Apple has been pretty outspoken about marketing HomePod as a high-end speaker that just happens to have Siri, as opposed to a smart speaker built around its AI assistant, like the Google Home or Amazon Alexa.
A new study published by Loup Ventures suggests one reason why: Because Siri is seriously lagging behind its rivals in terms of its functionality and ability to answer questions.
The report placed HomePod as the best sounding smart speaker (which pretty much everyone has acknowledged), but said that it ranks behind Google Home, Amazon Alexa, and Microsoft Cortana as an AI assistant. Of 782 queries fielded, Siri managed to answer 52.3 percent of them. That’s behind the 57 percent managed by Cortana, the 64 percent managed by Alexa, and the impressive 81 percent managed by Google.
The problem wasn’t related to HomePod’s inability to understand questions. As the report notes, Siri understood 99.4 percent of queries, but simply doesn’t have the AI smarts to answers them. It is, however, good at local questions (“where can I find a good cup of coffee around here?”) and commerce queries (“help me buy some new shoes”), where it outranks Alexa and Cortana, but falls behind Google.
Potential to improve
All isn’t lost, however. The low score partly reflects the fact that, right now, a lot of domains simply aren’t supported by HomePod.
“Adding domains will quickly improve Siri’s score. Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct. This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%).”
Loup Ventures points out that Apple’s limiting of HomePod’s domains should change over time, which will make it a far more valuable AI assistant. For now, though, folks who buy HomePod are predominantly doing so for the high-end audio. (For a look at this aspect of HomePod, check out our initial thoughts review here.)
Have you tried out HomePod yet? If so, what have been your thoughts on Apple’s latest product? Let us know in the comments below.