Apple has been pretty outspoken about marketing HomePod as a high-end speaker that just happens to have Siri, as opposed to a smart speaker built around its AI assistant, like the Google Home or Amazon Alexa.
A new study published by Loup Ventures suggests one reason why: Because Siri is seriously lagging behind its rivals in terms of its functionality and ability to answer questions.
The report placed HomePod as the best sounding smart speaker (which pretty much everyone has acknowledged), but said that it ranks behind Google Home, Amazon Alexa, and Microsoft Cortana as an AI assistant. Of 782 queries fielded, Siri managed to answer 52.3 percent of them. That’s behind the 57 percent managed by Cortana, the 64 percent managed by Alexa, and the impressive 81 percent managed by Google.
The problem wasn’t related to HomePod’s inability to understand questions. As the report notes, Siri understood 99.4 percent of queries, but simply doesn’t have the AI smarts to answers them. It is, however, good at local questions (“where can I find a good cup of coffee around here?”) and commerce queries (“help me buy some new shoes”), where it outranks Alexa and Cortana, but falls behind Google.
Potential to improve
All isn’t lost, however. The low score partly reflects the fact that, right now, a lot of domains simply aren’t supported by HomePod.
“Adding domains will quickly improve Siri’s score. Some domains like navigation, calendar, email, and calling are simply not supported. These questions were met with, “I can’t ___ on HomePod.” Also, in any case that iPhone-based Siri would bring up Google search results, HomePod would reply, “I can’t get the answer to that on HomePod,” which forces you to use your phone or give up on the question altogether. Removing navigation, calling, email, and calendar-related queries from our question set yields a 67% correct response, a jump from overall of 52.3% correct. This means added support for these domains would bring HomePod performance above that of Alexa (64%) and Cortana (57%), though still shy of Google Home (81%).”
Loup Ventures points out that Apple’s limiting of HomePod’s domains should change over time, which will make it a far more valuable AI assistant. For now, though, folks who buy HomePod are predominantly doing so for the high-end audio. (For a look at this aspect of HomePod, check out our initial thoughts review here.)
Have you tried out HomePod yet? If so, what have been your thoughts on Apple’s latest product? Let us know in the comments below.
Luke Dormehl is a U.K.-based journalist and author, with a background working in documentary film for Channel 4 and the BBC. He is the author of The Apple Revolution and The Formula: How Algorithms Solve All Our Problems … and Create More, both published by Penguin/Random House. His tech writing has also appeared in Wired, Fast Company, Techmeme and other publications.
7 responses to “HomePod may sound great, but Siri still sucks”
I got the Homepod this weekend. It’s very annoying that Siri sucks, I repeatedly said “Play my Latest Favourites playlist”, it just wouldn’t do it, I had to change the name of the playlist and it eventually worked, but for all that money, and for such a basic request, it’s set me up for trepidation for future use. I guess that’s the price you pay for adopting new products so early?
“Where can I find a good cup of coffee around here?”
Seriously. From your own house which you’re living for about 20 years?!
They may have had to downsize to afford their pricey new speaker.
Now that’s funny!!!
I agree with this article completely…Siri sucks! For whatever HomePod may or may not do with more domains, the painful fact is Siri is awful on any device, in any circumstance. To imply that more domains will somewhat help Siri’s performance is the proverbial “fiddling while Rome burns”.
It seems ironic, if not sarcastically laughable, that Siri at this point is so neglected in dedicated development by Apple that previous wannabe’s like Google and Amazon are defining the market and the direction it will take.
Siri sucks. That is incontrovertible. But one feature that would go a long, long way to making me a happier HomePod owner (got mine Friday) would be ability to access my iTunes library via Siri or be able to use the SiriusXM app. That this device only connects to Apple Music is a real problem in my eyes. The sound is indeed great. Love it. But whereas I planned to purchase several for around the house, I now will likely not do this and instead wait for a different solution.
The first Apple product that I am seriously considering returning. Siri is totally broken. Sound is good on the device once you can get your selected music playing.