Apple may have been late to the smart speaker game but its HomePod is showing signs it could one day outperform them all.
But only if Apple gives Siri the power.
In Loup Ventures annual smart speaker comparisons, Google Assistant understood all 800 questions and answered nearly 88 percent of them correctly. In its first Loup Ventures test, the HomePod with Siri at the helm misunderstood just three questions and managed to answer correctly nearly 75 percent of its queries.
Google Home remains the top performing smart speaker with the HomePod second. It just passed the Amazon Echo and Alexa by a couple of percentage points. The Invoke with digital assistant Cortana was fourth.
HomePod score not bad for a first test
Apple has been tweaking Siri and the company’s push to be a leader in artificial intelligence has directly fed into the rise of Siri’s IQ. This past summer, in a test of digital assistants on mobile devices, Siri jumped more than 10 percentage points from its previous score.
“We separate digital assistants on your smartphone from smart speakers because, while the underlying tech is similar, the use cases and user experience differs greatly,” Gene Munster and Will Thompson wrote on the Loup Ventures website today. “It’s less helpful to compare, say, Siri on your iPhone and Alexa on an Echo in your kitchen.”
The top four smart speakers were peppered with questions in five categories: local, commerce, navigation, information and command.
Most of the questions misunderstood by the speakers included a proper noun, such as the name of a local restaurant. But Loup Ventures noted improvements across all speakers in voice recognition and language processing.
Screenshot: Loup Ventures
Google Home was tops in four of the five categories but fell short in the command category to the HomePod.
“HomePods lead in this category may come from the fact that HomePod will pass on full Sirikit requests like those regarding messaging, lists and basically anything other than music to the iOS device paired to the speaker,” Munster and Thompson wrote. “Siri on iPhone has deep integration with email, calendar, messaging and other areas of focus in our Command category. Our question set also contains a fair amount of music-related queries, which HomePod specializes in.”
HomePod was weakest in the information and commerce categories, with some questions answered with I can’t get the answer to that on HomePod. Loup Ventures says Apple limits Siri’s tasks on HomePod, making it more of a home speaker than a smart speaker.
“With scores nearing 80-90 percent, it begs the question, will these assistants eventually be able to answer everything you ask?” the authors wrote. “The answer is probably not, but continued improvement will come from allowing more and more functions to be controlled by your voice.”
