Siri's IQ scores higher in Loup Ventures' annual test of virtual assistants

IQ test of virtual assistants shows Siri’s smarts are on the rise


Siri Alexa in voice report
So selfless, that Siri.
Photo: Apple

If Siri seems to be of greater service to you lately, an annual IQ test comparing virtual assistants shows its definitely getting smarter.

Loup Ventures annually peppers the various digital assistants with five categories of questions and when it comes to accuracy and comprehension of a user’s question, Google Assistant remains the best. Siri, however, is right behind Google while Amazon’s Alex and Cortana lags behind in that order.

Siri’s marks in past Loup Ventures tests were far from impressive. Just last year, it could only answer 66 percent of the questions correctly and only understood queries 95 percent of the time. This year, Siri’s scores jumped to 78 percent on correct answers while understanding questions 99 percent of the time.

Google Assistant got its first perfect score in the “Understood Query” category and correctly answer questions 85 percent of the time.

Alexa was third with correct answers with 61 percent to Cortana’s 52 percent. Both tied in “Understood” at 98 percent.

Siri's IQ
This year’s report card was better for Siri.
Graphic: Loup Ventures

Siri’s IQ improvement impresses judges

The five categories of questions are Local, Commerce (ordering an item online), navigation, information an command (like an appoint reminder).

“Google Assistant has the edge in every category except Command,” LoupVentures reported. “Siri’s lead over the Assistant in this category is odd, given they are both baked into the OS of the phone rather than living on a third-party app (as Cortana and Alexa do).

“We found Siri to be slightly more helpful and versatile (responding to flexible language) in controlling your phone, smart home, music, etc. (With) a fair amount of music-related questions… Apple, true to its roots, has ensured Siri is capable with music on both mobile devices and smart speakers.”

Loup Ventures said Google Assistant’s overall lead is not surprising because Google’s main strength is its search engine.

The venture capital firm, with offices in Minneapolis and New York City, expects Siri to be even more versatile with the upcoming release of iOS 12 and Siri Shortcuts.

This fall, Loup Ventures will conduct a test on home speakers Amazon Echo, Google Home, Cortana, and HomePod.

Source: iPhone Hacks and Loup Ventures


Daily round-ups or a weekly refresher, straight from Cult of Mac to your inbox.

  • The Weekender

    The week's best Apple news, reviews and how-tos from Cult of Mac, every Saturday morning. Our readers say: "Thank you guys for always posting cool stuff" -- Vaughn Nevins. "Very informative" -- Kenly Xavier.