Although it sometimes doesn’t understand everything you say, it’s hard not to like Siri. After all, the voice-controlled assistant has made it easier then ever to perform all kinds of tasks on a smartphone using only the natural language that we use on a daily basis. But as we are well aware, Siri isn’t perfect.
Especially when it comes to answering your questions. In fact, Apple analyst Gene Munster believes she’s still two years behind Google after she only managed to answer 68% of the 800 questions he asked in a quiet room.
Munster threw 1,600 questions at Siri altogether. In addition to the 800 asked in a quiet room, another 800 were asked on a busy street in Minneapolis. He then threw the same 1,600 questions at Google. Let’s be clear: not Google Voice, just traditional, text-based Google search.
Rather unsurprisingly, Munster found that Google is far more accurate than Siri after it answered 86% of his questions accurately. Munster gives Google search a B+ grade, while Siri could only achieve a D.
Although it does seem like an unfair test — at least to me — Munster did discover some interesting things.
Siri is great at comprehension, understanding over 80% of the queries presented to it. That doesn’t come close to the 100% comprehended by Google, however. Bearing in mind, again, that these queries were typed into Google — not spoken.
On a noisy Minneapolis street, Siri comprehended 83% of our queries. We believe this is the most accurate representation of Siri’s comprehension because Siri is rarely used in a perfectly quiet setting. As a point of comparison, we performed the test in a quiet room where Siri was able to comprehend 89% of queries. Siri will need to improve from a B in comprehension to at least an A if it is to be considered a viable alternative to Google (A+), which could take two or more years.
Siri isn’t all that accurate. It was able to answer just 62% of the questions put to it out on the street, and just 68% of the questions put to it in a quiet room.
Siri accurately answer 62% on the street and 68% in a quiet room. (See page 2 for difference between comprehension and accuracy). We believe accuracy is where Siri needs the most improvement if it wants to rival Google. Currently, we measure Google at 86% accuracy in the US based on comScore result page per search data.
In order to become a viable search alternative to Google, then, Siri must match or surpass Google’s accuracy grade of B+. It’s currently on a grade D, according to Munster.
Finally, Munster believes that in iOS 6, Siri’s dependency on Google will decrease by 12%, from 60% to 48%.
We estimate that changes to Siri iOS 6 will decrease dependency on Google by 12%, from 60% to 48%. Apple has made several significant changes to Siri on iOS 6 that will decrease Siri’s dependency on Google. First and foremost, Apple’s in-house maps app will eliminate Siri’s dependency on Google for navigation. Second, Siri’s new integration of Yahoo Sports, Open Table, Rotten Tomatoes, and Fandango will provide answers for sports scores and statistics, restaurant reservations, movie show times, and ticket purchases.
It’s clear Siri still has a long way to go until it can replace typing when it comes to answering your questions, then. But it would have been interesting to see how it fares against Google Voice search. And as we must keep reminding ourselves, Siri is still in beta.
- Source Business Insider