A new search engine wants to keep App Store reviews honest and accurate by completely ignoring the ones that may come from paid shills or through developers’ relentless badgering of users.
As of this writing, the system claims it has audited over 17 million reviews and ignored 1.1 million of them. And if you’re wondering if the hot new game or feature-filled calendar app you’re about to download is really worth your time, you might want to check this site out.
AppRecs is the service, and it’s totally free online. All you have to do is search for the app you’re interested in directly or browse through the list of about 35,000 “truly great” ones. You can filter those down based on price, average rating, category, and popularity.
The creator of AppRecs, Seattle-based software engineer Mark Edmond, describes his methods in a lengthy thread on Reddit, but it basically boils down to organizing reviews based on levels of trustworthiness. Edmond places “organic” App Store reviews — those users write on their own initiative — and ones developers gently request at the top of the scale. Below those are the less trustworthy reviews, which include aggressively requested, filtered/cherry-picked, network-sourced, reward-driven, and just straight-up paid ones.
Edmond says that it’s hard to determine exactly which App Store reviews might be less than genuine, but he’s developed a few flags to look out for.
“For example, let’s say a reviewer has posted nothing but 5-star reviews and has reviewed 1,000+ apps — we flag those reviews as untrustworthy,” he told Reddit. “How about prolific reviewers A and B, both of whom have reviewed nearly the same set of apps, all of them with positive ratings? Probably networked or paid.
“In some cases, the reviews are cut-and-pasted from previous reviews and are thus easy to flag. In other cases, we can look at overall stats for each app — median review length, median number of reviews posted by the reviewers, and other metrics — and establish that something fishy is going on.”
AppRecs starts with a Java app that pulls in a massive amount of data from Apple searches and feeds and separates the relevant info in a database. Then it begins sifting through all of that data looking for things that trigger the flags using aggregation and recommendation engines before feeding all of its findings into the site generator, which updates the engine online.
Because it uses a series of automated tasks based on educated guesses, Edmond isn’t sure what the system’s error rate is. But he’s looking for ways to improve it with machine learning, and it’s definitely off to a good start.