Emoji are many things to many people. Depending who you ask, they’re fun, flirty, confusing, a whole new language, or the demise of language itself.
Google’s Emoji Scavenger Hunt adds one more to the many meanings of emoji: a way to play with machine learning.
The technology involved in this browser-based game is complex, but the idea is simple. An emoji appears, and using your phone’s camera you hunt around for examples of it IRL. Maybe you’ll see an emoji of a clock, or a shoe, then it’s your task to find one before the timer runs out. The emoji game will also take guesses at the identities of any object it happens to see along the way. So maybe make sure to tuck away anything you don’t want a computer voice announcing to your coworkers.
According to Google, “While you search, the neural network will try and guess what it’s seeing—proof that machine learning can be used for more than serious applications … This is just one example of how machine learning can be used in fun ways.“
Success means little more than earning a little trophy icon and not annoying your housemates or coworkers. But the real reason to play this game is to interact with machine learning in a novel way. And, one might suspect, to help refine Google’s machine learning algorithms for image recognition.
More than child’s play
Usually, taking advantage of neural networks requires connecting to a massive, distant set of cloud servers. But in this case, all of the processing takes place on the phone. The secret sauce is Tensorflow.js, an open-source framework for machine learning using JavaScript.
So instead of connecting to a distant server farm, all computation is happening directly on your device. The folks behind Emoji Scavenger Hunt — a joint effort of Google’s Brand Studio and PAIR teams — insist that the game saves none of the photos it takes, and invite people to review the open source code on GitHub.
Searching for a shoe isn’t exactly the future of gaming. But using our phones to quickly and accurately recognize objects like this is an impressive feat. It’s an ability that will become ever more integral to the everyday uses of our phones, from shopping to games to augmented reality. It also raises some security and privacy concerns, as phones will become able not just to see but to make sense of everything that passes in front of their cameras.
Emoji Scavenger Hunt works on the latest versions of Chrome on phones using up-to-date builds of Android OS, or on newer iPhones running Safari on the latest iOS. However, as the team says, this is merely an experiment, and might not always work as expected. Either way, happy hunting, and if the game asks you to find an eggplant, just let the timer run out.
