eBay’s new tech controls your iPhone with head movements


HeadGaze uses the iPhone X 3D camera to move an on-screen cursor with subtle head motions.
HeadGaze uses the iPhone X's 3D camera to move an on-screen cursor with subtle head motions.
Photo: eBay

The TrueDepth camera on the iPhone X is what makes FaceID and Animoji possible. But a brilliant intern at eBay used to tech to create head-tracking software that can be used to control the iPhone.

HeadGaze has just been made open source, so any developer can build it into their own software.

“Our team built a model that creates a virtual stylus that follows the motion of your head (up, down, side to side), taking the head’s 3D information from ARKit and applying 3D geometry mapping to get the location of the ‘cursor’ on the screen,” wrote Muratcan Cicek, eBay Intern and PhD Candidate at University of California, Santa Cruz, in an official blog post.

HeadGaze includes a user interface that allows a user with motor impairments to scroll the screen or move between pages. Pausing the cursor over a button registers as a virtual “tap.”

HeadGaze is for everyone

Cicek and his fellow developers created HeadSwipe as a proof of concept. This app allows users to shop on eBay without touching the iPhone X. But HeadGaze isn’t limited to that software. “Our modular code design allows developers to easily integrate our features into their existing or future apps with minimum code change,” promises Cicek.

The doctoral candidate sees potential benefits for all kinds of users. “HeadGaze enables you to scroll and interact on your phone with only subtle head movements. Think of all the ways that this could be brought to life. Tired of trying to scroll through a recipe on your phone screen with greasy fingers while cooking? Too messy to follow the how-to manual on your cell phone while you’re tinkering with the car engine under the hood? Too cold to remove your gloves to use your phone?”

Muratcan Cicek and his team at eBay developed HeadGaze.
Muratcan Cicek and his team at eBay developed HeadGaze.
Photo: eBay

The iPhone X is the sole current phone with a TrueDepth camera, and therefore the only one capable of running this software. However, Apple is expected to unveil three new models tomorrow with TrueDepth support.