iPad installation plays music based on the face you are wearing


The "Sound of Emotion" will play music based on facial expressions at eh Market Street Protyping Festival in San Francisco. Photo: Sound of Emotion/Neighborland

You’re in a mood and your face can’t hide it. Now imagine if that face was the source of music.

The “Sound of Emotion,” a musical project created with facial expression recognition technology, will be on display at the Market Street Prototyping Festival in San Francisco. The festival runs today through April 11 on Market Street between Embarcedero and Van Ness.

The installation will use four iPad and for selected genres, each device will represent a single instrument, such as bells, a didgeridoo and drums.

The instruments will get activated through the iPad’s emotion detection software, which can recognize up to seven facial expressions. When visitors approach the device, the face you are wearing will trigger the instrument to play in the mood it sees.

The installation was one of 50 chosen for the festival, which showcases the Bay Area’s tech know-how and allows people to interact with the projects. The festival is partnership with the San Francisco Planning Department, the Yerba Buena Center for the Art and the Knight Foundation.

The design team on the Sound of Emotion includes, Paul Pillitteri, Seth Neiman, Josh Coleman, Howard Cohen, Evan Kuester and Amy Tedesco Pillitteri.

Winning projects will get funding, workspace and mentorship.


Daily round-ups or a weekly refresher, straight from Cult of Mac to your inbox.

  • The Weekender

    The week's best Apple news, reviews and how-tos from Cult of Mac, every Saturday morning. Our readers say: "Thank you guys for always posting cool stuff" -- Vaughn Nevins. "Very informative" -- Kenly Xavier.