What Apple’s Vibrating Pen Tells Us About the Future of Everything



The site Patently Apple Wednesday posted a detailed analysis of a new Apple patent application for an iPen, a vibrating pen that makes noise.

The application describes a haptic stylus containing a tiny speaker, which is designed to be used on touch screens.

Apple watchers are scratching their heads over this one. Apple is going to sell tablets with pens like the Microsoft Tablet PC, or phones with pens like the Samsung Galaxy Note?

Not exactly.

But the iPen patent does hint at amazing and brilliant things to come — for Apple and the entire PC industry.

What Is the iPen?

Haptics technology generates vibrations that give the user tactile feedback. The most primitive form of haptics can be found in your phone, which can vibrate to indicate an incoming call.

The iPhone even offers a little-known feature called Custom Vibrations.

To use Custom Vibrations, open the Settings app, tap “General” then “Accessibility” then turn Custom Vibrations on. Now in Settings tap “Sounds,” then “Vibration” under the heading “Vibration Patterns.” At the bottom, tap “Create New Vibration.” Start tapping on the screen, and your pattern of tapping will be recorded for use with any alert. You can make several of these. The most common use for Custom Vibrations is to be able to tell who’s calling when your iPhone is in silent mode.

Custom Vibrations is nice, but primitive. A more advanced version of existing haptics technology can be found in something like a Microsoft Xbox controller, which rumbles and shakes to enhance onscreen action. It’s amazingly effective.

Apple’s iPen haptics would be more akin to an Xbox controller than an iPhone.

The patent describes a product that recreates the experience of physical pens and paintbrushes. When you use a real paintbrush, for example, vibrations and sound tell you about the texture of the paper, the thickness of the paint, the coarseness of the brushes, the pressure applied by the painter and other subtle information. Apple’s pen would simulate this.

The patent also describes the use of haptics to give feedback about what’s onscreen as the pen slides across. Buttons, workspaces, and other onscreen objects could have boundaries and textures conveyed through haptic.

Why a Pen?

Apple is unlikely to do what existing products with styluses do. We’re not going to see iPads and iPhones with skinny little styluses inserted into a slot for the device. We already have a wide range of third-party pens and styluses for iOS devices.

Apple isn’t the kind of company that will add a usage model (like pen input) just because other companies offer it. And they’re not the kind that will avoid using it because others have it.

Apple will offer haptic pens because they’re a great idea. Apple is leading the charge into the post-PC world. And that means touch screens. But illustrators, architects, designers, cartoonists, photo editors and others aren’t going to use their fingers for fine drawing and image manipulation. They use pens now, and they’ll use pens in the post-PC world, too.

Average users want to sign documents, doodle, hand-write notes, annotate texts and photos by circling things and scribbling in the margins.

Why Haptics?

Apple’s “core competency” is thrilling user interfaces. Tactile feedback will be a powerful tool in Apple’s arsenal. But not an exclusive one. In fact, advanced, “high-fidelity haptics” will exist in every phone, tablet and post-PC device we use in the future.

Apple and other companies will build advanced haptics into everything we touch. In Apple’s case, future phones, tablets and desktop touch devices will all have advanced haptics. So will keyboards, touchpads, and the remote control for their upcoming TV set.

The iPen isn’t Apple’s first haptic patent. In fact, Apple has a pile of patents for haptic response and other forms of tactile feedback for touch devices.

Apple even has patents for a system that transforms the physical surface of a touch screen, interactively throwing up actual bumps and textures on the screen to fit the image.

That Apple is aggressively patenting haptic technologies, and planning to build them into all their products should not surprise anyone.

The whole history of user interfaces involves the application of every relevant new technology to making the virtual world feel and behave like the actual world.

And it won’t stop until we all live in a Holodeck. But that’s 100 years away. What’s coming next from Apple?

What Is Jonathan Ive Working On?

Apple’s industrial design chief, the freshly-knighted Sir Jonathan Ive, has the Resume of the Millennium for an industrial designer. He led Apple’s efforts to design all Apple’s iMacs, MacBooks, iPods, iPhones and iPads.

That’s a lot to brag about. But referencing a major secret project, Ive told a reporter this week that “what we’re working on now feels like the most important and the best work we’ve done.”

Wow! What would be more important and better than the iMac, iPhone or iPad?

I believe the answer can be only one thing: A desktop touch computer that does everything.

Imagine an elegant piece of hardware that looks like an iMac. It’s huge, say, 40 inches diagonally instead of 27.

It swivels from all-the-way vertical to all-the-way horizontal and anywhere in between. You can use it like an iMac, with a physical keyboard on a desk with a keyboard and Magic Trackpad. Or, you can use it like a drafting tablet at an angle, and use it like an iPad, using only your fingers.

Or, you can make it vertical, and use it as a TV or presentation screen.

The main mode, however, will be the drafting table angle.

Running your fingers across the surface, you’ll feel buzzes and bumps telling you the edges of buttons and documents, and simulating physical sensations as if the on-screen objects were real.

A gesture brings up a virtual keyboard under your fingertips. You’ll be able to touch type on it because haptics will let you feel the edges of keys and identify the orienting F and J keys for finger placement.

A special physical keyboard powered by Bluetooth 4.0 will be designed to rest on the screen itself, instantly recognized by the system and replacing the virtual keyboard with the physical one as the text input device for whatever application is running.

The touch interface will be vastly better than the iPad. And the keyboard options will be incredible. But the main interface will be voice commands using a future version of Siri. You’ll just talk, and Siri will figure out what you want and give it to you.

An electronic contract arrives in your inbox. Just grab your iPen and sign it. It will feel and sound like you’re signing a piece of paper on an oak desk.

You’ll be able to use that pen for drawing, painting, photo editing and annotating documents.

In short, the super desktop will do everything every Apple product currently does, but much better. It will have the power of OS X and more. It will have the elegance of iMac and more. It will have the visual quality of the iPad’s retina display. It will have the touch of iPad. It will have Siri, FaceTime, the LaunchPad and much more.

It will recognize objects. Place your iPhone on the screen, and it will recognize the phone and establish a connection. Hold a document up to the camera and it will scan and OCR it.

In short, it will be the ultimate Apple product. In some ways, the Apple desktop will follow work pioneered at Microsoft, Immersion and elsewhere. In other ways, Apple will lead the way.

Eventually, almost every PC will work like this, combining touch, haptics, sensors, voice and more into intuitive interfaces. But I also believe Apple has the best shot at having the most compelling version of the product.

That’s what I think Ive and the rest of the Apple team is working on. And haptic pens are just one small part of it.

So don’t be confused or surprised by Apple’s vibrating pen. It’s going to be mightier than the sword used to knight Sir Jony.


(Picture courtesy of KUPA America.)



Daily round-ups or a weekly refresher, straight from Cult of Mac to your inbox.

  • The Weekender

    The week's best Apple news, reviews and how-tos from Cult of Mac, every Saturday morning. Our readers say: "Thank you guys for always posting cool stuff" -- Vaughn Nevins. "Very informative" -- Kenly Xavier.