What Apple’s Vibrating Pen Tells Us About the Future of Everything

What Apple’s Vibrating Pen Tells Us About the Future of Everything

The site Patently Apple Wednesday posted a detailed analysis of a new Apple patent application for an iPen, a vibrating pen that makes noise.

The application describes a haptic stylus containing a tiny speaker, which is designed to be used on touch screens.

Apple watchers are scratching their heads over this one. Apple is going to sell tablets with pens like the Microsoft Tablet PC, or phones with pens like the Samsung Galaxy Note?

Not exactly.

But the iPen patent does hint at amazing and brilliant things to come — for Apple and the entire PC industry.

What Is the iPen?

Haptics technology generates vibrations that give the user tactile feedback. The most primitive form of haptics can be found in your phone, which can vibrate to indicate an incoming call.

The iPhone even offers a little-known feature called Custom Vibrations.

To use Custom Vibrations, open the Settings app, tap “General” then “Accessibility” then turn Custom Vibrations on. Now in Settings tap “Sounds,” then “Vibration” under the heading “Vibration Patterns.” At the bottom, tap “Create New Vibration.” Start tapping on the screen, and your pattern of tapping will be recorded for use with any alert. You can make several of these. The most common use for Custom Vibrations is to be able to tell who’s calling when your iPhone is in silent mode.

Custom Vibrations is nice, but primitive. A more advanced version of existing haptics technology can be found in something like a Microsoft Xbox controller, which rumbles and shakes to enhance onscreen action. It’s amazingly effective.

Apple’s iPen haptics would be more akin to an Xbox controller than an iPhone.

The patent describes a product that recreates the experience of physical pens and paintbrushes. When you use a real paintbrush, for example, vibrations and sound tell you about the texture of the paper, the thickness of the paint, the coarseness of the brushes, the pressure applied by the painter and other subtle information. Apple’s pen would simulate this.

The patent also describes the use of haptics to give feedback about what’s onscreen as the pen slides across. Buttons, workspaces, and other onscreen objects could have boundaries and textures conveyed through haptic.

Why a Pen?

Apple is unlikely to do what existing products with styluses do. We’re not going to see iPads and iPhones with skinny little styluses inserted into a slot for the device. We already have a wide range of third-party pens and styluses for iOS devices.

Apple isn’t the kind of company that will add a usage model (like pen input) just because other companies offer it. And they’re not the kind that will avoid using it because others have it.

Apple will offer haptic pens because they’re a great idea. Apple is leading the charge into the post-PC world. And that means touch screens. But illustrators, architects, designers, cartoonists, photo editors and others aren’t going to use their fingers for fine drawing and image manipulation. They use pens now, and they’ll use pens in the post-PC world, too.

Average users want to sign documents, doodle, hand-write notes, annotate texts and photos by circling things and scribbling in the margins.

Why Haptics?

Apple’s “core competency” is thrilling user interfaces. Tactile feedback will be a powerful tool in Apple’s arsenal. But not an exclusive one. In fact, advanced, “high-fidelity haptics” will exist in every phone, tablet and post-PC device we use in the future.

Apple and other companies will build advanced haptics into everything we touch. In Apple’s case, future phones, tablets and desktop touch devices will all have advanced haptics. So will keyboards, touchpads, and the remote control for their upcoming TV set.

The iPen isn’t Apple’s first haptic patent. In fact, Apple has a pile of patents for haptic response and other forms of tactile feedback for touch devices.

Apple even has patents for a system that transforms the physical surface of a touch screen, interactively throwing up actual bumps and textures on the screen to fit the image.

That Apple is aggressively patenting haptic technologies, and planning to build them into all their products should not surprise anyone.

The whole history of user interfaces involves the application of every relevant new technology to making the virtual world feel and behave like the actual world.

And it won’t stop until we all live in a Holodeck. But that’s 100 years away. What’s coming next from Apple?

What Is Jonathan Ive Working On?

Apple’s industrial design chief, the freshly-knighted Sir Jonathan Ive, has the Resume of the Millennium for an industrial designer. He led Apple’s efforts to design all Apple’s iMacs, MacBooks, iPods, iPhones and iPads.

That’s a lot to brag about. But referencing a major secret project, Ive told a reporter this week that “what we’re working on now feels like the most important and the best work we’ve done.”

Wow! What would be more important and better than the iMac, iPhone or iPad?

I believe the answer can be only one thing: A desktop touch computer that does everything.

Imagine an elegant piece of hardware that looks like an iMac. It’s huge, say, 40 inches diagonally instead of 27.

It swivels from all-the-way vertical to all-the-way horizontal and anywhere in between. You can use it like an iMac, with a physical keyboard on a desk with a keyboard and Magic Trackpad. Or, you can use it like a drafting tablet at an angle, and use it like an iPad, using only your fingers.

Or, you can make it vertical, and use it as a TV or presentation screen.

The main mode, however, will be the drafting table angle.

Running your fingers across the surface, you’ll feel buzzes and bumps telling you the edges of buttons and documents, and simulating physical sensations as if the on-screen objects were real.

A gesture brings up a virtual keyboard under your fingertips. You’ll be able to touch type on it because haptics will let you feel the edges of keys and identify the orienting F and J keys for finger placement.

A special physical keyboard powered by Bluetooth 4.0 will be designed to rest on the screen itself, instantly recognized by the system and replacing the virtual keyboard with the physical one as the text input device for whatever application is running.

The touch interface will be vastly better than the iPad. And the keyboard options will be incredible. But the main interface will be voice commands using a future version of Siri. You’ll just talk, and Siri will figure out what you want and give it to you.

An electronic contract arrives in your inbox. Just grab your iPen and sign it. It will feel and sound like you’re signing a piece of paper on an oak desk.

You’ll be able to use that pen for drawing, painting, photo editing and annotating documents.

In short, the super desktop will do everything every Apple product currently does, but much better. It will have the power of OS X and more. It will have the elegance of iMac and more. It will have the visual quality of the iPad’s retina display. It will have the touch of iPad. It will have Siri, FaceTime, the LaunchPad and much more.

It will recognize objects. Place your iPhone on the screen, and it will recognize the phone and establish a connection. Hold a document up to the camera and it will scan and OCR it.

In short, it will be the ultimate Apple product. In some ways, the Apple desktop will follow work pioneered at Microsoft, Immersion and elsewhere. In other ways, Apple will lead the way.

Eventually, almost every PC will work like this, combining touch, haptics, sensors, voice and more into intuitive interfaces. But I also believe Apple has the best shot at having the most compelling version of the product.

That’s what I think Ive and the rest of the Apple team is working on. And haptic pens are just one small part of it.

So don’t be confused or surprised by Apple’s vibrating pen. It’s going to be mightier than the sword used to knight Sir Jony.

(Picture courtesy of KUPA America.)

Related
  • TheSuperSteve

    I believe the answer can be only one thing: A desktop touch computer that does everything.”


    And that’s where I stopped reading. 
  • Tallest_Skil

    This is for drawing and image editing. Period. We’re not going to be using it as a means of UI interaction.

  • visualplant

    The writer obviously is no product designer. Even Tim Cook who is no product guy had enough sense to say a toaster-refrigerator wouldn’t appeal to consumers.

  • IamJAd

    Apple patented an iPen.  Now let me waste screen time telling you what I…and only I…  fantasize about.

  • sonofsci

    Uh……… wow.  Just wow. 

  • AppleJoeAJ

    I’m no design expert but I can see how this post makes sense. Its the ultimate union between all Apple products, its futuristic and elegant. I don’t know if Apple would do such a thing or if they have the means to do such a thing, but I wouldn’t be surprised if they did this and I would support them 100% if they did. This idea sounds like a really cool idea in my opinion :)

  • mr_bee

    I really, really, don’t understand CultofMac sometimes.  This is absolute nonsense, as is almost everything this man has ever written, yet he still has a job writing for this site, and the site still happily publishes this crap.  

    I don’t get it.  There are tens of thousands of better writers that would work for CultofMac, many for free.  Most of them have writing credentials, experience and know what they are talking about.  I really just don’t get how shite like this gets published over and over again and what the actual purpose of this is. 
  • mr_bee

    I’m no design expert but I can see how this post makes sense. Its the ultimate union between all Apple products, its futuristic and elegant. I don’t know if Apple would do such a thing or if they have the means to do such a thing, but I wouldn’t be surprised if they did this and I would support them 100% if they did. This idea sounds like a really cool idea in my opinion :)

    You do realise how dumb this sounds right?  

    A man with no design experience or talent writes a dumb article about design, and you post a comment admitting that …. you too know nothing about design, but that you think it all makes sense?  
    I mean WTF?  
    Further, you say, you don’t know if they will do it or even if it’s possible, but you wouldn’t be surprised if they did?  Again … WTF?  
    Unless you are 8 years old … well I guess I don’t have to finish that thought do I?  
  • technochick

    Or Apple won’t be giving us any kind of pen and their studies are to better interaction for the pens that other companies might make. This wouldn’t be the first time that Apple patented something they had zero intention of making and then licensed the patent to others that did, to use with Apple stuff or not. 

    As for the desktop touch computer. Uh no. To every detail Mike gives, a big no. His logic is so far distant from everything Apple has given us before in regards to how they design that it’s totally laughable. 
  • xMoonDevilx

    As an actual artist and designer….I’d love it. Rush out or save up to buy either in a heartbeat. Styluses wouldn’t be used by everyone…but be nice to have an option to, especially doing artwork, and if it were better than what is out there right now then by all means do it. If it combines into the theorized Cintique styled desktop…or variations with laptops…it would be well received, especially since Apple would at least make it affordable and providing more computer functionality with a Cintique on screen interface (ie. being like a Wacom tablet on screen instead of separate hardware). If not…Ill take a pen that works as this is prescribed to de.

  • Sean_Mapstone

    If it has a built in speaker I can see this being used by blind people, using the pen to scrub over or highlight text and have the pen read aloud. If it vibrates it could give feedback to the user for various actions.

  • SevanGrim

    yeah no. This is probably either Apple patenting something for patent holding sake, or an idea for a sold-separately stylus to go along with Apples idea for a future where everything is Apple synced. But either way we wont be seeing an ipen anytime soon. They know better than to create something this mundane. Its the kind of thing that used to kill products in the 90′s, and it would have the same effect now. And while Apple enthusiast would probably shell out the 40-100 bucks it would probably cost, the majority of people would either by a less-awesome 3rd party or do without. And Apple knows this.
     So, just like when everyone was SOOOO sure there was going to be an iphone 5 last year even though it would have meant a massive failure to reach the newly-created Verizon customers market, this is one of those things that has no chance of happening soon simply because it would be fiscally retarded. And this is why fanboys shouldnt run fanatical websites.

  • thelastofkris

    no sense of reality

  • lwdesign1

    Another option for filing the patent for the iPen could simply be strategy. If Apple files a patent, Samsung and all the other copycat corps will immediately assume this is the “next big thing” and start spending billions of R&D dollars trying to duplicate the thing. It could simply be corporate misdirection by Apple to have competitors waste their time and money, while Apple gets on with its real plans. I like the concept.

    I personally don’t like Mike’s idea of a 40″ tiltable screen I could draw on. Even when flat, this would be a LOT of area to try to work on–and what about heat from the screen causing sweaty forearms and smears and smudges on the screen. It wouldn’t be an idea working situation for me at all.
    I currently work on a 30″ Cinema Display with my keyboard and Kensington Expert Mouse and my Wacom tablet, and everything works brilliantly. I have no desire to work on a flat or angled surface reaching all over the place to get my work done. I DO like the idea of the Mac instantly recognizing an iPhone or iPad, and creating a connection via Bluetooth or similar technology.
  • kosherrunner

    http://www.cregle.com/


    its been on the market already, I have one
  • WackySooky
    I don’t think it’s such a stupid idea.

    Here’s another post getting to a similar conclusion but starting from a different point of view.

  • tinbert

    To me these ideas contain several brilliant outlooks to the future of computing. Of course, he’s speculating wildly, but I’m convinced that some of these ideas have great potential and will become reality – maybe not by Apple.

  • technochick

    I believe the answer can be only one thing: A desktop touch computer that does everything.”


    And that’s where I stopped reading. 

    I agree, to a point. I could see Apple making a custom order model iMac that does use this kind of touch UI for those that want to pay an extra $500. It could be used for Kiosks at museums etc. And if some prosumer user wants that option and is willing to pay, let them. 

    But I agree that it won’t be THE UI paradigm. Keyboard and mouse/trackpad/touchpad will be the way to go. But who knows, Apple might make good on all the various ideas about a keyboard that is the same size as the bluetooth one, has basically a smooth surface (no more crumbs getting under your keys) and a thin display underneath that can adjust the layout by what you are doing. Choose to go to DVORAK and that’s what you see. Go to another language and that’s what you see. Use an app like Final Cut and you can see the keyboard shortcuts like you used to buy those skins for. And so on. Perhaps in something like garageband your keyboard could become a little mini keyboard. You could also have a drawing mode, or even a ‘touch ui’ mode perhaps.
  • technochick

    Another option for filing the patent for the iPen could simply be strategy. If Apple files a patent, Samsung and all the other copycat corps will immediately assume this is the “next big thing” and start spending billions of R&D dollars trying to duplicate the thing. It could simply be corporate misdirection by Apple to have competitors waste their time and money, while Apple gets on with its real plans. I like the concept.

    As I said in another comment, Apple often patents major ideas that they really never intend to create themselves. They include the support to use that kind of thing and let those that know the mechanics way better do the actual design work. Like when they tried to create a speaker dock and it was only so so in quality. They dropped doing it themselves and started licensing the right to make a speaker with a dock connector the wiser folks. 

    So not only might Samsung etc think that Apple is directly doing this and try to copy or beat it, depending on the mechanics they would have to license the patent from Apple. So even when Apple doesn’t do it, they still can make money off it. AND if Apple does end up doing it directly or via a strong partnership with Wacom or such, they will show up Samsung who didn’t have the original R&D and had to rush to reproduce it and rarely does that make a better product. 

About the author

Mike ElganMike Elgan writes about technology and culture for a wide variety of publications. Follow Mike on Google+, Facebook and Twitter.

(sorry, you need Javascript to see this e-mail address)| Read more posts by .

Posted in iMac, Top stories |