The latest demonstration of Apple’s AI expertise is Keyframer, software that turns an image into a video based on natural language instructions.
It offers an early glimpse at the type of AI tools that might come bundled into future versions of macOS or iOS.
Keyframer: An Apple animation app built on LLM-based AI
In recent months, Apple faced increasing criticism for not doing more with artificial intelligence, arguably the hottest sector in tech. But CEO Tim Cook promises that Apple continues to put an enormous amount of effort into AI. The resulting software simply hasn’t been released yet. Cook said recently that he remains “incredibly excited” about announcements in this area coming “later this year.”
But we got a fresh taste of what’s ahead with a new research paper from three Apple researchers: “Keyframer: Empowering Animation Design using Large Language Models” (.pdf). It’s the brainchild of Tiffany Tseng, Ruijia Cheng and Jeffrey Nichols.
They describe their creation like this: “With Keyframer, users can create animated illustrations from static 2D images via natural language prompting.” Users just input an image, then describe how an animated version should look.
But, unlike so many current AI image generators, Apple’s animation software is intended for professional designers. It “supports exploration and refinement of animations through the combination of prompting and direct editing of generated output.”
Users can do much more than put in an image and a description and then accept the AI’s results. They can use tools built into Keyframer to make changes — say, pick different colors. It’s more of a collaboration between designers and AI.
Simplifying animation with AI
The researchers are exploring the possibility of using natural language to make otherwise complex tasks simpler. “The use of descriptive natural language prompts has the potential to reduce the technical expertise needed to create design artifacts and provide novices with educational opportunities to improve their design skills,” they say in the paper.
They did not develop their own large-language model. Nor did they use the one Apple itself reportedly created. They used OpenAI’s ChatGPT-4 instead.
They tested Keyframer with 13 participants with a range of animation and programming experience. “Participants were happily surprised by the efficacy of the system to turn natural language instruction into animations,” said the Apple researchers.
Apple researchers also recently unveiled an open-source AI model that edits images based on users’ natural language commands.
