Animated Drawings – An AI-Powered Sketch-to-Animation Tool by Meta AI

What is Animated Drawings?

Animated Drawings is an open-source tool developed by Meta AI that uses artificial intelligence to automatically turn hand-drawn characters into animations. It provides a full set of tools and algorithms that detect and segment characters from sketches, then animate them based on customizable settings. Users can control animation scenes, actions, and characters using configuration files. The tool also includes an automatic annotation generator to help users quickly create the necessary annotation files for animation. Animated Drawings supports multiple output formats, such as MP4 videos and transparent GIFs.

Key Features of Animated Drawings

• Automatic Animation of Hand-Drawn Characters:
  Automatically converts user-uploaded hand-drawn characters into animations, supporting various actions and scene settings.

• Flexible Configuration Files:
  Animation details such as character movement and scene setup can be precisely controlled via configuration files (e.g., YAML format).

• Automatic Annotation Generation:
  AI models detect characters in drawings and generate annotation files (e.g., joint positions, segmentation masks), streamlining the animation pipeline.

• Multi-Format Output:
  Supports exporting animations in formats like MP4 and transparent GIFs, catering to various usage needs.

• Multi-Character and Background Support:
  Enables the inclusion of multiple characters and background images in animations, enhancing content richness.

• Custom BVH File Support:
  Allows users to drive animations using custom BVH (Biovision Hierarchy) files, enabling compatibility with different skeletal motion data.

• Error Correction Tools:
  Provides an interface for manually correcting annotation errors generated by AI, ensuring more accurate animation results.

Technical Principles of Animated Drawings

• Image Detection and Segmentation:
  Uses computer vision techniques to detect and segment characters from user-uploaded sketches. Pre-trained object detection models identify character outlines and positions in the image, while segmentation algorithms create masks to separate the character from the background.

• Pose Estimation:
  Once a character is detected, the system uses pose estimation models (e.g., OpenPose) to locate key points like the head and joints. These key points are saved as annotation files, providing crucial data for animation generation. Accurate joint detection helps map natural motions to the drawn figure.

• Animation Generation:
  BVH (Biovision Hierarchy) files provide the skeletal motion data used to animate the drawings. The system uses motion retargeting to map BVH joint movements onto the hand-drawn character. An “As-Rigid-As-Possible” deformation algorithm ensures the animated figure retains a natural and coherent shape throughout the motion.

• Configuration and Rendering:
  Users define specific animation parameters such as characters, actions, and scenes through configuration files (e.g., YAML). These files guide the system in generating personalized animations. Finally, the system renders the animation as a video or GIF, offering options like transparent backgrounds for flexible export.

Project Links

• Official Site: https://sketch.metademolab.com/canvas

• GitHub Repository: https://github.com/facebookresearch/AnimatedDrawings

Use Cases for Animated Drawings

• Creative and Art Education:
  Inspire creativity in children and art enthusiasts by turning drawings into animations, making art education more engaging and interactive.

• Content Creation and Entertainment:
  Enable creators to quickly produce hand-drawn animated clips for social media, increasing engagement and content appeal.

• Interactive Exhibitions and Experiences:
  Enhance interactivity at museums, exhibitions, or live events by animating audience drawings in real time.

• Advertising and Marketing:
  Allow brands to produce personalized animated ads based on hand-drawn artwork, boosting consumer attention and social media visibility.

• Game Development:
  Quickly prototype character animations and create interactive games where users control characters through their drawings.