ToonComposer – An AI Animation Creation Tool Developed by Tencent in Collaboration with CUHK and Peking University

What is ToonComposer?

ToonComposer is a generative AI tool jointly developed by researchers from The Chinese University of Hong Kong, Tencent PCG ARC Lab, and Peking University. It can transform sketches into professional-grade animations within seconds. Based on generative post-keyframe technology, ToonComposer integrates the traditionally labor-intensive in-betweening and coloring processes into an automated workflow. With just a sketch and a single colored reference frame, it can generate high-quality animated videos. The tool also supports sparse sketch injection and region control, enabling artists to precisely direct animation effects, significantly reducing manual effort, improving efficiency, and bringing a revolutionary change to animation production.

Key Features of ToonComposer

• Generative post-keyframe: Automates in-between frame generation and coloring, requiring only a sketch and one colored reference frame to create a complete cartoon video, drastically reducing manual work.

• Precise sketch keyframe control: Artists can insert sparse keyframe sketches at any point on the timeline to guide animation, ensuring high accuracy and quality even in complex motion sequences.

• Region control: Users can intentionally leave blank areas in sketches and mark them with brush tools. ToonComposer intelligently fills these areas based on context or prompts, further reducing workload.

Technical Foundations of ToonComposer

• Sparse sketch injection mechanism: Uses positional encoding mapping to convert sketch frames into model-compatible tokens, injecting them into the latent space of the video generation model. This ensures temporal awareness of sketches during generation. A position-aware residual module allows users to dynamically adjust sketch control strength during inference, providing more flexible animation generation.

• Cartoon adaptation method: Based on Spatial Low-Rank Adaptation (SLRA), it adjusts only the spatial behavior of the model while preserving its temporal behavior, adapting modern video foundation models to the cartoon domain. This ensures the generated animation adheres to cartoon visual style while maintaining smooth motion.

• Region control mechanism: During training, random masking is applied to sketch frames, teaching the model to generate reasonable content in masked areas. During creation, users can specify regions for automatic generation without drawing full sketches, reducing workload and increasing creative flexibility.

Project Links

• Official Website: https://lg-li.github.io/project/tooncomposer/

• GitHub Repository: https://github.com/TencentARC/ToonComposer

• HuggingFace Model Library: https://huggingface.co/TencentARC/ToonComposer

• arXiv Paper: https://arxiv.org/pdf/2508.10881

• Online Demo: https://huggingface.co/spaces/TencentARC/ToonComposer

Application Scenarios of ToonComposer

• Animation production: Independent creators can quickly generate animation prototypes, avoiding tedious in-betweening and coloring, and focus more on creativity and storytelling.

• Game development: Developers can rapidly produce animation assets for characters or scenes, cutting manual production costs and improving efficiency.

• Education & training: Serves as a teaching tool in animation education, helping students quickly grasp production workflows and boosting learning engagement.

• Advertising & marketing: Agencies can rapidly create animated ads tailored to client needs, saving time and costs during fast iteration and content revision.

• Film & TV production: Helps generate preliminary animated sequences for films and TV shows, enabling directors and production teams to quickly evaluate creative ideas and scene effects, reducing early production costs.