OmniSVG – StepFun Collaborates with Fudan University to Launch an End-to-End Multimodal Vector Graphics Generation Model

What is OmniSVG?

OmniSVG is the world’s first end-to-end multimodal SVG (Scalable Vector Graphics) generation model jointly developed by StepFun and Fudan University. Leveraging a pre-trained Vision-Language Model (VLM) and an innovative SVG tokenization method, OmniSVG parameterizes SVG commands and coordinate parameters into discrete tokens, achieving a decoupling of structural logic and geometric details. This enables OmniSVG to efficiently generate diverse high-quality SVG graphics, ranging from simple icons to complex anime characters.

The main functions of OmniSVG

• Multimodal Generation: OmniSVG is the first end-to-end multimodal SVG generation model capable of generating high-quality SVG graphics based on text descriptions, image references, or character references. It can produce diverse graphics ranging from simple icons to complex anime characters.
• Efficient Generation and Training: Leveraging the pre-trained vision-language model (VLM) Qwen-VL, OmniSVG employs an innovative SVG tokenization method that parameterizes SVG commands and coordinate parameters into discrete tokens. This separates structural logic from geometric details during training, achieving a training efficiency more than three times faster than traditional methods. It can handle sequences of up to 30,000 tokens, enabling the generation of complex SVGs with rich details.
• Dataset and Evaluation: The OmniSVG team has released the MMSVG-2M dataset, which contains 2 million multimodally annotated SVG resources, covering three subsets: icons, illustrations, and characters. They have also proposed a standardized evaluation protocol, MMSVG-Bench, for assessing the performance of conditional SVG generation tasks.
• Editability and Practicality: The generated SVG files feature infinite scalability and full editability, allowing seamless integration into professional design workflows such as Adobe Illustrator and others. This enhances the practicality of AI-generated graphics in fields like graphic design and web development.

The Technical Principles of OmniSVG

• Based on Pre-trained Vision-Language Models (VLM): OmniSVG is built upon the pre-trained vision-language model Qwen-VL. The model can deeply integrate image and text information, providing a strong foundation for multimodal generation.
• SVG Tokenization Method: OmniSVG innovatively parameterizes SVG commands and coordinate parameters into discrete tokens, enabling the processing of SVG generation in a way similar to natural language processing. This approach improves training efficiency while retaining the ability to generate complex SVG structures.
• End-to-End Multimodal Generation Framework: OmniSVG supports direct generation of SVG graphics from various input methods, including text descriptions, image references, or character references. This end-to-end generation framework can produce vector graphics that are rich in color and vivid in detail, overcoming many limitations of traditional methods.
• Efficient Training and Long Sequence Processing: Compared to traditional methods, OmniSVG achieves a training speed improvement of over 3 times and can process sequences up to 30,000 tokens in length. This enables it to generate complex SVG graphics with rich details.

The project address of OmniSVG

• Project Website: https://omnisvg.github.io/
• Github Repository: https://github.com/OmniSVG
• HuggingFace Model Hub: https://huggingface.co/OmniSVG
• arXiv Technical Paper: https://arxiv.org/pdf/2504.06263

Application scenarios of OmniSVG

• Brand Icon Design: OmniSVG can quickly generate brand icons based on text descriptions. Designers no longer need to start from scratch, significantly reducing manual design time.
• Web Development: Icons are an indispensable element in web development. OmniSVG can generate vector icons based on text descriptions or image references. These icons can be scaled without loss of quality and are suitable for various resolutions, from mobile devices to 4K monitors.
• Character and Scene Design: In game development, OmniSVG can be used to generate graphic materials such as game characters and scenes, adding a unique artistic style to the game.
• Dynamic Character Generation: Based on character references, OmniSVG can generate vector graphics that maintain the same character features but with different poses or in different scenes.
• Rapid Prototype Design: Content creators can use OmniSVG to quickly generate prototypes of icons, illustrations, or character graphics, accelerating the creative process.