PaddleOCR-VL – a multimodal document understanding model open-sourced by Baidu PaddlePaddle

What is PaddleOCR-VL？

PaddleOCR-VL is a multimodal document understanding model open-sourced by Baidu’s PaddlePaddle team, optimized for low-compute environments with only 0.9 billion parameters. It achieved a world-leading score of 92.6 on the OmniDocBench V1.5 international benchmark, surpassing major models such as GPT-4o.
The model adopts a two-stage architecture: PP-DocLayoutV2 handles layout analysis, while PaddleOCR-VL-0.9B performs content recognition. Supporting 109 languages, it can accurately process complex document elements such as tables, formulas, and charts, outputting structured data in Markdown or JSON formats. Its lightweight design makes it suitable for on-premise deployment, especially in privacy-sensitive fields like medical report processing and ancient manuscript digitization.

Key Features of PaddleOCR-VL

• Intelligent Document Structure Parsing:
  Automatically detects and recognizes text, tables, formulas, and charts while preserving natural reading order.

• Multilingual Support:
  Recognizes and processes documents in 109 languages, including Chinese, English, Japanese, and Korean.

• Lightweight and Efficient Deployment:
  Optimized for mobile devices, local servers, and other resource-constrained environments.

• Multimodal Understanding:
  Handles mixed text-image documents with high precision.
  On the OmniDocBench V1.5 benchmark, the model excelled in handling medical reports, vertically written ancient texts, and mathematical formulas, producing structured outputs in JSON or Markdown format.

Technical Principles

1. Two-Stage Processing Architecture

• Stage 1 – Layout Analysis:
  The PP-DocLayoutV2 model identifies semantic regions such as text, tables, and formulas, and predicts human reading order (with an average error of only 0.043).

• Stage 2 – Content Recognition:
  PaddleOCR-VL-0.9B performs fine-grained recognition on the localized regions, outputting structured text, tables, and formulas.
  This approach avoids the hallucination and misalignment issues common in end-to-end models, improving stability in complex layouts.

2. Multimodal Fusion Core Architecture

The model integrates three major components:

• Visual Encoder:
  A NaViT dynamic-resolution encoder adapts to various document sizes and resolutions, preserving fine-grained details.

• Language Model:
  Built on the lightweight ERNIE-4.5-0.3B, providing strong language understanding and generation capabilities.

• Cross-Modal Alignment Mechanism:
  A vision-language fusion module converts visual features into structured textual representations.

3. Dynamic Resolution & Lightweight Design

The NaViT encoder supports adaptive resolution adjustment, allocating compute resources based on document complexity.
With only 0.9B parameters, the model runs efficiently on CPUs, achieving 14.2%–253.01% faster inference than comparable models.

4. Unified Multi-Task Framework

Uses an instruction-driven system to handle text, tables, formulas, and charts within a single model—eliminating the need to switch between specialized models and simplifying deployment.

Project Links

• Official Website: https://ernie.baidu.com/blog/zh/posts/paddleocr-vl/

• HuggingFace Model Hub: https://huggingface.co/PaddlePaddle/PaddleOCR-VL

• arXiv Paper: https://arxiv.org/pdf/2510.14528

• Online Demo: https://huggingface.co/spaces/PaddlePaddle/PaddleOCR-VL_Online_Demo

• Official Try-Out Portal: https://aistudio.baidu.com/application/detail/98365

Use Cases of PaddleOCR-VL

• Large-Scale Document Digitization:
  Converts paper archives, historical manuscripts, and contracts into editable digital formats.
  Handles multilingual and complex layouts (tables, formulas) with high accuracy.

• Financial and Business Invoice Processing:
  Automatically extracts key information—such as amounts, dates, and company names—from invoices, receipts, and bank statements to streamline financial auditing and tax management.

• Academic Research and Educational Digitization:
  Parses text, formulas, and charts in research papers and textbooks, enabling knowledge extraction and structured organization for scientific information management and intelligent education tools.

• Multilingual Global Document Processing:
  Supports 109 languages, including Arabic, Russian, and Japanese, making it suitable for multinational enterprises, translation platforms, and multilingual archive management.

• Privacy-Sensitive Local Deployment:
  With its lightweight 0.9B-parameter design, the model can run efficiently on CPUs or edge devices, ideal for government or medical data environments with strict privacy requirements.

• Intelligent Knowledge Base and Retrieval Systems:
  When integrated with RAG (Retrieval-Augmented Generation) technology, PaddleOCR-VL can convert scanned documents into structured data, enhancing enterprise knowledge management and retrieval precision.