Granite-Docling-258M – A lightweight vision-language model launched by IBM

What is Granite-Docling-258M?

Granite-Docling-258M is a lightweight vision-language model launched by IBM, designed for efficient document conversion. It can transform documents into machine-readable formats while fully preserving layouts, tables, formulas, and other structural elements. With only 258M parameters, the model delivers excellent performance at low cost and supports multiple languages, including Arabic, Chinese, and Japanese. By leveraging the DocTags format to precisely describe document structures, it minimizes information loss. Granite-Docling-258M integrates seamlessly with the Docling library, offering strong customization and error-handling capabilities, making it a powerful tool for enterprise-level document processing.

Key Features of Granite-Docling-258M

• Accurate Document Parsing: Capable of precisely recognizing and parsing text, tables, formulas, charts, and other elements within documents, providing a clear and accurate foundation for subsequent processing.

• Structure-Preserving Conversion: Converts documents into digital formats while fully retaining the original layout and structure, ensuring the output closely matches the source for easier reading and editing.

• Multi-Modal Input Support: Accepts both image and text inputs, enabling the processing of scanned documents, handwritten notes, and digital files, broadening application scenarios.

• Multilingual Document Processing: Handles documents in multiple languages, offering convenience for multinational enterprises and multilingual environments.

• Efficient Data Extraction: Quickly extracts key information and structured data from documents, improving productivity and reducing manual effort.

• Flexible Output Formats: Supports conversion into multiple common formats, such as Markdown, HTML, and JSON, allowing users to adapt results for various use cases.

• Powerful Customization: Through integration with the Docling library, users can tailor workflows for document processing, achieving personalized conversion and analysis.

• Enterprise-Grade Reliability: Optimized for stability, reducing errors and anomalies in large-scale applications, making it suitable for enterprise-level deployment.

Technical Principles of Granite-Docling-258M

• Architecture:

  • Vision Encoder: Uses siglip2-base-patch16-512 to efficiently process image inputs and extract visual features from documents.

  • Vision-Language Connector: Employs a pixel shuffle projector to link visual features with the language model, enabling multimodal integration.

  • Language Model: Based on Granite 165M, capable of handling and generating natural language to ensure accurate content conversion.

• DocTags Format: A universal markup language that precisely describes document elements (charts, tables, formulas, etc.), their context, and positioning. Optimized for LLM readability, DocTags outputs can be directly converted into formats such as Markdown, HTML, or JSON.

• Training Data: Combines public datasets with internally synthesized datasets, such as SynthCodeNet (code snippets), SynthFormulaNet (mathematical formulas), SynthChartNet (charts), and DoclingMatrix (real document pages). These high-quality labeled datasets enable the model to learn document structure and content more effectively, improving accuracy and robustness.

Project Resources

• Official site: IBM Granite Docling Announcement

• Hugging Face model hub: Granite-Docling-258M

• Online demo: Granite-Docling-258M Demo

Application Scenarios

• Enterprise Document Management: Quickly digitizes paper documents for easier storage and retrieval, boosting efficiency.

• Academic Research: Processes large volumes of literature, helping researchers access and analyze materials faster.

• Government Archives Digitization: Accurately converts historical archives for long-term preservation and convenient access.

• Education: Teachers can efficiently organize teaching materials, and students can easily obtain electronic study resources.

• Multilingual Document Processing: Supports multinational enterprises in handling diverse language documents, breaking down language barriers, and facilitating international collaboration.