OmniSQL – Open-source Text-to-SQL Model, converting natural language into SQL queries

What is OmniSQL?

OmniSQL is an open-source text-to-SQL model that efficiently converts natural language questions into SQL queries. Leveraging an innovative data synthesis framework, it has generated the first large-scale text-to-SQL dataset, SynSQL-2.5M, which contains 2.5 million high-quality samples covering more than 16,000 cross-domain databases. The samples span multiple complexity levels and language styles. OmniSQL offers three model versions: 7B, 14B, and 32B. During the fine-tuning process, it incorporates high-quality annotated data from Spider and BIRD.

The main functions of OmniSQL

• Text-to-SQL Conversion: OmniSQL can understand questions posed by users in natural language and convert them into corresponding SQL query statements.
• Support for Multiple Databases and Complex Queries: OmniSQL supports various database types and can handle SQL queries of varying complexity levels, ranging from simple single-table queries to complex multi-table joins, subqueries, function calls, and Common Table Expressions (CTEs).
• Provide Thought Chain Solutions: In addition to generating SQL query statements, OmniSQL offers a thought chain solution for each sample. This thought chain demonstrates the logical reasoning process from understanding the natural language question to generating the SQL query. It helps users better understand the model’s decision-making path and also facilitates developers in debugging and optimizing the model.
• Multiple Model Version Options: OmniSQL provides three different model sizes: 7B, 14B, and 32B. Users can choose the appropriate model version based on their actual needs and available computing resources. The different model sizes strike a balance between performance and resource consumption. Smaller models run faster and consume fewer resources, while larger models may perform better in certain complex query scenarios.

The Technical Principle of OmniSQL

• Automatically generated by the database: OmniSQL analyzes network tables, infers business scenarios, and automatically constructs a database structure with multi-table relationships and primary/foreign key constraints with the assistance of large language models. Enhanced strategies are employed to increase the number of columns and optimize the structure, making the generated database better aligned with practical applications.
• Complexity-Aware SQL Query Generation: Define four complexity levels and leverage the SQLite function library, including aggregate functions (e.g., SUM, AVG) and window functions (e.g., ROW_NUMBER, RANK), to generate various types of SQL queries. The system can intelligently select an appropriate complexity level based on user queries and provide suitable SQL statements.
• Styled Question Reverse Translation: Utilize the SQL-to-Question strategy to reverse-translate SQL queries into natural language questions in nine different linguistic styles. Semantic analysis ensures semantic consistency before and after translation, improving the efficiency and accuracy of natural language-to-SQL conversion and adapting to diverse user language preferences.
• Chain-of-Thought (CoT) Solution Synthesis: Through a step-by-step reasoning generator, intermediate derivation steps are added to samples. During training, the model learns the conversion from questions to SQL while also learning the reasoning logic at each step. This enhances reasoning accuracy and reliability, presents a transparent reasoning process to users, and builds trust.
• Large-Scale Data Synthesis and Training: OmniSQL is based on its data synthesis framework to generate a large-scale, high-quality training dataset, SynSQL-2.5M. The dataset contains over 2.5 million samples, covering more than 16,000 cross-domain databases. By training on such a large and diverse dataset, OmniSQL can learn the mapping relationships between natural language expressions and SQL queries across different domains and styles, resulting in stronger generalization ability and adaptability.

The project address of OmniSQL

• Github Repository: https://github.com/RUCKBReasoning/OmniSQL
• arXiv Technical Paper: https://arxiv.org/pdf/2503.02240

Application scenarios of OmniSQL

• Enterprise Data Analysis: With its natural language query feature, OmniSQL enables non-technical users to easily retrieve the required information from databases.
• Education Field: In SQL teaching, OmniSQL’s Chain-of-Thought (CoT) solution helps beginners better understand the conversion process from natural language questions to SQL queries. Teachers can use OmniSQL to generate query examples, allowing students to master SQL concepts and techniques through hands-on operations.
• Cross-domain Adaptability: Based on its data synthesis framework, OmniSQL can quickly generate specific domain datasets. In the medical field, it can generate EHRSQL datasets to support medical research; in the scientific research field, it can generate ScienceBenchmark datasets to assist in scientific research data analysis.