SurveyForge – An AI-Automated Academic Survey Generation Framework

What is SurveyForge?

SurveyForge is an innovative framework developed by Shanghai AI Lab in collaboration with institutions such as Fudan University and Shanghai Jiao Tong University, designed to automate the generation of high-quality academic surveys. The framework adopts a two-stage design: outline generation and content generation.

In the outline generation stage, it employs a heuristic learning mechanism driven by dual databases (a research paper database and a survey outline database) to produce logically structured and coherent outlines. In the content generation stage, the Scholar Agent Navigator (SANA) uses a memory module and a time-aware re-ranking engine to accurately retrieve literature and generate high-quality content.

SurveyForge’s parallel generation and coordination mechanism enables efficient production of coherent long-form documents. It can generate a survey of around 64k tokens in just 10 minutes, with a cost of less than $0.5. The team has also developed SurveyBench, a multi-dimensional evaluation benchmark for objectively assessing the quality of generated surveys.

Key Features of SurveyForge

• Efficient Survey Generation: Generates approximately 64k-token surveys in just 10 minutes, with a cost of under $0.5.

• High-Quality Outline Creation: Uses a dual-database driven heuristic learning mechanism to produce logically structured outlines that rival those written by humans.

• Accurate Literature Retrieval: The SANA agent accurately retrieves high-quality, topic-relevant papers, avoiding irrelevant citations and enhancing the survey’s authority.

• Content Optimization and Coordination: Through parallel generation and refinement, each section is independently produced and then coordinated to remove redundancy and ensure logical consistency across the full document.

Technical Principles of SurveyForge

Outline Generation Stage

• Dual-Database Driven Learning: Builds a research paper database (~600,000 arXiv CS papers) and a hierarchical survey outline database (~20,000 survey article outlines). It fuses knowledge across these databases to retrieve topic-relevant materials and learn expert-level structuring patterns.

• Recursive Construction Strategy: First generates high-level global outline, then recursively refines each section into more detailed substructures, achieving structured learning from global to local levels.

Content Generation Stage

• Scholar Agent Navigator (SANA):

  • Sub-query Memory Module: Retains the set of literature retrieved during outline generation as contextual memory. Combined with the original query, it ensures decomposition remains focused and avoids semantic drift.

  • Retrieval Memory Module: Uses the entire topic-related literature as global memory, retrieving the most relevant papers for each sub-query based on embedding similarity, improving retrieval accuracy and semantic coherence.

  • Time-Aware Re-ranking Engine: Groups retrieved papers by publication date and applies top-k filtering within each group based on citation count—balancing authoritative classics and cutting-edge research.

• Parallel Generation and Coordination: Each section is generated independently, guided by a shared memory system to maintain thematic consistency. A refinement phase eliminates redundancy and smooths transitions for a coherent final output.

Project Links for SurveyForge

• GitHub Repository: https://github.com/Alpha-Innovator/SurveyForge

• HuggingFace Dataset: https://huggingface.co/datasets/U4R/SurveyBench

• arXiv Paper: https://arxiv.org/pdf/2503.04629

Application Scenarios for SurveyForge

• Academic Research: Assists newcomers, interdisciplinary researchers, and experts in quickly gaining a comprehensive understanding of a field and improving literature review efficiency.

• Education: Helps educators design courses and students learn domain knowledge efficiently.

• Industry: Supports technical research and industry analysis for informed R&D decision-making.

• Policy Making: Offers scientific support for policymakers in technology assessment and policy planning.

• Personal Learning and Research: Enables individuals to quickly explore new topics and assists in writing papers.