InftyThink – An Infinite Depth Reasoning Paradigm Jointly Launched by Zhejiang University and Peking University

What is InftyThink?

InftyThink is an innovative large model reasoning paradigm that breaks through the limitations of traditional models in long-chain reasoning tasks. By employing a segmented, iterative approach, it decomposes complex reasoning processes into multiple short segments. After each segment, it generates an intermediate summary, enabling chunk-based thinking. This “zigzag” memory pattern—periodically discarding old details while retaining new summaries—effectively reduces computational complexity, allowing models to handle theoretically infinite-length reasoning chains.

Technical Principles of InftyThink

• Iterative Reasoning with Staged Summarization:
  InftyThink decomposes traditional monolithic reasoning into several shorter reasoning segments. After each segment, a concise summary is generated, which then serves as context for the next stage of reasoning. This mimics the human cognitive process of incremental summarization and enables infinite-depth reasoning while maintaining contextual coherence. It addresses the challenges of limited context length and high computational complexity found in conventional long reasoning approaches.

• Fixed Computational Overhead and Context Window:
  InftyThink implements a “zigzag” memory usage pattern. After each short reasoning step, the previous context is cleared, and only the summary is retained. This significantly reduces the computational cost of inference. Compared to traditional paradigms, InftyThink achieves a more efficient balance between reasoning depth and computation.

• Decoupled from Model Architecture with High Compatibility:
  InftyThink does not rely on modifications to the model architecture. Instead, it restructures training data into a multi-step reasoning format. It is compatible with existing pretrained models and integrates seamlessly with fine-tuning and reinforcement learning pipelines, offering strong practical deployability.

• Data Reconstruction Techniques:
  InftyThink has developed methods for converting existing long-form reasoning datasets into an iterative format. For instance, it transforms the OpenR1-Math dataset into 333,000 training instances, making it suitable for training under this paradigm.

Key Advantages of InftyThink

• Breaks Context Window Limitations:
  Through iterative reasoning and intermediate summarization, InftyThink can handle reasoning chains of theoretically unlimited length, overcoming the constraints of traditional model context windows.

• Reduces Computational Costs:
  Compared to traditional long-form reasoning methods, InftyThink reduces the need for direct processing of long sequences, significantly lowering computational costs.

• Improves Reasoning Performance:
  In complex reasoning tasks, InftyThink handles long-sequence information more effectively, improving both accuracy and generation throughput.

Project Resources

• Official Website: https://zju-real.github.io/InftyThink/

• GitHub Repository: https://github.com/ZJU-REAL/InftyThink

• arXiv Paper: https://arxiv.org/pdf/2503.06692

Application Scenarios for InftyThink

• Mathematical Problem Solving:
  InftyThink can tackle complex mathematical problems by using segmented, iterative reasoning and intermediate summarization to progressively solve long-chain math problems.

• Logical Reasoning:
  In tasks requiring long-sequence logical inference, InftyThink can significantly improve both accuracy and efficiency.

• Code Generation:
  For code generation tasks, InftyThink can incrementally produce complex code logic while maintaining coherence and correctness.

• Intelligent Tutoring:
  In intelligent tutoring systems, InftyThink can provide step-by-step detailed explanations based on students’ questions, helping them better understand and master knowledge points.

• Drug Discovery:
  InftyThink can be used to predict 3D structures of drug targets and their binding affinity, accelerating the drug development process.