FS-DFM – A diffusion language model jointly developed by Apple and The Ohio State University

What is FS-DFM？

FS-DFM (Few-Step Discrete Flow-Matching) is a diffusion-based language model jointly developed by Apple and The Ohio State University for fast long-text generation. The model treats the sampling steps as explicit parameters during training, enabling it to generate high-quality text in far fewer steps. By combining reliable update rules with strong teacher guidance, FS-DFM ensures accurate probabilistic updates without over-adjustment.

In language modeling benchmarks, FS-DFM achieves the perplexity level of a 1024-step discrete flow baseline with only 8 sampling steps, while improving sampling speed by 128×, greatly enhancing efficiency and throughput.

Key Features of FS-DFM

Efficient Sampling: Achieves the same performance as traditional 1024-step diffusion models with only 8 steps, improving speed by 128×.
Long-Text Generation: Designed for generating long texts, addressing the efficiency bottleneck faced by traditional autoregressive models in long-sequence generation.
Stability and Controllability: Optimized sampling rules and teacher guidance ensure a stable and controllable generation process.

Technical Principles of FS-DFM

Based on the Discrete Flow-Matching (DFM) Framework:
FS-DFM is built on the Discrete Flow-Matching (DFM) framework, which learns a probabilistic path from a noise distribution to a target distribution to generate text. DFM leverages the properties of Continuous-Time Markov Chains (CTMC) to enable parallel text generation, significantly improving efficiency over traditional autoregressive models.

Explicit Sampling Steps:
FS-DFM introduces explicit step parameterization, training the model to maintain consistent generation quality under different step budgets. This means the model can reach the same performance as traditional methods (e.g., 1024 steps) with far fewer steps (e.g., 8 steps), greatly reducing computational cost and generation time.

Reliable Update Rules:
To ensure stability and accuracy in few-step generation, FS-DFM introduces a reliable update rule that controls the direction and magnitude of probabilistic updates, preventing overshooting and maintaining a stable generation trajectory.

Cumulative Scalar:
FS-DFM introduces the concept of a Cumulative Scalar, which integrates the scheduler’s rate over time intervals to provide correct probabilistic flow for each finite step. This enables effective updates during few-step generation, ensuring sufficient update strength in early stages and preventing the process from stagnation.

Project Information

arXiv Technical Paper: https://arxiv.org/pdf/2509.20624

Application Scenarios of FS-DFM

Content Creation: Quickly generates high-quality long-form text such as articles, stories, and news reports, helping creators boost productivity.
Intelligent Customer Service: Enables fast generation of detailed responses, improving response time and user experience.
Language Translation: Efficiently generates long text for large-document translation, improving translation speed and accuracy.
Creative Writing: Assists writers and creators by generating story outlines, scripts, or poems to spark new ideas.
Education: Generates educational materials such as course outlines, teaching cases, and exercises to support instructors in lesson preparation.