FlowDirector – A Video Editing Framework Jointly Developed by Westlake University and Central South University

What is FlowDirector?

FlowDirector is a novel training-free video editing framework jointly developed by the AGI Lab at Westlake University and Central South University. It is designed to perform precise video edits based on natural language instructions. Unlike traditional methods that rely on latent space inversion—often causing temporal inconsistency and structural distortion—FlowDirector directly models the editing process in data space, using Ordinary Differential Equations (ODEs) to guide smooth transition paths.

FlowDirector introduces a Spatially Attentive Flow Correction (SAFC) mechanism to ensure temporal-spatial consistency in unedited areas, and applies a Differential Averaging Guidance (DAG) strategy to enhance semantic alignment. The framework demonstrates outstanding performance across multiple video editing benchmarks, significantly improving instruction fidelity, temporal coherence, and background preservation, offering a new solution for efficient and coherent video editing.

Key Features of FlowDirector

• Precise Semantic Editing: Modifies video content at the semantic level according to natural language prompts—for example, replacing a “bear” in the video with a “dinosaur.”

• Temporal-Spatial Consistency: Maintains the temporal continuity and spatial structure of the video during editing, avoiding content misalignment or stylistic mismatches.

• Localized Editing with Global Preservation: Utilizes attention-guided mechanisms to edit only the target regions while preserving the original content and dynamics of untouched areas.

• Efficient Training-Free Editing: Requires no additional training; directly leverages pretrained Text-to-Video (T2V) models, reducing editing cost and increasing efficiency.

• Support for Diverse Editing Tasks: Capable of handling complex tasks such as object replacement, texture transformation, localized attribute modification, and object insertion/deletion.

Technical Foundations of FlowDirector

• Editing Flow Generation: Based on pretrained T2V models, FlowDirector computes the difference in velocity fields between the source and target videos to generate a direct editing path. This path is driven by an ODE-based smooth transition, avoiding the structural distortions common in latent-space-based methods.

• Spatially Attentive Flow Correction (SAFC): Uses attention-guided masks derived from attention maps related to the editing task. These masks control which regions are editable and which should remain unchanged. By applying these masks to the velocity field during ODE integration, non-target areas are “frozen,” ensuring their preservation.

• Differential Averaging Guidance (DAG): Inspired by Classifier-Free Guidance (CFG), DAG generates multiple candidate editing flows and computes their difference signals to enhance semantic alignment. These signals are then used to refine the editing trajectory, ensuring the result aligns closely with the target semantics while preserving structural integrity.

Project Links for FlowDirector

• Project Website: https://flowdirector-edit.github.io/

• GitHub Repository: https://github.com/Westlake-AGI-Lab/FlowDirector

• arXiv Paper: https://arxiv.org/pdf/2506.05046

Application Scenarios for FlowDirector

• Visual Effects Production: Quickly generate visual effects from simple text instructions—for instance, replacing a “car” with a “dragon”—to add creativity to film and video productions.

• Commercial Video Editing: Tailor video content to match advertising scripts by replacing product appearances or backgrounds with brand-aligned elements, enhancing relevance and appeal.

• Animation Production: Accelerate animation workflows by modifying characters or scenes—for example, changing a character’s clothing from “red” to “blue,” or converting a “forest” setting into a “city.”

• Personalized Video Editing: Allow users to easily customize short videos by replacing pets with other animals or adding fun effects, increasing video engagement and entertainment value.

• Creative Content Generation: Enable creators to rapidly generate videos in specific themes or styles, such as transforming a normal landscape into a cyberpunk-style scene—meeting the demand for creative content on social media.