WorldScore – A Unified Evaluation Benchmark for World Generative Models Launched by Stanford University

What is WorldScore?

WorldScore is a unified evaluation benchmark for world generation models proposed by Stanford University. It decomposes world generation into a series of next-scene generation tasks and achieves unified evaluation of different methods through explicit layout specifications based on camera trajectories. WorldScore evaluates three key aspects of generated worlds: controllability, quality, and dynamics. The benchmark includes a carefully curated dataset comprising 3,000 test samples, covering diverse worlds that are static and dynamic, indoor and outdoor, as well as realistic and stylized.

The main functions of WorldScore

• Unified Evaluation Framework: WorldScore provides a unified evaluation framework for measuring the performance of different world generation models. It decomposes the world generation task into a series of next-scene generation tasks, achieving unified evaluation of different methods through explicit layout specifications based on camera trajectories.
• Evaluation Dimensions: Worlds are evaluated across three key aspects: controllability, quality, and dynamism.
• Multi-scenario Generation: WorldScore is the only benchmark that supports multi-scenario generation, enabling the evaluation of models’ performance in generating consecutive scenes.
• Unity: It offers a comprehensive evaluation framework capable of assessing 3D, 4D, image-to-video (I2V), and text-to-video (T2V) models in a unified manner.
• Long Sequence Support: It supports the generation of multiple scenes, evaluating models’ performance in long-sequence generation tasks.
• Image Conditioning: It supports image-based conditional generation, making it suitable for image-to-video generation tasks.
• Multi-style: It includes datasets with various visual styles, enabling the evaluation of models’ generation capabilities across different styles.
• Camera Control: It evaluates models’ ability to follow camera trajectories, ensuring that the generated scenes align with specified camera movements.
• 3D Consistency: It assesses the geometric stability of scenes, ensuring that the generated 3D scenes remain consistent across different viewpoints.

The Technical Principle of WorldScore

• Diverse Datasets: The WorldScore dataset contains multimedia data with dynamic and static configurations, suitable for image-to-video and image-to-3D tasks.
  ◦ Dynamic Configuration: Includes fields such as images, visual motion, visual style, motion type, style, camera path, objects, and prompts.
  ◦ Static Configuration: Includes fields such as images, visual motion, visual style, scene type, category, style, camera path, content list, and prompt list.
• Dataset Scale: The dataset is divided into training and test sets, with 1,000 samples for the dynamic configuration and 2,000 samples for the static configuration.
• Camera Trajectory-Based Layout Specification: A clear camera trajectory-based layout specification is provided to enable unified evaluation across different methods.
• Multi-Modal Data Support: Supports various modalities of data, including images, videos, and 3D models, making it suitable for multi-modal content generation tasks.

The project address of WorldScore

• Project Website: https://haoyi-duan.github.io/WorldScore/
• GitHub Repository: https://github.com/haoyi-duan/WorldScore
• arXiv Technical Paper: https://arxiv.org/pdf/2504.00983
• Hugging Face Dataset: https://huggingface.co/datasets/Howieeeee/WorldScore

Comparison of WorldScore Benchmark Tests

WorldScore differs from other existing benchmark tests in several aspects. Here is a detailed comparison:

Application scenarios of WorldScore

• Image-to-Video Generation: Generate high-quality video content for applications in video production, animation design, and other related fields.
• Image-to-3D Generation: Convert 2D images into 3D models for use in virtual reality, augmented reality, and 3D modeling scenarios.
• Dataset Support: The dataset includes multimedia data with dynamic and static configurations, suitable for various tasks and assisting researchers in optimizing and improving models.
• Research and Development: The WorldScore dataset provides a standardized testing platform for researchers to develop and validate new 3D/4D scene generation algorithms.
• Autonomous Driving Scene Generation: Generate realistic 3D scenes for the training and testing of autonomous driving systems, helping to enhance the safety and reliability of these systems.