AI Enables Zero-Shot Transfer in 3D Games

Researchers from Peking University and the University of California, Los Angeles (UCLA) have unveiled ROCKET-2, an intelligent agent that marks a significant breakthrough in zero-shot transfer capabilities of AI within 3D gaming environments.

Pretrained solely in Minecraft, ROCKET-2 demonstrates the remarkable ability to adapt directly to previously unseen 3D games, such as Unreal Engine 5 and DOOM, without any additional fine-tuning.

At the heart of ROCKET-2 is an innovative mechanism called Cross-View Goal Alignment, which overcomes the limitations of traditional instruction spaces and enables more natural and efficient human-AI interaction.

Compared to its predecessor, ROCKET-2 achieves a 3x to 6x increase in inference speed and attains near 100% state-of-the-art performance across a variety of tasks. These advancements highlight the system’s powerful capabilities in active understanding and generalization, pushing the boundaries of what AI can achieve in complex, dynamic 3D environments.