GameRealm Dataset
GameRealm brings together diverse game environments, multiple viewpoints, and multimodal observations, including RGB, depth, camera parameters, point tracks, audio, text, and control inputs.
Content Diversity
Viewpoint Diversity
Single-view
A first-person window into interactive environments.
Stereo
Paired perspectives for studying spatial structure.
Multi-view
Complementary viewpoints of a shared scene.
Panoramic
A wider context for immersive world understanding.
Modality Diversity
Depth
GT Per-pixel distance and scene structure.
Camera
GT Camera intrinsics and extrinsics.
Tracking
GT Point trajectories through space and time.
Audio
GT Sound synchronized with visual observations.
Text
Qwen Language describing scenes and events.
Keypress
GT Keyboard and mouse inputs aligned with gameplay.
Finetuned Models Based on GameRealm
Explore models finetuned on GameRealm for interactive world modeling and visual geometry learning, with demonstrations of action-conditioned video generation, 3D reconstruction, and point tracking.
Interactive World Model
Predict how interactive worlds evolve under actions.
Visual Geometry Learning
Recover 3D structure and track motion across views.
World Model (Finetuned Helios)
Multi-View World Model (Finetuned Helios)
Panoramic World Model (Finetuned LTX-2.0)
Reconstruction (Finetuned Pi3 and Fast-FoundationStereo)
Tracking (Finetuned TAPIP3D)
CONTINUE EXPLORING
Project resources
Research materials will be linked here as they become available.
Technical report
The ideas, design, and evaluation behind GameRealm.
Research code
Implementations and tools for getting started.
Dataset
Data access, annotations, and documentation.
Pi4
Explore the Pi4 project and its interactive demonstrations.
Citation
