GameGen-X: Interactive Open-world Game Video Generation
Part I for the basic functionality showcase, and Part II for the key features of GameGen-X (0:47).
Part I for the basic functionality showcase, and Part II for the key features of GameGen-X (0:47).
Character Generation
Geralt of Rivia
Arthur Morgan
Eivor
Jin Sakai
Astroneer
Ice Magician
RoboCop
Security Guard
Environment Generation
Spring
Summer
Autumn
Winter
Lake
Sea
Lavender Field
Pyramid
Action Generation
Motorcycling (first-person)
Driving
Flying
Sailing
Motorcycling (third-person)
Walking
Riding
Carriage
Event Generation
Raining
Snowing
Thundering
Sunrising
Firing
Sandstormig
Tsunami
Tornado
Open-domain Generation
Cybermonk roaming in China town
TimeMaster standing in another dimension
Traveler with a cloak walking on Mars
Magic steam airship soaring in the clouds
Ghost walking under the blood moon
Venom Druid touring Runeforest
Angel looking at the Holy Kingdom
Mechanical life passing through the ruins
Structural Instruction Prompts
Fire on the sky
Dark and star
Sunset happens
Fog emerges
Operation Signals
Move left (A)
Move right (D)
Move left (A)
Move right (D)
Video Prompts
Canny Prompt
Output Video 1
Motion Vector
Output Video
In-domain Generation Comparision
GameGen-X
OpenSora-Plan
OpenSora
CogVideoX
GameGen-X
OpenSora-Plan
OpenSora
CogVideoX
Open-domain Generation Comparision
GameGen-X
OpenSora-Plan
OpenSora
CogVideoX
GameGen-X
OpenSora-Plan
OpenSora
CogVideoX
GameGen-X
OpenSora-Plan
OpenSora
CogVideoX
GameGen-X
OpenSora-Plan
OpenSora
CogVideoX
GameGen-X
OpenSora-Plan
OpenSora
CogVideoX
Control Comparision
GameGen-X
Luma
Kling
Tongyi
GameGen-X
Luma
Kling
Tongyi
GameGen-X
Luma
Kling
Tongyi
OGameData Summary: OGameData is a comprehensive multi-genre open-world video game dataset, which contains generation and control subsets. Sourcing over 32,000 videos from local engines and the internet, each video ranges from several minutes to several hours in length. The dataset features more than 150 next-generation games across various genres, including open-world RPGs, FPS, racing games, action-puzzle games, and more. It also covers different perspectives (first-person, third-person) and styles (realistic, Eastern traditional, cyberpunk, post-apocalyptic, Western fantasy, etc.). After a rigorous selection process that spanned six months and involved multiple human experts and advanced model algorithms, we have curated over 4,000 hours of high-quality video clips, ranging from 720p to 4k resolution. These segments were meticulously annotated by GPT-4O, providing a rich source of labeled data for training and validation purposes. The OGameData is expected to become an invaluable resource for researchers and developers, enabling the exploration of various applications such as video game generative AI development, interactive control, and immersive virtual environments. Its imminent open-source release will offer the scientific community unprecedented access to a broad spectrum of video game data, fostering innovation and collaboration across multiple disciplines.
OGameData Preview Version: https://drive.google.com/file/d/1PhCM-_bnQKHAn2Otvgq0iQefAip9yVHy/view?usp=drive_link
OGameData for Generation Training
A person in a trench coat and hat walks along a riverbank, approaching wooden houses on a misty morning. In this atmospheric sequence from the action-adventure game Red Dead Redemption 2, Arthur Morgan is depicted walking along a serene riverbank, highlighted by his distinctive wide-brimmed white hat and dark trench coat. The environment features a tranquil riverside setting bathed in golden sunlight with mist lingering over distant forests. The camera tailing Arthur captures steady shots that subtly reveal more of the lush greenery and rustic buildings emerging on the left bank as he proceeds forward, reflecting the game's characteristic blend of exploration and attention to scenic detail,
Cars drive through a city intersection at sunset, with a horse statue visible in the background. In this sequence from the open-world game Grand Theft Auto V, a sleek black muscle car is seen navigating through a downtown intersection at dusk. The scene captures the essence of urban life with palm trees lining the streets and modern buildings in the background, one prominently featuring an imposing silver statue of a rearing horse. As the sky glows with a soft purple and pink hue, suggesting early evening, various camera angles provide expansive and cinematic views: starting with wide shots to establish the setting and transitioning into closer perspectives that capture details like traffic lights turning green and an approaching garbage truck on one side. The environment effectively conveys a rich, immersive atmosphere typical of GTA V’s detailed cityscapes.
OGameData for Instruction Tuning
Environmental Basics: Widen the path in front of the main character as they walk forward. Main Character: Move steadily along the path, decreasing distance to distant buildings. Environmental Changes: Enhance visibility and detail of approaching village structures over time. Sky/Lighting: Maintain clear skies and consistent daylight throughout. aesthetic score: 5.47, motion score: 15.37, camera motion: pan_right, perspective: third, shot size: full.
Environmental Basics: Show a lush, green countryside path lined with stone walls and trees under a sunset sky. Main Character: Have the main character riding steadily forward on horseback along the path. Environmental Changes: Slowly move the horse and rider deeper into the scene along the path. Sky/Lighting: Maintain consistent golden sunset lighting throughout. aesthetic score: 5.54, motion score: 8.45, camera motion: zoom_in, perspective: third, shot size: full.
Acknowledgements: Our project page is borrowed from DreamBooth.