Alibaba's new AI model turns your prompts into 3D worlds
What's the story
Alibaba has unveiled its latest artificial intelligence (AI) model, Happy Oyster. The innovative model can generate interactive videos and 3D environments that mimic real-world physics. Happy Oyster goes up against Tencent's Humayun HY-World 2.0 model, which can reconstruct full 3D worlds from video clips.
Model innovation
Model goes beyond traditional video generation tools
Alibaba has touted Happy Oyster as a revolutionary AI model that goes beyond traditional video generation tools. The company explained, "Unlike traditional AI video tools that follow a one-shot workflow of 'write a prompt, wait for rendering, receive a finished clip,' Happy Oyster keeps listening and responding throughout the generation process." This means the scene reacts in real time and continues to evolve as users provide instructions.
Operational modes
Directing mode lets users intervene at any moment
Happy Oyster operates through two main modes: Directing and Wandering. The Directing mode creates a "running physical world where lighting, gravity, character motion, and scene causality remain continuous and coherent over time." Users can intervene at any moment using text, voice or image prompts to change camera angles, direct characters or alter storylines in real time.
Model versatility
Wandering mode lets users create infinitely extendable world
The Wandering mode of Happy Oyster lets users create and explore an infinitely extendable world from a single text or image prompt. Alibaba sees this tech powering everything from real-time film production to storyboarding and next-level gaming experiences. The company also said that the model can be used for rapid storyboarding, interactive short series where viewer choices make every watch unique, and concept development in gaming sector.