Genie 3: A New Frontier for World Models [Jack Parker-Holder and Shlomi Fruchter] – 743

Today, we’re joined by Jack Parker-Holder and Shlomi Fruchter, researchers at Google DeepMind, to discuss the recent release of Genie 3, a model capable of generating “playable” virtual worlds. We dig into the evolution of the Genie project and review the current model’s scaled-up capabilities, including creating real-time, interactive, and high-resolution environments. Jack and Shlomi share their perspectives on what defines a world model, the model’s architecture, and key technical challenges and breakthroughs, including Genie 3’s visual memory and ability to handle “promptable world events.” Jack, Shlomi, and Sam share their favorite Genie 3 demos, and discuss its potential as a dynamic training environment for embodied AI agents. Finally, we will explore future directions for Genie research.

🗒️ For the full list of resources for this episode, visit the show notes page: https://twimlai.com/go/743.

🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1

🗣️ CONNECT WITH US!
===============================
Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/
Follow us on Twitter: https://twitter.com/twimlai
Follow us on LinkedIn: https://www.linkedin.com/company/twimlai/
Join our Slack Community: https://twimlai.com/community/
Subscribe to our newsletter: https://twimlai.com/newsletter/
Want to get in touch? Send us a message: https://twimlai.com/contact/

📖 CHAPTERS
===============================
00:00 – Introduction
7:11 – What is a world model?
14:49 – Milestones of Genie research
24:32 – Genie 3
27:46 – Challenges
30:07 – Genie 3 examples
33:48 – Model capabilities
35:49 – Key aspects of the model
39:40 – Consistency as an emergent property
42:11 – Promptable word events
47:24 – SIMA agent
50:56 – Limitations
56:08 – Future directions

🔗 LINKS & RESOURCES
===============================
Genie 3: A new frontier for world models – https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/
Genie 2: A large-scale foundation world model – https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/
Genie: Generative Interactive Environments paper – https://arxiv.org/abs/2402.15391
A generalist AI agent for 3D virtual environments – https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/
Genie: Generative Interactive Environments with Ashley Edwards – 696 – https://twimlai.com/podcast/twimlai/genie-generative-interactive-environments/

📸 Camera: https://amzn.to/3TQ3zsg
🎙️Microphone: https://amzn.to/3t5zXeV
🚦Lights: https://amzn.to/3TQlX49
🎛️ Audio Interface: https://amzn.to/3TVFAIq
🎚️ Stream Deck: https://amzn.to/3zzm7F5