Qualcomm’s AI250 Attacks the AI Inference Memory Bottleneck | Durga Malladi Interview

AI’s memory bottleneck is breaking. Qualcomm is directly attacking the Total Cost of Ownership (TCO) for AI inference with its new AI200 and AI250 data center solutions, featuring a projected 10x leap in effective memory bandwidth.

I sat down with Durga Malladi, SVP & GM of Technology Planning, Edge Solutions, and Data Center at Qualcomm, for a deep dive into this new hardware. We discuss how their "Near-Memory Computing" architecture in the AI250 is an assault on the token generation bottleneck, specifically targeting the ‘decode’ phase to boost tokens-per-second and crush TCO.

This isn’t just about raw performance; it’s a strategic "mix and match" play. Durga explains how hyperscalers & CSPs can integrate the AI250’s capabilities alongside their own custom silicon, offering a new level of flexibility. We also cover how Qualcomm is leveraging its decades of experience with the Hexagon NPU (from phones to auto) to create these new, highly scaled, direct liquid-cooled rack-level solutions.

🔔 Subscribe to our channel for more great content just like this: https://youtube.com/twimlai?sub_confirmation=1

📖 CHAPTERS
===============================
00:00 – Introduction to Qualcomm’s New AI Inference Products
00:33 – AI200 – Direct Liquid Cooled Rack
01:23 – AI250 – Innovative Memory Architecture
02:26 – Target Customers: Hyperscalers and CSPs
03:08 – Lessons Learned Since the Original AI 100
04:40 – Hexagon NPU & Continuity of Architectures Across Devices
05:58 – Qualcomm’s Edge in a Competitive Market
07:50 – Deep Dive: Near-Memory Computing Architecture
09:57 – How Architecture Impacts Inference Performance
10:45 – Software and Ecosystem Strategy
12:30 – Go-to-Market Strategy for Data Center AI
13:45 – The Product Roadmap and Annual Cadence
14:38 – Conclusion

🤝 ABOUT OUR GUEST
===============================
Durga Malladi is Senior Vice President and General Manager of Technology Planning, Edge Solutions, and Data Center at Qualcomm Technologies, Inc.

🔗 LINKS & RESOURCES
===============================
Qualcomm Announces AI200 & AI250: https://www.qualcomm.com/news/releases/2025/10/qualcomm-unveils-ai200-and-ai250-redefining-rack-scale-data-cent
Qualcomm Data Center Offerings: https://www.qualcomm.com/artificial-intelligence/data-center
Speculative Decoding & Efficient LLM Inference: https://twimlai.com/podcast/twimlai/speculative-decoding-and-efficient-llm-inference/

🗣️ CONNECT WITH US!
===============================
Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/
Follow us on Twitter: https://x.com/twimlai
Follow us on LinkedIn: https://linkedin.com/in/twimlai
Join our Slack Community: https://twimlai.com/community/
Subscribe to our newsletter: https://twimlai.com/newsletter/
Want to get in touch? Send us a message: https://twimlai.com/contact/

📸 Camera: https://amzn.to/3TQ3zsg
🎙️Microphone: https://amzn.to/3t5zXeV
🚦Lights: https://amzn.to/3TQlX49
🎛️ Audio Interface: https://amzn.to/3TVFAIq
🎚️ Stream Deck: https://amzn.to/3zzm7F5