Do AI Tokenomics Matter More Than Model Benchmarks?

As reasoning models consume more tokens and AI systems become more expensive to run, understanding what those tokens actually buy is becoming increasingly important. In this episode, Stanford professor and Big Spin co-founder Chris Potts joins us to discuss AI tokenomics and his research into “tokenflation”—the possibility that token usage is growing faster than the measurable value those tokens produce.

We explore how to measure the return on AI spending, why benchmarks alone provide an incomplete picture of model progress, and what inference-time scaling means for the economics of increasingly capable models. Chris also explains why expert AI users tend to get better results by challenging and iterating with models, how AI fluency affects outcomes, and why more efficient architectures could change the underlying economics. We also discuss DSPy, interpretability, the limits of today’s transformer architectures, and where Chris sees opportunities for more fundamental innovation in AI.

🗒️ Full show notes including references: https://twimlai.com/go/776.

🔔 Subscribe to our channel for more great content just like this:
https://youtube.com/twimlai?sub_confirmation=1

📖 CHAPTERS
===============================
00:00 – Introduction
05:33 – Linguistics in the Age of Language Models
09:25 – Scale Limitations in NLP Research
12:54 – Challenging the Bitter Lesson Mindset
15:08 – Relationship Between Data, Mechanistic Interpretability, and Efficiency
17:03 – DSPy
21:32 – Prompt Optimization and Model Variability
24:35 – Tokenomics and the Rising Cost of AI
28:13 – Measuring Token Purchasing Power with a CPI
31:55 – Inference-Time Scaling
35:36 – Defining Value Across Different AI Tasks
38:33 – AI Value Creation
40:38 – Predicting AI Costs
42:45 – Tokenflation
46:22 – AI Fluency
50:12 – Key Lessons of AI Fluency Work
54:44 – Future Directions

🗣️ CONNECT WITH US!
===============================
Subscribe to the TWIML AI Podcast: https://twimlai.com/podcast/twimlai/
Follow us on Twitter: https://twitter.com/twimlai
Follow us on LinkedIn: https://www.linkedin.com/company/twimlai/
Join our Slack Community: https://twimlai.com/community/
Subscribe to our newsletter: https://twimlai.com/newsletter/
Want to get in touch? Send us a message: https://twimlai.com/contact/