🚨The Bitter Lesson: Grok 4's breakthrough and how Elon leapfrogged the competition in AI
Ask about All-In Podcast
Episode Description
Full episode: https://youtu.be/KypnjJSKi4o
Follow the besties: https://x.com/chamath https://x.com/Jason https://x.com/DavidSacks https://x.com/friedberg
Follow on X: https://x.com/theallinpod
Follow on Instagram: https://www.instagram.com/theallinpod
Follow on TikTok: https://www.tiktok.com/@theallinpod
Follow on LinkedIn: https://www.linkedin.com/company/allinpod
Intro Music Credit: https://rb.gy/tppkzl https://x.com/yung_spielburg
Intro Video Credit: https://x.com/TheZachEffect
Brief Summary
Elon Musk's AI strategies emphasize general learning over human knowledge, reshaping industry standards and applications.
Key points
- The Bitter Lesson. A fundamental architectural decision in AI emphasizes general learning over human knowledge.
- Implications for AI Companies. The shift towards general computation raises questions about the strategies of major AI players.
Episode highlights
- The Bitter LessonA fundamental architectural decision in AI emphasizes general learning over human knowledge.“You're always better off when you're trying to solve an AI problem.”0:53Jul 12, 2025
- Implications for AI CompaniesThe shift towards general computation raises questions about the strategies of major AI players.“What is Llama been doing?”2:07Jul 12, 2025
Transcript
two things. One is how quickly starting in March of 2023, so we're talking about less than two and a half years, what this team has accomplished and how far ahead they are of everybody else as demonstrated by this. But the second is a fundamental architectural decision that Elon made which I think we didn't fully appreciate until now. And it maps to an architectural decision he made at Tesla as well. And for all we know, we'll figure out that he made an equivalent decision at SpaceX. And that decision is really well encapsulated by this essay, the bitter lesson by Rich Sutton. And Nick, if you can just throw this up here, but just to summarize what
this says, it basically says in a nutshell that you're always better off when you're trying to solve an AI problem taking a general learning approach that can scale with computation because it ultimately proves to be the most effective. And the alternative would be something that's much more human labored and human involved that requires human knowledge. And so the first method, what it essentially allows you to do is view any problem as an endless scalable search or learning task. And as it's turned out, whether it's chess or go or speech recognition or computer vision, whenever there was two competing approaches, one that used general
computation and one that used human knowledge, the general computation problem always won. And so it creates this bitter lesson for humans that want to think that we are at the center of all of this critical learning and all of these leaps. And what these results show is a general computational approach that doesn't require as much human labeling can actually get to the answer and better answers faster. That has huge implications because if you think about all these other companies, what is Llama been doing? They just spent 15 billion to buy 49% of scale AI. That's exactly a bet on human knowledge. What is Gemini
doing? What is OpenAI doing? What is Anthropic doing? So all these things come into question. I just think it's an incredible moment in technology where we see so many examples. Travis is another one. What he's just talked about, you know, the bitter lesson is you could believe that, you know, food is this immutable thing that's made meticulously by hand by these individuals. Or you can take this general purpose computer approach, which is what he took, waited for these cost curves to come into play, and now you can scale food to every human on Earth. I ju I just think it's a it's so profoundly important.





