🚨The Bitter Lesson: Grok 4's breakthrough and how Elon leapfrogged the competition in AI

All-In Podcast

Ask about All-In Podcast

Episode Description

Full episode: https://youtu.be/KypnjJSKi4o

Follow the besties: https://x.com/chamath https://x.com/Jason https://x.com/DavidSacks https://x.com/friedberg

Follow on X: https://x.com/theallinpod

Follow on Instagram: https://www.instagram.com/theallinpod

Follow on TikTok: https://www.tiktok.com/@theallinpod

Follow on LinkedIn: https://www.linkedin.com/company/allinpod

Intro Music Credit: https://rb.gy/tppkzl https://x.com/yung_spielburg

Intro Video Credit: https://x.com/TheZachEffect

Brief Summary

Elon Musk's AI strategies emphasize general learning over human knowledge, reshaping industry standards and applications.

Key points

  • The Bitter Lesson. A fundamental architectural decision in AI emphasizes general learning over human knowledge.
  • Implications for AI Companies. The shift towards general computation raises questions about the strategies of major AI players.

Transcript

two things. One is how quickly starting in March of 2023, so we're talking about less than two and a half years, what this team has accomplished and how far ahead they are of everybody else as demonstrated by this. But the second is a fundamental architectural decision that Elon made which I think we didn't fully appreciate until now. And it maps to an architectural decision he made at Tesla as well. And for all we know, we'll figure out that he made an equivalent decision at SpaceX. And that decision is really well encapsulated by this essay, the bitter lesson by Rich Sutton. And Nick, if you can just throw this up here, but just to summarize what

this says, it basically says in a nutshell that you're always better off when you're trying to solve an AI problem taking a general learning approach that can scale with computation because it ultimately proves to be the most effective. And the alternative would be something that's much more human labored and human involved that requires human knowledge. And so the first method, what it essentially allows you to do is view any problem as an endless scalable search or learning task. And as it's turned out, whether it's chess or go or speech recognition or computer vision, whenever there was two competing approaches, one that used general

computation and one that used human knowledge, the general computation problem always won. And so it creates this bitter lesson for humans that want to think that we are at the center of all of this critical learning and all of these leaps. And what these results show is a general computational approach that doesn't require as much human labeling can actually get to the answer and better answers faster. That has huge implications because if you think about all these other companies, what is Llama been doing? They just spent 15 billion to buy 49% of scale AI. That's exactly a bet on human knowledge. What is Gemini

doing? What is OpenAI doing? What is Anthropic doing? So all these things come into question. I just think it's an incredible moment in technology where we see so many examples. Travis is another one. What he's just talked about, you know, the bitter lesson is you could believe that, you know, food is this immutable thing that's made meticulously by hand by these individuals. Or you can take this general purpose computer approach, which is what he took, waited for these cost curves to come into play, and now you can scale food to every human on Earth. I ju I just think it's a it's so profoundly important.

Related episodes