00:00:00
I would say the rate of progress is exponential right now on at least three key dimensions. So number one is the
00:00:05
algorithms themselves. The models are improving at a rate of I don't know three to four times a year. They're not
00:00:10
just getting faster and better. But qualitatively they're different. Remember we started with pure LLM chat
00:00:16
bots. Then we went to reasoning models. We didn't even get to the agents part of
00:00:20
it yet, but that's the next big leap after reasoning models. We're just starting to scratch the surface there.
00:00:25
Then you've got the chips. Depending on how you measure it, each generation of chips is probably three or four times
00:00:30
better than the last. It's not just the individual chips that are getting better. They're figuring out how to
00:00:34
network them together. Like with NVL72, it's like a rack system to create much better performance at the data center
00:00:40
level. And that would be the third area where you're seeing basically exponential progress. Just look at the
00:00:44
number of GPUs are being deployed in data centers. So when Elon first started training Grock, I think they had maybe
00:00:50
100,000 GPUs. Now they're up to 300,000. They're on the way to a million. Same thing with OpenAI's data center.
00:00:56
Stargate and within a couple years they'll be at I don't know 5 million GPUs, 10 million GPUs. The algorithms,
00:01:02
the chips and the data centers are all improving or scaling at a rate of I don't know 3 to 4x a year. That's 10x
00:01:09
every 2 years. Where people don't understand exponential progress is that if you're getting better at 10x every 2
00:01:15
years, that doesn't mean you'll be at 20x in 4 years. It means you'll be at 100x. So you multiply those things
00:01:20
together, the algorithms, the chips, and then the raw compute that's available, you're talking about a millionx
00:01:25
increase. Some of which will be captured in price reductions, some of it will be
00:01:30
in the performance ceiling, and then some of it will just be in the overall amount of AI compute that's available to
00:01:35
the economy. But the impact of this thing is going to be absolutely massive. And I think people still don't even
00:01:40
appreciate that fact cuz they don't understand exponential progress.