00:00:00
I would say the rate of progress is
00:00:01
exponential right now on at least three
00:00:04
key dimensions. So number one is the
00:00:05
algorithms themselves. The models are
00:00:07
improving at a rate of I don't know
00:00:08
three to four times a year. They're not
00:00:10
just getting faster and better. But
00:00:13
qualitatively they're different.
00:00:14
Remember we started with pure LLM chat
00:00:16
bots. Then we went to reasoning models.
00:00:18
We didn't even get to the agents part of
00:00:20
it yet, but that's the next big leap
00:00:22
after reasoning models. We're just
00:00:23
starting to scratch the surface there.
00:00:25
Then you've got the chips. Depending on
00:00:27
how you measure it, each generation of
00:00:28
chips is probably three or four times
00:00:30
better than the last. It's not just the
00:00:32
individual chips that are getting
00:00:33
better. They're figuring out how to
00:00:34
network them together. Like with NVL72,
00:00:36
it's like a rack system to create much
00:00:38
better performance at the data center
00:00:40
level. And that would be the third area
00:00:41
where you're seeing basically
00:00:42
exponential progress. Just look at the
00:00:44
number of GPUs are being deployed in
00:00:46
data centers. So when Elon first started
00:00:48
training Grock, I think they had maybe
00:00:50
100,000 GPUs. Now they're up to 300,000.
00:00:52
They're on the way to a million. Same
00:00:53
thing with OpenAI's data center.
00:00:56
Stargate and within a couple years
00:00:57
they'll be at I don't know 5 million
00:00:59
GPUs, 10 million GPUs. The algorithms,
00:01:02
the chips and the data centers are all
00:01:05
improving or scaling at a rate of I
00:01:07
don't know 3 to 4x a year. That's 10x
00:01:09
every 2 years. Where people don't
00:01:11
understand exponential progress is that
00:01:13
if you're getting better at 10x every 2
00:01:15
years, that doesn't mean you'll be at
00:01:16
20x in 4 years. It means you'll be at
00:01:18
100x. So you multiply those things
00:01:20
together, the algorithms, the chips, and
00:01:22
then the raw compute that's available,
00:01:24
you're talking about a millionx
00:01:25
increase. Some of which will be captured
00:01:27
in price reductions, some of it will be
00:01:30
in the performance ceiling, and then
00:01:31
some of it will just be in the overall
00:01:33
amount of AI compute that's available to
00:01:35
the economy. But the impact of this
00:01:37
thing is going to be absolutely massive.
00:01:39
And I think people still don't even
00:01:40
appreciate that fact cuz they don't
00:01:41
understand exponential progress.
