00:00:00
So much of what's happening today in the AI industry is extremely inhumane. >> But this is me playing devil's advocate.
00:00:06
And logically, it could be the case that the civilization that accelerate their research with AI is going to be the
00:00:12
superior civilization. >> No, it's not. This is a prediction that you're making, right?
00:00:15
>> Making Zuckerberg's making. >> And do you know what the common feature
00:00:19
of all of them is? They profit enormously off of this myth. You know, I have all these internal documents
00:00:24
showing that they're purposely trying to create that feeling within the public so
00:00:28
that they can extract and exploit and extract and exploit. So, what do we do about it?
00:00:32
>> We need to break up the empires of AI. >> You know, I've been covering the tech
00:00:36
industry for over 8 years, interviewed over 250 people, including former or current OpenAI employees and executives.
00:00:42
And I can tell you that there are many parallels between the empires of AI and the empires of old, right? like Lelay
00:00:48
claimed the intellectual property of artists, writers, and creators in the pursuit of training these models.
00:00:52
Second, they exploit an extraordinary amount of labor, which breaks the career ladder because someone gets laid off and
00:00:57
then they work to train the models on the very job that they were just laid off in, which will then perpetuate more
00:01:03
layoffs if that model then develops that skill. And when they talk about that there's going to be some new jobs
00:01:08
created that we can't even imagine, a lot of the jobs that are created are way
00:01:12
worse than the jobs that were there. And then there's the environmental and public health crisis that these
00:01:17
companies have created and how they're able to also spend hundreds of millions
00:01:21
to try and kill every possible piece of legislation that gets in their way and will censor researchers that are
00:01:27
inconvenient to the empire's agenda. But what I'm saying is not that these
00:01:32
technologies don't have utility. It's that the production of these technologies right now is exacting a lot
00:01:37
of harm on people. But we have research that shows that the very same capabilities could be developed in a
00:01:44
different way that doesn't have all of these unintended consequences. So let's
00:01:48
talk about all of that. This is super interesting to me. My team given me this report to show me how many
00:01:56
of you that watch this show subscribe. And some of you have told us according to this that you are unsubscribed from
00:02:01
the channel randomly. So favor to ask all of you. Please could you check right now if you've hit the subscribe button
00:02:06
if you are a regular viewer of the show and you like what we do here. We're approaching quite a significant landmark
00:02:10
on this show in terms of a subscriber number. So if there was one simple free thing that you could do to help us, my
00:02:16
team, everyone here to keep this show free, to keep it improving year over year and week over week, it is just to
00:02:22
hit that subscribe button and to double check if you've hit it. Only thing I'll
00:02:24
ever ask of you, do we have a deal? If you do it, I'll tell you what I'll do.
00:02:28
I'll make sure every single week, every single month, we fight harder and harder
00:02:32
and harder and harder to bring you the guests and conversations that you want to hear. I've stayed true to that
00:02:36
promise since the very beginning of the D of Sio and I will not let you down. Please help us. Really appreciate it.
00:02:42
Let's get on with the show. Karen, how you've written this book in front of me here called Empire of AI:
00:02:52
Dreams and Nightmares in Sam Alman's Open AI. I guess my first question is what is the research and the journey you
00:03:00
went on in order to write this book we're going to talk about and the subjects within it today
00:03:04
>> I took a strange route into journalism I studied mechanical engineering at MIT
00:03:09
and so when I graduated I moved to San Francisco I joined a tech startup I became part of Silicon Valley and I
00:03:16
basically received an education in what Silicon Valley is about because a few months into joining a very missiondriven
00:03:21
startup that was focused on building technologies that would help facilitate the fight against climate change. The
00:03:27
board fired the CEO because the company was not profitable. And this was in hindsight a very pivotal moment for me
00:03:35
because I thought if this hub is ultimately geared towards building profitable technologies and many of the
00:03:43
problems in the world that I think need solved are not profitable problems like climate change. Then what are we
00:03:50
actually doing here? like what how did we get to a point where innovation is not actually necessarily working in the
00:03:57
public benefit and sometimes even undermining the public benefit in pursuit of profit. In that moment, I had
00:04:03
a bit of a crisis where I thought, well, I just spent 4 years trying to set myself up for this career that I now
00:04:11
don't think I am cut out for. And I thought, well, I might as well just try something totally different. I've always
00:04:20
liked writing and that's how after 2 years I landed at a role at MIT technology review covering AI full-time
00:04:28
and that gave me a space to then explore all of these questions of who gets to decide what technologies we build how
00:04:35
does money and ideology also drive the production of those technologies and how do we ultimately make sure that we
00:04:42
actually reimagine the innovation ecosystem to work for a broad base of people all around the world. And so that
00:04:51
is kind of how I then set off on this journey of ultimately writing a book. I didn't realize that I was working
00:04:58
towards writing a book, but starting in 2018 when I took that job was essentially the moment in which I began
00:05:06
researching the story that I I document in it. >> A very timely time to start working in
00:05:11
artificial intelligence. For anyone that doesn't know, this is pre OpenAI chat
00:05:14
GPT launch moment that shook the world. But in writing this book, you interviewed a lot of people and went to
00:05:21
a lot of places. Can you give me a flavor of how many people you've interviewed, where it's taken you around
00:05:26
the world, etc. >> I interviewed over 250 people. So over 300 interviews, over 90 of those people
00:05:32
were former or current OpenAI employees and executives. So the book covers the inside story of opening eyes's first
00:05:40
decade and how it ultimately got to where it is today. But I didn't want to write a corporate book. I felt very
00:05:47
strongly that in order to help people understand the impact of the AI industry, we would also have to travel
00:05:54
well beyond Silicon Valley. These companies tell us that AI is going to benefit everyone and that's their
00:05:59
mission. But you really start to see that rhetoric break down when you go to the places that look nothing like
00:06:07
Silicon Valley, that speak nothing like Silicon Valley, and that have a history and culture that are fundamentally
00:06:12
different as well. And that's where you start to really understand the true reality of how this industry is
00:06:21
unfolding around us. >> Karen, I often try and steer conversations, but in this situation, I
00:06:26
feel like it's probably my responsibility to follow. So with that in mind, I'm going to ask you where does
00:06:32
this journey begin and where should we be starting if we're talking about the
00:06:34
subjects of empire of AI, AI generally artificial intelligence and also I'd say
00:06:40
one thing I'm really keen to do in this conversation which is I often see in
00:06:43
conversations is left out is let's assume that our viewers know nothing about AI.
00:06:48
>> Yeah. So they don't know what scaling laws are or GPUs or comput or whatever
00:06:52
and let's try and keep this as simple as we possibly can in terms of language or
00:06:57
explain all the complicated language so that we can bring as much people with us
00:07:00
as we possibly can. >> Yes. >> Where should we start? >> I think we should start with when AI
00:07:06
started as a field. So this was back in 1956 and there were a group of scientists
00:07:13
that gathered at Dartmouth University to start a new discipline, a scientific discipline to try and chase an ambition.
00:07:20
And specifically an assistant professor at Dartmouth University, John McCarthy decided to name this discipline
00:07:25
artificial intelligence. This was not the first name that he tried. The previous year he tried to
00:07:32
name it Automata Studies. And the reason why some of his colleagues were concerned about this name was because it
00:07:38
pegged the idea of this discipline to recreating human intelligence. And back then, as is true today, we have no
00:07:47
scientific consensus around what human intelligence is. There's no definition
00:07:52
from psychology, biology, neurology. And in fact, every attempt in history to quantify and rank human intelligence has
00:08:02
been driven by nefarious motives. It's been driven by a desire to prove scientifically that certain groups of
00:08:10
people are inferior to other groups of people. There are no goalposts for this field and there are no goalposts for the
00:08:19
industry when they say that they are ultimately trying to recreate AI systems that would be as smart as humans. How do
00:08:26
we even define what that means? And when are we going to get there if we don't
00:08:31
know how to define the destination? And what that effectively means is that these companies can just use the term
00:08:39
artificial general intelligence which is now the term to refer to this ambitious
00:08:44
um goal to recreate human intelligence. They can use it however they want to and
00:08:49
they can define and redefine it based on what is convenient for them. So in OpenAI's history, it has defined and
00:08:55
redefined it many times. When Sam Alman is talking with Congress, AGI is a system that's going to cure cancer,
00:09:02
solve climate change, cure poverty. When he's talking with consumers that he's
00:09:07
trying to sell his products to, it's the most amazing digital assistant that
00:09:11
you're ever going to have. When he was talking with Microsoft, you know, in the
00:09:16
deal that OpenAI and Microsoft struck where Microsoft invested in the company, it was defined as a system that will
00:09:23
generate hundred billion of revenue. And on OpenAI's own website, they define it
00:09:28
as highly autonomous systems that outperform humans in most economically valuable work. This is like not a
00:09:36
coherent vision of one technology. These are very different definitions that are
00:09:41
spoken out loud to the audience that needs to be mobilized to ward off regulation or get more consumer buy in
00:09:50
into the the industry's quest or to get more capital more resources for continuing on this journey with
00:09:58
ambiguous definitions. I mean, speaking about different definitions through time, in 2015, in a blog post that Sam
00:10:06
Waltman wrote before open air was officially announced, he explicitly outlined the existential risk by saying,
00:10:12
"Development of superhuman machine intelligence is probably the greatest threat to the continued existence of
00:10:18
humanity. There are other threats that I think are more certain to happen, for example, an engineered virus, but AI is
00:10:24
probably the most likely way to destroy everything >> in general." When Alman is writing for
00:10:31
the public or speaking for the public, he does not just have the public as the audience in mind, there are other people
00:10:38
that he is trying to motivate or mobilize when he says these things. And in that particular moment, Alman was
00:10:46
trying to convince Elon Musk to join him on co-founding OpenAI. And Musk in particular was spending all of his time
00:10:55
sounding the alarm on what he saw as a huge existential threat that AI could pose. And so in that blog post, if you
00:11:03
look at the the language that Alman uses side by side with the language that Musk
00:11:07
was using at the time, it mirrors all the things that Musk was saying >> identical. I mean, 10 years ago, Musk
00:11:13
was going on podcast saying, tweeting, whatever, that the greatest existential risk to humanity was AI.
00:11:19
>> Yeah. And so you know like his parenthetical there are other things that we that might actually be more
00:11:26
likely to happen like engineered viruses. It's because up until then Alman had been talking just about
00:11:32
engineered viruses. And so now that he needs to pivot to speak to an audience of one to Musk. He needs to kind of
00:11:40
resolve the contradiction between what he's now elevating as his new central
00:11:45
fear to be the same as Musk's new central fear with what he had previously been saying. So that's why he's like I
00:11:51
think this is now even though before I said this >> and are you saying that Sam Alman
00:11:57
manipulated Musk because Elon did end up donating a huge amount of money to um open AAI and co-founding it I believe
00:12:06
with Sam Alman. Elon Musk did end up co-ounding it with Altman. And certainly from Musk's perspective, he does feel
00:12:12
manipulated because he feels like Alman was engineering his language in a way that would make Musk trust him as a a
00:12:24
partner in this endeavor. And of course then Musk is leaves. Um and through some
00:12:31
of the documents that came out during the the lawsuit that Musk and Altman are engaged in now, it has become clear that
00:12:38
there was a degree to which Musk was actually muscled out a little bit. And so that's why he's left with this
00:12:47
very intense personal vendetta against Altman, saying that somehow Alman tricked him into being part of this. So
00:12:54
in in 2015, Sam Alman is writing these blog posts saying this is, you know, one of the greatest existential threats. At
00:12:59
the same time, in 2015, Musk is doing some very famous speeches at the time at MIT. He said that AI was the biggest
00:13:06
existential threat and compared developing AI to summoning the demon. And what you're saying here is you're
00:13:12
saying that Samman was just mirroring the language that Elon was using to get Elon involved in open open AAI. And
00:13:18
later it appears and again there's a legal case taking place now that Sam might have muscled Elon out in some
00:13:24
capacity. >> Yeah. So we know from the lawsuit and the documents that have come out in the
00:13:28
lawsuit that Ilia Sgver who is the chief scientist of OpenAI at the time and Greg
00:13:35
Brockman chief technology officer at the time when they were deciding whether or
00:13:40
not to maintain OpenAI as a nonprofit because it was originally founded as a nonprofit. They decided okay we need to
00:13:46
create a for-profit entity but the question was who should be the CEO of this for-profit entity. Should it be
00:13:50
Musk or should it be Alman? because it's they were the two co-chairmen of the
00:13:54
nonprofit. And in the emails, it became clear that Ilia and Greg first chose Musk to be the CEO.
00:14:05
But through my reporting, I discovered that Altman then appealed personally to Greg Brockman, who was a friend of his
00:14:13
that they had known, they had known each other for many years through the Silicon
00:14:16
Valley scene, and said, "Don't you think that it would be a little bit dangerous
00:14:22
to have Musk be the CEO of this company, this new for-profit entity, because, you
00:14:28
know, he's a famous guy. He has a lot of pressures in the world. He could be
00:14:33
threatened. He could act erratically. He could be unpredictable. And do we really
00:14:38
want a technology that could be super powerful in the future to end up in the hands of this man? And that convinced
00:14:46
Greg and Greg then convinced Ilia, you know, I think there's a point here. Do
00:14:52
we really want to give this much power to Musk? And that is why Musk then leaves because then they the two switch
00:15:00
their allegiances. They say, "Actually, we want Altman to be the CEO." And then
00:15:04
Musk is like, "If I'm not CEO, I'm out." >> So, it sounds like Sam again managed to
00:15:08
persuade someone to do something. >> Mhm. >> I guess this begs the question, what do
00:15:14
you think of Sam Orman? >> I think he's a very controversial figure. >> You did an interesting pause. It's a
00:15:22
pause where someone tries to select their words. Well, this is this is this is what's so interesting
00:15:30
about those interviews is people are extremely polarized on Alman there. No one has in between feelings about him.
00:15:39
Either they think he's the greatest tech leader of this generation akin to the
00:15:43
Steve Jobs of the modern era or they think that he's really manipulative and
00:15:49
an abuser and a liar. And what I realized because I interviewed so many people is it really comes down to what
00:15:58
that person's vision of the future is and what their goals are. So if you align with Altman's vision of the
00:16:06
future, you're going to think he's the greatest asset ever to have on your side
00:16:10
because this man is really persuasive. He's incredible at telling stories. He's
00:16:14
incredible at mobilizing capital, at recruiting talent, at getting all the inputs that you need to then make that
00:16:20
future happen. But if you don't agree with his vision of the future, then you
00:16:26
begin to feel like you're being manipulated by him to support his vision even if you fundamentally don't agree
00:16:34
with it. And this is the story especially of Daria Amade, CEO of Enthropic, who was originally an
00:16:41
executive at OpenAI. So for people that don't know, Dario now runs anthropic
00:16:45
which is the maker of Claude. A lot of people probably are more familiar with Claude.
00:16:49
>> Yeah. And it's one of the biggest competitors to OpenAI. And Amade at the time when he was an ex
00:16:57
executive at OpenAI, he thought that Alman was on the same page with him and then over time began
00:17:06
to feel that Altman was actually on exactly the opposite page of him and felt that Altman had used Amade's
00:17:15
intelligence, capabilities, skills to build things and bring about a vision of the future that he actually
00:17:23
fundamentally didn't agree with. And so that's why people end up with this bad
00:17:28
taste in their mouths. And so, you know, I've been covering the tech industry for
00:17:33
over eight years and covered many companies. I've covered Meta, Google, Microsoft in addition to Open AI. and
00:17:39
OpenAI and Altman is it's the only figure that I've seen this degree of polarization with where people cannot
00:17:48
decide whether he's the greatest or the worst. >> You mentioned Dario there and I found it
00:17:56
really what I found really interesting is to look at how people's quotes evolve
00:17:59
over time with their incentives. So I was looking at all of the all of the things they've said on the record on
00:18:04
podcasts in their blog post to see how it's evolved over time and Dario who was
00:18:08
the former VP of research open AAI and has now moved on to enthropic who are taking a slightly different approach to
00:18:14
developing AI said back in 2017 while he was still at open AI that this is a quote I think at the extreme end is the
00:18:22
Nick Bostonramm style of fear that an AGI could destroy humanity. I can't see
00:18:27
any reason in principle why that couldn't happen. My chance that something goes really quite
00:18:33
catastrophically wrong on the scale of human civilization might be somewhere between 10% and 25%.
00:18:40
And also you mentioned Ilia who was a co-founder of OpenAI and then left. I guess the first question I'd ask is why
00:18:47
did I leave? >> It's a great question. So he was instrumental in trying to get
00:18:54
Sam Alman fired and he's another one of the people who over time began to feel
00:19:00
like he was being manipulated by Alman towards contributing something that he didn't believe in. And for
00:19:07
>> you know >> because I interviewed a lot of people Ilia in particular had
00:19:12
two pillars that he cared about deeply. One is making sure we get to so-called AGI and the other is making sure that we
00:19:22
get to it safely. And he felt that Altman was actively undermining both things. He felt that Alman was creating
00:19:31
a very chaotic environment within the company where he was pitting teams against each other where he was telling
00:19:37
different things to different people. >> Have you ever spoken to him? >> I have. So, so I interviewed him in 2019
00:19:43
for a profile that I did of OpenAI um for MIT Technology Review >> and back in 2019, he has a quote where
00:19:51
he says, "The future's going to be good for AIs regardless. It would be nice if
00:19:54
it was also good for humans as well. It's not that it's going to actively
00:19:58
hate humans or want to harm them, but it's just going to be so powerful. And I
00:20:01
think a good analogy would be the way that humans treat animals. It's not that
00:20:04
we hate animals. I think humans love animals, and I have a lot of affection for them. But when the time comes to
00:20:10
build a highway between two cities, we are not asking the animals for permission. We just do it because it's
00:20:15
important to us. And I think by default, that's the kind of relationship that's
00:20:19
going to be between us and AI, which are truly autonomous and operating on their
00:20:25
own behalf. And that was in 2019, the year that you interviewed him. >> One of the things that I I feel like we
00:20:30
should take a step back to examine is going back to this idea of what even is artificial intelligence and what do we
00:20:36
mean by intelligence? And a huge part of the views of the different people and the quotes that you're reading derives
00:20:43
from a specific belief that they each have in this question of what is intelligence, what constitutes
00:20:50
intelligence. For Ilia, he has throughout his research career felt that ultimately our brains
00:21:00
are giant statistical models. This is not something that you know we actually know but this is his own hypothesis also
00:21:08
the hypothesis of his mentor Jeffrey Hinton who also was on this podcast. This is why they have such a strong
00:21:15
conviction in the idea of building AI systems that are statistical models and that this particular approach is going
00:21:22
to lead to intelligent systems as we are intelligent. It's a hypothesis that they
00:21:28
have. It's not one that has been proven by science. And some people vehemently
00:21:33
disagree with them on this particular thing. But if you step into their shoes and take on that hypothesis and assume
00:21:41
that it's true, that our brains are in fact statistical engines and that these
00:21:48
systems that they're building are also statistical engines, that they're making
00:21:51
bigger and bigger and bigger until they become the size of the human brain. That's why they say that making this
00:21:59
comparison where the system will become equal to human intelligence and then maybe exceed human intelligence is
00:22:06
relevant in their framework. And um Ilia gave a talk at one point at this really
00:22:12
prominent AI research conference that happens every year called neural information processing systems. It's a
00:22:18
mouthful, but he gave this keynote where he shows this chart of the size of brains and the intelligence of a
00:22:27
species. And it's roughly linear. The bigger the size of the brain, the more
00:22:32
intelligent the species. And so for him, he thinks he's building a digital brain
00:22:39
because he he thinks brains are just statistical engines. So from that logic it's like okay if we then build a bigger
00:22:47
statistical engine than the human brain then based on this chart it will be more
00:22:53
intelligent and then we will be subjected to the same treatment that we've subjected animals but it's really
00:23:00
important to understand that these are scientific hypotheses of specific individuals within the AI research
00:23:05
community and there's a lot a lot of debate about whether this is in fact the
00:23:10
case and some of The biggest critics say it's very reductive to think of our
00:23:15
brains as simply just statistical engines. >> Why why does it matter to know the
00:23:21
mechanism? Is it not just important to know the outcome which is that it's going to be
00:23:27
able to do make a video for me or agents are going to be able to do the work that
00:23:31
I do. Does it does it really really matter for us to know the mechanism behind it?
00:23:36
>> Yes and no. So it matters because these companies they are driving their future actions
00:23:44
based on this hypothesis. So they have decided we think that this hypothesis is true like we should just
00:23:54
continue building larger and larger statistical models in the pursuit of artificial general intelligence. And
00:24:00
that's then having global consequences like in order to continue doing that they're hoovering up more and more data.
00:24:07
They're building more and more data centers. They are having uh they're, you
00:24:11
know, exploiting more and more labor in order to continue on this path. Here's a
00:24:16
question that I think is important to ask is why are we trying to build AI systems that are duplicative of humans?
00:24:23
We're kind of having this conversation right now where we've just taken the
00:24:27
premise of this industry as a good thing. Like they said that we should be building AGI, so we say that we should
00:24:34
be building AGI. I would like to ask like why are we doing that? Why is it that we are building a technology that
00:24:42
is ultimately designed to replace and automate people away? That is not the enterprise of technology. Like we should
00:24:51
be building technology and the purpose of technology throughout history has been to improve human flourishing, not
00:24:58
to replace people. And so this is like a a critical part of my critique of these
00:25:05
companies and and these scientists that have just adopted this goal and have relentlessly pursued it and have had
00:25:12
enormous capital and enormous resources to pursue it. Is is this the right goal?
00:25:16
What like why are we doing this? Why can't we just build AI systems that do things like accelerate drug discovery
00:25:24
and improve people's health care outcomes, which are systems that have nothing to do with the statistical
00:25:30
engines that they're trying to build to duplicate the human brain? >> So why are they doing it? I mean, you've
00:25:35
interviewed all these people. I think it's what, 300 people in total, 80 or 90
00:25:39
of them from OpenAI, the maker of CHACHBC. Why do you think they're doing it?
00:25:44
I think it's because they're driven by an imperial agenda. And that is why I
00:25:48
call these companies empires of AI. >> What do you mean by an imperial agenda?
00:25:52
What does that term mean? >> Empire is the only metaphor that I've ever found to fully encapsulate all of
00:25:59
the dimensions of what these companies do and the scale that they operate and what motivates them to do what they do.
00:26:07
And there are many parallels that you see between what I call the empires of AI and the empires of old. They lay
00:26:15
claim to resources that are not their own in the pursuit of training these models. That's the data of individuals,
00:26:20
the intellectual property of artists, writers, and creators. Their land grabbing in order to build these
00:26:25
supercomputer facilities for training the next generation models. Second, they exploit an extraordinary amount of
00:26:30
labor. They contract hundreds of thousands of workers all around the world including in the US to ultimately
00:26:38
make these technologies. We can talk about that more. And they also design their tools to be labor automating so
00:26:46
that when the technologies are deployed, it also affects labor rights because it
00:26:52
erodess away labor rights. And this is a political choice that they have. Third,
00:26:57
they monopolize knowledge production. And so they project this idea that they're the only ones that really
00:27:01
understand how the technology works. And so if the public doesn't like it, it's
00:27:05
because they don't actually know enough about this technology. They do this to
00:27:08
the public. They do this to policy makers. And they've also captured the majority of the scientists that are
00:27:14
working on understanding the limitations and capabilities of AI. >> You think they're gaslighting the public
00:27:20
in a way? >> They are. Yeah. So if most of the climate scientists in the world were
00:27:25
bankrolled by fossil fuel companies, do you think we would get an accurate picture of the climate crisis?
00:27:31
>> No. >> And in the same way they employ and bankroll the AI industry employs and
00:27:37
bankrolls most of the AI researchers in the world. So they set the agenda on AI research in soft ways simply by
00:27:44
funneling money to their priorities so that only certain types of AI research are produced. But they also will censor
00:27:52
researchers when they do not like what the researcher has found. And so I talk about the case of Dr. Timmy Gabru in my
00:28:00
book who was the ethical AI team co-lead at Google when she was literally hired to critique the types of AI systems that
00:28:10
Google was building. She then co-wrote a critical research paper that was showing
00:28:15
how large language models specifically were leading to certain types of harmful outcomes. And in an attempt to try and
00:28:24
stop this research from being published, Google ended up firing Gabru and then fired her other co-lead Margaret
00:28:31
Mitchell. And so they control and quash the research that is inconvenient to the
00:28:41
empire's agenda. >> Did you have an example where this is happening to journalists as well that
00:28:46
are asking questions of their team members? I think I was watching a video of yours where there was a young man
00:28:52
that was saying he had someone show up at his door, knocked on his door and asked for information, emails, text
00:28:58
messages, and this person was from one of the big AI companies. >> This was opening. I started subpoenaing
00:29:03
some of its critics. Yeah. Um as a as part of a what's what appears to be a campaign of
00:29:11
intimidation, but also what appeared to be a campaign of fishing for more information to figure out to map out the
00:29:18
network of critics further. But this was a man who runs a small watchdog nonprofit and they had been doing a lot
00:29:26
of work during that time to try and ask questions about OpenAI's attempt to convert from a nonprofit to a
00:29:34
for-profit. Ultimately, OpenAI was successful in that conversion. But during the period where it was sort of
00:29:40
existential for open AI to complete this conversion, there were a lot of civil society groups and watchdog groups like
00:29:47
MIDAS who were trying to prevent the process from happening in the dead of night. They were trying to get more
00:29:56
transparency. They were trying to have more public debate about this because it's unprecedented. And it was then that
00:30:03
um there was a knock on his door and he was served papers. >> What did the papers say?
00:30:09
>> The papers asked him to reproduce every single piece of communication that he
00:30:15
had had that might have involved Musk. So this was like this strange paranoia that OpenAI had that Musk was somehow
00:30:21
funding these people to block the conversion. None of them were actually funded by Musk. So in this particular
00:30:27
case their request he simply was just answered you know I I don't have any documents because this doesn't exist.
00:30:33
>> So going back to this point of empires you were saying that one of the factors
00:30:36
of an empire is a land grab and then the next one was >> was labor exploitation
00:30:42
>> labor exploitation. The third one, controlling knowledge production. >> And one of the other ones that's really
00:30:50
important to understand about the AI empires in particular is empires always have this narrative that they they say
00:30:59
to the public like we're the good empire and we need to be an empire in the first
00:31:04
place because there are also bad empires in the world. And if you allow us to take all the resources and use all of
00:31:12
the labor, then we promise we will bring you progress and modernity for everyone.
00:31:18
>> We will bring you to this utopic state akin to an AI heaven. But if the evil
00:31:24
empire does it first, we will descend into a hell. >> And the evil empire being in this case,
00:31:30
>> in this case, most often it's China. But actually in the early days, Open AI
00:31:35
evoked Google as the evil empire. >> So all of their decisions were about we
00:31:40
need to do it first because otherwise Google, this evil corporation that's driven by profit, us as a benevolent
00:31:47
nonprofit. Like this is a this is a critical contest of who wins. >> Do you think the people building these
00:31:56
AI companies believe that the outcome is going to be all good now? Do you think they think that it's going to be it's
00:32:04
going to serve everyone? It's going to be the age of abundance. Everything's
00:32:07
going to go up well. What do you think they believe? What do you think Sam believes?
00:32:10
>> So, so this is so funny is such a core part of the mythology that they create
00:32:15
around the AI industry includes the belief that it could go very badly. It goes hand in hand. like they need that
00:32:25
part of the myth in order to then say and that's why we need to be in control
00:32:30
of the technology because that's the only way that it's going to go really
00:32:33
really well and Alman has said publicly you know the worst case lights out for everyone but best case we cure cancer we
00:32:42
solve climate change and there's abundance and Dario Amade same kind of rhetoric was like worst case
00:32:49
catastrophic or existential harm for humanity best case mass human flourishing. So this is like two sides
00:32:57
of the same coin. Like they have to use both of these narratives in order to continue justifying an extremely
00:33:06
anti-democratic approach to AI development where there should not be broad participation in developing this
00:33:12
technology. They must be the ones controlling it at every step of the way. >> Sam Orman did a tweet saying, "There are
00:33:18
some books coming out about open AI and me. We only participated in two of them.
00:33:23
one by Kesh Hegy >> Keegy >> Khaggy focused on me and one by Ashley Vance on OpenAI.
00:33:31
Um he went on to say no book will get everything right especially when some people are so intent on twisting things
00:33:38
but these two authors are trying to you quote retweeted that tweet from Sam Alman and you said the unnamed book
00:33:47
empire of AI is mine. Do you believe that tweet from Sam Alman was in reference to your book?
00:33:55
>> 100%. Because there's only three books coming out about him >> and he had caught wind that your book
00:33:59
was coming out and >> he knew my book was coming out because I had contacted OpenAI from the very
00:34:04
beginning of my process and said I'm working on a book now. Will you participate in it? And actually
00:34:09
initially they said yes even though so my history with OpenAI I profiled the company for MIT technology review. I
00:34:16
embedded within the office for 3 days in 2019. my profile comes out in 2020, the
00:34:22
leadership are very unhappy. And in my book, I actually quote an email that I received that Sam Alman sent to the
00:34:30
company about my profile saying, "Yeah, this is not great." And from then on, the company's stance
00:34:40
to me was, "We are not going to participate in anything that you do. we are not going
00:34:48
to respond to anything any of the questions that you receive. And this was, you know, this was things that they
00:34:54
explicitly articulated. It wasn't like me inferring. Um, so I I had a a colleague at MIT Technology Review that
00:35:01
also covered AI. And at one point opening, I sent him this press release being like, "We would love for you to
00:35:06
cover this story." And he was like, "I'm really busy. Will you send it to Karen?"
00:35:10
And they were like, "Oh, no. We have a history. You understand?" And so, so for
00:35:17
three years they they refused to talk to me, but then I ended up at the Wall Street Journal where if they felt a a
00:35:25
bit compelled because it was the journal to reopen the lines of communication. And so I I I started having, you know,
00:35:33
more dialogue with them. Every time I wrote a piece, I would always send them here's my request for comment. I would
00:35:39
always ask them like, will you sit for interviews? And we did get to a more productive relationship. And then I
00:35:45
embarked on the book. So I I left the journal to focus on the book full-time. And I told them right away, I'm working
00:35:52
on this book. I want to continue this productive conversation where I make sure I reflect OpenAI's perspective in
00:36:01
the book. And so they were like, we can arrange interviews for you. You can come
00:36:05
back to the office. We'll set up some conversations. And then as we were going back and forth
00:36:13
on this, the board fired Sam Alman. And that's when things started going kind of south because the company
00:36:21
started becoming very sensitive to scrutiny. And so then they started pushing kicking the can down the road,
00:36:27
down the road, down the road. And I kept saying, "Hey, when are we rescheduling
00:36:30
this? What's going on?" And then I get an email saying, "We are not going to
00:36:34
participate at all. You are not coming to the office. You're not doing interviews." and I had actually already
00:36:39
booked my tickets. So, I was already going to fly to San Francisco to have the the interviews. And so, then I told
00:36:48
them I was like, "That's fine. I will still engage in the process where I'll
00:36:53
give you extensive requests for comment. I'll ask through my reporting, I'll keep
00:36:57
you updated on all the things that I'm finding so that you can choose to still
00:37:01
comment." I gave them 40 pages of requests for comment. and I gave them over a month to respond to all of that.
00:37:09
So, this was when the tweet came out was we were doing all this back and forth trying to
00:37:15
and that's when Alman tweeted this. >> H >> and they never responded to a single one
00:37:23
of the one of the 40 pages. >> Sam Alman does a lot of interviews. >> Yeah.
00:37:28
>> You know, he's doing a lot of interviews all the time. He's done every podcast.
00:37:31
I've seen him on everything from Tucker Carlson to I think he's done Theo, Joe
00:37:35
Rogan, um podcasts all over the world. >> I wonder why he won't do mine.
00:37:45
>> Well, maybe. >> I don't know why. I I I don't know. I think I'm fair with everyone. I just ask
00:37:49
I just ask questions I genuinely care about. I don't come in with huge preconceptions or at least meet people
00:37:54
for the first time. But I've heard through the grape vine um that he doesn't want to do mine. I
00:38:00
mean, going back to what you were saying earlier that with this the way that OpenAI and these
00:38:06
companies control research, you asked, do they also do this with journalists? I mean, yes, the answer is yes. And
00:38:14
apparently they they also do it with anyone who has, you know, a broad mass communications platform.
00:38:20
>> It's not just about the conversation that you're going to have with them.
00:38:24
It's about who you also choose to platform. And there's this huge problem in
00:38:30
technology journalism where companies know that a really big carrot that they can give to technology journalists is
00:38:38
access. >> Yeah. Yeah. Yeah. >> And they will withhold that access at
00:38:44
the drop of a hat if they catch wind that you're speaking to someone that they didn't want you to speak to.
00:38:49
>> This is so true. And I don't think the average person really truly understands
00:38:53
this. >> Yeah. So, this kind of sounds like theory as you say it, but I'm not going
00:38:57
to name names here because I don't think it's important, but there is a particular person in AI who um whose
00:39:05
team have basically dangled the carrot of them coming here for like 18 months. And I'm like, you don't you don't have
00:39:11
to dangle the carrot. I'm going to speak to whoever I want to regardless of the
00:39:14
carrot or not. And when this person comes, if they want to come, I'll I'll
00:39:17
give them a fair shot. I'll ask them all genuinely curious questions about what
00:39:21
they're doing, their incentives. I won't gotcha them. I don't have a history of
00:39:24
ever gotchering anybody. Even if I dis like even if I have a different of opinion, I'll ask the question.
00:39:29
>> Yeah. >> But they dangle carrots and they say, "Well, if you know he he's thinking
00:39:33
about it, let's think about a date." And what what the strategy is, and I don't
00:39:36
think they they think those people don't understand, is if we just dangle it for
00:39:39
long enough, then they will um perform in the way that we want them to do and they'll be
00:39:46
>> they'll be pleasant about us. They won't be critical. They won't give a give a
00:39:52
critics. >> Our critics. >> And I think a lot of their game is just dangle the carrot forever.
00:39:57
>> Yes. Yeah. >> That's like the optimal outcome is if we just dangle it. If we just tell them,
00:40:00
yeah, look, we're just trying looking at the schedule. >> It just doesn't work. I think in the
00:40:04
modern world, you just have to go there and give your opinion and allow the clash of ideas in the public forum, let
00:40:08
the viewers un decide for themselves. >> Yeah. >> What they think. >> Yeah.
00:40:12
>> Um, but this is a Yeah. This is such a huge part of their machinery is the way
00:40:18
that they use these tactics to massage the public image of these companies and make sure that information that they
00:40:24
don't want out and even opinions that they don't want out there go out there.
00:40:28
>> Mhm. >> And so this is this is you know I feel very lucky now that opening I shut the
00:40:36
door early on me >> at the time I didn't feel lucky. I felt like I had screwed myself over. I was
00:40:43
nicer access to a journalist, right? Like you're supposed to report the truth and you're
00:40:51
always supposed to report in the interest of the public. Like that is the point of journalism. And in that moment
00:40:58
it I I was like relatively junior in my career. I was like, did I misunderstand what journalism about is is about? Like
00:41:06
>> should I have actually been playing the access game? >> Mhm. >> But it was too late. I had the door shut
00:41:11
to me and so I had to build my career understanding that the door the front door was never going to be open.
00:41:18
>> Yeah. >> And that actually really strengthened my own ability to just tell it like it is
00:41:26
like objective. Yeah. And just report what I see are the facts being presented to me irrespective of whether the
00:41:33
company likes it or not. And most often the company really does not like it but >> I can continue to do the work. They
00:41:40
don't need to open the front door for me. I was still able to do more than 300
00:41:44
interviews. >> So Sam Alman gets kicked off the OpenAI executive team. Did you find out why that happened?
00:41:57
>> Yeah, there's a scene by scene recounting >> from who? I can't remember the exact
00:42:04
number of sources, so I don't want to misquote myself, but it was around six
00:42:08
or seven people that were directly involved or had spoken to people directly involved in the decision-making
00:42:13
process. So, Ilia Satskever is seeing these serious concerns about the way that Altman's behavior is
00:42:27
leading to bad research outcomes and poor decision-m at the company. He then approaches a board member, Helen
00:42:38
Toner. Ilia, for anyone that doesn't know, is the the co-founder we mentioned
00:42:42
earlier. The co-founder of OpenAI we mentioned earlier. >> Yes. And he kind of does a bit of a
00:42:50
sounding board thing to Helen just because Ilia is freaking out. He's like he's been like sitting on this these
00:42:56
these concerns for a while and he's like if I tell this to someone, this could
00:43:01
also be really bad for me if Alman finds out. And so he asks for a meeting with Toner
00:43:12
and in that first meeting he's like re like he barely says a thing. He's
00:43:18
just like dancing around trying to figure out hey is this someone that I can maybe trust to divulge more
00:43:25
information. >> And Toner's role and responsibilities at OpenAI were >> she was a board member.
00:43:29
>> Just a board member. >> Yeah. And and specifically an independent board member. So opening eye
00:43:34
when it was a nonprofit the board was split between people who had a stake financial stake in the company and then
00:43:40
people who were fully independent and this was meant to be a structure that would balance the decision-m to be in
00:43:47
the benefit of the public interest rather than to be in the benefit of the for-profit entity that opening I then
00:43:51
created >> and Ilia as a non-independent board member was approaching toner as an independent
00:44:01
board member her to try and see whether or not she was potentially seeing or hearing the same things that he was
00:44:10
about the effect that Alman was having on the company. This then sets off a series of conversations first between
00:44:17
Ilia and Helen and then between Amir Moratti and some of the board members. Samir Moratti was at that point the
00:44:25
chief technology officer of OpenAI where these two senior leaders essentially through these conversations and through
00:44:31
documentation that they're pulling together like email, Slack messages and so forth, they convey to the independent
00:44:37
board members, three independent board members, we are very concerned about Altman's leadership like he is creating
00:44:47
too much instability at the company and it is like he is the root of the problem. It's not they they they were
00:44:57
trying to say to these independent board members like the problem will not be fixed unless Alman is removed because of
00:45:04
the way that he's pitting teams against each other and creating this environment
00:45:09
where people are unable to trust each other anymore and they're competing rather than collaborating on what's
00:45:14
supposed to be this really really important technology. When you say instability,
00:45:20
that's a that's quite a vague term. That could mean lots of things. Like
00:45:23
instability could mean pushing people hard to work harder, >> right? >> What do you mean by instability in spec
00:45:28
as specific terms as you can possibly say them? >> When chat GBT came out in the world,
00:45:34
OpenAI was wholly unprepared. >> They didn't think that they were launching a gangbusters product.
00:45:41
>> Yeah. They thought they were releasing a research preview that would help them
00:45:46
get the data flywheel going, collect a bunch of data from users that would then inform what they thought would be the
00:45:53
gang busters product, which was a chatbot using GPT4 and chat GBT was using GPT 3.5.
00:46:01
And because of that, there were servers crashing all the time because they they weren't they had to scale their their
00:46:10
infrastructure, you know, faster than any company in history. And there were um there were all of these outages. They
00:46:17
were trying to also hire faster than any company in history to try and have more
00:46:20
personnel there. And they were then sometimes hiring people that they were like, "Actually, we made a mistake. We
00:46:26
shouldn't have hired you." So they were firing people left and right. and people
00:46:29
were just disappearing off of Slack and that's how their colleagues would learn
00:46:33
that they were no longer at the company. And so it was yes like many fast growing
00:46:39
companies a very chaotic environment and a particularly chaotic environment because it was extra fast like they had
00:46:48
to accelerate more than any other startup. And on top of that mirror Morati and Ilasgiver felt that Alman was making it
00:46:58
worse like he was not actually effectively ameliorating the circumstances of the chaos. He was
00:47:05
actually sewing more chaos, getting these teams to be more divided. And this is where it's important to
00:47:13
understand that the executives and the independent board members, they're all
00:47:19
operating under this idea that they're building AGI and that AGI could either
00:47:24
be devastating or utopic to humanity. And so it's not yes it's like any other
00:47:32
company and no it's not like any other company. You cannot have like in their
00:47:37
view you cannot have this degree of chaos as the pressure cooker for creating a technology that they in their
00:47:44
conception could make or break the world. And so that is basically what the independent board members also begin to
00:47:53
reflect on. They have these conversations amongst themselves where they're like,
00:47:58
"Well, based on what we're hearing about Altman's behavior, like if this was an
00:48:02
Instacart, would that warrant firing him?" And they concluded, "Maybe not,
00:48:08
but this is not Instacart." And that's why they were like, "Well, crap. Maybe this is actually this does
00:48:15
rise to the to the bar where we should consider replacing him because we are ultimately building a technology that we
00:48:24
think could have transformative impacts either in the positive or negative direction. And so that is what happens.
00:48:31
It's like these two executives and then the independent board members also they
00:48:35
were hearing other feedback as well from their connections within the company with other people in the industry. At
00:48:40
one point, Adam D'Angelo, who is one of the independent board members and the
00:48:44
CEO of Kora, uh, which is, you know, start a tech startup in the valley, he is at a party in San Francisco, and he
00:48:52
starts to hear some of these rumors that there's something weird about the way
00:48:58
that OpenAI has structured its OpenAI startup fund, which was this fund that they the company had created to start
00:49:06
investing in other startups. >> Mhm. and he realizes they'd never really seen
00:49:13
documentation about how the startup fund had been set up from Alman. And finally
00:49:17
they get the documents and it turns out that OpenAI startup fund is not OpenAI's
00:49:21
startup fund. It's Altman's startup fund. And this was something like one of
00:49:28
several experiences that the independent board members were also having where they're like there's something not right
00:49:33
about the fact that there continuously are inconsistencies inconsistencies between the way that Altman is
00:49:40
portraying what is being done versus what is actually being done. And so when these
00:49:47
two executives approach the board or the independent board members, then they're
00:49:51
like, "Okay, this lines up with also the experiences that we've been having."
00:49:58
And at that point, they then have this series of very intense discussions where they're meeting almost every day talking
00:50:06
about should we actually really consider removing Altman? And in the end they conclude, yes, we
00:50:16
should. And if we're going to do it, we need to do it quickly. Because they were
00:50:20
very concerned that the moment that Alman found out, his persuasive abilities would make it impossible to
00:50:26
do. And so they end up firing Altman without telling anyone. You know, they don't talk to any stakeholders to get
00:50:36
them on the same page. Microsoft gets a call right before they execute the action saying, "We're going to fire
00:50:42
Altman." >> And Microsoft, for anyone that doesn't know, are a lead investor in OpenAI at
00:50:45
the time. >> Yes. One of the only investors in OpenAI at the time. And that is what then
00:50:54
devolves the whole thing because every single person that is affected by this decision is now extremely angry that
00:51:02
they were not involved. And that is what then creates this campaign to bring Altman back. And then Alman is
00:51:10
reinstalled as CEO days later. >> This company that I've just invested in,
00:51:15
it's grown like crazy. I want to be the one to tell you about it because I think
00:51:17
it's going to create such a huge productivity advantage for you. Whisper Flow is an app that you can get on your
00:51:22
computer and on your phone on all your devices and it allows you to speak to your technology. So, instead of me
00:51:26
writing out an email, I click one button on my phone and I can just speak the email into existence and it uses AI to
00:51:33
clean up what I was saying. And then when I'm done, I just hit this one button here and the whole email is
00:51:37
written for me. And it's saving me so much time in a day because Whisper learns how I write. So on WhatsApp, it
00:51:44
knows how I am a little bit more casual. On email, a little bit more professional. And also, there's this
00:51:48
really interesting thing they've just done. I can create little phrases to automatically do the work for me. I can
00:51:53
just say Jack's LinkedIn and it copies Jack's LinkedIn profile for me because
00:51:57
it knows who Jack is in my life. This is saving me a huge amount of time. This company is growing like absolute crazy.
00:52:02
And this is why I invested in the business and why they're now a sponsor of this show. And Whisper Flow is
00:52:06
frankly becoming the worstkept secret in business, productivity, and entrepreneurship. Check it out now at
00:52:11
whisperflow spelled w i s p r l o w.ai/steven. It will be a game changer for you.
00:52:20
There's a phase a lot of companies hit where they're no longer doing the most
00:52:23
important thing, which is selling. And they get really bogged down with admin. And it's often something that creeps up
00:52:28
slowly and you don't really notice until it's happened. Slowly momentum starts to
00:52:32
leak out. This happened to us and our sponsor Pipe Drive was a fix I came across 10 years ago. And ever since my
00:52:38
teams across my different companies have continued to use it. Pipe Drive is a simple but powerful sales CRM that gives
00:52:43
you the visibility on any deals in your pipeline. It also automates a lot of the
00:52:47
tedious, repetitive, and time-conuming parts of the sales process, which in turn saves you so many hours every
00:52:52
single month, which means you can get back to selling. Making that early decision to switch to Pipe Drive was a
00:52:57
real gamecher, and it's kept the right things front of mind. My favorite feature is Pipe Drive's ability to sync
00:53:03
your CRM with multiple email inboxes so your entire team can work together from one platform. And we aren't the only
00:53:09
ones benefiting. Over 100,000 companies use Pipe Drive to grow their business. So, if something I've said resonates,
00:53:15
head over to pipedive.com/ceeo where you can get a 30-day free trial. No credit card or payment required.
00:53:24
How does a CEO of a major company get fired by the board? Because board members, there's a quote in your book on
00:53:30
page 357 where you say about Ilia saying, "I don't think Sam is the guy
00:53:35
who should have the finger on the button for AGI." Now, I I asked myself this
00:53:39
question. You know, I work with lots of people here. We have 150 people that work in this business and
00:53:46
those people know me best. >> Yeah. >> They see me on camera. They see me off
00:53:50
camera. So if they said that we don't think Steven is the right person to host
00:53:54
the direc >> Yeah. >> It would take a lot for them to say that. >> Yeah.
00:53:58
>> They must have seen some [ __ ] off camera for them to go we don't think he's the
00:54:02
right person to be on camera. Yeah. >> Or for whatever reason. And in the case
00:54:05
of AI, which is much more consequential than a podcast that is, you know, filmed
00:54:09
in my old kitchen. Um it almost sends a chill down one's body to think that the
00:54:14
co-founder of a business has gone to the board and said this isn't the guy to
00:54:18
lead this consequ I mirror Marotti then also said I don't think Alman is the
00:54:23
right guy >> and then they both left later. >> So then Altman comes back and lo and
00:54:28
behold Ilia never comes back. So his concerns about the fact that Alman founding out would be bad for him
00:54:35
manifested. He ended up not coming back and Miriam Marotti then left shortly thereafter.
00:54:41
>> Quite a lot of these people leave, don't they? Open AAI >> they do. So if you consider
00:54:49
one of the origin stories of open AI is this dinner that happened at the Rosewood Hotel,
00:54:57
which is a very swanky hotel um right right in the heart of Silicon Valley that uh was one of Elon Musk's favorites
00:55:04
whenever he was coming up from LA to the Bay Area. And there was this dinner that
00:55:08
was there where Altman was intending to recruit the OG team that would start OpenAI. So he's kind of telling everyone
00:55:18
you might have a chance to meet Musk because Musk is going to come to this dinner dinner. And he cold emails Ilia
00:55:24
and gets Ilia to then come because and Ilia specifically wants to come because he wants to meet Musk. And he also
00:55:31
emails all these other people including Greg Brockman, Dario Amade. These are all people that ended up working at Open
00:55:37
>> and they all almost all of them not not every one of them but almost all of them
00:55:42
end up working at OpenAI >> and leaving >> almost all of them end up leaving
00:55:49
specifically after they clash with Alman >> and Ilia he left and launched a company
00:55:55
called Safe Super Intelligence. >> Yeah. >> Which is I mean that's an indirect if
00:56:02
I've ever heard one. Do you know what I mean? Do you know what I mean? If someone like co-ounded this podcast with
00:56:10
me and then they left and started a podcast called Safe Podcasting, I I'd take that as a slight.
00:56:19
I' I'd have people knocking on their door and asking for their texts. One of
00:56:24
the things that is happening here is >> it is not a coincidence that every
00:56:32
single tech billionaire has their own AI company. >> Mhm. >> They want to create AI in their own
00:56:40
image and that's why they keep not getting along. And in fact, it's not just don't get along, they end up hating
00:56:49
each other after working together. >> Mhm. and then splinter off into their
00:56:54
own organizations. So after Musk leaves, he starts XAI. After Dario leaves, he starts Anthropic. After Ilia leaves, he
00:57:01
starts Safe Super Intelligence. After Meera leaves, she starts thinking machines lab. They want to have control
00:57:12
over their own vision of this technology. And the best way that they have derived from their experiences of trying
00:57:24
to put their vision into the arena is by creating a competitor and then competing
00:57:30
with OpenAI and with all the other companies out there. Do you think some of these AICOs realize that they are
00:57:35
quite literally summoning the demon as Elon said 10 years ago, but they don't
00:57:40
really care because being the person that summoned the demon is makes you consequential and powerful and
00:57:47
historical even if the outcome is potentially horrific. Even if there's like a 20% outcome of it being horrific.
00:57:53
I remember I think it was Dario, he's the one that said there's somewhere
00:57:57
between a 10% and 25% chance of things going catastrophically wrong on the scale of human civilization. 25% is a
00:58:07
one in4 chance. If you put bullets in a fourchamber revolver and said Steven, the upside is
00:58:17
you could become a multi-gazillionaire and be remembered forever. The downside is that there would be a bullet in your
00:58:22
head. There is no chance that I would take take that bet with a 25% potential chance of things going catastrophically
00:58:30
wrong. >> So, I have a very long answer to this because do they know if they're summoning the
00:58:37
demon? It really depends on what we define as summoning the demon. And in this particular case, to go back to what
00:58:44
we were saying before, there's a mythology that the AI industry uses where summoning the demon is an integral
00:58:52
part of convincing everyone that therefore they can be the only ones that are developing
00:59:00
this technology. >> I got it. So on one end, you got to say if we don't, China will and that's
00:59:05
terrible. >> Yeah. But if we let anyone else do it other than me, then we're [ __ ] as
00:59:09
well. >> Exactly. >> So that means that I have to do it and you have to give me money and support.
00:59:14
>> Exactly. So when they're saying these things, we should understand it as not as like a
00:59:21
genuine prediction based on what they're seeing because first of all, we don't
00:59:24
predict the future. We make it. We should understand this as an act of speech to persuade other people into
00:59:32
believing that they should seed more power, more resources to these individuals. And so, do they know that
00:59:39
they're summoning the demon? I mean, they are purposely trying to create this this
00:59:47
feeling within the public that they are because it is a crucial part of their power.
00:59:53
But do they if we were to define just do they realize that the things that they are doing are having already
01:00:01
really harmful impacts all around the world on vulnerable people, vulnerable communities, vulnerable countries.
01:00:09
That's where I'm like maybe yes, maybe no. and they don't really care because
01:00:15
in the frame of mind like I sometimes use the analogy that the AI world is like Dune.
01:00:22
>> Dune for anyone that doesn't know Dune >> science fiction epic written by Frank
01:00:25
Herbert and it's set in this intergalactic era where there are all these houses and they're fighting each
01:00:32
other for spice. So it's a call back to colonialism and empire and they all are
01:00:37
trying to control the spice. But one of the features of this story is that there
01:00:41
are these myths that are seated on the different planets about a a religious myth basically about the coming of the
01:00:48
Messiah that are used as ways to control the people. And Paul at Trades when he arrives at
01:00:56
the planet Iraqis uh with with the intention of um trying to then fight against the empire and um avenge his
01:01:06
father's death. He steps into a myth that has been seated on this planet that
01:01:11
says that one day there will be a Messiah that comes and saves the planet. So he steps into the role of the Messiah
01:01:18
and leans into this idea in order to better control the people and rally them behind him as a leader to help with this
01:01:27
quest. He knows that it's a myth in the beginning, but because he lives and breathes and embodies it, it kind of
01:01:37
starts to blur in his mind whether this is really a myth or whether he's really
01:01:40
the messiah. And this is what I think happens in the AI world. On one hand, there are all these executives that
01:01:51
actively engage in mythmaking because, you know, I have all these internal documents that I write about in the book
01:01:57
where they are very keenly aware of how to bring the public along with them by showing them dazzling demonstrations of
01:02:06
the technology by using crafting a mission that will sound really good uh and and and make people give more
01:02:15
leniency to their companies. So they know they're doing the mythmaking and also I think many of them lose
01:02:23
themselves in the myth because they have to live and breathe and embody it day in
01:02:28
and day out. And so when you know Daario says he thinks that 10 to 25% of the future could be catastrophic or or
01:02:37
whatever the probability is 10 to 25%. He is actively engaging in the mythmaking but also he's losing himself
01:02:44
in the myth. Like I think if you were to ask him, "Do you genuinely believe
01:02:47
that?" He would be like, "Yes, I genuinely believe that." Because there's
01:02:51
been a blurring of when he's saying something just to say something versus when he actually believes what is he's
01:03:01
required to believe in order to then continue doing the things that he's doing.
01:03:09
>> And this is the whole psychology of cognitive dissonance, right? where you
01:03:12
the brain struggles to hold two conflicting worldviews at the same time. So it's it's incentivized or it
01:03:18
endeavors to dismiss one. So if you you know if you wanted to be a healthy person but also a smoker. Um and I
01:03:24
pointed out that smoking is bad for you. The first words out of your mouth are going to be yes but
01:03:28
>> smoking helps me with stress. Yeah, but I only do it when I think I don't know I
01:03:34
kind of see that at the moment because these companies have to raise extortionate like huge amounts of money
01:03:39
to fund their AI research and they're building out all of these data centers.
01:03:44
>> So when they're out in the public, they're always fundraising. All of these
01:03:47
major companies are fundraising all the time at the moment. >> So you can't be fundraising and saying,
01:03:51
"I'm going to destroy your children's future potentially. There's 25% chance
01:03:54
that your children aren't going to have a great life." Which might be the truth. I mean that is
01:03:59
actually what they say Dario. This is what famously Dario Amade does. He's like
01:04:03
>> he does that but the others Sam's not doing that as much anymore. >> Yes. And it's because you know
01:04:10
it goes back to like each of them kind of distinguish themselves a little bit as as the brand that they need to
01:04:16
project. >> Do you think any of them are more have a stronger moral compass than others? cuz
01:04:22
I think Dario often gets the credit for having more of a, you know, more of a backbone and being more conscious of
01:04:28
implications. >> He does get a lot of credit for that. >> He's from Claude and Anthropic. For
01:04:34
anyone that doesn't know, >> I don't think it truly matters that question, the answer to that question,
01:04:43
because to me, >> even if you were to swap all the CEOs for someone that people would say is
01:04:49
better at running these companies, it doesn't fix the problem that I identify
01:04:54
in the book, which is that there is a system of power that has been constructed where these companies and
01:05:00
the people running these companies get to make decisions that affect billions of people's lives. lives around the
01:05:05
world and those billions of people do not get any say in how it goes. >> Those people, they can go to the polls,
01:05:13
right? So, if the public are sufficiently educated, they can go to the polls and pick a leader that says
01:05:18
they're going to legislate or pass laws or try and pass laws. >> Yes. But at the speed and pace at which these
01:05:26
companies operate and at the sheer scale and size, they're able to also spend
01:05:31
extraordinary amounts of money, hundreds of millions in this upcoming midterms to
01:05:35
try and kill every possible piece of legislation that gets in their way and craft legislation that would codify
01:05:40
their advantage. And so to me, I think sometimes as a society, we obsess a little bit with
01:05:50
are these leaders good or bad people? And to me the bigger question is is the governance structure that we've created
01:05:59
a sound one or that allows broad participation or an anti-democratic one that has consolidated this
01:06:05
decision-making power in the hands of the few because no person is perfect. It does I don't I don't care who is on at
01:06:12
the top of these companies. they're not going to have the ability to make decisions on behalf of so many people
01:06:18
around the world who live and talk and um and and have a culture and history that are fundamentally different from
01:06:26
them without things going wrong. And so that is why throughout history we've moved from empires to democracy.
01:06:36
It's because empire as a structure is inherently unound. it does not actually
01:06:42
maximize the chances of most people in the world being able to live dignified lives.
01:06:49
>> I'm going to try and take on their point of view. So, this is me playing devil's
01:06:52
advocate. Okay. But Karen, if the US don't continue to accelerate their research with AI, at some point, China's
01:07:02
model is going to become so smart and intelligent that we're basically going
01:07:07
to have to rent it off them and we're going to be, you know, they'll get the
01:07:09
scientific discoveries. They'll discover the new era of autonomous weapons and we
01:07:14
will be their backyard. And like logically that argument does appear to be pretty
01:07:21
true. >> No, it's not. >> If we scale up, if we just imagine any
01:07:25
rate of change with this intelligence, at some point we're going to come to a
01:07:29
weapon that could theoretically disable um all of the United States electricity,
01:07:34
their weapons systems. It would know exactly how to disable the United States from a cyber perspective because it
01:07:41
would be that smart. All you've got to imagine is any rate of improvement of
01:07:44
any period any sort of long period of time. So this is a theory that might be true and if it's true
01:07:52
>> I mean yeah any theory might be true >> but but if but but you know again going
01:07:57
to this point of like even if it's a small percentage it's worth paying attention to on the other side of the
01:08:00
foot. This is a theory that people talk about. It could be the case that the most intelligent civilization is going
01:08:09
to be the superior civilization. Logically, that's a pretty sound thing to say. No.
01:08:14
>> So, there's a lot of a lot of fundamentals in this argument that would
01:08:19
need to be true in order for this to be a viable argument. And let's knock them
01:08:23
down one by one. So the first one is that these systems are intelligent and that
01:08:31
just scaling them is going to bring us more intelligence. So far so true. >> No, it's actually not because first of
01:08:39
all again we don't actually know if these systems are like intelligence is not it's not like the right analogy
01:08:46
almost. It's sort of like it's like is a calculator a calculator can do math problems faster than a
01:08:53
human. Does that make it intelligent? >> It has a narrow intelligence because
01:08:57
they're solving a narrow problem which is like 1 plus 1 equals 2. But >> and these systems, they actually also
01:09:03
are quite narrowly intelligent in the sense that even though these companies say that they're everything machines
01:09:09
that can do anything for anyone, they actually can only do some things for some people. This is like the jagged
01:09:14
frontier of these AI models like some of the capabilities are quite good, other capabilities are not that good. You know
01:09:20
why that happens? is because the company can only focus on advancing certain types of capabilities. It can't
01:09:26
literally focus on advancing all types of capabilities. They have to actually set their mind to advancing a certain by
01:09:32
gathering the data that is needed for that capability by taking uh you know getting a bunch of human contractors to
01:09:39
annotate and train the model to do that exact thing. And so scaling these models is actually a
01:09:48
perpendicular question to are we actually getting more cyber capabilities specifically and
01:09:56
more military capabilities specifically. >> I would argue that most of the most of
01:10:00
the top people in AI believe that the intelligence is going to continue to scale for some time. a lot of them do
01:10:06
like Jeffrey Hinton does. >> And again, it's it's back to his hypothesis about how human intelligence
01:10:12
works and what the appropriate model of the brain is. His hypothesis throughout his career has been the brain is a
01:10:19
statistical engine. >> But that's his hypothesis and that is not universally agreed upon especially
01:10:25
among people that are not in the AI world. When you talk with neuroscientists and psychologists,
01:10:29
people who actually study human intelligence in the human brain, that is where you start to get a lot of debate
01:10:35
and disagreement about this particular view that Hinton has. And so this is kind of like one of the one of the
01:10:44
things is like AI is already being used in the military and has been used in the military for a
01:10:50
long time. But ex specifically accelerating large language models isn't just the only path for getting
01:11:01
military cap. like the companies would have to choose to specifically pick military capabilities to accelerate not
01:11:08
just like general intell it's like you know what I'm saying like they create
01:11:12
this myth that they are actually pushing the frontier of all of the capabilities
01:11:17
of the model but that's not what's actually happening internally and I have
01:11:20
I had hundreds of pages of documents on like how they were specifically training
01:11:24
models they pick what capabilities they want to advance and you know how they pick them it's based on which industries
01:11:31
countries would be able to pay them the most money for their services. So they pick finance, law, medicine, healthcare,
01:11:40
commerce. It's not actually intelligent like a like a a baby where you the the
01:11:47
more that you that the baby grows up, they start having this like general these general abilities.
01:11:52
>> I think I have jagged intelligence. I'll be honest. I wasn't going to say it, but
01:11:57
I think I know a little I know a little bit about uh No, I know a lot about a little bit.
01:12:02
>> Yeah, but if but you also have the capability to learn and acquire knowledge by yourself. And you also have
01:12:06
the ability to choose what you're going to learn and acquire by yourself. >> It's not easy and it takes a lot more
01:12:11
time than these models. It seems less compute, but >> and you can learn how to drive in one
01:12:16
place and then immediately know how to drive in another place. These models cannot do that. Every time a
01:12:21
self-driving car is shifted to another location, it has to completely retrain on that location. It's like all the
01:12:28
self-driving cars. I mean, we're sitting in Austin right now and there's all
01:12:30
these self-driving cars that are driving through Austin. But when one of them learns, they all
01:12:35
learn >> which is which >> well it's just because it's a it's an
01:12:40
operating system that is has an AI model as part of it and you're training the AI
01:12:45
model and then you deploy that AI model across all the self-driving >> a big advantage because if one optimist
01:12:51
robot learns one thing in one factory they all learn it and imagine that imagine if humans if we all learned what
01:12:57
all the other humans learned that would be that would give us such an unbelievable competitive advantage. I
01:13:02
mean one of the ways we did that is through communication. >> They could not because they could be
01:13:05
learning the wrong thing which has also happened again and again with these technologies is that all of them then
01:13:10
learn the wrong thing and they all have the same failure mode. I mean part of the resilience of human society is that
01:13:15
we do have different expertises and we also have different failure modes. >> I think sometimes we hold AI models to a
01:13:21
higher standard than we hold humans to. And in a weird because I I' I'd hear on
01:13:25
stage we're in we're in Austin at the moment and I'd hear people go ah but you
01:13:29
know them AI models they hallucinate sometimes. I'm like, "Have you met a
01:13:32
human?" Like, I I hallucinate all the time. I can barely spell or do math. >> So,
01:13:40
>> yes, but it's it's once again like using this analogy that was specifically
01:13:43
picked in the early days of the field as a way to market these technologies. like
01:13:48
we're repeatedly using the intelligence analogy and relating these machines to
01:13:52
human intelligence as a a way to try and gauge whether or not it is good or worthy or capable in society. I think
01:14:01
the output is the thing that really m is the most consequential which is like okay it might have a different brain and
01:14:06
a different system but does it arrive at the same capability like does it is it able to do surgery on someone's brain is
01:14:12
it able to drive a car like my car drives itself in in Los Angeles I don't touch the steering wheel and I can drive
01:14:17
for many many hours and in here in Austin I just saw the ones the other day where they've removed the steering wheel
01:14:22
and the pedals the new cyber cabs so I go it doesn't really matter if it's
01:14:25
using a different system if it's navigating through the world as a car it has a better safety record than human
01:14:30
beings Um then as far as I'm concerned, intelligence or not, it's like >> yes, you know,
01:14:36
>> but that was not the original argument that you made, which was like these
01:14:40
systems are just generally going to become more intelligent across different things based on the prediction. This is
01:14:46
a prediction that you're making, right? Like that and this is a prediction that
01:14:49
all the AI um >> Ilia's making, Dario's making, Elon's making, Zuckerberg's making, man's
01:14:54
making, Dennis is making. >> And do you know what the common feature of all of them is? They profit
01:14:59
enormously off of this myth. >> Elon has recently spearheaded the construction of Colossus, a massive
01:15:05
supercomputer in Memphis housing a 100,000 GPU specifically to scale up their API models faster than their
01:15:12
competitors. It appears that they've all converged around this idea that you can
01:15:16
brute force your way to greater, more generalized intelligence. They've converged around the idea that you can
01:15:22
brute force your way into models that they can sell to people for automating certain tasks that are that are
01:15:29
financially lucrative. >> And I heard Elon say that if you're a surgeon, there's just no point. He was
01:15:33
like, don't train to be a surgeon. He says in a couple of years time, Optimus
01:15:37
and AI generally are going to be better than any surgeon that's ever lived. >> Yeah. You know,
01:15:41
>> do you think these things are true? Well, you know, I I'm pretty sure it was
01:15:44
Hinton that famously slash infamously said there would be no need for radiologists anymore.
01:15:50
>> There would be no need for radiologists anymore in he set a deadline that we've
01:15:54
already passed. I don't remember how many years. Radiology is doing great as a
01:16:00
profession. >> Do you think it will be in 5 years? >> Okay. So, this this once again goes back
01:16:05
to this question of like why do we build technology and why should we specifically be building AI? Okay. And
01:16:11
for me like the whole project of technology development advancement is not to advance technology for
01:16:17
technologies sake. >> It's to help people. And there have been lots of research
01:16:23
that has shown that actually the best outcomes for people in a healthcare setting is for the radiologist to have
01:16:31
the AI model in their hands and for the for the human expert to use the AI model as a tool as an input into
01:16:43
their judgment. And it is that combination that leads to the most accurate and early diagnoses of certain
01:16:51
types of cancer that then help improve the prognosis of the patient. >> Do you believe that in the coming years
01:16:58
all the cars pretty much all the cars on the road will be driving themselves? >> No.
01:17:01
>> You don't you don't think so? >> Mm-m. >> How come?
01:17:03
>> Because of the way the technology works. >> Because because these are statistical I
01:17:09
mean currently the way that AI models are primarily developed. They're statistical engines. You have what's
01:17:15
called a neural network, which is a piece of software that has a bunch of densely connected nodes and
01:17:22
>> like parameters. Is this what they call parameters? >> Yeah, pretty much. And you're just
01:17:26
pumping a bunch of data into it and then it's analyzing the data and creating
01:17:31
this all of these finding all these correlations in the data, finding all these patterns and then it's through
01:17:36
those patterns that the machine is then able to act autonomously, right? And so the way that they're training a
01:17:43
self-driving car is they're they're recording all this footage and then they
01:17:48
have tens of thousands or hundreds of thousands of human contractors that draw literally around every single vehicle in
01:17:57
the footage, every single pedestrian, every single traffic light, every single lane marking and label it exactly as
01:18:04
such. So that then it's fed into an AI model that can identify all of these different components and then it's
01:18:11
connected to another piece of software that is not AI that's saying okay if you
01:18:17
if the AI model recognizes the pedestrian we do not run over the pedestrian. If the AI model recognizes a red traffic
01:18:26
light we stop. And so the like the thing about statistical engines is that it's
01:18:32
based on probabilities. It's not based on deterministic logic. So systems make errors all the time and
01:18:41
it's impossible. It is technically impossible to get them to stop making errors.
01:18:48
>> Humans make errors way more than >> systems in this case. Like the safety
01:18:53
record is like isn't it like 10 times more safe to be driven in a Tesla with
01:18:57
autonomous driving than it is to for a human to drive? >> It depends on the place. It depends on
01:19:02
whether the Tesla was trained to specifically navigate the place that you're driving.
01:19:05
>> Get drunk >> because if it's in Mumbai, >> in some place in Vietnam, no, it would
01:19:12
not be safer. I WOULD MUCH RATHER be driven >> by someone that has been driving in that
01:19:18
place their whole life. I'm I'm not arguing against like the fact that in
01:19:22
certain places where the car has been explicitly trained to drive in this place that it has a better safety record
01:19:29
than the humans that are driving in that place. But you specifically asked if I think that all of the
01:19:34
>> most cars >> most cars in the world in the US >> in the United States cuz we're here.
01:19:40
>> I don't actually think that it's like imminently on the horizon
01:19:43
>> 10 years. >> No, I don't think so. >> I sat with Dra from Uber and he's pretty
01:19:47
convinced that his 9 million couriers will be replaced by autonomous vehicles. >> I mean, how long have has self-driving
01:19:53
cars been invested in thus far? It's been more than 10 years. And what percentage of
01:19:59
cars right now are autonomous >> on the US roads? I mean, so part of it is it's actually not a technical
01:20:07
problem, right? Like part of it is also social problem like do people even trust
01:20:11
getting into these vehicles? Part of it is also a legal problem which is if the car the self-driving car kills someone,
01:20:19
which it has happened. >> Yeah, it has happened. >> Who is responsible? So, in the case in
01:20:24
LA, it was both Tesla and the driver because the driver dropped their phone, they looked down, and this was a couple
01:20:30
of years ago, I believe. Um, and they went to grab their phone and they hit someone, and so it went to court, and
01:20:36
they were held both responsible, both the driver and Tesla. Um, in terms of Tesla,
01:20:44
pretty much everyone that gets the car, it comes with autonomy now for pretty much most people, I believe.
01:20:49
>> Partial autonomy. Yeah, it's called full self-driving at the moment where it's
01:20:51
like >> I mean, yes, it is called full self-driving. >> Full self-driving supervised where you
01:20:56
kind of have to be looking in the d. You have to be looking in the right direction, but
01:20:59
>> Yeah. So, it's partial autonomy. >> And here in Austin, it's full autonomy
01:21:04
cuz there's no steering wheel. >> Yeah. >> On the new car. Um, so you can't drive
01:21:07
it anyway. But it is, you know, the Model Y is the undisputed highest selling car, bestselling car in the
01:21:13
world across all brands. Well, I guess my point here is like these predictions where they say AI is going to completely
01:21:22
change transportation and driving. It's going to completely change lawyers aren't going to have jobs. Accountants
01:21:26
aren't going to have jobs. Um, do you believe that they are true? Do you believe that there's going to be mass
01:21:31
job displacement? >> Okay, so I do think that there is going to be huge impacts on employment and we
01:21:37
already seeing those impacts. It is not simply because the AI models are just automating those jobs away. It
01:21:44
is specifically because the models are improving in certain capabilities based on what the
01:21:51
companies that are developing them choose to improve them on. And executives at other companies are then
01:21:58
deciding to fire or lay off their workers because they think that AI can replace the worker irrespective of
01:22:06
whether that might be true. And there, you know, there have been cases of like the CLA CEO who laid off a bunch of
01:22:11
people thinking that he would replace everyone with AI and then it didn't actually work and he had to ask some
01:22:15
people to come back. >> I actually DM'd him about this. If you're hearing this, this is because
01:22:19
I've DM'd Sebastian and he's fine with me sharing this. >> He said, because I've heard his name
01:22:24
mentioned a lot and so when I when we talked about AI in the past and people mention Sebastian and Cler as the
01:22:29
example, I wanted to clarify with him what the truth was. >> He said, "It's great to hear from you.
01:22:33
Um, I think sometimes people struggle with two things can be true at the same time. I think it might be time to come
01:22:39
back on your podcast. To your point, this is the media misinterpreting my tweet. We are
01:22:44
doubling down on AI more than ever. Cler is shrinking with almost 100 employees per month due to AI. We used to be 7,400
01:22:52
at the peak. A year ago, 5,500. Now we're 3,300. And by the end of summer, so this was
01:23:00
last year, will be 3,000 people. AI handles 70% of our customer service conversations at this moment. This is
01:23:08
because we have realized that with AI, the production cost of software comes down to almost zero. Just like
01:23:13
manufacturing used to be all handcrafted and then the machines came. Code used to
01:23:17
be all handcrafted up until a few years ago. And now it is machine produced. And
01:23:23
ultimately we pay people more than ever for the unique handcrafted man-made stuff. China is a bank. People will want
01:23:31
to connect to humans not only machines. They want us to be personable, relatable, even flawed. So we need to
01:23:38
make sure while we are automating replacing with AI in parallel, we make sure we offer a super available human
01:23:46
experience. I'm really glad you read this because I think it touches on some
01:23:50
really important nuances to the AI. Yeah. Like the impact that AI is going to have on employment. So I think
01:23:58
the there's often these binary narratives. It's like AI is going to come for every job.
01:24:04
>> Mhm. >> Or people say AI is not actually working and it's not actually coming for jobs.
01:24:09
And like the reality is it's coming for jobs. There are definitely jobs that are
01:24:14
being automated away because of the capabilities of their models. And there's also jobs that are being lost
01:24:19
because executives are deciding to lay off the workers even if the models don't
01:24:23
match the capabilities because it's good enough. Like they would rather have the
01:24:26
good enough model for way cheaper >> or they made a mistake with hiring. They
01:24:30
blowed their team and it's a great convenient thing to say. >> Exactly. Like there's there's there's
01:24:34
many reason but like clearly we're already seeing impacts on the job market. Like the um US jobs report that
01:24:40
came out earlier this year showed that there has been a decline in hiring is a slowdown in hiring across especially
01:24:49
white collar professional industries. And you saw Anthropic's report the new
01:24:54
this week. The TLDDR is it matches kind of what you were saying where they Anthropic looked at exactly how people
01:25:00
were using their models and they looked at like what people are saying. >> Yeah.
01:25:04
>> And they said that there's been a 40% reduction in entry- level jobs in
01:25:08
particular and then they made this graph which has gone viral over the internet.
01:25:11
The red shows where we are now in terms of capability and based on how people are currently using the models they
01:25:17
prediction >> extrapolated out that the blue part will be the disrupted parts. This is the
01:25:21
things that they say AI can do right now, but people don't realize it yet. So, if you look at it, it's like it's
01:25:27
kind of all the stuff you would expect. >> Yeah. >> It's the physical real world human stuff
01:25:31
>> which robots maybe can do someday like construction or agriculture that are
01:25:35
untouched, but like office and admin, um like saying finance stuff, math, >> and notice that these are all the things
01:25:42
that I just named that they purposely >> finance, math, law, >> media and arts. That's me cooked.
01:25:48
>> Yeah. office and admin. I mean they do focus a lot on like assistant type and
01:25:55
managerial work. >> So but but the the other thing that the CLO CEO said was
01:26:02
but people also want human experiences. So it's not actually just about the capabilities of the models. It's also
01:26:08
about what people want like some things they would turn to AI for and some things they wouldn't irrespective of
01:26:16
whether or not AI is capable of doing it but because of a preference that they want humanto human interaction
01:26:24
>> and so what we're seeing right now is yeah the the thing that happens with
01:26:30
every wave of automation which is that there is a bunch of entry-level work that gets automated away and there There
01:26:38
are also new jobs created, but the jobs that are created are one in one of two categories. There are people that get
01:26:45
even higher skilled jobs and what he was saying like we pay people more for like
01:26:49
the handcrafted code now >> and there's also the people who get way worse jobs and so there was this amazing
01:26:57
article in New York magazine that was talking about how a lot of people are getting laid off and then they end up
01:27:05
working in data annotation which is the labor that I've been referring to throughout this conversation that
01:27:11
companies need in order to teach their models the next thing that the companies are trying to automate. And so like a
01:27:18
marketer gets laid off and then they go and work for a data annotation firm to train the models on the very job that
01:27:27
they were just laid off in which will then perpetuate more layoffs if that model then develops
01:27:33
that skill. And the article was talking about how this has become a huge catchall for a lot of people that are
01:27:44
struggling with finding job opportunities right now, including like awardwinning directors in Hollywood that
01:27:51
are actually secretly doing this data annotation work to put food on the table. And so when they talk about
01:27:58
there's going to be mass unemployment and then there's going to be some new
01:28:02
jobs created that we can't even imagine, I think a lot of these narratives rarely
01:28:06
talk about like first of all, why are some jobs going away? It's not just because of the model capabilities, it's
01:28:11
also because of executive choices and because of the rhetoric that they use if they want to just downsize. Um, but the
01:28:18
other thing that is rarely talked about is the jobs, a lot of the jobs that are created are way worse than the jobs that
01:28:26
were there >> and it breaks the career ladder. So, it's the entry level and the mid tier
01:28:31
jobs that get gouged out. It's higher order jobs and then way more lower order
01:28:37
jobs that get created. And so, how do people continue to progress in their careers? There's no more rungs on the
01:28:45
ladder. >> I actually don't know the answer to this question. And I've been furiously trying
01:28:48
to find a good answer to this question because I can, you know, everything is theory. And for my audience, I would say
01:28:55
most of my audience don't run businesses. A lot of them do, a lot of them aspire to, but they don't run
01:28:59
businesses. So, they're kind of, they're also in the land of theory. They're
01:29:02
hearing lots of different things. Jack Dorsey does his tweet saying he's halfing his headcount because of AI.
01:29:06
They don't know what's true. They don't know the sort of internal economics at
01:29:09
Jack's company and did he bloat the company during the pandemic and he's
01:29:12
just using this as an excuse to make this share price spike seven points because his investors now think they're
01:29:16
an AI company or whatever. >> Mh. >> It's hard to pass through. So eventually
01:29:20
I go, okay, what am I doing? >> I have hundred hundreds of team members, probably 70 companies I invest in, maybe
01:29:26
five or six that I'm like the lead shareholder in. What am I actually doing on a day-to-day basis right now? I am
01:29:31
I'm also I also consider myself to be head of recruitment >> but in the last month in particular I
01:29:36
have met extremely capable candidates in terms of cultural alignment hard work those kinds of things but I've had to
01:29:42
take a great deal of pause because when I run the experiment of can I get an AI agent to do that exact same thing the
01:29:48
answer is increasingly yes >> especially in a world of open clause >> and so what I'm curious like
01:29:56
>> now you confront this decision where you're seeing in this short-term period
01:30:01
you could just choose the AI agent and in the long-term period there is no career ladder. So, so who
01:30:11
are you promoting into these senior roles? Like what how do you resolve it for your own company?
01:30:16
>> Yeah, it's a good question. So, there's kind of two ways I'm thinking about it.
01:30:18
I think really deep expertise is very very valuable because if you're now the
01:30:22
orchestrator of potentially AI agents, it's really about um having a deep understanding of the right question to
01:30:28
ask and and that's someone who has deep expertise on something. So I need my CFO
01:30:33
>> because if she's going to be orchestrating our team of agents that might be doing financial analysis or
01:30:37
whatever else, she needs to understand what to tell them to do in our company. >> Mhm.
01:30:43
>> And in turn financial analysts can't do that. They need this the 50 odd years of
01:30:47
experience that you know CLA has. On the other end, I need Cass. Cass is 25. Cass
01:30:53
knows everything about AI agents. He's a young Japanese kid who's highly highly
01:30:58
curious. You know, on the weekend, he's building AI agents to solve problems in
01:31:01
my life. I need those two kinds of thinking, which is highly proficient agent maxing young kids or they don't
01:31:07
necessarily need to be young, but like really lean in high curiosity. That's
01:31:11
creating a force multiplier in my business. And then I need deep expertise. Now the everything else
01:31:16
outside of there is another one I've thought of another group is like people
01:31:19
with extremely great IRL people skills >> because we do meet people in real life.
01:31:26
We greet you when you arrive here. We greet we when we go for lunch with big clients that we have whether it's Apple
01:31:31
or LinkedIn or whoever it might be. We, you know, we need to smoosh. >> Mhm.
01:31:35
>> And we have teams who, you know, are in person in the office. So, we we do a lot
01:31:39
of stuff IRL and increasingly we're building communities even for this show.
01:31:42
We're doing community events all around the world. So, we need people that are
01:31:44
good at that as well. IRL, bringing people together in real life and organizing stuff. Those are the three
01:31:49
groups of people that I'm like, you know, irreplaceable right now. And if you were to to all of the all the roles
01:31:58
that could be done by AI agents, if we were to replace them with AI agents, do you think you would still have these
01:32:02
three roles pools of people to hire and promote into the three critical things that you need in the long term?
01:32:10
>> If things carry on at the the current rate of trajectory, >> yeah,
01:32:14
>> one could assert that even those roles would experience pressure. If you just
01:32:18
imagine like people think of things either statically or linearly or exponentially. Yeah,
01:32:22
>> you imagine an exponential rate of improvement, which is kind of what I've
01:32:25
seen. Even like a 10% compounding rate of improvement at some point, >> at some point, at some point, I think
01:32:34
what remains is actually the IRL irreplaceably human stuff, human to human, our Maslovian needs of being in
01:32:41
person like we are now aren't going to change. We need connection. Humans get
01:32:44
very sick when they don't have other human beings in their life and strong, deep relationships. 100% agree. So that
01:32:51
stuff is going to matter a whole lot. I have this contrarian weird take that actually maybe this is the first
01:32:56
technology that's going to deliver on the promise of making us human and connected because we're going to be
01:33:00
rendered useless of everything else other than what humans are good at. Cuz all the other technology said, "Oh,
01:33:05
we're going to make you more connected, connecting the world." And they
01:33:08
disconnected the world and isolated the world. But maybe this is the one. It's
01:33:10
so intelligent now that it doesn't need us to [ __ ] around in spreadsheets anymore.
01:33:13
>> Do you see that actually happening in real time right now that it's making us more
01:33:20
able to be in person, connected with one another, having deeper social community
01:33:26
engagements. >> Yes. >> Yes. >> And I'll give you some data points.
01:33:31
>> Okay. >> Data point number one, the Financial Times released a report on social media
01:33:35
usage. And what they saw is 2022 was the peak and it's plateaued ever since. The
01:33:40
generation that's plateaued the fastest and heading down is the younger generations. The boomers are still off
01:33:46
to the races, right? So on Facebook and stuff. And then you look at the way Gen Alfa are using social media. They're not
01:33:52
posting as much. They call it uh posting zero. They're scrolling sometimes, but
01:33:56
they're in dark social environments like WhatsApp and Snapchat and iMessage.
01:33:59
They're not like performing to the world. They also value IRL experiences much more than any other generation.
01:34:04
They're like not getting smashed. We're seeing every brand has a run club.
01:34:08
um I mean runs exploding around the world and we're seeing this real sort of
01:34:13
sort of almost like innate realization that like technology let us down at some fundamental level like dating apps let
01:34:20
us down social networking kind of has let us down and we're seeing I think maybe a bifocation of society where a
01:34:26
lot of people are going [ __ ] this like I want to go back to what it is to be a
01:34:29
human >> and I I would imagine that in such a world where intelligence is so
01:34:33
sophisticated that we no longer needed to sit at laptops and like I think screen time is going to continue to
01:34:38
fall. I think you go into an office, you're not going to see people sat at laptops. You're gonna see something
01:34:41
completely different. And I think maybe, you know, and then we talk about robots
01:34:47
and Optimus robots. Elon says there'll be 10 billion Optimus robots. Elon has
01:34:51
been wrong with timing before. He's almost never been wrong on the big things completely. He's just his timing
01:34:59
is got a bad track record. Um, so I think he's he's probably right. You know, I think I've I've got some people
01:35:05
on the way from Boston Dynamics and these other big companies like Scale AI, and they're actually bringing the robots
01:35:09
here to show it, like folding laundry, doing the dishes. I'm not saying that's
01:35:12
what I would want in my home, but I think factory work is going to completely change. I think a lot of
01:35:16
manual labor is going to completely change, and I think we're going to be forced to do what only we can do. Um,
01:35:22
Sebastian, who's the CEO of Cler, has actually just called me. >> Hello, Sebastian. You're right.
01:35:30
>> Hey, how are you? >> I'm good. How are you? It's been a while.
01:35:34
>> It has been a while since you're on the show. I was just saying we do need to
01:35:37
get you back on. >> I I just I just had a couple of simple questions cuz you know I do a lot of
01:35:41
interviews and um Clan has always mentioned because I think the media has said that you like double down on AI
01:35:46
then you reversed because it didn't work out. So I know I spoke to you a while
01:35:50
ago and we exchanged a couple of DMs about it but that was more than a it was almost a year ago now.
01:35:54
>> So I just wanted to get an update on Cler's business AI agents and all of
01:35:58
that if possible. First and foremost, we were early on uh released um AI uh to support our customer service which had
01:36:06
that uh initial uh benefit of uh more calls being dealt with by AI which customers liked because those calls or
01:36:13
chat messages were much much faster and more qualitative. Then since then that has actually expanded slightly. Um what
01:36:21
we did however try to communicate as well is that we believed in a world of where AI is cheap and available the
01:36:28
value of human interaction will be regarded as higher. So the future of customer service VIP is a human um we
01:36:37
have then hence doubled down on providing more of that but at the same time the efficiency gains within the
01:36:42
company has continued. I mean we used to be about 6,000 people and and now we are
01:36:49
less than 3,000 which is 2 3 years since we stopped recruiting and at same point
01:36:54
in time our revenue has doubled right so you can clearly see that AI has allowed
01:36:59
us to be do more with less people but we have avoided layoffs and instead relied
01:37:05
on natural attrition when people kind of move on to other jobs. I mean from my perspective we will continue to be very
01:37:14
you know not really recruit much. I mean we recruit a little bit here and there but we expect that kind of natural
01:37:19
attrition of 10 15% per year to continue and to become fewer. I think the big breakthrough was really in November
01:37:27
December last year where even the kind of more most skeptical uh engineers who were like very
01:37:35
well-renowned and and appreciated like the founder of Linux and stuff like that basically said that coding has now been
01:37:42
resolved and hence is not you know uh you don't need to code anymore and that
01:37:46
was kind of a common sentiment. So I think in in coding that's definitely an
01:37:51
engineering work that has been a tremendous shift in the last six months. >> What do all these people go do
01:37:57
Sebastian? >> I am optimistic. I mean I think obviously people will have a lot of
01:38:02
opinions about this topic but I still believe that we are going to move towards a richer society. Now in the
01:38:09
short term there could be more worry about what happens if people don't get a
01:38:14
job and and so forth. But I think in the longer term, I I am optimistic what it means for society and humanity.
01:38:21
>> Thank you so much, Seb. I'll chat to you soon. Thank you for taking the time. I
01:38:24
appreciate you, mate. Thanks. >> All right. All right. Byebye. Byebye. >> You know the little traditional SIM card
01:38:30
that goes inside of our phones. They haven't changed at all since they were invented in the '90s. You have this
01:38:36
physical piece of plastic that means you're locked into one carrier, one network, and the second you cross a
01:38:42
border, that carrier can start charging you whatever they want. But there are alternatives and today's sponsor SY is
01:38:48
one of them. It's an eSIM app that gives you a safe and secure data connection in
01:38:53
over 200 destinations. All of their ESIMs have built-in cyber security which is great if you're traveling for work
01:38:58
and looking at confidential material. I've been using SY whenever I travel because the connection is always
01:39:03
reliable and it saves me a ton of roaming fees. It also means I don't have to deal with all of the faf that
01:39:09
surrounds sorting out a SIM everywhere I go. If you want to give it a try, download the sale app from the app store
01:39:14
now and scan the QR code on screen. And if you want 15% off your first purchase,
01:39:19
use my code D O A when you get to check out. That's D O A for 15% off. Keep that
01:39:28
to yourself. This is something that I've made for you. I've realized that the Dio
01:39:32
audience are strivals that we want to accomplish. And one of the things I've learned is that when you
01:39:40
aim at the big big big goal, it can feel incredibly psychologically uncomfortable
01:39:46
because it's kind of like being stood at the foot of Mount Everest and looking
01:39:49
upwards. The way to accomplish your goals is by breaking them down into tiny small steps. And we call this in our
01:39:56
team the 1%. And actually this philosophy is highly responsible for much of our success here. So, what we've
01:40:02
done so that you at home can accomplish any big goal that you have is we've made
01:40:06
these 1% diaries and we released these last year and they all sold out. So, I asked my team over and over again to
01:40:13
bring the diaries back, but also to introduce some new colors and to make some minor tweaks to the diary. So, now
01:40:18
we have a better range for you. So, if you have a big goal in mind and you need a framework and a process and some
01:40:26
motivation, then I highly recommend you get one of these diaries before they all
01:40:30
sell out once again. And you can get yours at the diary.com. And if you want the link, the link is in
01:40:36
the description below. >> Any thoughts? Well, I actually had thoughts on something that you said
01:40:42
before he called, >> which is you were saying that the Jenzers like there's this trend that
01:40:48
they're actually disconnecting from technology. So, they're becoming more in
01:40:51
person. And then there's this other class of workers that are actually leaning into the technology, but then
01:40:56
becoming more human because they're leaning into the technology >> because they're realizing that they
01:41:01
should actually just be spending more time doing inerson interactions rather than staring at a spreadsheet. And so
01:41:08
they're no longer doing the typing, whatever. I really want to go back to this New York Magazine piece that just
01:41:12
came out >> because what you're describing is true for a very specific category of people,
01:41:18
which is often like the business owners and leadership within companies that actually can make these decisions on how
01:41:25
they spend their time and what they ultimately do with their time. But what the piece talks about is the working
01:41:34
class like people like people who are not business owners that are then having to experience being laid off and then
01:41:43
working for the data annotation industry which is now one of the top jobs on LinkedIn by the way. Um the yeah so
01:41:51
LinkedIn had a report that showed the top 10 jobs with the highest growth in the last year and data annotation is on
01:41:59
that list. >> And for anyone that doesn't know what data annotation is.
01:42:02
>> Yeah. So data annotation is the process of teaching these chat bots or or any AI
01:42:09
system to do what they ultimately are able to do. So the fact that chat GBT can chat is because there were tens of
01:42:16
thousands or hundreds of thousands of people that were literally typing into a large language model and showing it.
01:42:23
This is how you're supposed to then respond when a user types in a prompt like this. Before they did that work,
01:42:31
chatgbt didn't exist. Like it just it would just you would prompt the model and the model would generate some text
01:42:37
that was not in dialogue with the person. It would kind of generate something that was adjacently related.
01:42:42
Is this what they call reinforcement learning where you kind of you give it like a
01:42:45
>> it's a part of the process of reinforcement learning. So you do data
01:42:48
annotation which is literally um showing lots of different um you know examples of things that you
01:42:55
want the model to know and then reinforcement learning is getting the model to then train on those examples
01:43:00
iteratively in a way that then >> gives the model some of those capabilities. And what the New York
01:43:07
Magazine piece highlighted is many many of the people that are getting laid off now or or or are struggling to find
01:43:14
work. And these are highly educated people. They're college graduates, PhD graduates, law degree graduates,
01:43:21
doctors, um and again like award-winning directors that are that are then struggling to find employment in the
01:43:29
economy because the economy has been very much restructured by AI. they are then finding themselves being serving
01:43:36
this industry and the industry is designed in a way that is extremely inhumane because what the companies the
01:43:45
companies that use these data annotation services like there's these third party
01:43:48
providers that are data annotation firms an open AI a gro um a Google they will hire these firms to then find the
01:43:58
workers to perform the data annotation tasks that they need for these These firms, these third party firms, they are
01:44:05
incentivized to pit workers against each other because they want this data annotation to happen at speed and as
01:44:12
cheaply as possible so that they can also compete with one another in this middle layer to get the the the bid the
01:44:19
the contract from the the client. And so all of these workers that were interviewed for this New York Magazine
01:44:27
story talk about how they actually no longer have an ability to be human because they are waiting at their laptop
01:44:35
to be pinged on Slack for when a project is going to open up for data annotation
01:44:40
because they've tried job hunting. They literally can't find anything else. This
01:44:44
is the thing that's going to help them put food on the table for their kids.
01:44:46
And there was this one woman who said like, "I have so much anxiety about when
01:44:52
the project is going to come, when it's going to leave that when the project
01:44:56
came, it was right when my kid was coming off of off of school." And I just
01:45:01
started tasking furiously because I don't know what's going to go and I need
01:45:04
to earn as much money as possible in this window of opportunity. So then my when my kid came home and tried to talk
01:45:10
to me, I screamed at my child for for distracting me. And then she was like, "I've become a monster and I'm not even
01:45:19
allowed to go to the bathroom or take care of my kids, let alone myself, because this industry that is absorbing
01:45:28
more and more of the workers that are being laid off, is mechanizing my life, atomizing my work, devaluing my
01:45:38
expertise, and then harvesting it for the perpetuation of this machine that all of these AI executives are saying is
01:45:46
then going to come for everyone else's jobs. And so what you were saying about
01:45:52
these this class of workers, the business owners that get to become more human because there are all of
01:45:59
these AI models now doing the tasks that they don't have to do anymore. It is at
01:46:03
the cost of the vast majority of people who are not business owners that are struggling to find work getting absorbed
01:46:11
into the work of then providing these technologies that the business owners can use
01:46:18
>> and instead of becoming more human they feel like their humanity has been
01:46:23
squeezed and diminished and they have no ability to have control, agency and dignity in their lives anymore. I think
01:46:32
this is a big I think this is a big question that kind of pertains to this graph here which is you know all of
01:46:37
these people if we believe anthropics prediction of who will be disrupted these people in these industries like
01:46:43
arts and media legal um life and social sciences architecture and engineering computer and maths business and finance
01:46:52
and management and also office and admin. These people if we believe this would have to retrain at something else
01:46:58
and unlike the industrial revolution where you might get 10 20 years to retrain because factories take a long
01:47:03
time to build. The distribution layer that AI sits on top of is the open internet. So this is why chat can go and
01:47:09
get hundreds of millions of users in no time at all and become the fastest growing company of all time. Um one of
01:47:15
my fears is that this disruption takes place at a speed where we can't transition.
01:47:21
And that was you know that I think you you you said that sentence in the passive voice the transition would
01:47:28
happen at a speed but who is driving that speed? >> Um >> it's the companies
01:47:34
>> and their race with one another. >> Yeah. And so they are driving the
01:47:38
transition to happen at a speed at which it would be really hard to take care of
01:47:46
all of the people that would be bulldozed over by >> this is one of the crazy questions that
01:47:50
no one can answer for me when I sit with these people that are AI CEOs. So I go,
01:47:54
"So what happens to the people if this is if you agree that this is going to
01:47:56
happen at super speed?" You know, I spoke to that CEO of Uber, Dar, who said
01:48:00
very similar things to what you're saying is, you know, there'll be data
01:48:03
labeling jobs, for example, for the drivers. But um they can't all become data labelers. And there's a question
01:48:09
around meaning and purpose and fulfillment. And that comes from losing your meaning in life. I s also sit here
01:48:15
with so many people who talk about how their father lost their job in Iran or some some other country and came to the
01:48:22
United States and had to be a a toilet cleaner on particular case was a doctor in Iran but came to the US and was a
01:48:28
toilet cleaner and had to deal with the sense of shame that that particular person felt and the lack of dignity that
01:48:33
that caused and how that made that person's self-esteem feel and the depression alcoholism that transpired
01:48:38
from that. um if this happens at a large scale across society, there's going to
01:48:43
be a ton of consequences like that. >> I mean, this is this is like the core
01:48:47
themes of my work. And the reason why I'm critical of these companies is that
01:48:50
they are creating technologies in a way that creates the halves and have nots in
01:48:56
an extreme form that we have. It's it's exacerbating the inequality that we
01:49:01
already see in the world. Like the people who have things will have way more riches. they'll have way more free
01:49:08
time. They'll be allowed to be more human. But the people who don't have
01:49:12
things are even being squeezed even more. And it's not just from a work perspective. I mean, I talk in my book
01:49:23
also about the environmental and public health crisis that these companies have created where they are building these
01:49:31
colossal supercomput facilities. there and and in in comm community like communities all around the world and
01:49:39
they specifically pick some of the most vulnerable communities. We're sitting in
01:49:42
Texas right now. Open AAI's largest one of its largest data center projects is
01:49:48
being built in Abalene, Texas as part of the Stargate initiative which was an effort announced at the beginning of
01:49:54
Trump's second administration to spend $500 billion on AI computing infrastructure.
01:50:00
This facility consumes will when it's finished will consume more than a gigawatt of power
01:50:07
which is over 20% over 20%. So this is actually a little bit inaccurate now. Um this was
01:50:15
something that circulated online for a while but there's updated numbers >> just for someone that can't see cuz
01:50:20
they're listening on Spotify or something. It's a picture of the size of
01:50:23
this facility. >> So this is not the Abene Texas one. This is a meta facility. Yeah. So, let's
01:50:29
first talk about opening eyes facility in Texas. That one would be the size of Central Park and it would run a million
01:50:37
computer chips and it would require the power of more than 20% of New York City.
01:50:45
>> Do you know one of the things which I found confusing, so I'd like to like
01:50:48
alleviate the dissonance is I thought you were saying earlier that you didn't
01:50:51
think the job disruption promises were real. No, what I was saying is that when we
01:50:59
talk about what these executives predict about the future, we need to understand
01:51:05
that they are ultimately trying to influence the public in a way that allows them to continue maintaining
01:51:11
control over the technology. >> But objectively, do you think that the job disruption that they talk about
01:51:16
where >> Yeah. Yeah. I mean I I mentioned >> real >> well I >> I don't want to comment specifically on
01:51:21
like this chart but it's like we've already seen in job reports that there
01:51:25
is a restructuring of the economy happening right now. Yeah. >> But but going back to like the data
01:51:30
center. So this supercomputer facility it's a meta supercomputer facility >> is being built in Louisiana
01:51:37
>> and it would be four times the size of the Abene Texas one and use half of the
01:51:43
average power demand of New York City. So it's one the size of Manhattan. This
01:51:46
makes it seem like almost all of Manhattan, but it's it would be 1/5 the size of Manhattan. When these facilities
01:51:52
go into these communities, what happens? Power utility increases, grid reliability decreases. The facilities
01:52:01
also need fresh water to generate the power for powering them as well as fresh water to cool. And there have been lots
01:52:08
of documented stories of communities that are already really constrained in their freshwater resource. they're under
01:52:13
a drought when a facility comes in and then there are people the community is actually like competing with this
01:52:19
facility for fresh water. I talk about one of those communities in my book and also sometimes these facilities instead
01:52:25
of connecting to the grid they instead a a power plant pops up next to it. So in
01:52:31
Memphis Tennessee where Musk built Colossus the supercomputer for training Grock he used 35 methane gas turbines to
01:52:41
power the facility. This is a working-class community, a black and brown community, a rural community that
01:52:47
was not even told that they would be the hosts of this facility. And they discovered it because they literally
01:52:54
smelled what seemed like a gas leak in all of their living rooms. And that's
01:52:59
when they discovered that these methane gas turbines were taking away their right to clean air. And this is a
01:53:08
community that's already been facing a history of environmental racism. They
01:53:12
had already had lots of struggles to access their right to clean air. And now there's this huge supercomput that's
01:53:21
landed in their midst that is pumping thousands of tons of toxins into their air, exacerbating the asthmatic symptoms
01:53:30
of the children, exacerbating the respiratory illnesses of other people. that it's it's one of the communities
01:53:36
that has the highest rates of um lung cancer and so >> and that supercomputers taking their
01:53:44
jobs >> and then they also have supercomputers taking their jobs. So, so this is what I
01:53:48
mean is like the halves and have nots are fundamentally being pulled apart even further. Like if
01:53:56
you in this version of Silicon Valley's future are in the misfortunate category
01:54:03
of being a have not, we are talking about you now getting a job that is way worse than what you had because you
01:54:11
might be doing data annotation >> and you might be treated as a machine rather than as a human to extract value
01:54:18
the value of your labor for perpetuating this labor automating machine that these
01:54:23
people are building. You might be competing with these facilities for freshwater resources. They're also
01:54:30
polluting your air. Your bills have increased. So, the affordability crisis is getting worse.
01:54:37
Like, how is that making people able to be more human? >> What do we do about it?
01:54:43
>> Yes. >> Okay. So, one of the analogies that I always use is AI is like the word
01:54:50
transportation. Transportation can literally refer to everything from a bicycle to a rocket. And we have nuanced
01:54:57
conversations about transportation where we always say we need to transition our
01:55:01
transportation towards more uh sustainable options. We need a transition towards you know public
01:55:08
transport, electric vehicles. And we don't we don't ever say everyone should
01:55:13
get a rocket to do every to serve all of their transportation needs, right? Like
01:55:18
we're in Austin. If you use a rocket to fly from Dallas to Austin, like that
01:55:22
would just make not no sense. It's just a disproportionate use of resources to
01:55:26
get the benefit of getting from point A to point B. This how we should think about AI. So all of
01:55:33
the models that we've been talking about, I like to think of them as the rockets of AI. They use an extraordinary
01:55:40
amount of resources and they provide benefit some dramatic benefit to some people but they're also exacting an
01:55:47
extraordinary cost on a large swath of people because of the like the costs of developing this technology.
01:55:57
Why don't we build more bicycles of AI? This is things like deep minds alpha
01:56:02
fold which is a system that predicts how proteins will fold based on amino acid sequences. It's really important for
01:56:10
accelerating drug discovery for understanding human disease and it won the Nobel Prize in chemistry in 2024.
01:56:18
And the reason why it's a bicycle of AI is because you're using small curated
01:56:23
data sets. you're just you just have data that has amino acid sequences and protein folding. So that means you need
01:56:32
significantly less computational resources to develop the system, which means significantly less energy, which
01:56:38
means less emissions, so on and so forth. And you're providing enormous benefit to people.
01:56:43
>> It feels like the horse has left the stable in this regard because they've already taken people's
01:56:50
IP, they've taken media, they they train on this podcast. We know they do because
01:56:54
it it shows that they do. Um I think there's a button actually in the back end of YouTube now that allows you just
01:56:58
to click it and it says we will train on your YouTube channel. Um so the horses kind of left.
01:57:04
>> Here's the thing. If the horse truly had left the stables, they wouldn't have to
01:57:08
train on anything anymore. Why is it that their appetite for data has actually expanded? It's because in order
01:57:15
to build the next generations of their technologies, in order to have the technologies continue to be relevant and
01:57:21
continue to update with the pace of new knowledge creation and society's evolvement, they need to train again and
01:57:30
again and again and again. And why are they employing actually more and more and more data annotation workers over
01:57:36
time? It's because they need more and more of that work over time. I mean, I've been reporting on data annotation
01:57:44
work for over 7 years now, and it's not gone down. It's gone it's increased.
01:57:50
>> Do you think there's any chance of it going down? Do you think there's any
01:57:54
chance of this sort of brute force scaling approach where you take data, you take computational power, energy,
01:58:00
and you, you know, you have um the data labelers and, you know, building out more and more parameters for the models.
01:58:07
Do you think there's any chance it's going to stop or go in a different direction other than the one it's going
01:58:11
in now? >> I would love to reframe the question and say what should we be doing in this
01:58:16
moment where it's not going down where we do recognize that actually these companies in this moment need continued
01:58:24
resources, inputs and labor to perpetuate what they are doing. >> Yeah. because this sounds like stop
01:58:30
>> and I just feel like stop is like a HUD. It feels like I just think you know with
01:58:35
the government in place they're supporting these companies like crazy. Globally this is happening. So I'm like
01:58:40
stop doesn't feel >> I always say we need to break up the empire and we need to develop
01:58:44
alternatives and we are already seeing a flourishing of incredible grassroots movements that are applying an enormous
01:58:52
amount of pressure to the way that the empire is trying to unfold its agenda. 80% of Americans in the most recent poll
01:59:00
think that the AI industry need to be regulated. >> Yeah. >> When was the last time that 80% of
01:59:05
Americans were on the same side of an issue? >> No. Yeah. When I have these
01:59:08
conversations on the podcast, the comment section are clear. >> Yeah. >> There's no there's no disagreement.
01:59:12
There's no one in there going, "Oh, no. I think they should crack on."
01:59:14
>> Yeah. Dozens dozens of protests against data centers have broken out all around
01:59:19
this country and the US, all around the world. >> So, what do we do about it?
01:59:23
>> So, these are thing people that are doing something about it. They are actually reasserting their agency and
01:59:30
exercising democratic contestation against the ways that the empires are going about their business.
01:59:36
>> What goal should we be aiming at? So, if I said to my audience, Janet at home,
01:59:40
because this is kind of what I see in the comments, it's hopelessness. It's
01:59:42
like, what can I do? I'm just a >> Yeah. Well, well, well, the goal is not
01:59:47
that we completely get rid of this technology. The goal is that these companies need to stop being empires.
01:59:52
And the way I define like a typical business versus an empire is that the empires are predicated on this idea that
01:59:58
they do not have to provide a fair exchange of value with the workers who work for them or the people who use them
02:00:04
or all of the other people that are involved in like the supply chain of producing and deploying these
02:00:08
technologies. They can extract and exploit and extract and exploit and get more value than what they offer. Whereas
02:00:15
typical businesses, there's a fair exchange. you you buy a service, you feel like you got the same amount of
02:00:20
value as the service that you provided. But like for these data annotation workers, for example, they do not feel
02:00:25
in any way that they're being paid the same value that they provide to these
02:00:28
companies. So that's like for me the north star is like we should be pushing
02:00:33
back and holding accountable these companies when they operate in an imperial way. And that's what we've seen
02:00:41
with all of these people that are now literally protesting in the streets against data centers and having an
02:00:46
enormous effect, by the way, actually stalling data center projects and also completely banning data centers from
02:00:53
being developed in their localities. We're seeing that with artisan writers that are suing these companies for
02:00:59
intellectual property infringement and creating a huge public conversation about what is it that we actually how do
02:01:05
we actually want to protect our intellectual property? It's like I three weeks ago I met Megan Garcia who is the
02:01:12
mother of Sul Settzer III who is the 14-year-old who died by suicide after being sexually groomed by a
02:01:21
characterized chatbot. And she when that happened I mean obviously was incredibly
02:01:30
devastated by what had happened to her son. She also decided to do something about it. She sued the companies and
02:01:37
that lawsuit then sparked many other parents and families who were actually experiencing similar things to sue these
02:01:44
companies as well. That has created an enormous public conversation about what these companies are actually doing when
02:01:52
they exploit and they extract. What is the cost to the lives of people around the world including children? So, what
02:02:01
do you think my audience should do if they if they agree with everything written in your book, Age Empire of AI,
02:02:06
Dreams and Nightmares, and Sam Mortman's Open AI? If they agree with everything
02:02:10
said here, if they agree with everything we've discussed today, they're concerned
02:02:13
about their kids, they they don't want everyone to become data labelers, they
02:02:17
don't think that's a, you know, particularly great solution, what what can they actually go and do?
02:02:22
>> When I was writing the book, the only discourse that was happening was this is
02:02:26
the best thing since sliced bread. >> Mhm. because of all of the actions of
02:02:30
these people like saying when they're comp they're they're not happy with the
02:02:35
things that these companies are doing. We now have 80% of Americans that want to regulate this industry. And so I
02:02:40
would say to people, think about all of the ways that your life intersects with the resources and the that the AI
02:02:49
industry needs to perpetuate what they do and also the spaces that they would need to deploy these technologies to
02:02:55
continue having broad-based adoption >> in their work. So you're a data donor to
02:03:02
these companies. You could withhold that data. And that's what those artists and
02:03:07
writers are are doing. like they're suing these companies to withhold to try
02:03:10
and create mechanisms by which that data would then be withheld. You probably have a data center popping up around
02:03:16
you. If you're at a school environment or a company environment, you're probably having a discussion in those
02:03:23
environments right now about what should the AI adoption policy be? And these companies they like I was talking with
02:03:30
some open air employees just the other day and they were telling me that it's
02:03:35
understood internally that the revenue targets for the company are extraordinary and they need things to go
02:03:44
flawlessly for it to all work out. And so they would need every single person to adopt this, every single space to
02:03:53
adopt this. They would need to be able to build their data centers at the speed that they're trying to build them. And
02:03:59
so what I would say to everyone of your viewers is let's not make it go flawlessly if we don't agree with what
02:04:05
they are doing. >> Ah, okay. I got you. >> And then let's build alternatives.
02:04:09
Because the thing is what I'm saying is not that these technologies don't have utility.
02:04:16
It's that specifically the political economy that has emerged to support the
02:04:20
production of these technologies right now >> is exacting a lot of harm on people. But
02:04:25
we have research that shows that the very same capabilities could be developed with much more efficient
02:04:33
methods with much less resource consumption. And we have a lot of different other AI systems at our
02:04:40
disposal that are like the bicycles of AI that we also know provide extraordinary benefit at very little
02:04:46
cost. So let's break up the empire and let's forge new paths of AI development
02:04:50
that are broadly beneficial to everyone. >> It's strange. I'm quite I think I'm I'm
02:04:56
I've trained myself to deal with dichotoies in my head. And this for me is such is a dichotomy where I as a CEO
02:05:04
and as a founder, as an entrepreneur and someone that loves technology, I think it's incredible. It's absolutely
02:05:08
incredible AI. It's just so amazing and incredible the things it's enabled me to
02:05:12
do and create. >> Yeah. Because it's designed to enable people like you.
02:05:16
>> And my car driving in the morning and being safer. Incredible. Um I think you
02:05:23
know the billion odd people that use AI tools or chat or whatever it might be, they'd probably say that it's added
02:05:28
value to their life. But and this is the part that people find confusing that you
02:05:32
can and I like I invest in companies that are you know heavily using AI but and the big butt is is it possible to
02:05:37
think that is true and also think that there are significant unintended consequences which technology in the
02:05:44
history of technology should have taught us to take a moment to pause to talk about because
02:05:48
>> I think this is absolutely like you can have both of these things in your head
02:05:53
and what I'm saying is that this tension doesn't have to be a tension because we
02:05:58
could actually preserve the utility and benefits of these technologies but actually develop and design them in a
02:06:05
different way that doesn't have all of these unintended consequences. >> Yes. And I think there needs to be a big
02:06:10
social conversation which is why I have so many conversations about AI in the show like there needs to be a big social
02:06:14
conse uh conversation about being intentional about the social impact um the social and environmental impact and
02:06:22
that conversation is not being had in the in government. From what I can see, the conversation takes place in the
02:06:28
industry and actually trying to pull it out of the industry and and open people's minds to it is hopefully what
02:06:33
we've been doing over the last couple of months with this subject because >> I think it's actually been it it has
02:06:38
been been happening everywhere outside of the industry and for local governments and state level governments
02:06:44
there have been huge conversations about this everywhere. Like I've been on book
02:06:48
tour, I've been to dozens of cities around the world. People are having these crucial conversations everywhere.
02:06:56
I have not gone to a single city. >> Yes. Everywhere. Even here in South by.
02:06:59
>> Yeah. I haven't gone to a single city where the room is not packed and people
02:07:03
are not wrestling with the same exact questions as every other person in every other room that I've been in.
02:07:08
>> Speaking of packed rooms, I know you've got to go cuz you've got you've got to
02:07:11
talk today. So, I'm going to we've got a last question which is the closing
02:07:14
tradition on this podcast. How would your advice to a friend with a terminal diagnosis differ from what you would do
02:07:21
yourself? >> That's a great question. >> Differ from what you would do yourself?
02:07:25
>> Oh my god. I have I I would tell them like enjoy like live life for yourself. Um you
02:07:33
wouldn't do it >> and take it easy. And yeah, I I I am not taking it easy.
02:07:39
>> Well, I think it's a good thing you're not taking it easy because you're
02:07:41
leading a conversation which is incredibly important. And I think that's the thing. I think the conversation is
02:07:46
the important thing. And so, you know, because of algorithms and echo chambers, it's so rare to have a conversation
02:07:52
>> these days, especially a long form one. I agree. >> Like this. So, I think they're so
02:07:56
important. And your book is for anyone that's curious about >> I think a lot of people would have
02:08:01
learned a lot of stuff today cuz I sit here with and interview AI people all the time and I've learned so much today.
02:08:06
From reading your book and the extensive objective perspective that your book takes, you you're able to unravel all of
02:08:12
these stories that we sometimes see in tweets and we don't know if they're true
02:08:15
or not because you've gone and met the people and you've done your research and
02:08:18
you're incredibly intelligent person, extremely intelligent person who clearly
02:08:23
has humanity's interests as your north star and that shows up in everything you
02:08:28
do and everything you say. So please continue to fight in the way that you are um because it's an incredibly
02:08:32
important one. people like you that are, I think, galvanizing the world to take the
02:08:39
collective action that we're starting to see everywhere. >> Yeah. >> Empire of AI: Dreams and Nightmares in
02:08:44
Sam Alman's Open AI by Karen How. I'll link it below for anyone that wants to
02:08:49
read this book. I highly recommend you do. It's a New York Times bestseller for
02:08:51
good reason. Karen, thank you. >> Thank you so much, Stephen. >> YouTube have this new crazy algorithm
02:08:56
where they know exactly what video you would like to watch next based on AI and all of your viewing behavior. And the
02:09:02
algorithm says that this video is the perfect video for you. It's different for everybody looking right now.