00:00:00
they wondered if there was a better way
00:00:01
to find information on the web on
00:00:03
September 15th 1997 they registered
00:00:06
Google as a website one of the greatest
00:00:09
entrepreneurs of our times someone who
00:00:13
really wanted to think outside the box
00:00:14
if that sounds like it's impossible
00:00:16
let's try it he took a backseat in
00:00:18
recent years to other Google leaders
00:00:20
Brin is now back helping Google's
00:00:22
efforts in artificial intelligence I
00:00:24
feel lucky uh that I fell into doing
00:00:27
something um that I feel really matter
00:00:30
you know getting people
00:00:33
[Music]
00:00:34
[Applause]
00:00:36
[Music]
00:00:45
[Applause]
00:00:47
information no introduction needed
00:00:50
welcome I I just agreed to this last
00:00:53
minute as you know I don't know where
00:00:54
you pulled up that clip so fast you guys
00:00:56
team is amazing kind of amazing this is
00:00:58
kind of amazing yeah
00:01:00
I thought Serge Ser just well he asked
00:01:04
to come check out the conference and I
00:01:06
was like definitely like come hang out I
00:01:08
didn't actually understand to be
00:01:10
perfectly honest I thought you guys just
00:01:12
kind of had a podcast and like a little
00:01:14
get together or something but yeah this
00:01:17
kind of mind-blowing congratulations
00:01:19
thank you well I'm glad you came out I'm
00:01:22
feeling a little bit shy but yeah wow
00:01:24
but thanks for agreeing to chat for a
00:01:26
little bit we're going to talk for a
00:01:27
little bit so this was not on the
00:01:28
schedule um but uh I thought it'd be
00:01:31
great to talk to you given where you sit
00:01:35
in the world as AI is on the brink of
00:01:39
and is actively changing the world
00:01:41
obviously um you know you founded Google
00:01:43
with Larry in 1998 and um you know
00:01:46
recently it's been reported that you've
00:01:48
kind of spent a lot more time at Google
00:01:50
working on AI I thought maybe and and a
00:01:53
lot of Industry analysts and pundits
00:01:55
have been kind of arguing that llms and
00:01:57
conversational AI tools are kind of an
00:01:59
ex Potential Threat to Google search
00:02:01
that's that's one of the and I think a
00:02:02
lot of those people don't build
00:02:03
businesses or they have competitive
00:02:05
Investments but you know we'll leave
00:02:07
that to the side um but there's this big
00:02:09
kind of narrative on what's going to
00:02:10
happen to Google and and where's Google
00:02:12
sitting with AI and I know you're
00:02:13
spending a lot of time on it so thanks
00:02:14
for coming to talk about it how much
00:02:16
time are you spending at Google what are
00:02:17
you working on yeah um honestly like
00:02:20
pretty much every day I mean like I'm
00:02:21
missing today which is you know one of
00:02:24
the one of the reasons I was a little
00:02:25
reluctant but I'm glad I came um but
00:02:30
I think as a computer
00:02:32
scientist I've never seen anything as
00:02:35
exciting as all of the AI progress
00:02:38
that's happened the last few
00:02:40
years thanks um no but it's it's kind of
00:02:45
mindblowing when I went to grad school
00:02:47
in the 9s you know AI was like kind of
00:02:51
like a footnote in the curriculum almost
00:02:54
like you like oh maybe you have to do
00:02:56
this one little test on AI we tried all
00:02:58
these different things they don't really
00:02:59
work that's it that's all you need to
00:03:01
know um and then somehow miraculously
00:03:05
all these people who are working on
00:03:06
neural Nets which was one of the big
00:03:09
discarded uh approaches to AI in like
00:03:12
the' 60s 7s and so forth um just started
00:03:16
to make progress a little bit more
00:03:17
compute a little more data a few clever
00:03:21
algorithms um and the thing that's
00:03:24
happened in this last decade or so is
00:03:28
just amazing as a computer scientist
00:03:31
like every month um you know well all of
00:03:35
you I'm sure use all of the AI tools out
00:03:37
there but like every month there's like
00:03:39
a new amazing capability and I'm like
00:03:41
probably you know doubly wowed as
00:03:43
everybody else is that computers can do
00:03:46
this um and
00:03:48
so yeah for me I really got back into
00:03:51
the technical work um because I just
00:03:54
don't want to miss out on this um as a
00:03:58
computer scientist is an extension of
00:04:00
search or a rewriting of how people
00:04:04
retrieve
00:04:05
information I mean I just think that the
00:04:07
AI touches so many different elements of
00:04:12
day-to-day life and sure search is one
00:04:14
of them uh but it kind of covers
00:04:17
everything um for example programming
00:04:21
itself right like the way that I think
00:04:23
about
00:04:24
it is very different now like you know
00:04:28
writing code from scratch feels really
00:04:30
hard compared to just asking the AI to
00:04:33
do it right um yeah sorry um so what do
00:04:38
you do then um actually I've written a
00:04:40
little bit of code myself just for Just
00:04:43
for kicks just for fun uh and then
00:04:46
sometimes I've had the AI write the code
00:04:48
for me um uh which was which was fun um
00:04:53
I mean just one example I wanted to see
00:04:56
how good our AI models were at Sudoku
00:05:00
so I had the AI model itself write a
00:05:02
bunch of code that would automatically
00:05:03
generate Sudoku puzzles and then feed
00:05:06
them to the AI itself and then score it
00:05:09
and so forth right um but it could just
00:05:11
write that code and I was like talking
00:05:14
to the engineers about it and you know
00:05:17
whatever we had some debate back and
00:05:18
forth like I came back half an hour
00:05:19
later it's done and they they were kind
00:05:22
of impressed because they don't honestly
00:05:23
use the AI tools for their own coding as
00:05:25
much as I think they ought to right um
00:05:29
so that's interesting example because
00:05:31
maybe there's a model that does Sudoku
00:05:33
really well maybe there's a model that
00:05:36
like answers information questions for
00:05:38
me about facts on the in the world maybe
00:05:41
there's an AI model that designs houses
00:05:44
um a lot of people are working towards
00:05:47
these ginormous general purpose llms is
00:05:51
that where the world goes some people I
00:05:53
think refer I don't know who wrote this
00:05:54
recently said there's a God model like
00:05:55
there's going to be a god model and
00:05:56
that's why everyone's investing so much
00:05:58
is if you can build the god model
00:06:00
you're done you got AGI whatever terms
00:06:02
you want to use there's this one thing
00:06:05
to rule them all or is the reality of AI
00:06:08
that there are lots of smaller models
00:06:10
that do application specific things
00:06:12
maybe work together like in an agent
00:06:14
system like what's the what what what is
00:06:16
the evolution of model development and
00:06:18
the how models are ultimately used to do
00:06:20
all these cool things um yeah I mean I
00:06:24
think like if you looked 10 15 years ago
00:06:28
there were different AI techniques that
00:06:30
were used for different problems
00:06:32
altogether like uh you know the chess
00:06:34
playing AI was very different than image
00:06:37
generation which was you know very
00:06:40
different um than like recently the
00:06:42
graph neural net at Google that like
00:06:44
outperformed every physics forecasting
00:06:47
model I don't know if you know this but
00:06:48
you guys publish this pretty aesome
00:06:50
embarassed I but it was like a totally
00:06:53
different Arch it was a different system
00:06:54
it was trained differently and it ended
00:06:56
up in that particular so there
00:06:58
historically there have been different
00:07:00
systems and even recently um like the
00:07:04
international math Olympiad that we
00:07:06
participated in we got um silver metal
00:07:09
as an AI actually one point away from
00:07:11
gold um but we actually had three
00:07:14
different AI models in there there was
00:07:16
one very uh formal theorem proving model
00:07:19
that actually did basically the best
00:07:21
there was one uh specific to Geometry
00:07:24
problems believe it or not that was just
00:07:25
a special kind of AI uh and then there
00:07:27
was a general purpose language model
00:07:30
um but uh since then we've tried to take
00:07:33
the learnings from that that was just a
00:07:35
couple months ago uh and triy to infuse
00:07:38
some of the sort of knowledge and
00:07:40
ability from the formal prover into our
00:07:42
general language models um that's still
00:07:45
working progress but I do think the
00:07:48
trend is to have a more unified model I
00:07:52
don't know if I'd call a god model uh
00:07:55
but to have certainly sort of shared
00:07:58
architectures and and ultimately even
00:08:01
shared
00:08:02
models um right so if that's true you
00:08:06
need a lot of compute to train and
00:08:09
develop that model that big
00:08:12
model uh yeah yeah I mean you definitely
00:08:15
need a lot of compute I I think like
00:08:17
I've I've read
00:08:19
some articles out there that just like
00:08:22
extrapolate they're like you know it's
00:08:24
like 100 megawatt and a gwatt and 10
00:08:26
gwatt and 100 gwatt and I don't know if
00:08:28
I'm quite a believer in you know that
00:08:31
level of
00:08:32
extrapolation um partly because
00:08:36
also the algorithmic improvements that
00:08:38
have come over the course of the last
00:08:40
few years uh maybe are actually even
00:08:44
outpacing the increased compute that's
00:08:46
put into these
00:08:48
models so is it irrational the buildout
00:08:52
that's happening everyone talking about
00:08:55
the Nvidia Revenue the Nvidia profit the
00:08:58
Nvidia market cap supporting all of what
00:09:01
people call the hyperscalers and the
00:09:02
growth of the infrastructure needed to
00:09:04
build these very large scale models
00:09:06
using the techniques of today is this
00:09:09
irrational or is it rational because if
00:09:10
it works it's so big that it doesn't
00:09:13
matter how much you well first of all
00:09:14
I'm not like an economist or like a
00:09:16
market Watcher the way that you guys
00:09:18
very carefully um watch companies so I
00:09:20
just want to disclaim my abilities in
00:09:22
the space um I think that I know uh for
00:09:28
us we're kind of building out compute as
00:09:29
quickly as we can and we just have a
00:09:31
huge amount of demand I mean for example
00:09:33
our Cloud customers just want a huge
00:09:36
amount of tpus gpus you name it um you
00:09:41
know we just can't we have to turn down
00:09:42
customers uh because we just don't have
00:09:45
the compute available uh and we use it
00:09:47
internally to train our own models to
00:09:49
serve our own models and so forth
00:09:51
so I guess I think there are very good
00:09:54
reasons that companies are currently
00:09:56
building out comput at a fast pace um I
00:09:59
just don't know that I would look at the
00:10:04
training Trends and extrapolate three
00:10:06
orders of magnitude ahe just blindly
00:10:08
from where we are today but the
00:10:10
Enterprise demand is there out there you
00:10:12
know I mean they they want to do lots of
00:10:14
other things for example running
00:10:16
inference on all these AI models
00:10:18
applying them to all these um new
00:10:21
applications um yeah there doesn't seem
00:10:24
to
00:10:25
be uh a limit right now
00:10:29
and where have you seen the greatest
00:10:32
success surprising success in the
00:10:35
application of models whether it's in
00:10:38
robotics or biology what are you like
00:10:40
seeing that you're like wow this is
00:10:41
really working and where are things
00:10:43
going to be more challenging and take
00:10:45
longer than I think some people might be
00:10:48
expecting um yeah I mean uh now that you
00:10:52
mentioned those well I I would say in
00:10:54
biology you know we've had Alpha fold
00:10:56
for quite a while um and I'm not
00:10:59
personally a biologist but when I talk
00:11:01
to biologists out there like everybody
00:11:03
uses it and it's more recent uh variants
00:11:07
um uh and that is I guess a different
00:11:10
kind of AI but like I said I do think
00:11:12
all these things tend to
00:11:14
converge um you know
00:11:18
robotics for the most part I see in this
00:11:21
sort of wow stage like wow you could
00:11:24
make a robot do that with just you know
00:11:27
this general purpose language model or
00:11:29
just a little bit of fine tuning this
00:11:30
way or that and it's like
00:11:34
amazing uh but maybe
00:11:36
not for the most part yet at the level
00:11:40
of robustness that would make it like
00:11:42
day-to-day useful but you see a line of
00:11:44
sight to
00:11:45
it
00:11:47
um yeah yeah I mean it would be it's I
00:11:50
don't see any particular Google the
00:11:52
robotics business and then spun it out
00:11:55
or sold it we've had like had aot five
00:11:58
or six robotics businesses they just
00:12:00
weren't the timing wasn't right yeah um
00:12:03
yeah unfortunately I don't know I guess
00:12:05
yeah I think that was just a little too
00:12:07
early to be perfectly honest I mean
00:12:09
there was like Boston Dynamics um what
00:12:11
was called um start stamp I don't even
00:12:14
remember all the ones we had anyway
00:12:17
we've had like five or six
00:12:18
embarrassingly yeah um but they're very
00:12:22
cool um and they very
00:12:25
impressive
00:12:27
um it yeah it just feels kind of silly
00:12:30
having done all of that
00:12:32
work uh and seeing now how capable these
00:12:36
General language models are that include
00:12:38
for example vision and image and they
00:12:39
multimodal and they can understand the
00:12:42
scene and everything and not having had
00:12:44
that at the time uh yeah it just feels
00:12:47
like you were sort of on a treadmill
00:12:50
that wasn't going to get anywhere
00:12:51
without the modern AI technology you
00:12:54
spend a lot of time on core technology
00:12:56
do you also spend a lot of time on
00:12:57
product visioning where things going and
00:13:00
what like the human computer interaction
00:13:03
modalities are going to be in the future
00:13:04
in a world of AI everywhere like what's
00:13:07
our life going to be like I mean I guess
00:13:09
there's water cooler chitchat about
00:13:12
things like
00:13:13
that um sh care to share
00:13:17
any
00:13:20
I um trying to think of things that
00:13:23
aren't embarrassing um struggling but uh
00:13:26
friends I I guess it's like just really
00:13:31
hard
00:13:33
to you know just forecast like you know
00:13:36
to think five years out because you know
00:13:38
based on the base technical capability
00:13:40
of the AI is what enables the
00:13:43
applications um and then sometimes you
00:13:45
know somebody will just whip up a little
00:13:47
demo that you just didn't think
00:13:50
about
00:13:51
um and it'll be kind of mind-blowing
00:13:55
yeah um uh and uh and of course then
00:13:59
from demo to actually making it real in
00:14:01
production so forth takes time um I
00:14:04
don't know if you've played with like uh
00:14:05
the Astra model but it's just sort of
00:14:08
live video and audio and you can chat
00:14:10
with the AI about what's going on in
00:14:12
your environment you'll give me access
00:14:14
right uh yeah I'll get well once I have
00:14:16
access um I mean I'm I'm sort of
00:14:20
sometimes the slowest to get some of
00:14:22
these
00:14:23
things um but it's um yeah there's like
00:14:26
a moment of wow
00:14:30
uh and you're like oh my God this is
00:14:33
amazing and then you're like okay well
00:14:36
it does a correctly like 90% of the time
00:14:39
but am I really like is that then worth
00:14:42
it if 10% of the time it's kind to make
00:14:43
a mistake or taking too long or
00:14:46
whatever and then you have to work work
00:14:48
work work work work work to get to
00:14:50
Perfect all those things make it
00:14:51
responsive make it available whatever
00:14:53
and then you actually end up with
00:14:56
something kind of amazing I heard a
00:14:58
story
00:15:00
that you went in you were on site I
00:15:03
should have mentioned this to you before
00:15:04
you came on stage see if you were cool
00:15:05
about talking about here we are um and
00:15:08
there like a bunch of Engineers showed
00:15:09
you that you could like use AI to write
00:15:11
code and it was like well we haven't
00:15:13
pushed it in Gemini yet um because we
00:15:15
want to make sure it doesn't make
00:15:17
mistakes and there was this like
00:15:18
hesitation culturally at Google to do
00:15:20
that and you were like no if it writes
00:15:22
code push it and you really and a lot of
00:15:24
people have told me this story because
00:15:25
they said and um or you know I've heard
00:15:28
this that it was really important to
00:15:30
hear that from you the founder in being
00:15:33
really clear that Google's conservatism
00:15:36
you know can't rule the day today and we
00:15:39
need to kind of see Google push the
00:15:41
envelope is that accurate is that kind
00:15:43
of huh how you've spent some time or I
00:15:45
don't remember the specific in just to
00:15:47
be honest but uh but I'm not
00:15:51
surprised um I mean I guess that's the
00:15:53
question for me is like as Google's
00:15:54
gotten so big there's more to lose
00:15:59
I think there's like this um yeah I
00:16:02
think there's a little bit of fearful I
00:16:04
mean language models to begin with like
00:16:06
we invented them basically with a
00:16:08
Transformer paper that was um whatever
00:16:10
six eight years ago something like that
00:16:13
um and uh oh no one by the way is back
00:16:15
at Google now which is awesome Cong um
00:16:19
and um yeah we were we were too timid uh
00:16:22
to deploy them um and you know for a lot
00:16:26
of good reasons like whatever they some
00:16:28
make mist mistakes they say embarrassing
00:16:30
things whatever you know um they're you
00:16:34
know sometimes they're just like kind of
00:16:36
embarrassing how dumb they are even
00:16:37
today's like latest and greatest things
00:16:40
like make really stupid mistakes people
00:16:43
would never make um and at the same
00:16:47
time like they're incredibly powerful
00:16:50
and they can help you do things you
00:16:52
never would have done and um you know
00:16:55
like I've like programmed really
00:16:57
complicated things with my kid like
00:16:59
they'll just program it because they
00:17:00
just ask the AI using all these really
00:17:03
complicated apis and all kinds of things
00:17:05
that would take like a month to like
00:17:08
learn so I just think that that
00:17:10
capability is
00:17:12
Magic
00:17:14
and uh you need to be willing to have
00:17:17
some
00:17:18
embarrassments uh and take some risks
00:17:21
and um and I think we've gotten better
00:17:23
at that and well you guys have probably
00:17:25
seen some more
00:17:26
embarrassments um but you're comfor
00:17:29
I have super voting you're still like I
00:17:31
mean you're comfortable with the
00:17:32
embarrassments at this St it's so to do
00:17:35
this like I mean not not particular on
00:17:38
the basis of my stock but I I um but as
00:17:40
a you know I mean but am I comfortable
00:17:43
um I mean I guess I just think of it is
00:17:48
this something magical we're giving the
00:17:52
world yeah and I think as long as we
00:17:54
communicate it properly like saying like
00:17:56
look this thing is amazing
00:17:59
and we'll periodically get stuff really
00:18:02
wrong uh then I think we should put it
00:18:07
out there and let people experiment and
00:18:09
see what new ways they find to use it um
00:18:12
I just don't think this is the
00:18:14
technology you want to just kind of keep
00:18:16
close to the chest and hidden until it's
00:18:18
like
00:18:19
perfect do you think that there's so
00:18:21
many places that AI can affect the world
00:18:24
and so much value to be created that
00:18:27
it's not really a race between Google
00:18:29
and meta and Amazon like people frame
00:18:32
these things as kind of a race is there
00:18:34
just so much value to be created that
00:18:35
you're working on a lot of different
00:18:37
opportunities and it's not really about
00:18:39
who builds the the model that score the
00:18:42
llm that scores the best that there's so
00:18:44
much more to it I mean how do you kind
00:18:45
of think
00:18:46
about um the world out there and
00:18:49
Google's place in it I mean I I think
00:18:52
it's very
00:18:54
helpful to have competition in the sense
00:18:57
that all these guys are vying and um we
00:19:00
just we were number one for on olysis
00:19:02
for a couple weeks by the way uh just
00:19:04
now and I think we're last time I
00:19:06
checked we're still beat the Top Model
00:19:08
there's just some El stuff so you do
00:19:09
care yeah yeah not
00:19:13
saying not but um uh and uh um and I you
00:19:18
know we've come a long way since um you
00:19:21
know a couple whatever years ago um chat
00:19:24
GPT launched or and we were quite a ways
00:19:27
behind uh I'm really pleased with all
00:19:29
the progress we made so we definitely
00:19:31
pay attention I mean I think it's great
00:19:34
that there are all these AI companies
00:19:36
out there be it uh US Open AI anthropic
00:19:41
um you name it there's um mistol it's
00:19:45
it's a I mean it's a big fast moving
00:19:47
field but I guess your question is yeah
00:19:50
I mean I think there's tremendous
00:19:52
value uh to humanity and I I think if
00:19:55
you think
00:19:57
back uh you know like when I was in
00:20:00
college let's say and there wasn't
00:20:02
really a proper internet or like web the
00:20:05
way that we know it today like the
00:20:07
amount of effort it would take to get
00:20:08
basic information the amount of effort
00:20:11
it would take to communicate with people
00:20:14
you know before cell phones and things
00:20:17
um like we've gained so much
00:20:20
capability uh ac across the world uh but
00:20:24
the sort of the new AI is another big
00:20:27
capability
00:20:29
uh and pretty much everybody in the
00:20:30
world can get access to it in one form
00:20:32
or another these days and I think it's
00:20:34
super exciting it's awesome uh sorry we
00:20:37
have so such limited time Sergey thank
00:20:39
you so much for joining us please join
00:20:40
me in thanking Sergey thank
00:20:42
[Applause]
00:20:44
you thanks yeah