00:00:00
they wondered if there was a better way to find information on the web on September 15th 1997 they registered
00:00:06
Google as a website one of the greatest entrepreneurs of our times someone who really wanted to think outside the box
00:00:14
if that sounds like it's impossible let's try it he took a backseat in recent years to other Google leaders
00:00:20
Brin is now back helping Google's efforts in artificial intelligence I feel lucky uh that I fell into doing
00:00:27
something um that I feel really matter you know getting people [Music] [Applause]
00:00:36
[Music] [Applause] information no introduction needed welcome I I just agreed to this last
00:00:53
minute as you know I don't know where you pulled up that clip so fast you guys team is amazing kind of amazing this is
00:00:58
kind of amazing yeah I thought Serge Ser just well he asked to come check out the conference and I
00:01:06
was like definitely like come hang out I didn't actually understand to be perfectly honest I thought you guys just
00:01:12
kind of had a podcast and like a little get together or something but yeah this kind of mind-blowing congratulations
00:01:19
thank you well I'm glad you came out I'm feeling a little bit shy but yeah wow but thanks for agreeing to chat for a
00:01:26
little bit we're going to talk for a little bit so this was not on the schedule um but uh I thought it'd be
00:01:31
great to talk to you given where you sit in the world as AI is on the brink of and is actively changing the world
00:01:41
obviously um you know you founded Google with Larry in 1998 and um you know recently it's been reported that you've
00:01:48
kind of spent a lot more time at Google working on AI I thought maybe and and a lot of Industry analysts and pundits
00:01:55
have been kind of arguing that llms and conversational AI tools are kind of an ex Potential Threat to Google search
00:02:01
that's that's one of the and I think a lot of those people don't build businesses or they have competitive
00:02:05
Investments but you know we'll leave that to the side um but there's this big kind of narrative on what's going to
00:02:10
happen to Google and and where's Google sitting with AI and I know you're spending a lot of time on it so thanks
00:02:14
for coming to talk about it how much time are you spending at Google what are you working on yeah um honestly like
00:02:20
pretty much every day I mean like I'm missing today which is you know one of the one of the reasons I was a little
00:02:25
reluctant but I'm glad I came um but I think as a computer scientist I've never seen anything as
00:02:35
exciting as all of the AI progress that's happened the last few years thanks um no but it's it's kind of
00:02:45
mindblowing when I went to grad school in the 9s you know AI was like kind of like a footnote in the curriculum almost
00:02:54
like you like oh maybe you have to do this one little test on AI we tried all these different things they don't really
00:02:59
work that's it that's all you need to know um and then somehow miraculously all these people who are working on
00:03:06
neural Nets which was one of the big discarded uh approaches to AI in like the' 60s 7s and so forth um just started
00:03:16
to make progress a little bit more compute a little more data a few clever algorithms um and the thing that's
00:03:24
happened in this last decade or so is just amazing as a computer scientist like every month um you know well all of
00:03:35
you I'm sure use all of the AI tools out there but like every month there's like
00:03:39
a new amazing capability and I'm like probably you know doubly wowed as everybody else is that computers can do
00:03:46
this um and so yeah for me I really got back into the technical work um because I just
00:03:54
don't want to miss out on this um as a computer scientist is an extension of search or a rewriting of how people
00:04:04
retrieve information I mean I just think that the AI touches so many different elements of
00:04:12
day-to-day life and sure search is one of them uh but it kind of covers everything um for example programming
00:04:21
itself right like the way that I think about it is very different now like you know
00:04:28
writing code from scratch feels really hard compared to just asking the AI to do it right um yeah sorry um so what do
00:04:38
you do then um actually I've written a little bit of code myself just for Just for kicks just for fun uh and then
00:04:46
sometimes I've had the AI write the code for me um uh which was which was fun um
00:04:53
I mean just one example I wanted to see how good our AI models were at Sudoku so I had the AI model itself write a
00:05:02
bunch of code that would automatically generate Sudoku puzzles and then feed them to the AI itself and then score it
00:05:09
and so forth right um but it could just write that code and I was like talking to the engineers about it and you know
00:05:17
whatever we had some debate back and forth like I came back half an hour later it's done and they they were kind
00:05:22
of impressed because they don't honestly use the AI tools for their own coding as
00:05:25
much as I think they ought to right um so that's interesting example because maybe there's a model that does Sudoku
00:05:33
really well maybe there's a model that like answers information questions for me about facts on the in the world maybe
00:05:41
there's an AI model that designs houses um a lot of people are working towards these ginormous general purpose llms is
00:05:51
that where the world goes some people I think refer I don't know who wrote this recently said there's a God model like
00:05:55
there's going to be a god model and that's why everyone's investing so much is if you can build the god model
00:06:00
you're done you got AGI whatever terms you want to use there's this one thing to rule them all or is the reality of AI
00:06:08
that there are lots of smaller models that do application specific things maybe work together like in an agent
00:06:14
system like what's the what what what is the evolution of model development and the how models are ultimately used to do
00:06:20
all these cool things um yeah I mean I think like if you looked 10 15 years ago there were different AI techniques that
00:06:30
were used for different problems altogether like uh you know the chess playing AI was very different than image
00:06:37
generation which was you know very different um than like recently the graph neural net at Google that like
00:06:44
outperformed every physics forecasting model I don't know if you know this but you guys publish this pretty aesome
00:06:50
embarassed I but it was like a totally different Arch it was a different system it was trained differently and it ended
00:06:56
up in that particular so there historically there have been different systems and even recently um like the
00:07:04
international math Olympiad that we participated in we got um silver metal as an AI actually one point away from
00:07:11
gold um but we actually had three different AI models in there there was one very uh formal theorem proving model
00:07:19
that actually did basically the best there was one uh specific to Geometry problems believe it or not that was just
00:07:25
a special kind of AI uh and then there was a general purpose language model um but uh since then we've tried to take
00:07:33
the learnings from that that was just a couple months ago uh and triy to infuse some of the sort of knowledge and
00:07:40
ability from the formal prover into our general language models um that's still working progress but I do think the
00:07:48
trend is to have a more unified model I don't know if I'd call a god model uh but to have certainly sort of shared
00:07:58
architectures and and ultimately even shared models um right so if that's true you
00:08:06
need a lot of compute to train and develop that model that big model uh yeah yeah I mean you definitely
00:08:15
need a lot of compute I I think like I've I've read some articles out there that just like
00:08:22
extrapolate they're like you know it's like 100 megawatt and a gwatt and 10 gwatt and 100 gwatt and I don't know if
00:08:28
I'm quite a believer in you know that level of extrapolation um partly because also the algorithmic improvements that
00:08:38
have come over the course of the last few years uh maybe are actually even outpacing the increased compute that's
00:08:46
put into these models so is it irrational the buildout that's happening everyone talking about
00:08:55
the Nvidia Revenue the Nvidia profit the Nvidia market cap supporting all of what
00:09:01
people call the hyperscalers and the growth of the infrastructure needed to build these very large scale models
00:09:06
using the techniques of today is this irrational or is it rational because if it works it's so big that it doesn't
00:09:13
matter how much you well first of all I'm not like an economist or like a market Watcher the way that you guys
00:09:18
very carefully um watch companies so I just want to disclaim my abilities in the space um I think that I know uh for
00:09:28
us we're kind of building out compute as quickly as we can and we just have a huge amount of demand I mean for example
00:09:33
our Cloud customers just want a huge amount of tpus gpus you name it um you know we just can't we have to turn down
00:09:42
customers uh because we just don't have the compute available uh and we use it internally to train our own models to
00:09:49
serve our own models and so forth so I guess I think there are very good reasons that companies are currently
00:09:56
building out comput at a fast pace um I just don't know that I would look at the
00:10:04
training Trends and extrapolate three orders of magnitude ahe just blindly from where we are today but the
00:10:10
Enterprise demand is there out there you know I mean they they want to do lots of
00:10:14
other things for example running inference on all these AI models applying them to all these um new
00:10:21
applications um yeah there doesn't seem to be uh a limit right now and where have you seen the greatest
00:10:32
success surprising success in the application of models whether it's in robotics or biology what are you like
00:10:40
seeing that you're like wow this is really working and where are things going to be more challenging and take
00:10:45
longer than I think some people might be expecting um yeah I mean uh now that you
00:10:52
mentioned those well I I would say in biology you know we've had Alpha fold for quite a while um and I'm not
00:10:59
personally a biologist but when I talk to biologists out there like everybody uses it and it's more recent uh variants
00:11:07
um uh and that is I guess a different kind of AI but like I said I do think all these things tend to
00:11:14
converge um you know robotics for the most part I see in this sort of wow stage like wow you could
00:11:24
make a robot do that with just you know this general purpose language model or just a little bit of fine tuning this
00:11:30
way or that and it's like amazing uh but maybe not for the most part yet at the level
00:11:40
of robustness that would make it like day-to-day useful but you see a line of sight to
00:11:45
it um yeah yeah I mean it would be it's I don't see any particular Google the robotics business and then spun it out
00:11:55
or sold it we've had like had aot five or six robotics businesses they just weren't the timing wasn't right yeah um
00:12:03
yeah unfortunately I don't know I guess yeah I think that was just a little too early to be perfectly honest I mean
00:12:09
there was like Boston Dynamics um what was called um start stamp I don't even remember all the ones we had anyway
00:12:17
we've had like five or six embarrassingly yeah um but they're very cool um and they very
00:12:25
impressive um it yeah it just feels kind of silly having done all of that work uh and seeing now how capable these
00:12:36
General language models are that include for example vision and image and they multimodal and they can understand the
00:12:42
scene and everything and not having had that at the time uh yeah it just feels like you were sort of on a treadmill
00:12:50
that wasn't going to get anywhere without the modern AI technology you spend a lot of time on core technology
00:12:56
do you also spend a lot of time on product visioning where things going and what like the human computer interaction
00:13:03
modalities are going to be in the future in a world of AI everywhere like what's
00:13:07
our life going to be like I mean I guess there's water cooler chitchat about things like
00:13:13
that um sh care to share any I um trying to think of things that aren't embarrassing um struggling but uh
00:13:26
friends I I guess it's like just really hard to you know just forecast like you know
00:13:36
to think five years out because you know based on the base technical capability of the AI is what enables the
00:13:43
applications um and then sometimes you know somebody will just whip up a little demo that you just didn't think
00:13:50
about um and it'll be kind of mind-blowing yeah um uh and uh and of course then from demo to actually making it real in
00:14:01
production so forth takes time um I don't know if you've played with like uh the Astra model but it's just sort of
00:14:08
live video and audio and you can chat with the AI about what's going on in your environment you'll give me access
00:14:14
right uh yeah I'll get well once I have access um I mean I'm I'm sort of sometimes the slowest to get some of
00:14:22
these things um but it's um yeah there's like a moment of wow uh and you're like oh my God this is
00:14:33
amazing and then you're like okay well it does a correctly like 90% of the time but am I really like is that then worth
00:14:42
it if 10% of the time it's kind to make a mistake or taking too long or whatever and then you have to work work
00:14:48
work work work work work to get to Perfect all those things make it responsive make it available whatever
00:14:53
and then you actually end up with something kind of amazing I heard a story that you went in you were on site I
00:15:03
should have mentioned this to you before you came on stage see if you were cool about talking about here we are um and
00:15:08
there like a bunch of Engineers showed you that you could like use AI to write code and it was like well we haven't
00:15:13
pushed it in Gemini yet um because we want to make sure it doesn't make mistakes and there was this like
00:15:18
hesitation culturally at Google to do that and you were like no if it writes code push it and you really and a lot of
00:15:24
people have told me this story because they said and um or you know I've heard this that it was really important to
00:15:30
hear that from you the founder in being really clear that Google's conservatism you know can't rule the day today and we
00:15:39
need to kind of see Google push the envelope is that accurate is that kind of huh how you've spent some time or I
00:15:45
don't remember the specific in just to be honest but uh but I'm not surprised um I mean I guess that's the
00:15:53
question for me is like as Google's gotten so big there's more to lose I think there's like this um yeah I
00:16:02
think there's a little bit of fearful I mean language models to begin with like we invented them basically with a
00:16:08
Transformer paper that was um whatever six eight years ago something like that um and uh oh no one by the way is back
00:16:15
at Google now which is awesome Cong um and um yeah we were we were too timid uh to deploy them um and you know for a lot
00:16:26
of good reasons like whatever they some make mist mistakes they say embarrassing
00:16:30
things whatever you know um they're you know sometimes they're just like kind of
00:16:36
embarrassing how dumb they are even today's like latest and greatest things like make really stupid mistakes people
00:16:43
would never make um and at the same time like they're incredibly powerful and they can help you do things you
00:16:52
never would have done and um you know like I've like programmed really complicated things with my kid like
00:16:59
they'll just program it because they just ask the AI using all these really complicated apis and all kinds of things
00:17:05
that would take like a month to like learn so I just think that that capability is
00:17:12
Magic and uh you need to be willing to have some embarrassments uh and take some risks
00:17:21
and um and I think we've gotten better at that and well you guys have probably seen some more
00:17:26
embarrassments um but you're comfor I have super voting you're still like I mean you're comfortable with the
00:17:32
embarrassments at this St it's so to do this like I mean not not particular on the basis of my stock but I I um but as
00:17:40
a you know I mean but am I comfortable um I mean I guess I just think of it is this something magical we're giving the
00:17:52
world yeah and I think as long as we communicate it properly like saying like look this thing is amazing
00:17:59
and we'll periodically get stuff really wrong uh then I think we should put it out there and let people experiment and
00:18:09
see what new ways they find to use it um I just don't think this is the technology you want to just kind of keep
00:18:16
close to the chest and hidden until it's like perfect do you think that there's so
00:18:21
many places that AI can affect the world and so much value to be created that it's not really a race between Google
00:18:29
and meta and Amazon like people frame these things as kind of a race is there just so much value to be created that
00:18:35
you're working on a lot of different opportunities and it's not really about who builds the the model that score the
00:18:42
llm that scores the best that there's so much more to it I mean how do you kind of think
00:18:46
about um the world out there and Google's place in it I mean I I think it's very
00:18:54
helpful to have competition in the sense that all these guys are vying and um we
00:19:00
just we were number one for on olysis for a couple weeks by the way uh just now and I think we're last time I
00:19:06
checked we're still beat the Top Model there's just some El stuff so you do care yeah yeah not
00:19:13
saying not but um uh and uh um and I you know we've come a long way since um you
00:19:21
know a couple whatever years ago um chat GPT launched or and we were quite a ways
00:19:27
behind uh I'm really pleased with all the progress we made so we definitely pay attention I mean I think it's great
00:19:34
that there are all these AI companies out there be it uh US Open AI anthropic um you name it there's um mistol it's
00:19:45
it's a I mean it's a big fast moving field but I guess your question is yeah I mean I think there's tremendous
00:19:52
value uh to humanity and I I think if you think back uh you know like when I was in
00:20:00
college let's say and there wasn't really a proper internet or like web the way that we know it today like the
00:20:07
amount of effort it would take to get basic information the amount of effort it would take to communicate with people
00:20:14
you know before cell phones and things um like we've gained so much capability uh ac across the world uh but
00:20:24
the sort of the new AI is another big capability uh and pretty much everybody in the
00:20:30
world can get access to it in one form or another these days and I think it's super exciting it's awesome uh sorry we
00:20:37
have so such limited time Sergey thank you so much for joining us please join me in thanking Sergey thank
00:20:42
[Applause] you thanks yeah