00:00:00
what's up people of the internet people of the internet yes it's David uh today we got a little bonus episode for you
00:00:07
don't worry we've got a regular episode on Friday still so stay tuned for that but I wanted to dig a little bit deep
00:00:12
into what exactly AI is right I think we all have been hearing about that for months now um years possibly but nobody actually has really explained like what
00:00:24
it is or how it works right like people just say that things are AI but what does that even mean um
00:00:30
so I want to get an answer for that so I called up my friend from Google who definitely knows what that means and we
00:00:36
have a little nice conversation about how this all works so hope you enjoy uh Daniel was gracious enough to come on the podcast and of course we met at my Cafe classic uh so yeah we're going to
00:00:48
debrief after this but um enjoy we've been talking a lot about uh Ai and generative Ai and all of the
00:01:01
stuff that's happening in the world right now and it's very confusing so we thought it would be actually pretty useful if we got someone who knew what
00:01:07
they were talking about to come on the podcast we weren't just speculating constantly so today uh we have denu
00:01:14
banga with us uh he is Google's head of generative AI or director of generative AI so we're going to have a long conversation about what that means all right so danu if you were to explain to
00:01:27
someone including me what you do at Google goal or as your job what would that be we we try to incubate uh generative AI Solutions into production
00:01:40
grade applications for uh companies startups and or Enterprises so that would that include like a company comes to you they say we want to use generative Ai and then you work with
00:01:50
them to actually integrate it into their product that's correct uh but we do we do have many other teams that really
00:01:55
focus on that longtail work of integration so to say our interest is to figure out what are the patterns that
00:02:02
are not necessarily common at this point or the new ones and then really turn that into 10x scare packages so as you
00:02:09
know many of these technology items especially within the AI space are fairly new right so they demand new
00:02:16
technologies they demand new approaches to Technologies and then what we do is to try to figure out what are the
00:02:21
patterns within this a bit open ecosystem at this point and package these patterns into applications that we
00:02:28
can now either to the teams that are more consistently working on with with with customers on putting that into
00:02:35
their product or open source these capabilities so that some folks can use that so instead of just throwing in a
00:02:42
chatbot that is based on a large language model you're actually integrating a specific solution that
00:02:47
makes more sense to that company exactly think about the early days of programming when when people were writing code right so you have a bunch
00:02:54
of folks writing programs and then sometime I think about the ' 80s and the '90s there's this coming pattern around
00:03:00
design patterns that emerged where some folks would say hey put these things together and then it's going to be called the specific pattern so to say and then based on the design pattern you
00:03:11
somwhat create a new language and a new mechanism for people to use technology in a bit more consistent manner so it's
00:03:19
it's what we do so we try to understand what are the design patterns of AI and geni and then put that into either
00:03:26
technology and or educational artifacts for people to use so I want to get into a little bit about what AI actually is
00:03:34
cuz we talk a lot about Ai and generative Ai and all the stuff on the podcast cuz it's like the only
00:03:39
conversation happening right now and for the last year um but I think that something that confuses a lot of people
00:03:45
is the fact that you see all these companies that are saying we have ai now we have ai now and nobody really knows
00:03:51
what that means sometimes it means they added a large language model chatbot sometimes it means they added some stuff
00:03:57
under the hood that is doing a lot of work sometimes it means they're just rebranding something that wasn't really AI into AI so in your words what is AI
00:04:06
in relation to what we're seeing in the industry right now so um to me right AI is uh it's a system so to say it's a collection of tools
00:04:19
techniques science and engineering capabilities um and we get when we get to talk about generative AI I'm also
00:04:25
going to talk about it in terms of a system because I don't think it's necessarily one single thing m
00:04:30
um and it has evolved over time but if you look at that system that AI system overall um it's it's again like I said a
00:04:37
collection of tools and technologies that are really geared towards uh providing human cognitive capabilities
00:04:44
to uh computers and make it so that these computers can accelerate the processes through which we uh we produce um different things in technology so you
00:04:55
can think of AI as being a collection of planning and scheduling and sensing the
00:05:00
the world around us and understanding that world into a set of cognitive uh uh container so to say and being able to do other things out of that level of
00:05:13
understanding so AI is somewhat bringing intelligence human intelligence analog to the computers overall and so I know
00:05:21
that's a bit of a complex definition but that's that's where we are now in understanding that as a system and when
00:05:27
you try to break that down into what that means in terms of Technology then it comes in three major forms um one of
00:05:34
a major form which is a umbrella term which is AI it encompasses things like uh like I say planning sensing
00:05:40
scheduling and then processing that data that you sensed um with a a certain list
00:05:46
of tools and then these tools are usually borrowed from the mathematical worlds of uh statistics and probability
00:05:54
and then combining the collection of these tools is what we traditionally call machine learning MH um so AI is
00:06:00
bigger than machine learning and then within machine learning you have a set of tools that are mathematical statistical and whatnot and then the
00:06:07
subset of these tools which is also the essence of uh generative AI is a family of techniques called Deep learning and so deep learning is uh involved with
00:06:18
using neural networks so to say which is almost an artificial representation or analog or modeling of what the brain could possibly look like to the full extent of our understanding of it and
00:06:32
try to trying to represent essentially a data structure that would be used to process um and set of techniques that
00:06:39
would be used to process the data that is sensed just to reel it back for a sec so that people understand the difference
00:06:44
between those three can you in a couple of sentences Define the difference between machine learning or like individually what is machine learning what is deep learning and then what is um what was the third one you said AI I
00:06:58
guess AI yeah between those three can you can you define them in like two sentences each got it so with AI you
00:07:05
want the machine to do things that seem human sort of say right imagine being here and someone asks you hey David what is the color of the car in the garage you would have to do a few things you
00:07:17
would have to plan the way you would get out and get to the garage you would have to look at this artifact in the garage
00:07:23
and understand it as a car and then you would have to understand colors and then look at that and say okay the color is
00:07:29
right for example so there is this set of steps so to say that you have to carry out as a human intelligent person
00:07:36
that would say okay I'm going to plan my way out I'm going to plan my way into the garage I'm going to look at this object detect that object as a car and
00:07:43
then eventually detect the colors right so there are a few things that you do now if you were to break that so that's
00:07:50
AI so to say uh imagining that the system could do that imagine asking a robot to do the same set of tasks then
00:07:56
overall I would consider that to be AI now if you break that into some levels of deeper details the and taking out of
00:08:03
planning and scheding what are the techniques that you use possibly for navigating this ecosystem all the way up
00:08:09
until you got to the garage right what are the different techniques that you use in order to analyze that object and
00:08:17
understand that as a car and so that that set of techniques is what Ma learning is machine learning okay so
00:08:23
it's like Machine Vision and like object recognition that kind of stuff would be the machine learning Tech te that get
00:08:30
applied on top of AI that create the machine learning mechanism exactly so machine learning could be considered
00:08:36
just the the mathematical underpinning set of artifacts that you would use as a subset of AI okay right so and then deep learning is just one of these techniques
00:08:47
that so within the concept the context of machine learning there are different techniques like one of them is called uh
00:08:53
nearest neighbors it's usually preceded with a u with a K so K is for number we can say what are the four nearest neighbors to David and danu and then as
00:09:05
a matter of fact we would look at all the people that are within these buildings and then understand what are
00:09:10
the people that have a distance that is the four uh closest distance to us so those are the four nearest neighbors that's just one technique out of many techniques there's another technique
00:09:20
called support Vector machines and there are many other techniques like regression classification and so on and
00:09:25
so forth now deep learning is a set of all of these techniques that uses neural networks as a representation of the data that you would use to process in order to identify objects classify objects and
00:09:39
so on and so forth so you get AI as a bigger bucket that has other things including planning and scheduling and
00:09:45
sensing uh you get machine learning that is more focused on the mathematical and statistical and probability techniques
00:09:51
yeah okay and then you get deep learning that is just one of the application of machine learning techniques that focuses
00:09:57
more on artificial neural and then deep learning did that become something that became very popular in the vernacular because it was uh it was discovered that it was a very good way to do machine
00:10:09
learning did people try to do a bunch of different machine learning techniques but deep learning just became the most
00:10:15
useful one that is correct and when we so bringing bringing it back to your initial question which was how does how
00:10:22
do these techniques or how do these definitions relate to the current state of affairs of machine learning it's very
00:10:29
related because um machine learning has been applied for a while deep learning also for the last 10 20 um years so say so the techniques have been around but
00:10:40
then the techniques were boosted based on the Advent of a couple of capabilities and until we started
00:10:45
observing that deep learning was really doing two things one it was that it was able to process a large amount of data
00:10:53
and so the traditional machine learning techniques supervised learning and so on they would they would tend to PL to when
00:11:00
when you give it too much data so it would give you some performance and at some point it wouldn't really give you
00:11:06
more scale it doesn't scale so you start having diminishing returns so you spend a lot of computer capabilities but
00:11:12
you're not really getting good results um but with deep learning it was seen that you can one paralyze that
00:11:18
aggressively if you have a lot of computer capabilities gpus and or tpus mhm and two it wouldn't necessarily
00:11:25
Plateau that means that you can give it a lot of things going up and so what what we saw was that um the techniques have been applied
00:11:36
for the last many years but their techniques are increasingly getting better and better given the Advent of
00:11:43
additional capabilities that are supporting that increasing performance MH what is that additional capability
00:11:49
that has really tipped the scale especially in the last year uh let's go back to the last six
00:11:57
years so to say with the invention of the Transformer architecture do you want to explain what that is so the
00:12:02
Transformer architecture is is uh was created in 2017 and before that there were many other architectures within the
00:12:09
Deep learning ecosystem that were used to process data scale um one of the abilities for these systems to process text for example or sequences of data
00:12:20
things like music things like video things that have to deal with frames um was had been studied for many years right so we had sequences sequence models we had things that we called lstm
00:12:32
long short-term memory models that essentially made it possible for someone to uh process sequence data and even possibly generate sequence data but the
00:12:43
problem with those architectures was that if you have a text if you have an entire page and then you want to either
00:12:50
summarize that or analyze that then you have to put the entire thing into the model and so we started having
00:12:56
limitations with the capabilities that the machine maches themselves would have to host that amount of text in order for
00:13:03
you to ask a specific question of that text for example what is this text talking about or generate a summary of
00:13:10
this text whatnot so there were some skating issues because if you were to synthesize the entire page of text it's
00:13:16
hard it's more computationally expensive to the n+1 degree to generate or to synthesize more and more text as you add words ex exactly and it it scale it scale quadratically
00:13:29
right the other thing is that to improve the quality specifically when you have to analyze things like text you want to
00:13:35
maintain a certain say grammatical structure if uh if you're being asked a question about a sentence sometimes the
00:13:42
answer is really towards the end of a sentence but you have to maintain the context with the beginning of a sentence
00:13:48
so there was this idea of essentially um keeping or maintaining a structure of the content that you're analyzing by do
00:13:56
um applying different mechanisms and one of the mechanisms that was invented with the Transformer architecture in 2017 is
00:14:02
what we call the attention mechanism so the attention mechanism is a mechanism through which within the neural network
00:14:09
it's possible for you to maintain a structure or keep information about how let's say specific words are related within the text that you're analyzing so essentially you're coming up with a
00:14:20
mechanism through which you can an you can analyze a large amount of text while still maintaining the information about
00:14:27
how these specific tokens and or words words is just one representation or tokens are one representation of words are related within that context now it
00:14:38
gets very expensive computationally and on memory and storage to get that done and that was the challenge pre 2017 what the Transformer architecture brought um
00:14:48
about was the ability to process these large amounts of data maintain the structure that they have and it not
00:14:57
being extremely uh expensive on the hardware the storage and the computer so then it was possible by basically
00:15:04
parallelizing some of these architectures to make it possible for you to process a very large amount of
00:15:09
data build extremely scalable very very internet skill models if you had the hardware for it and then eventually being able to get some intelligence out of it and then a couple of things
00:15:20
started happening one is that you remember when I said that if you were able to uh basically break through the
00:15:28
plat Towing of diminishing returns when you start having more and more and more performance so we started seeing things
00:15:34
going this way where you you get more and more performance and eventually you get new abilities you get emerging
00:15:39
abilities out of the same models when you say emerging abilities do you mean things that we didn't expect exactly um
00:15:47
traditionally we would train what we call supervised models based on tasks and so essentially what I would be would
00:15:53
be that you would go to the model and say what's the color of this object and it would say red so that's an that's a
00:15:59
model that is trained towards understanding given an object what is the color and so the way you do that is
00:16:07
that you you give it a lot of examples that are labeled and you say this is a mug the mug is red and black this is
00:16:13
another mug the this mug is white and so on and so forth you tag it manually exactly and the next time you show it
00:16:19
some data it will give you you know mug MH um but it's it's expensive to basically train one model that can recognize mugs recogniz people also
00:16:30
answer a question and so on and so forth so being able to give multiple tasks so
00:16:35
to say to a single model that you train once was a challenge but with to make it
00:16:41
multimodal yeah multimodal has a couple yeah you can make it multitask and multimodel multimodel essentially to a
00:16:48
level of Simplicity really means that you're able to get the model to analyze images and text and audio and video at
00:16:55
the same time right and then it could it could be m to model input single output IE you you train the model to see images
00:17:02
text audio video but you're only asking it questions about text or the textto text format um that Paradigm is called a
00:17:10
picture is worth more than a thousand words so you can essentially get multiple pictures within the model get
00:17:16
it to learn from it but the way you interact with it is still in a text man okay so we started seeing benefits where
00:17:22
very very large models that had seen a lot of data coming out of the entire not the entire but a huge part of Craw
00:17:29
website for example uh a huge part of you know data that is available out there started behaving in such a way that they had this almost general purpose intelligence um they could do reasoning
00:17:43
up to a certain extent and that is tested by giving it some mathematical problems and then it would do
00:17:49
derivations so to say assuming that it had seen some of these derivations in um some mathematical books for example or writings so it would learn that structure leveraging that attention
00:18:00
mechanism and being able to derive the answer step by step and give you a specific answer and is that still
00:18:05
considered an emergent property if it was being fed different levels of ver of derivations through different text input that's a very good question so the thing that makes the thing that makes that an
00:18:17
emerging property is the fact that it's doing that in a multitask fashion so remember initially we would train one model to do one thing so if it was one model that was trained only on doing a
00:18:29
derivation over a specific mathematical problem set that would be very simple it wouldn't be considered emerging sure but
00:18:35
if you train one model that can do that on a mathematical Corpus at the same time take an an SAT exam at the same
00:18:43
time uh give you a summary of a specific piece of uh text that you give it and at
00:18:49
the same time write code at the same time optimize code and review code at the same time so those are the different kinds of emerging property is that a multitask so to say large model um is
00:19:02
able to do in your opinion is are those emergent properties kind of subsets of the attention mechanism like is that the thing that really allows it to do these kind of things one analog that um I
00:19:16
would I would give you is you know in physics for example when you have particles that are moving at a very very
00:19:24
fast pace so to say in a contained environment then you start get temperature yeah right heat and and
00:19:30
whatnot and if they move faster then you get higher and higher temperature temperature itself or heat itself is not
00:19:37
necessarily something that is uh is a is a physical artifact it's it's an emergence of that fast movement but that movement itself is very simple right
00:19:48
right so similarly the attention mechanism makes it so that uh the specific elements that you feed the
00:19:55
model get to learn about each other and so they get this interaction mode through which they basically function
00:20:01
they have this simplistic function mechanism at a very very low level yeah and there's almost this transformation this phase transition that happens where the higher level thing which is the
00:20:13
model starts giving you some of these specific behaviors in a multitask fashion has skill sets you didn't
00:20:19
anticipate it to be able to have that are based on things you did give it but you didn't realize we're connected
00:20:25
exactly a couple of other emerging properties one of my favorite is is called in context learning where
00:20:31
basically your large model now would learn from what we call demonstrations so again traditionally you would want to
00:20:37
give an input to the model and then the model will give you an answer that is a straight um uh input output relationship
00:20:43
but some of these models today you could say hey give me an answer that looks like this or here are four
00:20:49
demonstrations of the kinds of questions that I will be asking you therefore going forward from now I need you to be
00:20:56
answering these questions in this manner mhm and for some reason it's able to remember that context learn from these
00:21:02
demonstration that you gave it and then start giving you answers going forward that sounds like that and that's why uh
00:21:08
is that something we didn't expect it to be able to do exactly that's why systems like uh like chat GPT or Bard are are
00:21:15
very interesting in that sense because you can even basically tell the the system hey you are a knowledgeable
00:21:23
scientist about this field given that background start answering my question questions and then it will be giving you
00:21:30
some very interesting and then there are many ways you can get creative about that space right you can say you are a very funny and creative artist start
00:21:38
giving me answers Within These specific uh steps and the last emerging property I'm going to talk about is what we
00:21:44
called uh um Chain of Thought or or reasoning I think I I spoke about it a bit earlier where the the model or the
00:21:53
AI system is able to give you a stepbystep breakdown on on how it came up with the answer right right so that's very interesting too and that's definitely not something we expected it
00:22:04
to be able to do exactly okay so that was a lot uh I think there's a lot of answers to this question but effectively
00:22:10
it seems like AI is sort of the outer layer where you try to teach machine like human analoges to a machine and then you've got machine learning which is a subset of AI and deep learning
00:22:22
which is a subset of machine learning and then when you feed these models just these enormous amounts of data you end
00:22:28
up with these emergent properties that you're not really expecting uh we're going to get a little bit deeper into
00:22:35
those emergent properties and very philosophical next so stick around I think I'm going to go get a coffee
00:22:41
[Music] first so I think that um because large language models and chat Bots and things
00:22:58
like Dolly are sort of the only things that a lot of normal people in their everyday life have seen AI affecting is
00:23:06
there what what else is the Transformer transforming like what what uh what industries are being kind of like pulled up by Ai and what's actually driving
00:23:16
that because I think that most people just see like oh we've got chat GPT oh now this random app that I never talked
00:23:23
to has a chat bot for some reason right but like we hear all acoss every industry that every industry is being uplifted by AI so is that also Transformer based and how is that
00:23:35
working since it's not using a language model right so the Transformers started the revolution so to say right so the
00:23:42
ability to have these emerging properties and since then so that was in 2017 it's been what 6 years now mhm um
00:23:49
since then there's been a lot of evolution of that specific architecture there's been a lot of creativity around
00:23:56
you know building some of these AI systems systems generative AI systems that can generate uh images or text or
00:24:03
given some text give you some image or giving some image give you some text that's captioning and or applying this
00:24:10
paradigm shift so to say into many uh uh Industries and many applications there are two ways I would say we can look at this one is old school AI is not gone right so
00:24:25
we're still using that we're still applying some of these techniques or recommended systems uh you know when you
00:24:31
go on the website you're still being recommended some out factum things to to buy and or uh suggestions of books to
00:24:38
read and whatnot so many of these initial applications of AI are they're really really uh useful for very large companies that have their abilities and
00:24:50
this is one thing that I really like talking about very big companies that have the ability to hire hundreds of
00:24:55
Engineers so to say or dozens of Engineers highly trained highly paid that can build some of these highly
00:25:03
tuned systems that would scale to say hundreds of millions of users right for the businesses that are not the multi-billion dollar businesses we're seeing New Opportunities open up because
00:25:14
these industries can now use some of these generative AI systems in the past you needed about 7 months to 18 months to build an application with programmers
00:25:25
designers uh product managers and so on and so forth but now if you have a vision well you can go on Bard and say
00:25:32
hey this is my vision help me iterate on that give me five ideas that are related to this and then after that you can say
00:25:39
hey now write a a product requirement specifications for a system that may look like that and then you can say hey based on all of these interaction write a project plan M and you can iterate on
00:25:51
that context with the with the bot the chat bot so to say and then after that you can say hey considering these
00:25:57
artifacts or considering everything that we've talked about help me write a design document that I could use to
00:26:04
implement this app this solution and it would do that and then you can say now I need you to help me implement this in
00:26:10
Python you know designed the apis for me write the implementations of the apis for me write the system designed for me
00:26:18
it could even help you draw some of these things and so what you're seeing is that you're moving from a a life
00:26:24
cycle where you had to use about 18 months with a team of 10 to even get an idea into a good shape to probably a matter of hours to weeks working with
00:26:37
prompts and being very creative in the way you interact with that bot or as a smaller group you interact with that bot
00:26:43
yeah to come up with a solution that is pretty pretty good yeah and so what I see is that many Industries um many
00:26:50
startup and Enterprises have really really taken advantage of that I've seen good examples in media I've seen good
00:26:56
examples in in healthcare and Life Sciences I've seen good examples in financial services but in all things
00:27:02
essentially I'm seeing a lot of uh movement do you use these kind of systems in your own work to build your
00:27:08
own apis and stuff you use Bard for your own work yeah I use Bard I use Bard every day every time I have an idea
00:27:15
every time I want to process something I use B to iterate on the idea wow um I use B for uh outlines if I if I need to give a talk for example a conference um
00:27:27
usually for me the process of creating content would be based on the work that it depends on the topic but based on the
00:27:33
work that I do and based on some research I try to come up with a specific outline that really touches on
00:27:39
the points that I would like to talk about and so I use bar to create to help me create that outline and then I may
00:27:44
feel the outline myself and give it back to Bard and say hey help me summarize this and or help me extract specific
00:27:51
talking points out of this yeah uh and then I can say hey make this a bit more creative and make this a bit more you
00:27:57
know in in different types of tones so there's this that's one mode of interaction the other mode of
00:28:02
interaction is the one that I spoke about earlier which is when I have an idea rough idea say I want to create a
00:28:08
system that helps you determine what coffee you're going to drink in the morning based on prior whatever like
00:28:15
just a toy example like that and so I can formulate specific questions and interact with Bard in that way and I
00:28:22
could have a prototype before the end of the day that works that is implementing python full stack that's and back
00:28:28
everything yeah that's like a productivity explosion exactly I want to reel it back a little bit because we uh
00:28:34
we talked about AI we talked about machine learning we talked about deep learning but the big thing that's being
00:28:39
that's on everyone's mind in the last year is generative AI which you've talked about multiple times so far but
00:28:45
we didn't really Define what generative AI is and what makes it different from those uh other forms of AI so can you
00:28:52
give a quick explanation of what generative AI actually is remember we talked talked about AI overall being a
00:28:59
system not just one thing uh so and machine learning being a set of techniques that are more mathematical in nature deep learning being one of these techniques that focuses a bit more on
00:29:10
neuron networks so by virtue of of getting something that is a lot more fundamental generative AI is a deep learning technique so it's still using
00:29:21
the Deep learning Technologies but generative AI is really focus on generating or creating a specific
00:29:28
artifact and so that artifact could be an image it could be a piece of text or it could be a piece of audio or it could be something else so yeah that's that's a very simplistic definition of what
00:29:39
generative AI is and what what is the foundation of generative AI like what allows that to work because we see
00:29:45
things like generative fill in Photoshop we see generated music now like the every single creative industry and
00:29:52
non-creative Industry is being sort of upended by this generated content what is allowing systems to actually generate content instead of just classifying content yeah so that's a beautiful
00:30:04
question in a sense that it there's a very very strong common denominator among all of these things and that's the
00:30:09
Transformer architecture we spoke about earlier right so what we've s what we've seen is that applying the same technique
00:30:17
and then changing the question a little bit gives you exactly uh content that is
00:30:22
generated that that you're interested in for example we can say using these lower
00:30:28
Transformer architecture help me generate an image you can give um the the generative AI problem as given images of different artifacts
00:30:39
like animals like cats and dogs or whatnot create something that looks like some of these things uh using I don't
00:30:46
know interpolation or extrapolation different techniques and make it look like the family of things that I've
00:30:52
shown you in the past and it will give you something that doesn't exist in real life maybe the image very high fidelity
00:30:58
image of a dog or cat that doesn't exist in real life but really really looks like the samples of the things that
00:31:04
you've shown it in the past so the ability for these models to essentially create uh content in different
00:31:11
modalities is to generative ability yeah so we think about like large language models being fed into a Transformer
00:31:17
right and that's just like give me all of the text that has ever been written on the internet and we can develop relationships between words but when
00:31:24
you're when you're generating an image or you're generating audio what is being fed into the transformer in that way
00:31:31
right because we we see uh you know there's um a lot of genetics work that's being worked on with Transformers too
00:31:38
what kind of data do you feed into Transformers to actually make that work in a variety of different fields right
00:31:45
so in general you would give it today and text was very easy easier to acquire that's why you hear of large language models today a lot more right and I and
00:31:56
the result also so from generating text were a lot more impressive and exciting to look at that's why in my opinion that
00:32:02
field somewhat took over but you're right so there are you could consider the input to be pretty much anything
00:32:08
that could be put into a sequence you a video for example is a sequence of frames right so you could give multiple
00:32:15
videos uh broken down into frames uh to uh a Transformer based architecture and it gets a bit more complex in the way those sequences of process or structure is maintained the many techniques around attention
00:32:29
mechanism and so on and so forth but let's consider that to be a black box and then it knows how to do that then
00:32:35
what you give it is a set of frames which are videos so to say and then you say give me something that looks like
00:32:41
that so in that sense you've given it videos of or of set of frames um you could also have a mechanism through which you give it videos and text which
00:32:51
we do today there is this um encoding uh model that is called clip essentially putting together images and videos I mean and text and which is the foundations of do and a lot of AI image
00:33:04
generation at least the foundational technique of U of these kind of abilities where you you you teach the
00:33:11
model to recognize images and text together as a as a joint uh entity so to say and the process through which you do that is by getting the image is processed with what we call tokenizer
00:33:23
and or encoder specific to an image and that turn turns that into a vector we call that an embedding and then you do
00:33:30
you go through the same kind of process with the text where you turn the text into a vector and then once you have
00:33:36
these two vectors you can then combine them with basically algebra and then at a higher level you have the task and
00:33:42
Order question that you want the model to answer in one scenario you could you would want the model to say for example
00:33:48
given an image explain the content of this image for me or you may have the reverse problem which is given a text
00:33:55
generate an image that contains the the information so to say that I've provided in this text which is the business of um
00:34:03
Mid Journey so as a kind of to break that down you're depending on the field that you're trying to use Transformers on you are turning data into numbers and
00:34:15
you're comparing those numbers to each other and then getting an output correct so because you're able to take video or
00:34:22
images or text and vectorize them and turn them into tokens you can compare them to each other even though they're different types of media correct correct that that is that is excellent
00:34:33
and the the one of the things that makes it really work beautifully is because once you take the images or video or
00:34:41
audio you incode that into an initial Vector that that process is called tokenization M then once you get the
00:34:48
tokens and by the way the tokens can be a bit more complex for example the tokenizers could learn to not just use a
00:34:54
word per token mapping but it could also split words into two or three if that word has a bit more multiple meanings and complexities or if it finds it effective so Sub tokenization sub
00:35:05
tokenization so you may have a situation where uh a five-word uh sentence gives you 12 or 15
00:35:13
tokens or maybe less so it's a matter it's a the concept is more about information preservation within the subst structure that is a vector rather than a onetoone mapping between the
00:35:24
words and and and the vector right same thing with an image an image is a two-dimensional structure which has a
00:35:30
third dimension of red green blue right right so if you flatten that entire thing into a pixel intensity over that
00:35:38
entire two times times three so to say Dimension then then you get a larger Vector but that's just a simplistic
00:35:45
tokenization where you say hey I'm going to flatten an image flatten that more by reging blue and then after that I'm
00:35:52
going to have a vector representing the pixel from there you can have a deeper tokenization that may consider the structure for example the adjacency of object or the distance between objects
00:36:04
or even some deeper level of understanding of the objects within that image at the end of the day you go from
00:36:10
a piece of artifacts like audio and in anodia you use spectrograms and and you turn that into specific order artifacts so you you go from an asset to a vector
00:36:23
mhm now there's this other step called embedding which is basically doing a projector of that a projection of that
00:36:30
Vector onto a vector space that is shared by every other piece of artifacts I mean other piece of data in that space
00:36:36
it's like a normalization like a normalization but then by that projection what you essentially do
00:36:41
especially if you have a multimodal model like if you work with an image and if you work with Tex for example then
00:36:47
you tokenize them each which is a onetoone relationship between an image and the text and the tokenizer that works for them and once you have these
00:36:54
two vectors you do that projection onto that shared Vector space so to say and the beautiful thing about that is that and you do that through training the beautiful thing about that is that once
00:37:05
you land these things within the same space they become of the same nature right so you can start comparing them yeah so you can start assigning relations and making uh having
00:37:16
statements like a car c a r written in text form compared to the image of a car compared to the image of a car so these vectors will be closed in location so a
00:37:26
Rosetta Stone you're taking you're taking one language and another language and you're sharing them in a certain way
00:37:32
and then once you have this shared say you translate them all to Latin then you can do whatever you want from there and
00:37:38
the common the common substrate of all of these different things assumption at least is that there's information that
00:37:44
is preserved in these different types of artifacts so you're almost doing an information extraction exercise right
00:37:50
describe that what do you mean by information so it may be a longer conversation but at the end of the day
00:37:56
at the end of the day so information and I know you had a whole uh uh video about
00:38:02
the nature of information it could be contextualized to the piece of artifacts that you're working uh you're working
00:38:08
with but in a very very simple manner information is this uh entity or this thing that could give you it's hard to Define information without using information yeah yeah you have to it's
00:38:21
this thing that can give you a bit of a pattern right so and we usually base that pattern on uh the notion of order
00:38:28
disorder Symmetry and so on and so forth yeah but if you have something that can
00:38:34
um give you a pattern about in indifference and or disorder about a specific subsystem then you start having
00:38:40
information for example if I do this nothing has changed very much so if you were on the receptive end of that pattern you won't really get much
00:38:52
information but if I do there's a difference in what I did before and what I'm doing now sure now you may not understand why I'm doing
00:39:03
that but you would understand that there's a difference between what I the way I tapped the frequency at which I
00:39:09
tapped my hands before and the frequency at which I tapped it after then you've gained information yeah so it's the same
00:39:15
way that you may understand some differences within an image for example looking at a contour and then something changed between this and this then you may realize that these may be two
00:39:26
different object and so on and so forth and within text as well you may have difference maybe between words or between paragraphs and between different structures so you have some form of
00:39:36
information and the beautiful thing about information is that it could be combined so it's the evolution of
00:39:42
information is what you're maintaining it's the extraction of information and or differences in patterns within
00:39:48
different modalities of data artifact but the beautiful thing about that is that it could be combined at a certain
00:39:54
level or compared that's what makes it possible for you to essentially extract information out of an image by
00:40:00
understanding how different it is or how many different patterns exist within that image yeah and extracting
00:40:05
information out of a piece of text by understanding how many different patterns exist within that text and then putting that together in a normalized
00:40:13
space through which you can start comparing them and then reversing that you can now combine text to images and
00:40:18
basically have that relation maintained with all of that combined is that would you say would that be the fundamentals of like a general AI that could do everything we're we're getting into the
00:40:30
realm of AGI I would love your opinion on that if you feel comfortable talking about of
00:40:36
course of course so um what is intelligence according to you according to me this is such a big question um I've thought about this a
00:40:48
lot my personal opinion on this at this point is uh well for the listeners we're
00:40:54
going to Define AGI really quickly AI is artificial general intelligence effectively meaning you can ask an AI to
00:41:02
do anything that a human could be able to do or possibly even more right and it could be able to help you with that would you agree that that's the definition or do you have an expanded definition that's that's somewhat why
00:41:15
I'm I'm um asking the question of what is intelligence because agreeing on AGI being artificial general intelligence assumes that we agree on what intelligence sure okay my my definition
00:41:29
of intelligence would be wow thanks uh the ability to synthesize information and create uh create new actions based on
00:41:45
information that you weren't explicitly told to do that'd be probably my definition of intelligence that's a
00:41:51
that's a decent definition um would you disagree that the context in which you have have to do that specific workflow
00:41:58
that you define has to be defined at e I.E you have to do it within the context of I don't know literature or uh
00:42:06
robotics Automation in a sub field for example having a robot that can control a specific arm either for surgery and it
00:42:13
would be a different thing if that robot controlled an arm say in a restaurant and so on and so forth I think that I
00:42:20
think that when we talk about the generalization of intelligence or even information
00:42:26
we're making a bold claim that goes beyond what we understand so far about the nature of these things uhuh right so
00:42:34
I I see so if I want to break down the problem of AGI uh again I might have already expressed that I'm not a very big fan of that definition because I don't really think we know exactly what we mean when we say that sure um but if
00:42:46
we want to get into a practical uh realm I think that may be possible to essentially and which is the the state in which we are now Now by getting these models to progress in their ability to
00:43:01
impact the world as well so we we discussed the software version of the AI so far which is you give it data it could recognize it or at this point it can also generate data sure but what is
00:43:13
a software real world interaction mode at this point right so we have many systems for example in healthcare and life sciences that have to deal with the real world in the way that say a
00:43:23
hospital equipment functions or in the way that um a robotic arm that controls cameras functions so you get many other things about a real world that may have to do with intelligence so I think a lot
00:43:37
of the work that we're doing on improving the quality of these AI systems has to bring things all the way
00:43:44
up to these definitions of AI that I mentioned earlier which involve in and include planning scheduling and acting
00:43:50
and sensing as well so when you start augmenting these systems with these additional capabilities is and you start
00:43:57
training agents that are able to plan and schedule and act on in the real world then you get that sense of a gii
00:44:04
that closer to the definition you gave it right now the we the ability to do that at that level at that scale gets challenged by where are you sensing what kind of information and also where are
00:44:19
you acting in which kind of world environments right and if you want to look at the real world in which we
00:44:25
operate and you want to look at all the types of interactions and actions that can happen it the the number of
00:44:31
possibilities is larger than the number of atoms in the universe right and so how would you have a generally
00:44:38
intelligent system that knows how to act at this entire world I find that quite a challenging thing to believe but if you
00:44:44
constrain the problem if you make the problem simpler as simple as I want to have a generally intelligent system that would learn how to use all the hospital equipments within the hospital hospital
00:44:56
system then maybe you have the opportunity to have an AGI system that can essentially take in the task and execute that effectively so that is my
00:45:08
um uh techno Optimist view of the possibilities of AGI by training agents that have World represent representations but these are simpler worlds representations that are
00:45:19
constrained by the problem space in which you want these systems to operate right and then being able to plan
00:45:25
schedule sense and act including the other type of capabilities that they can do okay interesting so danu doesn't
00:45:33
think that we're going to have this one omniscient AGI artificial general intelligence that's going to be handling
00:45:40
everything but he rather thinks that we're going to have these smaller more specialized AIS that kind of handle
00:45:46
different tasks and help us do stuff a lot faster this is actually not that different from that whole conversation around the Tesla bot right like where
00:45:53
you can have a robot that's like a human that does human tests or you can have a bunch of really small robots that handle
00:45:59
the test that we already do on a daily basis kind of the same thing pretty interesting um in the next segment we're
00:46:05
going to get into the problem of AI hallucinating which is where it just makes up a ton of random stuff and uh
00:46:13
that's clearly a problem I was very curious about that so that'll be a fun conversation plus we need to see how
00:46:19
fast danu can type so uh get ready for that [Music] uh Lis wanted to hop in and ask a question real quick yeah sorry I really liked what you said about um defining
00:46:43
intelligence as it pertains to AGI and I thought David brought up a really important kind of intelligence like intuition and deduction and the ability to uh EXT ex ract not just pieces of
00:46:57
information but threads and systems of information from multiple kinds of contexts but there's lots of other kinds
00:47:05
of intelligence that people like cognitive scientists like to Define and classify things like spatial reasoning
00:47:12
um things like uh engaging in dialectical thinking um and these are all intelligences that we've observed in ourselves and so when we think about
00:47:23
sort of a general purpose Swiss Army knife AI do you think that we should be limiting that to the kinds of tasks that our brains do on a daily basis or do you
00:47:35
see that there's going to be almost like new methods of thinking and new cognitive strengths that emerge as uh
00:47:43
these neural networks get stronger that's that's a super interesting question I for in a practical sense um
00:47:53
I'm actually with you David on that definition right because I think that that's the form of intelligence that
00:48:00
could be mechanistically or mechanically implemented in a piece of software as in
00:48:07
program right so by our own intuition we can think about doing things like that by breaking that down into steps that
00:48:14
kind of um intelligence that you're talking about to me it's a bit more like that emergence right like that emerging
00:48:22
ability that and I don't think we've gotten to the point where we can perceive what those are yeah or
00:48:29
intuitively like a new color yeah intuitively know what exactly we need to do in order for the models to have that
00:48:38
spatial awareness or that other kind of ability now we can program that by having segmentation models and having distance calculations and then coming up with some mathematical theistic through
00:48:49
which we can claim that we've achieved that capability but I would argue that the way we learn us is not exactly the way we teach the machines how to do that right so there
00:49:02
there's definitely a lot of a lot more research and we may stumble upon you know we may basically strike luck and
00:49:08
then find out that other kinds of scaling mechanisms or the way it works in physics today is that you have
00:49:14
smaller systems you have a simple interaction mode like magnetization or just Collision analysis or the different
00:49:21
forces that we're working with about four of them and based on these simple interaction mode you get the entire
00:49:27
universe the way we know it at least it's the current theory yeah fundamentals of physics yeah at least it's it's the way we we know but there
00:49:34
may be another way right we may have just sensed it in our own apparatus of sensing in that kind of form and we're
00:49:40
able to explain it the way we explain it but it's still a projection on on the on the screen that we're looking at and
00:49:45
doing our analysis so I'm I'm super excited about the possibility of us finding out more cognitive routes so to say in the way these systems learn and
00:49:57
right now the best tools that we have essentially in our laboratory of AI are these deep learning tools the Transformers and there are many other architectures that are being built
00:50:07
around that things like uh memory orware neuron Network architectures or things like the abilities to pull from a vector store and augment the knowledge uh with
00:50:18
the retrieval augmented generation capability so I feel like the more we add interaction mode and information
00:50:26
uh uh retrieval and use um utilization capabilities Within These models the more possibilities we have to have this additional emerging capability that is a
00:50:37
lot more cognitive than the mechanic way we've been doing things so I think that's an open question I think it's a
00:50:43
beautiful question and I hope we get lucky In Our Lifetime to find a way to get that done yeah me too we've stumbled
00:50:49
upon a lot of random stuff in science so there's definitely a possibility that we have that happen which would be big
00:50:56
I think it's Richard fan that say that science is the belief in the ignorance of the expert so I I think that if we if
00:51:03
we really take it as a as a basic principle that we could stumble upon some things and then we we believe that
00:51:09
whatever we know so far is may may or may not be way then we have an opportunity to really in incorporate new
00:51:16
information and all knowledge that can get us faster and further yeah I want to pull this back uh a little bit back to some practical stuff again um a big
00:51:27
philosophical yeah no I love the philosophical conversation I love the philosophical um I think that one thing
00:51:32
that people think about when they think about generative AI uh is the problem of
00:51:38
hallucinations and uh for people that don't know hallucinating is basically when you generate something that just
00:51:45
isn't right or isn't true um in large language models you can ask a question and it will confidently lie to you
00:51:52
sometimes and how do you look at how we're we're going to solve that problem because it seems like part of generative
00:52:00
Ai and part of large language models in general is that it's just it's parting information based on probabilities and those probabilities are not always going to be correct so
00:52:12
you're I'm assuming working on ways to make these AIS more accurate um accuracy is obviously going to be a major a major problem and something that we need to solve over the next couple of years how
00:52:24
do you look at solving the hallucination problem the the problem of hallucination
00:52:29
so the way these models work now is that you give it a lot of data and the question you're really asking of it is
00:52:35
um give me let's simplify a token to a word and working within the text domain give me the next word based on these word that I gave you right so if you if
00:52:46
the if you qualify the problem as write a novel for me or write a paragraph or write a summary of something then traditionally what would happen is that you would give it the beginning of uh uh
00:53:01
a sentence and you would say complete this sentence for me and so it's that sentence completion so to say that is
00:53:07
based on probability and even basing that on probability in within the context of this conversation is the
00:53:13
simplification so there's there's a lot more going on but the basic principles of how it works is that it would let's
00:53:20
assume that it works off of the most probable word to follow that word that existed
00:53:25
and then taking that longer sentence as an input figuring out what is the most probable word that could follow and so
00:53:32
on and so forth what that means if you simplify the problem just at that level and if I say give me a complete the
00:53:39
sentence doctor something works at John Hopkins or something like that then it would just put a name there right right so the the the question you haven't
00:53:51
asked is make sure that that name is an existing human being that is really a doctor at John Hopkins and whatnot right so fundamentally it's a different question
00:54:05
to ask off of that system and then we're back on the reason I call these things a system in the beginning is because yes
00:54:11
you may have a model that gives you the next word prediction or the next token prediction but then you still need to do
00:54:17
a lot more work on top of that input and that output and even at processing sometimes to make sure that the output
00:54:23
and the response that you get out of it is a truthful one or real one right or a
00:54:29
less toxic one if the answer is toxic and you don't want to serve toxicity to your users so there are many
00:54:35
pre-processing and post-processing activities that need to happen one to uh uh make sure that the context I mean the the answer of the model is grounded we call that concept grounding grounded in
00:54:47
reality MH and the second is to make sure that the cont the output of that model um goes by your certain set of
00:54:55
responsible AI principles right so those are two things but fundamentally the way the science works is that it would give
00:55:01
you something whether that thing is true or not sure it's it's about it's your job to make sure that that thing becomes
00:55:08
true and so the way that happens then now is that you need to associate that response to basically a source of truth right what is truth yeah yeah what what
00:55:20
is truth what is reality and that's that's another thing where you another another reason why you probably want to
00:55:27
contain and contextualize that use case sort to say down to a source of truth right uh give me Dr blah that works at John Hopkins then you need to probably have a database of all the doctors that
00:55:38
work in that hospital and make sure that after you get the name of a doctor because the model will give you that you
00:55:44
check that against that database and if that person doesn't exist or you can say fill this specific spot off of the names
00:55:51
the list of names and the database and constraint and constraint so so that's why Bard now has that Google button
00:55:57
where you can ask a question and then you can double check it that's another context that's another mechanism for for
00:56:05
that but that's not exactly why it has that it has that button um the just to just to land on the concept on the concept of hallucination so it was named hallucination because it could give you
00:56:17
some some answers that seem real but they're not necessarily real but this is her normal functioning mode of these
00:56:24
Technologies um the reason it took us a while to release Bart for example was not because
00:56:29
well we invented the Transformers so we've known how to do this thing for a long time yeah but it's it's all of the
00:56:36
additional Tech set of technologies that we had to build and principles that we had to really build around the behaviors
00:56:41
of a model that really get us to you know one the requirements of build addition building additional Technologies and then two the challenge around making these Technologies
00:56:52
deterministic in the sense that you always want a specific answer so you have to do a lot more evaluations you
00:56:59
have to do a lot more checks and balances you have to add the number of metrics like is this model answering a
00:57:05
question when it doesn't know the answer you probably want to codify that into something that gets checked yeah and so
00:57:10
on and so forth so there's been a lot of work that we've done on one uh really having clear and concise responsible AIS
00:57:18
um principles and then two turning those into Technologies and or checking mechanisms that could work in
00:57:24
conjunction with the creation the operation uh and the operation of a model and then three U making sure that
00:57:32
these cores and and outputs of checks are available so that that technology could be used on the cloud ecosystem for
00:57:37
example as part of platform so we work with research to understand what are these responsible AI principles that
00:57:43
could be turned into metrics and guard rails and so on those get turned into product capabilities that work alongside our models and then these models are exposed or are commercialized so say on
00:57:55
our Cloud platform this product called vertex Ai and you can go find it out on on on you know cloud. Google right so
00:58:02
that's that's how we we're essentially fighting the problem of uh of hallucination there's a lot more work
00:58:08
going on in that space mhm okay well I think I'm going to close it out here soon but I um I want to end with asking
00:58:14
if you think that there's anything that we missed anything that people would gain a lot from hearing about that they just are not hearing in popular media that's very important to the whole AI
00:58:27
story uh two things maybe one is the consumer applications of AI so bar chat GPT are very popular now so which is something that I'm I'm happy about because I think that it's really
00:58:41
bringing the conversation closer and closer to everyone and you and I have been working tech for a while so we may
00:58:47
have been aware of that coming up and coming together but I think it's a massive opportunity that today um people
00:58:54
that are um news editors or writers or artists or folks that work in different domains uh can use some of these things to help them write better to help them
00:59:07
um generate images that they can use as part of a Content that they produce and create to write better letters to write
00:59:13
better to do homework and and so on and so forth so I really love the the consumer application but one of the
00:59:19
things that I don't think get talked about a lot is the developer experience um um and also the way the barrier of
00:59:27
entry from creativity and product generation product creation standpoint it's getting really really lower with these set of Technologies and so I I I
00:59:41
really think that we are at the cusp of a new form of economy where creation uh of valuable items of
00:59:53
different kinds of forms would not just be a matter of a few being able to do that because they have a highly a high training and they've they spent years doing I don't know an undergrad in
01:00:06
computer science and so on and so forth but if you bring that level of assistive
01:00:11
creativity abilities to the masses so to say yeah I found that people have ideas
01:00:19
right like people are creative if you sit down and you tell someone let me take away the problem of knowing how to
01:00:25
implement these ideas as talk about your ideas you get many ideas to start emerging so I think that we're really at
01:00:33
the border of a transformation where the economy may take a different form if different people without the need to really understand in details how to implement some of these ideas are able
01:00:45
to one iterate on the ideas with the assistance of generative AI to validate some of these ideas with our ability to prototype those in a matter of hours rather than years and then three
01:00:58
test these ideas in the ecosystem and maybe find value for different people that they could commercialize these ideas for so I'm very optimistic about the possibilities of this in the future
01:01:09
all right well the last thing we're going to do we have a little game here that we play when we bring guests on
01:01:14
where we figure out how quickly they can type the alphabet it's a running scoreboard um
01:01:22
you can use either the MacBook keyboard you can useo thing yeah it's a keyboard test get yes okay so you'll take that so you get I'm going to ask the AI to TP
01:01:34
type this thing for me so you get three chances um wait what is the most optimized way of typing the whole alphabet if you as soon as you start typing it starts uh so as soon as
01:01:49
you type the letter A it'll start and does he have to hit enter at the end no you don't have to hit enter as soon as you hit Z it'll finish got it and and uh
01:01:57
no typos is allow so if you miss a letter like let's say you Miss B and go on to C it will not count you have to hit every single letter and you'll see at the top where it say type A okay um
01:02:13
that will tell you the letter you're supposed to do we give people tests at all or is it just three chances there's three total chances okay
01:02:23
ready you got to hit G oh it's harder than it looks definitely a lot harder than it looks that's okay you got to hit J this is why you get three chances don't worry about it I was extremely
01:02:42
[Music] slow okay so first run 26 seconds now just hit reset can I change a keyboard yeah you can change keyboard all right so now I
01:02:59
understand why there are options yeah so we have we have mechanical keyboard we also have the butterfly keyboard that um
01:03:06
Apple sells let me know I'm going to get a ke all right so I'll go mechanical you a mechanical let's do it set us up
01:03:12
however you want round two go for it fight nice okay 26 to9 yeah 26 to 9.8 much better that's a big come up much better last try last try but you guys aren't impressed that means that I'm not I'm
01:03:37
not that is not bad I was not far I was not far in front nine is is actually really good especially for a second ATT
01:03:43
we've seen some things in here that you would not believe I'll show you the scoreboard
01:03:48
after this and you'll be okay ready ready go [Music] nice 8.73 not bad honestly not bad okay
01:04:06
where where is that on the leaderboard David so here's the leaderboard fastest Tom Scott 3.5
01:04:13
seconds insane that was crazy to watch it was just um wow so let's see 8.73 is right above Brandon wow wow all right actually no faster than David
01:04:26
Blaine too you beat David bla you beat David blae he might be a magician but you're a
01:04:33
magician on the keyboard wow uh oh 8.7 73 right you also beat Hassan Minaj hey Hassan so you beat Hanan Minaj David bla and Brandon nice cool all right well
01:04:52
thank you again thanks for having thank you for coming um where can people find you on the
01:04:58
internet uh well I'm Dean banga on X now okay nowadays and I'm LinkedIn on LinkedIn as well so as Dan banga essentially awesome we'll link that in
01:05:09
description and uh do you want to shout out your any projects that you're finishing up right now or working on right now that Google can people can see
01:05:16
it Google so the the vertex AI platform is really the platform that I'm working on right so that that's what we we put out Solutions on and uh I say I would
01:05:28
say that look forward to many other more industri SL domain adapted uh uh capabilities around llms because I think
01:05:39
that large models are a big thing and I think it requires a lot of additional Technologies to actually make it work in
01:05:45
in applications and I think that this is the about the time where we need to come up with things like design patterns
01:05:51
right so if you think about a gang of four for example it's a book that was needed when programming needed some kind
01:05:57
of structure so I think we are at a place in time now where we need some kind of structure and how we build and
01:06:03
deploy large application large models in in in Enterprise environments and that's something that I'm working on awesome
01:06:09
sweet well everyone uh watching listening at home if you were surprised that we had an episode today don't worry
01:06:14
we have normal episode coming on Friday this was just a little extra story for you so uh hope you enjoyed it and we'll
01:06:20
see you on Friday Cheers peace [Music] n