00:00:01
we're speaking today with Justin got Schlick who leads the Machine programming research team at Intel labs
00:00:08
the newly formed research group focuses on the promise of machine programming which is a fusion of machine learning
00:00:16
formal methods programming languages compilers and computer systems Justin welcome to knowledge at wharton thank
00:00:24
you so much for being with us here today wonderful thank you for having me McCool
00:00:28
so I just read that description of what machine programming is but I think that you know given all the buzz around AI a
00:00:37
lot of people are familiar with machine learning but most of them like me don't
00:00:42
have a clue what machine programming means perhaps you could explain the difference between the two certainly
00:00:48
yeah so at the very highest level machine learning can be considered within the subset of artificial
00:00:55
intelligence and there's many different types of machine learning techniques one
00:01:01
of the most prominent right now are these things called deep neural networks and that's a lot of what people are
00:01:08
using to make the tremendous progress that we're seeing over the last decade
00:01:11
the machine programming is really the idea that we are trying to automate the development and maintenance of software
00:01:19
so the fundamental difference between the two is that with machine programming you can kind of think of all of the
00:01:28
field of machine learning being a subset of the field of machine programming but
00:01:34
in addition to using machine learning techniques which are these approximate types of solutions we'll also use other
00:01:40
things like formal program synthesis techniques that will give us mathematically proven correct pieces of
00:01:48
software and then between those two points you can kind of think of those as a spectrum you have the approximate
00:01:54
solutions here and the precise solutions here and then there's a fusion of a
00:01:58
number of different ways that you can combine these and every one of these things essentially is a part of the
00:02:04
bigger landscape of machine programming so if I understood you right machine programming is when you create software
00:02:14
that that can create more software right how would that happen and I wonder if you could give us a couple of examples
00:02:21
just to help our audience understand this yeah so the idea of creating software that creates its own software
00:02:31
is really the core idea between or inside of machine programming and a couple examples of that is we recently
00:02:39
built a system using genetic algorithms and what it does is it'll allow you to
00:02:45
take certain input output examples and then by running through a number of iterations we call them evolutions in
00:02:52
the genic algorithm space it will then automatically synthesize the program that will match the input and output you
00:03:01
do this sort of in the training phase and then it will take new input-output examples that it's never seen before and
00:03:07
then generate these new particular types of programs so that's like one example
00:03:12
that you might have in the space of machine programming and you would think about the impact that machine
00:03:19
programming might have on different industries which industries do you think are likely to be affected most by this
00:03:26
and over what period of time yeah this so this is a fantastic question and one that could require a very long response
00:03:36
but I'll try to keep it slightly abbreviated at the highest level one could imagine that any of the industries
00:03:42
that are predominantly based in software are going to benefit this benefit from this tremendously there was a recent
00:03:48
survey that was done in earlier 2019 that showed I think we have something like half a million computer scientist
00:03:58
positions that are open so these are programming positions in industry that we need to fill but we're only producing
00:04:05
roughly 10% of the actual programmers to fill those roles so what we're having in
00:04:13
the software industry is essentially a bottleneck of supply if we can start to automate some of the simple tasks you
00:04:23
know reading in a file parsing data helping us automate that software development testing this will I
00:04:31
think tremendously accelerates the rate at which software is being developed so I would say that that's probably the
00:04:38
first very obvious place the other area that I think is going to be impacted tremendously by this are autonomous
00:04:45
systems in most of the spaces in autonomous systems a core ingredient of those systems is software so for example
00:04:54
if one were to think about autonomous vehicles a large part of what's holding
00:04:59
us back from getting to level 4 level 5 autonomy which is the point where the car can essentially handle all of the
00:05:07
nuance behaviors of driving in downtown Philly or something of that nature a big
00:05:11
bottleneck of that is really the implementation and the algorithms of the machine learning systems if we can
00:05:19
automatically construct those these autonomous systems will also accelerate in their advancement I'd like to come
00:05:27
back in a little bit to the question about autonomous vehicles and the impact on the auto industry but given the fact
00:05:33
that a lot of our audience is in the financial services industry I wonder if we could go a little deep into that so I
00:05:41
mean I know that AI has made quite a significant impact for example in areas like fraud detection right do you think
00:05:48
machine programming can also have a major impact there and if so what what might that look like absolutely this is
00:05:56
a fantastic question and as I was mentioning earlier with machine learning systems their foundation is essentially
00:06:04
learning through statistical analysis in some sense this makes them probabilistic
00:06:08
that they are getting us close to the the right answer but there's in many cases we're not guaranteed that we'll
00:06:18
have the right answer when we think about machine programming we go back to that spectrum of how we have
00:06:23
probabilistic to very precise solutions it's my belief that for example in the
00:06:29
financial sector there are certain cases where probabilistic solutions aren't
00:06:33
sufficient so one might speculate that as you're doing some sort of financial
00:06:38
transaction having the probability that oh i sort of been rounding off the sense right
00:06:43
you know i'm close enough probably isn't sufficient if those transactions are
00:06:47
happening billions of times a day in that case we need a more precise solution and this is one of the areas
00:06:54
that i think you could use machine programming an interesting coming back to the other industry that you mentioned
00:07:01
and the auto industry what kind of impact do you think the machine programming will have on the whole drive
00:07:09
towards autonomy you i think you started talking about that in the context of philadelphia yeah but i wonder if you
00:07:15
could go a little more deep into what that might look like for sure so as i was mentioning earlier
00:07:21
we recently built a system that is using this dis genetic algorithm to automatically construct programs what i
00:07:29
didn't mention is one of the pieces that's part of genetic algorithm just
00:07:34
this thing called a fitness function the fitness function is essentially you can
00:07:38
think of it as a way that you grade the accuracy of the programs or the results that the genetic algorithm is giving so
00:07:45
genic algorithm produces a result and the fitness function says you get a b or you get an A historically though fitness
00:07:53
functions have been written by humans and not just any human really expert machine learning humans oftentimes what
00:08:01
we find is the complexity of the problem you're trying to solve is directly related to the complexity of the fitness
00:08:09
function so one could naturally infer then why would you write the fitness function just solve the problem yourself
00:08:15
so we took a look at this and what we did is we figured out a way using machine learning that we could
00:08:21
automatically create the fitness function without a human involved so now going back to your question if you think
00:08:29
about this type of thing in the autonomous vehicle space one of the things that's holding us back is the
00:08:34
advancement of ML systems and historically the advancements that we've had with ml systems have been through
00:08:40
humans creating them but if we use machine programming one could imagine like we have with the genetic algorithm
00:08:47
solution that the machine can actually start to invent it's own machine learning systems that
00:08:54
will then accelerate the progress of these autonomous systems so what are the implications of that well one of the
00:09:01
things that I've heard about that's holding back autonomous systems as you
00:09:06
are the autonomous vehicles is the fact that it might be too late to too far for
00:09:17
the system to make a certain decision yeah and and you because you don't want
00:09:23
to actually hit something right you you you you probably need software that can predict what's about to happen before it
00:09:30
actually happens is that one of the issues right here absolutely right so this is tremendous insight and and
00:09:37
actually we had a nurbs paper nerps is one of the leading research conferences in machine learning we had a nurbs paper
00:09:45
in 2018 that tried to start to address this problem so historically what you're
00:09:50
describing here is the space of anomaly detection so the Antonidas vehicle space when we
00:09:56
think about these various behaviors we think this is an anomaly and in particular that's a time series anomaly
00:10:02
so for example you're trying to prevent this vehicle from colliding with this
00:10:06
other vehicle or make sure it doesn't hit a pedestrian and as you pointed out
00:10:09
it's too late if you already have the event happened to you know detect it so
00:10:15
what we did in order to address this is we recreated the mathematical foundation
00:10:22
for anomaly detection specifically for time series and so with this now what our hope is is that the community will
00:10:29
adopt the new mathematical foundation we've created and then they can apply this for time series and Omni detectors
00:10:35
which will start to address those types of problems so and machine programming helps at all this absolutely so in the
00:10:42
context of autonomous vehicles you could think of using this mathematical foundation to better predict these
00:10:47
anomalies but when you think about machine programming or you think perming in general many of the problems we're
00:10:53
seeing today with software is we have these correctness bugs we have security bugs we have these privacy violations
00:11:02
all of these things in some sense are time-series in nature program is really just a sequence of
00:11:09
instructions one after another so if you take that mathematical foundation you can also apply it in the space of
00:11:16
machine programming which is exactly what we're doing well one thing I'm
00:11:21
curious about is that machine programming like a lot of other areas of AI has been around since the 1950s right
00:11:27
what's behind you know the sudden interest in machine programming now why is it picking up in such a big way and
00:11:36
why is Intel you know so interested in sort of investing it in in it in a big way this is a fantastic question and
00:11:46
I'll try to break it down to those two pieces first address why we're seeing
00:11:51
the you know the resurgence of this because it has been around since the 1950s and then why is Intel so
00:11:58
interested if we look at why it's taking off today I would say principally there's two reasons the first is I
00:12:05
believe we're at an inflection point and the second is I believe my colleagues
00:12:10
and I at Intel Labs and at MIT we've made an important sort of observation and how to think about the future of
00:12:17
machine programming so as far as the inflection point goes we believe there's
00:12:21
really or my view is really that there's three things that have created this the
00:12:26
first is we have tremendous advances in algorithms in machine learning and in formal methods things that didn't exist
00:12:35
say 12 months ago potentially are fundamental to the advancement of machine programming the second is we
00:12:45
have tremendous advances in compute today as the recent Turing Award winners Dave Patterson and John Hennessy how are
00:12:53
pointing out is we're living in sort of what they call the Golden Age of computing which they refer to as they
00:13:00
call them domain-specific architectures that for a long time it was really just the CPU but now we're seeing based on
00:13:07
the advances were having a machine learning in other areas we have these accelerators that are specific to these
00:13:12
domains and so it's creating a tremendous opportunity for acceleration of machine learning and formal methods that
00:13:20
wasn't possible and then the third piece is the abundance of big and dense data
00:13:26
for example there is a repository that's called github and github basically as a
00:13:33
place where people store their software and what we've seen by looking at github
00:13:38
is back in 2008 I think it had roughly 33,000 repositories 2009 when I looked at it earlier this summer I think it was
00:13:50
somewhere around over 200 million which is yeah it's it's a tremendous growth
00:13:56
it's it's actually nearly a four order of magnitude growth in a decade and that
00:14:02
kind of of growth and data as you probably know data really drives a lot of these machine learning systems so
00:14:10
this is created essentially a vehicle in which we can start to explore this space
00:14:14
so that's the inflection point but now goo the other point of the thing that's Intel and I think MIT observed is
00:14:23
fundamentally what we're seeing is the way we've historically done programming
00:14:28
we think is flawed that there's essentially a blurring of the programmers intention with these
00:14:36
algorithms and with the system level details what we really want to do as we move forward is we want the programmer
00:14:43
to just specify his or her intention that you you want to create a program that will tell you where the nearest
00:14:51
Starbucks is and you just say computer create a program that will always notify me when I'm near Starbucks and then the
00:14:58
computer handles all the details of the algorithm to implement it understands how to translate that to work on the
00:15:04
hardware that's on your cell phone or in a data center that type of thing so that
00:15:10
those are the two pieces that we think really are creating this opportunity for tremendous growth in machine programming
00:15:16
now to your second point about why is this interesting to Intel until obviously is very interested in advances
00:15:24
in hardware and me being at Intel for about a decade now one thing that I've
00:15:28
seen which is really exciting is Intel really used to be just a CPU company and we're not today today the
00:15:37
heterogeneous hardware landscape that we have at Intel is enormous we have neural network processors we
00:15:43
have neuromorphic processors we have GPUs we have a variety of accelerators we have FPGAs and we have a ton of CPUs
00:15:52
the problem though is programming these things that we can have all this tremendous harder but how can we
00:16:00
possibly expect that the average developer can program this and this is really why machine programming is
00:16:05
essential to Intel as Intel understands that with this new heterogeneous Hardware landscape that really is
00:16:12
required to advance all of the technology that we're seeing we need a way that it's simple enough
00:16:19
for the average programmer harness this massive amount of heterogeneous compute it sounds amazing the ISO since he was
00:16:30
speaking about the work that you did with MIT I understand that you wrote a paper called the three pillars of
00:16:36
machine programming mm-hm some time ago about some of these concepts and I was wondering if you
00:16:43
could share some of the main insights from that and how they relate to some of the things we're talking about certainly
00:16:49
yeah so back in I think it was 2017 a few of us from Intel labs teamed up with several people at MIT and we came up
00:17:00
with this vision of what if we did this thing called machine programming what would the landscape look like and the
00:17:09
main reason for this is we were seeing in these research venues that people were starting to explore machine
00:17:16
programming but they were a bit disorganized that there wasn't structure around the way we were thinking and so
00:17:25
the paper that we wrote the three pillars of machine programming is essentially a road map on how we want to
00:17:32
express and explore the research space there are three pillars intention invention and adaptation the intention
00:17:40
pillar is really what we would think of as the programmer is do in the future I don't really call these
00:17:46
people programmers I call them software creators because at the end of the day our our blue sky vision is these folks
00:17:54
won't write a single line of code they will express their intention either through natural language gestures visual
00:18:03
diagrams whatever is best for them and and for those hardcore programmers out there they can still write all the code
00:18:09
that they want so that's the intention pillar the invention pillar is it then
00:18:14
takes the programmers or the software creators intention and then translates that into the actual software these are
00:18:22
the algorithms the data structure is that type of thing once that's established then that work gets handoff
00:18:28
to the adaptation pillar the adaptation pillar then takes that code and then figures out ok what is the software and
00:18:36
hardware ecosystem look like for this particular program how do we need to augment it to make it run efficiently
00:18:43
securely correctly and then in the machine learning context accurately now in addition to Intel I'm sure there are
00:18:53
other companies that are also working on machine programming and I wonder if there are any companies with whom you
00:18:59
collaborate whose work you could talk about just to explore how this field is evolving absolutely so yeah we have many
00:19:08
collaborators in industry as well as academia and some of our industrial partners that are looking into this are
00:19:14
Microsoft and Facebook so Microsoft's they have a wonderful gentleman over there submit Gulani who's seen in in
00:19:23
many people's eyes is one of the founders of formal program synthesis and he actually has developed the system
00:19:30
inside of Excel that will automatically figure out what the users intent is they
00:19:37
call this flash fill so this is a good concrete example of real-world evidence that this is not just a research toy you
00:19:45
can actually build this into real products so Microsoft is deeply interested in this another company is
00:19:51
Facebook so Facebook is actually doing tremendous work in this space they recently published a paper about
00:19:57
a system called aroma and what aroma does is essentially works along the same lines of the three pillars it's
00:20:04
principally focused on trying to help with the intention that a programmer has an intention of trying to write some
00:20:13
code and doesn't quite know exactly how to write that code the aroma system then will take a little
00:20:20
bit of that code and do an analysis over a very large database and then send the
00:20:25
user back is this what you meant and it's sort of a human-in-the-loop machine
00:20:29
learning approximate solution that's sort of good early evidence that there's
00:20:34
a lot of while while we think of the space machine programming as being a very long journey there's things that we
00:20:41
can be doing today in industry that could be extremely valuable that sounds pretty remarkable again know
00:20:49
you spoke about several companies which countries do you think are making progress that you find impressive in the
00:20:57
area of machine programming in general in AI I've heard that China is you know
00:21:03
advancing in leaps and bounds could you talk about you know what's happening in
00:21:08
other parts of the world and what are some of the things that you are paying attention to absolutely
00:21:14
so as you pointed out China is doing tremendous things one of the things that they're doing is they have a very strong
00:21:20
sort of governmental infrastructural support for AI and it's my belief that the u.s. also has this but maybe not to
00:21:29
the level that China does it's something that I think as a country we probably
00:21:34
need to be a little bit more aggressive and progressive about there's also a lot
00:21:39
of involvement and advances that are happening in Europe and that's also tied
00:21:44
in both with their academic schools they have very strong ml leaders and academia
00:21:50
and then also the vision that they have through their governmental infrastructure which European country is
00:21:56
you think are doing the most interesting work that that's a great question off
00:22:01
the top of my head you know Germany is actually doing some really tremendous stuff as one might imagine part of that
00:22:08
has to do with the fact that they've been deeply involved in autonomous or vehicles and the natural
00:22:14
evolution is autonomous vehicles and then the byproduct of that is deep engagement in AI and machine learning no
00:22:21
D which innovations and machine programming do you think most promising and where do you think the next
00:22:29
breakthroughs will occur in the immediate future yeah so this is this is a really fascinating question and as I
00:22:37
was mentioning before there is a lot of low-hanging fruit where we can make advances and we can build things like
00:22:44
aroma or flash fill that are very useful but there are some core challenges that
00:22:50
at least with the folks that I'm interacting with that places like Stanford and MIT and Google and deeply
00:22:56
within Intel labs that we don't quite have the answer to the first is the structural representation of intention
00:23:08
so what I mean by this is oftentimes when we're writing code the programmers intention is diffused
00:23:16
across of the code what we really want to be able to do is understand how to properly represent the user's intention
00:23:24
and we don't quite have this there's a lot of advances that we've made
00:23:27
historically with things like compilers and static analysis tools that create different sort of graphical or tree
00:23:35
structures but when we've tried to apply these in the space of machine programming they don't quite fit that we
00:23:42
can sort of you know push the you know square peg in the round hole but it's
00:23:46
not the right match so I didn't tell we're thinking about this thing that
00:23:51
we're sort of roughly calling the abstract semantic graph and the idea here is this structure whatever this is
00:24:00
that we don't quite understand will be some sort of graphical representation of
00:24:04
the semantics essentially the intention of what the user wants once we figure out how to build this thing my belief is
00:24:14
the field of machine perming will see a tremendous spike of growth so a lot of people are working on this I'm working
00:24:21
with collaborators both in industry and then in academia folks at Penn Berkeley MIT we're all
00:24:28
thinking deeply about this hopefully we'll be able to figure out this this abstract semantic graph soon and until
00:24:38
we do I think that we'll just sort of work with the maybe not perfect solutions and try to you know edge our
00:24:45
way forward so if you figure it out what might some of the implications be so some of the implications will be the
00:24:54
programs that we'll be able to generate are likely to be orders of magnitude
00:24:58
more complex than the ones that we can create today for example in the space of formal program synthesis or approximate
00:25:07
solutions for machine programming we may be restricted to let's say programs that
00:25:12
are up to maybe a hundred instructions or less if we figure out how to build this abstract semantic graph it's my
00:25:20
belief that we will move from hundreds to thousands potentially millions of lines of code that so the implications
00:25:29
of this thing they're enormous so when any new technology comes along especially a machine learning or as you
00:25:38
described machine programming very often technologists have to justify these investments to the CFO or to the CA CEO
00:25:47
not just in terms of this is very cool technology but right this fits in with the ROI of where the business wants to
00:25:54
go or order fits in with the business strategy what are some of the metrics that you think about in terms of what's
00:26:03
the ROI in measuring the ROI of machine programming right this is a great question and of course as the leader of
00:26:12
the machine programming research group at Intel it's my job not only to work on
00:26:17
the research but then also justify its business value as you pointed out and one of the things that you you might
00:26:26
know is Intel's very interested in performance but we're not just interested in Hardware performance we're
00:26:33
also interested in software performance so one could imagine that if you have a programmer that's
00:26:38
writing code that's slow they might blame the Intel processors is being slow even though the problem is
00:26:44
not the processor it's actually the software one of the promises of machine programming and we're seeing early
00:26:51
evidence of this is that the code we can generate through these automated methods
00:26:56
will be superhuman in their performance correctness security so on and so forth one concrete example of that is one of
00:27:04
my colleagues Andrew Adams Jonathan Regan Kelly cave on photo Holly and Diaz are folks from MIT Stanford and
00:27:12
Facebook that were actually I think Andrew Adams just pivoted to Adobe research they have built a system called
00:27:20
halide it's a programming language that separates out the programmers intention
00:27:25
from the actual scheduling of that intention and in their recent paper this year that was published or I think in
00:27:32
June they've shown for the first time that the world's foremost experts in
00:27:38
this programming language can't compete with the machine that the machine is
00:27:43
producing code now that is regularly more efficient and I think it's by I'm
00:27:50
gonna be I'm just gonna guess here by at least 50% it might be upwards of you
00:27:55
know a hundred percent faster and this is the first time in the decade that they've been working in halide that
00:28:02
they've been able to achieve this but so this gives us promise that if we can do
00:28:07
it in halide maybe we can generalize this and start to improve the efficiency of code everywhere this is really
00:28:15
important to Intel because obviously we want everyone software to run as efficiently as possible and we don't
00:28:21
want people to mistakenly believe that our hardware slow when actually the problem is somewhere else no I I I hear
00:28:29
you and it actually reminds me of a broader concern that I have often heard about AI which is that as a lot of
00:28:36
automation begins to take place and AI systems get implemented that the impact of jobs could be considerable so for
00:28:45
example but autonomous trucks it has been a fear that lots of truck drivers could be losing their jobs if
00:28:55
you know autonomous vehicle stopped hauling you know goods across the highways so do you think that there is a
00:29:05
risk that if machine programming takes off that the same thing could happen to software programmer jobs and that this
00:29:14
this is something that the industry should be concerned about right this is an excellent question and one that I'm
00:29:21
asked quite often my my honest opinion is actually the inverse will happen is that through machine programming we will
00:29:31
create many jobs perhaps millions or tens of millions of jobs and the reasoning is actually very simple right
00:29:38
now we have a global population in the billions yet the programmer pool today is a very small percentage I don't know
00:29:48
the exact percentage but I think that it's roughly around like 1% of the global population with machine
00:29:54
programming what we are are trying to achieve is enable the entire global population to create software for
00:30:03
example my mother she's incredible you know entrepreneur she's created several
00:30:07
businesses and has just done fantastic but she's not a programmer so the entire
00:30:12
world of software is closed off to her and in fact this is one of the reasons that I became so compelled is I see
00:30:18
someone like her who's wildly creative she has some amazing ideas but because
00:30:24
software's closed off those ideas never get realized and hopefully with machine
00:30:29
programming with this intentionality that we're discussing earlier this will
00:30:32
this will create tens hundreds of millions of jobs it'll also keep the programmers that we have today employed
00:30:39
because there is work to do on building these very complex systems and as we expand intentionality we're going to
00:30:47
require those people those what we call it Intel those ninjas to be ensuring all
00:30:53
the subsystems that are part of those three pillars are advancing appropriately so since we were talking
00:31:00
about these advice consequences it reminded me of a conversation almost 15 years ago with
00:31:06
Andy Grove the former CEO of Intel whom we had interviewed in 2004 and and he once said that for every metric there
00:31:17
should be another paired metric that addresses the adverse consequences of the first so as you were thinking about
00:31:23
some of the metrics that you would use to measure the success or the ROI yeah off of machine programming what would be
00:31:31
some of the adverse consequences of machine programming and what metrics would you use just to keep an eye to
00:31:37
make sure things don't get out of control I'm really glad you asked this
00:31:42
question and on top of that before I answer I just want to say I'm a huge fan
00:31:47
of Andy Grove it's wonderful to be at a company with such strong legacy of leadership and you know we see the
00:31:55
impact that Andy has had even today that the company is really trying to follow a
00:32:01
lot of his principles I would agree with you wholeheartedly and in fact in a book
00:32:06
that we wrote in 2004 that knowledge at Wharton wrote he was identified as the top leader among 25 leaders in the past
00:32:13
25 years oh that's fantastic yeah I never got the chance to work with him personally but I know people that
00:32:19
have and from everything I hear he was not only a tremendous leader but also tremendous technologist which is sort of
00:32:27
a very rare combination but going back to your question about the adverse consequences we actually talked about
00:32:34
this a little bit in our three pillars paper and this is part of the reason why we we wrote this paper is that what we
00:32:42
were seeing in some cases of advances is there would be research for example one
00:32:48
of my colleagues that I'm very fond of Alvin Chung he's a professor at Berkeley
00:32:52
and he's doing this work called verified lifting and verified lifting essentially
00:32:57
uses formal program synthesis techniques to lift code from one programming language and then drop it down into
00:33:04
another programming language this is wildly useful for legacy systems that can't be maintained because we don't
00:33:11
have a programmer supply we can lift that code out put it in language where we have lots of
00:33:15
programmers however one of the things we noticed and I've discussed this with
00:33:20
Alvin so he will probably not be shocked when I mention this is that there's a
00:33:25
potential byproduct of that lifting that can reduce intentionality so for example
00:33:32
his work we would say principally falls in you know the invention and then the adaptation based on how that code is
00:33:42
transformed the intentionality of that code could be reduced for example things like variable names function names
00:33:49
things that are really important to programmers may not map properly to the new structure so as we're making forward
00:33:58
progress in machine programming what we've asked the community to do is think
00:34:02
of the context of the three pillars and then try to understand are you inadvertently hurting another pillar and
00:34:10
if you are clarify that so we understand that this is another thing that we now need to advance now since you mentioned
00:34:17
you know some of the academic collaborations I was I know that you spoke at Penn that did precise event
00:34:26
last week and I was wondering if we could end by talking a little bit about what kind of work you are planning to do
00:34:32
here at Penn oh yeah so I'm yeah very delighted that you asked this question
00:34:37
and I'm really happy to be here I had the honor of giving this talk at the precise industry day which was over
00:34:46
sold out I think people were sitting on the floor it was just very well attended
00:34:51
and rightfully so the lot of the thought leaders in the space of computer science
00:34:58
and formal methods in machine learning are part of the precise center recently I accepted invitation to help chair the
00:35:09
technologists regroup for precise and also act as their executive director for artificial intelligence my role with
00:35:18
precise and with Penn actually I think is twofold the first is with precise they have a very strong sort of
00:35:27
technical Horsham of industry collaborators and what I would like to do is ensure that
00:35:32
all of the industrial partners are working in a very complementary way that we understand what the core challenges
00:35:39
are and that we're not working in a way that's overlapping duplicating effort so
00:35:44
that's one part the other part that's really important to me is right now we
00:35:49
have sort of a lack of machine programming engineers and researchers in fact there's very few of us which makes
00:35:58
sense because the field even though it's been around since the 50s it's had
00:36:03
struggles to get to the point where it is today so what we're working on with Penn and
00:36:09
other academic Institute's is to start to incorporate curriculum changes and
00:36:15
get our undergrads get our grad students more familiar with it and then also generate the new leading minds through
00:36:23
the PhD programs that are then going to drive the research they're happening
00:36:27
both in academia and in the industrial labs that sounds wonderful and Justin thank you so much for taking the time to
00:36:36
explain all these things to us it's wonderful to meet you and very happy to have you here at knowledge at Wharton
00:36:42
yeah it's been my pleasure thank you so much for having me for more insight from
00:36:49
knowledge at Wharton please visit knowledge Wharton UPenn edu [Music]