00:00:00
welcome to the analytics at Wharton and
00:00:02
AI at Wharton podcast series on
00:00:04
artificial intelligence my name is Eric
00:00:06
Brad a professor of marketing and
00:00:08
statistics here at the Wharton School
00:00:09
I'm also the vice dean of analytics what
00:00:12
we're doing in this series is to explore
00:00:14
the role of artificial intelligence in
00:00:16
various aspects of business and today
00:00:18
certainly is no exception maybe one that
00:00:20
most people consider the most exciting
00:00:23
which is a artificial intelligence in
00:00:25
Innovation management I'm joined by two
00:00:28
of my colleagues uh first is valer
00:00:30
yakovich uh valer is executive director
00:00:33
of the Mac Institute for Innovation
00:00:35
management uh the Mac Institute focuses
00:00:37
on creating synergies between research
00:00:39
teaching and the practice of innovation
00:00:42
management within the school so valer
00:00:43
welcome to our podcast oh thank you glad
00:00:46
to be here I'm also joined by my friend
00:00:48
and colleague Christian turves uh
00:00:49
Christian is the Andrew mher professor
00:00:52
at the Wharton School he's a professor
00:00:53
and chair of the Wharton operations
00:00:55
informations and decisions Department
00:00:57
here he's also co-director of the Mac
00:00:59
Institution and he also holds a faculty
00:01:01
appointment in the Pearlman School of
00:01:03
Medicine Christian welcome to the show
00:01:05
thanks for having us it's great to have
00:01:06
you both so let's first start I hate it
00:01:09
when people use jargon so valer maybe
00:01:12
I'll start with you what is innovation
00:01:15
management and what does the Mac
00:01:17
Institute do and then we'll dive into
00:01:20
what role AI might have to play in that
00:01:22
oh let me try also I actually can rely
00:01:25
on Christian's favorite definition of
00:01:27
innovation I learned from my faculty
00:01:28
called directors so basically it's about
00:01:31
matching uh customer needs with the
00:01:35
technological solutions we have out
00:01:36
there and what we do basically we U our
00:01:41
priority our depart kind of departure
00:01:44
point is uh faculty research we fund it
00:01:47
in Innovation entrepreneurship and then
00:01:49
we try to translate it into um
00:01:51
experiential learning for students and
00:01:53
business practice for that purpose we
00:01:55
have a course with students Project
00:01:57
based we have corporate partners with
00:02:00
whom we work kind of trying to identify
00:02:02
their problems and provide some kind of
00:02:04
guidance thought leadership and so over
00:02:07
the years we identified basically four
00:02:09
areas which are the critical for us okay
00:02:12
it's about
00:02:14
um opportunities and risks uh how we
00:02:18
discover them and analyze them strategy
00:02:21
development uh organizing for Innovation
00:02:24
what kind of organizational structures
00:02:26
teams and so on you uh set up and uh
00:02:30
employ and finally uh value capture from
00:02:33
Innovation and so in my view today's
00:02:36
conversation is about kind of how
00:02:38
different types of AI in particular
00:02:40
generative AI U affect all these areas
00:02:44
basically that's why I think I'm here
00:02:46
yeah so Christian maybe you could tell
00:02:48
our listeners here on this this Sirius
00:02:49
XM Wharton podcast series I I'm going to
00:02:52
use the vernacular here all hell must be
00:02:55
breaking loose at the Mac Institute I
00:02:57
mean if you guys are focusing on
00:02:58
innovation and I think most people would
00:03:01
argue one of the big areas of
00:03:03
application of these large language
00:03:05
models like Bard and chat GPT is
00:03:08
innovation like how do you get started
00:03:11
like how do you as a scholar think about
00:03:13
your research how do you think about as
00:03:14
a A Center Director how do you think
00:03:16
about as the chair of the department
00:03:18
with all the hats you wear where do you
00:03:20
even get started and how do you think
00:03:22
about it I would narrow it down to to
00:03:24
three dimensions of innovation that when
00:03:27
I teach Executive Education when I teach
00:03:29
rbes students I would want to focus on
00:03:31
right there is the initial idea and that
00:03:33
is this combination of solution and need
00:03:36
uh that hopefully creates some form of
00:03:37
value that is something that the student
00:03:40
needs to be able to manage from I have
00:03:41
an idea towards launching a venture and
00:03:44
I think W has been wonderful at doing
00:03:45
that AI is helpful at that level because
00:03:49
a lot of the things that used to be very
00:03:51
expensive very difficult to do such as
00:03:53
market research such as prototyping
00:03:55
through generative II got a lot better
00:03:58
the second thing that we want talk is
00:04:00
about is in in bigger organization
00:04:02
established organizations there's a
00:04:04
pipeline of ideas flowing uh they don't
00:04:07
have just one idea they have thousands
00:04:09
of ideas and that's a process that needs
00:04:11
to be managed and AI has been as we show
00:04:14
in some of our research been really good
00:04:16
at at fueling that process of filling
00:04:19
that pipeline of ideas and then the
00:04:21
third thing is that process needs to
00:04:22
have a Direction I mean most
00:04:24
organizations unlike the unless they
00:04:26
Venture Capital firms they have some
00:04:28
form of strategic intent
00:04:30
and I have to help managers kind of
00:04:32
think about possible future states of
00:04:34
the world possible disruptive threats
00:04:37
and again CH GPT or other kind of gener
00:04:40
of models can help me imagine a world
00:04:43
for which I would should be prepared
00:04:45
that I myself would have want to imagine
00:04:47
so let me ask you valy as well um
00:04:50
whether it's in general or the two
00:04:52
specific uses that Christian mentioned
00:04:54
which sit in my area of the world too
00:04:56
which is marketing research and
00:04:57
prototyping um would f are firms as
00:05:01
you're talking to them would they really
00:05:03
replace direct marketing research or
00:05:06
creating minimal viable products and
00:05:08
doing prototyping would they really
00:05:10
trust artificial intelligence with this
00:05:13
crucial step of the let's call it new
00:05:16
product development process or what are
00:05:18
you seeing out there well actually they
00:05:21
definitely don't uh transfer decision
00:05:23
making to these large language models
00:05:26
but what we do see they bring them in
00:05:28
and trying to kind of um take all the
00:05:32
Corpus of knowledge they accumulated
00:05:34
over years uh bring it into kind of in
00:05:37
interactions with these models and try
00:05:39
to automate some parts of the process
00:05:41
and actually we are doing the same here
00:05:43
we the consistent uh request we have for
00:05:46
example from the medical school or
00:05:48
engineering school when we work with
00:05:49
them on their inventions and trying to
00:05:51
kind of uh develop Pence Innovation
00:05:54
ecosystem uh they ask us for market
00:05:56
research and the demand is so
00:06:00
substantial we can't meet it with the
00:06:03
available tools and available resources
00:06:06
so what we are doing now we are trying
00:06:07
to actually uh figure out how to use l
00:06:11
language models in assessing let's say
00:06:13
potential uh commercialization potential
00:06:15
of in new inventions in the School of
00:06:18
Engineering so could maybe Christian
00:06:20
since I know you're both you've written
00:06:22
a number of books you've written stuff
00:06:24
on Innovation and Innovation tournaments
00:06:26
you've been an
00:06:27
entrepreneur can you take our list ERS
00:06:29
kind of this stepbystep process imagine
00:06:32
the Mac Institute wanted to partner with
00:06:34
pennovation to try to help think about
00:06:37
the economic value or how to best launch
00:06:39
projects how would you use artificial
00:06:42
intelligence to help support that
00:06:44
process I think in any Innovation
00:06:46
process there are two key functions that
00:06:48
is the generation of opportunities
00:06:50
create more better and higher variance
00:06:53
opportunities it's a very derian process
00:06:55
so we need to create variety first so
00:06:57
let me start with that one
00:07:00
large lot of argument against generative
00:07:03
AI is that it tends to work for the you
00:07:07
know the the center mass of the
00:07:09
distribution it doesn't give you very
00:07:11
good ideas in the long tail it doesn't
00:07:13
if it's not in that trained Corpus you
00:07:16
won't see it in some so how do you think
00:07:19
it does in that first phase yeah I know
00:07:22
I'm I'm glad that you asked so we have
00:07:23
done a study based on MBA generated
00:07:26
ideas of which we have thousands in our
00:07:28
database so after teaching Innovation
00:07:30
for many years and we compare this with
00:07:32
large language models and to our
00:07:35
surprise the large language models are
00:07:37
actually better of creating what we call
00:07:39
High variance Innovations of polarizing
00:07:42
Innovations of Innovations where the
00:07:43
payoff distribution has a high variance
00:07:46
which is good because Innovations ideas
00:07:49
have this real option flavor that if the
00:07:51
idea is bad we just cancel it we don't
00:07:54
execute just for our listeners as
00:07:55
measured by whom how do you measure the
00:07:57
variance like do you use humans or you
00:07:59
ask AI engine to score the one that
00:08:02
would be uh Magic uh so we get get to
00:08:05
your second part in the moment when I
00:08:07
get to my second part which is a
00:08:08
selection step uh so how do we evaluate
00:08:10
the quality of the ideas uh we do
00:08:12
purchase intense studies uh that
00:08:14
Standard Market Research we Rego on amk
00:08:18
prolific or other platforms we showcase
00:08:21
textural descriptions of the ideas we
00:08:23
asked for purchase intent probabilities
00:08:25
of large crowds which is not perfect but
00:08:29
again best practice of what you guys in
00:08:30
marketing do very before you get to the
00:08:32
selection piece let me ask you let's
00:08:34
imagine there's a world where people are
00:08:37
doing what you're doing and thousands of
00:08:40
these types of let's call it generative
00:08:42
studies are done and M Turk and all of
00:08:45
that's done won't eventually an AI
00:08:48
engine be able to forecast the stuff
00:08:51
that we're using humans for to evaluate
00:08:54
right now like right now we just have a
00:08:55
data problem but if we had the data AI
00:08:58
could do that too it's a really
00:08:59
interesting question right I mean what
00:09:01
we've seen with humans over many many
00:09:03
years is the selection decision is
00:09:05
always the hard one right right I mean
00:09:07
coming up with ideas is I don't want to
00:09:09
say easy but it's it's something that
00:09:12
humans can do and now ai can do when we
00:09:15
try to use AI to predict the quality of
00:09:18
an idea it still struggles and again
00:09:20
it's not too surprising in the sense
00:09:22
that humans even Venture capitalists are
00:09:24
really having a hard time predicting the
00:09:26
odds H Val any thoughts about that about
00:09:28
the I'll call it the idea Generation by
00:09:31
the way I love this bifurcation into the
00:09:34
idea generation stage versus the
00:09:37
selection phase and I as a statistician
00:09:40
I would imagine with enough data
00:09:43
eventually and enough variation you know
00:09:46
as we always say you need variation in X
00:09:47
to be able to have good selection models
00:09:50
and you need to of course observe
00:09:51
outcomes over time I would imagine that
00:09:53
if we're sitting here 5 years from now
00:09:55
ai engines may be able to do better A
00:09:57
lot better on the selection phase than
00:09:59
they they do now but what are your
00:10:00
thoughts well actually I think if we
00:10:02
look a couple years back we thought AI
00:10:05
will never be creative we always thought
00:10:07
AI will be predictive but creativity
00:10:10
based on existing data and so on and
00:10:12
suddenly we're surprised we find that
00:10:14
generative AI is quite creative and but
00:10:17
if you think more about it it's not so
00:10:19
surprising one how do we Define
00:10:21
creativity we go back to shet famous
00:10:24
economist he said it's about Rec
00:10:25
combination of existing ideas and
00:10:28
because um the large language models are
00:10:30
trained on such a huge volume of
00:10:32
information which encompasses all kinds
00:10:34
of diverse opinions if the task is not
00:10:38
very well defined it actually does
00:10:40
better because it can produce all kinds
00:10:42
of opinions you can imagine all kinds of
00:10:45
customer profiles you can imagine and uh
00:10:48
this variability becomes very helpful
00:10:51
for a combination right that's why these
00:10:53
findings are quite uh consistent with
00:10:56
what we know where how creativity
00:10:57
operates and and by the way the nice
00:10:59
thing that uh chrisan is doing he's
00:11:00
sharing a lot of his findings with the
00:11:02
press and social media so I had read
00:11:03
that article and I'm glad that I read it
00:11:05
no I I give myself a good grade because
00:11:07
that is what I read from the article so
00:11:09
I'm glad that I interpreted the findings
00:11:11
appropriately um so there's the
00:11:14
technology piece there's the Innovation
00:11:17
piece but what about the company
00:11:19
adoption piece so what are you guys
00:11:21
seeing in the Mac Institute part of the
00:11:23
world are companies embracing this as
00:11:26
the next great opportunity or as or as
00:11:29
companies thinking my God this is a
00:11:31
threat to my business model what do you
00:11:33
see um happening out there in the world
00:11:36
uh so we just on Wednesday evening a
00:11:38
session in Executive Education with the
00:11:41
uh customer analytics program that
00:11:43
you're well familiar with of course
00:11:44
which I taught on on Monday all right
00:11:46
and so we're talking about with the
00:11:47
participants of what is it means for
00:11:49
their business and I think many of them
00:11:52
are struggling making sense of the
00:11:54
technology they know it's big but they
00:11:56
have a hard time for like what do I do
00:11:58
next Where do I get started and I think
00:12:01
I hate to say this to a marketing
00:12:02
Professor right but I mean you start
00:12:04
with your customer Journey you start
00:12:06
with your customer pain points it's not
00:12:08
the right strategy to say like let's AI
00:12:10
everything you look for the customer
00:12:12
Journey what are the customer needs
00:12:14
where are the pain points and then
00:12:16
identify those and think along those
00:12:18
customer Journeys where could AI be the
00:12:20
right solution you now have a new set of
00:12:22
tools and you can go through your
00:12:25
existing pain points that you might have
00:12:26
known for many years and fix those plus
00:12:30
you can find through the sensing
00:12:32
technology of AI by having it read
00:12:34
customer reviews by interviewing it you
00:12:37
can find new pain points along the way
00:12:38
that you might have not been aware of I
00:12:40
see so valer um what are you seeing
00:12:43
since uh you know as the executive
00:12:44
director of the Mac Institute a lot of
00:12:46
your role is to interface with companies
00:12:48
you know us as faculty directors we
00:12:50
obviously do a lot of research we also
00:12:52
interface with companies but you're
00:12:53
really on the front lines what are you
00:12:55
seeing today and how are companies
00:12:58
thinking that the Mac Institute can help
00:13:00
them well uh I think
00:13:04
uh right now that's the major disruptive
00:13:08
technology that preoccupies manager
00:13:11
attention basically uh I mentioned
00:13:13
briefly that we have this experiential
00:13:15
learning piece at the Mark Institute we
00:13:17
do projects with companies this semester
00:13:19
out of seven projects four are about
00:13:21
generative Ai and I kind of looked at
00:13:24
them before can you tell us can you tell
00:13:25
our listeners without giving necessarily
00:13:27
the company or something can you tell us
00:13:28
what are the projects what they a few
00:13:31
kind of pretty much their names their
00:13:33
titles smart Supply Chain management
00:13:35
using generative AI uh AI Synergy
00:13:39
strategizing and operationalizing
00:13:41
intelligent transformation it's a very
00:13:43
general kind of topic one is directly
00:13:45
relevant to in AI in Innovation
00:13:47
management it's intelligence software
00:13:49
testing with a AI ml Innovations right
00:13:53
so software testing is one major part of
00:13:56
any Innovation or testing prototype
00:13:58
typing in general and finally evaluating
00:14:01
the potential of disrupting the mortgage
00:14:03
title industry through AI technology
00:14:06
what I also see I was last week in
00:14:08
Silicon Valley meeting with our
00:14:09
corporate Partners meeting with startups
00:14:11
some of them gaining a lot of traction
00:14:13
for example going back to Innovation
00:14:15
management there is one company that um
00:14:18
pretty much automated the process of
00:14:20
patent writing which is a extremely
00:14:23
labor I saw an academic talk on that
00:14:26
maybe about two or three months ago
00:14:28
where I I thought it was remarkable yeah
00:14:31
I mean you can think it's very well
00:14:34
structured language right and very
00:14:35
specific very hard to understand
00:14:37
language if you read the patent uh but I
00:14:39
can imagine it's pretty straightforward
00:14:41
to train um machine learning AI
00:14:44
generative AI L language model to do it
00:14:47
and uh what I notice when companies deal
00:14:49
with these things what they need to do
00:14:51
they they obviously take an existing uh
00:14:54
L language model Foundation L language
00:14:56
models they don't develop their own but
00:14:58
then they need sa safeguards uh a wall
00:15:02
between the vendor of the model and
00:15:04
their own knowledge base and vendors now
00:15:06
willing to provide it at the same time
00:15:09
also a number of startups emerged that
00:15:11
actually uh offering these companies um
00:15:15
security privacy and other tools to also
00:15:17
not only Safeguard their own knowledge
00:15:20
base but the knowledge base of their
00:15:21
clients that they're going to use in
00:15:23
order to deliver value to clients so I
00:15:25
think this privacy uh confidentiality
00:15:28
are key issues and security of these
00:15:31
models what we see going on and uh
00:15:34
another example of very interesting
00:15:35
creative application for Innovation
00:15:37
management I encounter it it's a company
00:15:40
that have a huge databas base of cancer
00:15:44
patients and uh now they're trying to
00:15:47
engage large language models to match it
00:15:50
with uh
00:15:51
fda's uh uh database of clinical trials
00:15:55
that are going on in order to find the
00:15:57
right subjects for the right trial
00:15:59
so apparently it's a huge value added
00:16:02
that can be done now at large scale
00:16:04
using this model so Christian let me ask
00:16:06
you on the you know I'll call it biggest
00:16:08
opportunity side and the biggest area
00:16:11
where you don't you still don't see AI
00:16:13
being used much what are you seeing as
00:16:15
like if you had to give a lecture
00:16:17
tomorrow to your MBA students and say
00:16:20
this is the most sophisticated
00:16:22
interesting value added application of
00:16:24
AI this is what I've seen the last three
00:16:27
years and if you also had to give the
00:16:28
same lecture and say and here's an
00:16:30
example where I think there's an
00:16:31
opportunity but I haven't seen anything
00:16:33
yet what would those be so in terms of
00:16:36
word works I think anything that is
00:16:38
simple text writing and simple all the
00:16:40
way to a new patent but it is a writing
00:16:43
task AI is amazing and that I think that
00:16:46
n has been cracked it's only going to
00:16:48
get better I think as faculty we all get
00:16:50
these inquiries Professor bradl could
00:16:52
you summarize this paper for me I mean
00:16:54
don't do this right AI can do this and
00:16:57
so let me ask you you um part of both of
00:17:00
our jobs is as Journal editors and
00:17:03
reviewers should I ask should I no the
00:17:07
the journals have a policy against this
00:17:09
potentially right now maybe not in the
00:17:10
future but let's say there was no policy
00:17:13
should I take an article jam it into an
00:17:15
AI engine ask it to give me a summary of
00:17:18
it in addition to my reading of it as a
00:17:21
reviewer you're should not but as
00:17:23
somebody who wants uh to stay current
00:17:26
with the literature having basic the AI
00:17:29
give you every morning uh a two-minute
00:17:32
summary of a paper that otherwise takes
00:17:34
an hour or two hours to read I think
00:17:36
that would be a very healthy thing
00:17:37
forget the ethical or moral parts of it
00:17:40
why do you say as a reviewer I should
00:17:42
not I have I could give a reason but I'm
00:17:44
not here I'm here to interview you who
00:17:46
cares what I think what do you think so
00:17:47
as a reviewer you have to make you have
00:17:49
to turn over every stone to make sure
00:17:51
that there's not the fall the flaw in
00:17:54
the paper in the methodology I think
00:17:56
that is something which is a COR Corner
00:17:58
that I don't think AI is ready to do
00:18:01
I've tried this AI is doing a decent job
00:18:05
when you feed it a PDF of a paper saying
00:18:07
like look there might be some issues
00:18:09
with endogenity or what have you some
00:18:11
pretty generic ones I don't think it has
00:18:14
the Precision to dial in and say like in
00:18:17
equation seven the erot term is
00:18:18
correlated with the explanatory variable
00:18:21
I see right so I think there uh we still
00:18:23
have to do the homework um the second
00:18:25
type of work is the one that puzzles me
00:18:27
the most is analytical types of things
00:18:31
right I mean so uh especially at the
00:18:33
beginning when I gave GPT my MBA exam a
00:18:37
year ago please tell our listeners about
00:18:39
this because this by the way I I I would
00:18:42
guess it is probably the most publicized
00:18:45
article that has come out of Wharton in
00:18:47
the last 5 years no I'm just saying it
00:18:49
was on every major News Network it was
00:18:51
republished everywhere so please tell
00:18:53
people about the study what you found
00:18:55
and then what you're still puzzled about
00:18:58
uh so over the winter break my kids and
00:19:01
I were sitting together my kids are in
00:19:03
college or through college who were
00:19:04
talking about GPT like everybody else
00:19:06
probably in the world and so the
00:19:07
question came up like that you're
00:19:09
teaching in this MBA course you think
00:19:10
that GPT could take your exam and so we
00:19:13
literally took my MBA exam and fed the
00:19:15
questions cut and pasted them in the
00:19:17
prompt line and it did really well it
00:19:20
did what I would have given it a solid B
00:19:23
if not a B+ over the subsequent months
00:19:27
then was GP before coming out it is now
00:19:30
well in the a range basically the type
00:19:32
of questions and my my questions are
00:19:35
many cases so to say like 10 lines of
00:19:37
text with some computations in there
00:19:40
find the Bott compute the inventory cost
00:19:43
uh do some queuing analysis um GPT is
00:19:46
amazing at that which again is
00:19:49
counterintuitive because it's a language
00:19:50
model right it has it has no
00:19:52
representation inside of what capacity
00:19:55
even is uh you tell it to find a a rout
00:19:58
through uh traveling salesman problem
00:20:01
connect uh cities in the right sequence
00:20:04
to minimize Transportation time it does
00:20:07
a pretty decent job already right out of
00:20:09
the box what is new since is uh with GPT
00:20:14
4 also we have these plugins now one of
00:20:16
the set of plugins is from W from Al
00:20:18
Alpha which is kind of the the the power
00:20:20
side for analytics that's going to help
00:20:23
and that's going to help right but even
00:20:25
the plain vanilla GPT out of the box has
00:20:28
gotten pretty good at doing analytics
00:20:31
task so I imagine just as a lay person
00:20:35
most of our listeners here would have
00:20:37
not used wol from analytics unless they
00:20:39
nurds like you and me right I mean
00:20:41
that's an Insider type of tool sure you
00:20:42
now have basically a user interface that
00:20:46
lets you do sophisticated analytics that
00:20:49
lay persons can do and in make inquiries
00:20:52
into exploring into analyzing hard
00:20:54
mathematical problems data sets
00:20:57
operations research type of problems I
00:20:58
think that is super exciting H so uh
00:21:01
valer could you tell us about what role
00:21:03
do students play at the Mac Institute
00:21:08
and um what do you think like when
00:21:11
students I'm sure ask you all the time
00:21:12
they ask me all the time especially when
00:21:14
around machine learning like Professor
00:21:16
bradow what should I be studying now
00:21:18
what do you tell students what should
00:21:19
they focus on becoming great prompt
00:21:21
Engineers should they focus on being
00:21:24
able to take the output of chat GPT
00:21:26
integrate it with their own beliefs
00:21:28
and then help in decision making how do
00:21:30
you see that well basically uh we have
00:21:34
they have a challenge now uh and
00:21:37
Christians research showed what the
00:21:39
challenge is you have to figure out CH
00:21:41
GPT can do better some tasks you have to
00:21:44
figure out where you belong and there is
00:21:46
actually another uh recent study done by
00:21:49
a large group group of researchers I on
00:21:52
them Ethan mik our colleague in
00:21:54
management uh it was done at the Boston
00:21:57
Consulting group where they really in
00:21:59
the randomized experiment looked what
00:22:01
Consultants can do what can't do with
00:22:04
generative fi and roughly speaking my
00:22:07
reading of the paper is that on
00:22:10
exploration their productivity
00:22:12
drastically increases with the Gen L
00:22:15
language models on problem solving more
00:22:19
specific contextual tasks which require
00:22:22
more Precision understanding of the
00:22:25
context and so on those who use
00:22:28
I do worse so basically we know um Ethan
00:22:33
actually Ethan and his colleagues talk
00:22:36
about this kind of changing Frontier
00:22:38
where which task can be done or cannot
00:22:40
be done by
00:22:42
geni U and it's hard to identify and
00:22:45
it's a moving Target right but they need
00:22:48
to experiment themselves they have to
00:22:50
innovate and reinvent their careers in
00:22:52
some sense uh so disruptive uh this
00:22:55
technology is uh yes so Christian let me
00:22:58
ask you um if we were sitting here 5
00:23:01
years from now what do you think will
00:23:04
have changed in those five years is it
00:23:07
the language models will get better at
00:23:09
prediction which I'm sure the answer to
00:23:11
that is yes the application areas that
00:23:14
we haven't even thought of will get done
00:23:15
that's probably yes but what do you
00:23:17
think are the big changes our listeners
00:23:19
should know that is coming in the next
00:23:21
five or so years I think we have to stop
00:23:23
thinking about the substitution game
00:23:26
where will the NBA student be replaced
00:23:28
by GPT the amount of work that is going
00:23:31
around in the world is not constant if
00:23:34
we make it efficient and cheap enough
00:23:35
there's new work that is going to Bubble
00:23:37
Up right so I just wrote as my latest
00:23:40
paper a paper on ethical advice and AI
00:23:43
can AI give me ethical advice and we
00:23:45
show in the paper that it is basically
00:23:47
as good as the ethicist in the New York
00:23:49
Times on providing advice to or to
00:23:52
readers or people who face ethical
00:23:55
dilemas so what does this mean one
00:23:57
hypothesis it it puts the ethics
00:23:58
advisors out of business but that's what
00:24:01
not not what I think what I think is
00:24:02
going to happen is we're going to go to
00:24:04
a a virtual zero marginal cost ethics
00:24:07
advisor a lot more and get more ethical
00:24:10
advice right and so this productivity
00:24:12
gain is not putting people out of work
00:24:15
the amount of work that we can
00:24:17
productively serve is going to grow up
00:24:19
and so the effect on employment is
00:24:21
highly ambivalent H yeah please uh just
00:24:24
to add to this uh I just worked on the
00:24:27
paper with Peter Capelli and S tber yeah
00:24:30
on kind of trying to think through this
00:24:31
organizational implications of
00:24:33
generative Ai and um one point we are
00:24:36
trying to make is that jobs consist of
00:24:39
multiple tasks and what we see so far
00:24:41
some tasks indeed can be automated but
00:24:45
then the amount of information you have
00:24:48
to process produced by generative fi is
00:24:51
at such a scale that sometimes uh it
00:24:53
becomes costly you need to now make
00:24:55
sense of this and the problem is that
00:24:58
these models really the kind of
00:25:00
explainability is still a big problem so
00:25:03
I talked to some engineers in this um uh
00:25:06
area related to healthc care and they're
00:25:09
saying that new types of models now they
00:25:11
believe might replace L language models
00:25:14
which actually can so to say understand
00:25:16
things at a large more conceptual level
00:25:19
when instead of predicting the next word
00:25:21
as these models do you kind of uh based
00:25:24
on what you know so far you predict the
00:25:27
next kind of the high level concept and
00:25:30
the text Will Follow from that concept
00:25:32
this is the way we think right and so
00:25:35
apparently there is quite a bit of
00:25:37
attraction in that area um so we'll see
00:25:40
large language models is it the last
00:25:41
word uh or we'll have a totally quite
00:25:45
different technology which is already
00:25:47
out there for for example uh visual
00:25:50
images basically when you see a part of
00:25:52
the image you reconstruct the second
00:25:54
part completely instead of specific
00:25:56
pixels right so there is a lot of
00:25:58
development again it's a moving Target
00:26:00
which uh is exciting to watch these
00:26:03
developments and we need to adjust
00:26:04
quickly and kind of explores things as
00:26:07
they appeared well this has been our
00:26:09
episode on AI and Innovation management
00:26:12
I'd like to thank my colleague uh Valeri
00:26:14
yakovich the executive director of the
00:26:16
Mac Institute and my colleague uh
00:26:18
Christian tursh the chair of the oid
00:26:20
department here at the warten school the
00:26:21
co-director of the Mac Institute uh
00:26:23
thank you both for joining me on this
00:26:25
analytics at Wharton AI at Warton
00:26:27
podcast here
00:26:28
thank you very much for having for
00:26:29
having
00:26:34
us
