00:00:00
welcome welcome to the analytics at
00:00:03
Warton AI at Wharton podcast series on
00:00:05
artificial intelligence my name is Eric
00:00:07
bradow professor of marketing statistics
00:00:09
and data science here at the Wharton
00:00:10
School I'm also the vice dean of
00:00:12
analytics and I'm the one that's been
00:00:13
hosting this podcast series today's
00:00:16
episode is on an area that you know I've
00:00:18
said it many times even when my two
00:00:19
guests are not here I think that
00:00:21
artificial intelligence and machine
00:00:23
learning in combination with industry is
00:00:25
going to solve the healthcare problems
00:00:27
we have today Ai and Healthcare is such
00:00:29
an important area and I can't imagine
00:00:31
two colleagues better to talk with me
00:00:33
about that topic first I have my
00:00:35
colleague Marissa King Marissa is the
00:00:37
Alis y hung president's distinguished
00:00:40
professor at the Wharton School uh her
00:00:41
research has significantly contributed
00:00:43
to our understanding of a wide range of
00:00:45
pressing Healthcare issues ranging from
00:00:48
prescription drug abuse crisis to
00:00:50
clinician burnout Welcome to our podcast
00:00:53
it's a pleasure to be here and then next
00:00:55
and certainly last last not last but not
00:00:58
least is my friend and colleague Hamza
00:01:00
bastani Hamza is associate professor
00:01:02
operations information decisions as well
00:01:04
as a colleague in the statistics and
00:01:06
data science department her research
00:01:08
focuses on developing machine learning
00:01:10
algorithms for Learning and optimization
00:01:12
in healthcare and as I know very well
00:01:14
because I've interviewed her for other
00:01:15
podcasts uh she's done a lot of work
00:01:17
with Greece and Sierra Leon to deploy
00:01:20
algorithms at a Countrywide scale Hamza
00:01:22
welcome to the podcast as well thanks so
00:01:24
much for having me so let me start with
00:01:26
the beginning maybe uh Marissa I'll
00:01:28
start with you um for those people that
00:01:30
aren't familiar with the applications of
00:01:32
AI and Healthcare what are they if you
00:01:35
could just give us a broad overview of
00:01:37
the kinds of problems in healthcare
00:01:38
people are trying to use artificial
00:01:40
intelligence to use and uh to solve and
00:01:42
maybe it's in combination with machine
00:01:44
learning and other types of
00:01:45
algorithms yeah machine learning and
00:01:48
artificial intelligence have touched
00:01:49
almost all aspects of healthcare at this
00:01:51
point if you think of everything from
00:01:54
who and how you get reminders to pick up
00:01:56
prescriptions uh from who's reading your
00:02:00
Radiology reports to even how you're
00:02:02
being triaged in the emergency
00:02:03
department machine learning plays a key
00:02:05
role in all of those facets so why don't
00:02:09
maybe before I jump to Hamza here let me
00:02:11
ask you a question so um are the
00:02:13
reminders that we're all getting are
00:02:15
those being determined using some
00:02:17
optimal algorithm um is triage being
00:02:20
done in a more like not entirely 100%
00:02:23
human basis and um in terms of who is
00:02:27
reading my charts um like is that being
00:02:31
done a lot in an automated way so maybe
00:02:32
just to give us a baseline there yeah
00:02:34
and pretty much in every one of those
00:02:36
applications machine learning and AI is
00:02:38
playing a critical role so when you get
00:02:41
those reminders it's almost certainly
00:02:42
coming from an AI powered reminder the
00:02:44
same is true if you're thinking about uh
00:02:46
La reminders to pick up labs and get
00:02:48
your blood work done so that's the point
00:02:50
of care for for patients but clinicians
00:02:53
are also starting to deploy this for a
00:02:54
wide range of uses if you think about
00:02:57
Radiology reports that's arguably the
00:02:59
place where AI had the greatest
00:03:00
penetration so many many of our
00:03:02
Radiology reports are read Now by
00:03:04
machines and then finally if we think
00:03:06
about triaging in the emergency
00:03:08
department that's another important area
00:03:10
of application that's really reducing
00:03:12
overall length of stay within the
00:03:13
emergency department and we know that
00:03:14
when length of stay is reduced that has
00:03:16
important implications from everything
00:03:18
from um complications to long-term
00:03:20
mortality so AI is already playing a
00:03:23
critical role in a lot of domains of
00:03:24
healthcare so Hamza as someone who's I
00:03:26
know both a statistical methodologist
00:03:28
but also cares about the practical
00:03:30
application of the work how do you think
00:03:33
about the kinds of problems that Marissa
00:03:35
just laid out like do you try to solve
00:03:37
them in some idic setting do you try to
00:03:40
solve them with real data do you try to
00:03:42
kind of develop algorithms and then you
00:03:44
know try to run field experiments
00:03:45
actually launch them in the field how do
00:03:47
you think about your role as similar to
00:03:49
me as we're statistical methodologists
00:03:52
how do you think about your role in
00:03:53
helping solve these problems I think it
00:03:56
has to start with the data so I think in
00:03:57
healthcare there's a lot of variation so
00:04:00
only recently in the last couple decades
00:04:01
have we started digitizing the whole
00:04:03
health record but even now like EKG
00:04:05
readings for example aren't digitized in
00:04:07
most health systems and so um I think
00:04:10
the first thing we need to figure out is
00:04:12
is this a use case where an algorithm
00:04:14
given the data that is digitized is able
00:04:16
to do better than a human or at least
00:04:18
comparably to a human in a way that's
00:04:20
Equitable and also like transports well
00:04:22
to other Health Systems um and if that's
00:04:25
the case then we start thinking about
00:04:26
algorithms and rcts because there's lots
00:04:28
of other things that come up like
00:04:30
whether we're able to effectively
00:04:31
integrate it into the workflow whether
00:04:33
there's Buy in from stakeholders and so
00:04:35
on but I think it has to start with the
00:04:37
data so Marissa Hamza said a lot of
00:04:39
things I want to ask you about let me
00:04:41
let me fire these through in a rapid
00:04:43
fire kind of way how much are people in
00:04:45
Industry worried about
00:04:48
Equity it's easy to say you're you care
00:04:51
about it but how much does it impact
00:04:53
like let's imagine I could have an an
00:04:56
algorithm that improves the outcomes for
00:04:58
some population of people but not others
00:05:00
it's not Equitable but it's still from a
00:05:02
population perspective it could still
00:05:04
benefit Society how are people thinking
00:05:06
about Equity yeah I mean I think it's
00:05:08
certainly a point of concern in large
00:05:10
part because some of the key issues
00:05:12
arising around Equity with algorithms
00:05:14
that have already been deployed have
00:05:15
been made quite public um but at the
00:05:18
same time I do think it's a second order
00:05:19
consideration and that's going to I
00:05:21
think have a really important long-term
00:05:22
business implications because in the
00:05:24
long run as businesses start to deploy
00:05:26
these algorithms um when issues around
00:05:28
equity come to light I think it's going
00:05:30
to have really significant impact for
00:05:32
their bottom line so I think in very
00:05:34
short order it will go from being a
00:05:36
second order consideration to a primary
00:05:38
consideration or it should and one of
00:05:40
the things I've been talking to all of
00:05:41
our uh guests on the podcast series
00:05:43
about is what are the hindrances to
00:05:45
actually getting this stuff implemented
00:05:47
so what do you see you know there's
00:05:49
always whether you want to call it
00:05:50
algorithm aversion there's like well how
00:05:52
do I know this machine learning
00:05:54
algorithm is right how what kind of
00:05:56
barriers are you seeing or hesitancy
00:05:57
you're seeing in the field especially
00:05:59
when when it's
00:06:00
Healthcare yeah I think as a first order
00:06:04
consideration one of the biggest issues
00:06:06
is that as Hamza mentioned data is
00:06:08
certainly an issue but a bigger issue is
00:06:10
that to deploy these algorithms well you
00:06:12
need to actually have a very deep
00:06:14
understanding of clinical workflows for
00:06:16
them to be incorporated so even when
00:06:18
clinicians are willing to accept them um
00:06:20
you still have to integrate them into
00:06:22
workflows and that point of integration
00:06:24
seems to be one of the biggest
00:06:25
challenges at the moment so that's a
00:06:27
perfect segue to my question for Hamza
00:06:29
so
00:06:30
um I've that's probably the biggest
00:06:32
problem I've had in my career and you're
00:06:33
going to tell me now how to solve that
00:06:35
where I develop algorithms but you know
00:06:38
the fact is getting them actually in
00:06:39
someone's workflow is really tough how
00:06:42
did how do you think about that how did
00:06:43
you deal with that when you were working
00:06:45
with Greece with Sierra Leon to actually
00:06:47
you know you have an algorithm but you
00:06:48
can't just hand it off to someone and
00:06:50
say good luck right I think um that's an
00:06:53
excellent question and uh I'm doing my
00:06:55
best over here I dep I think it depends
00:06:58
on the complexity of the setting and how
00:06:59
well a human is trained to answer that
00:07:01
particular question uh compared to how
00:07:03
well an algorithm might be so I think in
00:07:05
a lot of public health questions uh
00:07:07
where you're trying to forecast demand
00:07:09
for Health Resources or you're trying to
00:07:11
figure out who which population to
00:07:13
screen and it's kind of rapidly evolving
00:07:15
that's kind of the work we've done um it
00:07:17
makes sense like even policy makers or
00:07:20
public health experts think that
00:07:21
algorithms are better suited to
00:07:23
assessing those situations because
00:07:24
there's large volumes of data um I mean
00:07:27
assuming that you've built an
00:07:28
interpretable system that they can look
00:07:29
into and you know um check the reasoning
00:07:33
of the algorithm I think in healthcare
00:07:35
it's harder because for example a very
00:07:37
famous example is this sepsis alarm
00:07:40
that's in a lot of icus that's being
00:07:41
deployed now uh and I think a big
00:07:44
challenge is those algorithms they're
00:07:45
not always aware of the private
00:07:47
information that the physician has so a
00:07:48
lot of Doctors Express frustration that
00:07:51
when the alarm goes off they already
00:07:52
knew that the patient was crashing
00:07:54
they're working actively to you know
00:07:55
stabilize the patient and this thing is
00:07:57
just you know irritating them and so
00:07:59
this causes something called alarm
00:08:00
fatigue so I think algorithms need to be
00:08:04
exactly as Marissa said designed in a
00:08:06
way that is aware of what knowledge the
00:08:08
human decision maker has and is able to
00:08:11
complement it in a useful way and that's
00:08:13
not how we do machine learning so that's
00:08:15
a fascinating idea let me let me give
00:08:17
you my example that I always like to
00:08:18
give in sports and then I'll translate
00:08:20
it to healthcare which is what is the
00:08:22
role of Scouts in sports like can't I
00:08:25
just measure everything and then just
00:08:26
the algorithm's going to tell me who the
00:08:28
better player is but they may have
00:08:30
private information that the algorithm
00:08:32
doesn't see and so I always talk about
00:08:33
blending the two can you talk to me
00:08:35
Marissa a little bit about how it's it
00:08:37
shouldn't really be AI or humans it
00:08:39
really should be Ai and humans in
00:08:41
healthcare yeah in healthcare in
00:08:43
particular this is not negotiable in
00:08:45
many ways in large part because of
00:08:47
Regulation so if you think about what's
00:08:48
happening at the Forefront of algorithm
00:08:50
development almost all the models at
00:08:52
this point are thinking about a
00:08:54
physician or clinician with an AI
00:08:57
co-pilot and that model I think is going
00:08:59
to be the one that is the one that's
00:09:00
most likely to Prevail both for
00:09:02
regulatory reasons which you in many
00:09:04
ways can't get around with but also
00:09:05
forgetting clinici and buy in which is
00:09:07
absolutely essential and seems to be a
00:09:09
huge hurdle at the moment so um Mera
00:09:12
just mentioned a word which is you know
00:09:13
an interesting word when we develop
00:09:15
algorithms which are whether it's
00:09:16
regulation or restrictions how do you
00:09:18
think about that when you're kind of
00:09:20
saying well here would be the optimal
00:09:22
solution in kind of an unrestricted
00:09:24
world with unlimited data and the
00:09:26
ability to do whatever you want or now
00:09:29
that there restrictions here's kind of
00:09:30
whether you want to call it the loss of
00:09:32
efficacy or here's the you know how do
00:09:34
you think about restrictions when you're
00:09:36
thinking about building algorithms I
00:09:39
think it's a great question uh I think
00:09:41
in most of these cases humans still have
00:09:43
this valuable private signal so we do
00:09:45
want them to override the algorithm uh
00:09:47
but I've heard a lot of concerns that um
00:09:49
you know people are worried about
00:09:51
malpractice lawsuits and things like
00:09:52
that and so they would rather if if an
00:09:54
algorithm is for example FDA approved
00:09:57
they would rather be more conservative
00:09:58
and Ur towards the algorithm so we've
00:10:01
had a lot of talk about algorithm
00:10:02
aversion but I think it also goes the
00:10:03
other way that sometimes there's over
00:10:04
Reliance on algorithms because it clear
00:10:07
creates a more established like uh paper
00:10:09
trail but ideally we would have better
00:10:12
training so Physicians are able to
00:10:14
understand what are the limits and the
00:10:16
capacities of these algorithms what is
00:10:18
the correlation between the information
00:10:20
that's bringing to the table and their
00:10:21
own private signal so that they're able
00:10:23
to more effectively combine these
00:10:25
signals in a I guess beian way uh and I
00:10:29
think that will be necessary to get
00:10:30
actually good outcomes so you just
00:10:32
mentioned something I was not aware of
00:10:33
maybe Marissa you could educate me and
00:10:35
our listeners here on SiriusXM and in
00:10:37
this AI at Wharton podcast series do
00:10:39
these algorithms have to be FDA approved
00:10:42
and if the answer is yes usually the
00:10:44
gold standard for approval is randomized
00:10:47
experiments can you run a is it I
00:10:49
mean imagine running a randomized
00:10:52
experiment and now people are dying so
00:10:54
how do you think about running like
00:10:56
getting kind of the gold standard of
00:10:58
evidence in cases is where you're
00:10:59
testing an algorithm how is that thought
00:11:01
about yeah all these algorithms require
00:11:04
FDA approval did know they do and there
00:11:06
have been more than 500 algorithms that
00:11:08
have already been approved for use in
00:11:10
clinical settings and so you can get a
00:11:11
sense of just how many algorithms there
00:11:13
are that exist and I think the lack of
00:11:15
clinical integration is also highlighted
00:11:17
by how few of those are actually used in
00:11:19
practice um so certainly um regulation
00:11:22
is a key point for this um and there's a
00:11:24
lot of debate over how well they're
00:11:26
actually being regulated so the current
00:11:28
standard is trying to compare them to
00:11:30
existing clinici and performance um but
00:11:32
we know right that algorithms that
00:11:34
particularly when they're exposed to new
00:11:35
data will often times deteriorate so
00:11:37
you're like the question is both how
00:11:39
well does it work compared to clinicians
00:11:41
but also on how large of data sets and
00:11:43
those are two current criteria that are
00:11:45
really key um the other interesting
00:11:48
piece about on the regulation side is my
00:11:50
under and I'm not a lawyer uh but my
00:11:52
understanding is that ultimately
00:11:55
responsibility still lies with a
00:11:56
clinician so if there is going to be a
00:11:58
loss suit um even if an algorithm does
00:12:01
have FDA approval the final the
00:12:04
responsibility finally lies with the
00:12:06
clinician um so that's another
00:12:08
Regulatory and legal challenge in terms
00:12:10
of getting large scale deployment so
00:12:13
could you give us a sense maybe Marissa
00:12:15
um of these I think you mentioned about
00:12:17
500 algorithms could you give our
00:12:18
listeners here on SiriusXM just a sense
00:12:21
of like what are these algorithms like
00:12:23
like what would be if you want to think
00:12:25
about them from the most impactful and
00:12:27
efficacious to the least What would near
00:12:29
the top and what would be like I don't
00:12:31
know like for example let's imagine you
00:12:34
had something that could red EKGs that
00:12:35
could prevent heart attacks at a much
00:12:38
higher rate than a h that would seem to
00:12:41
be be given the frequency of heart
00:12:43
attacks that would seem to be pretty
00:12:44
efficacious and important can you give
00:12:45
us a sense of like or the way I like to
00:12:47
describe it I'm an effect Siz person
00:12:49
tell me the things that you think the
00:12:51
algorithms that are having the big
00:12:52
effect sizes on a large population yeah
00:12:54
the algorithms that seem to be enjoying
00:12:57
the greatest success in having the
00:12:58
biggest impact on Healthcare and
00:13:00
healthare outcomes do seem to be the
00:13:01
ones that are focused on Radiology so if
00:13:03
you imagine that you show up at the
00:13:06
emergency department and that you may be
00:13:08
having a stroke um the ability to get
00:13:10
that scan and read that scan quickly is
00:13:12
critical so time is of the essence in
00:13:14
Saving Lives um and the deployment of AI
00:13:17
to you read Radiology reports which then
00:13:19
do set an alert that speeds up the rest
00:13:22
of clinical care so a human is looking
00:13:24
at it um but that acceleration seems to
00:13:27
have a huge impact on both Health
00:13:29
outcomes and the cost and quality of
00:13:31
care so I would put the more Radiology
00:13:34
focused machine red ability to read um
00:13:38
various scans of whatever nature those
00:13:40
may be or Radiology reports seem to be
00:13:41
the area where there's been the greatest
00:13:43
penetration um so I think that's where
00:13:46
the greatest impact lies where things
00:13:47
get trickier or when you think about
00:13:49
things that um require a deeper
00:13:52
integration into clinical care and I
00:13:54
think where the they're facing the
00:13:55
biggest uh point of resistance is
00:13:57
actually if we think about things that
00:13:58
are directly patient facing MH so uh
00:14:02
Marissa just mentioned something that I
00:14:04
haven't thought about for a while I used
00:14:05
to spend a lot of time working on
00:14:07
methods that I called realtime
00:14:08
approximation methods I haven't worked
00:14:10
on these in a while but as Marissa said
00:14:12
someone comes in you know I hate to put
00:14:14
it this way but I'll use the word basian
00:14:16
um I can't run my basian mcmc sampler
00:14:19
overnight to get some result that gives
00:14:22
me something and of course the patient
00:14:23
may have died by then how much do you
00:14:25
think about you know um as we as AC mics
00:14:29
are supposed to in theory we're supposed
00:14:31
to come up with good answers the fact
00:14:33
that they may not be real Time That's I
00:14:35
hate to put this way but that's someone
00:14:36
else's problem how much do you think
00:14:37
about that when you're trying to come up
00:14:39
with Solutions like when you were
00:14:40
working with coid testing you know
00:14:43
someone's coming through the skin and
00:14:44
you can't say well give me a few hours
00:14:45
why don't you just sit over here on the
00:14:47
side while we decide which you know what
00:14:48
the likelihood of you having Co is how
00:14:50
do you think about real time nature of
00:14:52
things yeah I think um sometimes that
00:14:55
changes uh the algorithm that you use so
00:14:57
sometimes it's better to use do
00:14:58
something that's uh computationally
00:15:00
easier uh that might be slightly less
00:15:02
accurate um because it's actually
00:15:04
practical and another big thing we do is
00:15:06
batching like batched updates uh so you
00:15:10
can't so like uh like Marissa was saying
00:15:12
one of the issues I think the FDA is not
00:15:13
monitoring is making sure that these
00:15:15
algorithms actually evolve over time as
00:15:17
the patient population changes and
00:15:19
adapts to thing things to events like
00:15:21
coid uh I think that should be part of
00:15:23
the regulation but isn't yet uh but we
00:15:25
do do need to monitor these algorithms
00:15:27
as like uh the ICD system changes as uh
00:15:30
scanning imagery uh changes and so on uh
00:15:33
and I think doing batched updates on as
00:15:36
fresh data comes in is kind of a
00:15:38
critical part of that and that makes it
00:15:39
that kind of solves a computational
00:15:41
issue I see uh we're here on the
00:15:43
analytics at warten AI at Warton podcast
00:15:45
series we're talking about Ai and
00:15:46
Healthcare I'm here with my colleagues
00:15:47
mura King the alisy hung president's
00:15:50
distinguished professor at the Wharton
00:15:51
School and Hamza bastani associate
00:15:53
professor of operations information
00:15:54
decisions and statistics and data
00:15:56
science um let me ask you a question
00:15:58
question when we as
00:16:00
academics approach these problems maybe
00:16:02
even approach companies I can imagine
00:16:05
one of two reactions like what are you
00:16:07
doing here or number two oh thank you
00:16:10
the academics have come to help us um so
00:16:14
what is the reaction when we as I mean
00:16:17
it's not that we we care I mean I think
00:16:18
I can safely the three of this people in
00:16:20
this room care more about our the way
00:16:22
our research impacts practice then
00:16:24
probably we're in the top desile of
00:16:26
Wharton faculty but what's the reaction
00:16:28
of Industry when academics want to get
00:16:30
engaged
00:16:31
here yeah I I think that the engagement
00:16:35
of academics is absolutely critical and
00:16:36
I think this is where analytics at
00:16:38
Wharton has particularly a huge role to
00:16:40
play if you think about the nature of
00:16:41
the Health Care system in general it's
00:16:43
highly fragmented fragmented with
00:16:45
stakeholders having various um strong
00:16:49
positions for a variety of different
00:16:50
reasons so you have right insurers you
00:16:52
have Regulators you have the people
00:16:54
delivering care um so there's a many
00:16:57
many players in this space and to solve
00:16:59
healthcare's biggest challenges you need
00:17:01
to get them all to work together
00:17:02
collaboratively and because many times
00:17:04
the positions from which they're arguing
00:17:06
right they may have um different
00:17:08
incentives or misaligned incentives and
00:17:09
so having a neutral convener who can
00:17:11
bring together um all those parties and
00:17:14
Tackle them from a place um of
00:17:16
scientific basis and a place of
00:17:18
neutrality and act as a really convening
00:17:20
organization is really really critical
00:17:23
um so certainly algorithmic tools can be
00:17:25
useful um but in order for them to have
00:17:28
broad penetration to tackle the most
00:17:30
pressing challenges you need
00:17:31
coordination among stakeholders and I
00:17:33
think that's where Academia can play a
00:17:34
really important role so actually
00:17:36
Marissa reminded me of something that's
00:17:38
in neither of your bios but it's going
00:17:39
to change after today I should have
00:17:42
mentioned even more importantly that
00:17:43
analytics at won is launching a
00:17:45
healthcare analytics lab under Hamza and
00:17:47
Marissa's leadership for those people
00:17:49
interested which is I would think
00:17:51
everybody you could go to analytics.
00:17:53
won. up.edu Warton Healthcare analytics
00:17:57
lab and see about about all the work
00:17:59
that we're that Hamza and Marissa will
00:18:01
be leading Us in could you talk uh Hamza
00:18:04
about the role of uncertainty like an
00:18:06
algorithm comes up with a suggestion or
00:18:09
a recommendation but one of the things
00:18:11
that's important is you know if it's
00:18:14
640 does the clinician have the right to
00:18:17
know that this is 6040 like the model
00:18:19
saying this is better than this but
00:18:21
maybe it's not that much better how do
00:18:24
you think about that when you're kind of
00:18:26
whether it's a dashboard you're creating
00:18:28
or you're providing recommendations
00:18:29
you're doing some form since you're an
00:18:31
oid which means you also care about
00:18:32
optimization when you think about
00:18:34
optimization what is the role and how do
00:18:36
you think about uncertainty I think it's
00:18:38
super important as you probably agree uh
00:18:41
but I think uh so I'll talk about the
00:18:43
humans first so when you're thinking
00:18:44
about human AI collaboration I think
00:18:46
uncertainty is one of those critical
00:18:47
pieces of information that you have to
00:18:49
convey so they know when they should
00:18:51
override it and when they shouldn't um
00:18:54
but I think one of the challenges has
00:18:56
been that in behavioral experiments when
00:18:58
you show uncertainty people often tend
00:19:00
to over trust the algorithm because they
00:19:02
think oh not only did it give me a point
00:19:04
estimate but it also gave me a measure
00:19:06
of uncertainty uh so I think this is
00:19:08
part of the training thing that has to
00:19:09
happen that uh we want people to
00:19:12
intervene in a preferential way when the
00:19:14
algorithm is uncertain for optimization
00:19:17
it's a little bit easier we've built a
00:19:19
lot of tools in in stochastic
00:19:20
optimization that account for
00:19:22
uncertainty uh so that we're targeting
00:19:24
for example the right quantile of
00:19:26
uncertainty rather than just using the
00:19:27
mean or estimates because typically in
00:19:29
underserved populations we'll have a lot
00:19:31
more uncertainty and we don't want that
00:19:33
to result in you know them getting uh
00:19:36
fewer
00:19:37
resources um the next question I'm going
00:19:39
to ask Marissa is my favorite question
00:19:41
to ask anybody and when it comes to
00:19:43
analytics and I can't you know usually
00:19:45
with me it's about some sort of food or
00:19:47
beverage so I'm taking outside my
00:19:49
personal life I'm talking about my
00:19:50
professional life here if you think
00:19:52
about the way that AI can impact in
00:19:54
healthcare I'm going to give you one of
00:19:56
three options okay okay you can have
00:19:59
better
00:20:00
data you can have better mathematical
00:20:03
models or you could have better
00:20:05
adherence by people in the field to what
00:20:08
we as academics you can't pick all three
00:20:10
you you we all want all three which one
00:20:13
is the big if you'd like impedance right
00:20:16
now to advances we're making is it lack
00:20:19
of data lack of better models or is it
00:20:22
we've got all that stuff just you know
00:20:24
these damn people just won't listen to
00:20:26
us I think it's a ladder um so if you
00:20:29
think about the data challenges the data
00:20:31
challenges still Loom large um but we
00:20:33
have now the ability to work with large
00:20:35
enough data sets that this is starting
00:20:37
to become a solvable problem I think
00:20:39
particularly on the electronic health
00:20:41
record front it's still a challenge in
00:20:43
the sense that most of the time we're
00:20:44
going hospital by Hospital deploy these
00:20:46
things um but it's still like the data
00:20:48
is okay um the second piece is the
00:20:50
algorithms seem right like the
00:20:52
algorithms I don't think are the
00:20:53
challenge uh if you just even look at
00:20:55
how many your FDA approved right and I
00:20:57
feel like h and I could sit down
00:20:59
probably tomorrow and write an algorithm
00:21:00
that would certainly improve care in
00:21:02
many many ways um the big is I'm
00:21:04
counting on that but H the biggest
00:21:07
challenge is really implementation and
00:21:09
integration and the same is true as was
00:21:12
true with data in the sense that most of
00:21:13
the time these things have to be rolled
00:21:15
out system by System hospital by
00:21:16
Hospital doctor's office by doctor's
00:21:18
office um and until and even within
00:21:21
those roll outs um there's a lot of
00:21:23
clinical resistance so I as I watched
00:21:25
these things be deployed in various
00:21:27
settings right um I can't tell you like
00:21:29
I don't even want to disclose how often
00:21:31
times the algorithm is overridden um
00:21:33
just for and medicine is particularly
00:21:36
challenging in the sense that clinicians
00:21:40
have a lot of expertise and a lot of
00:21:41
authority and um there's a strong
00:21:44
difference to that expertise and
00:21:45
Authority as there should be but it
00:21:47
makes um changing the way that they
00:21:49
think um and questioning their judgment
00:21:52
um particularly difficult so I think
00:21:53
it's the latter if I could improve
00:21:55
anything it would be adoption and
00:21:57
implementation so HS I know an issue
00:21:59
you've thought quite a bit about and are
00:22:00
planning on doing a lot of work on is
00:22:02
kind of educating people in these
00:22:05
methods so what's the process like I my
00:22:08
brother's a cardiologist I'm pretty sure
00:22:11
if you I mean he's also a researcher so
00:22:13
I'm pretty sure if you present him the
00:22:14
ideas of confidence intervals and
00:22:16
prediction methods he might get it um
00:22:19
but many doctors that's not their job
00:22:21
like so how do you present information
00:22:24
or how do we even educate this large
00:22:27
population on these algorithms cuz you
00:22:29
use the word before I think
00:22:30
interpretable and explainable maybe
00:22:32
they're like how do I know this is just
00:22:34
some black box data's coming in
00:22:35
something's coming out how do you think
00:22:37
about your role in kind of educating the
00:22:41
if you'd like the distribution channel
00:22:42
in this case the Physicians on these
00:22:45
methods I think uh one big challenge is
00:22:48
that people don't know what training
00:22:49
data was used to train the algorithm and
00:22:51
I mean giving it to them wouldn't be
00:22:53
super useful anyway because it's not
00:22:55
interpretable but I think um a lot of
00:22:57
the reasons that uh we want humans to
00:23:00
override these algorithms is because the
00:23:02
data that they're seeing is possibly an
00:23:04
outlier or has a different distribution
00:23:06
uh than the data that it was trained on
00:23:08
so for example maybe they didn't see
00:23:09
people of this particular type um or
00:23:12
maybe the Imaging system that they're
00:23:14
using now is a little bit different from
00:23:15
uh the one that was used in the training
00:23:17
data uh and I think partly it's on us to
00:23:20
provide these signals but I don't think
00:23:22
they're immediately interpretable so I
00:23:23
think some kind of training where we
00:23:25
show them historical examples of when
00:23:27
the algorithm went wrong and when they
00:23:29
might have had a right prior and when
00:23:31
the algorithm didn't go wrong and when
00:23:32
they overwrote it even know even just
00:23:34
showing them their own decisions
00:23:35
historically and seeing when they um
00:23:38
should or shouldn't have overwritten the
00:23:40
algorithm like would help a lot because
00:23:41
people need feedback and there isn't
00:23:44
spec like healthcare is very expensive
00:23:46
and these people's time is very
00:23:47
expensive so like setting aside the time
00:23:49
to do that training I think is important
00:23:51
and costly well that's what that was
00:23:53
going to be my next question for Marissa
00:23:54
do you see whether it's Hospital groups
00:23:56
or do you see I don't know the American
00:23:57
American Medical Association or the
00:23:59
American Association of Surgeons I'm
00:24:00
just I don't know I think that's assume
00:24:02
that's a real one I mean do you see them
00:24:04
coming to people like yourselves and say
00:24:07
help us train Us in Mass come up with
00:24:10
video series or something so that these
00:24:13
people that are overburdened overworked
00:24:15
in general you could argue underpaid
00:24:17
like how are we going to train them
00:24:19
because we want to save
00:24:21
lives yeah I think unfortunately or
00:24:24
fortunately depending on how you look at
00:24:25
it the greatest opportunity is actually
00:24:27
just going to come from pain um we know
00:24:29
that we're facing a clinical healthcare
00:24:31
worker shortage so by 2033 um there's
00:24:33
going to be a massive shortage of
00:24:35
healthcare workers and if you already
00:24:36
look at the issues around burnout um
00:24:39
with a majority of clinicians burnt out
00:24:41
um there's already an enormous amount of
00:24:43
pain and overwork and so I think in many
00:24:45
ways that we're most likely to see
00:24:47
adoption coming coming from the bottom
00:24:48
up where clinicians asking right and I I
00:24:52
didn't talk about clinical notation but
00:24:54
that's a huge area in which um large
00:24:56
language models and machine learning can
00:24:58
play a role by starting to do some of
00:25:00
the work right and the tasks that
00:25:01
clinicians don't need to be doing and by
00:25:03
saving them that time right then you can
00:25:05
allow them to reconnect with patients
00:25:07
deliver higher quality care and so I
00:25:09
think in many ways um education is
00:25:11
certainly going to be more important but
00:25:13
based on my experience people are really
00:25:14
much more willing to adopt things when
00:25:16
there's a a real need for them and right
00:25:18
now in healthcare there's a strong need
00:25:20
for help um to augment clinical
00:25:22
workflows but particularly with the
00:25:23
things that clinicians don't need to be
00:25:24
doing and so I think very soon we're
00:25:27
going to be seeing
00:25:28
yeah I just had an example the other day
00:25:29
where someone I'll just say with a large
00:25:31
Investment Bank told me that all of
00:25:34
their meetings are now recorded and now
00:25:37
the agent or the investment adviser
00:25:39
doesn't need to take notes because all
00:25:41
of that is automatically put into the
00:25:43
system any types of decisions they made
00:25:45
get automatically implemented because
00:25:47
they're now in this case voice recorded
00:25:49
and automatically and so now the
00:25:50
investment adviser can spend time on
00:25:52
training and other forms of doing her
00:25:54
his or their job actually better which
00:25:57
is a great really great point so maybe
00:25:59
in the last minute or two that we have
00:26:00
let me ask each of you i' I'm trying to
00:26:02
ask each person in this podcast series
00:26:04
about this so let's say we're sitting
00:26:06
here 10 years from now which the
00:26:08
invitation is open we're sitting here 10
00:26:10
years from now that'll be my 38th year
00:26:12
at Wharton we're sitting here 10 years
00:26:13
from now what are we talking about that
00:26:16
either you or you think the field of AI
00:26:19
and healthc Care from an algorithmic or
00:26:21
you know data perspective what have we
00:26:23
seen over the past 10 years what's your
00:26:26
hope and dream at Le least even if it's
00:26:28
not going to happen I think over the
00:26:30
past 10 years we will definitely have um
00:26:32
adoption on on notes for example or
00:26:34
things that Physicians don't want to do
00:26:36
I think we'll have more adoption in
00:26:38
developing countries where there isn't I
00:26:40
know we're we're short staffed even in
00:26:41
the US but uh where there isn't as much
00:26:44
health worker staff to reach underserved
00:26:46
communities so automation is sort of
00:26:48
being more adopted in those locations uh
00:26:51
and I think definitely for things like
00:26:52
Radiology I think we'll still be facing
00:26:54
challenges with you know things like
00:26:56
alarm fatigue and adoption for places
00:26:58
where we have clinician experts and I
00:27:00
think uh with models like gbt coming out
00:27:03
I think it'll be easier to educate
00:27:05
people on machine learning um and be
00:27:07
able to you know better enable this
00:27:09
human AI collaboration but I feel like
00:27:11
that is going to be a big challenge even
00:27:12
10 years from now and what do you think
00:27:13
mer so what are we going to see out
00:27:14
there whether it's in the field or we as
00:27:16
academics are doing I mean both homs and
00:27:19
I's goal at the analytics lab is to try
00:27:21
to improve access to care and quality
00:27:23
care for all and I think that that's
00:27:25
hopefully where analytics will take us
00:27:27
well I'd like to thank both of you for
00:27:29
our podcast series episode here on AI
00:27:31
and Healthcare i' like to thank again my
00:27:33
uh colleague Marissa King and Hamza
00:27:34
basani thank you again for joining me
00:27:37
thank
00:27:43
you
