00:00:01
we're here today with Wharton professors
00:00:03
Joe Simmons and Kate Massey to talk
00:00:05
about some of their new research which
00:00:07
focuses on algorithm aversion and how to
00:00:09
stop it Joe and Katie thanks for being
00:00:11
with us today
00:00:12
thank you so first of all could you give
00:00:14
us a brief summary of this research and
00:00:16
also I know this is actually the papers
00:00:17
actually a follow-up to something that
00:00:19
you had done previously yeah so so we're
00:00:22
studying a phenomenon called an
00:00:23
algorithm aversion which is basically
00:00:24
the tendency for people to not want to
00:00:28
follow specific evidence-based rules
00:00:31
when they make decisions even though a
00:00:34
lot of the research that we do in
00:00:35
judgment decision-making shows that
00:00:37
that's exactly the way that you should
00:00:38
be making judgments and forecasts so a
00:00:41
lot of people just want to you know rely
00:00:43
on their gut or go to the seat of their
00:00:45
pants they don't want to rely on
00:00:46
consistent evidence-based rules and they
00:00:48
should and so we've been studying for a
00:00:50
couple years now you know first of all
00:00:52
why don't they or under what
00:00:54
circumstances don't they want to rely on
00:00:55
these algorithms and then our second
00:00:57
paper the one you you asked about is
00:00:59
about how to get people to be more
00:01:01
likely to rely on algorithms and so the
00:01:05
answer to our first question we
00:01:07
basically found that people are if you
00:01:08
tell them you're going to make a
00:01:10
forecast an algorithm is going to give
00:01:12
you some advice or you can go with your
00:01:16
own opinion what do you want to do and
00:01:18
you just ask them that they're actually
00:01:20
okay with saying I'll use the algorithm
00:01:22
however once you give them some practice
00:01:25
and let them see how their algorithm
00:01:27
performs now all of a sudden they don't
00:01:29
want to use it anymore and that's
00:01:31
because they see the algorithm make
00:01:33
mistakes and once they see algorithms or
00:01:36
computers make mistakes they don't want
00:01:38
to do it anymore even though the
00:01:40
algorithm or computer is going to make a
00:01:42
smaller mistake or more infrequent
00:01:44
mistakes than they themselves are going
00:01:46
to make the algorithm supposed to be
00:01:47
perfect right so people want algorithms
00:01:50
to be perfect and they expect them to be
00:01:51
perfect even though of course what we
00:01:54
really want is for them to simply be a
00:01:56
little more a little better than than
00:01:59
the humans and so our second paper our
00:02:03
first paper is kind of pessimistic and
00:02:05
shows that once people see the
00:02:06
algorithms do its thing they don't want
00:02:08
to use it our second paper shows that
00:02:10
actually you can get people to use
00:02:11
algorithms as long as you give them a
00:02:13
little bit of
00:02:14
control over ETSU say the algorithm
00:02:16
tells you that this person is going to
00:02:19
have a GPA of 3.2 what do you think
00:02:22
their GPA is going to be and they don't
00:02:24
want to just go with a 3.2 the algorithm
00:02:26
says but if you say you can adjust it a
00:02:28
little bit you can adjust it by 0.1 then
00:02:31
they're like okay I'm fine to use the
00:02:33
algorithm and so we basically find as
00:02:35
long as you give people a little bit of
00:02:36
control over these things they're more
00:02:38
likely to use them and that's that's
00:02:40
pretty good news so this is we
00:02:46
operationalize this in experimental
00:02:48
context but we're motivated by a
00:02:49
real-world context so some of the early
00:02:52
ideas for this research came from
00:02:54
working with companies where we would go
00:02:56
in with models for decision-making and
00:02:59
in particular this is about hiring and
00:03:01
recruiting new employees and based on
00:03:03
many years worth of data and some pretty
00:03:05
good analytics we'd have advice and we
00:03:08
were sure that we had the best advice
00:03:09
going and yet those organizations would
00:03:11
be reluctant to use it because they want
00:03:13
to rely on just their intuition they're
00:03:15
reluctant to use those models so it's
00:03:17
very common and hiring it's very common
00:03:19
in performance evaluation it's even
00:03:21
common increasingly in some fields where
00:03:23
they automate decision-making like how
00:03:25
to manage a hedge fund or what should
00:03:28
the sales forecast be for some product
00:03:30
so those are all places where
00:03:32
increasingly automatically generated
00:03:34
forecasts or advice is available we call
00:03:36
it an algorithm and the final
00:03:39
decision-maker has discretion over
00:03:40
whether they listen to that advice or
00:03:42
whether they use their own or they use
00:03:43
some blend so your key takeaway was
00:03:46
basically that people are a little less
00:03:47
averse to using algorithms that they
00:03:49
have some control but there is a
00:03:51
conclusion that surprised you in how
00:03:53
much control you had to give them or you
00:03:56
could give them to make them feel better
00:03:57
so tell us a little bit about that well
00:03:59
we were agnostic on how much control
00:04:01
would be necessary to kind of get them
00:04:02
to buy end the downside of given and
00:04:04
controls they start degrading the
00:04:06
algorithms they make they're not as in
00:04:08
most domains about as good as the model
00:04:09
and so the more of their opinion is in
00:04:12
there the worse it performs and so in
00:04:14
some sense you'd like to give them as
00:04:15
little control as possible and yet still
00:04:17
have them Buy in we didn't know what the
00:04:19
answer that would be and we got early
00:04:21
evidence that it wasn't gonna be very
00:04:22
much as we started testing the limits of
00:04:24
it and we found that we could give them
00:04:26
just a little bit of control
00:04:27
move something around 5% or so and they
00:04:31
would be much more interested in using
00:04:32
the algorithm and then if you give them
00:04:34
more it doesn't increase their list at
00:04:36
all it's just give them a little bit
00:04:38
it's about the same as given the
00:04:39
moderate influence yeah and cave was
00:04:43
mentioning what's nice about that is
00:04:45
that every when they adjust algorithms
00:04:47
they make them worse but if they can
00:04:49
only adjust it this much they can only
00:04:50
make it is that much worse and they're
00:04:52
more like entrance they're just more
00:04:53
likely to use it in that case their
00:04:55
final judgments will end up being
00:04:57
correlated with the algorithm close to
00:04:59
perfectly and so we can't get people to
00:05:02
use algorithms 100% we can get them to
00:05:04
use algorithms 99% and that bat
00:05:06
massively improved their judgments so
00:05:09
tell me a little bit about so with this
00:05:10
paper like applying it in real life like
00:05:12
how could if I'm a business owner or
00:05:14
even someone who's going to be charged
00:05:16
with using one of these algorithms how
00:05:18
might I apply this research in real life
00:05:20
so the the overarching lesson would be
00:05:23
that you don't simply impose a
00:05:26
monolithic model or blackbox model and
00:05:28
say this is how you use judgment people
00:05:29
will fight that this is how you this is
00:05:31
how you should cause by your decision
00:05:33
making people will fight that you want
00:05:35
to let them have discretion and that's
00:05:37
going to look different in different
00:05:38
places so consider graduate school
00:05:40
making admissions they they rank their
00:05:44
applicants and at some point they cut a
00:05:46
line and they make exceptions they move
00:05:47
people around you can automate some of
00:05:49
that process even if you use their
00:05:51
judgment to provide inputs to the model
00:05:55
you can use them a model automatic model
00:05:57
to say these are the folks that you
00:05:58
should take on one hand you could say
00:06:00
here's a model this is what it says take
00:06:02
it or leave it
00:06:03
we're going to automate the process
00:06:04
you'll basically have a revolt on your
00:06:06
hand
00:06:07
but if you say here's a model its
00:06:08
advisory we suggest that you consider it
00:06:11
if you want to move things around move
00:06:12
them around we've actually worked with
00:06:14
schools on in exactly this way and what
00:06:16
you find is that they're a little
00:06:17
skeptical early on they lean on the
00:06:19
model some and over time they lean on
00:06:21
the model more and eventually they're
00:06:23
practically using the entire model as it
00:06:25
is even though they have discretion to
00:06:27
change as much as they want so sort of a
00:06:28
get-to-know-you process there's very
00:06:30
much a good to know you process with it
00:06:31
and now how important is it with these
00:06:34
models I mean just I would also make the
00:06:36
presentation if it will be important
00:06:37
that you make sure that people know that
00:06:39
they have this control
00:06:40
but you're sort of presenting it in such
00:06:42
a way that like here's what you can
00:06:43
control and here's how much control or
00:06:45
can it be a little more available Matt
00:06:47
yeah I basically think the important
00:06:49
thing is to avoid an all-or-nothing
00:06:51
framing like you are all you you have to
00:06:54
stick with the algorithm 100% of the
00:06:56
time if people think that based on how
00:06:59
you've described it they are going to
00:07:01
push back but instead you can frame it
00:07:03
as you know even like ninety nine
00:07:06
percent of time we're gonna go with the
00:07:07
algorithm but you have the option to
00:07:08
change the algorithm or to not go with
00:07:11
the algorithm at a given moment I think
00:07:12
that that's going to go and make people
00:07:14
a lot more amenable to using it I mean
00:07:17
in other context of this might matter is
00:07:18
like self-driving cars I mean you can
00:07:20
imagine people saying like I'm not
00:07:22
feeling comfortable being in a
00:07:23
self-driving car if they have no control
00:07:25
whatsoever but if they say well there's
00:07:27
this sort of thing you can do it's a
00:07:29
little bit difficult and unusual but
00:07:31
there's this thing you can do to gain
00:07:32
control over the car in circumstances
00:07:35
where you might need to do it now we
00:07:37
found that people never need to use this
00:07:38
but it does exist we would predict that
00:07:41
in that circumstance you will be much
00:07:42
more amenable to getting in getting in a
00:07:45
self-driving car because there's some
00:07:47
control it's like autopilot is usually
00:07:50
safer than real pilots but people want a
00:07:52
pilot there even though lots of plane
00:07:54
crashes are due to pilot error they feel
00:07:56
better about that and so I think our
00:07:58
research sort of speaks to that a bit
00:08:00
now are there other stories in the news
00:08:03
that might apply to this research
00:08:04
Grenoble self-driving cars have
00:08:05
obviously been in the news quite a bit
00:08:07
how about election forecasts yeah so I
00:08:10
think you know back in November of
00:08:13
course we had a presidential election
00:08:15
that surprised the world and there are a
00:08:20
bunch of people out there predicting
00:08:21
based on past polling information what
00:08:24
the election was going to to look like
00:08:26
like probably the most famous cases made
00:08:29
silver who's who writes for
00:08:30
fivethirtyeight.com
00:08:31
and he basically said that there is a
00:08:33
70% chance that Hillary Clinton would
00:08:36
win the election and a 30% chance that
00:08:37
Donald Trump would win and of course you
00:08:41
know Donald Trump won and there was a
00:08:43
lot of pushback against Nate Silver at
00:08:45
the end of it being like you know you
00:08:46
were you know you were wrong and and we
00:08:50
think in part of your model with your
00:08:52
model your model was wrong
00:08:53
and the thing is it wasn't necessarily
00:08:56
wrong 30% happens 30% of the time but I
00:09:00
think that it's it's the kind of thing
00:09:03
where when individual I think when
00:09:05
individual pundants
00:09:06
go out there and say one thing is going
00:09:08
to happen I don't think they get as much
00:09:09
blowback as when the person who uses
00:09:11
statistics and the model and an
00:09:13
algorithm that people expect to be right
00:09:15
100% of the time when they actually wind
00:09:17
up getting it wrong in that particular
00:09:19
case so I think the blowback that we've
00:09:21
seen in in the direction of nate silver
00:09:24
has been in line with what we found
00:09:26
before so it circles back to their first
00:09:27
paper where people are just much harder
00:09:29
on models and algorithms from there than
00:09:33
they are on people they're just more
00:09:34
forgiving they we've explored it a
00:09:36
little bit but the bottom line is that
00:09:38
they're held to a higher standard yeah
00:09:40
now does that I mean does that make
00:09:42
sense or is that I mean I would think I
00:09:44
can see what people would feel that way
00:09:46
although no one's perfect or anything is
00:09:48
perfect well you want there a variety of
00:09:50
reasons we think people do this one of
00:09:52
them is that they believe that people
00:09:54
can improve over time whereas a model is
00:09:56
relatively fixed and that's the way they
00:09:59
feel in a way both of those things are
00:10:01
not necessarily true models can improve
00:10:02
over time and people don't necessarily
00:10:04
improve over time so the psychology of
00:10:06
that I think is compelling but it's not
00:10:09
necessarily correct the certainly would
00:10:12
be some settings where people can
00:10:14
improve more than a model but we think
00:10:16
that people have that intuition more
00:10:17
than actually should now is anything -
00:10:20
kind of sets this research apart from
00:10:21
other work that's been done in this area
00:10:24
that we're not the first to talk about
00:10:29
the difference between model judgment
00:10:31
and human judgment it's been established
00:10:34
for decades now that models are quite
00:10:36
good we're relatively early into trying
00:10:39
to understand why that is and then how
00:10:41
you fix it yeah so right so not many
00:10:45
people have documented previously the
00:10:48
reasons why people are averse to using
00:10:50
algorithms there's been some anecdotal
00:10:52
research there's been some writings
00:10:54
about how people don't like these things
00:10:55
but no one's really looked at it
00:10:57
systematically before and so that's
00:10:58
that's what we sort of started and again
00:11:01
back to the motivation motivation was we
00:11:03
work with organizations we want them to
00:11:05
use more models we need to know how to
00:11:06
break down that
00:11:07
is you don't you can't prescribe
00:11:09
anything into you better understand of
00:11:10
why and exist because you give someone a
00:11:13
model and I don't use it you might as
00:11:14
well know delirious frustrated we did
00:11:17
never wrote a few times and now is there
00:11:19
anything what's next for this research
00:11:21
we took a couple of ideas are you know
00:11:24
we continue to play with some factors
00:11:27
that might contribute to the people's
00:11:29
reluctance to use acronyms but we also
00:11:31
want more real-world tests of it so if
00:11:33
we go work with professionals with real
00:11:36
money at stake do they fall in the same
00:11:39
biases and is there other ways for us to
00:11:42
help them so we have a couple of
00:11:44
organizations we've talked to over time
00:11:45
and they're interested in running
00:11:47
experiments on their employees or their
00:11:48
own their customers to see if what we
00:11:50
see in the lab takes place in the field
00:11:52
as well great thank you both so much for
00:11:54
being with us today yeah
00:12:07
you
00:12:15
[Music]
