00:00:01
we're here today with Wharton professors Joe Simmons and Kate Massey to talk
00:00:05
about some of their new research which focuses on algorithm aversion and how to stop it Joe and Katie thanks for being
00:00:11
with us today thank you so first of all could you give us a brief summary of this research and
00:00:16
also I know this is actually the papers actually a follow-up to something that you had done previously yeah so so we're
00:00:22
studying a phenomenon called an algorithm aversion which is basically the tendency for people to not want to
00:00:28
follow specific evidence-based rules when they make decisions even though a lot of the research that we do in
00:00:35
judgment decision-making shows that that's exactly the way that you should be making judgments and forecasts so a
00:00:41
lot of people just want to you know rely on their gut or go to the seat of their
00:00:45
pants they don't want to rely on consistent evidence-based rules and they should and so we've been studying for a
00:00:50
couple years now you know first of all why don't they or under what circumstances don't they want to rely on
00:00:55
these algorithms and then our second paper the one you you asked about is about how to get people to be more
00:01:01
likely to rely on algorithms and so the answer to our first question we basically found that people are if you
00:01:08
tell them you're going to make a forecast an algorithm is going to give you some advice or you can go with your
00:01:16
own opinion what do you want to do and you just ask them that they're actually
00:01:20
okay with saying I'll use the algorithm however once you give them some practice
00:01:25
and let them see how their algorithm performs now all of a sudden they don't
00:01:29
want to use it anymore and that's because they see the algorithm make mistakes and once they see algorithms or
00:01:36
computers make mistakes they don't want to do it anymore even though the algorithm or computer is going to make a
00:01:42
smaller mistake or more infrequent mistakes than they themselves are going to make the algorithm supposed to be
00:01:47
perfect right so people want algorithms to be perfect and they expect them to be
00:01:51
perfect even though of course what we really want is for them to simply be a little more a little better than than
00:01:59
the humans and so our second paper our first paper is kind of pessimistic and shows that once people see the
00:02:06
algorithms do its thing they don't want to use it our second paper shows that
00:02:10
actually you can get people to use algorithms as long as you give them a little bit of
00:02:14
control over ETSU say the algorithm tells you that this person is going to have a GPA of 3.2 what do you think
00:02:22
their GPA is going to be and they don't want to just go with a 3.2 the algorithm
00:02:26
says but if you say you can adjust it a little bit you can adjust it by 0.1 then
00:02:31
they're like okay I'm fine to use the algorithm and so we basically find as
00:02:35
long as you give people a little bit of control over these things they're more
00:02:38
likely to use them and that's that's pretty good news so this is we operationalize this in experimental
00:02:48
context but we're motivated by a real-world context so some of the early ideas for this research came from
00:02:54
working with companies where we would go in with models for decision-making and in particular this is about hiring and
00:03:01
recruiting new employees and based on many years worth of data and some pretty good analytics we'd have advice and we
00:03:08
were sure that we had the best advice going and yet those organizations would be reluctant to use it because they want
00:03:13
to rely on just their intuition they're reluctant to use those models so it's
00:03:17
very common and hiring it's very common in performance evaluation it's even
00:03:21
common increasingly in some fields where they automate decision-making like how to manage a hedge fund or what should
00:03:28
the sales forecast be for some product so those are all places where increasingly automatically generated
00:03:34
forecasts or advice is available we call it an algorithm and the final decision-maker has discretion over
00:03:40
whether they listen to that advice or whether they use their own or they use some blend so your key takeaway was
00:03:46
basically that people are a little less averse to using algorithms that they have some control but there is a
00:03:51
conclusion that surprised you in how much control you had to give them or you could give them to make them feel better
00:03:57
so tell us a little bit about that well we were agnostic on how much control would be necessary to kind of get them
00:04:02
to buy end the downside of given and controls they start degrading the algorithms they make they're not as in
00:04:08
most domains about as good as the model and so the more of their opinion is in there the worse it performs and so in
00:04:14
some sense you'd like to give them as little control as possible and yet still
00:04:17
have them Buy in we didn't know what the answer that would be and we got early
00:04:21
evidence that it wasn't gonna be very much as we started testing the limits of
00:04:24
it and we found that we could give them just a little bit of control move something around 5% or so and they
00:04:31
would be much more interested in using the algorithm and then if you give them more it doesn't increase their list at
00:04:36
all it's just give them a little bit it's about the same as given the moderate influence yeah and cave was
00:04:43
mentioning what's nice about that is that every when they adjust algorithms they make them worse but if they can
00:04:49
only adjust it this much they can only make it is that much worse and they're
00:04:52
more like entrance they're just more likely to use it in that case their final judgments will end up being
00:04:57
correlated with the algorithm close to perfectly and so we can't get people to
00:05:02
use algorithms 100% we can get them to use algorithms 99% and that bat massively improved their judgments so
00:05:09
tell me a little bit about so with this paper like applying it in real life like
00:05:12
how could if I'm a business owner or even someone who's going to be charged
00:05:16
with using one of these algorithms how might I apply this research in real life so the the overarching lesson would be
00:05:23
that you don't simply impose a monolithic model or blackbox model and say this is how you use judgment people
00:05:29
will fight that this is how you this is how you should cause by your decision making people will fight that you want
00:05:35
to let them have discretion and that's going to look different in different places so consider graduate school
00:05:40
making admissions they they rank their applicants and at some point they cut a line and they make exceptions they move
00:05:47
people around you can automate some of that process even if you use their judgment to provide inputs to the model
00:05:55
you can use them a model automatic model to say these are the folks that you should take on one hand you could say
00:06:00
here's a model this is what it says take it or leave it we're going to automate the process
00:06:04
you'll basically have a revolt on your hand but if you say here's a model its
00:06:08
advisory we suggest that you consider it if you want to move things around move them around we've actually worked with
00:06:14
schools on in exactly this way and what you find is that they're a little skeptical early on they lean on the
00:06:19
model some and over time they lean on the model more and eventually they're practically using the entire model as it
00:06:25
is even though they have discretion to change as much as they want so sort of a get-to-know-you process there's very
00:06:30
much a good to know you process with it and now how important is it with these models I mean just I would also make the
00:06:36
presentation if it will be important that you make sure that people know that they have this control
00:06:40
but you're sort of presenting it in such a way that like here's what you can
00:06:43
control and here's how much control or can it be a little more available Matt
00:06:47
yeah I basically think the important thing is to avoid an all-or-nothing framing like you are all you you have to
00:06:54
stick with the algorithm 100% of the time if people think that based on how you've described it they are going to
00:07:01
push back but instead you can frame it as you know even like ninety nine percent of time we're gonna go with the
00:07:07
algorithm but you have the option to change the algorithm or to not go with the algorithm at a given moment I think
00:07:12
that that's going to go and make people a lot more amenable to using it I mean
00:07:17
in other context of this might matter is like self-driving cars I mean you can imagine people saying like I'm not
00:07:22
feeling comfortable being in a self-driving car if they have no control whatsoever but if they say well there's
00:07:27
this sort of thing you can do it's a little bit difficult and unusual but there's this thing you can do to gain
00:07:32
control over the car in circumstances where you might need to do it now we found that people never need to use this
00:07:38
but it does exist we would predict that in that circumstance you will be much more amenable to getting in getting in a
00:07:45
self-driving car because there's some control it's like autopilot is usually
00:07:50
safer than real pilots but people want a pilot there even though lots of plane crashes are due to pilot error they feel
00:07:56
better about that and so I think our research sort of speaks to that a bit now are there other stories in the news
00:08:03
that might apply to this research Grenoble self-driving cars have obviously been in the news quite a bit
00:08:07
how about election forecasts yeah so I think you know back in November of course we had a presidential election
00:08:15
that surprised the world and there are a bunch of people out there predicting based on past polling information what
00:08:24
the election was going to to look like like probably the most famous cases made silver who's who writes for
00:08:30
fivethirtyeight.com and he basically said that there is a 70% chance that Hillary Clinton would
00:08:36
win the election and a 30% chance that Donald Trump would win and of course you know Donald Trump won and there was a
00:08:43
lot of pushback against Nate Silver at the end of it being like you know you were you know you were wrong and and we
00:08:50
think in part of your model with your model your model was wrong and the thing is it wasn't necessarily
00:08:56
wrong 30% happens 30% of the time but I think that it's it's the kind of thing
00:09:03
where when individual I think when individual pundants go out there and say one thing is going
00:09:08
to happen I don't think they get as much blowback as when the person who uses
00:09:11
statistics and the model and an algorithm that people expect to be right 100% of the time when they actually wind
00:09:17
up getting it wrong in that particular case so I think the blowback that we've
00:09:21
seen in in the direction of nate silver has been in line with what we found before so it circles back to their first
00:09:27
paper where people are just much harder on models and algorithms from there than
00:09:33
they are on people they're just more forgiving they we've explored it a little bit but the bottom line is that
00:09:38
they're held to a higher standard yeah now does that I mean does that make sense or is that I mean I would think I
00:09:44
can see what people would feel that way although no one's perfect or anything is
00:09:48
perfect well you want there a variety of reasons we think people do this one of them is that they believe that people
00:09:54
can improve over time whereas a model is relatively fixed and that's the way they
00:09:59
feel in a way both of those things are not necessarily true models can improve over time and people don't necessarily
00:10:04
improve over time so the psychology of that I think is compelling but it's not
00:10:09
necessarily correct the certainly would be some settings where people can improve more than a model but we think
00:10:16
that people have that intuition more than actually should now is anything - kind of sets this research apart from
00:10:21
other work that's been done in this area that we're not the first to talk about
00:10:29
the difference between model judgment and human judgment it's been established
00:10:34
for decades now that models are quite good we're relatively early into trying
00:10:39
to understand why that is and then how you fix it yeah so right so not many people have documented previously the
00:10:48
reasons why people are averse to using algorithms there's been some anecdotal
00:10:52
research there's been some writings about how people don't like these things
00:10:55
but no one's really looked at it systematically before and so that's that's what we sort of started and again
00:11:01
back to the motivation motivation was we work with organizations we want them to
00:11:05
use more models we need to know how to break down that is you don't you can't prescribe
00:11:09
anything into you better understand of why and exist because you give someone a model and I don't use it you might as
00:11:14
well know delirious frustrated we did never wrote a few times and now is there anything what's next for this research
00:11:21
we took a couple of ideas are you know we continue to play with some factors that might contribute to the people's
00:11:29
reluctance to use acronyms but we also want more real-world tests of it so if we go work with professionals with real
00:11:36
money at stake do they fall in the same biases and is there other ways for us to
00:11:42
help them so we have a couple of organizations we've talked to over time and they're interested in running
00:11:47
experiments on their employees or their own their customers to see if what we see in the lab takes place in the field
00:11:52
as well great thank you both so much for being with us today yeah you [Music]