00:00:01
today we're speaking with Wharton professor 8 on green about his research on expert decision-making and
00:00:07
specifically umpire calls in baseball so it sound happy with us thanks for joining us yeah my pleasure so can you
00:00:14
give us an overview of your research yeah so generally what I'm interested in
00:00:17
is decision-making by experts expertise particularly in realms for which there are predictions available from machines
00:00:25
and machine based models algorithms or in the case of umpires in baseball data from stereoscopic cameras behind home
00:00:31
plate of every major league ballpark that we use to benchmark the calls that umpires make and so this is a great
00:00:37
setting for studying decision-making by experts because we have experts who are supposed to abide by a very specific
00:00:43
decision rule so the pitcher throws a pitch if the pitch is in this imaginary box the official strike zone defined by
00:00:50
the width of home plate on the floor and the batter stance then the umpires supposed to call a strike otherwise he's
00:00:55
supposed to call a ball and so what we do is we use these data from the stereoscopic cameras that take a
00:01:00
sequence of images of every pitch from its release from the pitchers hand until it crosses the region above home plate
00:01:06
to basically observe to what extent the umpire abides by this decision rule to make his calls based solely on the
00:01:13
location of the pitch and so I think the most interesting thing that comes out of
00:01:16
the data is basically this deviation from that benchmark and very systematic way and so there's something in baseball
00:01:23
called the count the count is keeps track of the sequence of pitches between a pitcher and a batter over the course
00:01:28
of an at-bat if the count reaches for balls that's good for the batter he walks if it reaches three strikes that's
00:01:34
good for the pitch or the batter strikes out and so what you see is that instead
00:01:38
of the umpire just using the location of the pitch to make as calls pitches at the same location are sometimes called
00:01:45
balls or sometimes called strikes depending on the count and in particular the strike zone expands dramatically
00:01:50
when the count favor is the batter and so when account favors the batter the umpire responds by favoring the pitcher
00:01:55
and vice versa when account favors the pitcher the Empire responds by favoring the batter and it's particularly extreme
00:02:02
so basically you can think about a pitch that crosses say the top boundary of the
00:02:07
official strike zone so this pitch and what I'll call baseline count the count
00:02:11
at the beginning of the advance euro strikes this an umpire calls a strike fifty percent
00:02:17
of the time it calls a ball fifty percent of the time you can think of as being a different between a ball and a
00:02:22
strike but when the counts say has three balls and 0 strikes when it's strongly
00:02:26
favored is the batter well then this pitch is almost always called a strike and the reverse is true in the opposite
00:02:32
countless your balls and two strikes the same pitch at the same location is almost always called a ball so so why
00:02:39
does this happen so they're potentially a number of stories that I can explain
00:02:43
this result let me tell you about a particularly interesting and counterintuitive one and the argument
00:02:49
here is that what the pitcher is do or what the umpire is doing is he's trading
00:02:54
off accuracy for bias or rather he's trading off bias for accuracy he's being
00:03:00
purposefully biased consciously or unconsciously so he's varying the strike zone that he enforces with the count
00:03:06
he's not making his decisions based solely on the location of the pitch but the argument is this actually helps them
00:03:11
make more accurate calls and why is this the case well imagine yourself as an umpire you're squatted behind the
00:03:20
catcher you're looking out over his head towards the pitcher the pitcher winds up
00:03:24
he throws a 90 plus mile an hour pitch it's there in an instant it has some lateral movements and vertical movement
00:03:29
you have to decide whether this pitch is inside or outside some imaginary box it's an incredibly difficult problem and
00:03:36
if you relied only on your observation of the location of the pitch you'd probably make mistakes on a regular
00:03:42
basis frequently when the pitch is closed it'd be hard to say whether it was just inside the strike center just
00:03:47
outside the strike zone but fortunately for you you have other information at your disposal you have expectations that
00:03:54
you've built up over many years of being a professional umpire expectations about
00:03:59
where the pitcher is going to throw in a certain count and whether the batter is
00:04:02
going to swing and so for instance you might reasonably expect that when the count has three balls and 0 strikes that
00:04:09
is when favor is the batter that the pitcher is going to try to throw a strike and so if the pitch is close and
00:04:14
you're unsure whether it was just inside the strike zone or just outside the
00:04:16
strike zone you may err on the side of calling a strike the pitch that you expect now think about what happens in
00:04:23
an 02 count so in this count you expect that the batter is going to swing it anything close
00:04:28
because if he doesn't he runs the chance of striking out whereas he can prolong
00:04:32
liat bad if he fails the pitching off for instance and so imagine you see a pitch that appears close to you but the
00:04:38
batter chooses not to swing how can you rationalize that decision well you can rationalize it by saying that he
00:04:43
observes something that you didn't that his vantage point was such that he believed the pitch to be a ball and so
00:04:47
you might err on the side of calling a ball and so this Bayesian updating this basically rational way of processing
00:04:54
other information that you have creates this trade-off between bias and accuracy
00:04:58
it helps the umpires become more accurate at the cost of having them systematically change the strike zone
00:05:03
that they enforce with this variable the count that has nothing to do with their
00:05:07
directive so what would you say a business practitioner should take away from your research yeah so I think what
00:05:16
umpires are doing is they're statistically discriminating so they have a directive to make their calls
00:05:23
based solely on the location of the pitch but that's very difficult to do it's very hard to observe the exact
00:05:29
location every time and so what they do instead is they say well this other informations other information is
00:05:34
correlated with the location in the pitch it could help me on average make more accurate calls and so as I said
00:05:40
before they basically trade-off bias for accuracy it's a statistical discrimination at least opportunities
00:05:47
for statistical discrimination are just they're everywhere and they're everywhere in the workplace in
00:05:52
particularly in you know the hiring process so for instance you know when we hire we have a benchmark that sounds
00:05:59
very similar to the umpires directive we want to hire the best person person is going to do the best of the job there's
00:06:05
going to be the best fit but it's hard in the interview process looking at a CV
00:06:09
or even interview i'm a person often to decide who is the best or how good is
00:06:14
this person how good is this person going to be in the job and so we may rely on other factors factors that are
00:06:20
either implicitly or explicitly banned that we shouldn't be using perhaps but
00:06:24
factors that we believe perhaps rightly as in the case of the umpires or even erroneously to give us information about
00:06:31
this person's fit and so if we're right we're going to get a little more
00:06:34
accuracy but it's going to come look at the cost of bias it's going to come at
00:06:38
the cost of systematic being able to systematically predict who we hire based on factors
00:06:44
that have nothing to do at least directly with the dimension that we're trying to hire along so what are you
00:06:50
going to look at next going to stay stay in baseball or look elsewhere for research yeah so I mean it's a baseball
00:06:56
is an opportunity to use machine based models these cameras to say how good of a job umpires are doing but I think
00:07:07
there are a lot of interesting cases in which the decisions that individuals make that experts make can be informed
00:07:15
by algorithms by machine based predictions and so one of the things that I'm interested in particular now is
00:07:23
making predictions about the election so it's particularly timely I think a lot
00:07:26
of us are interested in the probability that Hillary Clinton is going to wait on
00:07:29
the probability that Donald Trump will be our next president and so one place you may go to get information about this
00:07:35
you may go to 538 nate silver's website and one thing that nate silver is doing
00:07:39
this election season that he hasn't done in previous election seasons is he's
00:07:42
providing multiple models so in the past he told you the probability that Hillary
00:07:47
Clinton would win is seventy-seven percent now he's telling you if you believe this model it's seventy two
00:07:52
percent if you believe this model it's eighty-four percent and sometimes there's really quite a deviation between
00:07:57
these two models well what are these two models well basically they're making
00:08:03
different assumptions about the world and your decision as to which model you listen to is really a decision about
00:08:09
what what you believe the data generating process to be what do you believe the world to look like and in
00:08:14
particular there's one model that says we should only listen to the pulse we
00:08:18
should only listen to what people are saying right now and there's another model that says actually there are lots
00:08:23
of predictors economic indicators for instance like that historically have been very predictive of election
00:08:30
outcomes and so we should listen to those as well as your decision about which model to listen to her how to
00:08:35
balance these two pieces of information basically comes down to your belief about whether this election season is
00:08:41
totally different from the past in which case you should only listen to the polls
00:08:45
or if you believe that this is just another draw and some stable distribution that is similar to
00:08:51
everything else that's come before and so generally what I'm interested in
00:08:56
is how can we frame questions what types of information can give people to make them think that this moment the present
00:09:04
is just like the past and the past is a good predictor of the present and what types of information how can we frame
00:09:10
questions to get people to think actually know the process is not stationary at all this moment is unique
00:09:16
in time great that's fascinating thanks very much for joining us a time yeah my
00:09:21
pleasure thank you you