00:00:00
Welcome, welcome to Wharton Moneyball. Welcome to a full hour of sports analytics
00:00:05
here on the Wharton Podcast Network. This is Cade Massey hosting this week at the moment with my longtime friend and collaborator,
00:00:12
Eric Bradlow. Shane Jensen is out this week, sadly, tragically, but Audie Weiner will join us just a
00:00:20
little bit late. Audie is in transit at the moment. Some combination, as you guys know, some combination
00:00:24
of us are here almost every week of the year and have been for more than 12 years now talking sports analytics with you
00:00:31
guys. We are going to run our usual format this week. We're gonna run a guest in the first
00:00:36
half hour and then we'll do open lines. In the second half hour, me, Audie, and
00:00:39
Eric will kick around a few things. I am piped in from Austin, Texas. Eric is from Huntsman Hall, West Philly, on
00:00:48
campus. Mark, I'm assuming it's New York. I don't know that for a fact.
00:00:51
Mark Brody is our guest in the first half of the show. He is a longtime friend of the show
00:00:56
and we're always delighted to get time with you. Mark, thanks for making time for us.
00:01:01
Thanks for having me. And yes, I'm in New York, New York, and looking out over the Hudson River, which
00:01:06
you can't see in my background because it's the way I'm looking, not behind me.
00:01:10
Right. Mark, as some of you guys know, is at Columbia, the Columbia Business School there in
00:01:15
New York. He is the Carson Family Professor of Business there. He came out of the quant finance world,
00:01:23
but he is best known in the sports analytics world for his contributions to golf.
00:01:27
He flat out invented the concept of strokes gained, which is just kind of ridiculous.
00:01:32
It's one of the great sports analytics stories of our lifetime and continues to be engaged
00:01:37
in the sport at multiple levels. He has a great book, Every Shot Counts. We'd strongly recommend his book, Every Shot Counts.
00:01:45
He consults to professional golfers and he has been involved with revising the ranking systems of
00:01:54
various associations, including the World Golf Ranking System, which is like the biggie, but also NCAA
00:02:01
ranking system. So something we wanna talk about at some point with Mark is that, that's a fun
00:02:06
topic for us. But maybe we'll start, Mark, with just, this is kind of, in some ways, peak golf
00:02:12
season. We're a few weeks post-masters, we're a week before the PGA, which is gonna be
00:02:17
in the Philly area. The majors are all lined up now, one a month, which is fun these days.
00:02:24
What is on your mind, what is on your docket right now in the world of golf? Well, I would say that I don't really
00:02:32
follow golf as closely as some fans do, certainly not as closely as Rufus Peabody does.
00:02:40
But I look at more kind of longer -term problems and actually my day-to-day stuff recently has been with college golf rankings
00:02:49
and professional disc golf rankings. And this is the ranking season for college golf because, like the March Madness basketball tournament,
00:03:00
the main use of college rankings, besides bragging rights throughout the year, is determining which teams
00:03:07
go to regionals. And that's run by the NCAA. And then from regionals, you go to the
00:03:14
nationals and then you crown the national champion. So it wasn't clear to me how this
00:03:20
ecosystem worked until getting into the rankings, but the colleges, universities themselves choose their schedule and
00:03:29
teams run their events. And it's kind of in coordination with the NCAA, but the NCAA isn't running all of
00:03:37
these events. So we're keeping track of all that's going on and then ranking the teams throughout the
00:03:42
year. And then at this point in the year, then the qualifiers for regionals are set.
00:03:50
So there's some interesting problems there in the rankings. There's some interesting problems also, which is more
00:03:55
operations research, I guess, than sports analytics, which is how do you assign the teams that
00:04:01
you have selected into the six regions? Okay. So that comes up in many sports, but
00:04:09
I was involved in it this year in golf. This sounds like a ball, frankly. Let me say, interesting that you mentioned Rufus
00:04:18
at the top there. I happen to know that Rufus and Mark occasionally play golf in the New York area.
00:04:23
So these guys are friends. But yeah, Rufus, not a trivial part of his living is made watching golf, paying attention
00:04:31
to golf very closely. But one of the first things we ever did together when he was shortly after he
00:04:37
was at Yale, the Yale women's golf coach at the time asked us if we could help develop a ranking system for women's golf,
00:04:46
because they felt they were disadvantaged being in the Northeast, playing different tournaments, playing a different
00:04:51
time of year. They thought whoever the cabal was in Oklahoma that ran the college golf rankings, maybe they're
00:04:58
still the cabal. Maybe you're in with the cabal now, Mark. I don't know.
00:05:00
But that coach was not happy with it. And we dabbled a little bit. It was good fun.
00:05:05
But I could imagine that's great fun to get into. It is absolutely great fun.
00:05:11
And I would argue that the previous rankings did appear to be biased in certain ways
00:05:18
that if you played a stronger schedule, that you would be ranked higher for an equal
00:05:25
level of performance. But that requires some analytics in order to prove that. Right.
00:05:31
So I've heard a lot from coaches, but then when I develop this new ranking system,
00:05:38
which has been in place for, this is the third year, I then hear questions from coaches.
00:05:43
And I realized that just because people are criticizing the ranking system, doesn't mean they're right.
00:05:49
You really need to go into the analysis to figure out back then, was Penn, was it biased against teams in the Northeast or
00:05:57
not, just because... And that's another thing that's almost as important
00:06:02
as the analytics is being able to convey the underlying rationale and to communicate why it's
00:06:10
fair and to convince people that it is fair. Because I don't know how many times coaches
00:06:16
have come up with the question to me, we have beaten this other team twice in the season.
00:06:22
By that means they played in the same event and they finished higher in the final
00:06:27
leaderboard. We've beaten them twice, therefore we should be ranked higher than them, but we're not.
00:06:31
Your system is messed up. And so it's like, well, how do you answer that question?
00:06:37
And the most direct way I could find to answer that question was to show cycles.
00:06:41
So I found in the data, just looking at wins and losses, team A has beat team B two or three times.
00:06:50
Team B has beat team C two or three times. And if you argue that A has to be ranked better than B and B should
00:06:58
be ranked better than C, then A should be ranked better than C, but C beat A two or three times total.
00:07:08
And then you realize you can't use that logic. It's completely impossible to have a ranking system
00:07:14
that would satisfy that criterion. Right, so I think it's the NFL that by some point in the season every year,
00:07:21
the circle's complete, that everybody has beat somebody and transitive property takes us all the way
00:07:25
around. Eric has a question. So I just want to say real quickly, Eric, you don't know, like you probably don't
00:07:31
quite understand that he's hinting at the greatest playoff tournament format in sports, you know, the
00:07:37
one I get fired up about every May and June, the college golf tournament. Mark, I don't know if you've heard me
00:07:42
rant, every year I make the guys pay attention because they play this, you know, three
00:07:47
or four or five rounds of stroke play to identify the 16 or whatever it is, teams that play match.
00:07:53
And it's just fricking brilliant in my opinion, but let's leave it there for now.
00:07:57
Eric wants to jump in with a question. No, I just had a clarification question, Mark.
00:08:01
Are the rankings, like literally it's a deterministic function on who qualifies based on the rankings
00:08:09
or it's used like in college football, like it's an input to some subjective committee.
00:08:13
It's a clarification question. I'm just asking which one is it? It's more the latter.
00:08:19
It's an input, not a rule, but in practice, more of the divisions follow the ranking
00:08:26
exactly than don't, but where it comes up is usually in borderline cases, when you draw
00:08:35
the line and you take a look at the teams right around that line and somebody will make an argument.
00:08:41
And I think it could be a valid argument that the rankings aren't perfect. And there might be some reason to take
00:08:48
the team that's ranked one out rather than one in, you know, across the line.
00:08:54
But also when you do that, then the committee that makes the ultimate decision needs to
00:09:00
justify it because it's the team that got kicked out that was inside the bubble is
00:09:05
like, oh, well, you better have a good reason for not letting us get in. But I assume Mark that the distribution of
00:09:11
team, let's call them strengths or rankings is what we classically see that there's greater separation
00:09:17
out in the right tail, but that right near the cut score, there tends to be bunching.
00:09:22
And therefore there's a significant number of teams that could argue, I should be above the
00:09:27
cut line or not. I assume that that follows for rankings as well. I think it happens in just about every
00:09:33
stat that I've looked at, including rankings. So you're exactly right there.
00:09:38
Mark, let's talk about, well, first one more thing on the college thing. You said it's interesting how they assign them
00:09:43
to regions. Yes. And you said this is kind of an OR question, but what are the considerations?
00:09:48
So you said there's six regions. What are the considerations? Yeah. So once you decide on the 72 teams
00:09:56
and it's not in rank order because the conference champions get in. So Columbia women got in this year because
00:10:03
they won the Ivy league championship. And so conference champions get a free pass
00:10:10
into the regionals. And when they're not ranked in the top 72, that means there's teams that are ranked
00:10:18
higher that get in place of other teams. But once you have the 72 teams, the six regional hosts, the six locations have been
00:10:27
determined years in advance. So now the question is, how do you put those 72 teams and assign them to
00:10:34
six regions? And the multiple- Why don't we listen to, we're all math people here.
00:10:40
Is there an objective function, literally a mathematical objective function that's maximized by that assignment?
00:10:47
Except it's an OR problem, there has to be an objective function, right? Well, the problem is there's three or four
00:10:52
objectives. Good, good. And the question is, how do you weight them? But by far the number one objective is
00:11:00
to make sure that the top six seeds, the top six ranked teams, which would be the number one seeds in each region have
00:11:09
the easiest and an equal shot of getting through. In other words, you wanna make sure there's
00:11:15
regional balance. You don't wanna put a bunch of strong teams in one region and a bunch of
00:11:20
weak teams in another. So the number one criteria is so that the number one seeds have the best chance
00:11:27
and a nearly equal chance of getting through and the same with the two seeds and
00:11:31
three, four, five seeds. And even, so that's one objective, sort of this regional balance in terms of strength of
00:11:39
each region. The other that I found out that they care about is they don't want one region
00:11:46
to be overloaded with a bunch of teams from one conference. So it seems like it's a repeat of
00:11:51
a conference championship and they have to duke it out somehow. Somebody is gonna get left out just because
00:11:57
they happen to be in the same region as same conference. Well, Mark, let me ask you a question.
00:12:01
If I gave you the 72 teams, suppose we all agree, I'm not saying we're agreeing
00:12:07
on what those teams are, but I give you the list of 72 teams. Would you write down the criterion?
00:12:14
Like you just mentioned, the top team should have the easiest path. We don't want too many teams in the
00:12:18
same. Would some algorithm now just pump out those 72 broken into the six or would there
00:12:26
still be subjectivity there? So yes, actually to both because the last one is travel.
00:12:33
So I put that in as an objective, but it's less important than I had thought
00:12:37
it would be. But I put it into an optimization problem and it will spit out an assignment.
00:12:44
And then they will look at that assignment in more detail and find reasons for doing
00:12:50
swaps. And because there are many, many assignments of these 72 to six regions that are nearly
00:13:00
comparable, this to me is a way of choosing among many good alternatives, but the algorithmic
00:13:08
viewpoint gets you to a really good starting place, which is very hard to do manually,
00:13:13
which is what they were doing before. So what the standard practice is called the
00:13:19
S-curve, which is teams one through six go into these regions and seven goes where
00:13:25
the six and then you go around. Right, right, right. So we made the algorithm so that it
00:13:34
starts with the S-curve and then when it does swaps, it preserves seeds so that only two seeds are swapped or three seeds
00:13:42
are swapped or four seeds. So in some sense, the S-curve isn't completely blown apart, which again would be harder
00:13:50
to justify or explain. This is a great example of man-machine collaboration and set the board with the algorithm
00:13:58
and then tweak as you need to tweak, but you're going to do so much better as a baseline if it's set with, look,
00:14:04
it's not Mark Brody deciding what the weights are. You had a long consultation about what those
00:14:09
weights were before you ran the optimization. Adi Weiner has joined the show and he
00:14:13
has a question from his office on a campus at University of Pennsylvania. Yeah, great to see you, Mark.
00:14:18
I just want to, just only one question based on my partial understanding of the conversation.
00:14:23
This seems like a job for AI, can it do it? At least get you at right towards a decent set of solutions pretty quickly?
00:14:32
I think AI is good at some things, but not this. No. It's like saying, let's give a traveling salesman
00:14:39
problem to AI and expect to get a better solution than our current algorithms? No.
00:14:45
It isn't. I would expect you to use the current algorithm. That's basically what I would expect.
00:14:48
Oh, sure, sure. Yeah, so yeah, if you think of it that way, but if you don't give AI
00:14:55
enough guidance and it goes to try and figure this out on its own, it's not going to come up with a good solution.
00:15:03
Right. And Mark, you just mentioned something that some of our audience will understand and some won't.
00:15:07
And it's worth pausing and explaining what is the traveling salesman problem?
00:15:12
Sure, so the traveling salesman problem is one of the most famous OR problems. And it says, if you have a list
00:15:18
of cities, think of a bunch of cities in the United States and you want a salesman to go on a tour, visit each
00:15:25
of those cities in some order, and you want to minimize the total travel that they
00:15:31
take. And it turns out if you have 20 cities, then there's 20 factorial, which is a
00:15:40
large, large number, possible orderings of these traveling salesman tours. And it's very, very difficult to find the
00:15:49
optimal solution. Well, 20 is pretty small now, so that can be done. But it's a problem that's in a class
00:15:56
of very hard problems. It means as the number of cities increases, it becomes harder and harder to find the
00:16:03
solution. But for any kind of reasonable set of cities that we have today, like the 72
00:16:11
college teams to assign them, that can be done relatively very fast on modern computers.
00:16:18
And Mark's making a general point that I think Adi set up quite intentionally, which is
00:16:22
whenever we have well-established models and algorithms for a solution, then they do most of
00:16:30
the work. And unless they're fed it directly, AI wouldn't come up, because that's a well-honed, decades
00:16:37
-long optimized model. Okay, let's jump to the pros, because you're also involved with the World Golf Rankings.
00:16:45
And now we have a different set of problems, because this is, I was talking with
00:16:51
Rufus about this just a few weeks ago, actually. World Golf Rankings versus like Data Golf Rankings.
00:16:58
It's a little bit like the college football playoff debate that happens every season between best
00:17:04
versus most deserving. And the best is the predictive model, that's kind of the Data Golf model.
00:17:09
If you're gonna bet who's gonna win this weekend, you want a model like that.
00:17:13
Versus most deserving, which is what I take to be the World Golf Rankings. So I'm sure there's best in there, but
00:17:19
you're also trying to reward accomplishment. So it's more backward, it's at least some
00:17:24
backward looking, whereas the best, quote best, is gonna be strictly forward looking.
00:17:30
So that's the setup. And then can you talk to us about how y'all decide, talk about weights.
00:17:36
I mean, you gotta decide what weight to put on different levels of accomplishment in order
00:17:40
to fix these World Golf Rankings. And importantly, Mark, you weren't, you know, this
00:17:43
thing existed many years before you got involved. So what are the changes since you've been
00:17:46
involved? So Cade, you hit on the point that I think many people when they look at
00:17:54
rankings don't realize that the objective could be predictive, forward looking, or it could be backward
00:18:01
looking reward for performance. And it's easier to see the difference in the official World Golf Rankings if you just
00:18:10
used a who's the best, what's the most predictive, then you wouldn't give any special treatment
00:18:19
for majors beyond that they have a strong field. You wouldn't give any special reward for winning
00:18:26
by- Winning, yeah. One shot for winning, period. Because it could be, you know, you won
00:18:31
in a playoff and you didn't differentiate yourself from the other two or three players that
00:18:35
were in the playoff, whereas a predictive system, one stroke matters, not very much, but in
00:18:41
the official World Golf Rankings, winning means a lot. And winning majors means a lot more.
00:18:49
So the board of the official World Golf Rankings and the way it has been set up for, you know, since the 1980s when
00:18:56
it first started, is that wins are rewarded a lot, heavily, like the ratio is 100
00:19:04
for first, 60 for second, 40 for third. And that decline, that exponential decline really says
00:19:10
one stroke at the top matters a huge amount. Missing the cut, whether you miss the cut
00:19:15
by one stroke or 20 strokes, you get zero points, which in a predictive sense clearly
00:19:21
isn't gonna be good. So the official World Golf Rankings objective is clearly not to be predictive.
00:19:27
It's a reward for performance, kind of backward looking, but it's of course related to skill
00:19:33
because the better players win more often. But the biggest change was trying to make
00:19:41
it unbiased within this reward for performance system. So you wanna make sure if you're giving
00:19:47
points out this way, that a strong field gets the right total number of points compared
00:19:54
to a weaker field. So you can still have the same exponential payoffs, but the main question then is, how
00:20:00
many points do you give to a strong field versus a weak field event or between a PGA Tour event and the Hero World
00:20:08
Challenge, which is 20 players or between that and some Asian Tour event? So that's- How do you do that?
00:20:16
How do you do that? Let me clarify the question first. You're saying it's always gonna have this steep
00:20:20
decay, say 160, 30. So 1.6.3, that ratio off the top three. But for a minor tour event, it might
00:20:30
just be at a base of 10 points. Versus a PGA event might be, I don't know, 50, and then a major might be
00:20:36
100 or whatever. What's the ratio of those things? Yeah, we of course know, Mark, I don't
00:20:40
know if your system relates this at all, but it's not principled really in any way.
00:20:44
We know how they do it in tennis, for example, which is exactly that. It's 2,000 for the majors, 1,000
00:20:50
for the Masters 1,000, which is why they call them that. You know, ATP 500, there's 500 points, 250.
00:20:57
So how is it done in golf? And is it related at all to this kind of two for one system, two for
00:21:03
one ratio system? Definitely not. And that was the problem that that's kind of arbitrary and that leads to biases.
00:21:12
And there were biases that you could easily see in the old system. And so the question is, what's the principled
00:21:18
way to do it? And the idea is you take an unbiased measure of skill, in this case, a strokes
00:21:25
gained measure, and you actually have a curve which converts that into kind of points or
00:21:34
endowments that the players bring to an event. And that endowment, you know, weaker players that
00:21:41
have, you know, worse strokes gain bring fewer points to an event and stronger players bring
00:21:46
more. And that total endowment of the players in the field is the number of points that
00:21:52
are redistributed. And so then you get on average in any field, the players don't move up or
00:21:59
down because their points that they bring to the event are just redistributed. So the players that do well are better
00:22:07
than the points they brought, move up in the rankings, the players that do worse move
00:22:12
down. And that's- It's a standard system. It's a standard of any ranking system ought
00:22:17
to be that way that if your expected performance, if your performance matches your expected performance,
00:22:22
you don't change the rankings, right? And if you outperform your expectation due to
00:22:28
the rankings, then you should move up or move down. So it's a very- So Bradley Terry
00:22:32
is optimized using the EM algorithm and you just, if you're at your expectation, you don't
00:22:38
change. Well, so this is, it's lovely because you said principled and I was kind of like,
00:22:42
well, I want to see this because it wasn't clear to me how you're going to
00:22:45
come up with a principle for these ratios. How off was it before, Mark? Without throwing anybody under the bus or too
00:22:50
much shade, like how big an adjustment was that? Not just that, good, Kate. How the hell did Mark get them to
00:22:55
actually adopt that system? I want to hear that part of it too. Yeah, yeah, right.
00:22:59
Well, there's two parts of that question. So one is, I mean, we kind of showed with Dick Rendleman, a colleague of mine,
00:23:08
that there is great bias, but it's easier anecdotally to see that you could have a
00:23:13
particular player, in this case, say from the Japanese tour, and they used to have these
00:23:18
World Golf Champion events where they'd bring the top ranked players, top 72 or 75, and
00:23:27
invariably, players from some of these lesser tours that happened to just make it into the
00:23:33
top 75 would be destroyed in the tournament by many, many strokes. And you could see that they always, when
00:23:43
you got them together in head-to-head competition with the other players, you could see
00:23:50
that their ranks were misaligned, were out of whack. So to answer Eric's question, how did this
00:23:59
happen? Well, Dick Rendleman and I wrote this paper, and like academics, we thought this would affect
00:24:05
some change, but no, unlike, you know, we showed drive for show, putt for dough is
00:24:12
not right. Here, when we showed the official World Golf rankings in the past iteration were biased, people
00:24:17
said, yeah, we knew that. It was like, but then how do you get it to be fixed? And basically they got a, the official World
00:24:28
Golf Rankings Board decided that they needed a mission statement, and the mission statement was to
00:24:35
fairly rank golfers, not to just reward certain tours for one reason or the other.
00:24:41
And they did an analysis, independent of what Dick Rendleman and I had done, five other
00:24:49
groups did independent analyses that came to the same conclusion that the rankings as currently they
00:24:56
stood at the time were biased. And so now they had a mission statement, they want unbiased rankings, and they have independent
00:25:03
academic expertise saying they're biased, so what's the fix? And that's what led to the change.
00:25:11
It's interesting, so then there's an, so let me just check. I just want to think about this.
00:25:16
So if I'm a good player, I want Sheffler or potentially Sheffler or McElroy, well, when
00:25:25
they're not in the field, there's less than total points brought to that event, which means
00:25:30
there's less, I'll call it bonus for winning. So beating them, I mean, it's what you'd
00:25:35
want. You'd want that to be true, but of course your probability of winning goes down.
00:25:40
So have you ever thought of the problem of like, if I could, if I had, if I'm a, let's say I'm the number
00:25:45
10 player in the world, okay? I don't even know who that is, but let's, I could make up a name, but
00:25:50
let's say, I don't know, whoever. Let's say I'm the number 10 player in
00:25:53
the world. The number 10 player in the world golf rankings is Xander Shuffley. Okay, that's a great name.
00:25:59
So I'm Xander Shuffley. Should I want McElroy and Sheffler not to show up? Now, the bad news is they're not, those
00:26:08
points aren't coming their way, but of course my probability of being higher up the exponential
00:26:12
curve goes up. So which, have you thought about the problem of optimal field depending on where I am
00:26:19
in the rankings? Absolutely. So this was designed so that, that question that you just asked, you're indifferent between whether
00:26:28
Scotty and Rory show up or not if you're Xander Shuffley. Right, right. Beautiful.
00:26:33
Take it, but if you take it to an extreme and you say, we're going to give Eric Bradlow a sponsor's exemption into a
00:26:42
no-cut signature event, okay? Yeah. You would be, you would not be in the top 10,000 of the official world
00:26:52
golf ranking, yet because it's a no-cut event, you might get a few points and
00:26:58
that would be unfair. So it works for almost all events, but not if you take a large, large range
00:27:05
from, the Scotty Shufflers of the world to the 10,000th ranked player in the world.
00:27:10
So your point is academically, it's perfectly correct. In practice, you don't have that large a
00:27:17
dispersion in the quality of the field. I'm sorry, but just to be clarified, does
00:27:20
that mean that an expected value, it's sort of balanced? It's indifferent?
00:27:25
Yes. Okay, but not necessarily, but you could have other loss functions that could be optimized differently.
00:27:31
On average, when you, when players, professionals play in events, they neither go up nor down
00:27:38
in the rankings, unless like I said, you're Eric Bradlow on a sponsor's exemption, then on
00:27:44
average, you're going to go up. All right, Mark, one, I'm curious, especially now
00:27:50
that this has been jiggered a little bit, when it comes to like writer cup selection.
00:27:53
Yep. How did the world golf rankings play into that versus predictive models like data golfs?
00:28:02
And some of it, are the captains restricted or some guys automatically qualify, but that's a
00:28:07
world golf rankings automatic qualification. And then, but they've scrunched that down a
00:28:12
little bit to give the captains more discretion. Do you have any sense of whether captains
00:28:16
just want the predictive or do they put a little value in the most deserving? Cause they might think that it captures an
00:28:23
ability to win under pressure counting. So they've changed from like 10 out of
00:28:27
12 or automatic qualifiers now, like six out of 12. So the captains tend to have six discretionary
00:28:34
picks. And I would say most of them are based on predictive and looking forward.
00:28:40
And then the question is, well, something like strokes gained or the data golf rankings, whatever,
00:28:47
don't take into account necessarily course fit, recent play. And the question is, how do you weight
00:28:53
those factors? And so you're still trying to make it predictive, but you're including additional features that aren't
00:29:02
necessarily in the raw numbers. Okay. Okay. Mark, we are coming up on the time we'll need to let you go, but I
00:29:09
do want to ask one kind of general question about you and your experience with golf
00:29:13
analytics. When you created this concept of strokes gained, you probably could have only fantasized that it
00:29:19
would be as impactful as it has been. But I'm curious years on now, what is one place it's gone or something that's come
00:29:27
from it that has surprised you or that you never would have anticipated? Because it strikes me that it's this tool,
00:29:33
it's a method really, and any good method is going to find new and unexpected uses.
00:29:39
So I'm just curious your experience with it. Where has it gone that you might not
00:29:42
have anticipated? Well, I did not anticipate that most professional players and coaches and college coaches and college
00:29:52
players would be looking at this, but for the most part they are. I didn't really anticipate that it would form
00:29:59
the underpinnings of an official world golf ranking. And I didn't really anticipate that many betters
00:30:07
like Rufus would also be using this as step one in their betting system. So it has had a lot more uses
00:30:16
than I ever imagined. And also surprising when I started teaching the sports analytics course at the Columbia Business School
00:30:25
is the strokes gain concept is now in most modern stats that are coming out. So rushing yards over expected and many other
00:30:35
kind of new stats in football and many other sports have this idea of performance over
00:30:41
expectation. And that's basically what strokes gained is for golf. Well, it's XG in golf and in soccer.
00:30:49
Expected goals in soccer, expected goals in hockey. Guys, it's April or May, it's hockey.
00:30:55
Come on now, Eric. I just have to ask Mark one question. When I think, and this is one academic
00:31:01
to another. When I think of some of the things that I'm most quote unquote famous for in
00:31:06
academia, it's none of the things that are particularly the most technical of my work.
00:31:10
So can you just give us a sense briefly, like I assume strokes gained is not the most technical thing you've ever written.
00:31:17
And so does it bother you? It doesn't bother me at all. I'll just tell you.
00:31:21
But does it bother you at all that the thing you may become most famous, or you are most famous, I don't know, is
00:31:26
or the most cited for is something that isn't as technical as like your training?
00:31:32
No, it doesn't bother me because as academics, I think we try and come up with
00:31:37
ideas that have an impact. And this is an idea that's had way wider an impact, but not on academics, but
00:31:45
on the real world, at least as far as the golf world or the sports world is concerned.
00:31:50
So I'm not embarrassed at all. And I, you know, in many of my talks, it said, you know, how, who would
00:31:58
have guessed that you could get so much mileage out of a concept which is subtracting
00:32:02
two numbers? Right. Well, there you go. That's a good counsel. That's good counsel to us and some of
00:32:10
our listeners that it doesn't have to be the absolutely most technically proficient thing to make
00:32:14
a contribution. Why don't we leave it there? Mark, greatly appreciate your being with us.
00:32:19
Always enjoy the chance to visit with you. Thanks so much, Cade and Eric and Adi.
00:32:24
It's always a pleasure. Thank you. Mark Brody, Carson Family Professor of Business at
00:32:28
the Columbia Business School, the creator, literally the creator of Strokes Gained and a longtime friend
00:32:34
of Wharton Moneyball. Come back and join us after the break. We have another half hour still ahead.
00:32:41
Welcome back. Welcome back to Wharton Moneyball. Welcome to the second half of this week's
00:32:46
show. Tuesday afternoon, as it usually is when we're recording, show will go up bright and early
00:32:52
Wednesday. I think Dion gets this thing going early Wednesday. Eric Bradlow's in here with me.
00:32:57
This is Cade. Adi Weiner's in here now, rolled in to his office about halfway through the first segment.
00:33:03
Shane Jensen out and about doing Shane Jensen things. He'll be back. I think he's blood on a bloody committee
00:33:07
right now, the poor guy. He'll be back just off the line with Mark Brody. He always enjoyed talking Mark.
00:33:15
I didn't know he'd done all those golf rankings things. That was good fun.
00:33:18
And that's a neat little extension of his Strokes Gained. And tournament design as a result of it.
00:33:23
Yeah, it affects tournament design. That's great. We have a handful of topics to hit,
00:33:30
some big, some small, but I'd mentioned last week on the show that he had done
00:33:34
a little work on the ABS. Original primary research that Adi is prone to do on occasion just kind of spins it
00:33:40
off. Some of us eat lunch kind of naturally, some of us drink coffee, some of us
00:33:44
go for walks. And Adi just spins off original primary research on things like the challenges in baseball.
00:33:50
So what do you got, Adi? Yeah, so it's really, it's all new. So we're really interested in trying to figure
00:33:56
things out. And lots of analysis are being done. In fact, the MLB has its own website
00:34:02
devoted to challenges created by, as usual, Tom Tango, who is not a real name. That's a real person, that's not a real
00:34:09
name. And he's looking at essentially challenge rates and success rates, but he kind of creates his
00:34:17
own denominator of challengeable pitches, and he's kind of have his own thing.
00:34:21
So what I ask the question is, think about a universe where every wrong play was
00:34:28
turned to the right one. Okay, good. And now accumulate all the, what I call run expectancy flip, caused by flipping the wrong
00:34:37
play to the right play. We're just talking about called balls and strikes now.
00:34:41
Exactly. And only when they're challenged. Well, no, no, no. No, no, no. All of them.
00:34:46
All of them, right? And so every team will have a run, a defensive set of run expectancy they could
00:34:52
reclaim by challenging, and offensive run expectancy that they can reclaim by challenging.
00:34:58
So why I wanted to ask the question, how much of that pile, if you will, that's available to them, are they reclaiming?
00:35:05
Okay, first, great, cool. But give us some context. How big is that pile? And how's it distributed?
00:35:11
Well, it's actually pretty big. It balances out. So your opponent is missing or is being
00:35:18
denied about the same amount that you are. So it's not like, it's a bias.
00:35:22
But if you can recover more than your opponent, that's actually an advantage, substantial.
00:35:27
Per game, it's getting close to about a quarter to a half a run. All right. So just remind me, per team?
00:35:37
Yeah, per team. It balances out. So it's equal, so it's not like one side is advantage.
00:35:41
That's like 5% of the number of runs that are scored. So that's a lot. It's a little less than that, but yes,
00:35:48
it's a lot, yeah. Okay, and just correct me if I'm wrong. How many, before I answer your question on
00:35:53
percent, can you just remind me of how many challenges I get and when, if I get it right?
00:35:59
Because it matters. Because if I could infinitely challenge, and that's a different thing, but obviously I cannot.
00:36:04
Obviously, you'll get all of it if you can infinitely challenge. Right. You just keep challenging until they make it
00:36:09
right. And at that point, you give up the system and you just let the robots determine
00:36:13
all the value. But that's not the system. The current system allows you two.
00:36:18
And every time if you make a mistake, your amount is decremented. So you can do as many as you
00:36:23
can until you hit two errors. So it's a classic. So that's a wonderful system.
00:36:28
It is a good system. Because it actually is a lot of fun to sort of think about from a operational,
00:36:34
strategic perspective, what should the teams do? And the players are kind of working them
00:36:39
out. Now we've learned a few things. We've learned that catchers are much better than
00:36:43
hitters and pitchers. Catchers are best at this. They're the best view. They also don't have the emotion.
00:36:49
Jazz Chisholm is the worst. He has challenged pitches right down the middle. And he's already an internet meme for like
00:36:56
looking at the camera going, oh, what am I doing? Most players- Catchers have caught a million
00:37:04
pitches. And so they're calibrated for this. Exactly. But they make errors as well.
00:37:09
The catcher error rate is around 60%. And the position player- Zero? No, no, sorry, getting backward.
00:37:15
Success rate is around 60%. And the position players are in the mid 40s. Well, that already gives us some information.
00:37:22
So because look, the two factors that help me determine reclaim percentage is the hit rate
00:37:29
or how successful am I when I challenge, but also the distribution of when I ask
00:37:34
for it, because I want to ask for it, for example, if it's bases loaded and it's three, two count and a marginal pitch
00:37:40
comes in, I should challenge that pitch because that's potentially worth a lot in terms of
00:37:46
expected number of runs. Absolutely. So people have recognized that your decision to
00:37:53
challenge depends on two factors, your probability that you're right, which is how bad the call
00:37:59
was. And of course, the run expectancy that would flip. Didn't I just say that?
00:38:03
Exactly right. And people have recognized that, but people have not. There's no algorithm that anyone has made as
00:38:08
yet. And I'm working, our team is working to create that. We don't know the algorithm right now because
00:38:14
there's a problem. You only get a resource of two. So you have to husband that resource and
00:38:19
think about it properly. So the question that I went out to answer and did answer is a very simple
00:38:25
one, which is what percentage of the run expectancy that is out there for you to
00:38:31
reclaim are you actually getting? And among the amount that you're missing, are
00:38:36
you missing it because you're running out of challenges and therefore you can't challenge?
00:38:40
That's really horrifying, by the way, if you're watching baseball and it's the eighth inning and
00:38:43
there's a strikeout and it's a bad call and you're out of challenges that you're pulling
00:38:47
your hair out. So how much of the missing challenge is cost is, missing RE is caused by lacking
00:38:54
the ability to challenge because you're all out. And how much is it missed because you're
00:38:58
just not challenging? Can I ask you a clarifying question? Sure. Why you decided to study runs as opposed
00:39:04
to win probability? Eric Bradlow with the home run question. The answer is it's a lot more stable
00:39:14
and it doesn't depend on context and it'll be the same no matter what inning you're
00:39:20
in. And it is just, and there's a lot more data on it that's reliable. So win probabilities can barely move with a
00:39:29
strike in the first inning, but a run expectancy is a measurable and determined quantity.
00:39:35
So it really is a statistics argument. Yeah, the only reason I'm asking because I
00:39:39
think it's a good question, I'm glad you thought of it, obviously, but also the reason
00:39:43
why you might see a percentage lower than one might expect is because you've created a
00:39:50
metric, but the team, I'm being generous, the team may be trying to reclaim win probability,
00:39:56
not runs. And therefore that's why you see a lower percentage in your metric than you might expect.
00:40:02
Actually, real quickly, you could imagine psychologically regret is more closely associated to win probability than
00:40:08
runs. Even though I 100% agree with what Adi's saying and doing. And by the way, that's something Adi has
00:40:14
said over the years about expected points versus win probability in different sports.
00:40:19
So I kind of knew where he was coming from, but you could imagine regret, like even the hair pulling out in the eighth
00:40:23
inning, that's a little bit closer. All right, do you want my answer, Adi, on the percent?
00:40:26
Let me just before, just to respond on that as well, like in fourth down decision
00:40:31
-making, the right thing to do is win probability. In decision-making in football, everything should be
00:40:36
win probability, but a lot of decisions are made on expected points. And one of the reasons for that is
00:40:41
there's just many more independent possessions. Possessions are valued on the point basis.
00:40:47
Win probabilities are all related, right? So you have much more data to accurately
00:40:51
estimate things when you're dealing with possessions and the points rather than win probability.
00:40:55
And so we know much more about run expectancy than we know about win probability.
00:41:00
And so we have to recognize the limits of our data. So also one of the things that I
00:41:03
also tell my students, start with the easy thing, for God's sakes. No, I don't.
00:41:09
All right, so you want to make some guesses? What percentage of the available RE is being
00:41:14
reclaimed? I got it. I understand the question. And give me a defense and an offensive
00:41:19
number. Well, clearly we know that defensive is going to get more because catchers are better.
00:41:26
Yeah. I don't have a calculated answer, but my intuition is like just a very small fraction.
00:41:31
I don't know, 10%, 15%. Eric? I was going to guess a third. Okay, so you're both in the ballpark.
00:41:38
It is small. And no one is at a third, but defense is around 27% and offense is just around 19.
00:41:47
And that's only through April 29th when I ran the data. Eric, we should have asked if we could
00:41:52
pool our guesses and we would have hit it. You would have nailed it. Yeah, exactly.
00:41:56
Model average. That would have been the right thing to do as you guys all know. Ensemble, we just have to demonstrate it.
00:42:01
Now the real question is, the second question, which I know the answer to, is how
00:42:05
much of the missed challenge, which is roughly between 75% to 80%, how much of that missed challenges are caused by not having
00:42:13
challenges? Small number. Quarter to a third. I'm going to guess more than that.
00:42:23
I'm not going to, I was going to say half. Now we're going to average. Which side?
00:42:28
That doesn't matter. Eric's half is this. So how much is lost of the remaining
00:42:32
chunk that we're missing? X, take out the ones you're getting. Yeah, if I had more challenges, I could
00:42:38
regain half of that. That's my thought. Okay, so I'm not asking the question, that's
00:42:43
a different, that's a totally different question. If I had three challenges, how much can
00:42:46
I regain? I'm asking you just an observational question of the challenge run expectancy that you have not
00:42:52
reclaimed, how much of it is missed during the periods of time where you have no
00:42:57
challenges? Yeah, yeah. Then it's small. Then I'm going to go back to what
00:43:02
Kate's saying. I'm going to go with a quarter. It's actually much smaller than that.
00:43:09
Much smaller. They are barely missing any challenge because they don't have it.
00:43:15
It's about 5%. So they happen to pick the right number of attempts too? No, I just think they're underusing their challenges.
00:43:21
That's my guess. They're underusing their challenges. That's another explanation.
00:43:26
I got plenty of challenges because I'm not using them. They're just not using them.
00:43:29
And when they use them, but they don't use them enough. I think the teams, once they blow one,
00:43:35
are just under, I just think they underuse them. And particularly later in the game, I think
00:43:40
they're underusing them. Seventh innings, eighth inning, and the ninth. I mean, come on.
00:43:46
Just use the goddamn challenge. Is this the same as like sitting your guy with two fouls in the first half?
00:43:52
Yes. You're trying to preserve something and you're being too conservative about the preservation.
00:43:57
Yep. Or it's like, you're starting actually to see Adi's point actually in the NBA now.
00:44:02
Like you're starting to see at the end of games, like you know it was a foul, but what the hell?
00:44:08
Let's challenge this thing and see what happens. There's 15 seconds on the clock.
00:44:11
And you know what? We're going to lose the game if this thing isn't overturned.
00:44:15
There's no cost for me to see. You know, maybe some gets in the ref's eye and he says, no, that's not a
00:44:20
foul. You know, who knows? Actually, I'm starting to see it in the playoffs.
00:44:24
I've seen ridiculously bad challenges in high leverage situations. Are there no penalties for over?
00:44:32
You get a certain number. But it's just the end of the game. You're going to lose the game if this
00:44:36
isn't reversed. Give it a shot. So that's the real question. So the answer, the unknown now is what's
00:44:43
the right policy. Right. Yeah, right. Don't know it. Are you observing different teams following different policies?
00:44:51
Is there much heterogeneity? Is there already learning? Oh, Adi could have been talking about two
00:44:55
different things, Adi. Do you mean MLB's policy? Or do you mean what Cade was talking
00:44:59
about, which is the team policy? I'm not changing MLB. Oh, okay. By the way, so I'm able to rank
00:45:05
teams by their ability to recover challenges. And actually Philadelphia is very good on the
00:45:09
offensive side. They've been recovering a lot of challenge. They may be doing that because they're actually
00:45:13
challenging a lot. And in fact, a lot of, I mean, this gets to the psychology. When you challenge and are wrong, they often
00:45:21
feel the heat for that. And I think in a way that's probably not deserved. A good challenging team should be not ending
00:45:29
the game with challenges. That's, you screwed up if you did that. Which means that you have to bear the
00:45:33
brunt of what might be considered silly challenge. Can you tell us the probability?
00:45:38
If I have on the Y-axis, the probability of successful challenge, and on the X -axis, the distance from the strike zone, what
00:45:45
does that curve look like? I actually have, so I don't have that number because it's very correlated with that.
00:45:55
But I'll start with the first number, which is, I can tell you the probability of
00:45:59
an umpire error as a function of the distance from the strike zone. Okay. I fit that curve.
00:46:04
And by the time you get to three inches, the probability is nearly zero. Yeah. And it interpolates fairly linearly.
00:46:12
What is the diameter of a baseball? Just under three inches. All right, so it's basically, once it gets
00:46:18
to more than a baseball outside of the zone. They tend to be very, very good. That's good.
00:46:24
I'm glad, thank you. The umpires, by the way, the umpires are awesome. Yeah.
00:46:28
I mean, we laugh at them and think, oh, 92.5 is their success rate. Do you have any idea how many pitchers
00:46:35
are right on the border? These are professional pitchers we're talking about.
00:46:39
They throw it on the corners and on the edges. And when you're on the edges, their probabilities
00:46:44
are roughly 50%. I mean, it's very hard to get right. The trouble is, and this is one of
00:46:49
the reasons the challenge system is so great, is that there are outliers. I mean, there are human beings.
00:46:53
And the outlier, if you can zap those, then you've made a lot of progress. Why don't we let that sit there for
00:46:59
now? This sounds like an ongoing project. I'll be able to come back to us on some things.
00:47:03
I have some other, I'll do some other baseball trivia for y'all. Trivia is the wrong word.
00:47:07
I'll put you through your paces a little bit. Did y'all happen to see this Wall
00:47:10
Street Journal article about the Brewers the other day? They looked at how over-performing they are
00:47:15
over the years. So, FanGraph's preseason predictions for number of wins over an 11-year period of time, 2015
00:47:23
through 2025. So 11 seasons. The Brewers have been the most outperforming of the FanGraph's expectations.
00:47:29
And you can go to like Pakoda, you can go to somebody else's system. They're also the top there.
00:47:34
What do you think the average number of wins above expectation the Brewers have achieved over
00:47:40
the last 11 years? Five. So what is the, in wins? How far in wins above expectation?
00:47:49
Observe your wins minus FanGraph's expected wins. Sum it over 11 years and divide by
00:47:54
11. Yeah, and I've guessed five. I would guess three. Seven and a half. What?
00:48:02
Right? And that's 11 years set. Dodgers are five, 5.1. Astros are 4 .8. You're Yankees, who seem like underachievers to
00:48:10
me, are actually four. But the Brewers are 7.6. And- Does that say more about their model than
00:48:16
anything else? Well, the key, I think the article, is it Jared Diamond? He's always doing these kinds of articles.
00:48:23
I'm not sure if it's him. Let me look real quick. Yeah, Jared Diamond, as a matter of fact.
00:48:26
This is May 4, so yesterday. They say it's development. You know, this is a small market team,
00:48:33
small budget team. And they say they are turning, making players better more than they're expected to and more
00:48:42
than other teams are. And if that's true, I just- They're also relying heavily on youngsters because if they're
00:48:48
not youngsters, they would be priced into the preseason expectation. So if I'm getting this right- Unless
00:48:54
they're just turning them into, they take a veteran and they make them better than they
00:48:58
were before in some way. I mean, this was the story for the Astros all those years.
00:49:01
Like Astros were first into development. Now we talk about baseball being this way.
00:49:04
But apparently, seemingly, at least by this article, they're ahead of the league on that.
00:49:10
But Adi's got a good point in that, like when residuals like this tend to be
00:49:14
persistent, any good statistician would go back to the model and say, I'm missing something here,
00:49:21
you know? But what if it's a team effect? And then you're just going to put your
00:49:23
thumb on the scale and have a real Brewer's Fixed Effect? That's not the way you want to go.
00:49:28
That's a big number. That's seven and a half. That's a big number. Right?
00:49:31
Yeah, no, because listen, they don't tell us what their forecasting system is.
00:49:34
Yeah. I don't know what that is. That is a very big number. And it is time for us to just
00:49:39
shout out a former student of ours, Eric Babitz, who is the Director of Player Acquisition
00:49:43
for the Brewers and has been there since he graduated five, six years ago. At the Brewers?
00:49:48
At the Brewers. Oh, how about that? Well, heck, let's get him on the show and find out whether Jared Diamond's article is
00:49:53
actually capturing what's going on. I mean, he's right in the middle of that stuff.
00:49:56
Because he's charged with finding people that aren't going to break the bank because they're a
00:50:00
small budget team, and yet they believe they can make better. That's his charge.
00:50:04
That's his charge. That's interesting. Okay, let me give you one other baseball
00:50:07
question. I don't know how I wasn't calibrated for this, but top, so the women's softball season
00:50:14
is winding down. Their postseason is great fun. Top shortstop in the country, Issa Torres, Florida
00:50:22
State. She's got a big bat, she makes offensive contributions, but she's also great defensively.
00:50:27
What do you think, just let's say, I don't know if this is true, but let's
00:50:31
say she has the top fielding percentage in the country. What do you think her field, a shortstop,
00:50:35
women's college softball shortstop, what do you think the top fielding percentage would be?
00:50:41
And I'll give you the number of, well, you might even guess, what's the number of
00:50:43
chances? There's a simple thing to guess. How many games are there in a season?
00:50:46
30 or so? About 30, yeah. And I would guess chances are probably in the, there's so many strikeouts, probably between six
00:50:54
or seven chances a game. So I'd go for about 150, so. You're dead on, 142.
00:51:00
I thought it'd be higher, 142. So Issa Torres, 142 chances this season, led the country in fielding percentage.
00:51:06
What do you think it was? So I bet she got about 138 of those 143, would be my guess.
00:51:14
So that's 97, 98%. That's what I was, I was gonna guess somewhere between 95 to 98%.
00:51:19
What's your upper limit? Well. 100. Well, I know that's your true max, but
00:51:27
we do agree. She's 100%, she had no errors this year. She had no errors, that's amazing.
00:51:32
Playing shortstop. Now, let me ask you a question about that. So let's suppose you have a known statistic.
00:51:38
So let's imagine there's a play that happens now or whatever during the season.
00:51:43
And the scoring person says, you know what, let's tilt this to the no side. First baseman should have scooped that ball.
00:51:52
Yeah, exactly. That was not a bad throw. Okay, that is wonderfully cynical, Eric.
00:51:58
Well done. Okay. Let's do one last one. And this is more involved. And this is the paper, y'all saw
00:52:04
that Dylan Weawad. Dylan Weawad is with Slack. He's a quantitative researcher with Slack.
00:52:09
He is a social psychologist out of Simon Fraser, making him Canadian. And he did, Northwestern has had, Northwestern, Kellogg's
00:52:19
Graduate School of Business at Northwestern, as Eric knows anyway, has had postdocs for years.
00:52:24
Great postdoc program. He did that. And then he's moved on to Slack. But he's rediscovering his love of hockey and
00:52:30
is running some quantitative analysis. And so I wanna ask y'all. So he sent us this paper and he's
00:52:34
a published academic. And so when he writes a paper, he's gonna write it soundly like an academic.
00:52:39
So it's this impressive, only 15 pages, but very strong methodologically. I wanna ask y'all, because he takes
00:52:45
on an interesting challenge. He goes to measure depth in hockey. And one of the reasons he sent it
00:52:52
to us was that we talked to Eric Tolsky a few weeks ago, the general manager of the Carolina Hurricanes.
00:52:58
And Tolsky's whole thing is basically depth. They've built their whole roster to be deep,
00:53:04
like less top-heavy and just deep. All four front lines, all three back lines, whatever.
00:53:09
They're gonna have guys that can play all the way. Is there an agreed upon measure?
00:53:12
Like you can imagine like entropy of skillset. You can imagine max minus min. You can imagine taking lines and looking at
00:53:19
the summation of some metric and looking at lowering the variance. Is there a well, I'm saying, is there
00:53:25
a, no, okay. But you mean something like this. But no, but you mean something like this
00:53:29
when you say depth. If we take Dylan to have done a thorough lit review, and I don't know that
00:53:35
that was his objective, so we don't know, but he's acting as if he's creating this.
00:53:39
I suspect teams, some teams have this. Some teams, certainly Tolsky has done this.
00:53:43
But as far as we know, there's no accepted measure. So I'm asking y'all how you go
00:53:48
about doing it. Just a neat little paper about Dylan. I just came up with three measures right
00:53:53
there. I just, so you have some strength parameter for a player. And so depth would be, well, I mean,
00:54:01
it depends on meaning across the whole team, or you could mean across lines. You could, matter of fact, around the team.
00:54:08
Well, you know, there's classic measures of, you know, variance is one. You could have some sort of- Just
00:54:14
look at the difference. How much, what's the rate of scoring per minute among each line and look at the,
00:54:21
how that erodes from, look for big drop in three and four in some way. So you want to use scoring, Adi, as
00:54:27
opposed to like some sort of strength measure. Well, you can make a composite measure because
00:54:31
there's obviously offense and defense. But I think the lines are different. I mean, they're misaligned, right?
00:54:36
You're out of my pen here, folks. No, no, no, but let's- No, no, let's- But isn't there offensive players and
00:54:42
defensive players- No, no, but this- But no, you're asking a good question. Suppose- Let's say, let's, straight offense.
00:54:46
We're going to play- Yeah, so I want to look at your rate of scoring and then compare that across the lines, either
00:54:53
per possession per minute, probably per minute. Okay. And then there's also a power play issue
00:54:57
because I think you bring in your top line. You could do that, yes. You could do power plays, sure.
00:55:02
Okay, so let me just walk you through a few of the steps that he does because I think it's thoughtful and interesting.
00:55:07
Let me say it's relevant also because he ran the numbers on this year's NHL and
00:55:12
Carolina is like a standard deviation beyond everybody else in this measure. And in his analysis, it shows that it's
00:55:19
predictive. It makes contribution above baselines on the likelihood of a team winning.
00:55:26
Okay, so that's how you're controlling for a bunch of other things that you'd expect.
00:55:30
It's there's incremental contribution. That's right. Can I just shout out to, yeah, go
00:55:35
ahead, finish. Well, I'm going to give you a little bit of a, I'm going to give you
00:55:37
a couple of steps on methodology because I thought they were nice. First, it's at the player level as opposed
00:55:42
to the line level. So he'll take a stat like ice time or closer to what you might care about,
00:55:49
like XG, expected goals. He looked at also things like assist and shots on goal, but it ends up with
00:55:55
a suite of these statistics. And in each case, he asked, he asked what's the compliment of the Gini coefficient.
00:56:02
So y'all know the Gini coefficient is this IO kind of, it's a concentration measure.
00:56:08
And so he's asking how concentrated is the ice time or shots on goal or expected
00:56:13
goals among the players or the offensive player or the skaters. It's all obviously defensive players can have XG
00:56:20
as well. And so that's one, that's the basic measure, but then he models the whole thing as
00:56:27
a latent variable. And so I think this is something that we ought to listen to. I think there's a nice contribution because he's
00:56:32
saying, look, it's not one thing. XG, like the compliment of the Gini coefficient
00:56:37
of XG for a team is an imperfect measure of depth. But if we get three or four of
00:56:43
these things and ask, is there a latent variable? I'd say this is basic measurement science.
00:56:49
You don't measure a construct with one thing. You use a bunch of manifest variables that
00:56:54
represent some underlying basic concept, which is why I was thinking of some sort of strength
00:57:00
model. You could call that latent thing a strength thing, which is manifested itself in different observables.
00:57:09
I'm excited to look at his paper, Cade. Thanks for bringing it to our attention.
00:57:13
I will just point out that that was the qualitative or quantitative or qualitative question that
00:57:18
we asked our high school students in our competition. Is that right? They were asked to essentially evaluate whether depth,
00:57:24
we didn't call it that, we call it line variance, how that affected winning.
00:57:29
And that was the final question for their high school students. And we didn't- Maybe I should just
00:57:35
say a few words, what you're talking, I mean, I obviously know what you're talking about.
00:57:39
You should just say a few words. What is the Wharton High School Data Science Competition?
00:57:42
So Wharton Analytics and AI at Wharton, we sponsored a major competition for high school students
00:57:48
and we created a fake hockey league. They played each other all season. And then we asked the students to forecast
00:57:55
the outcomes of the playoffs. And that was all very forecasty and very rigorous and qualitative.
00:58:03
But then we had them ask an open -ended question, like a hockey question, which is,
00:58:07
what is the relationship between having a depth, which is the right word for it, and
00:58:14
winning? And we asked them to study that. And it was actually very hard because there
00:58:20
was a very strong confounder, which is the top lines also play against the top defenses.
00:58:27
And so you have to control for the quality of the defense that they play against
00:58:32
to measure the quality of your top line. And if you didn't do that, you thought
00:58:36
that your underlying, your lower lines were better than they were because they were playing against
00:58:42
weaker defenses. And I will tell you that one team managed to do that out of the 700
00:58:49
submissions. Well, I know who it was because they presented actually at the Wharton AI and Analytics
00:58:54
Board Meeting. I assume you mean the Pingree team. Is that the team that won? They were the team that won.
00:58:58
I'm not sure they're the team that figured that out because it was so esoteric that
00:59:03
there were so many components to the program. All right, guys. So we'll let that one go.
00:59:07
If anyone's interested in chasing it down, Dylan's website is hockeydecoded.com, hockeydecoded.com.
00:59:15
Thanks, Dylan, for sending that to us. And listen, guys, we're always interested in the
00:59:19
work. And to be fair, the more put together the work is when you send it to us, the better, but we'll take work in
00:59:25
progress. We'll take data. We'll take websites. And Dylan, maybe he got lucky.
00:59:30
He got me in a moment where I could pause and read the paper, but we love that you guys send us stuff on
00:59:35
occasion and we learn from it. So please keep it up. Fellas, what else around the world of sports
00:59:42
right now before we have to run? Well, just one thing for me. I always love talking about tennis, but Yannick
00:59:49
Sinner has done something no one's ever done. He's won five straight Masters 1000 titles.
00:59:55
And no, not Federer, not Djokovic, not Nadal has ever done that. And my prediction is, I don't think, unless
01:00:02
he gets injured, I don't think he's going to lose unless, until Alcaraz comes back.
01:00:09
I mean, the gap between him and everybody else, look, he just played Zverev, who's number
01:00:15
three in the world. He beat him 6-1, 6-2. It's the fourth straight match. He hasn't given up more than six games
01:00:23
to Zverev. And that's the number three player in the world, okay? And Djokovic is injured right now.
01:00:28
Maybe Djokovic beats him on a bad day for Sinner, but the gap is so large that he may win every tournament he competes
01:00:37
in, assuming he stays healthy right now. I don't see any reason why he wouldn't
01:00:41
be. I have to start looking at the betting odds, but you have to put him at minus 400 minimum, minimum for any tournament he
01:00:49
plays right now. That takes the fun out of it, doesn't it? That's a little too heavy.
01:00:53
It does. How long is Alcaraz out, you said? It's hard to know. I mean, he's not playing the French, which
01:00:58
is upcoming. They're not at the French yet. They still have, I forget which one's next,
01:01:02
the Italian Masters or whatever. But there's one more tournament and then the
01:01:05
French, and Alcaraz is already saying, because of his wrist, he's not playing the French.
01:01:10
So, yeah. So that's what caught my eye. All right. We should note that two Philadelphia franchises are
01:01:19
in the second round of the playoffs. That's not nothing. How long has it been?
01:01:22
How long has it been since that happened? Both Flyers and the Sixers are into the
01:01:24
second round? That's pretty good. They're both getting whipped right now in that
01:01:29
second round, but presumably the Sixers will bounce back. I assume that that Knicks first game was
01:01:34
just because of how rough that Sixers- Well, the Knicks, by the way, have set up, I mean, whatever this is worth, it's
01:01:39
a small sample. The Knicks have now set a playoff record. They've won four, three straight, but they've won
01:01:44
four straight, I think, playoff games now by 25 points or more, which has never been
01:01:48
done before. So the Knicks are on a heater. They were down 2-1, just remember, mind
01:01:52
everyone, they were down 2-1 to the Hawks. They beat them three straight by plus 25.
01:01:57
They beat the Sixers by 39 last night. So whatever heater they're on, they're on a
01:02:03
wrecking crew mission right now. And you're right, the Sixers may bounce back
01:02:07
and lose by 15. Now, one trick, concern potentially, is that they were winning, and then the team starts to
01:02:14
try to get desperate, and then it ends up creating a gigantic margin if it doesn't
01:02:18
work out. They were beat pretty early on this game. Yeah, I have to tell you, I was
01:02:22
in- It cuts out pretty quick. I was in Penn Station when the Knicks were playing their last game of their first
01:02:26
series, and at the bar in Penn Station, they had the game going on, and I turned over and looked, and I saw the
01:02:32
score, and I'm like, that can't be real. Right, it was like 90 to 30 or something.
01:02:36
No, it was an unbelievably unreal score. But there's this thing in tennis that I
01:02:41
think is known, that tennis players, if they get behind in a set, sometimes just say,
01:02:47
I'm gonna lose this set, I'm gonna conserve energy. And I think this happens in basketball series
01:02:52
as well, that sometimes a team will get sufficiently behind, they're like, we're gonna conserve energy
01:02:56
this time, we're gonna come back- Well, Cade, I mean, let's remember, the Sixers played
01:03:00
seven games. They ended on Saturday. So they had to fly down from Boston. By the time they was blown out, I
01:03:08
know for a fact, Embiid, Paul George, Tyrese Maxey, they were all pulled in the early
01:03:12
to mid part of the third quarter, the third quarter, because they're playing again tomorrow.
01:03:17
There's no, I mean, I'm saying there's no rest. There's one day rest.
01:03:20
You obviously, the game's over. Tank this game and go for the next one. I'm worried about that in the Eastern NHL
01:03:28
playoffs, the Canes are up on the Flyers 2-0. They're big favorites anyway, they might sweep them.
01:03:33
The team I'm pulling for, the Sabres, haven't yet started their series against the Habs.
01:03:39
And so it could be that the Canes get out and have like a 10-day break before- Get rusty.
01:03:46
Then you hope for us, that's exactly right. All right. Let me give you one other thing, guys,
01:03:49
before we go. The NHL lottery, draft lottery is today. So there's a few wrinkles here that are
01:03:57
just a little bit interesting. But by the way, the presumed number one pick, it's not 100% sure, but the
01:04:01
presumed number one pick is that kid that's at Penn State. He kind of unexpectedly went to Penn State
01:04:07
when Penn State was practically just an intramural team still, Gavin McKenna. And by the way, he's like six foot
01:04:13
170. It's just amazing to me that a guy on the hockey ice, six foot 170, which
01:04:18
is a very average size guy, could be that good. But setting that aside, the odds, the top
01:04:24
odds for the worst performing team last year, the Vancouver Canucks, it's 25 and a half
01:04:29
percent, 25% chance. It's pretty, pretty good. And then it drops to Blackhawks are 13,
01:04:34
Rangers are 11. And here's something for us to pull for tonight. In absentia, Shane Jensen's Calgary Flames are the
01:04:41
fourth highest probability of getting the number one pick, nine and a half percent.
01:04:44
So we should all be pulling for the Flames. They haven't done quite as much flattening as
01:04:50
like the NBA has done with the draft lottery, et cetera. Or where they haven't even, there's tiers.
01:04:56
Like bottom four in the NBA are all at the same percentage now, so in other words, you can tank, but there's no reason
01:05:02
to tank from third to second. They haven't, I don't think they've done that
01:05:06
yet, but they're talking about making that last change she just said. But there's another wrinkle in here that I've
01:05:10
never seen before. We talk about tournament design, but here's lottery design.
01:05:13
Of course, we're hearing a lot about it from the NBA, but I think this is just gonna exacerbate what you just said.
01:05:17
They have separate draws for the number one and the number two pick. I think that means in the NBA, they'll
01:05:25
count down like 10, nine, eight, seven to one. And so it's one draw all the way
01:05:30
through. I think that means they're gonna choose a ball for number one, and they're not just
01:05:36
gonna do a full, well, would it make any difference? And they're gonna, that team will obviously be
01:05:42
removed for the next draw. Why would it be any different? That team's probability should be, that probability, let's
01:05:50
say it's 25.5%, that probability should be distributed equally. In other words, it doesn't violate what we
01:05:56
call the IIA property, meaning the remaining 74 .5% probability still has the same ratio
01:06:03
amongst all the other teams. So there should be no impact. Yeah, so why would it matter?
01:06:09
Maybe I have it, maybe I get, so I don't know. I don't know what they're doing and what
01:06:13
they're doing wrong. That must not be what they're doing. Anyway, I would just wanna, something for a
01:06:16
future show, but I have a group of students from my Moneyball Academy over the summer.
01:06:20
They studied the rate at which value is lost as you go down the draft curve for all sports.
01:06:28
And hockey surprisingly gets the incredible amount of value from the top of the draft, and
01:06:33
it drops quickly, but way more than other sports. I was surprised. I didn't realize how predictable a top-ranked
01:06:41
hockey player is in terms of value. That's a lot. The difference in predictability is what you're talking
01:06:46
about, though, the drop. Yeah, but even, it starts high, and then it stays high and it goes down, but
01:06:52
it's amazing. So the metric they used was all-star appearances, and they are a lot at the
01:07:00
top of the draft. Interesting, and did you have any sense of where the other sports come in?
01:07:04
Yes, we have that. We'll have to talk about that at a later date, because it's a great little study.
01:07:10
Well, it's surprising given the long development time on most hockey players.
01:07:16
It's much more like, it's in the direction of baseball as opposed to like a football
01:07:19
or basketball. Okay, guys, that's a pretty solid show. Well, well over an hour here.
01:07:25
It feels like we haven't talked in a while, and we did some today. We'll let it go there.
01:07:30
For the whole team here, Eric Bradlow and Adi Weiner joining me, Shane Jensen in absentia,
01:07:35
Dion Simpkins, the big boss behind the scenes making this thing happen for more than 12
01:07:39
years now, Marissa Reyna, our producer, and Deep Patel, boss lady, keeping us all in line.
01:07:46
Thanks to that whole team. Thank you guys for listening. Come back and join us next time.
01:07:49
Between now and then, enjoy your sports.