00:00:00
Welcome to Wharton Money Ball.
00:00:02
Welcome to a full hour of sports analytics
00:00:05
here on the Wharton Podcast Network.
00:00:07
This is Cade Massey hosting this week with
00:00:09
the whole crew.
00:00:11
We have Shane Jensen in here.
00:00:12
We have Audie Wynder in here.
00:00:14
Eric Bradlow, who was not scheduled to be,
00:00:16
but slid in here pleasantly at the last
00:00:19
minute.
00:00:19
So we're gonna be fully staffed for this
00:00:21
show.
00:00:22
We've got a double guest show this week.
00:00:25
We usually run guests for half the show
00:00:27
and open lines the other half, but we
00:00:29
wanted to hit the NFL draft with two
00:00:32
barrels this week, and we had the chance
00:00:35
to get two great guests, and so we
00:00:36
jumped on it.
00:00:38
First up, Richard Thaler.
00:00:40
Richard Thaler is a professor at the University
00:00:43
of Chicago.
00:00:44
He also is a Nobel Prize winner.
00:00:47
Was it 2017, 2018?
00:00:49
I should know that, Dick, and I don't.
00:00:51
2017 or 18?
00:00:53
He- 17.
00:00:55
So we've been doing this show for 12
00:00:58
years, guys, and we finally have a Nobel
00:01:00
laureate on the show.
00:01:01
This is the first.
00:01:02
Dick, thank you for making time.
00:01:04
We'd talk with you anytime.
00:01:06
We're happy to have time with you.
00:01:07
That doesn't mean it's the first time you
00:01:09
invited me.
00:01:10
Well, that's true.
00:01:12
That's true, and I don't know why that
00:01:14
is, honestly.
00:01:15
We would take you on a lot of
00:01:16
different topics, but we decided to grab you
00:01:18
on the NFL draft.
00:01:21
We've just come through the draft.
00:01:23
We're gonna have two guests.
00:01:24
We can kind of debrief the draft, but
00:01:26
we thought we'd chat with you first, Dick,
00:01:27
and I'm gonna take the perspective, I think
00:01:31
the easiest way to think about it, I'm
00:01:33
gonna take the perspective of your wife, the
00:01:34
honorable professor, former professor, Franz LeClaire, who has
00:01:40
not really loved the project we've been working
00:01:43
on all these years.
00:01:44
She doesn't really understand why we waste our
00:01:46
time with the NFL, and so I wanna
00:01:48
ask you that question, Dick.
00:01:49
Why is a Nobel laureate messing around studying
00:01:53
the NFL draft?
00:01:54
I mean, why is that worthy of your
00:01:56
time?
00:01:57
Yeah, well, as you know, I have one
00:01:59
criterion for anything I choose to do, which
00:02:04
is it has to be fun, and so
00:02:09
studying sports is fun.
00:02:13
Now, of course, that is not a satisfactory
00:02:18
answer to Madame LeClaire, but my serious answer
00:02:23
to that question is that we don't have
00:02:28
that many opportunities to study business decision-makers
00:02:36
making decisions, so there's a big merger going
00:02:45
on, and there's competing bidders, and we don't
00:02:50
know what they're thinking and how they came
00:02:52
up with the numbers, how the process evolved.
00:02:56
In sports, we get to watch, so some
00:03:02
team continues to punt on fourth down, fourth
00:03:08
and one everywhere on the field.
00:03:10
We get to see that, and most of
00:03:16
my life, we would just say, what are
00:03:19
they doing?
00:03:21
But many years ago, you and I got
00:03:25
the idea that the NFL draft would be
00:03:30
an interesting institution to study because it involves
00:03:38
so many different things, so I mean, the
00:03:42
institution is interesting.
00:03:43
Well, why do they have a draft at
00:03:46
all?
00:03:46
You could ask, like in European soccer, there's
00:03:51
nothing like a draft, but they have it,
00:03:58
and there's this odd chart, and our
00:04:08
history of this went back to learning about
00:04:12
this chart and actually calling up the guy
00:04:16
who made it, and that's an interesting story.
00:04:21
The guy was a partner of Jerry Jones
00:04:25
at the Dallas Cowboys, an engineer, and the
00:04:28
coach at the time, Jimmy Johnson, asked him
00:04:31
a question.
00:04:31
He said, look, we get offers to trade
00:04:34
the fourth pick for some other picks, and
00:04:37
we have no idea what the right price
00:04:39
should be.
00:04:40
Can you figure it out?
00:04:42
And this guy's an engineer, so he's a
00:04:45
numbers guy, and he got all the trades
00:04:50
that had happened up to that point, and
00:04:55
my impression, maybe your recollection of this conversation
00:04:58
is better than mine, but my recollection is
00:05:01
he said, kind of wrote them all down,
00:05:06
and then kind of drew a line on
00:05:08
a piece of graph paper, and he made
00:05:12
the first pick worth 3,000, and the
00:05:16
last pick worth one, and there's a smooth
00:05:20
curve you can draw through that, and so
00:05:28
for a while, that was proprietary with the
00:05:31
Cowboys, but then assistant coaches move around.
00:05:36
Now, Google NFL draft chart, and it will
00:05:42
pop up.
00:05:44
It hasn't changed in 30 years, and teams
00:05:51
use that as like a price list.
00:05:56
I think of it as like the blue
00:05:58
book for used cars.
00:06:00
You can look up what your 10-year
00:06:04
-old Honda Civic should sell for, and so
00:06:08
the paper we wrote a long time ago
00:06:11
that my wife mocks us about asks the
00:06:15
question of is that price list the right
00:06:21
one?
00:06:22
So you can trade the first pick for
00:06:25
the eighth and ninth picks, according to that
00:06:27
chart, or for half a dozen second-round
00:06:30
picks, and that was the origin of our
00:06:36
research paper, and what we found is that
00:06:41
price list is very wrong.
00:06:44
It's much too steep at the beginning that
00:06:48
you'd much rather have the eighth and ninth
00:06:50
picks than the first pick, and we've now
00:06:59
had a re-dive into the idea of
00:07:02
this asking, since we published a paper about
00:07:07
this 15 years ago, has anything changed?
00:07:13
And the short answer is no, so it's
00:07:16
been nice talking to you, Cade.
00:07:19
Well, let's take it one step further, and
00:07:22
then Adi wants to jump in with a
00:07:23
question, but I would say what did you
00:07:25
learn?
00:07:25
What do you think we have learned?
00:07:27
We came back to it.
00:07:28
God knows, I hope we learned something from
00:07:30
coming back to this.
00:07:32
It was bad enough that we spent so
00:07:33
many years working on it the first time,
00:07:34
and now we're gonna do it again.
00:07:36
What have we learned this second time around,
00:07:38
Dick?
00:07:38
In what way do you think about this
00:07:39
differently because of what we've done the last
00:07:41
couple years?
00:07:44
Well, so I think there were two big
00:07:47
questions we started with this time.
00:07:53
One is, do they still use this Jimmy
00:07:56
Johnson chart?
00:07:59
And the second is, have they gotten any
00:08:04
better at picking players?
00:08:08
Now, the reason why that chart is wrong
00:08:12
is that, you know, I said the first
00:08:16
pick is worth the same as eighth and
00:08:18
ninth.
00:08:19
That suggests the first player has to be
00:08:22
twice as good, excuse me, or twice as
00:08:28
valuable, or more precisely, how much he contributes
00:08:36
minus what you have to pay him.
00:08:39
The surplus has to be twice as much
00:08:42
for the first one as to any pair
00:08:45
you could trade for.
00:08:48
And for that to be true, you'd have
00:08:51
to be really good at predicting.
00:08:53
Mm-hmm.
00:08:54
And our favorite stat from the original paper
00:08:58
was one we called the better than the
00:09:01
next guy stat, which is take any two
00:09:05
players who play the same position who are
00:09:09
drafted one after the other.
00:09:12
So the third running back compared to the
00:09:15
fourth and so forth.
00:09:17
And we say, what's the probability that the
00:09:21
earlier player is better than the next one?
00:09:25
And in the original paper, the answer to
00:09:28
that question was 52%.
00:09:31
Now, if they're perfect, it's 100.
00:09:34
If they're random, it's 50.
00:09:37
It's a little better.
00:09:39
And it's a little higher in the first
00:09:42
round.
00:09:44
And so that was the first question is,
00:09:46
well, now we've got stats and the combine
00:09:50
and AI and who knows, maybe they've gotten
00:09:55
a lot better at this.
00:09:57
And the answer is they haven't.
00:09:59
That number is now 53%, first round 58%.
00:10:05
So they've not all of a sudden mastered
00:10:10
the art of predicting who's gonna be good.
00:10:15
And the second one is, do they still
00:10:20
use that chart?
00:10:21
And the answer is yes, kind of ridiculously
00:10:26
so.
00:10:28
So there was one trade early in the
00:10:31
draft, the Browns traded down from six to
00:10:36
nine or 10.
00:10:39
And on the chart, that was worth 1
00:10:42
,600 points.
00:10:44
And they paid 1,602.
00:10:48
And so- Okay, those prices are like
00:10:55
floors.
00:10:56
So they're the asking price.
00:10:58
If the buyer's really keen, they'll sometimes pay
00:11:03
more than that.
00:11:05
But we don't see too many sales.
00:11:10
Well, and that finding, I think was-
00:11:13
I can't hear you.
00:11:15
Let me know if you can't hear this,
00:11:17
but the observation is that we thought they
00:11:20
might learn away from that curve.
00:11:22
We thought the prices might change.
00:11:25
I mean, they've had more than a decade
00:11:28
of learning.
00:11:30
And in fact, it went the other way.
00:11:32
The curve is the price.
00:11:34
And it's just unbelievable that that curve was
00:11:37
created based on 1980s trades.
00:11:40
It's been in existence for 35 years now,
00:11:42
and it hasn't changed.
00:11:44
Free agency has come, salary cap has come,
00:11:47
and the curve hasn't changed.
00:11:48
In fact, if you look at the coherence
00:11:50
to the curve, it's gotten tighter over time.
00:11:53
So it's one of the most surprising things.
00:11:54
Let's take a couple of questions.
00:11:56
Adi was first and then Shane.
00:11:57
Yeah, so I've actually had the opportunity to
00:12:01
dig into this a bit and even published
00:12:03
a paper on it, which I think Cade
00:12:04
is certainly aware of.
00:12:07
And I was really surprised when he told
00:12:10
me, Cade told me that it hadn't changed
00:12:12
at all.
00:12:12
Because we know a lot of people in
00:12:14
sports and they're pretty smart.
00:12:16
I mean, not everyone is smart.
00:12:18
I mean, I'm gonna be perfectly honest.
00:12:20
But there's a lot of smart people, including
00:12:21
some of our former students and very close
00:12:23
friends now are pretty high up in lots
00:12:25
of places.
00:12:26
And when I explained what you guys were
00:12:29
finding that it hasn't changed, the remark that
00:12:31
came back to me was, well, we don't
00:12:34
maximize expected value with our trades, which is
00:12:38
where your wrongness comes in.
00:12:40
The utility is expected value.
00:12:42
We actually look for tail distribution probabilities.
00:12:46
And so I went back and I looked
00:12:47
at that and I found that the trade
00:12:48
curve actually does match a utility function that
00:12:53
weights with much, much higher probability superstars.
00:12:58
So they've cut off, it's hard to measure
00:12:59
superstars.
00:13:00
It's a second contract.
00:13:03
So, and taking a lead from Cade, what
00:13:05
we did was with size of the payroll,
00:13:08
essentially the cap that you're devoting to one
00:13:10
player.
00:13:11
So if you call superstar something like 15
00:13:13
% of the cap, then, and you wanna
00:13:16
make a trade that balances the expected number
00:13:18
of superstars, then it actually does match.
00:13:21
And that seemed to be a very highly,
00:13:26
at least it puts a gloss on it.
00:13:27
I mean, for many trades are really dumb
00:13:30
because they aren't trading.
00:13:31
They think they're doing that, they're not.
00:13:32
They really should be trading expected value.
00:13:34
But for some teams, I think the smarter
00:13:36
ones, when they do trade up for a
00:13:38
higher pick, they have that in mind.
00:13:42
So we've done the following analysis to look
00:13:44
at that.
00:13:46
We've looked at every two for one trade
00:13:49
that has happened.
00:13:52
And then we've also looked at every feasible
00:13:56
two for one trade.
00:13:57
So I mentioned you could trade the first
00:13:59
pick for eight and nine.
00:14:01
You could trade it for seven and 12
00:14:04
and six and so forth.
00:14:07
And what we find is when you trade
00:14:11
down, you get the same number, you get
00:14:17
twice as many games started from the two
00:14:22
players and the same number of pro ball
00:14:26
appearances.
00:14:28
So you don't, if we're settling for all
00:14:34
-star appearances, you don't do better.
00:14:39
You break even on that and you get
00:14:41
- That's not enough.
00:14:42
All-star period, that doesn't quite do it.
00:14:44
You need superstar status, not just all-star.
00:14:48
Yeah, well, if you heard of, there's a
00:14:50
guy, kind of a tall guy, Tom Brady,
00:14:55
I think 199.
00:15:00
I rest my case.
00:15:02
Hold on, I'm gonna- Not an anecdote
00:15:04
on the one extreme?
00:15:06
For real?
00:15:08
Adi, what you need- Well, maybe I
00:15:09
can jump in with- No, no, let
00:15:10
me jump in.
00:15:11
Shane, Shane, let me jump in first on
00:15:13
this point.
00:15:15
And we should continue the conversation in other
00:15:17
places, Adi, but what you need for that
00:15:20
curve to be correct is that the team
00:15:23
places zero value on any level of performance
00:15:27
other than extreme superstar.
00:15:29
That's right.
00:15:30
You need zero value on anything other than
00:15:32
superstar.
00:15:33
And there's no evidence from any labor market
00:15:35
in professional sports that suggests that that's the
00:15:37
value they place on performance less than superstar.
00:15:40
I don't doubt, Dick and I said in
00:15:42
presentations in the 2010s, in the 2000s, we
00:15:45
said, look at this, the Pro Bowl curve
00:15:48
comes pretty close to the Johnson curve.
00:15:49
The Pro Bowl curve, I think you're right.
00:15:51
You said this thing a minute ago, you
00:15:52
said, this is what they have in mind
00:15:55
when they're making the trade.
00:15:56
Yes, it's something they have in mind.
00:15:58
It's the psychology of the trade up, but
00:16:00
it is far from a rational psychology.
00:16:05
They care about getting starts, even out of
00:16:07
first round picks.
00:16:08
They pay a great deal for mere starters
00:16:11
in every free agent market.
00:16:13
The psychology might be a heavyweight on superstars,
00:16:17
but there's no way that that rationalizes the
00:16:19
trade curve, given anything else that we observe.
00:16:22
Two quick comments.
00:16:23
First, you don't have to put zero.
00:16:25
You can have an S-shaped curve and
00:16:26
you'll get the same thing.
00:16:28
Secondly.
00:16:28
I'm gonna have to understand that better.
00:16:30
I don't accept that as a rationalization.
00:16:33
You can have a logistic function that approximates
00:16:35
it.
00:16:36
Oh my God, this is gonna be the
00:16:37
most extreme indicator function.
00:16:38
Are you kidding me, Artie?
00:16:39
Come on.
00:16:39
It doesn't go that quickly.
00:16:40
The second thing is, and I think this
00:16:43
is sort of relevant, when you talk about...
00:16:50
Well, you know what?
00:16:52
I'm just gonna pass.
00:16:53
All right, so I'll just say the following.
00:16:56
In the formal analysis of the paper, which
00:16:59
I think is gonna be out anytime this
00:17:02
decade, the way we value players is by
00:17:11
what percentage of the salary cap they get
00:17:15
in their first free agent contract.
00:17:18
Yes.
00:17:19
That's what I do.
00:17:19
That'll include superstars.
00:17:23
And we'll have this conversation again when the
00:17:30
paper is done, and otherwise we're gonna just
00:17:33
go into the weeds.
00:17:35
Shane.
00:17:36
Yeah, and I hope I'm not just asking
00:17:38
the same kind of question from a different
00:17:40
direction, but earlier when you were talking about
00:17:44
what they could have learned and how you
00:17:45
kind of expect things to be updated, you
00:17:48
mostly talked about that you would have gotten
00:17:50
at least over time a more accurate evaluation
00:17:55
of the actual talent.
00:17:58
But that's only, I think...
00:18:00
I just wanna clarify that that's only kind
00:18:02
of half of the story in terms of
00:18:04
like whether a one is worth a seven
00:18:06
and an eight or whatever.
00:18:07
It's not just that accurate, like the accurate
00:18:11
evaluation of their various talent.
00:18:13
It's actually also a big part of the
00:18:15
story is how the talent distribution, or I
00:18:17
guess how the value distribution falls off away
00:18:21
from one, right?
00:18:23
I mean, if you had sort of, I
00:18:25
guess, an extreme enough distribution of actual value,
00:18:29
the number one pick could be worth seven
00:18:34
or eight.
00:18:34
It's just in actual football, it doesn't seem
00:18:37
to drop off like that.
00:18:38
I guess it's maybe the empirical part of
00:18:39
this.
00:18:40
You know, if you look at the first
00:18:45
round picks over the last couple of decades,
00:18:49
there are a lot of busts.
00:18:52
Just look at the quarterbacks drafted by the
00:18:55
Chicago Bears until the recent one.
00:18:59
And they're, you know, it's not the case
00:19:04
that they hit very often.
00:19:07
And, you know, as Mr. Brady shows, or
00:19:12
for God's sakes, Mr. Irrelevant, Brock Purdy, it's
00:19:17
not the case that the best players are
00:19:20
all in the top five.
00:19:22
They're not.
00:19:25
It is also the case that there are
00:19:26
just more busts at the top than we
00:19:28
remember.
00:19:29
We think of those guys, because they are
00:19:32
the highest likelihood of superstar, we think that
00:19:34
the likelihood of superstar is higher than it
00:19:36
actually is.
00:19:36
Eric was gonna jump in with a question.
00:19:38
Yeah, so Professor Thiel, I wanted to ask
00:19:40
you, so you also studied learning, right?
00:19:43
And so I would think, I'm a Bayesian,
00:19:46
but that doesn't mean just in Bayesian computation.
00:19:49
I do think about how people update beliefs,
00:19:52
and I do a lot of work on
00:19:53
that.
00:19:53
So if the chart is mispriced in some
00:19:58
way, why aren't teams learning this over this
00:20:02
long period of time, especially given, in my
00:20:05
view, and your guys' empirical study, the overwhelming
00:20:07
evidence that they do?
00:20:10
Yeah, so I think most, so I think
00:20:14
almost every team has somebody on it who
00:20:21
has read our paper most have somebody who's
00:20:26
been able to update it.
00:20:28
There are many other updates around the professional
00:20:34
sports world.
00:20:37
So, and I think a lot of teams
00:20:40
have another chart they think reflects actual value.
00:20:46
But my impression from talking to people in
00:20:50
the league is that the Jimmy Johnson chart
00:20:54
is the list price.
00:20:58
And if you're selling, that's your asking price.
00:21:04
And you'll get a lot of nos.
00:21:09
You'll hear from teams, yeah, we wanted to
00:21:13
trade out, God knows what the Rams were
00:21:18
thinking, but you'll hear, oh, they wanted to
00:21:23
trade out, okay, then what price were they
00:21:26
willing to take?
00:21:30
And we don't see very many trades below
00:21:36
the chart.
00:21:38
So smart teams would have to be willing
00:21:41
to sell at a discount.
00:21:45
Right.
00:21:46
So, and there's an old story, well, remember,
00:21:51
right when the first version of our paper
00:21:53
came out, the 49ers had the first pick
00:21:59
and they needed a quarterback.
00:22:01
And for months, they were debating between two
00:22:04
hot prospects and they couldn't make up their
00:22:07
mind.
00:22:08
And a reporter, a local reporter called me
00:22:12
and said, what advice would you give to
00:22:14
the Niners?
00:22:15
And I said, I'd announce a sale on
00:22:20
the first pick, 20% off.
00:22:24
And that's not what they did.
00:22:27
And they used the first pick to draft
00:22:32
Alex Smith.
00:22:34
And there's another guy who was playing on
00:22:38
the other side of the bay, who had
00:22:40
a pretty good career and he went 24th.
00:22:49
So, no one seems to, my advice is
00:22:53
so worthless.
00:22:55
No team has yet adopted the strategy of
00:23:00
not knowing what to do with the first
00:23:03
pick and announcing a sale, because it makes
00:23:05
them look bad.
00:23:06
Well, this is the thing that I want
00:23:07
to emphasize because it's a very fair question.
00:23:10
It seems to be an off equilibrium place
00:23:13
we're at.
00:23:13
The price that is so rigorously adhered to
00:23:17
doesn't seem to match the value of the
00:23:19
assets that come from those picks.
00:23:22
And so the question is, why does it
00:23:23
persist?
00:23:24
Every team in the league has a chart
00:23:26
and this is what happens guys.
00:23:27
The trades come up and they've got Jimmy
00:23:29
Johnson chart and they've got their own charts.
00:23:31
And they compare it on all these charts
00:23:34
and they know that the price is basically
00:23:36
gonna be Jimmy Johnson.
00:23:37
And then they decide how badly they want
00:23:39
it by looking at their own charts.
00:23:41
And so I asked a guy just last
00:23:43
week, we were texting just before the draft.
00:23:45
This is a guy who has run the
00:23:47
trades for multiple teams.
00:23:49
And I said, why would you say some
00:23:51
teams are not willing to accept less than
00:23:52
Jimmy Johnson, basically fail or sale, even if
00:23:55
their other charts really like the trade at
00:23:57
a lower price?
00:23:58
So this is key.
00:23:59
Why does Jimmy Johnson pricing persist even when
00:24:02
lots of teams have sharper charts that diverge
00:24:05
from Jimmy?
00:24:06
Yeah, so this is the guys I got
00:24:07
permission, I'm not gonna tell you who it
00:24:09
is or use any team names, but he
00:24:10
gave me permission to use his text.
00:24:13
Why is he selling down?
00:24:14
He says, this is the answer.
00:24:15
He gives me the answer.
00:24:16
He says, perception.
00:24:18
This is quote from our text last week
00:24:19
to my question.
00:24:20
He says, perception, how another team was looked
00:24:23
at, for example, when they got way too
00:24:25
little in that general manager's first trade.
00:24:28
You never wanna look like the sucker.
00:24:30
And the safest way not to is to
00:24:32
not deviate from what has gotten that deal
00:24:34
done before, which is Jimmy Johnson.
00:24:37
And that's the story.
00:24:39
Taylor talked about the interview with the guy
00:24:41
who created the chart, Mike McCoy.
00:24:43
He told us that in 2006.
00:24:45
He says, this way, no one gets skinned.
00:24:47
He says, it's CYA, CYA, because these guys,
00:24:51
they don't wanna look bad.
00:24:53
They don't wanna look bad in the press.
00:24:54
They don't wanna look bad to their peers.
00:24:55
And that's enough to keep them.
00:24:57
Now, one last wrinkle's needed.
00:25:02
The guys who wanna buy at lower prices
00:25:05
can't access it.
00:25:06
They can't short.
00:25:07
There's no shorting in this market.
00:25:08
So the only guys that could move it
00:25:10
are the guys that would sell at lower
00:25:11
prices.
00:25:12
But there are these basically punitive measures if
00:25:17
you sell below Jimmy Johnson.
00:25:20
It's hard to buy that they won't do
00:25:22
that, that you're essentially saying that they're letting
00:25:25
something that's been known for 20 years get
00:25:27
in the way of, I mean, listen, look
00:25:29
at fourth down decision-making.
00:25:30
It took forever, but now they all do
00:25:32
it.
00:25:32
It's done, right?
00:25:33
It took forever.
00:25:35
They don't all do it.
00:25:36
So they get it like half right.
00:25:45
We didn't know how this was gonna come
00:25:48
out.
00:25:48
What I'm telling you is it doesn't look
00:25:51
like they've learned anything.
00:25:53
Well, let me ask you another question.
00:25:54
How do you know how to build a
00:25:56
Super Bowl-winning football team?
00:25:59
What's the secret sauce?
00:26:01
Maybe there's something, I mean, why would we
00:26:03
in the outside know how to do that?
00:26:04
Because implicitly you're saying that you're not doing
00:26:06
it right because we know how to build
00:26:07
the team and you don't.
00:26:09
How do we know that?
00:26:11
No, I think, look, no one would hire
00:26:15
anybody that's on this call to do such
00:26:17
a job, but look, right?
00:26:21
I'm sitting in the office of a money
00:26:25
management firm that has my name on it.
00:26:29
And we don't think we know how to
00:26:32
run companies, but we think that we can
00:26:36
put together a portfolio of stocks that outperforms
00:26:43
its benchmark.
00:26:45
And that's by being right 53, 54%
00:26:48
of the time, right?
00:26:51
So I think teams, the smarter teams are
00:26:58
doing better, but they basically are, that's because
00:27:03
they trade down more often and never trade
00:27:08
up.
00:27:11
But they don't wanna look bad and they
00:27:16
run a big risk of getting fired.
00:27:18
Look, 10 of the 32 coaches got fired
00:27:21
last year.
00:27:22
I don't know the data on GMs, but
00:27:27
you can't afford to look dumb.
00:27:31
I actually look forward to hearing about that
00:27:33
study that shows that the teams that don't
00:27:36
ever trade up and don't trade or trade
00:27:40
down more often, they do well.
00:27:42
I don't know if it's false or not,
00:27:43
but is that true?
00:27:45
It's something we looked at.
00:27:47
We used to call this the golden, what
00:27:51
the Holy Grail.
00:27:54
The Holy Grail, yeah, right.
00:27:55
There's so much else.
00:27:58
Some teams are better at fourth downs.
00:28:03
Do they do better?
00:28:06
The Steelers were terrible at fourth downs and
00:28:10
somehow still managed to win.
00:28:13
So there's blocking and tackling and the stuff
00:28:18
we geeks care about.
00:28:22
Well, Adi, the last thing we do in
00:28:24
our first paper is to run an analysis
00:28:26
of the accumulation of draft capital in one
00:28:30
year and the downstream consequences on the playing
00:28:32
field.
00:28:34
So it's wins.
00:28:35
What we care about is wins, right?
00:28:36
And what we find is the accumulation of
00:28:39
draft capital doing the trade down, managing their
00:28:42
draft capital in the way we would recommend
00:28:43
is positively correlated with downstream wins.
00:28:47
Now, is it direct or does it just
00:28:49
reflect the kind of sophisticated management?
00:28:51
We don't know, but we find a reliable
00:28:53
relationship between exactly what you just said.
00:28:56
It's the last test in the published paper.
00:28:58
Trading down, accumulating draft capital is positively related
00:29:01
with downstream wins.
00:29:03
Eric has a question.
00:29:04
Yeah, I want to ask Professor Thaler another
00:29:05
question.
00:29:06
Here's another way to rationalize, and I wonder
00:29:08
if you guys have looked at this.
00:29:10
Suppose, since as you know, utility functions also
00:29:13
sometimes have uncertainty bought into them.
00:29:15
In other words, we take the mean utility,
00:29:17
we might add an uncertainty term.
00:29:19
Is it also possible that the whole mechanism
00:29:22
is due to the top-end picks are
00:29:25
seen as much more certain with lower standard
00:29:27
error?
00:29:28
Later picks is seen as much more variable.
00:29:31
Therefore, a rational person would like, even it
00:29:35
seems overvalued if you're maximizing expected utility, but
00:29:38
if it's a risk-adjusted expected utility, it's
00:29:41
not.
00:29:42
Because that could also rationalize it.
00:29:44
People overestimate the certainty with which the top
00:29:48
picks.
00:29:48
So it's a variance story, not just a
00:29:51
mean.
00:29:54
Well, trading down even past the first round
00:30:02
works like two-thirds of the time.
00:30:06
And it's just not the case.
00:30:11
After the first pick, it's better than the
00:30:15
next guy's 52%.
00:30:18
And you can't predict the tails.
00:30:23
So I can't think of any rational basis
00:30:32
by which you wouldn't prefer twice as many
00:30:36
starts and the same number of pro bowls.
00:30:40
And to be specific about the variance, the
00:30:42
variance actually goes down later in the draft.
00:30:44
It doesn't go up, especially if you're looking
00:30:46
at surplus.
00:30:48
Surplus, which is after you've paid them, you've
00:30:50
got this very high volatile left tail at
00:30:54
the top of the draft.
00:30:55
And later in the draft, yes, you've got
00:30:57
outliers, but mostly you've got a bunch of
00:30:59
low performances and standard deviations are coming down.
00:31:04
Last comment.
00:31:05
The big variance comes from Tom Brady and
00:31:08
Brock Purdy.
00:31:13
Guys, we're going to need to cut Professor
00:31:15
Thaler loose.
00:31:17
Any last questions for him?
00:31:23
What questions remain for you, Dick?
00:31:25
Give us one last question you have about
00:31:27
the draft.
00:31:28
Now that you've looked at it, come back
00:31:30
to it, looked at it some more.
00:31:31
You've got decades now of looking at this.
00:31:34
What questions remain for you about it?
00:31:40
Well, I think the, within the team
00:31:47
dynamics and the role of the owners, I
00:31:54
think would be really interesting to study.
00:31:59
And let's just say they don't always help.
00:32:05
Okay, why don't we wrap it there?
00:32:08
Dick, thanks so much for making time for
00:32:10
us.
00:32:10
Appreciate it.
00:32:11
Good luck with everything you have rolling out
00:32:13
there.
00:32:14
Just off the line with Professor Richard Thaler
00:32:18
dialing in from California.
00:32:20
He spends most of his time, half his
00:32:22
time out there talking NFL draft.
00:32:25
We are gonna spend the next half hour
00:32:26
with a one-on-one fight between me
00:32:28
and Adi about our papers.
00:32:30
We'll suspend Ben for next week.
00:32:32
We have to work some things out, okay?
00:32:37
Is that really happening?
00:32:38
No.
00:32:41
Alternatively, we'll fight- Shane and I could
00:32:44
moderate that discussion if you'd like.
00:32:46
Okay, I'm ready.
00:32:47
I am ready, but we're gonna- I've
00:32:49
been to an econ seminar, and those are
00:32:50
pretty bloody, so.
00:32:53
I've never been to one, and I've started
00:32:55
to wade into that territory and realizing it.
00:32:58
Yeah, Adi decided to turn the first half
00:33:00
of the show into an econ seminar, so
00:33:02
we'll take it up.
00:33:02
We'll take it up elsewhere.
00:33:05
We've got instead Ben Robinson.
00:33:07
We are an all-in-one show.
00:33:09
All-NFL draft show this week.
00:33:10
We're gonna return to previously scheduled topics like
00:33:13
baseball.
00:33:14
Adi has some primary research, some new primary
00:33:16
research he wants to share.
00:33:18
We'll talk about it.
00:33:19
But in the meantime, we still are processing
00:33:20
what happened over the weekend.
00:33:22
NFL draft had its thang.
00:33:24
Ben Robinson, for those of you who don't
00:33:27
know, created and runs Grinding the Mox.
00:33:31
Ben has been on our show many times
00:33:33
over the years.
00:33:34
He was, and I think maybe it's safe
00:33:37
to say, the first aggregator of Mox out
00:33:42
there.
00:33:42
He was an early, at least an early
00:33:44
aggregator of Mox.
00:33:47
And he has, I would say he's grinded
00:33:50
over the years and built a little bit
00:33:53
of an empire, an influencer.
00:33:57
He's got a partnership now with the NFL.
00:33:59
He consults to teams.
00:34:02
More and more, the conversation around the draft
00:34:05
is around things like this, like the consensus
00:34:07
and big boards and how teams perform versus
00:34:10
those measures.
00:34:12
And so in many, in a very real
00:34:13
way, Ben's work has become central to the
00:34:16
conversation around the NFL draft.
00:34:19
It's been fun to see.
00:34:20
We felt like we caught him pretty early.
00:34:22
He's making, I think he's really making the
00:34:24
conversation smarter out there.
00:34:25
Even the analytics of it, like how do
00:34:27
you aggregate these things is become super interesting.
00:34:31
And so the methodologies are interesting.
00:34:33
So Ben, welcome back.
00:34:35
Thanks for making time for us.
00:34:36
Glad to have you.
00:34:37
Oh, it's an honor to be here.
00:34:39
And in terms of being an early Mox
00:34:42
draft aggregator, I was not the first.
00:34:44
I initially thought I was the first.
00:34:46
Okay.
00:34:46
But would you guess Brian Burke was the
00:34:48
first?
00:34:49
Of course he was.
00:34:50
Of course he was.
00:34:52
Tough to be first on anything ahead of
00:34:54
Brian Burke, turns out.
00:34:57
But you've certainly invested more over the years.
00:35:01
Give us a sense.
00:35:02
Give us an update of state of the
00:35:04
union of Grinding the Mox through the 2026
00:35:07
draft.
00:35:08
What's important?
00:35:09
What are the high points?
00:35:10
How should we think about your enterprise at
00:35:11
this stage?
00:35:12
Well, yeah.
00:35:13
Like I said, it's an honor to be
00:35:14
on.
00:35:15
So yeah, no, Grinding the Mox started off
00:35:16
as kind of a humble side project and
00:35:18
now we're a data and insights company.
00:35:22
And so, you know, this year I have
00:35:26
two team clients that I've worked with and
00:35:28
I, this year added five more.
00:35:30
So we're working with about a quarter of
00:35:32
the league is using our data in some
00:35:34
shape or form to inform their draft preparation
00:35:37
and draft planning and draft strategy as well.
00:35:41
All the internal tools that teams have at
00:35:43
their disposal to think about the draft and
00:35:45
how they want to maneuver and plan to
00:35:47
get either the players they want or the
00:35:49
kind of positions they wanted to target in
00:35:51
the draft.
00:35:52
But overall, we're trying to, you know, do
00:35:54
what we've always done, which is, you know,
00:35:56
using mock drafts, the wisdom of the crowds
00:35:58
and machine learning to predict the draft.
00:36:01
And so we have a really consistent hit
00:36:04
rate of getting about 80 to 85%
00:36:08
of the top 100 players matched.
00:36:10
This year was a little different, but then
00:36:12
in the first round, we're usually getting between
00:36:14
28 and 29 of the 32 in our
00:36:17
top 32.
00:36:18
And so that type of consistency is really
00:36:19
hard to repeat by most experts, because as
00:36:22
we know, in the wisdom of crowds, these
00:36:24
things go up and down and up and
00:36:26
down.
00:36:26
So the data can help a lot in
00:36:28
setting those expectations accurately.
00:36:30
And like you said, making people smarter consumers
00:36:33
and fans of the draft.
00:36:35
So two clarifying questions, just to stay with
00:36:38
this for a moment, more background.
00:36:40
You said you have seven NFL class.
00:36:42
Now, how do they use your data?
00:36:44
So there's a lot of firewalls that exist
00:36:47
between me and the teams that I work
00:36:48
with.
00:36:49
So I don't necessarily get to go into
00:36:52
the draft rooms and hear, this is what
00:36:54
we did, this is how we used your
00:36:55
stuff.
00:36:56
My general sense of just the state of
00:36:58
the art and knowing what I know about
00:37:00
the industry is that lately teams have been
00:37:03
developing internal simulators.
00:37:06
So the types of simulators you see online,
00:37:08
the teams are developing that internally with a
00:37:11
combination of public data, their internal grades, maybe
00:37:15
you might pay for some betting data too
00:37:17
and throw that in there.
00:37:18
And the decision-makers are using that in
00:37:21
the lead up to the draft.
00:37:23
You know, when we talked a lot in
00:37:24
the past about fourth down decision-making, we
00:37:26
were often saying, you know who's really good
00:37:27
at making fourth down decisions?
00:37:29
People who get a lot of experience with
00:37:30
that.
00:37:31
In game, it just doesn't happen very often.
00:37:34
And so we were saying, hey, they should
00:37:36
play more Madden.
00:37:38
That is like more realistic.
00:37:40
You'll get more of these scenarios where you'll
00:37:41
get to say, okay, this is the risk,
00:37:42
this is the reward.
00:37:43
And it's the same for the draft.
00:37:45
Having these like internal tools, these simulators that
00:37:48
can give more GMs more bites at the
00:37:51
apple.
00:37:51
You only get a certain number of picks
00:37:53
every year, a very small number.
00:37:55
And so being able to thread the needle,
00:37:57
think about your strategy.
00:37:58
If this thing happens, how does my strategy
00:38:00
change?
00:38:01
Having more bites at the apple is a
00:38:03
big piece of internal draft planning.
00:38:05
And so my data feeds into some of
00:38:07
those products.
00:38:09
I think some teams are also real time
00:38:11
in the draft using it to better understand
00:38:13
what's likely to happen in the next few
00:38:16
picks.
00:38:17
So for example, if a team gets a
00:38:18
call and is offered a trade and they
00:38:21
have to ask, if I trade back, what's
00:38:24
the chance my guy is still gonna be
00:38:25
there?
00:38:25
And they need a good sim to answer
00:38:27
that question.
00:38:28
And of course it has to be a
00:38:29
sim that's updated continually as people make picks.
00:38:31
And so I can imagine your data being
00:38:33
super helpful for that process.
00:38:35
Last clarifying question before we go to Eric,
00:38:38
how have you gotten better over time?
00:38:40
I'm guessing that these are more accurate than
00:38:42
they used to be.
00:38:43
What is the frontier for you as you
00:38:45
improve your model over time?
00:38:47
I think the main thing is about data
00:38:49
innovation.
00:38:52
Methodologically speaking, teams are largely using models that
00:38:55
have been around for a while.
00:38:58
You'll hear about like the, my model is
00:39:01
the model stuff that I sell to teams
00:39:02
is like a Bayesian hierarchical model.
00:39:04
It's not actually super complicated.
00:39:06
Those types of models have been around for
00:39:07
a while.
00:39:09
You'll hear about like other types of methodologies
00:39:11
as well.
00:39:12
But for me, it's a lot about data.
00:39:14
It's less about necessarily, hey, let's try to
00:39:19
like methodologically fix this issue.
00:39:21
Because there's a lot of uncertainty in the
00:39:23
draft.
00:39:24
And so to me, finding the ways to
00:39:26
inject new data sources is what's gonna make
00:39:28
your model, I think, better.
00:39:30
The better data you'll usually weigh out over
00:39:32
the better method in this case.
00:39:34
There is like some amount of evidence to
00:39:38
be made that if you wanna get a
00:39:40
very accurate simulation in some ways projection, though
00:39:44
a lot of the mock drafts are kind
00:39:45
of copying each other in very like mimicky
00:39:48
ways.
00:39:49
And so you still, but you still need
00:39:51
to be able to find those needles in
00:39:52
the haystack that matter.
00:39:53
And so you need to have access to
00:39:55
as much of the data as you possibly
00:39:57
can so that you can find the people
00:39:59
that are giving you the appropriate signal that
00:40:01
have the track record of being accurate.
00:40:03
And then don't just mimic what everyone else
00:40:06
is saying.
00:40:06
It actually has something that brings value to
00:40:08
the conversation.
00:40:09
So I think you still need data and
00:40:11
that's the thing.
00:40:12
So I've integrated big boards as well, just
00:40:14
like straight up player rankings into my data
00:40:16
as well over the years.
00:40:17
And that's been a meaningful addition to help
00:40:20
provide priors.
00:40:21
But to me, it's more about just the
00:40:22
data you can bring.
00:40:23
Like I mentioned before, the mock drafts, the
00:40:24
big boards, some type of, whether it's like
00:40:27
gambling odds or maybe even prediction market odds,
00:40:30
depending on those, that can add a lot
00:40:32
to a value and not necessarily the modeling
00:40:35
approach.
00:40:35
That can be fairly straightforward.
00:40:38
Eric.
00:40:39
I don't know if this has ever happened
00:40:41
on our 12 years of Moneyball where the
00:40:43
guest has literally answered every single point of
00:40:46
my question.
00:40:47
I was gonna ask him, would you rather
00:40:48
have better data or better models?
00:40:51
I was gonna ask him how he deals
00:40:52
with the correlated nature of the drafts.
00:40:55
Like I just copy what Cade Massey puts
00:40:58
up there and now you got seemingly two
00:41:00
observations that are convergent, but I've actually added
00:41:02
no information.
00:41:04
But I will ask one question then.
00:41:06
But first of all, thank you for the
00:41:07
clarification.
00:41:09
In a world where everyone seems to be
00:41:13
using neural nets and more complicated nonlinear types
00:41:17
of models today, despite I'm a Bayesian and
00:41:19
I'm Mr. Bayesian hierarchical model, I love fitting
00:41:22
them.
00:41:23
Why not jam a massive neural net against
00:41:27
your data where you can find all possible
00:41:30
combinations of things that help you predict outcomes
00:41:33
or like why in some sense be more
00:41:36
statistician and probabilistic-like and why not jam
00:41:39
the old black box computer science model at
00:41:42
it?
00:41:43
I think a lot of it has to
00:41:44
do with the uncertainty in the data that
00:41:47
you're having.
00:41:48
So the mock draft world of data is
00:41:50
a censored world of data.
00:41:52
Most mock drafts are only 32 picks long.
00:41:55
The first round is what gets the most
00:41:56
eyes, the most attention.
00:41:58
And so you're dealing with data that just
00:42:00
has a bunch of built-in issues I
00:42:04
think that statistical models do a better job
00:42:06
of addressing.
00:42:07
I think you can throw the machine learning
00:42:10
methodology at it too and you probably would
00:42:13
get pretty decent results.
00:42:15
But I do think that the uncertainty, like
00:42:17
being able to account for the uncertainty and
00:42:19
actually understanding the generation, like how your data
00:42:22
is generated, what it represents, you wanna try
00:42:25
to match that to the problem.
00:42:26
But then I think the other thing too
00:42:27
is just- You score yourself, Ben, beyond
00:42:28
just I'll call it the pure hit rate.
00:42:30
Like in other words, being off by one
00:42:32
is very different than being off by 15.
00:42:35
And in some sense, you could imagine a
00:42:38
black box model always in general underestimates uncertainty.
00:42:42
And so you wanna- And you feel
00:42:44
better about that in that way.
00:42:45
Yeah.
00:42:46
So I mean, you could come up with
00:42:47
a model that could be 95% right,
00:42:49
but then the confidence intervals are huge and
00:42:51
it just has no meaningful interpretation.
00:42:54
But to me, there's just a lot of
00:42:55
uncertainty with the draft.
00:42:57
And so you just kind of have to
00:42:57
be, get used to being wrong a lot.
00:43:01
And so, and then there's also these correlated
00:43:03
errors you mentioned.
00:43:04
So in the first round of the draft
00:43:07
this year, my mean absolute error, just like
00:43:10
very basic metric of prediction accuracy was very
00:43:14
average.
00:43:15
The first round, there might've been a couple
00:43:17
big picks, but for the most part, the
00:43:19
players went right around where we thought they
00:43:21
would go.
00:43:22
Especially if we value higher picks more than
00:43:25
later picks.
00:43:27
But then if you move into the top
00:43:28
100 and that's the first three rounds, this
00:43:32
year's draft, in terms of like my average
00:43:35
mean absolute error, just without adjusting for any
00:43:37
pick value, it was seven picks higher on
00:43:40
average.
00:43:41
So that means in the top 100 this
00:43:43
year, there was just a lot more uncertainty.
00:43:46
And that's just with my static pre-draft
00:43:47
model.
00:43:48
I don't have like a live model that
00:43:49
I use.
00:43:51
The teams obviously have these tools to be
00:43:53
able to make those assessments in draft.
00:43:55
But yeah, there was a lot more uncertainty.
00:43:57
And I think some of that was because
00:43:58
of these correlated errors that you could pick
00:44:01
up.
00:44:02
So one of the things that was the
00:44:03
story of round two and three of the
00:44:04
draft this year was the rise of the,
00:44:06
what I would call the overpriced blocking tight
00:44:09
end.
00:44:10
The teams, a lot of NFL teams are
00:44:13
following the money around what is innovation in
00:44:16
the NFL.
00:44:17
And the Rams ran a lot of three
00:44:20
tight end sets where you need a blocking
00:44:21
tight end to have the threat of the
00:44:23
run game and teams want to be able
00:44:24
to run those.
00:44:25
And so they were like, well, if that's
00:44:26
the new scheme innovation, we want to be
00:44:31
able to get that.
00:44:32
It bumped up the price in the draft
00:44:34
of the blocking tight ends.
00:44:37
Seth Walter from ESPN asked a bunch of
00:44:39
team sources about a bunch of players who
00:44:40
went earlier than expected and asked them, is
00:44:42
that really your reach?
00:44:44
And for the blocking tight ends, he said,
00:44:46
actually, I think they were just a lot
00:44:48
very undervalued by the draft community this year
00:44:50
and overvalued or maybe more valued correctly, right?
00:44:54
By the teams.
00:44:55
So these, you could pick these up in
00:44:57
a model, but if you just jammed everything
00:44:59
into a machine learning model, would miss that
00:45:01
all the time.
00:45:01
Real quick point about that dynamic.
00:45:04
This is a classic case of non-stationarity.
00:45:07
The world is changing.
00:45:08
So we don't know who's got the valuation
00:45:10
correct, but we know if everyone's playing offense
00:45:13
or a lot of teams are playing offense
00:45:14
differently now, however we valued positions in the
00:45:18
past will be less relevant.
00:45:20
And so it's an open question on where
00:45:22
the tight end value is going, but it's
00:45:24
just this perennial challenge of modeling when the
00:45:26
world is moving.
00:45:29
Adi.
00:45:30
So actually we did a little impromptu seminar
00:45:33
on draft valuations, but we weren't thinking at
00:45:37
all about kind of what you're thinking about.
00:45:38
Just to, so I want to clarify, you're
00:45:40
trying to predict where the players will actually
00:45:42
go in the actual draft.
00:45:44
Yes.
00:45:44
As opposed to figuring out, well, who's the
00:45:47
good player that's going to go lower or
00:45:49
higher?
00:45:50
And you're trying to model future value, which
00:45:52
is what we were talking in the first
00:45:54
half hour of our segment.
00:45:56
So my, so the question is, is that
00:45:58
something that teams want to know from you?
00:46:00
Or, I mean, are they just interested in
00:46:02
knowing the person they want, where is it
00:46:05
likely to go and how do they have
00:46:06
to trade or position their order so that
00:46:09
they make sure they get their people?
00:46:11
So to answer your first question, so I'm,
00:46:15
yeah, I'm trying to predict where players will
00:46:16
get drafted.
00:46:18
So my friend Arif Hasan creates every year
00:46:21
since like 2015, what he calls a consensus
00:46:23
big board, where he gets hundreds of player
00:46:26
rankings and aggregates them together.
00:46:28
Recently, he did a little analysis on his
00:46:30
sub stack, which is called Wide Left.
00:46:33
And he analyzed how good different publicly available
00:46:37
metrics were at predicting where players are going
00:46:39
to get drafted.
00:46:40
And also how good some of these metrics
00:46:43
were at predicting future value based on the
00:46:45
pro football reference, approximate value stat, which is
00:46:48
probably one of the best stats we have,
00:46:49
you know, for looking at value.
00:46:53
And my data predicts the draft the best.
00:46:55
I'm very proud of that.
00:46:57
I think I do a nice job.
00:46:59
It's good to hear.
00:47:00
My data predicts actual NFL success of those
00:47:04
publicly available metrics, the second best.
00:47:07
The thing that actually predicted NFL valued the
00:47:10
best was actually his board of player rankings.
00:47:13
But by me being closer to the actual
00:47:16
draft in the top 100, you end up
00:47:19
getting a lot more of those right than
00:47:22
wrong.
00:47:23
I still think he needs to do like
00:47:24
an additional analysis where he compares it to
00:47:25
the actual draft.
00:47:27
But by being closer to where these top
00:47:29
100 players should be selected, we do a
00:47:31
better job of predicting the draft than even
00:47:33
some of like the other publicly available aggregated
00:47:36
boards out there.
00:47:38
So sometimes you do want to try to
00:47:40
be as close to the market as possible.
00:47:43
And I know that Timo Rieske from, I
00:47:45
don't know if he's still at pro football
00:47:46
focus or not, but had done this analysis
00:47:48
in the past about, you know, do you
00:47:51
wanna be closer or further away from the
00:47:53
consensus?
00:47:54
Where does it matter?
00:47:55
When does it matter?
00:47:57
And so it does have some impact, whether
00:48:00
they do the teams wanna use my data
00:48:01
to predict where players are gonna be drafted
00:48:03
or where, yes.
00:48:04
Did they wanna use it to predict how
00:48:06
good players are gonna be?
00:48:07
Player evaluation, no.
00:48:09
And so I think the vanguard of player
00:48:12
evaluation is gonna be trying to beat the
00:48:13
draft, not trying to approximate the draft.
00:48:16
But I do think it's something you should
00:48:17
consider.
00:48:18
It might make your model a little smarter
00:48:19
pre-draft.
00:48:21
But yeah, I think teams are doing a
00:48:24
lot more different types of things on player
00:48:26
evaluation on the data side.
00:48:29
And I'm still not a hundred percent sure
00:48:31
that it's making its way into the decision
00:48:34
makers.
00:48:35
Like, I don't know how much it's infiltrating
00:48:36
decision-making.
00:48:37
There's a lot of alignment that has to
00:48:38
happen organizationally for insights to get from the
00:48:42
draft, from the analyst room into the draft
00:48:44
room.
00:48:45
I remember reading Future Value, which is a
00:48:48
great book about, you know, prospecting in baseball.
00:48:52
And there were teams that were drafting entirely
00:48:54
from models.
00:48:56
I don't think we're anywhere near that in
00:48:58
the NFL yet.
00:48:59
Scouts still dominate the conversation.
00:49:01
And I think we've also learned in other
00:49:03
sports that scouts have really important data that
00:49:05
they bring to the table about players.
00:49:08
But yeah, teams are using this to try
00:49:10
to kind of figure out how the draft
00:49:13
might play so that they can find those
00:49:14
specific players that they want.
00:49:16
And so what I find is that the
00:49:19
teams that are reaching the most in my
00:49:21
data for players are usually just, they just
00:49:23
don't consider that.
00:49:25
They just have thought before the draft.
00:49:31
And we might use the analytics department to
00:49:33
make sure we're in a position to get
00:49:34
the guy we want in that area that
00:49:37
we want.
00:49:37
But we're not using them to figure out
00:49:39
which players to pick.
00:49:40
We're super scouts.
00:49:41
We're the smartest guys in the room.
00:49:43
We're gonna pick the guys we want based
00:49:45
on the coaches we have to maximize the
00:49:47
opportunity to still have our jobs next year.
00:49:50
Can I ask a quick follow-up question?
00:49:52
Just like a real, because if you imagine,
00:49:55
if you look back at the draft years
00:49:57
later and you rank the players by quality
00:49:59
and then you rank them by their draft,
00:50:01
how often does the top five players in
00:50:03
the draft by quality get drafted in the
00:50:08
top five?
00:50:09
Is that?
00:50:10
Yeah, just by the way, as an extension
00:50:11
to the question I was gonna ask you,
00:50:13
Ben, by the way, was a related one
00:50:15
to Adi's, which is how much of the
00:50:17
variation does, if someone just did pure, forget
00:50:21
what position I need, forget any of that.
00:50:23
I just rank according to perception of quality
00:50:26
and I correlate that with grinding the mops.
00:50:31
Adi wanted the top five.
00:50:32
I wanted, in general, for the top hundred.
00:50:35
How much variation is sucked up just by
00:50:37
the pure quality rankings versus, let's call it,
00:50:40
the context of I need a tight end,
00:50:42
I need a running back, et cetera?
00:50:44
So we have a student who did this
00:50:45
for the NBA.
00:50:46
I know the number.
00:50:47
I'm curious to know what the number is
00:50:48
for the NFL.
00:50:50
This was exactly, I have another question for
00:50:52
Ben, but this is my question.
00:50:54
So the number for the NBA is one
00:50:55
and a half.
00:50:56
So of the top five players in a
00:50:58
draft class in the NBA, one and a
00:51:00
half are drafted in the top five.
00:51:02
Interesting.
00:51:03
I wish you hadn't given that number yet.
00:51:05
I was gonna ask everybody to guess.
00:51:07
Yeah.
00:51:07
Well, I would have guessed the answer to
00:51:09
my question was about a quarter to a
00:51:11
third of the variation in where people are
00:51:14
drafted, it would have been based on pure
00:51:18
quality.
00:51:19
Like if all Ben did was use a
00:51:22
consensus not of where people are gonna be
00:51:24
drafted, but of the rank order of quality,
00:51:28
that that might suck up 25% to
00:51:30
30% at most of where people are
00:51:33
actually drafted.
00:51:34
But Ben, maybe you know the answer to
00:51:35
this.
00:51:36
I don't really know the answer.
00:51:37
I think it's my data is very correlated
00:51:40
to the draft.
00:51:42
So for the most part, my data might
00:51:44
be saying, hey, like this guy is expected
00:51:47
to go here and he goes there.
00:51:49
There's like differentials between where players were expected
00:51:51
to go and where they go.
00:51:52
That's kind of what we know is that
00:51:54
what Timo Riske has brought into this conversation,
00:51:56
which is that draft reaches are real and
00:51:59
steals don't necessarily exist.
00:52:02
My bet is that it would probably be
00:52:04
more than one and a half.
00:52:07
So there's the classic, this is the classic
00:52:09
draft curve literature and actually our friend, Ron
00:52:13
Yurko at Carnegie Mellon, they did a little
00:52:16
bit of an update on the draft curve.
00:52:19
How do you, do you optimize your draft
00:52:21
curve for just value?
00:52:22
Do you optimize it for superstar?
00:52:24
And they were doing it with Madden rating
00:52:25
data.
00:52:26
And so that was a new analysis that
00:52:30
they released in recently that probably would do
00:52:31
a better job of telling you those sorts
00:52:33
of things.
00:52:34
I don't necessarily think we have the public
00:52:36
data to tell us necessarily who is that
00:52:40
or not.
00:52:41
I think the Madden data might be some
00:52:42
of the best data we actually have publicly
00:52:43
to do that across positions.
00:52:45
But I would like to think, I think
00:52:46
it would be higher than one and a
00:52:49
half, but I don't know if I would,
00:52:51
I would say probably maybe two.
00:52:52
I don't know if I would say anything
00:52:53
more than that.
00:52:54
I can answer the question this evening if
00:52:56
we just consider the second contract value, which
00:52:59
is I think a reasonable way to go
00:53:00
at it.
00:53:01
And I'd be curious to get everybody's number
00:53:03
out of five on average, how many of
00:53:06
the top five contracts, once they reach free
00:53:09
agency, how many of those top five in
00:53:11
a draft class were drafted in the top
00:53:13
five?
00:53:15
I'm going under the one and a half
00:53:17
number that Adi just said.
00:53:18
Under one and a half.
00:53:19
That's strong.
00:53:20
That is strong.
00:53:21
It's an average.
00:53:22
So you can pick a non-integer.
00:53:26
2.5. 1.2. Yeah.
00:53:30
Under.
00:53:32
I'm going under also.
00:53:35
Ben, let me ask you.
00:53:36
Not around 1.2. I like your number,
00:53:38
Eric.
00:53:38
You anchored me.
00:53:40
God damn it.
00:53:40
Let me ask you a question.
00:53:41
My feeling is that usually in a draft
00:53:44
class, you usually have a couple of quarterbacks
00:53:47
that go really high.
00:53:48
That's the thing.
00:53:49
The quarterbacks are gonna get you.
00:53:51
This year's class might be biasing you to
00:53:53
think that this is not an average quarterback
00:53:56
class, but I do think that, yeah, like
00:53:58
in a more normal quarterback class, there will
00:54:00
be two quarterbacks that go in the top
00:54:02
five and then they have a decent chance
00:54:04
of being in those numbers or an edge
00:54:07
rusher or a tackle or, you know.
00:54:10
So like when you get to drafts and
00:54:13
you see like are these non-premium positions
00:54:15
going in the top five a lot?
00:54:17
Sometimes.
00:54:18
This year it did.
00:54:20
And so, but yeah, overall, I think it's
00:54:23
more than one and a half, but yeah,
00:54:25
maybe I'm just over-indexing on my mental
00:54:28
model.
00:54:30
I'm gonna go under.
00:54:30
I'm gonna go under as well.
00:54:32
What is your thought?
00:54:33
I mean, I can't even tell you the
00:54:34
dismay of my son, one of my three
00:54:37
sons who does a lot of work in
00:54:38
sports analytics, trained by Adi.
00:54:41
When Jeremiah Love was drafted third, he was
00:54:44
apoplectic.
00:54:45
He just could not believe it.
00:54:48
So did you have him going that high
00:54:51
in the draft?
00:54:52
Forget whether he's the right, it's the right
00:54:53
choice.
00:54:54
No, I understand that's not what you look
00:54:55
at.
00:54:56
Did you have Jeremiah Love going like a
00:54:58
third to the Cardinals and was that what
00:55:01
the consensus said?
00:55:03
So in the grindingthemocs.com public expected draft
00:55:07
position board, he was my fourth ranked player.
00:55:10
In the Bayesian model, he actually was ranked
00:55:12
third.
00:55:13
So, but the idea, I think the vast,
00:55:17
that was kind of his ceiling was gonna
00:55:19
be pick three.
00:55:20
And he went there.
00:55:21
It definitely changed the complexion of the top
00:55:23
five of the draft.
00:55:24
I think it did a lot of other
00:55:26
teams some really strong favors because it definitely
00:55:31
shook up.
00:55:32
I think that if Jeremiah Love hadn't been
00:55:34
drafted third overall, he would have gone fourth
00:55:36
overall.
00:55:37
I think that would have been a big
00:55:38
disservice to Cam Ward who the Titans drafted
00:55:41
first overall last year to draft a running
00:55:43
back that doesn't really help him at all
00:55:45
or change the offense in a really meaningful
00:55:47
way for him and like the evaluation of
00:55:50
him as a quarterback.
00:55:51
But yeah, I had him going that high.
00:55:53
It's very regularly known that we'll have these
00:55:56
conversations around positional value of players especially at
00:56:00
the top of the player rankings.
00:56:02
They just tend to go for the most
00:56:04
part at the top of the player rankings.
00:56:06
The one position that we see year in
00:56:09
year out where if a player is ranked
00:56:11
high, and we saw that this year with
00:56:12
Ohio State safety, Caleb Downs, safeties get depressed.
00:56:17
Their value, they always are going a good
00:56:19
amount later than we thought in the draft.
00:56:22
So teams are willing to draft running backs
00:56:24
really high.
00:56:25
They're willing to draft linebackers really high.
00:56:28
Safeties for whatever reason, they get pushed down
00:56:31
relative to expectation.
00:56:33
So Caleb Downs was one of my three
00:56:35
biggest fallers in the draft.
00:56:37
He was my seventh ranked player and he
00:56:39
ended up going 11th.
00:56:41
And so that's sort of the one thing
00:56:44
that is the exception to the rule of
00:56:46
highly ranked players go high in the draft
00:56:49
as safeties.
00:56:50
And I don't have a great answer for
00:56:52
it, but we saw this year the Seahawks
00:56:56
win the Super Bowl, not drafting a safety
00:56:58
in the first round, although I did have
00:57:01
Nick M.
00:57:02
Unwary from South Carolina in last year's draft
00:57:04
as a late first round value in my
00:57:07
numbers.
00:57:08
But that safety position, if you can find
00:57:11
that perfect blend of someone who can play
00:57:15
the run in the box and then cover
00:57:17
tight ends and receivers up the seam, that's
00:57:19
become a lot more valuable.
00:57:21
But for some reason we're spending high draft
00:57:23
picks on blocking tight ends and not as
00:57:26
much on safeties that can make a difference
00:57:29
in the defense.
00:57:31
Ben, you mentioned that Downs was one of
00:57:32
your big three fallers.
00:57:34
Can you give us the others and then
00:57:36
can you give us those players that went
00:57:38
most outperformed?
00:57:40
They did better than expected.
00:57:42
Who were the surprises for you in that
00:57:43
direction?
00:57:44
So the three biggest surprises on day one
00:57:49
of the draft, number one was Alabama quarterback
00:57:52
Ty Simpson.
00:57:53
So that's like a whole podcast in and
00:57:57
of itself around what the implications are of
00:58:00
drafting a quarterback in the first round who
00:58:04
I did not think would go in the
00:58:05
first round at pick 13 overall.
00:58:10
But yeah, so he went at pick 13.
00:58:12
There were very few people who projected him.
00:58:15
I had this metric weighted mock drafts where
00:58:18
basically not every mock draft is created equal.
00:58:20
Some mock drafts are created closer to the
00:58:22
draft.
00:58:23
Some mock drafts are made by smarter predictors
00:58:26
and he only showed up in 4%
00:58:29
of mocks to the Rams that were in
00:58:31
his weighted mocks.
00:58:32
There was a lot of noise of him
00:58:34
going to be traded up for by the
00:58:37
Arizona Cardinals, but that didn't happen.
00:58:39
So that was the biggest shock of the
00:58:41
first round was Ty Simpson.
00:58:43
But the Minnesota Vikings, they drafted defensive tackle
00:58:46
Caleb Banks from Florida who I also thought
00:58:49
was a second round selection.
00:58:52
Coming out of the senior bowl, he was
00:58:54
one of my biggest risers in my data,
00:58:56
had a really lovely senior bowl.
00:58:59
He's a big man and he has been
00:59:01
dealing with some injuries his entire career at
00:59:03
Florida.
00:59:04
And he re-aggravated an injury at the
00:59:07
Combine and he was already kind of a
00:59:10
inconsistent player, but you can definitely watch a
00:59:12
couple of games and see like there are
00:59:14
some real flashes of brilliance there.
00:59:18
In terms of some of the biggest fallers
00:59:20
in my data, players that fell the most
00:59:23
kind of in the draft, at least on
00:59:24
night one, one of the biggest values in
00:59:26
the draft this year was Ohio State linebacker
00:59:30
slash edge rusher, Arvel Reese.
00:59:33
He was in the running to be the
00:59:36
second overall pick.
00:59:36
And I think most people believe that if
00:59:38
he wasn't the second overall pick to the
00:59:40
Jets, that he would be the third overall
00:59:41
pick.
00:59:42
Didn't he go five?
00:59:43
He went fifth.
00:59:44
And so that was really surprising.
00:59:46
I think that that was one of the
00:59:48
biggest surprises of the draft.
00:59:50
And I think that if Jeremiah Love, like
00:59:52
I said, like that was, I think that
00:59:55
the Giants were probably thinking about some of
00:59:57
the other guys that would be available and
00:59:58
were completely taken aback.
01:00:00
I wouldn't have surprised me if they had
01:00:02
maybe had Reese number one on their board
01:00:04
at the beginning of the draft.
01:00:06
So they got a ton of value.
01:00:08
I think they had a great draft.
01:00:09
They also drafted a tackle at pick 10
01:00:11
and Francis Maui Noah from Miami.
01:00:14
So I thought that was really great value.
01:00:18
I also thought the Eagles got some good
01:00:20
value from this Makai Lemon draft selection.
01:00:24
I thought that was like a little surprise
01:00:26
that he fell a little bit.
01:00:27
Ruben Bain from Miami.
01:00:30
He fell a little bit in the draft.
01:00:32
So the draft is full of these like
01:00:34
little surprises.
01:00:34
So far I'm liking this.
01:00:35
It's the Eagles and the Buccaneers.
01:00:36
I'm liking this so far.
01:00:38
Don't interrupt him, Cade.
01:00:39
Let Ben keep talking.
01:00:40
As a Pats fan, I'm also happy the
01:00:42
Eagles selected a wide receiver.
01:00:43
Yeah, yeah.
01:00:45
But yeah, there's all these great stories.
01:00:49
The other guy that was the surprise really
01:00:52
was Omar Cooper Jr., the wide receiver from
01:00:55
Indiana.
01:00:57
Before the combine, he was thought of as
01:00:59
a second round pick and then people really
01:01:02
looked into him more and liked his profile
01:01:03
a lot.
01:01:04
And the Jets saved him the indignity of
01:01:07
having to be a day two pick and
01:01:09
took him via a trade up at the
01:01:12
end of round one.
01:01:13
So that was a big surprise.
01:01:14
I had him as my 20th ranked player
01:01:15
and he ended up going 30.
01:01:18
Ben, that reminds me of a feature of
01:01:21
your work that is notable, especially before the
01:01:24
draft.
01:01:25
You plot the trajectory of these players' projections
01:01:29
over time.
01:01:30
So you show guys not rising or falling
01:01:32
on draft day, but like in the lead
01:01:34
up to the draft, in the weeks up
01:01:35
to the draft, guys move up and down
01:01:37
the mock board.
01:01:38
What have you learned over time about characteristics
01:01:41
that lead guys to go up and down
01:01:43
ahead of time?
01:01:45
I think a lot of the time, in
01:01:48
the preseason, there's a lot of variance.
01:01:52
Over the cycle this year, there's usually a
01:01:56
lot more fallers than risers.
01:01:58
So the risers usually just come out of
01:01:59
nowhere, like you didn't know about them or
01:02:01
they were very low on your radar preseason.
01:02:05
This year, there were at least three quarterbacks
01:02:07
who I thought were gonna potentially have the
01:02:10
profile of being players that would be undrafted.
01:02:13
Penn State's quarterback, Drew Aller, Clemson's Cade Klubnick,
01:02:17
and LSU's Garrett Nesmeyer.
01:02:19
Coming into the preseason, all were considered potential
01:02:22
top quarterback prospects.
01:02:24
And along with some other prospects deciding not
01:02:26
to enter the draft this year, that made
01:02:28
the quarterback class really kind of unexciting.
01:02:32
But Fernando Mendoza, you also, it takes time
01:02:35
to kind of, for people to update their
01:02:37
priors.
01:02:38
So Fernando Mendoza has a couple of good
01:02:40
games.
01:02:41
You didn't think much of him coming into
01:02:42
the season, although I think some people did
01:02:44
think that he was potentially on the radar
01:02:47
for prospects.
01:02:48
But as he starts to really rack up
01:02:50
these things and you get the sense that
01:02:52
it's not just Indiana beating down on Ball
01:02:55
State or whoever, and they start to play
01:02:58
well against Penn State and Oregon, you kind
01:03:00
of begin to update your priors and then
01:03:02
you kind of, it takes a while for
01:03:04
those things to change in a meaningful way
01:03:07
for the crowd.
01:03:08
So the players that do come into the
01:03:10
consciousness, they come into it in a big
01:03:12
way.
01:03:12
So the big risers this year were Arvel
01:03:15
Reiss and Fernando Mendoza.
01:03:16
But Arvel Reiss is probably one of the
01:03:17
biggest risers in the draft this year, just
01:03:19
because in the preseason, they thought he was
01:03:22
just like a linebacker who was gonna be
01:03:24
playing linebacker.
01:03:25
No one really cared about him.
01:03:26
But then he started showing up in all
01:03:28
these games and you begin to kind of
01:03:30
consider him as potentially the top prospect in
01:03:33
the draft, somebody who you would pick at
01:03:35
second overall, despite not really having played Ed
01:03:38
Drescher at all as a full-time position.
01:03:40
All right, we're gonna have to cut you
01:03:42
loose, but Eric's got a last question for
01:03:44
you.
01:03:44
It's gonna be a yes, no answer.
01:03:46
Are there team effects in here?
01:03:47
Indiana wins- I was gonna ask, oh,
01:03:49
that team effect, okay.
01:03:50
No, no, team effects, like Indiana wins-
01:03:52
The college team, college team.
01:03:53
And now everybody, it relates to Cade's question
01:03:55
about risers.
01:03:56
Like Indiana wins the national title, then all
01:03:59
of a sudden wide receivers in Indiana are
01:04:01
going higher than expected and all.
01:04:02
How does that work?
01:04:04
I think a little bit.
01:04:07
I think you'd like to think that that's
01:04:08
correlated, right?
01:04:09
Like if Indiana's winning the title, they should
01:04:11
have a lot of draftable players.
01:04:14
There's so much attribution there.
01:04:16
Like coaching makes a big difference.
01:04:19
Obviously playing, you're playing at different positions means
01:04:21
different things.
01:04:22
Luck can mean different things.
01:04:25
But yeah, to me, yeah, there's definitely a
01:04:26
lot more of a microscope on players on
01:04:30
good teams.
01:04:32
And coming into the playoffs, I thought Fernando
01:04:34
Mendoza had done everything he needed to do
01:04:35
to prove that he was the number one
01:04:36
overall pick.
01:04:38
I thought he had done everything he needed
01:04:39
to do.
01:04:40
But then there were other players who even
01:04:41
produced well and people were still saying that
01:04:43
they needed to do more.
01:04:44
And so for example, that was kind of
01:04:45
like Ty Simpson.
01:04:46
Coming into the playoffs with Alabama, he had
01:04:51
been pretty up and down.
01:04:52
He had gotten hurt.
01:04:52
He had played well against some like low
01:04:54
level competition, struggled against some others.
01:04:57
And to be honest, I really didn't think
01:04:59
he did a lot more.
01:04:59
I was surprised that he went as high
01:05:02
as he did.
01:05:03
But yeah, in terms of team effects on
01:05:04
the college side, I think it's kind of
01:05:07
just baked in in the revealed preferences.
01:05:09
On the team side, you were mentioning like,
01:05:12
are there team effects in my model?
01:05:13
My model is team agnostic.
01:05:15
I do think that there's not enough historical
01:05:19
track record for most general managers for us
01:05:21
to be able to say anything meaningful.
01:05:23
And so I choose to believe that you
01:05:26
want to inject less certainty into your model.
01:05:28
And so I don't mean to throw any
01:05:30
shade, but ESPN has this draft predictor or
01:05:32
draft day predictor model.
01:05:34
And one of the things, and I'm going
01:05:36
to write about it this summer.
01:05:37
I've been collecting data from the draft day
01:05:39
predictor for a long time, like ever since
01:05:41
it's been live.
01:05:43
And if you do, I'm going to do
01:05:45
some type of like Breyer score analysis of
01:05:46
looking at what your prediction said around the
01:05:50
time when the player was going to go
01:05:51
and then the actual, you can make that
01:05:52
comparison.
01:05:54
And my guess is that when you do
01:05:55
tend to inject, like the teams are only
01:05:57
considering these players based on scouts, inks, needs,
01:06:00
or the teams are only going to consider
01:06:02
this player at this pick because of that.
01:06:05
You're just putting your finger on the scale
01:06:06
when you need to be letting it ride,
01:06:08
letting the randomness, the flatness of some of
01:06:11
these priors instead of trying to over-priorize
01:06:13
your model.
01:06:14
Super interesting.
01:06:15
You're full on flirting with us when you're
01:06:17
talking Breyer scores.
01:06:18
I mean, come on.
01:06:18
Just directly flirting.
01:06:21
Why not log loss?
01:06:23
Yeah, yeah, exactly.
01:06:25
I think we'll do both.
01:06:26
We'll do both.
01:06:27
Just an example.
01:06:28
Do it both ways.
01:06:29
Exactly, do it both ways.
01:06:31
All right, Ben, we should let you go,
01:06:33
man.
01:06:33
Thank you for making time.
01:06:34
Congrats on making it through another draft season.
01:06:36
Good job.
01:06:37
And we'll look forward to seeing what you
01:06:38
do next year.
01:06:39
Yeah, thank you guys so much.
01:06:40
And like you mentioned, my data is everywhere,
01:06:43
but this past year, grindingthemocs.com, my grinding
01:06:46
themocs substack, I'm writing there pretty regularly.
01:06:50
And then I also had my data in
01:06:52
NFL IQ, which is NFL Next Gen Stats
01:06:55
off-season dashboard, which was pretty cool and
01:06:57
exciting.
01:06:58
And yeah, no, this year was great.
01:07:00
Getting to work with about a quarter of
01:07:01
the teams in the league has been something
01:07:02
that I've been working towards for many years.
01:07:05
So it's a long time coming.
01:07:06
It's great to be back on with you.
01:07:08
Great fun, good work.
01:07:10
You've been grinding.
01:07:11
You have literally been grinding and it's good
01:07:12
to see a payoff.
01:07:13
That has been Robinson, founder and CEO of
01:07:16
Grinding the Mocs.
01:07:17
And that has been a full NFL Draft
01:07:19
Show.
01:07:19
We seem to be themed these days.
01:07:20
About two weeks ago, we did a full
01:07:22
golf show.
01:07:23
That one was unintentional.
01:07:24
This one was planned.
01:07:24
We'll get back to a little more open
01:07:27
-minded, broad-minded, including baseball.
01:07:29
And let me point out to some of
01:07:31
you guys, they're playing playoff hockey right now.
01:07:33
It's the best thing going.
01:07:34
It's great.
01:07:35
Enjoy it.
01:07:37
For the whole team here, for Eric Bradlow,
01:07:39
Shane Jensen, Adi Weiner.
01:07:41
For our guests, Ben Robinson and Richard Thaler.
01:07:44
For our production team, Dion Simpkins, making it
01:07:47
always happen.
01:07:49
Marissa Patel, I mean, Marissa Reyna.
01:07:51
And Dee Patel, the boss lady, Dee Patel.
01:07:53
Thank you guys for everything.
01:07:54
Thank you guys for listening.
01:07:55
Come back and join us next time.
01:07:56
Between now and then, enjoy your sports.
