00:00:01
Welcome, welcome to Wharton Moneyball,
00:00:03
the podcast edition here on the Wharton
00:00:06
podcast network. This is Eric Bradlo,
00:00:08
professor of marketing, statistics, and
00:00:10
data science here at the Wharton School.
00:00:11
As everyone who's listened to us for the
00:00:13
last 11 plus years knows, some
00:00:15
combination of myself, Kate Massie, Audi
00:00:17
Winer, Shane Jensen, sports never stops.
00:00:21
Data science and statistics never stops.
00:00:23
So, we're here every week on Wharton
00:00:25
Moneyball. I've always said that one of
00:00:28
my greatest honors of doing this show
00:00:30
again for the last 11 plus years is
00:00:32
having people that are living, breathing
00:00:34
analytics applied to sports all day
00:00:37
long. I just do it as a part-time job. I
00:00:40
got the regular job doing my own
00:00:41
research, teaching other stuff. And
00:00:43
certainly today is no exception. Um,
00:00:45
we're joined today by the I don't even
00:00:48
know how many times Aaron's been on the
00:00:49
air. I'm going to say 10 plus. It's got
00:00:51
to be more than that because whatever.
00:00:53
We're joined today by Aaron Shatz. is
00:00:55
chief analytics officer of FTN Fantasy.
00:00:57
A lot of people also know his work on
00:00:59
ESPN. A lot of our listeners certainly
00:01:01
know him as the founder of uh Football
00:01:04
Outsiders. A lot of people also know him
00:01:06
as the creator of Advanced Statistics,
00:01:08
including DVOA. And something I just
00:01:10
learned today, and we're going to talk
00:01:11
about this later on, is he's one of the
00:01:13
members of the Associated Press that
00:01:15
votes for the AllP Pro team and the
00:01:17
regular season awards. So Aaron, uh
00:01:19
welcome back to uh the Wharton Podcast
00:01:21
Network and Wharton Moneyball. Hey,
00:01:23
thank you for having me back. Always
00:01:24
good to be here, man.
00:01:26
>> Well, this is obviously a a great NFL
00:01:29
season. It's The reason I'm going to say
00:01:31
that and I'd love your a kind of
00:01:32
analytical look at it is, you know, I
00:01:37
can't tell you who's going to go I can't
00:01:39
even tell you in my view who's going to
00:01:40
go to the Super Bowl. I think this is
00:01:43
just a year with more uncertainty. You
00:01:45
know, it's like I think Shane read the
00:01:47
stat last week. This is like the first
00:01:49
time in 20 years there's been no Patrick
00:01:52
Mahomes or Tom Brady in the Super Bowl.
00:01:55
And so I'll just ask you, let's just
00:01:57
start at the highest level. How do you
00:02:00
see the 2025 NFL season right now? Who
00:02:03
are the true contenders? And if you had
00:02:05
to list, you know, I always like to give
00:02:07
the I'll call it the paro curve. like
00:02:10
how many teams would I have would you
00:02:13
have to give me to make it like an even
00:02:15
bet between making the playoffs and
00:02:17
between going to the Super Bowl and not?
00:02:19
How do you view the AFC and the NFC
00:02:20
right now?
00:02:21
>> Well, the first thing I'll say is it's a
00:02:23
very topsyturvy year. A lot of weird
00:02:26
stuff has happened. There are a lot of
00:02:28
teams where their record doesn't really
00:02:30
match what the underlying statistics
00:02:32
suggest about how good they are, right?
00:02:35
like Carolina, New England, Chicago
00:02:38
don't seem quite as good as their
00:02:39
records. Detroit, Kansas City,
00:02:42
Indianapolis have played better than
00:02:44
their records. Uh, some of these are
00:02:46
teams you would have expected in the
00:02:48
preseason, some of them are not. Uh, I
00:02:51
used my current ratings, current
00:02:52
ratings, and ran a simulation of the
00:02:55
whole season. And in that simulation,
00:02:58
the amount of times that Baltimore,
00:03:00
Detroit, Cincinnati, and Kansas City all
00:03:02
missed the playoffs was 1.1%.
00:03:06
>> Wow.
00:03:07
>> And that's not using preseason
00:03:08
projections. That's using how good the
00:03:10
teams have actually been this year.
00:03:12
>> So, we always like to ask that question.
00:03:14
So, now the question I I I'll ask you,
00:03:16
which is probably not a surprising
00:03:17
question, is it could mean one of a
00:03:20
couple things. It could mean the thing
00:03:22
that we all have in our simulations,
00:03:24
which is not enough uncertainty. And if
00:03:26
the answer is there isn't enough, where
00:03:28
is it coming from? Another possibility
00:03:30
is um 1.1% of events happen 1.1% of the
00:03:34
time. So, you know, let's not let's not
00:03:36
go overboard that you said 1 in 20
00:03:38
million. So, you didn't say that. The
00:03:40
third, of course, is your point
00:03:42
estimates of the team's strengths could
00:03:44
also be off. It's not an uncertainty
00:03:46
issue. It's kind of
00:03:47
>> It's an imperfect science. Yeah.
00:03:48
>> Yeah. It's an imperfect science. So
00:03:50
which of those three explanations or all
00:03:52
a combination of all three do you think
00:03:54
it is?
00:03:55
>> I think it's a combination of all three.
00:03:56
But the big thing is just that events
00:04:00
get ordered in different ways. And so
00:04:03
teams play one game where they're like
00:04:06
outstanding the whole game and then
00:04:07
another game where they lose by three
00:04:09
points. And this year is filled with
00:04:11
teams where they've just been either
00:04:15
really really good in one-score games or
00:04:18
really really bad in one-score games.
00:04:19
The other thing that's going on is the
00:04:21
is we'll talk about this in a little
00:04:23
bit. I think schedules being very
00:04:26
extreme
00:04:27
because of the way certain divisions are
00:04:29
good and bad this year. There are
00:04:31
certain teams that have very extremely
00:04:34
easy or extremely hard schedules. Um, as
00:04:37
far as uh the NFC is the better
00:04:39
conference, that's the first thing. Uh,
00:04:42
I my numbers differ from market. Is the
00:04:45
NFC the better conference on let's say
00:04:49
whether it's average or median or do you
00:04:51
think the top end because those are
00:04:53
different statements than saying the top
00:04:55
end of the NFC is better than the top
00:04:57
end of the AFC or both.
00:04:59
I would say both are true, but in
00:05:03
particular the top end of the NFC.
00:05:06
Um, the market I have Seattle and the
00:05:10
Rams much farther ahead of the rest of
00:05:12
the league than a lot of other people
00:05:15
who do advanced analytics.
00:05:17
Uh, the reason for that is almost
00:05:19
everybody else who does advanced
00:05:21
analytics uses some form of EPA, right?
00:05:25
Expected points added with some sort of
00:05:27
adjustment.
00:05:29
And I don't I use my own stat which is
00:05:32
DVOA.
00:05:33
And because of some of the ways that
00:05:35
DVOA works and EPA works, Seattle and
00:05:39
the Rams come out a lot higher in DVOA
00:05:42
than they do in EPA. One, some of it has
00:05:45
to do the increasy intricacies of, you
00:05:47
know, certain amount of yards and a
00:05:49
certain down and distance. And some of
00:05:51
it is easy to explain stuff like I
00:05:54
downweight turnovers because they're
00:05:56
less predictive. And Seattle has a lot
00:05:59
of turnovers. So if you do EPA,
00:06:02
Seattle's offense is not going to come
00:06:03
out as good as they do in DVOA because
00:06:07
I'm downweing the turnovers because less
00:06:10
predictive. So because of that, I have
00:06:13
Seattle and the Rams at almost a 5050
00:06:16
shot to win the Super.
00:06:19
>> I just want to be clear them com just to
00:06:22
be clear. Are you saying if you take the
00:06:24
Rams and Seattle, you have the Rams in
00:06:27
Seattle, I have the rest of the league
00:06:29
and that's a fair bet. Is that what
00:06:30
you're saying?
00:06:31
>> My numbers would say yes. It's a surp
00:06:34
It's surprising and I don't know whether
00:06:35
maybe I have them a little too high and
00:06:37
no one else would have this, but yeah. I
00:06:39
mean, everybody has right now has
00:06:41
Seattle and the Rams as their top two.
00:06:43
>> So, just give me an or this is what I
00:06:44
was going to ask you. whether one's EPA
00:06:46
based or DVO based. Can you give me a
00:06:48
sense? Or even if I went on to ESPN,
00:06:50
FBI, whatever it was, how much
00:06:53
probability would they give combined?
00:06:55
Are you at 50 and they're at 20 or
00:06:57
you're at 50 and they're at like 35? I I
00:06:59
always like to get a sense of the order
00:07:01
of magnitude of these discrepies
00:07:03
differences. I can look up ESPN while
00:07:06
we're here and tell you that for Seattle
00:07:09
and the Rams, they have them winning the
00:07:13
Super Bowl a combined 27% of the time.
00:07:16
>> Wow. So, you're almost twice as high.
00:07:18
So, that's we would consider
00:07:20
>> past everybody when it comes to those
00:07:22
two teams. Yeah.
00:07:25
>> Yeah. So, I just just for myself maybe
00:07:27
I'm thinking about this wrong. I'm
00:07:29
pretty sure most of our listeners and
00:07:31
this is the great thing about Wharton
00:07:32
Moneyball is saying, "Wow, would Aaron
00:07:35
really give me even money despite like
00:07:38
someone we both agree someone from the
00:07:40
AFC is making the Super Bowl. I don't
00:07:42
know who it is, but someone is." And we
00:07:44
could argue they have almost a 50%
00:07:47
chance to win that game. So, in your
00:07:50
view, well, let me ask a question. So,
00:07:52
there's this is the wonderful thing
00:07:53
about winning the Super Bowl.
00:07:56
We agree two things have to happen for
00:07:58
you to win the Super Bowl. If you're
00:08:00
Rams or Seattle, first you have to win
00:08:01
the NFC. Then you have to go to the
00:08:04
Super Bowl and win it. Which of those
00:08:06
two numbers is really, really high? It
00:08:10
sounds like it might be both. Like
00:08:12
here's something that could yield 50%.
00:08:14
>> The combined probability is 7070. That
00:08:17
would get you to 50%. Like they're the
00:08:19
one of the two of them has a 70% chance
00:08:22
of going in total. And if they go,
00:08:25
they're a twothirds favorite. Roughly
00:08:26
that get you to 50.
00:08:28
>> So my ratings are based on the idea of,
00:08:30
you know, how efficient is a team
00:08:31
compared to average.
00:08:33
>> Seattle and the Rams are two of the 10
00:08:36
best teams since 1978.
00:08:41
>> Measure. I know I have them way ahead of
00:08:44
everybody else. I know exactly how
00:08:46
you're measuring that. It's based on a
00:08:48
play-by-play breakdown and the success
00:08:51
on each play is compared to a baseline
00:08:54
that's adjusted for situation and
00:08:55
opponent.
00:08:57
And one thing is that I include special
00:08:59
teams and a lot of people who do
00:09:01
advanced stats don't include special
00:09:03
teams because special teams is less
00:09:04
predictive than offense and defense. But
00:09:07
it's not not predictive and I think
00:09:10
Seattle and Rams game that the fact that
00:09:12
Seattle has really good special teams.
00:09:14
So just to be clear, I compute this
00:09:17
>> I compute this metric for every play and
00:09:19
then I sum it over all the plays.
00:09:21
>> Yeah. Yes.
00:09:23
>> Okay. So Okay, I see. I just wanted to
00:09:26
understand the Okay. Wow.
00:09:27
>> So just to give you an idea of where the
00:09:29
numbers are, Seattle and the Rams are at
00:09:31
like 42 43%.
00:09:34
The next highest team that will make the
00:09:37
playoffs
00:09:39
is like Jacksonville right now, which is
00:09:42
at like 20%.
00:09:44
Like that's the gap I have between the
00:09:46
Rams and Seattle and everybody else. I
00:09:49
don't know if I would really do the Rams
00:09:51
and Seattle versus the Field to win the
00:09:52
whole thing.
00:09:54
>> If you gave me the 49ers though, I might
00:09:56
do it. If you gave me the NFC West
00:09:59
versus everyone else.
00:10:01
>> You almost made the whole MS NFC West.
00:10:03
>> I might I might do that.
00:10:05
>> Huh. That's very very very interesting.
00:10:08
Um, what aspect? Well, first of all,
00:10:11
could you just tell us since you must
00:10:12
since you're making a statement that
00:10:13
they're in the top 10 since 1978. Can
00:10:16
you give maybe I I don't know if you
00:10:18
have this handy, can you tell me since
00:10:20
I'm a football historian just like you
00:10:22
are, can you tell me who else is in that
00:10:24
list or do you have that list handy?
00:10:25
>> The top three through this is through 15
00:10:29
games. The top three are the 2007
00:10:32
Patriots.
00:10:33
>> They were pretty damn good. The 1991
00:10:36
Redskins.
00:10:37
>> They were good.
00:10:38
>> The 1985 Bear.
00:10:41
>> I knew the 85 Bears were going to be in
00:10:42
there somewhere. Wow. Yeah. So,
00:10:45
>> you also get some teams that didn't make
00:10:47
it all the way like the 2010 Patriots
00:10:51
and the 1983 Redskins who lost to the
00:10:54
Raiders in the Super Bowl and the
00:10:56
9549ers who got upset by the Panthers in
00:11:00
the divisional round. And then for some
00:11:03
reason the last three years I have had a
00:11:07
ton of teams come out as historically
00:11:09
good in a way that hadn't happened for a
00:11:12
dozen years before them. The 2023
00:11:15
Ravens, the 2023 49ers, the 2024 Ravens,
00:11:21
like come out as really historically
00:11:23
good. And I don't know why because I
00:11:25
haven't changed what I'm doing. And yet
00:11:29
there weren't any teams that were
00:11:30
anywhere near that high from 2011 to
00:11:32
2022 and then all of a sudden in the
00:11:35
last couple of years there have been
00:11:36
these teams that are ridiculously high.
00:11:38
H wow. Um so because if you had asked
00:11:43
most people I think I they would say
00:11:45
that the Rams and the Seahawks are
00:11:46
better appear to be better than others
00:11:48
this year but not I'll call them. Well,
00:11:50
I don't think anybody would put them
00:11:51
among the historically great teams.
00:11:54
>> And it's it's it's a little weird
00:11:56
because of the fact that I've had this
00:11:58
going for a couple of years and those
00:12:00
teams have not won the Super Bowl. But
00:12:03
it's not like my ratings always come out
00:12:06
with modern teams that high because from
00:12:08
2011 to 2022,
00:12:10
there were no teams that high.
00:12:12
>> It's just the last three years.
00:12:15
>> So, I'm going to ask you a question that
00:12:16
every listener here on Wharton
00:12:17
Moneyball. And again, I'm I'm joined
00:12:18
here by Aaron Shats. Aaron is the chief
00:12:20
analytics officer at FTN Fantasy.
00:12:23
Everybody also knows him from his
00:12:24
original work at Football Outsiders, his
00:12:27
work at DBA,
00:12:29
his work with DVOA. So, let me just ask
00:12:31
you, I'm always going to ask these kind
00:12:32
of questions because I know the answer
00:12:33
is no to each of them, but what the
00:12:35
hell? I'm going to answer them anyway.
00:12:37
Rams and Seahawks.
00:12:40
Um, if I'm correct, the quarterback for
00:12:42
the Seahawks is Sam Darnold, right?
00:12:44
>> Yes.
00:12:46
So you have no concerns?
00:12:49
>> No, I have concerns.
00:12:51
>> Okay. So, so let me ask you a different
00:12:53
question. Is that brought mathematically
00:12:56
into your model anywhere that it's Sam
00:12:59
Darnold's?
00:13:00
>> No. And it's something I want to look at
00:13:02
doing in the future, which is should I
00:13:04
have an additional career history
00:13:09
uh variable for quarterback that goes
00:13:12
past this year all the way back multiple
00:13:15
years of whether you trust a certain
00:13:17
quarterback compared to other quarter.
00:13:19
It's clear that Vegas does.
00:13:23
>> Absolutely.
00:13:23
>> Because there's no other reason why they
00:13:25
have the Bills as the favorite to come
00:13:27
out of the AFC. The Bills have not been
00:13:30
by any metric the best team in the AFC
00:13:33
this year. The only reason to have them
00:13:35
as the favorite to come out of the is
00:13:37
Josh Allen. Is the idea that you have
00:13:39
more historical trust in Josh Allen than
00:13:43
you do in Drake May or Bo Nicks or
00:13:47
Trevor Lauren?
00:13:49
>> So, which team in the AFC has been the
00:13:51
best team based on your Jacksonville? Is
00:13:53
it Jacksonville? Uh, if you look over
00:13:55
the course of the entire season, it's
00:13:57
Indianapolis.
00:13:59
If you weight it towards more recent
00:14:01
games, it's Jackson.
00:14:04
Where do you have I have debate with my
00:14:05
kids about this all the time who are big
00:14:07
analytics guys as well.
00:14:10
Isn't I mean, they they believe
00:14:12
Houston's a fraud. Where do you guys
00:14:15
have Where do you have Houston who I
00:14:16
think's won at least eight straight if
00:14:18
I've got this right?
00:14:18
>> We have them uh fifth.
00:14:22
>> We have fifth in the NFL. a fifth in the
00:14:24
NFL. They are number two on defense, but
00:14:27
they're only 21st on offense.
00:14:30
>> And there is that thing where offense is
00:14:31
a little bit more predictive than
00:14:33
defense. And that is that is true,
00:14:35
right? So, you're like, okay, maybe
00:14:38
they're not quite that good going
00:14:39
forward, but that defense is really,
00:14:41
really good and the offense has played
00:14:43
better in recent week.
00:14:46
>> Um, and okay, very interesting. So, let
00:14:48
me ask you another question. So, you've
00:14:50
met you made a comment in the notes here
00:14:52
about the Patriots and their schedule.
00:14:54
Um, could you talk more not only about
00:14:57
their schedule, but maybe this is why
00:14:59
one of the reasons why we have advanced
00:15:01
analytics because you can't just look at
00:15:04
one loss record and so could you talk a
00:15:06
little bit more about which teams you
00:15:08
see where and you mentioned it a little
00:15:10
bit when you're in your opening remarks
00:15:12
about who's one lost record or how about
00:15:14
the following. Let's say I did a by a
00:15:16
graph of w number of wins on the x- axis
00:15:20
and whether it's Aaron Shatz's measure
00:15:23
of team strength on the y ais. Which
00:15:25
teams would have the largest deviation
00:15:28
from that 45 degree line?
00:15:30
>> And a lot of these teams, anybody doing
00:15:32
advanced analytics has the dev
00:15:34
deviation? the Panthers,
00:15:37
the Bears, the Patriots, and on the
00:15:41
other side, the Lions, the Chiefs, and
00:15:44
the Colts. Although obviously the Colts
00:15:47
and Chiefs have a different quarterback
00:15:49
now than they did for most of the
00:15:51
season, and that affects how you would
00:15:52
predict things going forward. But as far
00:15:55
as how they've actually played during
00:15:56
the year, the Colts and the Chiefs have
00:15:58
been better than their record. And the
00:16:00
Lions have The Lions are going to be one
00:16:02
of the top DVOA teams ever to miss the
00:16:04
playoff.
00:16:05
>> Wow. So, I was going to ask you that
00:16:07
also. Do you ever as a I'll call it a
00:16:10
face validity check. Let's imagine that
00:16:13
every year DVOA based strength um has
00:16:18
lots and lots of outliers or deviations
00:16:21
where the DVA DVOA based strength does
00:16:24
not match let's call it the win loss. Do
00:16:26
you ever start to say maybe that's a
00:16:29
deficiency in DBOA because there's of
00:16:31
course there's randomness within a given
00:16:32
year but like across a long period of
00:16:35
time eventually you know if I look in
00:16:37
totality it should be properly
00:16:39
calibrated.
00:16:40
>> Yeah. I mean there's a lot of variance
00:16:43
in sports and there's a lot of variance
00:16:45
in the NFL. It's never going to be
00:16:46
properly calibrated but no I'm always
00:16:48
looking to improve it. I'm always
00:16:50
looking to improve the predictive
00:16:51
ability. I'm always looking to improve,
00:16:53
but a lot of these uh it's interesting
00:16:55
for a lot of the teams that I just
00:16:57
described,
00:16:58
the year-to-year DVOA is more consistent
00:17:01
than their year-to-year win loss
00:17:03
records.
00:17:03
>> Well, that's and that's what you would
00:17:04
hope.
00:17:05
>> Like Detroit, Indianapolis is an
00:17:06
exception, but Detroit was good last
00:17:08
year. Kansas City was good last year.
00:17:10
Carolina was not good last year. So, the
00:17:14
idea that Carolina is not as good as its
00:17:16
record makes some sense if you think
00:17:18
about who they were last year. And the
00:17:20
idea that the Chiefs are better than
00:17:21
their record makes sense if you think
00:17:23
about who they were last year,
00:17:25
>> right? Is it Am I correct in saying this
00:17:27
is just from my I haven't looked at it
00:17:29
analytically, but I would just give an
00:17:30
opinion that Well, I think I know, maybe
00:17:33
I don't know. I was going to say is
00:17:35
Well, it must be from your answer. The
00:17:37
strongest conference division, sorry, in
00:17:40
football has to be the NFC West, right?
00:17:41
Given you just said the Rams and Seattle
00:17:43
are historically strong. You said you
00:17:45
would take the 49ers. I think each of
00:17:46
them has at least 11 wins at this point.
00:17:49
Um, and so that's the strongest and the
00:17:52
NFC South is the weakest.
00:17:54
>> Yeah. Oh, yeah. The AFC South was
00:17:57
shockingly strong this year. The AFC
00:17:58
South,
00:17:59
>> that was going to be my second guess.
00:18:02
>> Really surprised. Indie Jacksonville
00:18:04
really surprised. Um, Houston, I think
00:18:06
people thought was going to be, you
00:18:08
know, just what they are, which is a
00:18:10
great defense with no offensive line.
00:18:12
Um, but uh right now in my ratings for
00:18:15
the whole season, so not weighted to
00:18:17
recent game. Uh, the Colts, the Texans,
00:18:20
and the Jaguars go four, five, six. But
00:18:24
with the NFC West, those two teams are
00:18:26
so far ahead and the 49ers are also good
00:18:29
that Yeah. the NFC West is the best
00:18:30
division. And the whole NFC South is in
00:18:32
the bottom half.
00:18:35
>> Wow. Yeah, because I you may remember
00:18:37
I'm a Buccaneers fan through my cousin,
00:18:39
etc. Uh what's happened the last three
00:18:42
weeks to the Buccaneers has been
00:18:43
embarrassing. Like I mean
00:18:45
>> they're the best team in the division.
00:18:46
Why can't they win games? It's very
00:18:48
frustrating.
00:18:49
>> Yeah. I mean they are the best team.
00:18:51
That's clear. And now they have all
00:18:52
their players back. So there's no excuse
00:18:54
for that. And they've just they lost at
00:18:57
home to the let me think if I got this
00:18:59
right. I was at the game with the Saints
00:19:01
so I know they lost that one and then
00:19:02
they lost to the Falcons at home and
00:19:04
then they just lost to the Panthers on
00:19:06
the road. So they just lost to, you
00:19:09
know, three teams in their division and
00:19:12
they all stink.
00:19:13
>> Yeah, it's I can't imagine. It's very
00:19:16
frustrating. Yes.
00:19:17
>> And of course they still control their
00:19:19
own destiny which is also the crazy
00:19:21
part. I mean they may end up going eight
00:19:23
and nine and winning the division
00:19:24
because the accord to you that's likely
00:19:27
to happen because uh the Panthers play
00:19:30
Seattle this week and that's a big game
00:19:32
for Seattle. The Bucks play the
00:19:34
Dolphins. Well, the the Bucks probably
00:19:37
>> starting a rookie quarterback.
00:19:40
They're gonna be
00:19:41
>> they should probably beat him. The Bucks
00:19:43
would have to win the division at nine
00:19:45
and eight because somebody's going to
00:19:46
have to win the last game of the season
00:19:48
between those two teams. But that may be
00:19:50
a historically weak division winner. And
00:19:53
whoever the five seed is um might be
00:19:56
happy they're the five seed.
00:19:57
>> I mean, there are other divisions, the
00:19:59
2020 NFC East, the 2004 NFC West. There
00:20:04
have been other divisions that are
00:20:06
weaker than this, but this is pretty
00:20:07
weak.
00:20:08
>> Now, the next thing in the notes that
00:20:10
you helped us uh create here, which I I
00:20:12
really appreciate that, is um I assume
00:20:14
when I asked you the question about DVOA
00:20:16
and your it being based on playbyplay
00:20:19
um and then you kind of add it up. Um
00:20:21
you put in the notes here the s the
00:20:23
search for historical playby-play data
00:20:25
and the upcoming edition edition of the
00:20:28
1977 season. So, is it fair for me to
00:20:31
say that what you and I completely agree
00:20:33
with this by the way that you've built
00:20:35
most of your analytical career for the
00:20:37
NFL on having this playby-play data as
00:20:41
opposed to just aggregated statistics
00:20:43
and that's why
00:20:45
>> Yeah, please go ahead.
00:20:45
>> When I got started, there were people
00:20:47
who were like, "Yeah, we did analysis
00:20:48
based on playbyplay and there was
00:20:50
nothing based on playbyplay in it." And
00:20:52
I was like, "Let's do something based on
00:20:54
playbyplay." And then I started
00:20:56
collecting old years and a lot of it's
00:20:58
been with the help of Jeremy Snyder who
00:21:00
people know on social media as quirky
00:21:02
research.
00:21:03
>> Okay.
00:21:04
>> And he has helped me and we have tracked
00:21:06
down and we keep going back and back and
00:21:10
back. So we've got back to 1978 now. We
00:21:12
have transcribed things off of video. We
00:21:15
have found old game books. Um and we
00:21:18
have enough to do 1977 this off season.
00:21:22
And it is no longer just mine. I have
00:21:26
shared it with Pro Football Reference.
00:21:28
Pro Football Reference now has all the
00:21:30
playbyplay going back to 1978 on their
00:21:33
site, searchable with all the
00:21:35
playby-play logs for all the games. I'm
00:21:37
very proud of doing that. It's been I
00:21:40
think it's a really huge addition to the
00:21:44
ability to study historic football
00:21:46
historically. And it's great. It's so
00:21:48
much fun for me when I'm like, "Oh,
00:21:50
yeah. The best teams to miss the
00:21:52
playoffs, the 1979 Redskins." Like, I
00:21:56
can go back that far and say they're one
00:21:58
of the best teams that didn't make the
00:22:00
playoffs. You know, let me ask you a
00:22:03
question. This might seem naive, but I'm
00:22:05
sure a lot of our listeners here on
00:22:06
Morton Moneyball are asking might be
00:22:08
thinking the same. What do you exactly
00:22:11
mean when you say playbyplay data?
00:22:13
Because one could dream. Let's let's
00:22:15
forget what you have. I'm going to
00:22:16
answer what my dream might be and then
00:22:19
you're going to tell me the data that
00:22:20
one actually would have conceptually.
00:22:23
You could have video motion data of
00:22:27
every player and their XY coordinates on
00:22:31
every single play and all of and I know
00:22:33
that's not the data you have but so what
00:22:36
do you mean by playbyplay data and how
00:22:39
do you compute kind of like an expected
00:22:42
performance
00:22:43
given that data
00:22:45
>> for a given play?
00:22:46
>> All my years are normalized so that
00:22:49
average is zero. So the same goes all
00:22:52
the way back to 78. Every year is
00:22:54
normalized so that I'm comparing teams
00:22:56
to just that year.
00:22:59
And what I have is basically everything
00:23:01
that was in the playbyplay up to 2004.
00:23:06
So like is it a pass or a run? Who was
00:23:08
the player?
00:23:12
Who was the receiver? What was the down
00:23:15
and distance? What was uh the time is
00:23:17
not in there for a lot of old years
00:23:19
until the last two minutes of each half,
00:23:22
but we have the time to start each
00:23:23
drive. And then the further back you go,
00:23:26
the more spotty it gets as far as like
00:23:28
the direction of runs or tackles or
00:23:31
passes defense, but we have like who had
00:23:34
sacks and who has interceptions and all
00:23:36
that data. Uh but we have targets,
00:23:39
right? Like no one ever had targets for
00:23:41
the 80s before before we collected that.
00:23:44
So, not just receptions, but targets
00:23:46
like is in this data. Um, we don't have
00:23:50
um yards after the catch and air yards.
00:23:53
That doesn't start till 2005.
00:23:56
>> We don't officially have whether a
00:23:58
quarterback run is a scramble, but I've
00:24:00
gone back to a lot of video and answered
00:24:03
whether a lot of those are. Uh, we don't
00:24:06
have a quarterback knockdowns. That
00:24:08
didn't start till 2006, I don't think.
00:24:11
And we don't have all the kind of
00:24:13
charting data that you now get from FTN
00:24:17
or SIS or PFF. Not we don't have any of
00:24:20
that going back before 2005. But
00:24:23
everything that people had from 99 to
00:24:25
2004, we basically now have for 78 to
00:24:28
98.
00:24:29
>> H I I can imagine that's ext and then
00:24:32
can you give us a sense of how just you
00:24:35
know maybe give me an example of how you
00:24:37
score a play. So, how is a play score?
00:24:41
>> Well, like you know, I give it a certain
00:24:43
number of success points based on the
00:24:45
yards and the down and distance, right?
00:24:47
Success in my percentages is you have to
00:24:49
get 45% of needed yards on first down,
00:24:53
60% on second down, and 100% on third or
00:24:56
fourth down, but there's like partial
00:24:58
numbers. It's it's not it's not it
00:25:01
doesn't go zero and one. It goes like
00:25:03
zero and 2 and 0.5 and 7, etc. And then
00:25:07
you get extra bonus for the extra
00:25:09
yardage. So a play will come out with
00:25:11
something like 2.3 success points if
00:25:14
it's like a 10 yard pass on first and 10
00:25:16
or something. And then I compare that to
00:25:18
what's the average for first and 10
00:25:21
passes in that area of the field.
00:25:23
>> Yeah. Okay. So that's what I meant. I
00:25:24
was also asking what do you condition on
00:25:27
when you compare a play? So it would be
00:25:29
similar
00:25:30
>> distance area of the field uh and then
00:25:32
adjust for the score, right? You know,
00:25:34
obviously it's easier to gain yardage if
00:25:36
you're losing by 40 late in the game.
00:25:40
>> So, this is actually a pretty opponent
00:25:42
adjustment based on the quality of the
00:25:44
opponent. So if I just look at this as a
00:25:47
big contingency table, this is a very
00:25:50
highdimensional contingency table
00:25:52
because I mean there's not that many
00:25:54
downs so that's not high dimension but
00:25:56
distance maybe have them uh blocked or
00:25:59
you know bundled into ranges or
00:26:01
something and then there's maybe there's
00:26:03
score difference and then there's maybe
00:26:06
>> what I try to do is smooth the curves.
00:26:08
>> Okay. So there is a smoothing that's
00:26:10
done but I didn't have like I I don't
00:26:12
think it makes sense to say that eight
00:26:15
yard gains are more common than seven
00:26:17
yard gains. It's more likely that our
00:26:20
sample size isn't big enough. It what
00:26:22
what's logical is to say that you know
00:26:26
you gain one more than you gain two you
00:26:28
gain two more than you gain three you
00:26:29
gain three you know etc. like it's it or
00:26:32
I guess four is the average and then the
00:26:34
curve goes like this, right? But like um
00:26:37
you know for judging between first and
00:26:40
20 and first and 25, you sort of group
00:26:42
that together because there aren't
00:26:44
enough actual like
00:26:45
>> first and 21s that are out. You know,
00:26:48
there aren't that many first and sevens
00:26:50
that aren't in the red zone.
00:26:52
>> Makes a lot of sense. So let me get to
00:26:54
the last topic for today that I want to
00:26:55
talk to you about which is as I
00:26:57
mentioned in when I introduced you to
00:26:59
start with you're one of the I I imagine
00:27:02
a tremendous honor for you and one
00:27:04
welldeserved as a member of the
00:27:06
Associated Press that votes uh for both
00:27:08
the regular season award and the allp
00:27:10
pro team. Um one of the topics you
00:27:12
brought up in your notes was the growing
00:27:14
role of analytics in awards voting. So
00:27:16
that could mean many things. It could
00:27:18
mean um the number of people that you
00:27:20
and I out of those 50 would consider as
00:27:22
analytics oriented is going up. That's
00:27:24
one way to measure it. Another way to
00:27:26
measure it is um you have knowledge of
00:27:30
the analytics that people are using.
00:27:32
Like here are the stats that people use
00:27:34
or those are provided in some way. What
00:27:36
did you mean when you said the growing
00:27:38
role of analytics in awards voting?
00:27:40
>> More the first than the second, but
00:27:42
they're definitely both true. I was the
00:27:44
first analytics person to be on the
00:27:46
panel.
00:27:48
uh starting in 2021, but they've done a
00:27:50
lot of changing of the panel over the
00:27:52
last few years. So now you have Mina
00:27:54
Kines
00:27:56
>> and you have Sam Monson who used to be
00:27:58
with PFF and you have Doug Ferrar who's
00:28:01
more of a film guy but understands
00:28:02
analytics used to write for me at
00:28:04
Football Outsider.
00:28:06
The other thing though is if you think
00:28:08
about it, people who are not necessarily
00:28:10
analytics people have access to more
00:28:14
analytics now than they used to. Think
00:28:15
about a guy like Dan Orlovski,
00:28:18
right? Dan is a former quarterback. He
00:28:21
is a film guy, but he works for ESPN and
00:28:24
he's very conversant with all of the
00:28:26
ESPN advanced stats that ESPN stats and
00:28:29
info does. And I'm going to guess that
00:28:32
when Dan puts his ballot together, he's
00:28:34
going to use some of that stuff. Now,
00:28:36
he's more probably going to use his eyes
00:28:37
and his scouting, but he's going to use
00:28:40
some of that stuff that didn't exist a
00:28:42
few years ago, or at least there was
00:28:44
nobody on the panel that would use any
00:28:46
of that a few years ago.
00:28:47
>> Right.
00:28:47
>> So, I think even the people who are not
00:28:49
necessarily analytics people are going
00:28:51
to use more analytics than they would
00:28:53
have used 10 years ago.
00:28:55
>> Can you point I don't want to talk about
00:28:57
this year because I know that's not only
00:28:58
is it off limits, you shouldn't talk
00:28:59
about it because you're a voter, but we
00:29:01
can talk about the past. So can you talk
00:29:04
about instances at least in your own
00:29:06
mind that somebody was like wow if it
00:29:10
wasn't for analytics that person never
00:29:12
would have been voted all pro or never
00:29:14
would have been an or maybe that's off
00:29:15
limits too just like I was just not that
00:29:17
your vote I'm not asking for your vote
00:29:19
or anything like that
00:29:20
>> I'm thinking because I have an answer
00:29:21
when it comes to my vote but I'm trying
00:29:24
I don't know about for the whole allpro
00:29:29
team or the whole or specific awards
00:29:31
that I think of anyone like that, but I
00:29:34
can think of one where I was very
00:29:35
different. I'm very famous among, you
00:29:39
know, people who understand this stuff.
00:29:41
When Lamar Jackson won MVP in 2023, he
00:29:45
got 49 of the 50 votes and I was the
00:29:48
exception.
00:29:50
>> So, can you tell Well, now that you said
00:29:51
that, can you tell us about that and
00:29:53
what what did you vote? And
00:29:54
>> I wrote a whole thing about how no
00:29:57
matter what stat you looked at, he came
00:30:00
in below Josh Allen and Dak Prescott.
00:30:04
>> My advanced stats, ESPN's advanced
00:30:07
stats, PFF grades, like SIS, anything
00:30:10
you looked at, he came in below those
00:30:13
guys. And I was like, I just can't vote
00:30:15
for him for MVP when he's not number one
00:30:18
and or two or two in any staff. And so I
00:30:22
went with Josh Allen.
00:30:24
Wow. Well, let me just say uh for those
00:30:26
people that follow us on whether you
00:30:28
want to call it X or Twitter, W
00:30:29
Moneyball, um there's no doubt I'm going
00:30:32
to be if we still call it tweeting, I'm
00:30:34
going to be tweeting that out because
00:30:36
that in itself is fascinating. It's also
00:30:39
fascinating to me that since let's
00:30:41
assume whether people use it or not as a
00:30:43
separate issue, that had to have been
00:30:45
sort of known by the voters. And so that
00:30:48
that to me is very interesting. Let me
00:30:50
ask I'm just asking you for speculation.
00:30:53
If that were to happen today, 5 years
00:30:56
from today, you agree the vote you're I
00:30:59
mean we would both hope the vote
00:31:00
wouldn't be 49 to1.
00:31:03
>> Today it's still there's still
00:31:04
narrative, right? Cuz Allen beat Jackson
00:31:07
last year even though last year the
00:31:09
opposite was true. Jackson had better
00:31:10
numbers.
00:31:11
>> Interesting. But in five years, I don't
00:31:14
know if somebody whose numbers are as
00:31:17
low as what Cam Newton's were in 2015 is
00:31:21
going to be able to win the MVP. I don't
00:31:23
know. I mean, you you'll have you'll
00:31:25
certainly have places where the person
00:31:27
who finishes third or fourth will beat
00:31:29
the person who finishes first in a stat,
00:31:32
especially because different metrics
00:31:33
have different players number one. But
00:31:37
five or six years from now, you may not
00:31:38
have a situation where you have a guy
00:31:40
who's 10th or 11th in stats, but he gets
00:31:43
a narrative going,
00:31:45
>> right?
00:31:45
>> And his team wins a lot of games because
00:31:48
of their defense and he gets a narrative
00:31:50
going and then he wins the award. You
00:31:52
may not have that anymore in a few
00:31:53
years.
00:31:55
Well, let me just my last question now
00:31:56
is um can you tell me a little bit about
00:31:58
what you're what you guys are working on
00:32:00
that you can talk I'm not necessarily
00:32:01
the nuts and bolts but what are you guys
00:32:03
doing at FTN Fantasy that our listeners
00:32:05
might be interested in like or you even
00:32:07
personally like what's the big project
00:32:08
you're working on now
00:32:09
>> I mean big projectwise I don't have
00:32:12
anything in particular except you know
00:32:14
still always looking to improve things
00:32:16
in the offseason you know improve
00:32:18
predictive ability um you know the thing
00:32:21
I'm really proud of at FTN is because we
00:32:23
do all of charting in our FTN stats hub.
00:32:26
If you're a subscriber to the site,
00:32:28
ftnfantasy.com,
00:32:30
you get DVOA for all kinds of charting
00:32:33
uh categories. So, you can look at who's
00:32:35
the best against man coverage or zone
00:32:38
coverage or with two high safeties and
00:32:40
which teams are the best with certain
00:32:41
run concepts, outside zone, inside zone,
00:32:44
power, man, etc. I love that stuff and I
00:32:47
love doing stuff to figure out what's
00:32:50
real and what is just really variable.
00:32:54
And then working on the 1977
00:32:57
and then the other thing that FTN is
00:32:59
doing that I have nothing to do with but
00:33:02
we are doing and is cool is basketball
00:33:04
charting. and we're going to have a
00:33:06
bunch of basketball stuff coming out uh
00:33:09
for people who enjoy basketball analy
00:33:12
well Aaron I'd like to thank you for
00:33:14
joining me here today on the podcast
00:33:16
edition of Wharton Moneyball I've been
00:33:17
talking to Aaron Shatz chief analytics
00:33:19
officer FTN Fantasy uh apparently it's
00:33:22
ftnfantasy.com
00:33:23
right is the uh is the website you can
00:33:26
also read
00:33:26
>> called FTN Fantasy we do a lot more than
00:33:29
just fantasy it's just we had to get it
00:33:31
all under one ceiling and that's the
00:33:33
ceiling we went
00:33:35
Well, again, Aaron, thank you again for
00:33:37
joining me today on Wharton Moneyball.
00:33:39
>> Hey, thanks for having me, man.
00:33:41
>> Welcome back to Wharton Moneyball here
00:33:43
on the Wharton podcast network. This is
00:33:45
Eric Bradlo, professor of marketing,
00:33:46
statistics, and data science here at the
00:33:48
Wharton School. Uh, I just got done and
00:33:50
finished talking to Aaron Shatz of FTN
00:33:54
Fantasy, chief analytics officer. Had a
00:33:56
great discussion of lots of different
00:33:58
topics. Um, and uh, some combination of
00:34:01
myself, Audi Winer, Shane Jensen, and
00:34:03
Cade Massie are here every week on
00:34:05
Wharton Moneyball. Uh, since it's just
00:34:07
me, um, and there's the a quote second
00:34:09
half of the show, I did something
00:34:11
interesting this week, which I hope all
00:34:13
of our listeners uh, will want to will
00:34:15
be interested in. I decided that since
00:34:18
I'm Wharton's vice dean of AI and
00:34:20
analytics, uh, why not use Generative AI
00:34:23
to answer a bunch of questions that I
00:34:25
have uh, I'd like to hear the answer to.
00:34:28
So, I'm going to tell you what I did.
00:34:30
I'm going to tell you the the exact
00:34:31
prompt I typed in, and I'm going to tell
00:34:33
you how, in my case, chat GPT. You could
00:34:35
use your own large language model, but
00:34:37
Wharton gives us a free enterprise
00:34:39
access to chat GPT. Um, I'll tell you
00:34:42
exactly what I said, what I asked it,
00:34:44
what the prompt was, and what it found.
00:34:46
And by the way, just for all of you that
00:34:47
want to know, this was chat GPT 5.2,
00:34:50
which is the newest version. And I also
00:34:52
used it in thinking mode trying to get
00:34:54
the if you'd like the highest quality
00:34:56
answer for anybody that wants to
00:34:57
reproduce what I did. So the first
00:35:00
question I asked it was give me a 95%
00:35:04
confidence interval for the number of
00:35:07
wins for the Oklahoma City Thunder in
00:35:09
the regular season. So let me say a
00:35:12
couple things. I'm interested in that
00:35:14
because they're 26 and three. They
00:35:16
certainly have the potential to break
00:35:17
the all-time record of 73-9
00:35:20
by the I think it was 2017 Golden State
00:35:23
Warriors who lost the title as everyone
00:35:24
remembers to LeBron in the finals. Um
00:35:27
but I'm also interested into how much
00:35:29
uncertainty it gives it. Okay. And so
00:35:32
the first thing is what this is first
00:35:35
something that differs from older
00:35:37
versions of large language models. You
00:35:39
may remember like when they first came
00:35:40
out like we've only trained the date up
00:35:42
until January the 1st of two years ago.
00:35:44
Well, that's all gone now. Now, I think
00:35:46
everybody knows that the first thing it
00:35:48
said is as of December 23rd, their
00:35:50
record is 26 and three. And they
00:35:51
literally played last night. So, they
00:35:53
know that they played last night. That's
00:35:55
in the database. So, that's good. Um, it
00:35:58
said it leaves 53 games in the regular
00:36:00
season. Sounds good. Um, then it did
00:36:03
something more spe more specific and
00:36:06
impressive and it started thinking and
00:36:08
it would come up on the screen what it
00:36:10
was thinking about. It says, "Look,
00:36:12
we're going to fit a beta binomial
00:36:14
predictive model." So, let me just be
00:36:16
clear to everybody what that means. So,
00:36:19
you can treat the 29 games they're
00:36:21
played as a binomial, a coin flipping
00:36:24
model. 26 have come up head success.
00:36:27
Three have come up failed. That means
00:36:28
they've lost. That's the binomial part.
00:36:31
And then we're going to put a prior
00:36:33
distribution on that coin flip. And
00:36:37
usually most people put a beta
00:36:38
distribution. You can view that as an
00:36:41
equivalent number of wins and losses. So
00:36:43
for example, if I said it's a beta coin
00:36:46
with a 1010, that would say my prior is
00:36:49
10 wins and 10 losses. I would add that
00:36:51
to the 26 and three. I'd have 36 wins
00:36:54
and 13 losses and that would be my
00:36:57
predicted win probability for the rest
00:36:59
of the season for the 53 games. So first
00:37:02
I was impressed that it knew about
00:37:04
basian inference that well. It knew that
00:37:07
doing some sort of shrinkage, which a
00:37:09
beta binomial is going to do because no
00:37:11
one's going to predict that they're
00:37:13
going to go, you know, 950 or whatever
00:37:15
26 and 3 is for the rest of the season.
00:37:17
You're going to have to shrink it down
00:37:19
to some prior and a beta binomial gives
00:37:22
you a formal way to do it because it
00:37:24
gives you an effective sample size. We
00:37:26
know the observed sample size 29, but
00:37:28
the prior is going to give us an
00:37:30
effective sample size. It then said the
00:37:33
95% interval is from 64 wins to 78 wins.
00:37:38
Now, first thing that surprised me was
00:37:40
the 78 number. I mean, that number I
00:37:43
mean, they're not winning 78 games.
00:37:45
They're not going 52 and one the rest of
00:37:48
the season, but I will say it also
00:37:50
computed this simulation game by game,
00:37:52
so it knew the schedule. 64, I think
00:37:56
that's just way too low. I mean, you
00:37:57
say, well, what about injuries? Well,
00:37:59
the first like month of the season they
00:38:01
played without their second best score,
00:38:03
Jaden Jackson. So, they're not I mean,
00:38:05
maybe um I just thought it was an
00:38:07
interesting interval. I guess it's
00:38:09
centered around 71 or 70 or 71 which
00:38:13
seems quite plausible to me that they
00:38:15
might do but either way. Um and then
00:38:17
lastly it said for a reference a more
00:38:20
optimistic assuming their current win
00:38:22
percentage continues exactly it leads to
00:38:24
an interval of 69 to 77 wins which means
00:38:28
centered at 73 plus or minus 4. So you
00:38:31
could view it as a standard deviation of
00:38:33
two. You're going to take two standard
00:38:35
errors of either side. So you're 73 plus
00:38:38
or minus 2 * 2 which gives you an
00:38:40
interval from 69 to 77. Either way I was
00:38:44
impressed by its thinking. I was
00:38:46
impressed by its use of the beta
00:38:47
binomial. I could have queried it more
00:38:50
to use this prior. It would have done
00:38:52
that calculation. It was simulating game
00:38:55
by game. Not bad. Not bad. So the next
00:38:59
question, since we're living in the
00:39:01
world of college football, I asked it,
00:39:04
what is the probability that the four
00:39:06
semifinal college football teams are
00:39:09
Indiana,
00:39:11
OSU,
00:39:12
Georgia, and Oregon. So that's the 1,
00:39:15
two, three, and the five seed. The
00:39:16
reason I picked the five seed Oregon is
00:39:18
that they're favored over Texas Tech.
00:39:20
And I said, give me an exact probability
00:39:22
estimate and a 95% confidence interval.
00:39:25
So, first thing it said is um it
00:39:27
recognized the matchup, so it told me
00:39:28
the matchups. That's not that hard. Um
00:39:31
it then used the draft kings money line.
00:39:34
It could have used lots of different
00:39:35
money lines, but that's fine. It then
00:39:38
gave me the probability of each of the
00:39:40
teams that I won winning. It then said,
00:39:43
let's assume they're independent. It
00:39:45
multiplied them together, and it comes
00:39:47
up with 17.94%.
00:39:50
It then computed a 95% confidence
00:39:52
interval based on those proportions. Um,
00:39:55
it then gave me a range of 16.72 to
00:39:58
19.68%.
00:40:01
U, that seems a little bit narrow to me.
00:40:04
Um, and then it said here, if you want
00:40:06
me to compute spreadbased probability
00:40:07
instead of money lines, it could do that
00:40:09
type of computation. So again,
00:40:12
this one was interesting because it rep
00:40:15
recognized that I was looking for a
00:40:17
compound event. you know the probability
00:40:19
of four things happening because I asked
00:40:20
for exactly all four of them. It used
00:40:23
the it found the money line to compute
00:40:25
those probabilities. It multiplied them
00:40:28
together um and it gave me a confidence
00:40:31
interval based on these uh proportions
00:40:34
and and what the likely uncertainty is
00:40:36
in those proportions. So I thought that
00:40:38
was pretty impressive. And again it's
00:40:41
more to me that it's impressive compared
00:40:43
to historically. If I had asked it six
00:40:46
months ago, a year ago, I don't think it
00:40:48
would have given me as precise an
00:40:49
answer. Um, and last but not least, I I
00:40:53
asked it um let's see here. I had one
00:40:56
other thing that I asked it for. Ah,
00:40:57
okay. Um, I asked it the question I
00:41:01
always like to ask, and this time I did
00:41:03
it for the NFL. I asked it, how many
00:41:08
teams do I need and what would those
00:41:10
teams be that would give a 50%
00:41:14
probability
00:41:16
to win the Super Bowl? In other words,
00:41:19
if I thought there were four teams that
00:41:20
added up to 50%, I I'm asking it to tell
00:41:23
me the number four and who those teams
00:41:24
are. Now, what's fascinating about this
00:41:26
is I didn't I hadn't even thought about
00:41:28
this at the time. I just asked Aaron
00:41:30
Shatz that question if you remember
00:41:32
during the first half of the show and he
00:41:34
said in his mind it's almost two teams
00:41:37
the Rams and the Seahawks almost add up
00:41:39
to 50% under his DVOA model. Um that's
00:41:43
not what uh but he also said under using
00:41:46
betting lines they would be about 27%.
00:41:48
And that's what um chat GPT 5.2 thinking
00:41:52
found. It said first it's converting bet
00:41:56
365 odds to an implied probability and
00:41:58
it gave me the formula of the implied
00:42:00
probability which is the way we always
00:42:02
do it. P equals 100 over the plus number
00:42:06
plus 100. So you take if it's plus 300
00:42:09
100 over 300 plus 100 is the implied
00:42:11
probability. It normalizes across all 32
00:42:15
teams to take care of the vig the
00:42:16
betting vig that's in there. And then
00:42:19
what it says, it gave me the top six
00:42:22
teams that give 49.65%.
00:42:25
Now to Aaron's credit, not surprisingly,
00:42:28
the Rams and the Seahawks are at the
00:42:30
top. Although according to this, that
00:42:31
would only be about 22% probability. It
00:42:34
then gave the Chiefs, who aren't even
00:42:36
going to make the playoffs, the Bills,
00:42:39
the Patriots, and the Broncos. And those
00:42:43
six teams, it adds up to 50%. Now,
00:42:46
here's the question. Certainly, I think
00:42:47
Aaron would take the No, of course, the
00:42:49
Chiefs aren't making the playoffs. The
00:42:50
Bills, the Patriots, and Broncos are.
00:42:53
Um, and by the way, the seven team for
00:42:55
us Eagles fans are the Eagles here. Um,
00:42:58
but I thought this was fascinating.
00:43:00
Again, I just want to say for our
00:43:01
listeners out there, use generative AI.
00:43:04
I I want to say I get I this is not a
00:43:07
specially trained generative AI model.
00:43:10
It's the one that if you have chat GPT
00:43:12
5.2 2 and possibly in thinking mode. Um,
00:43:16
you have the same ability I do to type
00:43:18
in what is the number of teams X that
00:43:21
are needed to get to 50% win
00:43:24
probability, order them by the highest
00:43:26
probability teams and tell me who the
00:43:28
teams are. That's all I typed in and
00:43:31
this analysis came in and it said here
00:43:33
and again it said closest to 50% using
00:43:36
the strict order of the teams is the
00:43:38
number six. That's pretty impressive and
00:43:41
this is a statistic we've been talking
00:43:43
about to understand uncertainty in the
00:43:45
league really for the 11 and a half
00:43:47
years we've been here on Wharton
00:43:49
Moneyball. So those are the three things
00:43:51
I did on chat GPT 5.2. I'll try to do
00:43:54
some more before next week's show and
00:43:56
also post some on our Twitter account
00:43:59
Moneyball. Um, just the last thing I
00:44:01
wanted to talk about maybe in the um, in
00:44:04
the last couple minutes I'm going to
00:44:06
talk today is um, since everybody knows
00:44:08
I'm a big tennis fan and we're trying to
00:44:10
get very soon we're going to get Paul
00:44:11
Anakone on and he would be great to talk
00:44:13
to about this. Um, as many people saw in
00:44:16
the last week Carlos Alcarez, the number
00:44:18
one player in the world, the holder of
00:44:19
six majors um, fired his coach. It'll be
00:44:23
very interesting to see what impact that
00:44:25
has on his performance during the year
00:44:28
because one of the things we've always
00:44:29
asked about is the impact uh of coaching
00:44:32
and so you know he's keeping a lot of
00:44:34
his team but his main coach Juan Carlos
00:44:37
Ferrero uh former number one in the
00:44:39
world uh winner of I think the French um
00:44:42
is no longer his coach. It'll very
00:44:43
interesting to see if there's any drop
00:44:45
off or if there's any narrative. So well
00:44:47
remember he also has to beat other
00:44:49
players and he has to beat especially
00:44:51
Yanx S. So you know can we decompose
00:44:54
whether maybe S's just getting better or
00:44:57
is it that Alcarez got worse? I think
00:44:59
this is the kind of thing that we're
00:45:00
going to need advanced analytics for
00:45:02
because I'm not sure like he's never won
00:45:04
the Australian Open. So we said, well,
00:45:06
let's say he loses. We can't just say,
00:45:07
oh well, if he had his other coach. No.
00:45:10
So we're going to actually have to dive
00:45:12
into the advanced analytics and
00:45:14
understand, you know, is he actually
00:45:17
performing worse? And then of course
00:45:19
there's also MA is match to match and
00:45:21
week-toeek variation. So we're going to
00:45:23
have to disentangle that from it. It
00:45:25
will not be an easy analysis. We're
00:45:27
certainly going to have to watch it for
00:45:29
some period of time and maybe use last
00:45:31
year as a if you like a control group
00:45:34
and compare his performance at similar
00:45:36
tournaments on similar surfaces and see
00:45:39
if there's some sort of market decline.
00:45:41
It's an interesting counterfactual on
00:45:43
how you might look at the impact of
00:45:44
coaching. So this has been uh this
00:45:47
week's show of Wharton Moneyball here on
00:45:49
the Wharton podcast network. Again, I'm
00:45:51
Eric Bradlow. On behalf of myself, Kate
00:45:53
Massie, Shane Jensen, and Aie Winer like
00:45:55
to thank you for joining us. I'd like to
00:45:56
thank Marissa Rena, like to thank D
00:45:59
Patel, and as always, I'd like to thank
00:46:01
Dion Simpkins, our associate producer,
00:46:03
sound engineer, the person that makes
00:46:05
all of this happen. So, between now and
00:46:07
next week, enjoy your statistics, enjoy
00:46:09
your sports. We'll see you next week
00:46:10
here on the Wharton Podcast Network and
00:46:13
Wharton Moneyball.
