RETURN TO CHANNEL

Search Captions & Ask AI

Wharton Moneyball Podcast – 10-Year Anniversary Episode

May 23, 2024 / 01:05:14

This episode of Wharton Moneyball celebrates 10 years of the show, discussing the evolution of sports analytics and data science. Guests include Kade Massie, Eric Bradlow, Shane Jensen, and AI Wier, who reflect on their experiences and insights gained over the decade.

The conversation begins with the changes in sports analytics since the show's inception in March 2014. The guests highlight how public perception of data and analytics has shifted, making the field more appealing to statisticians and data scientists.

They discuss the transition from basic on-field analytics to a broader understanding of sports, including business decisions, performance enhancement, and the integration of technology. The guests note the increasing importance of proprietary data and the challenges it presents for academic research.

As they reminisce about their time on the show, the guests share personal lessons learned, such as the significance of base rates in predictions and the value of regular discussions in enhancing their understanding of sports analytics.

The episode concludes with hopes for the future of the show, including potential offsite broadcasts and a focus on historical contributions to sports analytics.

TL;DR

Wharton Moneyball celebrates 10 years, reflecting on sports analytics evolution and personal growth among hosts.

Episode

1:05:14

00:00:10

hi this is Kade Massie practice

00:00:12

Professor here at the Wharton School

00:00:14

sitting with three of my closest faculty

00:00:16

colleagues and longtime Wharton

00:00:19

Moneyball collaborators Wharton

00:00:21

Moneyball is a show that's been on Sirus

00:00:22

xm's Business Radio since its beginning

00:00:26

10 years ago and we decided coming up on

00:00:29

the 10 year Ann we're about a week away

00:00:31

we thought we'd pause take a moment and

00:00:34

think back on those 10 years especially

00:00:36

because the world's kind of changed in

00:00:38

those 10 years what do we think we've

00:00:40

learned from it we spent a lot of time

00:00:41

with each other over the last 10 years

00:00:43

kind of a shocking amount actually doing

00:00:45

the show so we thought we'd gather a

00:00:47

little bit and talk about what that 10

00:00:49

years has meant to us I'm sitting with

00:00:52

aie Wier professor of Statistics Eric

00:00:54

bradow professor of marketing and

00:00:56

statistics and Shane Jensen professor of

00:00:59

Statistics as well a bunch of stats guys

00:01:02

and we've been doing sports analytics

00:01:03

we're also data science now we've been

00:01:05

added that since we began the show oh

00:01:06

there's a change in the last 10 years

00:01:08

statisticians have become data

00:01:10

scientists or at least some of them have

00:01:12

we can talk a little bit about that

00:01:13

that's an interesting development in

00:01:14

fact why don't we start there why don't

00:01:17

we start with the question of how you

00:01:19

feel the World of Sports analytics has

00:01:22

changed in the 10 years our first show

00:01:24

was March 2014 so it's been 10 years has

00:01:28

we've seen a few changes we're gonna

00:01:29

pitch questions out we're all going to

00:01:31

take a little bit of a chance Adam and

00:01:33

we'll wander through the reflections but

00:01:35

let's start there what has changed in

00:01:36

the World of Sports analytics in the

00:01:38

last 10

00:01:40

years well I think I mean Audi kind of

00:01:42

alluded to it a little bit here I think

00:01:44

certainly the public discourse around

00:01:47

analytics and data and everything I mean

00:01:49

in the last 20 years of me being a

00:01:51

faculty member in the last 10 years of

00:01:52

the show I think just sort of the public

00:01:54

consciousness of of of kind of data has

00:01:57

really changed a lot you know I no

00:01:59

longer have to go and explain to people

00:02:01

like why why I'm a why I'm a

00:02:02

statistician why I do why I work with

00:02:05

data I think so I think that really has

00:02:06

changed way of saying that it's much

00:02:08

cooler now it's caught up to the

00:02:10

coolness of Shane probably but I think

00:02:13

also on the other side I think Sports

00:02:15

has also become cooler within the sort

00:02:17

of statistical academic community in the

00:02:18

time I've you know in the last 10 years

00:02:20

as well I was kind of looking back I

00:02:22

started as a faculty in ' 04 and ' 05

00:02:26

was the first issue of the Journal of

00:02:27

quantitative analysis of sports it was

00:02:29

the first I think before when I was

00:02:31

first coming up through graduate school

00:02:32

I was interested in sports but I was

00:02:34

cautioned a lot like do not you know do

00:02:36

Sports as an academic topic you know

00:02:38

maybe have it as a side hobby but don't

00:02:40

make it your main thing and now we've

00:02:42

got Flagship journals in in in in in

00:02:45

sports anls and I think in the 10 years

00:02:47

we've been doing this show I think you

00:02:49

know Sports has become gone from kind of

00:02:50

being this sort of Black Sheep to this

00:02:52

something where this is where you see a

00:02:54

lot of the coolest kind of spatial

00:02:56

temporal data situations so it's almost

00:02:58

kind of on The Cutting Edge of

00:02:59

methodology now it certainly is accepted

00:03:01

I remember when you arrived that's when

00:03:03

I just got tenured that time and that

00:03:04

was the first time I decided to work on

00:03:06

Sports I waited until I had job security

00:03:09

before I could work with sports I

00:03:10

wouldn't I wouldn't give that advice

00:03:11

anymore I'd say Jump Right In I think

00:03:13

your comment Shane though about the

00:03:15

field is always going to chase the

00:03:17

coolest data and that's really I think

00:03:20

the difference that's happened I think

00:03:22

the reality is sports has data that

00:03:24

Rivals really any other industry today

00:03:27

and I think that's why Academia is

00:03:28

moving in that direction I think that's

00:03:29

why student interest is there I think

00:03:31

also it's a great testing ground for

00:03:34

learning statistical methods yeah it

00:03:36

just is you know one particular kind of

00:03:38

data that's blown up almost exactly in

00:03:40

the window we've been doing this show is

00:03:42

space show temporal so the kind the dawn

00:03:45

of that in my mind you can pick

00:03:48

different Don but one of the most

00:03:49

important Don was when Kirk goldsbury

00:03:53

got the data at Harvard from the NBA and

00:03:55

he ran across Luke Bourne and I believe

00:03:57

that was 2012 when Luke first got got

00:04:00

there as a faculty member so that was

00:04:02

less than two years but that was really

00:04:03

the beginning of and the NBA was before

00:04:05

the NFL that was just preceding our show

00:04:08

so we've been there kind of from the

00:04:10

beginning and we saw this thing blow up

00:04:11

and now it's secondhand well I will say

00:04:14

that Shane and our first paper on

00:04:16

baseball was spatial temporal data

00:04:18

evaluating Fielding but it was crappy

00:04:20

data right right done by video and human

00:04:22

beings and the enormous change was

00:04:25

leaving this crappy imprecise sparse ER

00:04:30

filled in incomplete data to Now

00:04:33

tracking but it Al also changed because

00:04:35

in baseball the best data was always

00:04:37

public and now the best data is private

00:04:40

and this is this is one of the things

00:04:41

that has changed I think not for the

00:04:42

better we sitting here in Academia could

00:04:45

produced Cutting Edge research that was

00:04:48

a value to the teams back then and I

00:04:51

think at this point if you don't have

00:04:52

proprietary data you're not saying

00:04:53

anything the teams want yeah I think it

00:04:55

kind of it goes hand in hand because I

00:04:57

think as the DAT has gotten richer or

00:04:59

the data has gotten very expensive to

00:05:00

collect and is is incredibly rich but is

00:05:02

now kind of like often sort of owned by

00:05:04

a league or something like that it's

00:05:05

less kind of accessible I think maybe

00:05:07

Kate to answer your question for me um I

00:05:10

was hoping someone would say something I

00:05:11

didn't say which Shane did um I think

00:05:14

the part that's gotten most interesting

00:05:15

to me is the problems meaning if you

00:05:17

think about the original Money Ball

00:05:19

right it was mainly about onfield stuff

00:05:21

like how are we going to get you know

00:05:23

let's go back to Billy Bean and

00:05:24

Moneyball how are we going to get

00:05:25

players that you know maximize our

00:05:27

chances of winning but now if you think

00:05:29

about the role of analytics I'm even

00:05:30

thinking about just the papers the

00:05:31

people here have written it's not just

00:05:33

the onfield part it's about the business

00:05:36

of sports it's about you know trying to

00:05:39

make decisions on sleep and training

00:05:41

patterns Etc which is to me the narrow

00:05:44

set of problems which was let's attack

00:05:46

should we go for it on fourth and one

00:05:48

you know are walks as valuable as

00:05:49

singles I'm not saying those aren't

00:05:51

important problems but the problems have

00:05:54

grown so diverse and that's the part

00:05:57

that's exciting to me and I think that's

00:05:59

what's really changed is that the

00:06:00

business side like when I started

00:06:02

working for the Eagles the onfield side

00:06:05

wanted something to do with me but the

00:06:07

business side wanted nothing to do with

00:06:09

me and now both sides tend to have

00:06:11

analytics so that to me when I was

00:06:13

thinking about the question I think the

00:06:14

problems are broader and a lot more

00:06:16

interesting and the maybe also to build

00:06:18

on Shane's earlier point and and what AI

00:06:20

just said the Divide now between

00:06:22

Academia and practice of all the fields

00:06:25

that I see you know my role as Vice dean

00:06:26

of analytics here I think Sports has the

00:06:29

closest connection between Academia and

00:06:31

practice that I've seen if I think about

00:06:33

people analytics and Neuroscience or

00:06:36

even big tech companies there's still

00:06:38

this divide but I think now there's this

00:06:40

real tightness between Academia and

00:06:43

practice when it comes to sports and

00:06:45

statistics and I think that's been

00:06:47

fantastic I mean how many students are

00:06:48

you sending to sports teams how many

00:06:50

times are we working with people with

00:06:52

sports teams whe it's the NFL Big Data

00:06:54

bowl or some other problem I think that

00:06:56

divide has really shrunk and that makes

00:06:58

to me Sports Antics the most exciting

00:06:59

field to be in so just for to finish up

00:07:03

I would more or less say the same

00:07:05

things statistics has always been very

00:07:07

close to sports because we can develop

00:07:09

new methods and really Advance the field

00:07:11

of Statistics as while using Sports data

00:07:14

so it really is a full-on statistics

00:07:16

data science problem just using Sports

00:07:19

and this has engaged so many people at

00:07:20

all levels um but the I have to say just

00:07:23

I can give you a number it wouldn't be a

00:07:25

statistic show without a number when I

00:07:26

first taught Moneyball Academy which was

00:07:28

our high school sta statistics program

00:07:30

um u high school sports analytics

00:07:32

program over the summer the first summer

00:07:35

we barely had more applicants than we

00:07:36

had space and now we have which is how

00:07:39

much space uh about 75 but the first

00:07:41

time we ran we ran for 50 we had 55

00:07:43

applications for 50 spots okay and this

00:07:46

past year we have two rounds and we have

00:07:48

already 350 and we I don't think we're

00:07:50

going to do a second round right that's

00:07:52

just show how it incredibly um the

00:07:55

interest in sports analytics even at the

00:07:57

Young level is just exploded MH MH the

00:08:01

one element that you alluded to Eric

00:08:03

that I just I think is worth

00:08:04

underscoring is that you're so right

00:08:06

that in the beginning we were talking

00:08:07

about more or less in-game decision

00:08:09

making and Personnel really the two the

00:08:11

play evaluation player evaluation and

00:08:13

the I think the principle third that has

00:08:15

evolved really only in the last seven or

00:08:18

eight years is performance enhancement

00:08:20

performance changes and and the classic

00:08:22

examples of the Astros bringing in these

00:08:25

pictures and teaching them to throw a

00:08:27

different way and that's a use of

00:08:28

Technology analytics that just wasn't

00:08:30

happening at all and it's entirely

00:08:32

different from the other from the other

00:08:33

two or as we've had guests on the show

00:08:35

where everything in your practice now

00:08:37

these people have sensors all over their

00:08:39

body so when is somebody tiring when is

00:08:42

someone getting their Peak Performance

00:08:43

and of course as scientist what we care

00:08:45

about is how do you maximize performance

00:08:47

which means if you can run an experiment

00:08:49

or manipulate things like let's try

00:08:51

different you know eating patterns

00:08:53

sleeping patterns Etc and let's see how

00:08:55

it manifests itself on the field there's

00:08:57

no reason why that shouldn't be part

00:08:59

part of sports analytics today you know

00:09:01

sensors all over the body is really good

00:09:03

data MH and I think that kind of

00:09:05

immediate feedback you know they be

00:09:07

being able to kind of train in a way

00:09:08

where you know if you're a p hitter and

00:09:10

you're trying to H learn how to hit a

00:09:12

fast ball you can kind of just have like

00:09:14

a lot of kind of data support like and

00:09:16

and kind of immediately learn sort of

00:09:17

like meth you know kind of mechanically

00:09:19

what you're doing right or wrong or free

00:09:21

throw shooting or whatever there's so

00:09:22

many examples where we now kind of have

00:09:24

almost IM you know immediate feedback

00:09:25

coaching and these kind of really helps

00:09:27

Player Development mhm

00:09:30

MH okay that question was about how the

00:09:32

field has changed this next one's more

00:09:34

or less how y'all have changed so what

00:09:36

have you learned in the 10 years doing

00:09:38

the show because of the show what do you

00:09:40

think you have what's an example of

00:09:42

something that you've

00:09:44

learned I can go I'll go first so um

00:09:47

I've learned a lot by listening to all

00:09:49

of you and uh the thing that kept coming

00:09:52

into my mind since I knew you'd asked

00:09:53

the question was base rates which is for

00:09:57

me to move away from a base rate so for

00:10:00

example let's imagine I took two

00:10:01

arbitrary teams just playing each other

00:10:03

I didn't know anything about them well

00:10:05

50/50 right okay so now I have these

00:10:08

other factors I bring in there so the

00:10:11

first thing is to move away from the

00:10:13

base rate of Shane always talks about

00:10:14

the coin flipping model to move away

00:10:16

from that you better have a good reason

00:10:19

and also I was probably one of those

00:10:21

people that thought effect sizes like

00:10:22

how much something really affected these

00:10:24

probabilities was much bigger than I I

00:10:26

now think it's much smaller like I used

00:10:28

to think well it's a 9010 game you know

00:10:31

Clayton kers on the Dodgers are playing

00:10:33

I don't know the lowly Pirates 90% for

00:10:35

the Dodgers no there's never you'd never

00:10:38

predict you think you're different now

00:10:39

than if we were sitting here 10 years

00:10:40

ago I do I think I'm different because

00:10:42

not you you were you were a cheered

00:10:44

professor of marketing statistics 47

00:10:47

years old 10 years ago and you were like

00:10:49

why you keep bringing up the age what

00:10:51

you were 47 10 years ago

00:10:53

too we're all born in 1967 not I think I

00:10:56

was only 37

00:10:59

I'm I'm not I'm not being critical at

00:11:01

all I'm honestly querying that's

00:11:04

phenomenal that that's been a

00:11:05

consequence of these 10 years I do I

00:11:07

think again it's both base rates in

00:11:10

other words to move away from just or a

00:11:12

always says if I have an empirical

00:11:14

frequency let's say I've got say you

00:11:15

know I've watched 10,000 baseball games

00:11:18

over 40 years here's the empirical

00:11:20

frequency something happens someone

00:11:21

asked you to make a prediction that's a

00:11:23

damn good starting place and to move

00:11:26

very far away from that you better have

00:11:28

a reason and I also think I adjusted too

00:11:31

much like well it's a left-handed

00:11:33

pitcher against this team momentum yeah

00:11:36

well we'll momentum will get to I still

00:11:39

believe in momentum that part hasn't

00:11:41

changed but I think I believe in base

00:11:43

rates and I don't move off them as far

00:11:45

as I think I used to so let me let me

00:11:48

just say what I've learned because it it

00:11:50

kind of matches in some level what you

00:11:51

were saying so when we started this show

00:11:54

um I knew a lot about baseball I still

00:11:56

know a lot about baseball but I was

00:11:57

really terrified how we were going to

00:11:58

feel two 2 hourss talking about sports

00:12:01

that it can't be all about baseball

00:12:03

because what am I going to say and uh

00:12:05

I've certainly learned a lot about

00:12:07

sports I think everyone is proud of my

00:12:08

accomplishments in the football arena in

00:12:10

particular um but all sports especially

00:12:13

but the way you're able to do it is

00:12:15

basically bring those base rate facts

00:12:18

and questions to any sport so I might

00:12:20

not know anything about golf but I can

00:12:22

contribute by asking what's the mean

00:12:24

what's the standard deviation what's the

00:12:26

what's the a typical player doing these

00:12:28

situations and sometimes what I found is

00:12:30

my lack of knowledge about a sport works

00:12:32

very well with the experts because it

00:12:34

brings you back to Earth it it stops you

00:12:36

from from overt telling the story and

00:12:38

and forces you to ask a specific

00:12:40

question and so one of the things that's

00:12:41

amazing is that that it was I don't know

00:12:43

how long it was but after a while we

00:12:45

realize we can fill two hours no problem

00:12:47

right AUD I just want to point out

00:12:49

you're connecting to Something in

00:12:50

Psychology called the inside view and

00:12:52

the outside view the inside view people

00:12:54

have very case-based detailed enriched

00:12:57

models of the world the outside view is

00:12:59

pretty much about Bas rights and often

00:13:01

the inside view can get skewed by their

00:13:03

cases and the outside view is really

00:13:04

helpful correction in that way um Shane

00:13:08

yeah I've been thinking about this too

00:13:09

and actually it's a little related to

00:13:11

what Eric was talking about a little bit

00:13:12

too I kind of feel like I I when I with

00:13:14

the show first started I mean you know I

00:13:15

was trained as a

00:13:17

statistician um I feel like I'm decent

00:13:19

at probabilistic kind of thinking and

00:13:21

thinking in the presence of probability

00:13:22

but I have learned repeatedly over this

00:13:25

decade of my own kind of how easy it is

00:13:28

for like like biases and probabilistic

00:13:31

thinking creep in I kind of I think of

00:13:32

the kind of I don't know what

00:13:34

inevitability kind of bias that I have

00:13:36

you bring it up like every college

00:13:38

football season right where we kind of

00:13:40

like halfway through the season we're

00:13:41

locked into a national championship game

00:13:43

no not that when he asked at the

00:13:44

beginning of the Season what probability

00:13:46

do you think is assigned to these four

00:13:48

teams and I'll say like 15% in case like

00:13:51

it's like less than 1% like total

00:13:54

overestimation and I think I've I've

00:13:55

learned from myself that in situations

00:13:57

especially like you know I think now of

00:13:59

like you know the Kansas City Chiefs and

00:14:01

the current Dynasty in in situations

00:14:02

where there's kind of you know a

00:14:04

dominant team or something like that my

00:14:05

own mind I tend to like when you've got

00:14:07

like something like a probability of0 75

00:14:09

or 08 and you round it up to one or you

00:14:12

round it to you you take a probability

00:14:14

that's in like the 15% or 10% range you

00:14:16

round you just your mind just BL rounds

00:14:18

it down to zero and so kind of thinking

00:14:20

about those sort of like you know like

00:14:22

even intermediary probability events

00:14:24

like a 20% you know a real mismatch in

00:14:27

sports would be like an 80 to 20 thing

00:14:29

thinking you know not thinking about

00:14:30

that as in an inevitable Victory like

00:14:33

you know thinking about that 20% and

00:14:34

kind of processing it you know every

00:14:36

time you see I think that's a lesson

00:14:38

I've kind of learned and it's been

00:14:40

instructive too to sort of see like me

00:14:42

looking at the Kansas City Chiefs as an

00:14:44

inevitable Dynasty it's because I'm not

00:14:46

a fan following that team and seeing all

00:14:48

remembering all the gut wrenches losses

00:14:50

they've had over that time as well I

00:14:52

think it's sort of like when you're

00:14:53

really closely following when you're

00:14:55

involved in a sport maybe you're less

00:14:57

prone to that inevitability byus cuz

00:14:58

you're kind of of seeing you know seeing

00:15:00

those 20% turn up maybe that's

00:15:01

interesting right yeah I'll give you a

00:15:04

couple quick ones uh one I've I'm struck

00:15:07

by how many times we figure things out

00:15:09

collectively that we'll start kind of

00:15:11

Meandering towards some subject and

00:15:13

we'll talk about it for seven or eight

00:15:14

minutes and after about 10 minutes we

00:15:16

kind of have a better understanding

00:15:17

collectively and it's connected to my

00:15:19

other one which is everyone's wrong

00:15:20

sometimes and I've learned that another

00:15:22

environments one of the nice things

00:15:23

about working in on faculty is that you

00:15:27

see some really smart people be wrong

00:15:29

it's helpful to see that we're all wrong

00:15:31

sometimes you guys are some of the

00:15:32

sharpest guys I have conversations with

00:15:34

and you're all wrong sometimes well

00:15:36

let's just take what the facts are I'm

00:15:38

wrong more than all of you because you

00:15:41

remember we've actually no no no for

00:15:43

years remember we've picked outcomes of

00:15:45

games and kept scores I think I have a I

00:15:47

have momentum that I'm the lowest every

00:15:51

single year we've done matter of fact I

00:15:52

don't think we've done this in a few

00:15:53

years matter I think it's cuz you guys

00:15:55

want to keep me at the bottom I we need

00:15:58

to go back

00:16:00

last place every year I just tell people

00:16:02

I'm in fourth place and like that sounds

00:16:04

impressive I don't tell them there's

00:16:05

only four of us making these predictions

00:16:07

well the redeeming thing is we haven't

00:16:09

exactly been comprehensive in our record

00:16:10

keeping so for all we know you might be

00:16:12

number one and we just hav written down

00:16:15

way well the last thing that I feel like

00:16:16

I've learned is the power of doing

00:16:19

something a little bit of something on a

00:16:21

regular basis I think the the cumulative

00:16:23

effect of doing this show even though

00:16:25

it's only been two hours a week and now

00:16:26

just one hour a week but we've done it

00:16:28

50 weeks a year for 10 years the

00:16:30

cumulative effect in terms of what I

00:16:32

know my relationship with y'all the

00:16:34

people we know in industry is amazing we

00:16:36

we what's the line like we tend to

00:16:38

overestimate what we can do in one big

00:16:39

push and underestimate what we can do

00:16:41

with a sustained regular push I think

00:16:44

this Show's been a like a tangible

00:16:45

demonstration of that to

00:16:47

me um all right what about themes that

00:16:51

have emerged on our show we we have come

00:16:54

to realize we didn't sit out with any

00:16:55

themes we didn't know how we were going

00:16:57

to talk about these things at all but

00:16:59

after a few years we kind of come back

00:17:01

to some of the same bits you guys have

00:17:02

just talked about base rates for example

00:17:04

that's definitely been a theme what else

00:17:06

would you say or some patterns or some

00:17:08

themes well come on it has to be what AI

00:17:10

mentioned I mean you guys I ridicule is

00:17:12

a fine word but in a professional nice

00:17:14

way you guys ridicule me about momentum

00:17:17

I mean that's definitely been and you

00:17:19

know I'm a strong believer that um there

00:17:22

is heavy if you want to call it a heavy

00:17:25

degree of State dependence like given a

00:17:27

success there's a much larger

00:17:29

put a fancier term to make momentum

00:17:31

sound more sophisticated no I think

00:17:33

moment I still believe momentum exists

00:17:36

but I'm willing to say that there's at

00:17:38

least heavy

00:17:40

non-stationarity and whether you want to

00:17:42

call that momentum or not matter of fact

00:17:44

i' I've joked as this I think you guys

00:17:46

know but our listeners know like I'm

00:17:48

from the old school basian World which

00:17:50

means I believe in Cross unit crossers

00:17:53

heterogenity what I think I'm going to

00:17:55

spend the next 20 years of my career on

00:17:57

is within person heterogenity which is

00:18:00

if you think about something we talk

00:18:01

about like a hidden Markov model like

00:18:02

people go through hot and cold States

00:18:04

well that's about a given unit that's

00:18:05

not a cross-unit statement I think we in

00:18:09

the show have spent a lot of time

00:18:11

talking about whether you want to call

00:18:13

it momentum non-stationarity Etc and I

00:18:15

think it's a great thing because I think

00:18:17

a lot of times we focus on differences

00:18:19

across units and not as much sometimes

00:18:22

there's Peak periods and weak periods

00:18:23

within a unit let me jump in because

00:18:25

that that's my favorite one that's

00:18:26

emerged over time I can name a few

00:18:28

others but that one I think is the most

00:18:30

interesting because it really has been a

00:18:32

slow thing to emerge and now it seems to

00:18:34

me terrifically important we come back

00:18:35

to it repeatedly you always named the

00:18:37

connection to marketing there but within

00:18:40

person variation and one of the reasons

00:18:42

it means so much to me is that I think

00:18:44

it's something that we miss like people

00:18:45

in the world underestimate it we tend in

00:18:48

the world to focus on betweeners

00:18:50

differences and we're talking about mean

00:18:52

differences and we underestimate the

00:18:53

within-person variance well you know my

00:18:55

favorite topic on that is when we talk

00:18:57

about whether it's tennis player or not

00:18:59

is that as players age you draw more

00:19:01

from a bodal distribution so we're even

00:19:03

know the greatest tennis player of all

00:19:05

time we could debate it's probably Novak

00:19:06

jokovic I begrudgingly have to admit

00:19:09

that but at the end of the day as

00:19:10

someone ages you start to see well maybe

00:19:13

95% of the time we'll see a draw from

00:19:15

the good distribution and a 5% of the

00:19:18

time it'll be bad and then maybe next

00:19:19

year because he's 37 38 it'll be 90% 10%

00:19:23

and that's what happens this is

00:19:24

definitely an Eric theme this an Eric

00:19:26

theme he's he's learning how toing from

00:19:28

me for me he's but he's learning how to

00:19:30

talk about it in a more structured way a

00:19:32

real quickly but but this is something

00:19:35

that you've given us an example of it's

00:19:37

with pitcher this is an empirical

00:19:39

observation you have about baseball

00:19:40

pitchers that we tend to think oh this

00:19:42

is how you rank them who's the best

00:19:43

who's the next Who's the differences and

00:19:45

you're like yeah yeah fine but a given

00:19:47

picture looks very different on

00:19:48

different days right they do not only do

00:19:50

they look very different on any given

00:19:51

day but any really almost any pitcher

00:19:54

can have a fantastic game in any in any

00:19:56

one moment and this is the kind of

00:19:57

things we we've Ted aled about

00:19:59

variability across Sports this is the

00:20:00

kind of thing that happens with baseball

00:20:02

pitching constantly doesn't happen so

00:20:04

much with with quarterbacks you expect

00:20:07

good quarterbacks to have fantastic

00:20:08

games obviously it does happen every now

00:20:10

and then but that kind of comparison

00:20:12

across Sports amount of variability but

00:20:14

if I were to answer like what what theme

00:20:16

that we've I mean

00:20:17

is and this is I guess so important in

00:20:21

in basic statistics we think about

00:20:23

regression and that just drives just

00:20:25

about everything we do and I don't mean

00:20:27

it just in regression as the forecasting

00:20:29

tool because the word regression is

00:20:31

generally used today to mean your

00:20:34

prediction your predicted value what is

00:20:35

you how did you regress this but the

00:20:37

word actually comes from to regress

00:20:39

which is to go back down and in sports

00:20:41

they use the word constantly and you and

00:20:43

we it's a tool for forecasting but our

00:20:45

so much of our dialogue is is

00:20:47

concentrated on how much to regress and

00:20:50

H and how what data we use to to figure

00:20:52

out and and what are we regressing to in

00:20:54

fact our just our last show what are we

00:20:56

regressing to to individuals rate to the

00:20:58

base rate of the of the of the of the

00:21:01

the the community the group and some of

00:21:03

our finest conversations have to do with

00:21:05

trying to understand how much regression

00:21:06

and that of course intercepts with our

00:21:08

basian conversations like how do we how

00:21:10

do we integrate priors into that and so

00:21:12

it's a really a great amalgam of some of

00:21:15

the most important themes of Statistics

00:21:17

regression basian shrinkage is is the

00:21:20

more the more complex way to think about

00:21:22

it and we've become artists in

00:21:24

describing these hard I think so

00:21:28

hard Concepts in our Sports and I think

00:21:30

it's made me a much better teacher

00:21:32

because I use it in class even not

00:21:33

necessarily with sports data has it made

00:21:35

you a better Reasoner I I think it does

00:21:37

do you think you're more you're less

00:21:39

prone to make that mistake you're more

00:21:40

prone to remember to strength your

00:21:43

forecast because of how often we talk

00:21:44

begrudgingly sometimes particularly when

00:21:46

it comes to my

00:21:47

Yankees I think and I think I've also

00:21:49

kind of like I I guess you could call it

00:21:51

maybe a cynicism or something like that

00:21:53

but I've kind of developed sort of like

00:21:54

a little bit more of I think a

00:21:55

protection against the sort of some of

00:21:57

the recency biases like when you you

00:21:59

know when when you see some unusual

00:22:01

performance or some new thing hit a

00:22:02

sport you know are you just looking at

00:22:06

kind of the random variation of a

00:22:08

particular pitcher like suddenly having

00:22:09

a stand up performance is that an

00:22:11

example of a non-stationary thing where

00:22:13

we can see that you know we can predict

00:22:14

that for that pitcher going forward or

00:22:16

are we just kind of biased by the

00:22:18

recency of a really stand out

00:22:19

performance I think back one of our um

00:22:22

you know back 2015 I think when the

00:22:23

Kansas City Royals won the World Series

00:22:26

we spent like weeks after that talking

00:22:28

about how they've revolutionized

00:22:30

baseball with the way they like got to

00:22:32

the World Series with like you know shut

00:22:34

down relief pitching and fielding and

00:22:36

all these types of things and you know I

00:22:38

mean they did do all that but how much

00:22:40

that that actually you know how much we

00:22:42

should have really regressed even more

00:22:44

those kind of Trends towards you know

00:22:46

the mean because it was it it wasn't

00:22:48

maintained you know can you can you go

00:22:50

back 10 years can you remember the first

00:22:52

show we're in a basement here at the

00:22:54

Wharton School we were in a different

00:22:56

basement when we started before the

00:22:58

Studio existed what were you thinking as

00:23:01

we Eric rounded us up let's give credit

00:23:03

to Eric I remember sitting his office

00:23:04

the first time and then we end up in the

00:23:06

studio over there Dion was there on day

00:23:10

one what were youall thinking on day one

00:23:13

remember I'll tell you what I was

00:23:14

thinking because I remember it I

00:23:15

remember two things first of all I

00:23:18

remember being nervous speaking live

00:23:20

remember it's been a while since we've

00:23:22

done a live show but the very first show

00:23:25

was a live radio broadcast there's no

00:23:27

opportunity to fix what we did and I

00:23:29

remember being concerned about that

00:23:31

remember being concerned about saying

00:23:32

something stupid that that you couldn't

00:23:34

take back or or have some tick or

00:23:36

whatever so that was the first thing the

00:23:37

second thing was really bothered me I'm

00:23:39

curious to know what your thoughts I

00:23:40

didn't know how we were going to fill

00:23:41

two hours um and and that's of course

00:23:45

been the the most pleasant surprise I

00:23:46

feel like we can we can fill two hours

00:23:49

in fact there was a a crazy time where

00:23:51

none of you were there and all the the

00:23:53

entire sound went out and there was no

00:23:55

Communications with and it was I was

00:23:57

left alone it was and I it was live and

00:23:59

I had to talk for a half an hour to no

00:24:02

one about nothing for a half hour and it

00:24:04

was no

00:24:06

problem that might be a professorial

00:24:08

skill that you've developed over time

00:24:10

it's possible yeah I was never worried

00:24:13

about us filling up time at all not in

00:24:15

the slightest because in my case I just

00:24:17

watch so many different sports and so

00:24:19

much Sports and I'm just used to also

00:24:21

taking notes when I'm watching sports on

00:24:23

things that also to me the beauty of

00:24:26

sports for me is that it's the ultimate

00:24:29

Trojan Horse to talk about any

00:24:31

statistical topic like should you go for

00:24:34

it on fourth and one should the coach do

00:24:36

this at this point in time should you go

00:24:38

for three should you foul in the last 10

00:24:40

seconds but every one of those I could

00:24:43

give and I'm sure you guys could do the

00:24:45

same could give an entire lecture just

00:24:48

on that topic so I was never ever

00:24:51

worried about it and also my my view was

00:24:54

um the one thing I've learned through

00:24:56

teaching and you mentioned about its

00:24:57

effect on teaching and and the radio's

00:24:59

really changed it in that way too is

00:25:01

that I figured if I'm having fun the

00:25:04

audience is having fun and that's always

00:25:06

been my motto for teaching as well and

00:25:09

part of the reason I brought this group

00:25:10

together I mean we've known each other

00:25:11

longer than I've actually known both of

00:25:12

them and my comment was is that we were

00:25:15

going to have fun no matter what talking

00:25:17

about sports and statistics and if we

00:25:19

had fun I figured it would be fine no

00:25:21

whether Sirius XM would keep us on the

00:25:23

air for a month a year or now 10 years I

00:25:26

actually wasn't worried about that I was

00:25:27

just like if we're having a good

00:25:29

conversation and we're having not only

00:25:30

fun but learning something if we're

00:25:32

learning something the audience will

00:25:33

learn something too but Eric you taught

00:25:35

that to us I mean I learned that from

00:25:37

you that we can talk about sports at a

00:25:38

high level and teach statistics and do

00:25:41

this for two hours with no problem but I

00:25:43

remember not knowing that at the time no

00:25:45

and I remember that I I can Echo the

00:25:47

nervousness of you know I mean again

00:25:49

we've learned since then also that

00:25:50

you're you're encyclopedic knowledge of

00:25:53

sports I can I can say stupid sh every

00:25:55

show and you're there to kind of as you

00:25:57

know sort of a uh instant fact Checker

00:25:59

so that that was reassuring and I also

00:26:02

do think I I kind of worried for a while

00:26:05

that we would sort of be able to kind of

00:26:06

generate to our you know like that

00:26:08

amount of material every week but as we

00:26:10

started doing the show and we started

00:26:12

seeing these kind of various themes

00:26:14

popping up I started seeing them

00:26:15

everywhere like I you know I would be

00:26:17

you know I'd be you start looking for

00:26:18

them you start looking for them and you

00:26:20

and you you do sort of like kind of

00:26:22

enabled me to sort of see connections be

00:26:24

between sports at like kind of the base

00:26:26

you know between versus Within ation

00:26:28

these kind of basic levels that you know

00:26:31

basically allowed me to kind of I think

00:26:33

Branch out to sports like tennis that I

00:26:35

didn't have much experience with before

00:26:37

the show even started Shane you're

00:26:39

talking about between sports makes me

00:26:41

think about um tournament design as a a

00:26:44

a late answer to an emerging theme I

00:26:46

never knew I was so interested in

00:26:47

tournament design until this show came

00:26:49

along what about favorite moments we

00:26:50

talked about how you were thinking about

00:26:51

it 10 years ago 10 years of shows any

00:26:54

favorite moments come to mind I'm sure

00:26:56

we've got lot but like what comes to

00:26:57

mind without no one's keep keeping track

00:26:58

or score but what might come to mind

00:27:00

well just cuz we were talking about the

00:27:01

early days in vance H I think one of the

00:27:04

I think maybe the first Super Bowl kind

00:27:06

of re I think it was Super Bowl 49 the

00:27:09

the Malcolm Butler interception was one

00:27:11

of my favorite shows we came in I mean

00:27:13

obviously you know my my team won it all

00:27:15

but it was more coming in and just sort

00:27:17

of like it was early on in our history

00:27:20

it just seemed like just kind of the

00:27:21

perfect time to talk about something

00:27:23

that had like obviously a lot of

00:27:24

consequence but also like a lot of

00:27:26

analytical strategy discuss and just

00:27:28

seeing that kind of moment being

00:27:30

discussed in the like 10 years to follow

00:27:32

and how how that kind of discussion of

00:27:35

that particular like play has been

00:27:37

informed by 10 years of kind of you know

00:27:40

increasingly sophisticated analysis that

00:27:42

that play has stayed the test of time

00:27:44

let me just say it's still discussed

00:27:45

today I I think uh I told AI just before

00:27:48

the show taping here what I was going to

00:27:50

talk about but part of his just I love

00:27:52

seeing your reaction each time Kade I'm

00:27:54

going to talk about Joey

00:27:56

Chestnut no no

00:27:59

yes to bring that up just I mean again I

00:28:02

want to say for our listeners here I

00:28:03

want to say it again Hot Dog Eating is a

00:28:05

real sport you can train for it and I'll

00:28:09

tell you the thing I like about it as

00:28:10

well is that you can measure

00:28:13

atypicality and that's the thing is that

00:28:16

you can measure how many how much how

00:28:18

many more hot dogs can this person eat

00:28:20

than that one and when Joey chessnut can

00:28:22

eat 20 25 more than other people and

00:28:25

people have now especially now he's been

00:28:27

like the 15 time mustard belt Champion

00:28:30

why haven't people caught up to him I'm

00:28:32

just saying you have to you have to

00:28:34

admire someone who's his exceedence is

00:28:38

so far greater than everybody else that

00:28:41

to me so part of it is just seeing your

00:28:43

reaction I just love it because I in my

00:28:45

heart I'm not just saying it to piss you

00:28:47

off I really think it's a sport and I

00:28:49

love talking and when he came on the air

00:28:50

and he talks about his training methods

00:28:52

and everything else I'm like this a real

00:28:54

scientist working here 20 years from at

00:28:56

our 20th anniversary Kate have totally

00:28:58

turned around on this I think no he

00:29:00

won't you're not you're not there yet I

00:29:01

can tell already so any other favorite

00:29:03

moments a so I have a bunch of favorite

00:29:05

moments that so it's interesting because

00:29:07

I don't not really concentrating on the

00:29:09

guest per se because we've had some

00:29:10

recurrent guests who are incredible um

00:29:13

and but the moments for me are when I

00:29:15

learned something that's shockingly new

00:29:17

to me so I remember when Rick Peterson

00:29:19

oh told something to us remind us Rick

00:29:21

Peterson was was a was a pitching coach

00:29:23

longtime pitching coach was the Oakland

00:29:25

A's and he was on our show quite

00:29:26

frequently in the beginning regular

00:29:27

guest as a regular guest and uh we used

00:29:30

to bring him in for about 10 minutes and

00:29:32

as a baseball analyst one of the big

00:29:34

strategic questions are why you not

00:29:35

bringing in your relievers at the high

00:29:37

leverage moments and we analysts were

00:29:39

always saying this is just dumb how come

00:29:40

they have this s conventional way of

00:29:42

doing things and you should you should

00:29:44

do some things differently and he said

00:29:46

you know you guys that's that's fine to

00:29:48

do this but you realize that there's all

00:29:50

this actual onfield issues that you

00:29:53

don't have any idea about like a pitcher

00:29:55

has to warm up and once you've warmed

00:29:57

them up and you don't bring them in we

00:29:58

might not be able to use them again and

00:30:00

we were sitting here listening to this

00:30:02

and going oh and I remember feeling

00:30:05

really schooled by an actual baseball

00:30:08

profession like a real Egghead moment

00:30:10

you and like boy I think we we have a

00:30:12

lot to say but we can't Implement

00:30:15

without actual onfield expertise okay

00:30:18

and that's just a general lesson that we

00:30:19

should be not just knowing ourselves but

00:30:22

preaching to other statisticians and

00:30:23

data that's right it's a humility that

00:30:25

that Rick was saying get over yourselves

00:30:27

on this you don't have the full story

00:30:29

and that was that moment of course there

00:30:31

are other moments as well like when mie

00:30:32

Betts told us that the real reason why

00:30:34

Bill Buckner let the ball go through his

00:30:36

legs because he was running and it got

00:30:37

him so nervous I enjoyed that moment

00:30:40

wilsonon sorry everyone I screwed that

00:30:43

up mie Wilson

00:30:45

yes you're talking about uh Rick

00:30:48

Peterson and and giving some background

00:30:50

that that was I opening to you reminds

00:30:52

me out of the blue of what Chris

00:30:54

Collinsworth told us when we talked to

00:30:56

him about Bill bellich he said he was at

00:30:58

some practice the Patriots practice

00:31:00

early in the season and he came away

00:31:02

thinking that belich was actually

00:31:06

burning some wins early in the season in

00:31:09

order to rotate defensive strategies and

00:31:12

to train his players to play multiple

00:31:14

strategies at a way that might cost them

00:31:16

early season in order to learn to be

00:31:18

better later in the season this was

00:31:20

Collinsworth interpretation of what

00:31:22

belich did and it was like oh wow that's

00:31:24

interesting and it did I do remember

00:31:26

that now that you say it one of those

00:31:27

really eye opening um I have another one

00:31:30

too and it's a surprising one for me

00:31:31

it's one of our horse racing

00:31:35

interviews no it was the one where we

00:31:37

were talking about like you know what

00:31:38

really take makes a horse like a a true

00:31:41

rener he's like oh it's the horse with

00:31:43

the biggest heart and we were all like

00:31:45

all of us cynically were like oh you

00:31:46

know we've heard that kind of wants it

00:31:48

the horse wants it the most but he was

00:31:50

literally talking about biologically the

00:31:52

horse with the largest heart is the one

00:31:55

most likely to win the race and I

00:31:56

thought that was you know that was is

00:31:58

kind of a fun little that's a top three

00:32:00

that's a consensus top three all time

00:32:01

also in that same interview he mentioned

00:32:03

that also every horse slows down during

00:32:05

the race just who slows down the least

00:32:08

it's kind of like the sinking fastball

00:32:09

kind of which also was a great

00:32:12

moment one of my favorites was I think a

00:32:15

category that I like is um when our

00:32:17

guests have little either they share

00:32:20

they have an epiphany or they share a

00:32:21

moment and we had one I think the first

00:32:23

time we ever had Brian Burke on the show

00:32:25

so Brian is an ESPN analyst and one of

00:32:28

the guys who had been on The Cutting

00:32:29

Edge of football analytics for a while

00:32:30

Brian had been an engineer by training

00:32:33

he flew for the military and then he got

00:32:36

into football analytics but his

00:32:39

background had him very black and white

00:32:42

and when he got into football analytics

00:32:43

he started seeing Shades of Gray he

00:32:45

literally said I had to start thinking

00:32:47

probabilistically for the first time in

00:32:49

my life and this is a you know highly

00:32:52

accomplished mid-30s probably guy and he

00:32:55

was saying that's what stats and

00:32:56

thinking about sports analytics did for

00:32:58

me it turned me from black and white

00:33:00

Reasoner to Shades of Gray and

00:33:01

probability so what you just said maybe

00:33:03

think about something that I'd love Audi

00:33:04

to talk about the work you're doing with

00:33:06

Ryan Brill on the fourth down in just a

00:33:08

second but the thing that we also I

00:33:10

surprised it didn't come up as one of

00:33:11

our themes also is is the role of

00:33:14

uncertainty which is even if you put a

00:33:17

probability on something that's your

00:33:19

point estimate of the probability like

00:33:21

probability is 60% you should go for it

00:33:23

on fourth and one okay but the

00:33:25

uncertainty is plus or minus 10% on and

00:33:28

so one of the things I think I meaning

00:33:29

with a coin we know with Precision what

00:33:32

that probability is but with these real

00:33:34

world team events we're estimating

00:33:37

something and and in general the you

00:33:38

know the classic rule of thumb is well

00:33:40

in fact I think what I'm about to say is

00:33:42

probably a theorem but audio correct me

00:33:44

which is by definition you're always

00:33:46

underestimating the probability like in

00:33:48

order you could say well a fair coin's

00:33:50

5050

00:33:52

maybe I don't know that like maybe the

00:33:54

way you flip it I'm just saying you can

00:33:56

always say there's some Ed factor that

00:33:58

you're not putting in there but I think

00:34:00

just to me the role of uncertainty even

00:34:03

in things that are uncertain we don't

00:34:05

take into account enough yeah and we

00:34:07

have to evaluate that and we've learned

00:34:08

to do that over the years remarkably um

00:34:11

and but sometimes you have to do it fast

00:34:12

but in particular you're talking about

00:34:13

so bril Wier right so we we we came to

00:34:16

this idea saying looking at to figure

00:34:18

out like what is the what is the

00:34:19

uncertainty in the models and what we

00:34:21

really disc models you're talking about

00:34:22

are the fourth down models we really

00:34:24

discovered that the data isn't really

00:34:25

that rich which means that given data is

00:34:28

a constraint and as a result you can

00:34:30

come up with an estimate but if you had

00:34:32

gotten a different set of data different

00:34:34

2,000 games you might ask well how might

00:34:36

the estimate change and that's really

00:34:39

what What's called the confidence

00:34:40

syndrome we realized that those things

00:34:41

were really wide almost shockingly wide

00:34:43

because there's not much data in

00:34:44

football because there's just not that

00:34:45

much data in football which led us to

00:34:46

the which connected with the Rick

00:34:48

Peterson comment which says wait a

00:34:49

minute if we don't really know we have

00:34:51

to have some humility about what we

00:34:52

don't know just cuz I've got a point

00:34:54

estimate doesn't mean that that's the

00:34:55

correct estimate and I can't oversell it

00:34:57

to the team because they have

00:34:59

information that I don't have and that

00:35:01

information should be more important

00:35:03

particularly when I don't have good

00:35:05

information and I've got to let them let

00:35:07

them go for it okay this connects to two

00:35:09

other of my favorite Point moments on

00:35:11

the show one is it connects back to our

00:35:13

regress your forecast and one of my

00:35:16

favorite standout moments me in memory

00:35:19

was the last bit of the show the last

00:35:23

show before the 2016 presidential

00:35:25

election so we've done a fair bit of

00:35:26

analytics on the presidential election

00:35:28

leading up to that and if yall remember

00:35:30

538 famously had Hillary Clinton as I

00:35:33

don't know 80% or something likely to

00:35:35

win that election and the last thing

00:35:37

that happened on that show was AI saying

00:35:39

ah you

00:35:41

know I'm gonna I'm gonna go with

00:35:42

something I'm be like really like how

00:35:44

much less I don't 67 I play that clip to

00:35:47

my class every election year yes I do I

00:35:50

put it on there because it's so 538s was

00:35:53

80% but almost every other forecast was

00:35:55

like 99% yeah they were actually the

00:35:57

most humble

00:35:59

Sil just said I just don't feel like it

00:36:02

and he did something completely

00:36:03

unpraised

00:36:07

dung for Hillary Clinton but something

00:36:09

doesn't feel right and we had a

00:36:10

conversation about what doesn't feel

00:36:12

right on the show and I was trying to

00:36:14

argue that it's got to be a lot lower

00:36:16

her probability of winning is a lot

00:36:17

lower and although we had no I had no

00:36:19

hard data to go on I was adjust looking

00:36:22

at previous elections within that

00:36:24

calendar cycle in other countries where

00:36:26

the populous candidate was coming out

00:36:28

big and it had and and overturning the

00:36:30

forecast that was a great moment so I I

00:36:32

what I like about it is that that's

00:36:34

probably that's 2016 so we're only two

00:36:35

years into the show we're talking about

00:36:38

we're we're all cultivating this

00:36:39

tendency to regress and pay attention to

00:36:41

base rates and the question was always

00:36:43

like are we like be you is that true

00:36:46

because in that moment I wasn't there's

00:36:48

no way I was thinking regress that

00:36:50

forecast it was only when AI said I'm

00:36:52

going to regress that forecast and now

00:36:54

because of that moment and because of

00:36:55

the reinforcement that that came about

00:36:57

from the

00:36:58

it does start maybe bringing us to that

00:37:00

habit but it's so easy to say it

00:37:02

intellectually but in the moment we

00:37:03

don't tend to do it you did it in that

00:37:05

moment and that was one of my kind of

00:37:07

what I talked about inevitability bias

00:37:08

or rounding those 80% up to one that was

00:37:11

a mo that was a real lesson for me you

00:37:13

know that was kind of a a real learning

00:37:15

moment I want to name one other that's

00:37:17

connected to this and that is uh and it

00:37:19

may be another question we can pursue in

00:37:21

a seconds like we've read some books

00:37:23

every now and then for some of the

00:37:25

people we interview and we read a book

00:37:27

called called escape from model Land by

00:37:30

Erica Thompson and this is written by a

00:37:33

modeler cautioning modelers to not get

00:37:35

too caught up in your model like you got

00:37:37

to recognize what we all love our models

00:37:39

and we begin to think the world is our

00:37:41

model but the model is only an imperfect

00:37:43

capture of the world and we have we have

00:37:45

to get out of that and it connects

00:37:47

exactly what ai's saying about what's

00:37:49

your data you know how how sure you of

00:37:51

these estimates you got to have that

00:37:53

humility it really comes back to that

00:37:54

humility well you may also remember for

00:37:56

years I asked Our Guest the following

00:37:57

question I always used to end when my

00:38:00

when I got my time to speak I'd always

00:38:01

ask the following question you could

00:38:03

have one of three things better you

00:38:05

could have better data better model or

00:38:08

better let's call it internal buyin I

00:38:10

never in all our years doing it I never

00:38:13

heard someone say a better model like

00:38:14

wow you're right my random Forest my XG

00:38:16

boost models just not working well

00:38:18

enough no no I need a better what you

00:38:21

always hear is either it would be

00:38:22

greater to have more impact at my

00:38:24

company because there's better

00:38:25

connection or I'd rather have a

00:38:27

randomized experiment or you know I have

00:38:29

Spar data on fourth and one I'd really

00:38:30

rather have an INF never did someone say

00:38:33

a more sophisticated model that can give

00:38:35

me another third significant digit on

00:38:37

something ever that's you I didn't I've

00:38:39

heard you answer that asked that

00:38:41

question in lots of environments on the

00:38:42

show and off I've stolen it on occasion

00:38:44

myself I didn't know that you had done

00:38:46

it enough to begin drawing inference

00:38:48

about it and I love that no one ever

00:38:50

said the model ever matter I'm pretty

00:38:51

sure the number could be zero but it's

00:38:54

interesting because you talked to all

00:38:55

the the representatives of of startup

00:38:57

companies that are selling data and

00:38:59

selling and selling pipelines like we

00:39:01

just talked to people from data bricks

00:39:03

and we have talked to another company uh

00:39:05

just called shot quality they aren't

00:39:07

selling models nobody's selling models

00:39:09

they just and and I always wonder why

00:39:11

they're not because there's a there's an

00:39:14

edge there right they're selling you

00:39:17

know serious strategy advice and data

00:39:19

and they just layer on top of it

00:39:20

somebody's random forest or deep net

00:39:23

deep neural net just

00:39:25

something we're sitting here 10 years in

00:39:27

D Wharton Moneyball all the all the

00:39:29

co-hosts longtime collaborators here

00:39:31

taking a moment to think about how the

00:39:33

world has changed over those 10 years

00:39:35

how we've changed some of the things

00:39:36

we've learned maybe some of the

00:39:38

highlights we were just talking about

00:39:40

Erica Thompson's escape from modeland

00:39:42

which came out she was studying partly

00:39:45

covid and talking about models in covid

00:39:47

there was this moment in time guys where

00:39:49

we were almost actually we were pretty

00:39:51

much full-time covid analysts so March

00:39:55

2020 hit and we start talking about Co

00:39:58

making sense of Co at some point we

00:40:00

decided we're going to stay on this and

00:40:01

we're going to dedicate the first half

00:40:03

hour I think of every week to co what

00:40:06

are your Reflections now that's been

00:40:07

we've we shifted back more than two

00:40:10

years ago now but it was a long stretch

00:40:13

when you think back on it now three

00:40:15

months was an hour we didn't have any

00:40:17

sports remember that that the beginning

00:40:19

that's right the first three months it

00:40:20

was all pandemic all the

00:40:23

time what Reflections do you have on

00:40:25

that stretch of Wharton I I remember it

00:40:27

extremely well because I really dived

00:40:29

into the data deeply you became our

00:40:31

expert and um but just I mean the lesson

00:40:33

that modeland the book which came out

00:40:35

much later we learned on the Fly because

00:40:38

there were modelers coming out with

00:40:40

forecasts for the spread of covid and

00:40:43

forecasts on diseases and on

00:40:45

hospitalizations and everything that we

00:40:47

wanted and policy was being built on it

00:40:49

and those forecasts were terrible and it

00:40:53

was the best Minds doing the best job

00:40:55

with the data they had and you couldn't

00:40:57

do anything couldn't do it and it was

00:40:59

just an incredible humbling lesson um

00:41:02

where I was disappointed was that people

00:41:04

would put out models with their

00:41:06

prediction intervals and their

00:41:08

prediction intervals nobody's overlapped

00:41:11

and that said to me this is a problem if

00:41:13

we have 10 different forecasts about

00:41:14

where we're supposed to be going down

00:41:16

the line really time series forecasts

00:41:17

with confidence bands or prediction

00:41:19

bands on top of them and they were all

00:41:21

so depressingly narrow and we looked at

00:41:23

them and this is just bad so not only

00:41:25

were the forecasts bad but much more

00:41:27

importantly the uncertainty was horribly

00:41:30

underestimated and that just kept

00:41:32

appearing over and over again as we move

00:41:34

through the pandemic it was almost

00:41:36

ridiculously tragic how little we we

00:41:38

could say with any accuracy I was going

00:41:40

to say the one thing I remember though

00:41:41

is that some things were relatively

00:41:43

stable like what I do remember is

00:41:45

because I remember each week I was the

00:41:47

one that was looking at the CDC data and

00:41:49

just seeing like in some sense like the

00:41:51

death rate conditional in you're getting

00:41:54

covid was actually fairly constant for a

00:41:57

long period of time now of course the

00:41:59

question is how many people are going to

00:42:00

get covid which people are going to get

00:42:02

covid what's the risk for a certain

00:42:04

population but there was a period where

00:42:06

I think the number was somewhere around

00:42:08

as I have a vague recollection maybe it

00:42:09

was 1% or 08 of 1% where if you looked

00:42:13

at every country and you said how many

00:42:15

people have Co reported report hug I

00:42:18

understand reported and how many people

00:42:20

were dying that number had a very narrow

00:42:24

Bandon interval for a fairly long period

00:42:26

of time so one thing that was

00:42:28

interesting because we talk about

00:42:29

moments when David Spiegel halter came

00:42:31

in as a guest later not there that day

00:42:33

um which he was amazing he was one of

00:42:36

the sharpest Minds on covid early

00:42:38

earliest on he had a beautiful infection

00:42:41

fatality curve that was by age that

00:42:44

turned out to be just about exactly

00:42:46

right and he published that in like

00:42:49

March of 2020 and we we publicized it on

00:42:52

our show and we talked about it and many

00:42:54

people came up to us that you were on we

00:42:56

provided solid good data on Co that they

00:42:59

weren't getting anywhere else I think

00:43:01

that's what I'd like to hear a little

00:43:02

bit more from you is like setting aside

00:43:04

the vagaries of modeling the pandemic

00:43:07

how what what do we do well what do we

00:43:09

not do well in that show I me did we

00:43:11

stay too long do we spend too much time

00:43:12

to little time like how did we because

00:43:15

that was a incredible moment of

00:43:17

uncertainty and I think one of the

00:43:18

things we did was we worked together to

00:43:22

reduce that uncertainty we were trying

00:43:23

to make sense of it like together that's

00:43:25

I think one of the virtues of what we

00:43:26

did what what are your thoughts on and

00:43:28

it gave us a vantage point at least for

00:43:29

me I mean it was obviously I think

00:43:31

probably a frustrating time for all of

00:43:32

us but it was particularly frustrating

00:43:33

for me kind of along kind of you know

00:43:35

Eric's way of kind of thinking about

00:43:36

things would you rather have good data

00:43:38

good models or or Buy in that was a time

00:43:41

in society where although we you know

00:43:44

could talk and we were mostly focused on

00:43:46

evaluting the models what we really

00:43:47

needed was better data and what we

00:43:49

really needed was actual buyin you know

00:43:51

on on public policy kind of initiatives

00:43:54

and so it was kind of I mean a little

00:43:56

bit of therap therapy for me but also

00:43:59

like you know actual like instructive to

00:44:02

kind of come in and hear mostly from

00:44:03

Audi like kind of the what was really

00:44:05

kind of going on in the kind of modeling

00:44:07

sphere and also to kind of just just be

00:44:09

able to kind of I I don't know Express

00:44:12

both of my kind of frustration but also

00:44:14

interest in in what was kind of going on

00:44:16

at a societal level at the time through

00:44:18

kind of the lens of analytics and

00:44:20

through the lens of trying to kind of

00:44:21

think about what we could actually

00:44:23

measure you know without bias Etc I have

00:44:26

to admit though I also I think back to

00:44:27

that period I also have to think about

00:44:29

um how I remember thinking to myself how

00:44:32

you know we're in we're exactly the same

00:44:34

age and so I remember there were times

00:44:37

where you would say yeah you know what I

00:44:38

would go out and have outside I'd have

00:44:40

some sort of party with friends and I

00:44:42

remember I said say it on the air like

00:44:44

oh I would never have done that our

00:44:46

internal no no no that was fascinating

00:44:48

to me and and by the way I don't think

00:44:50

of myself as overly risk averse in any

00:44:53

particular way and and you were you were

00:44:56

our expert so all most of the

00:44:57

information I was getting I was getting

00:44:59

from you just my interpretation of it

00:45:01

was like I'm not going to go for that

00:45:03

really even it's a low probability event

00:45:06

it's just not worth it and that to me I

00:45:08

I will remember all the decisions you

00:45:10

guys made and like like I was proud like

00:45:12

wow I got my like 80th shot not's like I

00:45:14

don't know if you should be getting that

00:45:16

many shots and I'm like I've got 80 go

00:45:19

ahead oh I was just going to say I can't

00:45:20

really it was a lesson how that

00:45:22

distribution of like kind of risk

00:45:24

tolerance versus risk adversity how much

00:45:27

there is across people within a person

00:45:29

even over time in that kind of in in in

00:45:32

that it was fascinating one of the

00:45:34

things we learned about covid during

00:45:35

that time which has been really useful

00:45:38

is the importance of observational data

00:45:40

is distinct from experimental data and

00:45:43

in sports we talk about confounding it's

00:45:45

just the most important Concept in all

00:45:47

any evaluation of a a complex game where

00:45:50

everybody interacts with each other

00:45:51

particularly basketball soccer football

00:45:53

much less so baseball but even that to a

00:45:55

degree and what this forced us to

00:45:58

constantly ask when we looked at a study

00:46:00

well these aren't these aren't

00:46:01

experiments and so much public policy

00:46:04

and so much discussion around covid was

00:46:06

built around observational data some of

00:46:08

it terribly done some of it better done

00:46:11

and we were able to explain that to to

00:46:13

our listeners to each other and make

00:46:15

sense out of it and this was this was

00:46:16

really one of the most and I think it

00:46:17

really carried us through it it puts

00:46:19

fart in Center the scientific method

00:46:21

which is at heart really what statistics

00:46:24

is serving the the acquisition of

00:46:26

knowledge and truth okay so I everything

00:46:29

you said I have to agree with entirely

00:46:30

of course but it also feels like one of

00:46:33

the things that happened in the pandemic

00:46:34

is

00:46:35

that experimental evidence by itself was

00:46:39

too isolated to really explain what

00:46:41

happens in the complicated World it goes

00:46:43

the other direction as well we have

00:46:45

experiments well you can't experiment

00:46:48

with you have a hard time experimenting

00:46:50

with societal uptake of policies and one

00:46:54

and the biggest confounder especially

00:46:56

early on was

00:46:57

people didn't act the way they were

00:46:59

supposed to act in the models and we

00:47:00

wouldn't have known that I don't think

00:47:02

from an experiment so in this case I

00:47:04

think it goes the difficult is you need

00:47:06

to go both ways we had to explain this

00:47:07

that why these weren't predictable what

00:47:09

was the issues and we did I think that

00:47:11

was one of our our strongest points with

00:47:14

within the field of Statistics as you

00:47:15

know nobody would consider me an expert

00:47:17

specifically in causal inference that's

00:47:19

not really the domain I operate in but I

00:47:21

kind of felt like you know with was

00:47:24

happen with my friends and kind of with

00:47:25

with the L people I was talking to like

00:47:27

having to explain just kind of very

00:47:29

basic what you can learn from an

00:47:30

experiment what you can not learn from

00:47:32

you know observ study is controlled

00:47:35

observational study all right guys let's

00:47:38

shift out of the pandemic and do a

00:47:40

lightning round before we end up with a

00:47:41

couple of final questions lightening

00:47:43

round back to favorite moments but let's

00:47:44

look at some specific uh some specific

00:47:47

narrow aspects of the show um any

00:47:51

favorite guest moments or interviews any

00:47:53

any things jump out to you over the

00:47:55

years we've talked about some so

00:47:58

far I mean you've had some big baseball

00:48:01

guys that you just love having bunny

00:48:03

chance we had Ron Bloomberg which I

00:48:04

enjoyed

00:48:05

immensely we I we I don't know if Matt

00:48:09

was the producer then but Sam Gwyn wrote

00:48:12

a book about HAL mummy the the the

00:48:14

famous football coach who was kind of

00:48:17

the beginning of the air raid and Sam

00:48:19

Gwen and so I had him on the show I

00:48:21

didn't know it was going to be on the

00:48:22

show I had just read an earlier book of

00:48:24

his that is really about the history of

00:48:26

Texas that was profound for me and I

00:48:28

come in some Wednesday morning and

00:48:30

Matt's lined up Sam Gwen for the

00:48:32

interview seg like oh my God I just this

00:48:34

guy just wrote this amazing book just

00:48:36

random you know pleasure any other guest

00:48:39

jump out well when we uh were on radio

00:48:41

roll for uh one of the Super Bowl I

00:48:43

think the Super Bowl in Miami um and we

00:48:46

got to interview Justin Tuck on the show

00:48:48

and I was wearing the same Tom Brady

00:48:50

coat and just the absolute look of

00:48:52

disgust on his face was kind of a a Gade

00:48:55

Mo that was a great interview to because

00:48:57

again Justin Tuck you know is not I

00:48:59

don't think the most you know

00:49:00

necessarily analytically minded like

00:49:03

athlete but it was a really informative

00:49:05

kind of discussion because we talked a

00:49:06

lot about you know I mean he was you

00:49:08

know on the defense that went up you

00:49:10

know went up against some great teams in

00:49:11

the Super Bowl and just sort of the way

00:49:13

he thought about you know battling Brady

00:49:15

and the way he thought about other

00:49:16

quarterbacks I just remember it being a

00:49:17

really cool discussion I loved also the

00:49:20

moments we've talked about we could call

00:49:21

them the more secondary sports like when

00:49:24

we've had people on golf talk about the

00:49:26

golf Analytics and you know like the

00:49:28

smarter players know where to forget

00:49:31

they know they're going to be inaccurate

00:49:32

so they choose a zone that they not

00:49:34

going to cause thems as much damage like

00:49:37

you could go for the left side of the

00:49:38

green but you know you should shoot for

00:49:41

the right side of the green because

00:49:42

here's how much variance there is in

00:49:43

your shot that's something that really

00:49:45

has always stuck with me that was one of

00:49:48

my most eye-opening interviews you i'

00:49:49

had forgotten about that that was one of

00:49:51

those moments where I'm listening to

00:49:52

this guy talk and it's Scott faucet

00:49:54

Scott faucet is the golf golf coach he

00:49:58

played some golf golf coach now in the

00:49:59

Dallas Fort Worth area and we chase him

00:50:02

down because of an article in the

00:50:03

newspaper and he starts talked on our

00:50:05

show about these guys have to act like

00:50:09

they're going to hit it perfectly

00:50:10

because you got to have confidence but

00:50:12

they have to plan knowing that they

00:50:14

can't hit it perfectly even the best

00:50:16

golfers in the world he was talking

00:50:17

about accepting uncertainty and

00:50:19

accommodating uncertain in your

00:50:21

decision- making it was absolutely

00:50:22

profound moment loved it yeah so I have

00:50:25

a few but I remember

00:50:27

some of my favorites were David Epstein

00:50:28

he's written two of my favorite books on

00:50:30

Sports one is a sports Gene and I

00:50:32

interviewed him about that book I may

00:50:33

have been alone this was a long time ago

00:50:35

and then he had a second book range

00:50:37

which he talked to us about the

00:50:39

importance of athletes trying out

00:50:41

different sports and how that's

00:50:43

important for for Success that was just

00:50:45

those were incredible interviews and

00:50:47

then of course I remember um Annie

00:50:49

Duke's first interview with us she came

00:50:51

live in the studio she she was writing a

00:50:53

book uh thinking in bets and she talked

00:50:55

about poker and reading opponents and I

00:50:57

remember one thing she said to us and

00:50:59

and we've she's been on many times she

00:51:00

comes from our our our my Moneyball

00:51:02

Academy every summer um but she talked

00:51:04

about how the importance of exactly the

00:51:08

opposite of what we do which is you know

00:51:10

calmly think about data and regress but

00:51:12

when you're in a procer tournament you

00:51:14

got to size up that opponent immediately

00:51:17

you don't have time to wait and watch

00:51:19

their re their play and accumulate data

00:51:22

and move yourself off the prior you've

00:51:24

got to look at everything they they do

00:51:26

how they touch their chips how they talk

00:51:29

to each other to quickly figure out what

00:51:31

type of type almost cluster type of

00:51:33

player because if you get it wrong

00:51:36

you're going to get hammered and it's

00:51:38

like very observable just the expertise

00:51:41

that goes into being a professional

00:51:43

poker player that's much more Beyond

00:51:45

just the the strategy of just do doing I

00:51:48

remember that too it was super

00:51:50

insightful I I I kind of found myself

00:51:51

coming back to that because you know it

00:51:53

was really about what she was talking

00:51:55

about is like very body language that as

00:51:57

a as kind of a professional in the field

00:51:59

you learn you know you really immerse

00:52:02

yourself in when coming out of Co one

00:52:04

thing or going through Co and coming out

00:52:07

of it I kind of from my own teaching I

00:52:09

got the I realized just how much I feed

00:52:12

off the body like you you are subtly

00:52:14

measuring the body language of people

00:52:15

like teaching in front of people in

00:52:17

person is a very different kind of

00:52:20

endeavor because you can read the body

00:52:22

language of people and very s you know I

00:52:25

I was I more kind of I I could perceive

00:52:28

what I was picking up on in very sort of

00:52:29

subtle signs and I think that's probably

00:52:31

you know for poker players that kind of

00:52:33

become that as their vocation that

00:52:34

almost becomes probably second nature in

00:52:36

the way I didn't even realize I was

00:52:37

doing it as a teacher a favorite offsite

00:52:44

moment well well I can start with one

00:52:47

you mentioned Justin Tuck before but on

00:52:49

our offsite moment I was quoting I think

00:52:52

Rufus Peabody or Kade Massie talking

00:52:55

about how you predict

00:52:57

future football games and how no no

00:52:59

player can change the line when they go

00:53:02

down except for a quarterback otherwise

00:53:04

it just doesn't affect it we're not

00:53:06

saying those players aren't important we

00:53:07

just we can't measure it and I told this

00:53:09

to Justus tuck and he looked at me and

00:53:12

this very imposing human being looked

00:53:14

like he was ready to take me down like

00:53:16

what are you saying I don't matter and

00:53:17

here I am trying to scramble wait a

00:53:19

minute of course that's what I'm saying

00:53:21

yeah

00:53:23

yeah one of one of mine was at the Miami

00:53:26

Super Bowl we ended up at a bar on South

00:53:30

Beach late after dinner and Eric eager

00:53:32

was there with somebody of his from the

00:53:34

Kansas City area yeah but this was

00:53:37

January or early February 2020 which

00:53:40

means pandemic was already in the air

00:53:43

but Eric had been using some um some

00:53:48

diffusion model based on pandemics in

00:53:50

one of his Sports papers and so he was

00:53:52

studying pandemic models in the months

00:53:54

prior to this and he sits there late at

00:53:56

night drinking some monster Margarita or

00:53:58

whatever it was on South Beach and he

00:54:00

said I think y'all should be looking at

00:54:01

this pandemic thing but you know what I

00:54:04

took out of that lesson that that

00:54:06

conversation I think we should be

00:54:07

talking talking to Eric eager that's

00:54:09

right he's not only been on our guest

00:54:11

many times but he's now a longtime

00:54:13

collaborator with us at Mone Moneyball

00:54:14

Academy so that was an incredible

00:54:16

offsite moment that's right that's right

00:54:18

honestly I was thinking of that uh

00:54:21

actual I think that same evening talking

00:54:23

with Eric eager because I was thinking

00:54:25

about that kind of in the cont Tex cuz

00:54:26

that was 2020 right before the pandemic

00:54:29

that was right before you know the

00:54:30

chiefs were about to play the 49ers I

00:54:32

think that was the Super Bowl and he was

00:54:35

already talking about a Chief's Dynasty

00:54:38

and I'm like this guy is this guy's

00:54:40

getting a little ahead of himself what

00:54:41

you know what does he know this guy's a

00:54:44

blow heart and you know it turns out

00:54:48

well I went to dinner that night with

00:54:49

you guys and the pre- dinner bar but I

00:54:51

went home before you guys went out with

00:54:52

Eric eager so I don't remember that you

00:54:54

traveling with family I was traveling

00:54:55

with family but what I do I when I think

00:54:58

about it is actually really I give a lot

00:54:59

of credit to SiriusXM I remember the it

00:55:02

was probably the time we interviewed

00:55:03

Justin Chuck and like we were there on

00:55:05

radio row and I have to admit we looked

00:55:07

legitimate like there was a big Serious

00:55:09

XM sign behind us and we were up there

00:55:12

and interviewers looking over like these

00:55:14

are some serious broadcast I was wearing

00:55:16

this Cod so I mean you

00:55:18

guys it actually made me feel good that

00:55:20

like you know I make it up CBS Sports

00:55:23

was across the way and like and there's

00:55:24

us four guys Wharton Money Ball and

00:55:26

we're there in radio row I I felt like a

00:55:28

legitimate broadcaster for a minute do

00:55:30

you remember when they had the NFL draft

00:55:33

here in Philadelphia I do and we went

00:55:35

did the show on site beforehand but then

00:55:37

we started walking around backstage and

00:55:40

we end up walking through the dadgum

00:55:41

Green Room players sitting around with

00:55:44

their families were like yeah we're not

00:55:46

supposed to be in here well it's funny

00:55:48

because some of my favorite off-site

00:55:50

moments was going down to spring

00:55:51

training bringing our equipment talk

00:55:53

about being somewhere you're not

00:55:54

supposed to be right and and I get

00:55:55

myself

00:55:57

press access and I wander into the

00:55:59

Yankee Dugout and the Yankee Dugout and

00:56:01

the Yankee Clubhouse and I'm walking

00:56:03

around with all the real reporters I'm

00:56:04

going what do I do with myself so I go

00:56:06

find CC Sabathia and I'm like no one's

00:56:08

talking to him he's which they were I

00:56:11

talked to him and I'm trying to explain

00:56:12

the opener to CC Sabathia did you tell

00:56:14

us remind us what the opener so the

00:56:15

opener is an idea that I've talked about

00:56:17

on the show for years is that the

00:56:19

baseball team can get an advantage by

00:56:20

beginning the game with one of their

00:56:22

good very good relief pitchers for

00:56:26

influence

00:56:27

perhaps when he was at Toronto not

00:56:30

Toronto Tampa Bay Rays I talked to him

00:56:32

just they were the first team to really

00:56:33

do it but I was trying to evangelize on

00:56:35

it and I said to Cece you know you

00:56:37

should be in favor of the opener and

00:56:39

he's like no we're starters we we're

00:56:41

we're trained to start the game and he's

00:56:43

an imposing human being and and it's

00:56:46

just a ridiculous idea that I would

00:56:47

start the game um in anything other than

00:56:50

the first inning and I said well you

00:56:51

know has some advantage and I start to

00:56:53

explain them and I one of the things I

00:56:54

said was you'll have an opport

00:56:56

opportunity to get the win without

00:56:58

having to pitch five innings and he just

00:56:59

without even missing a beat he says I

00:57:02

thought you stat heads don't value

00:57:06

wins that is

00:57:09

awesome there's so much in this story

00:57:11

absolutely incredible um okay last

00:57:13

lightning round question Eric you

00:57:15

mentioned the less popular sports like

00:57:17

golf but what about non sports are there

00:57:20

the pandemic we do sometimes wander off

00:57:23

the playing fields into other areas any

00:57:25

favorite non sport moment and to give

00:57:28

yall a moment to think about I'll tell

00:57:29

you I'll tell you my favorite it's the

00:57:31

Moneyball for fire guys the forest fire

00:57:33

guys we we just them well we

00:57:36

hypothesized that they must this was the

00:57:39

summer of I think it might have been

00:57:42

20120 and bad forest fires in the

00:57:44

American Northwest and we thought

00:57:46

someone is running analytics on that and

00:57:47

we asked Matt to run it down and he's

00:57:50

like shed I found these guys who were

00:57:52

publishing papers one of which is called

00:57:53

Moneyball for fire and now we have a

00:57:56

multi-year relationship with these guys

00:57:57

we've had on the the show multiple times

00:57:59

and they fight the same battles the

00:58:02

exact same culture battles that we see

00:58:05

fought in football organizations

00:58:07

basketball organizations baseball

00:58:08

organizations it's incredible yeah I

00:58:10

mean talk about a cool spatial temporal

00:58:12

data situation that basically pops up

00:58:14

every year and links to so much of the

00:58:16

kind of climatic things that are going

00:58:18

on right now as well it's one of my

00:58:21

favorite non non-sports topics was the

00:58:23

traffic engineer we had on years years

00:58:26

ago right before Thanksgiving you may

00:58:28

not remember this I don't think I was on

00:58:29

this show you may not have been on it I

00:58:30

wasn't alone I know there was a few of

00:58:32

us there remember um and he was talking

00:58:35

about money essentially Moneyball for

00:58:38

Designing highways and one of his

00:58:40

comments was that we in America don't do

00:58:43

things properly we just build wider

00:58:45

roads when we want to accommodate more

00:58:47

traffic but that can be accomplished by

00:58:49

learning how to

00:58:51

drive really we just don't drive

00:58:53

properly huh and he explained the

00:58:55

various ways in principle he was saying

00:58:58

stay out of the left lane unless you're

00:59:00

passing and how much trouble that causes

00:59:02

and and he explained how the models show

00:59:04

how much how your roads become

00:59:06

inefficient when people drive improperly

00:59:08

and he was of course saying but in

00:59:10

Germany where he was a German engineer

00:59:12

we don't build wide roads we teach

00:59:14

people to drive properly speaking of

00:59:16

Germans we have the DI the diet

00:59:19

researcher out of Duke whose name I'm

00:59:21

going to forget but he's given us some

00:59:23

great stuff over the years on calories

00:59:26

really good stuff all right we need to

00:59:28

wrap up and so I want to ask one last

00:59:30

question we've been talking about the

00:59:31

last 10 years of course and

00:59:33

appropriately hopefully we'll do another

00:59:35

10 years and More in front of us what is

00:59:38

one hope you have or ambition you have

00:59:42

for the show over the next 10 years I

00:59:45

think it's going to be really cool five

00:59:46

six seven years from now when we all

00:59:48

have Hall of Fame votes and we can on

00:59:50

the air we argue about the Hall of Fame

00:59:53

for weeks every year won't it be great

00:59:55

as voting members to do that I'd like us

00:59:59

to do an offsite at the hall of fame I

01:00:01

mean I'm saying that self Baseball Hall

01:00:02

of Fame but a matter of fact we could do

01:00:04

all of them I mean actually I have you

01:00:05

can actually do the baseball basketball

01:00:09

Football Hall of Fame in one swing I

01:00:11

mean there's not that far from each

01:00:12

other the major sports I don't even know

01:00:14

where's the Hockey Hall of Fame but

01:00:16

Toronto all right well it's not it's not

01:00:18

totally out of the question sounds

01:00:20

either way um I have such a passion for

01:00:22

the history of baseball especially and

01:00:24

the Baseball Hall of Fame that I think

01:00:26

it would be great to do a show from

01:00:28

there and I'm confident that we could

01:00:29

get a number of Hall of Famers to come

01:00:31

on the air and I'm sure they would talk

01:00:33

to us about the role that analytics

01:00:35

played in their careers and it would be

01:00:36

great to get a spectrum you know getting

01:00:38

someone whose major career was in the

01:00:40

70s and 80s in the pre-analytics ER then

01:00:42

get someone in the 90s and 2000s when it

01:00:44

was emerging and then get even the most

01:00:46

recent Hall of Famers that would be a

01:00:48

ton of fun yeah I mean I sorry to just

01:00:50

piggyback on that like I I I as I was

01:00:52

sort of thinking about the the the last

01:00:54

10 years I feel like we we do kind of

01:00:57

you know because we've been doing this a

01:00:59

while I think we have a historical

01:01:01

perspective things on things I would

01:01:02

like this to do even more of a kind of

01:01:04

like you know this year like you know 10

01:01:06

years ago what was going on both in like

01:01:08

sports itself and also in analytics you

01:01:10

know I think so many you know it the the

01:01:13

field has moved so quickly in say the

01:01:15

last 20 years that you know kind of like

01:01:18

you know we're the last keepers of the

01:01:20

generation without internet or whatever

01:01:22

we have we we have an obligation to

01:01:24

remind people what it was like and where

01:01:25

we Shane what we may find out in the

01:01:27

next 10 years is you know Audi was

01:01:29

talking about you know running more

01:01:31

sophisticated models maybe artificial

01:01:33

intelligence and these large language

01:01:35

models are going to change the way we do

01:01:37

even Sports analytics like maybe we're

01:01:39

going to be able to just upload a

01:01:40

massive amount of video and other stuff

01:01:43

and just say you know run you know

01:01:46

compress this using a Vari variation

01:01:49

encoder and just run a large just run

01:01:52

some analysis on this and it'll get

01:01:53

automatically done I have a feeling that

01:01:55

10 years from now the the use of video

01:01:58

data and other types of data will be

01:02:00

much more prevalent in sports because of

01:02:02

all these technology and AI that's going

01:02:04

on I think that's a fascinating uh

01:02:07

prediction um I of course I have to

01:02:09

second Eric's Hall of Fame show I mean

01:02:12

we're kind of a force in this department

01:02:14

it's my obsession and and just because

01:02:17

loving baseball and baseball history in

01:02:18

particular so I'm just seconding that

01:02:20

one um but if I just say I'd like to see

01:02:22

something in the next 10 years um I'd

01:02:24

like us to be to reflect a little bit

01:02:26

about what's happened on on the sports

01:02:28

analytics side kind of uh on the

01:02:30

research end and kind of reflect on our

01:02:33

particular area which is the academic

01:02:36

contribution of of sports analytics and

01:02:39

methods and just reflect on some of the

01:02:41

great papers the great contributors

01:02:43

interview some of the scientists

01:02:44

involved we haven't done too much of

01:02:46

that um we do the new stuff we're very

01:02:48

good with new but we haven't had the

01:02:49

opportunity maybe it's my bias of

01:02:52

reflecting historically about what's

01:02:53

happened and and kind of not a Time

01:02:56

based moment but uh you know talk to

01:02:58

some of the people who are really

01:02:59

involved in in pushing our our subject

01:03:01

forward that's imminently doable right

01:03:04

that's great yeah the road trip I'll

01:03:05

pitch for is the is a combine so the

01:03:08

combine week right now in Indianapolis

01:03:11

um the base save me any time CU you know

01:03:12

I watch the whole thing

01:03:14

anyway we you have to bring your second

01:03:17

and third and fourth screens for other

01:03:18

sports that's true you got to pick up on

01:03:20

that subtle body language of being in

01:03:22

person that we've discussed already

01:03:23

though you don't get that across the TV

01:03:26

there's a baseball combine now that

01:03:27

could be interesting yeah um but I do

01:03:29

think it'd be you know we were we saw

01:03:32

this technology evolve over the last 10

01:03:34

years and we were having conversations

01:03:36

with people when it first came out I'm

01:03:37

curious can we remain on that Cutting

01:03:40

Edge and what does it take for us to

01:03:41

remain on that Cutting Edge so you just

01:03:42

mentioned Ai and you talked about you

01:03:44

know dumping a bunch of video data but

01:03:46

it may be even earlier that what AI does

01:03:50

it levels the coding playing field Y and

01:03:53

other people can get into analytics

01:03:54

without having to go through years

01:03:56

basically of coding training could be

01:03:58

that could be one of the big things are

01:04:00

we going to be doing the interviews and

01:04:02

the research and the reading to really

01:04:03

know what happens next we we we did it

01:04:06

the last 10 years we need to keep on

01:04:08

whatever that was that kept us at the

01:04:09

front edge we need to keep on doing or

01:04:10

all the companies like Pro footb Focus

01:04:12

that do a lot of human coding well once

01:04:14

we have a lot of human coded data we

01:04:16

have supervised learning we can do so

01:04:18

now we we're going to use human coders

01:04:20

but we're going to use them to train our

01:04:21

models and then we're just going to let

01:04:23

our models rip on all the dat we can

01:04:26

collect in some large automated way as a

01:04:28

matter of fact over the years we've had

01:04:29

a lot of people talk about the different

01:04:30

you know value of having cameras and

01:04:32

locations and other stuff in stadiums I

01:04:34

think the volume of especially video

01:04:36

data we're going to have to figure out a

01:04:39

way to deal with that in a large

01:04:40

automated way all right guys thanks to

01:04:44

our audience and thanks uh big thanks

01:04:46

shout out before we give uh before we

01:04:47

leave to the crew here helping us bring

01:04:50

this to life our boss man matd dats

01:04:52

overseeing even the live show Michelle

01:04:54

young standing in as co-director and

01:04:56

Deion Simkins always vital to the show

01:05:00

on behalf of my colleagues and Friends

01:05:01

longtime collaborator Shane Jensen Eric

01:05:03

bradow AI wer thank you guys it's been a

01:05:06

great 10 years hope we stick around for

01:05:07

the next 10

Badges

This episode stands out for the following:

70
Best concept / idea
60
Best overall
60
Most influential

Episode Highlights

The Changing Landscape of Sports Analytics
Discussion on how sports analytics has evolved and become more accepted in academia.
“Sports has gone from being the Black Sheep to the cutting edge.”
@ 02m 50s
May 23, 2024
The Explosion of Interest in Sports Analytics
A significant increase in interest in sports analytics among students.
“We have already 350 applicants for our program.”
@ 07m 48s
May 23, 2024
The Emergence of Themes
After years of discussion, certain themes like momentum have re-emerged in our conversations.
“It has to be what AI mentioned.”
@ 17m 10s
May 23, 2024
Reflecting on the First Show
The nerves and uncertainties of the first live show are shared, highlighting growth over time.
“I remember being nervous speaking live.”
@ 23m 20s
May 23, 2024
The Importance of Fun in Teaching
The hosts discuss how enjoyment in conversation translates to audience engagement.
“If I'm having fun, the audience is having fun.”
@ 25m 01s
May 23, 2024
The Data Constraint
The richness of data significantly impacts our estimates and models in sports analytics.
“The data isn't really that rich, which means that given data is a constraint.”
@ 34m 24s
May 23, 2024
The Importance of Humility in Data
Understanding that we often underestimate uncertainty in our estimates is crucial.
“You have to have some humility about what we don't know.”
@ 34m 51s
May 23, 2024
Lessons from COVID-19 Modeling
The pandemic taught us about the limitations of models and the need for humility in predictions.
“It was just an incredible humbling lesson.”
@ 40m 59s
May 23, 2024
Accepting Uncertainty in Golf
A profound discussion on how golfers must balance confidence with the acceptance of uncertainty.
“You got to have confidence but accept uncertainty.”
@ 50m 10s
May 23, 2024
Annie Duke on Poker Strategy
Annie Duke shares insights on the importance of reading opponents quickly in poker.
“You don’t have time to wait and watch.”
@ 51m 04s
May 23, 2024
Eric Eager's Pandemic Prediction
A memorable offsite moment where Eric Eager discusses pandemic models before COVID-19.
“We should be talking to Eric Eager.”
@ 54m 07s
May 23, 2024
CC Sabathia on the Opener Strategy
A humorous exchange with CC Sabathia about the traditional starter role in baseball.
“I thought you stat heads don’t value wins!”
@ 57m 02s
May 23, 2024

Episode Quotes

The divide between academia and practice has shrunk.
Wharton Moneyball Podcast – 10-Year Anniversary Episode
The cumulative effect of doing this show is amazing.
Wharton Moneyball Podcast – 10-Year Anniversary Episode
If I'm having fun, the audience is having fun.
Wharton Moneyball Podcast – 10-Year Anniversary Episode
It was just an incredible humbling lesson.
Wharton Moneyball Podcast – 10-Year Anniversary Episode
You got to have confidence but accept uncertainty.
Wharton Moneyball Podcast – 10-Year Anniversary Episode
I thought you stat heads don’t value wins!
Wharton Moneyball Podcast – 10-Year Anniversary Episode

Key Moments

10-Year Reflection00:29
Academic Acceptance02:50
Growing Interest07:48
First Show Nerves23:20
Fun in Teaching25:01
Data Constraints34:24
Humility in Estimates34:51
CC Sabathia's Response57:02

Words per Minute Over Time

Vibes Breakdown

How AI and Analytics Are Changing Quarterback Evaluation and NFL Outcomes

January 08, 2026

01:07:55

How AI and Analytics Are Changing Quarterback Evaluation and NFL Outcomes

NBA Analytics, Tanking, and the Future of Team Building

February 19, 2026

01:04:12

NBA Analytics, Tanking, and the Future of Team Building

Hockey Analytics, Simulation, and Predictive Limits

April 22, 2026

59:22

Hockey Analytics, Simulation, and Predictive Limits

Baseball Analytics, NFL Parity, and College Football Playoff Odds

November 16, 2025

01:01:01

Baseball Analytics, NFL Parity, and College Football Playoff Odds

The 2025 NFL Season Through Data: DVOA, Analytics, Awards, and Predictions

January 05, 2026

46:23

The 2025 NFL Season Through Data: DVOA, Analytics, Awards, and Predictions

How Analytics and New Rules Are Changing Baseball

March 05, 2026

01:01:15

How Analytics and New Rules Are Changing Baseball

Inside the National Baseball Hall of Fame and Museum’s Evolving Mission

February 26, 2026

01:15:04

Inside the National Baseball Hall of Fame and Museum’s Evolving Mission

NFL Week 3 Analytics Insights and MLB Home Run Chase & Playoff Outlook

September 18, 2025

01:01:26

NFL Week 3 Analytics Insights and MLB Home Run Chase & Playoff Outlook

The Many Meanings of Baseball: History, Data, and Fan Experience

April 02, 2026

56:14

The Many Meanings of Baseball: History, Data, and Fan Experience

The Math Behind Sports Rankings and Golf Analytics

May 07, 2026

01:08:01

The Math Behind Sports Rankings and Golf Analytics

Understanding NFL Performance Variance Through Predictive Metrics

December 06, 2025

57:03

Understanding NFL Performance Variance Through Predictive Metrics

Baseball’s Hall of Fame Debate Is Changing

May 27, 2026

59:35

Baseball’s Hall of Fame Debate Is Changing