00:00:00
welcome everyone to the current edition of the Warton AI SiriusXM podcast series
00:00:06
here on artificial intelligence I'm Eric bradow Vice dean of analytics here at the Wharton School
00:00:12
also the KP Chow professor of marketing statistics and data science today's episode as all of our episodes are is
00:00:19
sponsored by analytics at Wharton and AI at Wharton and today we're going to talk
00:00:24
about a topic that I think it's hard to walk down the street or talk to anyone
00:00:28
in business and not have them speak about which is AI and Human Resources so I'm here today by two of my colleagues
00:00:35
the first is my colleague from the management department Matthew Bidwell Matthew is the Shing Jang and Y da
00:00:42
Professor a professor in the management department he's also the faculty director of a center that's a big part
00:00:48
of analytics at Wharton Wharton people analytics initiative and he's also the
00:00:52
academic director of Wharton Center for human resources program Matthew welcome here to our show thank you very much for
00:00:58
bringing me on Eric a it's great to have have you here I'm also joined by my
00:01:01
colleague Sunny Tay Sunny is an associate professor of operations information decisions at the Wharton
00:01:07
School and also teaches many of our courses on AI so Sunny welcome to the show thanks thanks for having me it's
00:01:13
great to have you both on such an important topic so let me start with the beginning Matthew maybe I'll start with
00:01:19
you um how since you're one of the you are the faculty director now of Wharton
00:01:25
people analytics um how do you think AI is going to affect the way that we manage people what are both the concerns
00:01:32
that you have and in equally importantly what are the big opportunities uh big question I mean
00:01:39
obviously we need to think a little bit about how we Define ai there's kind of
00:01:43
you know these days when we think about AI we kind of leap straight to chat GPT and large language models and so on or
00:01:50
what people a lot of people would call the generative AI part where the computer the large language model is
00:01:55
generating a response yeah um yeah if you look back historically last five years um people have used AI almost to
00:02:03
describe anything that involves numers so it's it's a broad range um yeah I
00:02:09
think with any of these Technologies as ever there's a lot of opportunities to
00:02:15
kind of improve how we manage people uh when we look at how people are managed so much of what goes on is kind of gut
00:02:22
decision making this kind of intuition and we have pretty much a century of research suggesting that our guts are
00:02:28
terrible decision makers that actually there's a reason why we should be thinking with our brains rather than our
00:02:33
stomachs um and so more broadly when we are more systematic when we are more thoughtful when we rely on data in
00:02:40
making decisions who do we hire who do we promote how do we manage people all those sorts of things we usually make
00:02:46
much better decisions and so I think the extent to which AI helps us be more systematic in doing that um it's going
00:02:54
to be really helpful it's already being helpful there are obviously big concerns
00:02:59
um I think kind of three spring to mind um so one big concern everybody has is bias
00:03:08
and discrimination um again we know there's a lot of bias in the labor market we
00:03:13
know our guts are discriminating all of the time um yeah the good news is probably most of the time AI is going to
00:03:21
be less discriminatory but we think it's going to be discriminatory um you know
00:03:25
particularly when we look at some of these more sophisticated large language models right they have been trained on
00:03:31
the Corpus of data that is out on the internet even when people aren't being
00:03:36
deliberately sexist and racist um that embodies a whole set of cultural assumptions so take sexism for example
00:03:44
there's been some very nice studies that kind of look at word embedding models
00:03:48
and other things that are trained on kind of the Corpus of text you see on the internet and they show not
00:03:53
surprisingly that we think words to do with careers are more closely related to men's names and words to do with kind of
00:04:00
home life and family more closely related to women's names and so once you start using those models to make
00:04:05
decisions about employment I think the risks of bias and discrimination are very serious um and I think one of the
00:04:12
things we worry about particularly more generally with kind of using algorithms rather than judgment in managing people
00:04:18
is if you have a manager that discriminates that's a problem for the people working for that manager if you
00:04:24
have an algorithm discriminates the fact that I can apply a hiring algorithm at scale across an entire company across an
00:04:32
entire industry the sheer volume of people that are potentially affected is huge um so I think that's one of the
00:04:39
advantages of scale and the disadvantages of scale yeah and so I think that is I think that is a very
00:04:45
live concern um if I can go on a little bit just talk about my other two concerns um I know we have many other
00:04:51
questions but I think another couple of things that we're thinking about um algorithmic algorithms like any
00:04:58
technology have often been applied in a fairly punitive way in HR um so I think the the classic example of this is
00:05:06
scheduling software um and so with scheduling software it's very tempting if you're an engineer sitting in kind of
00:05:14
your office trying to do the right thing you're like how do I increase productivity and the way I increase
00:05:18
productivity is by carefully matching people's schedules to shifts in demand
00:05:23
during the day and so I'm running Starbucks I want to give somebody a shift that starts at 7 in the morning
00:05:30
and then runs till 10: well by then we're through with the kind of office rush and then I want them to go away for
00:05:35
a while so I don't have to pay them and then maybe I want them to come back between 4 and 6 and so what you find is
00:05:42
these schedules end up creating schedules that are great for the company but have proved terribly damaging for
00:05:48
the people who actually have to try and fit their lives around what the algorithm thinks and probably frankly
00:05:54
end up causing long-term damage in the organization as well because you maximize that match between supply and
00:06:00
demand but you end up driving up attrition as people won't stay with those sorts of schedules and so I think
00:06:06
there's a broader issue I mean it's always ATT tension in managing people
00:06:09
you know how much do you take into account kind of what those people think but I think when you have people managed
00:06:15
by AI you have a bunch of assumptions that have being baked in by the schedulers by the engineers whoever they
00:06:22
are increasingly detached from what's going on in the ground and I think that
00:06:27
that often leads to some really bad and disrup Ive decisions and so I think done
00:06:31
well we can incorporate a lot of these algorithms um and you know manage people better but it does require us to really
00:06:39
think about how these algorithms are being used and have kind of that closed loop so we engineer something and then
00:06:45
we say okay what's actually happening and we're very alive to the problems
00:06:48
it's creating and go back and re-engineer it I think when you kind of just sit down do an optimization problem
00:06:54
then kind of put it out into the world and let everybody suffer the problems um that
00:07:01
creates a lot of damage too so those are some of the things I'm worrying about I
00:07:04
think one of the things that we always talk about is that you know first of all what is the objective function you're
00:07:09
optimizing that's the first thing and I think we would all agree and I'll turn
00:07:12
it over to sunny in just a second would be you know these things should be a decision support tool the minute you
00:07:19
automate them you have those dangers of you know it up you know if you'd like
00:07:23
maximizing some objective that may not be good for the employees and certainly may not be good for the firm so Sunny
00:07:30
let me turn things over to you since I know for a number of years you've been
00:07:33
one of our Pioneers in teaching AI to our students um what's changed like why
00:07:39
are things so why is everyone so excited today you know I'm a statistician I've
00:07:44
been here at Wharton for 28 years we've been doing you know kind of big data
00:07:48
science for a long time what's unique and what's changed about today that's
00:07:53
made I'm sure everybody want to take your class everybody be interested in every single thing you're working on so
00:07:59
I I think you I I think you know you touched on it just a second ago uh but this this this tension between a
00:08:06
decision support which is what a lot of technology has been doing for a few Generations now moving to this world uh
00:08:13
potentially um either recommending a decision or even automating decisions and that that's you know what we do at
00:08:20
work all day what we do is make decisions right and that's what businesses do organizations are are
00:08:24
optimized to do and so a technology that can uh Ser as that that can make decisions or recommend decisions has
00:08:32
implications uh for all parts of the organization people compare it to you know electricity has a potential to
00:08:37
change everything so I think that's one part of the uh the reason that people
00:08:41
are energized about uh this particular topic uh the other thing that's that's
00:08:46
exciting but also somewhat concerning is that I think there's more unpredictability right now around AI
00:08:52
than there has been for tools past right so we think about as we scale up these models people are seeing more emerging
00:08:58
capab capability that they would not have expected so let me press you on this just this one topic for a second so
00:09:04
I can imagine uncertainty in a few things one is um I type something into a large language model chat GPT Bing AI
00:09:12
Etc something comes out I type the same thing in maybe something doesn't the
00:09:17
same thing doesn't come out so that we could call that in the measurement literature test retest reliability
00:09:21
that's one possibility one is sunny Tay changes one word in the prompt prompt
00:09:27
engineering if you'd like something radically different comes out what form of uncertainty are you talking about or
00:09:33
maybe it's the measurement one that I think about or maybe it's a different
00:09:36
one right no for when I when I think about uncertainty in this context I'm thinking about the uh the question of
00:09:43
where these uh Technologies can add value to the jobs we do right and so if I as a person who uh designs jobs or an
00:09:51
organizational planner thinks about where it fits in uh the answer is changing in ways that I think are a
00:09:57
little bit unpredictable and so if we think about what it can do now what it can do uh tomorrow uh even the people
00:10:04
who are at the frontier of this technology find themselves quite surprised these days that we didn't
00:10:08
think it was going to be able to do that and so that kind of uncertainty um combined with the fact that it has the
00:10:14
potential to affect decisions everywhere uh have have have a lot of potential for
00:10:18
you know sort of thinking for for for change uh in ways that we don't exactly
00:10:22
know what's coming but it's is energizing in a way so Sunny maybe you could just clarify something for me and
00:10:27
for all of our listeners here on our show um the things that generative AI models can
00:10:34
do they don't I mean the algorithm itself the statistical engine itself doesn't just come up with it right I
00:10:42
mean somebody has to have programmed it to be able to do a certain type of problem it's not like it generates
00:10:50
solutions to problems it just generates let's say you wanted to you wanted your
00:10:55
AI engine to make some decision about how to optimally schedule something which Matthew said somebody a programmer
00:11:04
somewhere had to have said this is a problem this AI engine should solve it's
00:11:09
not like the AI engine searched around the world and said let's solve time card
00:11:14
scheduling problems the algorithm doesn't decide the problems humans help the algorithm decide which problems to
00:11:20
solve right humans on the input side absolutely do tell it do help it understand what problems to solve at the
00:11:26
same time they're incredibly general purpose so what capable and flexible enough to solve is is quite quite
00:11:32
impressive yeah I know Matthew you wanted to jump in here and talk about kind of this this idea of kind of the
00:11:38
breath of problems and what the future might be I mean well I think kind of on the uncertainty piece I think it's it's
00:11:44
very interesting I mean Sunny knows much more about this than I do just for anybody listening at home so I'm kind of
00:11:50
mainly curious to hear what he thinks but I mean STS me a lot of the uncertainty is is how good they're going
00:11:56
to get how quickly I mean I think you know you said and we're all talking about it and everyone wants to talk
00:12:01
about it it's partly because we're all so surprised by kind of just the leap in
00:12:08
capabilities over the last year um of these models kind of just blowing through the touring test in a way that
00:12:14
we thought was a long time off but the big question is have we now reached another kind of plateau where you know
00:12:22
at which point you kind of say these are neat tricks and they can do some things
00:12:27
quite well but the lack of accuracy the hallucinations all of those sorts of things are we ready to turn over large
00:12:36
processes to them wholesale I'm not sure or they're really going to improve and
00:12:40
the thing it makes me think about is self-driving cars I mean I remember like s or eight years ago when we were
00:12:45
thinking about buying our next car I thought this is the last car I'll ever buy because you know by the time I'm
00:12:51
ready to buy another car all cars will be self-driving you know there will be no steering wheels why would we have
00:12:57
them and it turned out you could get like 90% of the way but that last 10% proved really hard and so
00:13:05
the question in my mind is great Point could we see something similar or do we think really they are going to keep
00:13:11
improving at this rate yeah no absolutely I 100% agree I the challenges with all these tools right is are that
00:13:17
and you mentioned this I think a little bit when you talked about bias discrimination and managers is that all
00:13:21
these tools are embedded in a context where we have I don't know 200 250 years
00:13:26
of infrastructure about what to do when a manager gets it wrong right we understand how to deal with human
00:13:31
decision error anytime you're talking about putting one of these tools in place which include self-driving cars it
00:13:37
includes large language models and it gets it wrong we don't have that legal
00:13:41
organizational infrastructure and that has been an absolute uh has been a constraint and I I suspect it will
00:13:46
continue to be so we may well have hit a plateau it's just that whenever I think
00:13:50
about what the future is going to bring with these tools you know I think this is if I think about the history of
00:13:54
science a little bit this is uh somewhat rare in that you have a situation where
00:13:58
where you have a tool that can do certain things and the scientists are not trying to figure out how it works
00:14:02
right that hasn't happened very often in history and so this uncertainty is what
00:14:06
I I tend to tend to caveat some of those comments because of that uncertainty which is new I think when you compare it
00:14:11
to other technological innovation so let me ask you both next about what I'll
00:14:15
call application areas that you think are let's be positive people are about
00:14:20
extremely positive so for example one area that Matthew you already mentioned was about hiring let me let me ask you
00:14:26
the following if the following would be a good example or a bad example so I'm
00:14:30
even going back to my days so prior to coming to Wharton you two may not know this I was at the educational testing
00:14:36
service in Princeton and we were working on automated scoring algorithms for essays long time ago people may not have
00:14:43
called it AI but we were ingesting the words and trying to construct scores but not for the purpose that when Matthew
00:14:50
Bidwell takes the SAT that's the score he's going to get but how do I use
00:14:55
humans in a most efficient way when I have millions of essays to score how can I use an engine to basically do a first
00:15:02
pass algorithm and then humans will come in on the really tough ones so let's
00:15:06
take hiring as an example but you use any example you want why can't I use an
00:15:11
AI engine to do a first pass algorithm to look at a thousand résumés that I get for a job 950 get pruned off by the
00:15:19
algorithm and then the 50 that seem to have the credentials that match what I want I then use humans and intervention
00:15:27
to go in so do you have any concerns about that two-step process now bias and discrimination Can Happen by the
00:15:32
algorithm so maybe there's some in that 950 but what do you think about that and
00:15:36
also if you have a better example than mine I'm sure our listeners would love
00:15:39
to hear no that's great I would totally do that um so and I actually I mean there' have been some experiments with
00:15:46
this um generally they've worked out reasonably well I always say it's not
00:15:51
some I'm it's not so much that I'm a huge AI Optimist I'm just human skeptic
00:15:56
um so I mean think the challeng is when you look at people how people actually make hariring decisions it's so
00:16:01
haphazard that hariring actually I think is one of the places where this tends to
00:16:06
work really well now I mean we can get back to that question about kind of is it decision support is it actually
00:16:13
making the decision I mean one thing just to be aware of is I think in practice that is a very fine line I
00:16:20
think most of the evidence is people tend to do what they're told um and so frankly if the algorithm says
00:16:28
you should hire this person most of the time that's what they're going to do and
00:16:32
so you can't you know kind of when we say well there's a human in the loop so
00:16:36
it's okay yeah but are they really I mean once they've been told this is the
00:16:40
right way to do it they're going to largely follow the advice but yes I'm
00:16:45
actually I I think that is one of the better places I think there are concerns about bias um my guess is in the vast
00:16:53
majority of these cases the bias of the AI is still orders of magnitude less than the biases of the human rers so I'm
00:17:00
I'm reasonably bullish on that as a case so Sunny what's your thought on both the
00:17:04
example I gave Matthew just talked about and maybe what do you think is the most
00:17:08
promising area where AI can have a transformative effect on human resources today I want to underscore and pick up
00:17:14
on something you said which is Let's Be You Know positive people I think the uh
00:17:17
framing of the conversation uh in the in the broader uh the Press uh maybe has been too much uh Zero Sum with employers
00:17:26
and and employees or or manag and and workers and and and there may certainly be some of that but there uh
00:17:32
it seems that there's a lot of opportunity to use these tools in a way that um enhances employee experience
00:17:39
enhances employee well-being I was talking to an executive last week um that's using generative AI to write
00:17:46
performance reviews and first of all it saves them a ton of time but the second thing is that they're able to use that
00:17:51
excess time to do one-on-one mentoring and coaching uh so there are a lot of opportunities to maybe improve and they
00:17:57
also by the way uh provide more frequent performance reviews so it's almost on a
00:18:01
monthly basis instead of by anual so lots of opportunities um where where we can think about um the employee you know
00:18:08
a lot of these tools what they're doing is they're taking parts of the work that
00:18:11
we may not enjoy so much there so definitely there are definitely uh places where we can think about using AI
00:18:17
um at least as a first order uh first order ad first order application uh to think about how how can work be better
00:18:24
how can we do a higher touch better job and making sure our employees stick around are happy and are being
00:18:29
productive so we're here on the Wharton SiriusXM AI series uh on talking about
00:18:35
artificial intelligence this is sponsored by analytics at Wharton and AI at Wharton and again I'm joined today by
00:18:40
my colleagues Matthew Bidwell of the management department and sunny Tay of our operations information and decisions
00:18:45
department so Matthew let me just ask you um one of the things I love doing with my MBA students actually I've been
00:18:51
doing this for years now is I always start out one of the lectures and I say you know AI in that case machine
00:18:58
learning learning is coming for your job which kind of Industries or which areas
00:19:04
of the workforce do you see you know in some sense if you were advising our mbas
00:19:09
or undergrads like I don't know this seems like a pretty risky area to invest
00:19:14
in today as a career any particular jump out in you and thinking like wow I don't
00:19:19
know like for example I'll pick my home Department if I was in the Creative Marketing business today coming up with
00:19:25
advertisements I'd be thinking I don't know seems like AI engines could do a
00:19:31
pretty good job of you know coming up with a massive combination of features of ads that seem to be effective I'll
00:19:38
just pick one from my home Department of marketing any anything come to your mind
00:19:42
I'm nervous about this I mean i' I've chatted with sunny about this before
00:19:45
we've had kind of two decades of people making predictions about what work is going to go away based on AI and
00:19:55
in retrospect they've mainly been hilariously wrong um so I kind of feel like this is this an area that's very
00:20:02
hard to protect we've recently seen these things saying you know when we look at which jobs are going to be most
00:20:07
affected by these kind of new AI Technologies like things like English teachers are at the top of the list and
00:20:12
you're just like no no I mean if I think about which jobs are likely to be safest
00:20:17
from AI I cannot see chat GPT maintaining control in a class of 14y olds it's just not going to happen so I
00:20:24
think it's very hard I mean we are seeing I mean you mentioned kind of creative I think uh freelance graphic
00:20:29
designers have already really taken a big hit um so we're seeing some jobs um
00:20:36
essay Mills so if you made your money by ghost writing papers for college students I've got really bad news for
00:20:42
you um stack Overflow has just been laying off people kind of providing advice with
00:20:47
programs but we're see we're seeing kind of these narrow kind of slightly strange
00:20:52
kind of niches getting wiped out but I'm not yeah I'm nervous about making big
00:20:57
predic about what's going to what's going to be affective I'm with Matthew on this I'm
00:21:02
I'm relatively uh optimistic in the sense that uh we certainly you might expect to see some verticals uh affected
00:21:11
quite a bit maybe maybe customer service operations customer basing operations you know but that's been sort of true of
00:21:17
of tractors and zeros copers and everything in between by and large the evidence seems to be saying you know for
00:21:23
large language models for uh for example that all of us are going to be using them to some degree will make us a
00:21:29
little bit more productive the productivity gains will be uh promising but it'll be gradual and so uh we'll
00:21:35
able to be maybe be able to get rid of some parts of our job we don't like become a little bit more productive and
00:21:40
any job loss will be at a pace that the economy hopefully will be able to absorb
00:21:44
it without any problems yeah I mean I do think if You' been if you'd predicted
00:21:48
when the internet came in that one of the occupations that would be worst affected was journalists you know we'd
00:21:53
have asked you to show your working right I mean there are kind of it it's quite unpredictable how these things
00:21:58
play out so another question I'm sure our listeners here on SiriusXM uh and our podcast would like to know about is
00:22:05
how are you to as Educators using it in your own classes you like for example are you going to allow students to
00:22:12
submit assignments using generative AI are you going to encourage its use are you going to take certain parts of the
00:22:18
material that you're teaching students and say actually you'd be better off
00:22:21
just learning it through a generative AI engine so Matthew I'll start with you
00:22:25
and then I'll go to Sunny who you know one could argue your entire course is
00:22:29
about this so I'd like to start with you Matthew how are you going to use it in
00:22:31
the courses you teach um it's still a work in progress I have to say I mean so
00:22:36
I teach class on people analytics part of that I get people to um analyze data sets um you know I've tried throwing my
00:22:44
problem sets into chat GPT it's made some fairly Elementary errors which has
00:22:50
made me reassured that it's not going to make me completely redundant but I think
00:22:54
I'll be encouraging my students that that is a way to to work on it but that
00:22:59
they need to understand what the answers are that just expecting chat GPT to get
00:23:04
it right is going to lead them astray I think it's it's a big problem I think
00:23:10
for us though I mean as a kind of particularly saying the management department you know a lot of the way
00:23:16
kind of in the social sciences we have tended to evaluate people and get them to learn is go write a paper and it's
00:23:22
going to take us a while to figure out how to redo pedagogy when that is just so easy just to kind of get an AI to do
00:23:30
so it it we are one of the industries actually that I think is most affected in in some ways and we're not going to
00:23:35
lose our jobs but we're going to really have to I hope but we're really going to
00:23:39
have to change how we do what we do and sunny both in your answer I'd love to
00:23:42
hear since I know there's a at least in one of the two courses I'm well aware of
00:23:46
that you teach there's actually a significant coding portion so if you could talk you know I've heard some
00:23:51
people say it doesn't matter therefore whether you know r or python because you
00:23:56
can do the conversion back and forth therefore it's just you got to be able to program in something and we'll let
00:24:00
chat GPT do the rest but how how are you thinking about it yeah no so absolutely
00:24:05
so so I'm I'm somewhat fortunate in a way because Ai and analytics are so
00:24:09
Central to the courses I teach that these questions are I can just move them directly to the center you know and I
00:24:15
think what you said is absolutely right it's first order these days for students
00:24:18
to understand how you think about a coding workflow that involves large language models right so what changes
00:24:25
where does the time go how much coding do you need to be able to know to use this effectively these are all questions
00:24:30
we don't quite know the answer to but I think belong in the center of these types of courses and then another course
00:24:35
I teach on AI uh asked some of the bigger questions the questions Matthew raised bias ethics those sorts of things
00:24:41
that we're just not quite prepared for yet but that managers are absolutely going to have to deal with over the next
00:24:45
two decades or so uh those are also Central to uh how we how we how we spend our our class time and the the uh
00:24:52
there's just there's there's there's so many emerging questions every year new
00:24:55
questions it's been um I mean there's there never enough time so maybe just in
00:24:59
the last minute or so that we have I'll ask you each for a 15-second answer so
00:25:02
I'm an employee what do I need to know about AI that's going to help me do my job better
00:25:08
like what's the one thing I should know how to do as an employee Matthew any
00:25:12
thoughts um experiment I think basically just try things play with the technology
00:25:19
get online and see where it can take over parts of your job and make you more effective sunny I would say Embrace uh
00:25:25
we prepared to embrace change we're just during a period where I think the way we
00:25:29
do uh functions and operations and business processes is going to start to change quite rapidly and from an
00:25:34
employees pect perspective I think just mentally they should have that mindset that I should I need to uh stay on top
00:25:39
of how these things are changing well on behalf of analytics at Warton and AI at
00:25:43
Warton um I'd like to thank my colleagues Matthew Bidwell and sunny Tay for our episode here on AI and Human
00:25:49
Resources thank you for joining us thanks s thank you oh