00:00:02
our guests today are Adam gr and Kade Massie who lead Barton's people analytics initiative gentlemen welcome
00:00:09
to knowledge at Wharton thanks thanks for having us so Kate we when we spoke last year about people analytics that
00:00:16
was just before your conference and you have another conference coming up uh and
00:00:21
it seems to me that during this past year that interest in people analytics has really gone up why is people
00:00:27
analytics so hot well I agree with you that it does seem to be blowing up a little bit um it started I feel like it
00:00:34
started in technology more than any other industry and then Finance picked it up and now we see it kind of
00:00:39
everywhere um my sense is that people appreciate that this is a very important function and yet hasn't been approached
00:00:46
very in a very sophisticated way in the past and all of a sudden they realize you can use all these tools that we're
00:00:52
accustomed to using in marketing or Finance we can use them for hiring people and compensating people and
00:00:59
what's better than that given how how um important those things are to an organization so it's the it's the
00:01:03
potential people seeing the potential of this that's making it popular Adam what
00:01:08
are your thoughts on why it's becoming so popular yeah I I didn't even know it
00:01:13
existed until Kate and I ended up working with Google about five years ago and they had built this whole people
00:01:17
analytics team um that was a mix of traditional HR folks Consultants engineers and and folks like us who
00:01:23
study organizational behavior and it was amazing that they were able to take questions that used to be answered based
00:01:29
on intuition and actually run experiments and gather data to figure out what were the right choices to make
00:01:35
and you know I think that Google's gotten a ton of press for all the great work they've done in this area and other
00:01:40
leaders have started thinking why aren't we doing this shouldn't we be making all
00:01:43
of our important decisions based on evidence too yeah should we just be giving glasa Bach credit here because
00:01:47
he's been like an evangelist for this surprisingly he's been out there he's
00:01:50
the head of HR at Google and he's always been very willing to say hey this is
00:01:54
what we're doing this is what you should be doing kind of uncharacteristic for
00:01:57
someone to say that essentially to his competitors so one of the things I find very interesting about people analytics
00:02:03
is as as you said Alam how a lot of decisions HR decisions that used to be based on intuition are now sort of
00:02:10
migrating over into datadriven decisions can you give me some examples of say if
00:02:15
you take hiring as the first uh point of Engagement between a company and an employee how is hiring changing because
00:02:24
of people analytics so uh people are very interested now and can we can we identify from objective measures who's
00:02:34
going to work well in our firm so rather than having to bring them in here and talk to them in person can we grab you
00:02:41
know their their GPA in college and where they went to school and who they worked for and predict somehow from
00:02:47
these inputs how they're going to be if you could that'd be great right because
00:02:51
you save a lot of time you can process all these information all these applications real efficiently so that's
00:02:57
pretty promising at least that has a lot of appeal now there's more appeal to to
00:03:02
it than substance right now because it's really hard to do but there that would
00:03:06
be great if you can pull it off and it would it would it would be great um any investment you can make in making
00:03:12
that happen is going to have high returns but it's still new so there's no
00:03:17
there there's no silver bullets um but people are drawn to it because it could
00:03:21
be great if it happens So based on the evidence does it matter where you went to
00:03:26
school wow this is like a question that people with high school age kids are forever asking right they're like does
00:03:33
it matter if they go to Princeton versus the state school what do is there good evidence of that I don't know there's
00:03:38
good evidence on that right now I mean I I think it's a debate that continues to
00:03:40
rage um but my my favorite research on this uh is Caroline Hawkes and if I remember correctly what she shows is
00:03:47
that it's mostly selection effect so that if you you know if you come out of
00:03:51
an Ivy League school um you typically will end up with a higher income and more job opportunities uh but all of the
00:03:57
characteristics that led you there were visible before University chose you and it was essentially you know bringing in
00:04:03
an ambitious and talented group of students to begin with as opposed to something magical that happened in your
00:04:08
four years um that being said um we like to think that we're doing something of
00:04:13
use ex magical happens here at Wharton NBA magical happens um but you know that the what I what I like to say is
00:04:24
that probably Great Schools provide an advantage through the Network that people get to know maybe through the
00:04:32
training but there's it's kind of like stereotypes there there may be true
00:04:35
differences in social categories between people but people believe these differences are bigger than they
00:04:39
actually are and there's huge variation within the category so there's huge
00:04:43
variation in outcomes from people who come and go to pen and there's huge variations and outcomes from people who
00:04:48
go to pen State and there's a lot more overlap between these two things than
00:04:52
people think got so so so let's say the hiring step is over and now you on to
00:04:58
this uh next step of onboarding people what can uh people analytics do to improve the onboarding process so you
00:05:06
know we need to be careful not just you know uh law Google at every turn but they are an organization that has looked
00:05:12
at this explicitly and they did it in the best possible way they ran experiments on let's on board in this
00:05:18
way and then manipulate in a different group in a different way to see what actually makes a difference so it's not
00:05:23
just telling stories we're actually doing science here to figure out what makes the most difference and I I don't
00:05:28
I don't know all the details the study this is something they've been doing for
00:05:31
the last couple of years but they recognized the onboarding process as kind of really important and completely
00:05:35
underexplored so they went out and ran and and uh and experiment do you remember the details of this there
00:05:40
there's one finding that jumps out at me which is um when they they looked at all
00:05:44
the things that make a difference in the first few days or first few weeks of your time at Google um probably the most
00:05:48
critical thing to happen is just that you meet your manager on day one and people are busy right um You may have a
00:05:54
lot of direct reports um and maybe also as a new hire that you're being sent to
00:05:58
a lot of different places um but based on that evidence they they said look one of one of our rules for onboarding is
00:06:03
you've got to meet your manager on day one um and that's such a critical part
00:06:06
of building a bond between an employee and an employer um it's not to say that
00:06:10
we didn't know it was important for you to meet your manager but I think all of
00:06:13
us right myself included really underestimated how much of an effect that would have on day one and so I
00:06:18
think that's that's an example of the kind of thing we've learned from their
00:06:20
work something else I talk about a little bit that seems wise is that a person's success at a company often
00:06:27
depends heavily on who they work for and yet who they work for is essentially completely outside their control and so
00:06:33
this isn't exactly on boarding but it is early career early stage consideration
00:06:37
if things aren't working out you need to be wise about it and not just blame that
00:06:41
person and you probably want to see that person in more than one circumstance before you draw a too strong a
00:06:45
conclusion about this is just they're both examples of being more systematic
00:06:49
and scientific about evaluating your employees or trying to train your employees as opposed to the old school
00:06:54
kind of would do this because this is what we've always done got it so what once you start working in the company
00:07:01
increasingly we find that uh more and more of us are working in teams uh and teams are very often geographically
00:07:09
dispersed very often across different countries uh but also across Generations so you have Baby Boomers and Millennials
00:07:15
sort of having to figure out how to work together what can we learn through people analytics about the creation and
00:07:23
construction of high performance teams MH mhm so um an infinite number of things and this is such a rich area and
00:07:32
Adam I'm sure has his favorite examples one of the ones that I first think about
00:07:36
is recent work by Chris shabri and his colleagues on um Team IQ essentially and they do some really interesting stuff
00:07:45
that looks at the productivity of teams um as a function of their individual characteristics versus what they do
00:07:53
collectively and he finds that there is something they seem to find something like team IQ that is different from the
00:08:00
some of the parts it's not that if you put all the smart people together that
00:08:04
they produce the smartest work it's that they need to be people who understand
00:08:07
how to work with each other and work as a part of a team there's something unique about Team level intelligence
00:08:12
that's different from the sum of the individual level intelligence and I I think to build on
00:08:17
that there are probably ways that you can compose a team to enhance the likelihood right that that they'll be
00:08:22
intelligent as a group so one of the things I hear a lot is that diversity is good and there's no question that we get
00:08:28
a lot of value from diversity in terms of people bringing unique perspectives thoughts skills to the table but when
00:08:34
you look at personality research there are some characteristics on which it's
00:08:37
actually helpful to have similarity as opposed to variety so um if you look at the the data extroversion introversion
00:08:43
is the clearest trait where variety is useful um your team of whole extroverts essentially never starts working on the
00:08:49
task team of all introverts um often forget to bond and the data say that the most effective teams have a mix of the
00:08:55
two but don't necessarily stretch that into every other personality trait if
00:09:00
you look at a personality trait like agreeableness for example people who love social harmony one of the worst
00:09:05
things you can do is put them on a team with people who are extremely critical and skeptical um because the
00:09:10
disagreeable people are feeling like they have to walk on eggshells constantly meanwhile the poor agreeable
00:09:15
people they have this this catch22 of I can be really agreeable and act disagreeable like the disagreeable
00:09:20
people and then hate myself afterward or you know I can be really agreeable and then you know it doesn't doesn't quite
00:09:26
gel and so there it's actually helpful to have either similarity in personality
00:09:30
or a consistent Norm of how we're going to interact and so I think we have to be
00:09:34
really more thoughtful about composition than we have been have been in the past
00:09:38
probably and one of the things that people analytics brings to that task in general is just the inclination to study
00:09:43
it precisely and ideally to run experiments around it so again we're not just going to take conventional wisdom
00:09:49
we're not going to take you know um something that written by someone who used to run some teams we're going to
00:09:54
actually collect some data and run some experiments and figure ask these questions and figure them out
00:10:00
let's push a little bit further on the diversity uh aspect uh as you know there's been a lot of some controversial
00:10:06
news as well about uh say women and high technology companies for example uh has
00:10:13
people analytics come come up with any evidence that shows uh you know how gender roles and uh even racial roles
00:10:21
are are related to Performance I just wish we had someone who' done some writing on some gender issues if only
00:10:28
somebody don't look at me um sh Sandberg is the brains behind that operation um I uh I
00:10:37
I've learned a ton from working with with Cheryl on gender issues and um she
00:10:41
is a a wonderful researcher Maryann Cooper at Stanford who's um who's collaborated with us on on looking
00:10:46
through what did what did the data really show and I think the probably the the thing that I would say right now is
00:10:51
um there's a lot of academic research that hasn't been leveraged um so we we
00:10:55
know a lot about how to design for example Performance evaluations that actually lead people to judge
00:11:00
contributions as opposed to the person behind them um we know a lot about how to attract more female applicants in the
00:11:07
high-tech world for example um turns out the recruiter that comes and shows up matters a lot um our own Matthew Bidwell
00:11:14
here at Wharton has shown this in the finance realm that one of the reasons there are so few women in finance is
00:11:19
that they don't apply um at very high rates um they actually have a slightly
00:11:25
higher odds of getting hired because um Financial Services organizations are really trying to solve this gender
00:11:30
problem and bring in more women but where they they start to get discouraged is when a bunch of Partners show up who
00:11:35
are all men to recruit they say well I'm never going to get this job why bother
00:11:39
trying and I I just think there's a there's a pretty big gap between what
00:11:44
the social science shows on gender and what most organizations are actually doing um it's great to see the Facebooks
00:11:50
and Googles of the world really trying to make Headway on this um we've also
00:11:53
seen I think a growing number of consulting firms um make gender a big priority and their people Analytics work
00:11:59
so I know this is a big topic McKenzie is working on right now um Mercer has a whole initiative about how to create a
00:12:05
gender balanced Workforce uh that's completely data driven and I think that's only going to grow in the next
00:12:09
few years broadly how about um performance evaluations and compensation issues what does uh people analytics
00:12:18
have to say about these issues more broadly so I mean one we should be a little careful about realize we're
00:12:25
saying people land LS this and people land LS this I mean it's it's it's you
00:12:29
know it's not a terrifically well- defined space it's mainly just bringing
00:12:32
data where data hasn't been used in before and in kind of a Moneyball spirit
00:12:36
it's not taking conventional wisdom we're going to be evidence-based here
00:12:39
kind of regardless so um there there is there is an intersection though with some Fields like psychology and and the
00:12:47
biases that people bring so one of the one of the real motivating factors for for us because of the worlds we come out
00:12:53
of is we can actually improve decision- making by using these tools and performance evaluation is a classic
00:12:58
example because there have been so many biases that show up in performance evaluation so um PE it's it's giving
00:13:05
people analytics maybe too much credit because it's really just a vehicle to
00:13:08
bring in some psychology we've known about for decades but because we're
00:13:12
being more rigorous and trying to and focusing more on how we can improve this process um we're able to far it out some
00:13:18
of those biases so um in in many many many different ways so um trying to keep opinions for example independent so you
00:13:26
don't want people to judge I don't want to judge judge Adam after having had
00:13:30
heard you judge Adam we try to get the opinions independent like that we want to judge as much as possible blind so we
00:13:36
don't bring um information about a person to the table that isn't relevant
00:13:40
because we know for a fact from Decades of research that if we know things about
00:13:43
them it's impossible for us to to separate that so I think people in has just kind of help that cause we can't
00:13:49
give them too much credit because we've kind of known it but um it's been a
00:13:53
great vehicle to to ride and I think um Kate actually just touched on something that brings us full circle which is we
00:13:58
we talked about the important role Google has played and and really stimulating interest in this field but I
00:14:03
think Moneyball and the way that that Sports analytics has has taken off was probably the other Catalyst and and Kade
00:14:08
was on the the front lines of that at least what over a decade ago before I even knew it existed yeah we we sport
00:14:15
the sports professional sports has been ahead of um almost the entire non-sports
00:14:19
world in using these tools because their whole existence is the performance of these individuals and they've got better
00:14:26
data um they can see inputs they can see outputs precisely and so you can you can
00:14:32
often look at what's going on in sports around analytics now and know that 10
00:14:37
years from now those tools are going to trickle down and even just the rigor and
00:14:41
the scientific orientation of the people who use the numbers are better literally
00:14:46
because the guys who crunched numbers for baseball um 20 years ago were figuring things out so Google has come
00:14:53
up a couple of times in this conversation are there any other companies uh or organiz ganizations in
00:15:00
uh that that you have been impressed by the work they're doing in this area and
00:15:04
what other companies can learn from their experience well as soon as we open that box we got a lot a long list to go
00:15:10
down um you know across Industries and AD Adam has been very active lately with a lot of these firms but I want to start
00:15:15
out by saying there have been some firms who have focused on this for a long time
00:15:19
so like Delo and touch has had an analytics a Workforce analytics practice for a long time um some of the new folks
00:15:25
in uh in the financial services have been Goldman Sachs has take undertaken a big initiative over the last couple of
00:15:31
years um crit credit Swiss has been very interested with some full-time people but it now it's gotten now we have to
00:15:36
like really talked about everybody Johnson and Johnson is involved Adam's got a longer list I'm sure no I I'll
00:15:41
just add a few you know from from other Industries um Teach for America particularly on their their selection uh
00:15:46
or what they call Admissions um domain I mean they they've been tracking for
00:15:49
years what do they need to assess in the hiring process to figure out who's going
00:15:53
to be a star teacher and who's going to stick with us um Jet Blue has actually
00:15:57
made a lot of Headway in this area as well um they're doing work in several different domains one of my favorites is
00:16:02
recognition so how do you actually build a science of of recognition so that you
00:16:06
know when it's important to give people a sense of gratitude and appreciation um
00:16:10
do you make recognition public do you make it individual um there are all sorts of questions that the managers
00:16:15
have have basically answered on intuition for years that now you know the airline industry when there's so
00:16:21
many customer appreciation events that happen every day um is really sort of opening our eyes to how do you do this
00:16:27
more effectively the Teach for America uh is such a great example because you you know as we're talking about Google
00:16:33
and Goldman you wouldn't think necessarily go to the not for-profit world but they are the best we know on
00:16:39
the hiring I think I I was just talking about this at lunch actually that if if I could name one firm that knows the
00:16:44
most about their hiring practices it'd be those guys now they're kind of
00:16:47
ideally suited for it because they see 50 or 60,000 applications for kind of the same job and so it's this perfect
00:16:52
stream to get good at but we have learned I can tell you the Wharton um NBA admissions group has learned and
00:17:00
improved their processes because of the way Teach for America hires people absolutely they they've got a lot to
00:17:05
teach a lot of people so what's the secret ingredient of how do you how do you choose the best people to hire you
00:17:10
know you know what it is to recognize that you're never right and that you're
00:17:14
never going to be done those are the two unbelievable what Teach for America does
00:17:18
they say they say you know we say all the time we're never going to be done this is not a one-time project it's not
00:17:22
a one-year project we're never going to be done and the other thing they say
00:17:26
which is amazing is they say you know these are our metrics these are our objectives we know they're not right we
00:17:32
know they're wrong and that leads to an a continuous conversation about okay how
00:17:37
can we refine them what exactly in what way are they wrong and how can we how can we tweak them but they say we know
00:17:42
we know we're wrong that kind of humility is a great um counterbalance to analytics because you can get pretty uh
00:17:50
pretty confident about your model you can get kind of in love with your model you need the humility that says we're
00:17:54
wrong we know we're wrong humility is a good thing overall it is and I I think
00:17:58
actually it's part of the founding of of the evidence-based Management Field that
00:18:01
I think people analytics probably belongs to um Jeff Feer and Bob Sutton have have long said that if you want to
00:18:07
do analytics right you need an attitude of wisdom which is in their definition basically the willingness to act on the
00:18:13
best information you have while constantly doubting what you know and it's it's easy as Kade points out to
00:18:19
lose sight of of the doubt part um I would say though there's there's one
00:18:23
other thing from the hiring um perspective that that's been very eye- openening to me which is there a bunch
00:18:27
of data by Rick Jacob and his colleagues suggesting that the costs of a bad hire
00:18:31
are usually about triple the benefits of a good hire and I think a lot of selection is actually more about
00:18:36
screening out um than it is screening in um you're always going to have um you
00:18:41
know false positives and false negatives um but it's it's much more risky to to
00:18:45
bring in somebody that than you have to go and replace or do a bunch of cultural
00:18:49
damage repair um and that's I think where probably you want to put more of the emphasis now as if I were to sort of
00:18:56
switch gears and turn to the two of you as researchers uh what are some of the big questions that you are trying to
00:19:03
answer and what has surprised you most about what you learned so far um one of the things that I'm
00:19:09
working on right now is how to is exactly on this stuff is how to get people to be more open to um analytics
00:19:18
so explicitly so in some in some task we need to forecast what's going to happen
00:19:23
it's a market or Price or the performance of an employee and you might have some algorithm and you might have a
00:19:29
good algorithm and in most cases you need to blend that algorithm with some expert judgment it's not person or
00:19:35
computer it's best if you can blend these things and yet people are reluctant to take input from computers
00:19:41
especially the more expert you are in the field the more like you know no I need to use my you know my head as
00:19:46
opposed to that so we're trying to understand the psychology about what what leads people to resist those um
00:19:53
inputs and what can we do to help break that down one of the things that that I've gotten increasingly interested in
00:20:00
is um the the problem of of collaboration creep so to speak where we're constantly having to go to
00:20:06
meetings and answer emails and there's just massive explosion in interdependence and nobody knows how to
00:20:11
handle it everybody thinks collaboration is great uh but everybody is overwhelmed
00:20:15
with the amount of collaboration that they do um so I'm working on a project
00:20:18
with Rob cross and Reb Reb right now that that looks at uh the following question if you do a network analysis of
00:20:25
an organization and you ask people who do you depend on for critical knowledge and advice and expertise um what Rob
00:20:32
finds is there's a certain number of people that can write your name down um
00:20:37
and after that number you're at serious risk for Burnout and overload and I think that of course the unanswered
00:20:43
question is exactly where does that number fall um but it it turns out to actually be quite deadly uh to have
00:20:48
everyone depending upon you for expertise um and I'm interested in how do you redistribute um you know the help
00:20:55
um the Insight the connections uh so that it's not all bottleneck in one or two people right that's that's really
00:21:01
interesting and one final question if you were to look at people analytics and the state of knowledge of the field
00:21:07
today where are the biggest knowledge gaps and and what should be done to fill them um this I would say in in in in
00:21:20
being effective not just running better numbers but actually making change in organization so it's it's one thing to
00:21:25
have a fancy regression model or to really have some insight numerically it's a very different thing to actually
00:21:31
translate that into action and um it kind of doesn't matter how good your model is until you get good at that
00:21:37
translation and um it's kind of natural that's going to come later but right now
00:21:41
everyone's enamored with the models and the data and the analysis and it's not
00:21:45
going to matter unless they can actually persuade and change an organization I I think from my
00:21:50
perspective the probably the biggest unanswered question for people analytics uh is what Cade's working on right now
00:21:57
which is why don't more organizations do this and how can you get senior leaders
00:22:01
to realize that just because sometimes you know these these variables are hard to measure doesn't mean you shouldn't
00:22:07
bring better science to them and what does it take to to open the minds of leaders to to recognizing that if we had
00:22:13
more data it won't replace our jobs will actually give us the tools we need to
00:22:17
make better judgments great okay Adam thanks so much for speaking with knowledge at Wharton
00:22:23
and good luck with the conference thank you appreciate it [Music]