00:00:01
our guest today is Karthik hosonaga a
00:00:03
professor of technology digital business
00:00:06
and marketing at Wharton and we are
00:00:09
speaking with him about his recent book
00:00:10
titled a human's guide to machine
00:00:13
intelligence how algorithms are shaping
00:00:16
our lives and how we can stay in control
00:00:19
Karthik welcome to knowledge at Wharton
00:00:21
thank you so much for speaking with us
00:00:23
today
00:00:23
well cool thanks for having me it's
00:00:25
always a pleasure to talk to you and the
00:00:27
knowledge is important GRU thanks
00:00:29
so these days there's a growing buzz
00:00:31
about artificial intelligence and
00:00:33
machine learning and quite a few books I
00:00:36
have have come out recently on these
00:00:37
topics in all the conversations that are
00:00:40
going on what do you think are some of
00:00:43
the points that are being overlooked are
00:00:45
not sufficiently emphasized and how does
00:00:48
your book seek to fill that gap yeah
00:00:52
clearly there's a lot of buzz around AI
00:00:54
and machine learning which is a subfield
00:00:57
of AI I think the conversation tends to
00:01:02
you know either glorify the technology
00:01:07
or in many instances lately create a lot
00:01:11
of fear-mongering around it and you know
00:01:15
I don't think the conversation has
00:01:17
focused on you know what's the solution
00:01:21
how are we going to work with AI and
00:01:24
especially in the context of making
00:01:25
decisions and so my book is focused on
00:01:29
making decisions through intelligent
00:01:31
algorithms and certainly we have various
00:01:34
kinds of AI but one of the core
00:01:36
questions when it comes to AI is are we
00:01:39
going to use AI to make decisions if so
00:01:42
are we going to use it in a decision
00:01:43
support way are we going to have the AI
00:01:45
make decisions autonomously if so what
00:01:48
can go wrong what can go well and how do
00:01:51
we manage this because we know AI has a
00:01:54
lot of potential but I think there will
00:01:56
be some growing pains on our way there
00:01:58
and so those growing pains is what I
00:02:00
focus on how can algorithmic decisions
00:02:03
go wrong and how do we make sure that we
00:02:06
have control over the narrative of how
00:02:08
technology impacts the decisions that
00:02:11
are made for us or about us
00:02:14
I'd love to come back to the part of our
00:02:16
decision-making an algorithmic
00:02:18
algorithmic decision-making but I really
00:02:22
love the way you began the book with
00:02:23
some very striking examples about
00:02:26
chatbots and and how they interact with
00:02:29
humans I wonder if you could use that
00:02:32
illustration to talk a little bit about
00:02:34
how human beings interact with
00:02:37
algorithms and what some of the
00:02:38
implications are I began the book with a
00:02:41
description of Microsoft's experience
00:02:44
with a chat part called Xiao Bing in
00:02:47
China it's called Xiao Bing and
00:02:48
elsewhere in the world it's called Xiao
00:02:50
Weis this was a chat board created in
00:02:53
the avatar of a teenage girl and it's
00:02:58
meant to engaged in engage in fun
00:03:01
playful conversations with young adults
00:03:03
and teenagers and this chat pot has
00:03:07
about 40 million followers in China and
00:03:11
the reports say that roughly a quarter
00:03:14
of those followers have said I love you
00:03:16
to Xiao Weis so that's the kind of
00:03:18
affection and following Xiao Weis has so
00:03:21
inspired by the success of Xiao Weis in
00:03:23
China Microsoft decided to test a
00:03:25
similar chat bot in the US and they
00:03:28
created a chat bot in English which
00:03:31
would engage again in fun playful
00:03:32
conversations and targeted once again at
00:03:35
young adults and teenagers they launched
00:03:38
it on twitter under the name Tay ta y
00:03:41
and this chat BOTS experience was
00:03:47
actually very different and it's it was
00:03:48
very short-lived experience as well
00:03:50
because within an hour of launching the
00:03:52
chat pod turned sexist racist fascist it
00:03:57
tweeted very offensively and said things
00:04:00
like Hitler was right and Microsoft shut
00:04:04
it down within 24 hours and later that
00:04:06
year mi t--'s Technology Review rated
00:04:09
Microsoft Day as the worst technology of
00:04:12
the year and that incident you know
00:04:16
opened up this question for me which was
00:04:19
how can two similar chart parts or
00:04:24
pieces of AI
00:04:26
built by the same company produce such
00:04:30
different results and what does that say
00:04:33
about you know our decision to to use
00:04:37
algorithms for a lot of our decisions in
00:04:41
our personal and professional lives and
00:04:43
that's what helped start this
00:04:45
exploration into use of AI the extent to
00:04:50
which AI can be predictable biased as in
00:04:53
the case of Microsoft a and then of
00:04:55
course how do we what does that mean in
00:04:57
terms of using these systems to make
00:04:59
significant decisions for us so why did
00:05:02
T experiences differ so dramatically and
00:05:05
is anything that can be done about that
00:05:08
one of the insights that I got as I was
00:05:13
writing this book as I was trying to
00:05:14
explain the differences in behavior of
00:05:16
these two chatbots
00:05:18
was actually from human psychology
00:05:22
so psychologists describe human behavior
00:05:24
in terms of our nature and our nurture
00:05:27
and so our nature is our genetic code
00:05:29
and nurture is our environment and so
00:05:33
psychologists will attribute problematic
00:05:35
issues like alcoholism partly to nature
00:05:39
and partly to nurture and when I was
00:05:41
looking at algorithms I realized
00:05:43
algorithms have nature and nurture as
00:05:45
well so the nature for algorithms is not
00:05:49
genetic code but the code that the
00:05:51
engineer actually codes in or writes and
00:05:54
that's the logic of the algorithm the
00:05:57
nurture for the algorithm is the data
00:05:59
from which algorithms learn and so
00:06:02
increasingly as we are moving towards
00:06:05
machine learning we're moving from a
00:06:07
world where engineers used to program
00:06:11
the end-to-end logic of an algorithm
00:06:12
would actually specify what happens when
00:06:15
any situation happens if this happens
00:06:17
you respond this way if that happens you
00:06:19
respond a different way and so it used
00:06:21
to be all nature because the programmer
00:06:23
gave all the very minut specifications
00:06:26
of how the algorithm will work but as we
00:06:29
move towards machine learning we're kind
00:06:31
of telling algorithms here's data learn
00:06:33
from it and so nature starts to become
00:06:36
less important and nurture starts to
00:06:38
dominate so if you look at what
00:06:40
happened between tea and showers the
00:06:44
difference is in terms of their training
00:06:46
data in some ways and in particular in
00:06:49
the case of showers showers was created
00:06:51
to mimic how people converse on Tay and
00:06:55
so it was picking up how people are
00:06:57
talking to it and it would reflect that
00:07:00
and so there were many intentional
00:07:02
efforts to trip day and so there was
00:07:06
nurture there as well and part of it was
00:07:08
nature as well meaning that the court
00:07:11
could have specified certain rules the
00:07:13
court could have specified rules like do
00:07:16
nots say the following kinds of things
00:07:18
or do not get into discussions of these
00:07:20
topics and so on so it's a bit of both
00:07:22
and I think that's what in general rogue
00:07:26
algorithmic behavior comes down to that
00:07:29
come back a little later to the whole
00:07:31
question of what happens go rogue yeah
00:07:35
but I wanted to chat a little bit about
00:07:39
the way in which this algorithmic
00:07:43
decision-making itself has changed and
00:07:45
it wasn't there was a time when
00:07:49
decision-making algorithm
00:07:50
decision-making seemed to be almost like
00:07:52
Amazon will tell you what books you
00:07:55
should read or Netflix will recommend
00:07:57
which movies you should watch but
00:08:00
because of AI algorithmic algorithmic
00:08:03
decision-making has become a lot more
00:08:04
complex and I was wondering if you could
00:08:07
offer some examples and what are some of
00:08:10
the implications on the choices that we
00:08:13
make or don't make as a result of this
00:08:16
yeah algorithms pervade our lives and
00:08:19
sometimes we see it like Amazon's
00:08:22
recommendations and sometimes we don't
00:08:23
realize it but they have a huge impact
00:08:26
on decisions we make on Amazon for
00:08:28
example over 1/3 of the choices that we
00:08:31
make are influenced by algorithmic
00:08:33
recommendations people who bought this
00:08:34
also bought those people who viewed this
00:08:36
eventually bought that and so on on
00:08:38
Netflix over 80% of the viewing activity
00:08:42
is driven by algorithmic recommendations
00:08:44
they also drive decisions such as who we
00:08:47
date and Mary when you look at
00:08:49
applications or apps like tinder which
00:08:52
actually
00:08:53
ABB's we're algorithms create most of
00:08:56
the matches over there
00:08:57
they're also at the workplace you know
00:08:59
for example you make you apply for our
00:09:02
alone mortgage approval decisions are
00:09:05
made by algorithms increasingly you know
00:09:07
if you apply for a job resumes screaming
00:09:10
algorithms are deciding which ones to
00:09:12
invite for an interview and they're
00:09:14
making life-and-death decisions as well
00:09:16
there for example being used in the
00:09:19
criminal justice system in courtrooms in
00:09:21
the u.s. there are algorithms that
00:09:23
predict the likelihood that the
00:09:25
defendant will reoffend so that judges
00:09:27
can make sentencing decisions in
00:09:29
medicine we're moving towards
00:09:31
personalized medicine so that two people
00:09:33
with the same symptoms might not get the
00:09:35
same treatment it might be customized
00:09:37
based on their DNA profile algorithms
00:09:39
again will guide the doctors on those
00:09:41
decisions and also we're moving as AI
00:09:45
has progressed and advanced we're moving
00:09:49
to a point where the algorithms don't
00:09:51
merely offer decision support they can
00:09:53
function autonomously as well and
00:09:55
driverless cars are a great example of
00:09:57
that where we're trying to pretty much
00:10:00
say you can automate the whole process
00:10:02
and and have algorithms function without
00:10:06
a human making the final sort of
00:10:09
decision so as more algorithms influence
00:10:13
or make more and more decisions
00:10:14
is there anything like free will in the
00:10:18
world anymore well so free will is an
00:10:21
interesting concept and for the most
00:10:23
part I used to think of free will in a
00:10:25
very philosophical sense right and
00:10:27
philosophers have argued we don't even
00:10:29
have free will but that was in a very
00:10:31
different very as I said philosophical
00:10:33
interpretation but I think we have a
00:10:35
literal interpretation of free will now
00:10:37
in the context of algorithms which is
00:10:39
are you making the final choice and I
00:10:42
just said you know a third of your
00:10:43
choices are driven on Amazon by
00:10:45
recommendations 80% of viewing
00:10:47
activities on Netflix are driven by
00:10:50
algorithmic recommendations at YouTube
00:10:52
seventy percent of the time people spend
00:10:54
on YouTube is driven by algorithmic
00:10:56
recommendations so it doesn't feel like
00:10:59
algorithms are merely recommending to us
00:11:01
what we want and we make decisions I
00:11:03
mean think about a search on Google you
00:11:06
take the most
00:11:06
Oh Tarek of search terms like vintage
00:11:10
toy model trains or something like that
00:11:12
you will still find hundreds of
00:11:14
thousands perhaps even millions of
00:11:16
results for that search term we might
00:11:18
see less than 0.01% of the search
00:11:21
results because rarely do we even cross
00:11:23
page one the algorithm has decided which
00:11:25
pages we look at so yes they're making a
00:11:28
lot of choices for us so do we have free
00:11:31
well at some level yes but mostly I'm
00:11:33
going to say we don't have the level of
00:11:37
independent decision-making that we
00:11:39
think we do because most of us think we
00:11:42
get these recommendations we not
00:11:44
politely and then we do what we want but
00:11:46
indeed the algorithms are not jiggles in
00:11:48
very interesting ways and and I don't
00:11:51
think we have the level of independent
00:11:53
decision-making or choice that we think
00:11:55
we do mostly that's a good thing in some
00:11:57
ways because they're saving us time so
00:12:01
we can focus on leisure instead of
00:12:03
wasting our time sifting through lots of
00:12:05
alternatives but sometimes we become
00:12:07
very passive about how we use algorithms
00:12:09
and becoming that passive about
00:12:11
algorithm use can have consequences so
00:12:14
let's talk a little bit about those
00:12:16
consequences okay especially unintended
00:12:18
consequences yes there's a fascinating
00:12:21
part on your book about that and how do
00:12:23
design choices lead to unintended
00:12:27
consequences I was wondering if you
00:12:28
could speak about that yes so when I
00:12:31
mention unintended consequences in the
00:12:34
book I'm referred referring to
00:12:35
situations where you know you're trying
00:12:38
to optimize some aspect of a decision
00:12:43
and perhaps you manage to improve that
00:12:45
really well but then something else goes
00:12:47
wrong
00:12:48
so an example might be that when
00:12:51
Facebook was manually curating its
00:12:55
trending stories through human editors
00:12:57
as an editor you will appreciate that
00:12:59
it's real work right and they had people
00:13:02
doing that but then Facebook was accused
00:13:06
of having a left-leaning bias that these
00:13:08
editors were choosing left-leaning
00:13:10
stories and curating those more often so
00:13:13
they said you know algorithms can't be
00:13:15
accused of a political bias so they used
00:13:17
an algorithm to curate this they tested
00:13:19
it for political bias
00:13:20
it did not have any political bias but
00:13:23
there's something else it did which they
00:13:25
hadn't explicitly tested for which is as
00:13:27
we know it curated fake new stories and
00:13:29
and circulated them so that's an example
00:13:33
of unintended consequences and an
00:13:35
algorithm design can drive that in many
00:13:38
ways you know I've done a lot of work on
00:13:40
recommendation systems and how they
00:13:42
influence the kinds of products we
00:13:44
consume the kinds of media we consume
00:13:46
and I've specifically studied two kinds
00:13:49
of recommendation algorithms one kind of
00:13:53
algorithm is based or it's like the
00:13:56
Amazon people who bought this also
00:13:58
bought this so it's based on social
00:14:01
curation what are the what are others
00:14:03
consuming the other kind of algorithm
00:14:05
attempts to understand at a deep level
00:14:07
what is it that I'm recommending and the
00:14:10
algorithm is recommending and tries to
00:14:12
find items that are very similar to the
00:14:16
users interest an example of that would
00:14:18
be Pandora so Pandora's music
00:14:21
recommendations are not people who like
00:14:23
this song also like these other songs
00:14:25
Pandora actually has very detailed
00:14:28
information over 150 attributes for each
00:14:32
song musical attributes like how
00:14:34
rhythmic is the song how much
00:14:36
instrumentation is there in the music
00:14:37
and every time you say you like a music
00:14:40
a song or you don't like a song they
00:14:43
look at the musical qualities of the
00:14:44
song and then they adjust their
00:14:46
recommendations based on other songs
00:14:48
which have attributes similar to what
00:14:49
you have now I looked at both these
00:14:51
designs and I looked at which design is
00:14:54
more helpful in helping us find let's
00:14:58
say indie songs or very novel and niche
00:15:01
books or movies and at the time we did
00:15:05
the study and this was some time back
00:15:07
the conventional wisdom was that all of
00:15:09
these algorithms help in pushing the
00:15:12
long tail meaning these nice novel items
00:15:16
or indie songs that nobody's heard of
00:15:18
and what I found was that these designs
00:15:20
were very different the algorithm that
00:15:23
looks at the look said you know what
00:15:26
others are consuming people who bought
00:15:28
this also bought this it has a
00:15:30
popularity bias because it's trying to
00:15:32
recommend stuff
00:15:33
that others are consuming and so it
00:15:35
tends to lean towards popular items and
00:15:38
so it cannot truly recommend those
00:15:40
hidden gems but an algorithm like the
00:15:43
Pandora algorithm is it doesn't have
00:15:47
popularity as a basis for
00:15:49
recommendations so it tends to do better
00:15:50
and that's why what we've seen now is
00:15:52
companies like Spotify Netflix and many
00:15:55
others have changed the design of their
00:15:57
algorithms they've combined the two
00:15:59
approaches they've combined the the
00:16:02
social appeal of a system that looks at
00:16:06
what others are consuming and the
00:16:09
ability of the other design to surface
00:16:12
hidden gems let's go now to the point
00:16:17
you were up earlier about algorithms
00:16:19
going rogue yes why does that happen and
00:16:22
what can be done about it yeah if we
00:16:25
look at let me point a couple example
00:16:27
examples of algorithms doing rolled and
00:16:29
then we'll talk about why this happens
00:16:31
so I mentioned algorithms are used in
00:16:34
courtrooms in the u.s. in the criminal
00:16:36
justice system in 2016 there was a
00:16:39
report or study done by ProPublica which
00:16:42
is a nonprofit organization they looked
00:16:45
at algorithms used in courtrooms and
00:16:47
they found that these algorithms have a
00:16:49
race bias specifically they found that
00:16:51
these algorithms were twice as likely to
00:16:55
falsely predict future criminality in a
00:16:59
black defendant than a white defendant
00:17:01
and this was race bias in that algorithm
00:17:05
late last year Reuters carried a story
00:17:07
about Amazon trying to use algorithms to
00:17:10
screen job applications and the you know
00:17:15
again Amazon gets a million plus job
00:17:17
applications they hire hundreds of
00:17:19
thousands of people you know over the
00:17:21
last few years they've done that it's
00:17:24
hard to do that manually and so you need
00:17:26
algorithms to help automate some of this
00:17:27
but they found that the algorithms
00:17:30
tended to have a gender bias they tended
00:17:33
to reject female applicants more often
00:17:36
even when the qualifications were
00:17:38
similar now Amazon ran the test and
00:17:40
concluded and realized that there are
00:17:42
savvy companies so they decided not to
00:17:44
roll this out there's probably many
00:17:46
other companies that
00:17:47
using algorithms to screen resumes and
00:17:49
they might be prone to race bias gender
00:17:52
bias and so on so I've mentioned two
00:17:55
examples now race bias gender bias I
00:17:57
talked about fake news on on Facebook so
00:18:01
there's many examples in terms of why
00:18:03
they go rogue there's a couple reasons I
00:18:07
can share one is as we have gone from
00:18:10
those old traditional algorithms where
00:18:14
the programmer wrote up the algorithm
00:18:17
end to end and we've moved towards
00:18:19
machine learning we have created
00:18:22
algorithms that are usually more
00:18:24
resilient because these algorithms you
00:18:28
know can perform much better but they're
00:18:31
prone to biases that exist in the data
00:18:33
so if the data have biases so for
00:18:37
example if you tell a resume screening
00:18:38
algorithm you know here's data on all
00:18:41
those people who applied to our job and
00:18:44
here's the people we actually hired and
00:18:47
here are the people who we promoted now
00:18:49
figure out who to invite for job
00:18:51
interviews based on this data the
00:18:53
algorithm will observe that in the past
00:18:55
you are rejecting more female
00:18:57
applications or you were not promoting
00:19:00
women at the workplace and will tend to
00:19:03
pick up that behavior so it tends to
00:19:05
pick that up I think the other piece is
00:19:07
that engineers in general tend to focus
00:19:10
narrowly on one or two metrics you know
00:19:13
with a resume screening application you
00:19:16
will tend to measure the accuracy of
00:19:18
your model and if it's highly accurate
00:19:20
you'll roll it out but you don't
00:19:22
necessarily look at fairness and bias or
00:19:26
in Facebook's
00:19:27
example look at fake news and other
00:19:29
possibilities as well what are the
00:19:32
challenges involved in autonomous
00:19:37
algorithms making decisions on our
00:19:39
behalf well I think when you have
00:19:43
autonomous algorithms making decision on
00:19:45
our behalf I think one of the big
00:19:48
challenges is there is usually no human
00:19:51
in the loop so we lose control and that
00:19:53
is one one challenge and many studies
00:19:57
show that when we have limited control
00:20:00
we are less
00:20:01
likely to trust algorithms and so that
00:20:04
is one one aspect that is challenging
00:20:06
the other piece about having autonomous
00:20:09
algorithms is that again if there's a
00:20:11
human in the loop there's a greater
00:20:15
chance that the user can detect certain
00:20:17
problems and you know the likelihood
00:20:20
that problems get detected is therefore
00:20:22
greater I'm really glad you brought up
00:20:25
the point about trusting algorithms
00:20:26
because you tell this really fascinating
00:20:28
story in the book about a patient who
00:20:31
gets diagnosed with definitely fever I
00:20:35
wonder if you could share that story
00:20:36
with our audience and and spell out some
00:20:39
of the implications for trust in
00:20:43
algorithms and and what the implications
00:20:44
are yeah the story that I share is that
00:20:48
of a patient walking into a doctor's
00:20:50
office and the patient feels fine and
00:20:53
healthy and the patient and doctor are
00:20:56
joking around and doctor eventually
00:20:59
picks up the pathology report and he
00:21:03
suddenly looks very serious and he's
00:21:07
says or informs the patient that I'm
00:21:10
sorry to let you know that you have to
00:21:12
happen early fever and the patient
00:21:16
hasn't heard of that Bernoulli fever and
00:21:18
he asks what exactly is it and so the
00:21:20
doctor says it's a very rare disease and
00:21:24
it's known to be fatal and so I suggest
00:21:28
that you have this tablet and it will
00:21:32
you know reduce the chance that you will
00:21:34
have any problems and you know he says
00:21:38
here you take this you know three times
00:21:39
a day and and and then you go about your
00:21:42
life and I asked my readers if you know
00:21:48
that story is something that if they
00:21:51
were the patient they would feel
00:21:53
comfortable in that situation here's a
00:21:55
disease you don't nothing about and
00:21:56
here's a solution you know nothing about
00:21:58
and the doctor has given you the choice
00:22:03
and told you to go ahead but not given
00:22:06
you very many details and with that I
00:22:10
pose the question if an algorithm way to
00:22:13
again make this wreck
00:22:15
that you have this rare disease and we
00:22:17
want you to take this medication without
00:22:21
any information would you
00:22:22
and of course Mukul you've read the book
00:22:25
so you know Tappin le fever is not a
00:22:27
real disease I'm a fan of Sherlock
00:22:31
Holmes I used to read it a lot as a kid
00:22:33
and it's a disease in a one of the
00:22:35
Sherlock Holmes stories and you know
00:22:38
that inspired me to consider this
00:22:41
because even in the original Sherlock
00:22:43
Holmes story it turns out that the
00:22:45
person who has definitely fever doesn't
00:22:47
actually have it but setting that aside
00:22:50
it kind of brings up this question of
00:22:52
transparency you know are we willing to
00:22:55
trust decisions when we don't have
00:22:57
information about why a certain decision
00:23:00
was made the way it was and what I
00:23:02
highlight is sometimes we're seeking
00:23:04
more transparency from algorithms than
00:23:06
humans but in practice lots of companies
00:23:09
are imposing algorithmic decisions on us
00:23:12
without any information about why these
00:23:15
decisions are being made and are we fine
00:23:18
with that and a lot of research shows
00:23:20
that we're not fine with that research
00:23:22
for example that one PhD student then a
00:23:26
PhD student at Stanford looked at an
00:23:29
algorithm that would compute grades for
00:23:32
students and how did they do when they
00:23:35
just got their score versus they got
00:23:37
their score with an explanation and as
00:23:39
you expect when they have an explanation
00:23:41
they trust it more then why is it that
00:23:43
in in the real world there's a lot of
00:23:46
algorithms making decisions for us or
00:23:48
about us and we have no transparency
00:23:50
about those decisions and so I advocate
00:23:53
that we need a certain level of
00:23:55
transparency with regard to for example
00:23:57
what kinds of data were used to make the
00:24:00
decision so for example if we applied
00:24:04
for a loan and the loan was rejected we
00:24:06
would like to know why that was the case
00:24:07
if you applied for a job and it was
00:24:10
rejected it would be helpful to know
00:24:12
that the algorithm not only evaluated
00:24:15
what you submitted as part of your job
00:24:17
application would also looked at your
00:24:19
social media posts and so transparency
00:24:22
regarding what data was considered what
00:24:25
were the key factors that drove a
00:24:27
decision
00:24:27
is important but what's the end of the
00:24:30
book you recommend an algorithmic Bill
00:24:33
of Rights what exactly is that and why
00:24:35
is it necessary so the algorithmic Bill
00:24:39
of Rights is a concept that I borrowed
00:24:42
from the Bill of Rights in the US
00:24:44
Constitution and the history of the Bill
00:24:47
of Rights is that when the founding
00:24:48
fathers were setting up or drafting the
00:24:51
Constitution some people were worried
00:24:54
that we're creating a very powerful
00:24:57
government here in the US and the Bill
00:25:02
of Rights was created as a way to
00:25:05
protect citizens now today we are in a
00:25:08
situation where there's a lot of talk
00:25:10
about powerful tech companies you know
00:25:12
everywhere you see there's new stories
00:25:14
big tech companies and what are we going
00:25:15
to do about them and so there's a sense
00:25:18
that again consumers need certain
00:25:21
protections and so the Bill of Rights is
00:25:24
targeted at that but before I talk about
00:25:26
the Bill of Rights one aspect related to
00:25:28
the Bill of Rights I do want to address
00:25:30
is that a lot of consumers feel that
00:25:33
they're helpless against big tech and
00:25:35
against algorithms deployed by big tech
00:25:37
and personally I feel that consumers do
00:25:42
have some power and that power is in
00:25:45
terms of our knowledge our votes and our
00:25:47
dollars you know knowledge is about you
00:25:49
know we shouldn't be passive users of
00:25:51
technology we should be active and
00:25:53
deliberate about it we should know how
00:25:55
it's changing decisions we are making or
00:25:57
others are making about us and at some
00:26:00
level it seems like you know knowledge
00:26:01
is fine but what can I do with that
00:26:03
knowledge but I mean look at how
00:26:05
Facebook is changing their product
00:26:07
design today and that changes you know
00:26:10
support for encryption and so on is
00:26:11
because of push from the users and it
00:26:14
shows that when users complain changes
00:26:17
to happen votes there are another aspect
00:26:19
of that and votes are all about you know
00:26:22
us being aware of which elected
00:26:27
representatives understand the nuances
00:26:30
of algorithms and the challenges and how
00:26:34
to regulate them and it's about voting
00:26:36
for them right and the question is how
00:26:39
are these regulators going to protect us
00:26:41
that's where the bill of rights comes in
00:26:42
and the Bill of Rights I propose has a
00:26:45
few key pillars one pillar is
00:26:47
transparency transparency with regard to
00:26:49
the data used to make decisions and with
00:26:51
regard to the underlying decision itself
00:26:55
what were the most important factors
00:26:57
that led to a certain decision today
00:27:00
Europe's GDP are actually has certain
00:27:03
provisions like write to explanations
00:27:04
and information on the data that
00:27:07
companies are using and so I think some
00:27:09
of that transparency is needed and
00:27:11
companies should provide that another
00:27:14
pillar in my bill of rights is the idea
00:27:17
of some user control that we cannot be
00:27:21
in an environment where we have no
00:27:23
control over the technology we should
00:27:25
for example be able to with a simple
00:27:28
instruction tell Alexa you're not
00:27:30
listening to any conversation in the
00:27:33
house until I'd instruct you that it's
00:27:36
allowed right there's no such provision
00:27:38
we're told that the system is not
00:27:41
listening but then we're also hearing
00:27:42
from others that there are instances
00:27:44
where it listens even when you're not
00:27:46
actually saying Alexa and giving an
00:27:49
instruction and that control is very
00:27:52
important if you look at Facebook the
00:27:54
false news issue two years back there
00:27:56
was no way for users to alert Facebook's
00:27:58
algorithm and say this post in my
00:28:00
newsfeed is false news even though users
00:28:03
were seeing false news in their newsfeed
00:28:05
even today with just two clicks you can
00:28:08
let Facebook know that a certain news
00:28:11
post in your feed is either offensive or
00:28:14
it has false news and that feedback is
00:28:17
so important for the algorithm to now
00:28:19
correct itself and previously users had
00:28:21
no way even though you're observing
00:28:23
problems no way of informing the
00:28:26
algorithm or helping the algorithm
00:28:27
course correct
00:28:28
so some level of user control is also
00:28:31
another pillar I proposed and lastly I
00:28:33
have been advocating this idea that
00:28:35
companies should formally audit
00:28:37
algorithms before they deploy them as
00:28:39
especially in socially consequential
00:28:42
settings not every algorithm needs to be
00:28:44
audited
00:28:44
but algorithms in socially consequential
00:28:47
settings like recruiting need to go
00:28:49
through an audit process and that audit
00:28:51
process can be done by an internal team
00:28:53
or it could be outside
00:28:55
but it's got to be done by a team that
00:28:56
is independent of the team that
00:28:58
developed the algorithm and the audit
00:29:00
process is going to be important because
00:29:03
it will help ensure that somebody's
00:29:05
looked at things beyond say the
00:29:08
prediction accuracy of the model they
00:29:10
have looked at things like privacy
00:29:11
they've looked at things like bias and
00:29:14
fairness and so that will help curb some
00:29:17
of these problems with with algorithmic
00:29:19
decisions so I got we could keep talking
00:29:22
about this all day I've sort of come to
00:29:26
the end of the list of questions I had
00:29:28
for you are there any time points that
00:29:31
you would like to emphasize that I
00:29:33
haven't asked about one of the key
00:29:37
messages I want to share with people
00:29:39
even though I'm sharing many of the
00:29:41
challenges with algorithms in my book is
00:29:44
that I'm not an algorithm skeptic I'm
00:29:46
actually a believer in algorithms and I
00:29:48
don't want any listener or viewer or
00:29:52
reader to leave this becoming wary of
00:29:55
technology I think the message is not be
00:29:58
very but the message is engage more
00:30:01
actively and more deliberately and
00:30:02
influence be part of the process of
00:30:05
influencing how these technologies
00:30:07
develop so and the reason I say that is
00:30:10
that studies show that algorithms on
00:30:12
average are less biased than human
00:30:15
beings so if we say we don't want
00:30:16
algorithms we need to ask what's the
00:30:18
alternative
00:30:19
and the alternative is biased
00:30:21
furthermore my contention is that it is
00:30:23
easier in the long run to fix fix
00:30:26
algorithm bias than it is to fix human
00:30:28
bias but the challenge with algorithm
00:30:31
bias is just that that bias scales in a
00:30:34
way human bias does not scale and what I
00:30:36
mean by that is that a prejudiced judge
00:30:39
can impact the lives of maybe 200 300
00:30:41
people but an algorithm used in all the
00:30:46
courtrooms in a country or across the
00:30:48
world can influence the lives of
00:30:50
hundreds of thousands or even millions
00:30:51
of people similarly a bias recruiter can
00:30:54
affect the lives of hundreds of people
00:30:56
but a bias recruiting algorithm can
00:30:58
affect the lives of millions of people
00:31:00
so it's the scale we worry about and
00:31:02
that's why we need to take the issue
00:31:03
seriously
00:31:04
but the message mostly is I think we're
00:31:08
going
00:31:08
to a world where these algorithms will
00:31:10
help us make better decisions and will
00:31:13
have growing pains along the way and a
00:31:15
few examples I mentioned I think is only
00:31:18
just the beginning we'll hear many more
00:31:19
and we should engage actively now to
00:31:22
minimize those incidences Karthik thank
00:31:26
you so much for speaking with knowledge
00:31:27
at work thank you for having me it's
00:31:29
been a great pleasure for more insight
00:31:34
from knowledge at Wharton please visit
00:31:36
knowledge Wharton UPenn edu
00:31:42
[Music]
00:31:47
you
