00:00:04
an important team of my research is how
00:00:07
personalized recommendations and similar
00:00:09
algorithms affect consumer choice we are
00:00:12
all flooded by product choices today and
00:00:15
these kinds of personalized
00:00:17
recommendations play an important role
00:00:19
in helping us discover new products are
00:00:22
sorting through large choice sets and we
00:00:25
see a personalized recommendations in a
00:00:27
number of industries whether it's in
00:00:29
retail for example Amazon's of people
00:00:32
who bought this also bought this or in
00:00:34
media such as Netflix or YouTube we see
00:00:38
it with news as well for example Google
00:00:40
News will recommend personalized news
00:00:42
stories and we know they have a pretty
00:00:44
significant impact on consumer choice
00:00:46
for example at Amazon they drive
00:00:48
anywhere from a quarter to a third of
00:00:50
the choices that consumers make online
00:00:53
so although we know that they have a big
00:00:56
impact on consumer choice we don't fully
00:00:58
understand what kinds of products are
00:01:02
more likely to be accepted by consumers
00:01:05
when recommended and when do
00:01:07
recommenders work well and when they
00:01:09
don't so in my research with Professor
00:01:12
document Carnegie Mellon we look at two
00:01:15
main questions the first is what kinds
00:01:18
of products are more likely to benefit
00:01:21
from recommendations specifically our
00:01:25
mainstream products are niche products
00:01:27
more likely to benefit from
00:01:29
recommendations and the other question
00:01:32
we look at is what is it about a product
00:01:34
that that makes it more likely to elicit
00:01:38
a response from a consumer when it's
00:01:39
recommended for example the ratings of
00:01:41
the products or the price of the product
00:01:43
or the type of the product do they
00:01:45
influence whether recommendations are
00:01:47
effective for that product
00:01:53
so in one of our research studies we
00:01:55
look at whether recommendation systems
00:01:57
help us discover novel and nisha items
00:01:59
that we might not otherwise discover but
00:02:02
are a great fit for us personally what
00:02:06
we find in our study is that because
00:02:08
common recommendation systems are based
00:02:10
on sales and ratings for example you
00:02:12
know people who bought this also about
00:02:14
this they are unable to surface truly
00:02:17
novel items that have not been
00:02:18
discovered by many other people and this
00:02:22
tends to create a rich gets richer
00:02:23
effect for popular items and it might
00:02:26
also prevent consumers for finding a
00:02:29
better product matches because of this
00:02:32
bias for items that have been purchased
00:02:34
by others or that have been rated well
00:02:36
by others now this is a finding that a
00:02:40
lot of people find surprising because
00:02:42
many friends tell me that they do find
00:02:45
very new items that they previously did
00:02:47
not know about through recommendations
00:02:49
and in line with that we find that these
00:02:53
recommend recommendation systems can
00:02:55
push us as individuals to new items but
00:02:58
they push all of us towards the same new
00:03:01
items and does at the aggregate level we
00:03:03
don't see this great increase in
00:03:04
diversity of purchases from consumers so
00:03:08
in another research study we looked at
00:03:10
what is it about a product that makes it
00:03:13
more likely to elicit a response from a
00:03:15
consumer when it's recommended for
00:03:17
example we looked at interactions
00:03:18
between a products rating and the
00:03:21
recommendation response and we find that
00:03:23
as one would expect recommendations help
00:03:27
all kinds of products whether they rated
00:03:28
high or whether they rated low but
00:03:31
interestingly we find that it's the
00:03:34
products that have a low average ratings
00:03:37
that elicit a greater response from the
00:03:40
consumer that is the purchase
00:03:41
probability of a product goes up a lot
00:03:44
more when for lower rated products than
00:03:47
high rate higher rated products and this
00:03:49
tells us that recommendations and
00:03:51
ratings are in some way substitutes so
00:03:53
if a product has high ratings to begin
00:03:55
with then the recommendation has an
00:03:57
impact but it's not as great but when it
00:03:59
has low ratings you know in the absence
00:04:02
of the recommendation we might not even
00:04:04
respond to that product
00:04:05
but when that product is recommended
00:04:07
then we are willing to give the product
00:04:09
the benefit of doubt maybe the product
00:04:12
isn't right for everyone out there but
00:04:14
perhaps it's right for me and so we find
00:04:16
that recommendations and ratings can be
00:04:18
substitutes as well another aspect we
00:04:21
looked at is whether the type of the
00:04:24
product matters so we classified all the
00:04:26
products in our data set into what we
00:04:29
call utilitarian products and hedonic
00:04:31
products so utilit ate it utilitarian
00:04:33
products are products that serve some
00:04:35
functional purpose for example
00:04:38
appliances or groceries and hedonic
00:04:41
products are products that don't serve a
00:04:43
functional purpose and really appeal to
00:04:45
some sensory perception for example
00:04:48
jewelry and we found that
00:04:50
recommendations have a you know low to
00:04:53
moderate impact for utilitarian products
00:04:55
but it is for the hedonic products that
00:04:57
they have a very significant impact and
00:04:59
these hedonic products you know things
00:05:02
like jewelry where we don't really mean
00:05:04
it and when a recommendation kind of
00:05:06
suggests that this is a great fit for us
00:05:08
or people with similar tastes like this
00:05:11
product that really moves the needle in
00:05:13
terms of making us respond to that
00:05:15
recommendation we look at other things
00:05:18
like denote a description of the product
00:05:20
matters the price does it matter and ute
00:05:22
and we find what you would expect you
00:05:23
know people respond to recommendations
00:05:26
for lower price products and higher
00:05:28
price products and where there's better
00:05:30
description for the product and if there
00:05:32
is very limited description
00:05:37
so there are two conclusions that
00:05:40
surprised us that we didn't expect a
00:05:43
priori one was that recommendations
00:05:45
don't necessarily help us discover niche
00:05:50
products and that is interesting because
00:05:53
there has been a lot of discussion for
00:05:55
at least a decade now about how online
00:05:58
systems whether it's search engines or
00:06:00
personalized recommendations they will
00:06:02
help us find niche items and they will
00:06:04
help benefit you know what we call the
00:06:08
long tail the products that are not
00:06:11
super popular that almost don't get
00:06:14
produced that may get produced but don't
00:06:16
really sell much and the promise of
00:06:19
recommendation systems is they really
00:06:21
give a fair opportunity for these kinds
00:06:25
of products and we find that for the
00:06:27
common designs it doesn't happen and
00:06:30
that's pretty surprising there are some
00:06:32
designs where you you make design
00:06:34
modifications and you favor nisha items
00:06:37
you can make it work but most common
00:06:39
designs that are used at most retailers
00:06:41
they don't do that and we found that you
00:06:44
know it might have the opposite effect
00:06:46
now another result that surprised us was
00:06:48
that recommendations and ratings or
00:06:51
substitutes again our priori we expected
00:06:54
that you know people will respond to
00:06:57
recommendations when they are highly
00:06:59
rated and we did find that when they're
00:07:01
highly rated people respond to
00:07:03
recommendations but what surprised us
00:07:04
was that their responses even greater
00:07:06
when the products have a lower rating
00:07:09
and so that suggests that there's this
00:07:12
effect of recommendations as substitutes
00:07:15
for ratings and we hadn't predicted that
00:07:18
beforehand
00:07:23
so our research has implications for
00:07:24
retailers for producers and even
00:07:26
consumers for retailers to the extent
00:07:29
that their strategy is to offer a wide
00:07:31
product assortment is our research
00:07:33
suggests that the choices of technology
00:07:36
they make may not always be consistent
00:07:38
with that strategy and they need to
00:07:40
think harder about the technology
00:07:41
choices for example Amazon strategy is
00:07:44
that you can find any product on earth
00:07:46
at Amazon and really wide product
00:07:49
assortment is its strategy similarly
00:07:51
many online retailers also offered white
00:07:54
product assortment so our research
00:07:56
suggests that if you offer great product
00:07:58
assortment you also need to think about
00:08:00
how will consumers discover that white
00:08:02
product assortment and recommendations
00:08:04
are an important part of the solution
00:08:05
but they don't often work in practice
00:08:07
because they have this bias towards
00:08:09
products that have been bought before
00:08:10
and that have been rated before and so
00:08:13
our research suggests that they need to
00:08:15
think about technology choice and think
00:08:17
about how to modify this common design
00:08:19
so it is consistent with their product
00:08:22
choices in general for producers you
00:08:26
know research shows that recommendations
00:08:28
and similar algorithms they drive
00:08:30
consumer choice in a big way so
00:08:32
producers need to think hard about how
00:08:35
the game discovered by these algorithms
00:08:37
today producers are used to thinking
00:08:40
about how do I how does our product get
00:08:42
discovered by consumers they need to
00:08:45
also ask how do how does our product get
00:08:47
discovered by algorithms and for
00:08:49
consumers we find that these systems are
00:08:52
great at helping us as individuals
00:08:54
discover new products but at the
00:08:56
aggregate level we are not seeing that
00:08:57
diversity which is not necessarily
00:09:00
troublesome for consumers but it does
00:09:02
suggest that there are products out
00:09:04
there that could be this needle in the
00:09:06
haystack perfect product for you which
00:09:08
may not be surfaced by recommendations
00:09:10
so one has to be open to other sources
00:09:13
of discovery as well
00:09:18
so there's a lot of talk these days
00:09:21
about big data and analytics and how
00:09:23
there's so much data and companies are
00:09:25
building intelligent algorithms that can
00:09:27
help them be smart that can help
00:09:29
consumers find products they like and so
00:09:32
on and our research shows that these
00:09:34
efforts do work but at the same time we
00:09:39
have to be cautious about unintended
00:09:40
consequences so for example the idea of
00:09:44
recommendations is that they help us
00:09:46
find novel items but we don't want them
00:09:48
to have biases built into them where
00:09:50
they favor certain kinds of items versus
00:09:52
others and to the extent they favor
00:09:54
certain kinds of items that might have
00:09:57
an unintended consequence and we can
00:09:59
think about that say in the context of
00:10:00
news if we all consume news through
00:10:03
personalized recommendations we may not
00:10:06
always get that breadth of perspective
00:10:08
we want and so algorithms might be
00:10:11
driving a lot of our choice with media
00:10:12
and we need to think hard about how big
00:10:15
data and algorithms can be be biased and
00:10:20
and largely they work but they do have
00:10:22
some biases we need to be cautious about
00:10:28
so one of the things that's really novel
00:10:30
about our work is that our research is
00:10:34
informed by really large-scale data and
00:10:38
analysis of that data there have been a
00:10:40
lot of theories about how
00:10:42
recommendations impact consumer choice
00:10:44
what kinds of products they favor when
00:10:46
consumers respond to them and so on in
00:10:49
practice there had been very little
00:10:51
empirical evidence and that's partly
00:10:53
because in order to answer these
00:10:55
questions for example do they favor
00:10:57
niche items or mainstream items we need
00:11:00
a contrast between people exposed to
00:11:02
recommendations and unexposed to
00:11:04
recommendations and for most retailers
00:11:06
they observe consumers only after they
00:11:09
come to their website and are exposed to
00:11:11
recommendations and so our study is
00:11:13
based on and an experiment that was done
00:11:17
with a large retailer in North America
00:11:21
with whom we ran an a/b experiment where
00:11:23
some people got recommendations some did
00:11:25
not and we ran this for hundreds of
00:11:28
different product categories and so we
00:11:31
were able to not only get that contrast
00:11:33
needed to answer the question but we are
00:11:35
also able to generalize beyond a single
00:11:37
product category because we had so many
00:11:40
different products and that I think is
00:11:42
one of the things that sets this
00:11:44
research apart which is that it's based
00:11:46
on data it's based on concrete evidence
00:11:48
and it's based on a large enough sample
00:11:51
point and a large very representative
00:11:53
sample of consumers
00:11:59
so in terms of next steps we're trying
00:12:02
to generalize some of our results beyond
00:12:04
recommendations and think about all
00:12:05
kinds of search tools we have online and
00:12:08
so they include social tools and social
00:12:10
news feeds for example on Facebook we
00:12:12
discover products it includes search
00:12:14
engines we're also trying to understand
00:12:17
this world from the producers
00:12:19
perspective as I mentioned you know
00:12:22
there are implications for producers and
00:12:24
producers need to think hard about how
00:12:26
consumers will discover their products
00:12:29
through algorithms and so they need to
00:12:32
think about discovery by the algorithms
00:12:34
as well and there's very limited
00:12:36
understanding of what is it about a
00:12:38
product that makes a recommendation pick
00:12:41
that product among thousands of
00:12:42
potential candidates and so we're trying
00:12:45
to study that and hopefully we'll
00:12:47
provide some insights to producers so
00:12:49
that they can be active participants in
00:12:52
helping their products get discover
00:12:54
rather than passive observers
00:13:11
you
