00:00:04
an important team of my research is how personalized recommendations and similar
00:00:09
algorithms affect consumer choice we are all flooded by product choices today and
00:00:15
these kinds of personalized recommendations play an important role in helping us discover new products are
00:00:22
sorting through large choice sets and we see a personalized recommendations in a
00:00:27
number of industries whether it's in retail for example Amazon's of people
00:00:32
who bought this also bought this or in media such as Netflix or YouTube we see it with news as well for example Google
00:00:40
News will recommend personalized news stories and we know they have a pretty significant impact on consumer choice
00:00:46
for example at Amazon they drive anywhere from a quarter to a third of the choices that consumers make online
00:00:53
so although we know that they have a big impact on consumer choice we don't fully
00:00:58
understand what kinds of products are more likely to be accepted by consumers when recommended and when do
00:01:07
recommenders work well and when they don't so in my research with Professor document Carnegie Mellon we look at two
00:01:15
main questions the first is what kinds of products are more likely to benefit from recommendations specifically our
00:01:25
mainstream products are niche products more likely to benefit from recommendations and the other question
00:01:32
we look at is what is it about a product that that makes it more likely to elicit
00:01:38
a response from a consumer when it's recommended for example the ratings of the products or the price of the product
00:01:43
or the type of the product do they influence whether recommendations are effective for that product
00:01:53
so in one of our research studies we look at whether recommendation systems help us discover novel and nisha items
00:01:59
that we might not otherwise discover but are a great fit for us personally what we find in our study is that because
00:02:08
common recommendation systems are based on sales and ratings for example you know people who bought this also about
00:02:14
this they are unable to surface truly novel items that have not been discovered by many other people and this
00:02:22
tends to create a rich gets richer effect for popular items and it might also prevent consumers for finding a
00:02:29
better product matches because of this bias for items that have been purchased by others or that have been rated well
00:02:36
by others now this is a finding that a lot of people find surprising because many friends tell me that they do find
00:02:45
very new items that they previously did not know about through recommendations and in line with that we find that these
00:02:53
recommend recommendation systems can push us as individuals to new items but they push all of us towards the same new
00:03:01
items and does at the aggregate level we don't see this great increase in diversity of purchases from consumers so
00:03:08
in another research study we looked at what is it about a product that makes it more likely to elicit a response from a
00:03:15
consumer when it's recommended for example we looked at interactions between a products rating and the
00:03:21
recommendation response and we find that as one would expect recommendations help
00:03:27
all kinds of products whether they rated high or whether they rated low but interestingly we find that it's the
00:03:34
products that have a low average ratings that elicit a greater response from the
00:03:40
consumer that is the purchase probability of a product goes up a lot more when for lower rated products than
00:03:47
high rate higher rated products and this tells us that recommendations and ratings are in some way substitutes so
00:03:53
if a product has high ratings to begin with then the recommendation has an impact but it's not as great but when it
00:03:59
has low ratings you know in the absence of the recommendation we might not even respond to that product
00:04:05
but when that product is recommended then we are willing to give the product the benefit of doubt maybe the product
00:04:12
isn't right for everyone out there but perhaps it's right for me and so we find
00:04:16
that recommendations and ratings can be substitutes as well another aspect we looked at is whether the type of the
00:04:24
product matters so we classified all the products in our data set into what we call utilitarian products and hedonic
00:04:31
products so utilit ate it utilitarian products are products that serve some functional purpose for example
00:04:38
appliances or groceries and hedonic products are products that don't serve a
00:04:43
functional purpose and really appeal to some sensory perception for example jewelry and we found that
00:04:50
recommendations have a you know low to moderate impact for utilitarian products but it is for the hedonic products that
00:04:57
they have a very significant impact and these hedonic products you know things like jewelry where we don't really mean
00:05:04
it and when a recommendation kind of suggests that this is a great fit for us or people with similar tastes like this
00:05:11
product that really moves the needle in terms of making us respond to that recommendation we look at other things
00:05:18
like denote a description of the product matters the price does it matter and ute
00:05:22
and we find what you would expect you know people respond to recommendations for lower price products and higher
00:05:28
price products and where there's better description for the product and if there
00:05:32
is very limited description so there are two conclusions that surprised us that we didn't expect a
00:05:43
priori one was that recommendations don't necessarily help us discover niche
00:05:50
products and that is interesting because there has been a lot of discussion for at least a decade now about how online
00:05:58
systems whether it's search engines or personalized recommendations they will
00:06:02
help us find niche items and they will help benefit you know what we call the long tail the products that are not
00:06:11
super popular that almost don't get produced that may get produced but don't
00:06:16
really sell much and the promise of recommendation systems is they really give a fair opportunity for these kinds
00:06:25
of products and we find that for the common designs it doesn't happen and that's pretty surprising there are some
00:06:32
designs where you you make design modifications and you favor nisha items you can make it work but most common
00:06:39
designs that are used at most retailers they don't do that and we found that you
00:06:44
know it might have the opposite effect now another result that surprised us was that recommendations and ratings or
00:06:51
substitutes again our priori we expected that you know people will respond to recommendations when they are highly
00:06:59
rated and we did find that when they're highly rated people respond to recommendations but what surprised us
00:07:04
was that their responses even greater when the products have a lower rating and so that suggests that there's this
00:07:12
effect of recommendations as substitutes for ratings and we hadn't predicted that
00:07:18
beforehand so our research has implications for retailers for producers and even
00:07:26
consumers for retailers to the extent that their strategy is to offer a wide product assortment is our research
00:07:33
suggests that the choices of technology they make may not always be consistent with that strategy and they need to
00:07:40
think harder about the technology choices for example Amazon strategy is that you can find any product on earth
00:07:46
at Amazon and really wide product assortment is its strategy similarly many online retailers also offered white
00:07:54
product assortment so our research suggests that if you offer great product assortment you also need to think about
00:08:00
how will consumers discover that white product assortment and recommendations are an important part of the solution
00:08:05
but they don't often work in practice because they have this bias towards products that have been bought before
00:08:10
and that have been rated before and so our research suggests that they need to think about technology choice and think
00:08:17
about how to modify this common design so it is consistent with their product choices in general for producers you
00:08:26
know research shows that recommendations and similar algorithms they drive consumer choice in a big way so
00:08:32
producers need to think hard about how the game discovered by these algorithms today producers are used to thinking
00:08:40
about how do I how does our product get discovered by consumers they need to also ask how do how does our product get
00:08:47
discovered by algorithms and for consumers we find that these systems are great at helping us as individuals
00:08:54
discover new products but at the aggregate level we are not seeing that diversity which is not necessarily
00:09:00
troublesome for consumers but it does suggest that there are products out there that could be this needle in the
00:09:06
haystack perfect product for you which may not be surfaced by recommendations so one has to be open to other sources
00:09:13
of discovery as well so there's a lot of talk these days about big data and analytics and how
00:09:23
there's so much data and companies are building intelligent algorithms that can
00:09:27
help them be smart that can help consumers find products they like and so on and our research shows that these
00:09:34
efforts do work but at the same time we have to be cautious about unintended consequences so for example the idea of
00:09:44
recommendations is that they help us find novel items but we don't want them
00:09:48
to have biases built into them where they favor certain kinds of items versus others and to the extent they favor
00:09:54
certain kinds of items that might have an unintended consequence and we can think about that say in the context of
00:10:00
news if we all consume news through personalized recommendations we may not always get that breadth of perspective
00:10:08
we want and so algorithms might be driving a lot of our choice with media and we need to think hard about how big
00:10:15
data and algorithms can be be biased and and largely they work but they do have some biases we need to be cautious about
00:10:28
so one of the things that's really novel about our work is that our research is
00:10:34
informed by really large-scale data and analysis of that data there have been a lot of theories about how
00:10:42
recommendations impact consumer choice what kinds of products they favor when consumers respond to them and so on in
00:10:49
practice there had been very little empirical evidence and that's partly because in order to answer these
00:10:55
questions for example do they favor niche items or mainstream items we need a contrast between people exposed to
00:11:02
recommendations and unexposed to recommendations and for most retailers they observe consumers only after they
00:11:09
come to their website and are exposed to recommendations and so our study is based on and an experiment that was done
00:11:17
with a large retailer in North America with whom we ran an a/b experiment where some people got recommendations some did
00:11:25
not and we ran this for hundreds of different product categories and so we were able to not only get that contrast
00:11:33
needed to answer the question but we are also able to generalize beyond a single
00:11:37
product category because we had so many different products and that I think is one of the things that sets this
00:11:44
research apart which is that it's based on data it's based on concrete evidence
00:11:48
and it's based on a large enough sample point and a large very representative
00:11:53
sample of consumers so in terms of next steps we're trying to generalize some of our results beyond
00:12:04
recommendations and think about all kinds of search tools we have online and so they include social tools and social
00:12:10
news feeds for example on Facebook we discover products it includes search engines we're also trying to understand
00:12:17
this world from the producers perspective as I mentioned you know there are implications for producers and
00:12:24
producers need to think hard about how consumers will discover their products through algorithms and so they need to
00:12:32
think about discovery by the algorithms as well and there's very limited understanding of what is it about a
00:12:38
product that makes a recommendation pick that product among thousands of potential candidates and so we're trying
00:12:45
to study that and hopefully we'll provide some insights to producers so that they can be active participants in
00:12:52
helping their products get discover rather than passive observers you