00:00:04
and so my research deals with online
00:00:07
advertising attribution and it's easier
00:00:09
to explain it by giving an example so
00:00:12
which most consumers actually know so
00:00:14
let's suppose you went on a brand's
00:00:17
website let's a gap and you put into a
00:00:19
shopping cart a shirt and you didn't buy
00:00:21
the shirt and you don't continue to
00:00:23
surfing the internet what would happen
00:00:24
typically that you'll start seeing ads
00:00:26
for the same shirt again and again and
00:00:27
again on every website now the question
00:00:30
from the advertiser's perspective which
00:00:32
is gaps gaps retail is if in the end the
00:00:36
consumer did buy the shirt how do you
00:00:38
measure the effectiveness of each one of
00:00:40
those ad exposures the consumer head and
00:00:42
this is called attribution problem now
00:00:45
what most advertisers do today is they
00:00:48
use something called less touch
00:00:50
attribution or last click attribution
00:00:51
and the idea is very simple if me is a
00:00:54
consumer and I saw let's say 17
00:00:56
different ads and 17 different websites
00:00:58
and then purchase the product the last
00:01:00
ad they saw before purchasing the
00:01:01
product is going to get all of the
00:01:02
credit for this sale ok and motley me
00:01:05
research is trying to do is you trying
00:01:07
to determine is this method efficient or
00:01:10
not and what is this impact of this
00:01:11
method on different types of advertisers
00:01:14
and different types of publishers in the
00:01:17
online marketplace
00:01:22
in my research what I try to say is I
00:01:24
try to ask is this last click method
00:01:26
which is used by the majority of
00:01:28
advertisers efficient and why do they
00:01:30
use this method because it's a bit
00:01:32
counterintuitive why would they look
00:01:34
only the last add that the person sees
00:01:36
and what I do is I compared two types of
00:01:39
compensation schemes which are used on
00:01:41
line one is paying for every exposure of
00:01:43
an ad being seen and one is paying just
00:01:45
for every sale like a commission on a
00:01:47
sale that the publisher gets from the
00:01:48
advertiser so my key takeaway is that
00:01:51
actually paying for exposures is more
00:01:54
efficient and more profitable for the
00:01:56
advertisers than paying a commission on
00:01:58
sales she's very counterintuitive much
00:02:00
people think if you compensate a sales
00:02:02
person you should pick them a commission
00:02:04
only when they make sales and not when
00:02:06
they show off work but if you compensate
00:02:08
a publisher let's say Facebook and you
00:02:10
can pay Facebook for just showing the
00:02:12
ads or actually generating sales
00:02:14
actually my key takeaway is that paying
00:02:17
for showing an ads is more efficient and
00:02:20
more profitable for the advertiser the
00:02:22
reason being is that if you have more
00:02:24
than one publisher let's say Facebook
00:02:25
and Google are competing what you will
00:02:28
see is that there is going to be free
00:02:29
writing when one of them on the other so
00:02:32
because of this class click method are
00:02:33
going to compete among one another etc
00:02:35
and one of them is going to claim
00:02:37
they're gonna be very very very
00:02:39
efficient although maybe to just show
00:02:40
the last ads etc so this is one key
00:02:43
takeaway the second one is that the last
00:02:45
click method creates this competition
00:02:47
between publishers that creates kind of
00:02:52
a race to try and cheat the advertiser
00:02:54
out of money which is a bit strange but
00:02:57
the best way to explain it is just to
00:02:59
try and think what would you do if I
00:03:01
told you that the last add your show
00:03:03
gets the commission and you know a
00:03:05
consumer comes to your website and
00:03:07
there's a high probability the consumer
00:03:09
is going to buy the product and there's
00:03:10
another consumer that comes to the
00:03:12
website and you don't know if the
00:03:13
consumer is going to buy the product to
00:03:14
which consumer will you show that you
00:03:16
will show that the consumer is going to
00:03:18
buy anyway and just bomb the consumer
00:03:20
with ads to try to be the last one so
00:03:23
the other key takeaway is that when
00:03:24
you're using last click or last touch
00:03:26
methods all of the measurements and all
00:03:29
of the metrics you're getting are going
00:03:30
to be maximized you're going to get a
00:03:31
lot of sales through a lot of channels
00:03:33
and publishers but maybe they didn't
00:03:35
cause
00:03:36
any of those sales or just happen to be
00:03:38
the last ones
00:03:42
so the first conclusion which surprised
00:03:44
me what actually that paying for
00:03:46
impressions or exposures is more
00:03:48
profitable than paying for commissions
00:03:50
because if you ask any advertiser online
00:03:51
they say I want to pay only Commission I
00:03:53
went apparently when sales are being
00:03:55
generated and also this was my intuition
00:03:57
when I started this research and this
00:03:59
was very surprising to find and when I
00:04:01
tell this even to managers and
00:04:03
executives today they basically don't
00:04:05
believe me although I can show them the
00:04:06
math and show that it just works the
00:04:08
other surprising finding was that last
00:04:10
click and last touch is not necessarily
00:04:12
a bad method per se under some
00:04:15
conditions it increases the profits of
00:04:17
the advertisers but these conditions are
00:04:19
quite limited so if you ask advertisers
00:04:23
why do you use last click and last touch
00:04:25
typically they say it's just the best
00:04:27
method they know so far but I can show
00:04:29
that if it happens that your campaign
00:04:32
has these specific conditions it's
00:04:34
actually better to use last touch this
00:04:36
conditions are very specific is
00:04:38
typically when the first ad has the
00:04:40
biggest impact so there's a huge impact
00:04:42
to the first ad and not that not that
00:04:43
many to the other ones and consumers
00:04:45
don't visit the websites too often then
00:04:48
the last click method is very very
00:04:49
efficient
00:04:54
one takeaway you can take is that
00:04:57
realizing that whatever attribution
00:04:59
method you're going to use over time the
00:05:01
publishers are going to learn how to
00:05:02
maximize this metric so even if you told
00:05:05
the publishers I'm going to compensate
00:05:07
you according to who runs the fastest
00:05:09
someone will be the fastest runner and
00:05:11
they're going to be the best maximized
00:05:13
method so the idea is to try and sit
00:05:16
down with the publishers and with the
00:05:18
company you're using for doing
00:05:19
attribution and try to think is the
00:05:21
model that they're using makes sense so
00:05:23
does the model make sense to my
00:05:24
consumers and the other thing you need
00:05:26
to do is you need to be very carefully
00:05:29
design an experiment and say what would
00:05:32
happen when using a new attribution
00:05:33
method and what would happen without
00:05:35
using it you can just move to a new
00:05:38
method see all of the metrics maximized
00:05:41
and say oh I just made more money
00:05:42
because probably this isn't correct
00:05:48
so one interesting misperception it's
00:05:51
there's these ads color retargeting ads
00:05:53
these are very specific ads that you
00:05:55
only start seeing them after you put
00:05:57
let's say a product in the shopping bag
00:05:59
it in the industry they believe these
00:06:02
ads are very very useful and actually
00:06:04
are are very good at reminding the
00:06:06
consumer to come back to let's say gaps
00:06:08
website and purchase this shirt that
00:06:10
consumer was pondering to purchase and
00:06:14
my research showed that actually most of
00:06:17
these retargeting ads are probably very
00:06:19
inefficient and have no impact on the
00:06:21
consumers the only thing is that they're
00:06:24
being shown last before purchase because
00:06:26
if a consumer went to a website and put
00:06:29
something in the shopping cart there's
00:06:31
much higher probability the consumer is
00:06:32
going to buy the product anyway so if
00:06:34
you start showing that consumer ads
00:06:36
compared to consumers who just didn't
00:06:38
visit the website and they didn't put
00:06:40
anything at the shopping cart it would
00:06:41
appear you have a huge lift a huge
00:06:43
increase in sales on those consumers you
00:06:45
started showing ads too but even if you
00:06:48
don't show them as they will buy anyway
00:06:49
that's the point so the misperception is
00:06:53
basically if you don't run an experiment
00:06:55
and have a control group and says when
00:06:57
these consumers who visited my website
00:06:59
on half of them I do show ads on the
00:07:02
others I don't show ads and only then I
00:07:03
measure the difference you're going to
00:07:05
think that retargeting is a very very
00:07:08
useful channel and typically it or not
00:07:10
that but not that good
00:07:16
my method is unique in that I use game
00:07:18
theory and I actually did mostly in
00:07:20
analytical and theoretical analysis of
00:07:22
this problem and only then I tested it
00:07:24
on the data all of the other research on
00:07:26
this topic used a very empirical
00:07:28
approach that took a set of data they
00:07:30
ran algorithms on it and they try to
00:07:32
figure out which publisher is better
00:07:34
than the other one what I try to say is
00:07:37
I try to say what is the market using
00:07:39
currently in measurement and
00:07:40
compensation what would be the
00:07:42
consequences and can we observe that in
00:07:44
data and to that I basically found out
00:07:47
that a lot of the empirical applications
00:07:49
and measurements are basically incorrect
00:07:50
they assume that the market is efficient
00:07:54
in behaving correctly but if you do the
00:07:56
theoretical analysis you will see that
00:07:58
you can figure out from the data if the
00:08:00
market is inefficient or efficient and
00:08:02
in that case it's not very efficient the
00:08:04
publishers are basically showing ads to
00:08:06
consumers that will buy anyway which is
00:08:08
called adverse selection or essentially
00:08:10
trying to cheat the publisher while the
00:08:13
other empirical applications cannot
00:08:14
differentiate between really showing ads
00:08:17
to consumers who are influenced to
00:08:18
consumers who just would buy anyway and
00:08:20
you kind of stick them with ads to get
00:08:22
the Commission
00:08:27
after working on these three shots I
00:08:29
started getting a lot of communication
00:08:32
from companies saying hey Ron this is
00:08:34
very interesting and there's this
00:08:35
approach that you're suggesting but
00:08:37
there's at least 10 other companies that
00:08:39
claim their attribution product is
00:08:41
better or is the best how do we compare
00:08:44
how can we even tell which one is better
00:08:47
etc so one idea i'm trying to push and
00:08:50
this would be probably with a few
00:08:52
companies to experiment with is to try
00:08:54
to build a system where you generate
00:08:57
advertising data and you try different
00:09:00
attribution methods and you see which
00:09:01
one just matches this company type of
00:09:03
customer and the reason this is
00:09:05
interesting is that most a tribunal
00:09:07
companies don't actually publish size a
00:09:09
lot very transparent about the
00:09:10
algorithms which are say we have the
00:09:12
best algorithm and if you give us data
00:09:14
we can give you the results what we
00:09:16
won't really tell you what's going on so
00:09:18
this approach where you kind of test
00:09:20
through many of them and kind of run a
00:09:22
competition between all of them is going
00:09:24
to hopefully tell us which method works
00:09:26
better or not that we can understand
00:09:28
this process much much better
00:09:45
you
