00:00:01
we're here today with Wharton senior
00:00:03
fellow Chandra Hill to talk about her
00:00:05
new research which focuses on TV ads
00:00:07
online search and the connections
00:00:09
between them
00:00:10
Chandra thank you for being here today
00:00:11
thank you for having me first of all
00:00:13
could you give us a short description of
00:00:15
what you looked at in this research
00:00:16
absolutely so what we aim to achieve is
00:00:20
to find new ways to measure tv-out
00:00:22
effectiveness so let me take a step back
00:00:24
and kind of talk a little bit about how
00:00:26
people typically measure effectiveness
00:00:28
large brand advertisers will usually ask
00:00:31
another company to survey consumers and
00:00:35
ask them questions like did you see the
00:00:37
ad would you like to recommend the
00:00:40
product that was advertised to your
00:00:42
friends how did you feel about the ad so
00:00:44
questions about their attitudes they
00:00:47
might also look at sales data and
00:00:48
correlate that with the amount of spent
00:00:50
that they've made what we hope to do is
00:00:53
look at more granular data that reveals
00:00:55
itself in the searches people post on
00:00:58
large search engines and so what we're
00:01:00
hoping to do or have done is link TV ad
00:01:03
data at an aggregate level where they
00:01:05
you know can tell us precisely which
00:01:08
television show what time which
00:01:11
locations an ad was shown and then we
00:01:14
look at search data around that TV ad
00:01:16
before and after to see whether there
00:01:19
was an impact on the search behavior and
00:01:21
what we're actually trying to do is look
00:01:25
at the ability to coordinate advertising
00:01:27
efforts so not just on television but
00:01:30
also on digital platforms like sponsored
00:01:32
search so what we do is combine data
00:01:34
from TV ads so the location of those ads
00:01:38
so where they were shown not just the
00:01:40
location in terms of geography but also
00:01:42
which shows they were shown in and then
00:01:44
we link that to the search data not just
00:01:46
the searches but also conditioned on
00:01:49
somebody making a search did they click
00:01:51
on a sponsored search ad or not and we
00:01:53
combine data from all of these sources
00:01:55
to make causal claims about the impact
00:01:58
of TV ads on digital behaviors towards
00:02:01
measuring the effectiveness of TV ads so
00:02:04
this features kind of capitalizes on a
00:02:06
phenomenon that's going on then that
00:02:07
last couple years called second
00:02:08
screaming in that no longer do we watch
00:02:11
TV and just look at the TV but often
00:02:13
we're sitting on the couch
00:02:15
looking at the TV and then also
00:02:17
scrolling through our phones the whole
00:02:18
time so tell me like when you looked at
00:02:20
this a little more closely what were
00:02:21
some of the key takeaways that you found
00:02:22
um so first of all that's a great
00:02:24
observation and I should probably take a
00:02:26
step back and tell you the research
00:02:28
questions that we were interested in so
00:02:30
the number one research question we're
00:02:31
interested in is just that like how does
00:02:35
behaviors in response to TV ad manifest
00:02:38
themselves via these second screens and
00:02:41
what we found was that the response that
00:02:44
we were seeing so we do in fact see that
00:02:45
there's an increase in search behavior
00:02:47
after a TV ad is shown but that's
00:02:49
manifesting itself primarily on
00:02:51
smartphones so the smaller device the
00:02:54
more likely someone is to respond
00:02:56
directly after a TV ad digitally and we
00:03:01
also were interested in because we have
00:03:03
very granular level search data not just
00:03:07
in you know whether people are searching
00:03:10
more but as I mentioned this interaction
00:03:13
with the sponsored search ads and then
00:03:15
finally we wanted to look at how the TV
00:03:19
ads impacted different users in various
00:03:23
ways so for instance we're interested in
00:03:25
heterogeneous effects on demographics so
00:03:28
age and gender so do certain genders
00:03:31
respond differently to a particular
00:03:32
creative that's shown in a particular
00:03:35
television show
00:03:36
similarly we looked at device and that's
00:03:39
how we were able to discern that the
00:03:42
response was coming primarily from the
00:03:44
mobile phone so what were some of the
00:03:47
findings that were most surprising to
00:03:49
you I know one thing that I kind of
00:03:50
stood out to me is this idea that you
00:03:51
found that really when people are when
00:03:54
this increase in searching on your phone
00:03:55
is going on it's basically only amounts
00:03:57
to about three minutes that's right so
00:04:00
you hit the nail on the head in terms of
00:04:01
that three surprising I mean well the
00:04:03
surprising findings so there were two
00:04:05
that I think are obvious in hindsight
00:04:08
but we didn't necessarily anticipate so
00:04:10
the first one was one that we already
00:04:12
talked about by disaggregating the data
00:04:14
and looking at different cohorts of
00:04:17
people searching from smart phones
00:04:18
versus tablets versus PCs we were able
00:04:21
to see that the significant effect in
00:04:23
terms of the bounce and searches after
00:04:25
TV ad was happening only on mobile phone
00:04:28
right so that's the first thing that was
00:04:30
surprising to us although in hindsight
00:04:32
it makes sense right if you're sitting
00:04:33
in front of the television you're not
00:04:35
going to bring your desktop - you know
00:04:36
probably watch a television right and
00:04:38
then the second one was because we're
00:04:40
looking at very fine-grained windows so
00:04:42
really for the first time are we doing
00:04:44
this sort of granular response to TV ads
00:04:47
minute by minute we were able to see the
00:04:49
dynamic change in how people search
00:04:51
after TV ad and as you mentioned we
00:04:54
found that really we're seeing it either
00:04:56
in the first second or third minute
00:04:58
after the TV on and at first were like
00:04:59
wow we expected this thing to sort of
00:05:02
slow down but maybe tell off and the
00:05:04
reason is we suspect is that TV ad
00:05:06
segments are almost always exactly three
00:05:09
minutes and so people are probably
00:05:11
switching their attention back to the
00:05:14
television show after the TV and TV ads
00:05:16
are aired so if I am an advertiser
00:05:19
you're looking at you have this three
00:05:21
minute window people are on their phones
00:05:22
they're looking at these ads what can i
00:05:25
what can I do with this information how
00:05:26
can I use this and maybe synchronize
00:05:28
first of all are people even
00:05:30
synchronizing now with in this way like
00:05:32
trying to make sure if someone sees an
00:05:34
ad on TV they may also see it on mobile
00:05:35
and if not what can I do to kind of
00:05:37
capitalize on this information that you
00:05:39
find right so the implications of our
00:05:42
work I think are many right so the first
00:05:45
one is that because we're finding that
00:05:47
the search response to television ads is
00:05:50
manifesting itself primarily on mobile
00:05:52
phones and from prior research not ours
00:05:55
we know that people are more likely to
00:05:58
click on the first ad only on a mobile
00:06:01
phone when compared to say a PC or
00:06:04
desktop and that's primarily because of
00:06:06
the footprint right so you only see the
00:06:08
first ad so what that suggests is if
00:06:10
people really are moving to mobile phone
00:06:12
when they're watching television that if
00:06:14
you're an advertiser and you really want
00:06:16
to keep their attention you should spend
00:06:18
the money to make sure you're the first
00:06:20
ad that shows up for the advertiser so
00:06:23
that's the first one but then I think
00:06:25
the work has even broader implications
00:06:29
so because we can see who is responding
00:06:32
right so let's just say let's take two
00:06:35
examples let's say you have only one ad
00:06:37
creative like one TV ad one commercial
00:06:40
and
00:06:42
you want to know sort of for this let's
00:06:44
say it's a new product who's responding
00:06:46
you can launch that TV on and basically
00:06:48
you know look at the response in the way
00:06:51
that we have and see which types of
00:06:53
customers are responding and we're like
00:06:55
which geographies are responding and
00:06:57
that can help you sort of optimize your
00:06:59
other your other advertising efforts to
00:07:03
do more here or less there depending
00:07:05
upon what you find the other example I
00:07:08
wanted to point out is if instead you
00:07:10
have many ad creatives you don't know in
00:07:12
advance like maybe you've done some
00:07:14
focus test and you know which one small
00:07:16
groups like but you don't know in
00:07:17
advance what the broader audience will
00:07:21
respond best to you can launch all four
00:07:24
of those or however many ads and see who
00:07:28
is responding the best and then you know
00:07:30
adjust how you present those ads over
00:07:33
time so what this approach allows for is
00:07:36
to do near real time optimization of ads
00:07:39
with very aggregate level data now you
00:07:43
asked a question of like what are people
00:07:44
doing now right so for the most part
00:07:47
people are measuring advertising
00:07:49
effectiveness in the ways that I
00:07:50
mentioned when we first started so
00:07:52
asking you know people via surveys did
00:07:54
they see the ad or looking at sales data
00:07:57
but the future is quite different in
00:08:00
that now there are solutions for TV
00:08:04
networks and and even sort of solutions
00:08:07
that sit outside of TV networks that
00:08:09
allow people to buy advertisement
00:08:12
programmatically so right now we're not
00:08:14
all the way there so there's gonna be
00:08:17
programmatic buys that more advertisers
00:08:20
will do as well as something called
00:08:21
addressable TV where people can actually
00:08:23
advertise to individual households that
00:08:26
they know information right so if you
00:08:28
could do that then looking at this
00:08:30
aggregate level data kind of is
00:08:32
unnecessary but until we get there this
00:08:34
way is a good way to begin to optimize
00:08:37
campaigns this sort of is a beginning of
00:08:39
saying that my Pretty Little Liars crowd
00:08:41
is maybe a little different than my
00:08:42
scandal crowd which is maybe different
00:08:44
than my Monday Night Football exactly
00:08:46
now speaking of that what were there
00:08:48
some interesting things that you found
00:08:49
in terms of demographic differences and
00:08:51
how people reacted to this we did and
00:08:53
that was one thing that we thought would
00:08:55
be
00:08:55
interested to advertisers so because you
00:08:58
can see who's responding with respect to
00:09:01
we looked at really just age and gender
00:09:04
but still that's enough to give an
00:09:06
advertiser insights we could match the
00:09:09
demographics of the TV show for instance
00:09:12
if you look at sporting events those
00:09:14
tend to skew male and then ask when an
00:09:16
advertisement is shown in a sporting
00:09:18
event who what audience members are most
00:09:21
likely to respond and we found perhaps
00:09:24
obviously in hindsight that when a TV ad
00:09:26
is shown in a sporting event men are
00:09:30
much more likely to respond to it then
00:09:32
when an ad is shown in a sporting event
00:09:34
women really don't respond more than
00:09:37
they would otherwise so the idea is
00:09:40
maybe that your ads the the audience you
00:09:42
might want to go after with these ads
00:09:43
are the people that are already watching
00:09:45
anyway which I assume they know but then
00:09:47
also that it transfers over to online
00:09:49
searches as well that's right and so and
00:09:52
you can just check right so two things
00:09:54
right so one you can check that the
00:09:56
people that you're targeting are
00:09:57
actually the ones responding so that's
00:09:59
kind of like a validity check that your
00:10:01
strategy is a good one
00:10:03
but then in addition to that if you have
00:10:05
two types of shows that let's just say
00:10:08
men because we use that example are
00:10:10
likely to watch you can compare and see
00:10:12
like which type of show when an ad is
00:10:15
placed in it are men most likely to
00:10:17
respond to that ad because there could
00:10:19
be all kinds of things going on perhaps
00:10:21
in some shows people are more engaged
00:10:23
with the show and are less likely to
00:10:25
turn away from the commercials for
00:10:26
instance or you know get up and maybe
00:10:30
they're a longer show and they get up
00:10:31
and do other things so you can look at
00:10:34
the match between the type of show and
00:10:36
the audience that's responding and it
00:10:38
has two implications now just this
00:10:40
research play into also the idea that
00:10:42
more and more people are maybe turning
00:10:44
away from broadcast TV and going more
00:10:46
towards streaming for example because I
00:10:48
mean I know when I watch Hulu for
00:10:50
example because I have the kind where
00:10:52
you do get ads with it is that it's
00:10:53
asking me do I want the experience of
00:10:55
this or do I want here or do I want to
00:10:57
learn about do I want to travel video
00:10:58
about California or do I want something
00:11:00
about a cleaner I mean can this also be
00:11:02
applied to other like be on broadcast TV
00:11:05
so it can be applied to
00:11:08
other advertising strategies where you
00:11:11
have a specific time stamp associated
00:11:15
with the event so that could be you know
00:11:18
sort of placing a billboard in a
00:11:20
particular location and then taking it
00:11:22
away it could be a you have a radio
00:11:24
advertisement and it has a certain time
00:11:26
so the type of methodology that we
00:11:28
actually advocate for is one that allows
00:11:31
us to tease out the causality between an
00:11:33
event that has a specific time and
00:11:36
behavior that happens you know after
00:11:38
that event by comparing it not only to
00:11:40
what happened before but also to some
00:11:42
control group that we come up with but
00:11:44
so any event based advertising it would
00:11:48
work but to answer your question about
00:11:49
Hulu and let's say Netflix those those
00:11:53
solutions for sort of media consumption
00:11:56
or a little bit different in that they
00:11:58
know who you are right like they have
00:12:00
your information so what they can do is
00:12:04
closer to the addressable television
00:12:07
example that I mentioned earlier where
00:12:09
people can sort of now advertise
00:12:12
directly to individual household so
00:12:14
companies like Hulu and Netflix have the
00:12:17
ability already to do sort of one-to-one
00:12:19
advertising and they can use your your
00:12:21
behavior either on their own site or by
00:12:23
matching their data with third parties
00:12:27
to target to you directly
00:12:28
now what is their guess what sets this
00:12:31
research apart from other research
00:12:32
that's been conducted on this topic so
00:12:34
there are a few things that set it apart
00:12:36
right so the one thing is the
00:12:38
granularity of the data so because of
00:12:40
the scale of the data we were able to
00:12:42
look at minute by minute response by
00:12:44
different locations so that's one thing
00:12:48
the second thing is that for these
00:12:50
searchers we also have as I mentioned
00:12:52
demographics very crude high level
00:12:55
demographics but we were able to then
00:12:56
look at these heterogeneous treatment
00:12:59
effects at four demographics and then
00:13:01
also by device type which no one's done
00:13:03
before and then finally this combination
00:13:06
between not just looking at search
00:13:08
response but looking at the clicks so
00:13:11
conditioned on a search looking at the
00:13:13
sponsored ad clicks is something that's
00:13:15
also novel so looking at how firms might
00:13:17
begin to coordinate their advertising
00:13:20
efforts
00:13:21
something that hasn't yet been done
00:13:23
before when looking at response to TV
00:13:25
ads and what's next for this research I
00:13:27
know you've done a lot with social TV in
00:13:29
the past couple years but where were you
00:13:30
gonna go with this next so there are a
00:13:32
number of sort of obvious extensions so
00:13:34
we want to well and we are have already
00:13:36
started to bring in just organic search
00:13:39
so we focused in the first paper on
00:13:41
sponsored search and we can look at well
00:13:43
you know when an organic search response
00:13:46
actually is for the brands or not like
00:13:49
does that make a difference in their
00:13:50
likelihood to click after an ad and
00:13:53
we're also looking at other types of
00:13:56
digital responses not just search so
00:13:59
we're looking at clicks on webpages
00:14:01
associated with the brand and also
00:14:03
looking at where people are coming in
00:14:06
from when they make those clicks asking
00:14:08
questions around which specific
00:14:11
advertising platforms might be most
00:14:13
effective right after a TV ad campaign
00:14:16
and now after sort of those things that
00:14:20
we're already working on what we plan to
00:14:22
look at are assigning people to
00:14:25
different categories so instead of
00:14:28
thinking about it instead of assigning
00:14:30
them to demographics like male or female
00:14:32
we assign them to a place on the
00:14:34
purchase funnel so are they ready to buy
00:14:36
are they just seeking information are
00:14:39
they doing comparison shopping and we
00:14:41
want to know whether the TV ad is more
00:14:43
or less effective depending upon where
00:14:45
people are in that purchase funnel other
00:14:49
things that we'd like to do but we'll
00:14:53
you know need some convincing for a
00:14:54
partner or we have used observational
00:14:58
data techniques and we feel pretty
00:15:00
strongly that our method for teasing out
00:15:03
the causality the relation the causal
00:15:06
relationship between the TV ads and the
00:15:08
search response is pretty solid however
00:15:11
what we'd really like to do is run an
00:15:13
experiment while TV ads are running to
00:15:16
make sure that what we're finding for
00:15:19
this with the sponsored search results
00:15:20
are it's really true that in fact there
00:15:23
is the impact on sponsored search
00:15:25
results that we're seeing and I guess
00:15:27
the pie in the sky you know kind of
00:15:30
future work would be to actually run
00:15:33
experiments using addressable
00:15:34
TV right so our work I think will last
00:15:37
for quite a while because although
00:15:39
addressable TV exists today they're not
00:15:43
that many advertisers that have adopted
00:15:46
right away but what we want to see again
00:15:49
is whether these different combinations
00:15:52
of advertising lead to sort of more or
00:15:56
less spend more or less clicks more or
00:16:00
less search for information and in using
00:16:02
addressable TV solutions in combination
00:16:05
with experiment on sponsored research
00:16:07
you can get precisely at that answer
00:16:11
Chandra thanks so much for being with us
00:16:13
today thanks Rito
00:16:29
you
00:16:32
[Music]
