00:00:01
we're here today with Wharton senior fellow Chandra Hill to talk about her new research which focuses on TV ads
00:00:07
online search and the connections between them Chandra thank you for being here today
00:00:11
thank you for having me first of all could you give us a short description of what you looked at in this research
00:00:16
absolutely so what we aim to achieve is to find new ways to measure tv-out effectiveness so let me take a step back
00:00:24
and kind of talk a little bit about how people typically measure effectiveness large brand advertisers will usually ask
00:00:31
another company to survey consumers and ask them questions like did you see the ad would you like to recommend the
00:00:40
product that was advertised to your friends how did you feel about the ad so questions about their attitudes they
00:00:47
might also look at sales data and correlate that with the amount of spent that they've made what we hope to do is
00:00:53
look at more granular data that reveals itself in the searches people post on large search engines and so what we're
00:01:00
hoping to do or have done is link TV ad data at an aggregate level where they you know can tell us precisely which
00:01:08
television show what time which locations an ad was shown and then we look at search data around that TV ad
00:01:16
before and after to see whether there was an impact on the search behavior and what we're actually trying to do is look
00:01:25
at the ability to coordinate advertising efforts so not just on television but also on digital platforms like sponsored
00:01:32
search so what we do is combine data from TV ads so the location of those ads so where they were shown not just the
00:01:40
location in terms of geography but also which shows they were shown in and then we link that to the search data not just
00:01:46
the searches but also conditioned on somebody making a search did they click on a sponsored search ad or not and we
00:01:53
combine data from all of these sources to make causal claims about the impact of TV ads on digital behaviors towards
00:02:01
measuring the effectiveness of TV ads so this features kind of capitalizes on a phenomenon that's going on then that
00:02:07
last couple years called second screaming in that no longer do we watch TV and just look at the TV but often
00:02:13
we're sitting on the couch looking at the TV and then also scrolling through our phones the whole
00:02:18
time so tell me like when you looked at this a little more closely what were some of the key takeaways that you found
00:02:22
um so first of all that's a great observation and I should probably take a step back and tell you the research
00:02:28
questions that we were interested in so the number one research question we're
00:02:31
interested in is just that like how does behaviors in response to TV ad manifest
00:02:38
themselves via these second screens and what we found was that the response that
00:02:44
we were seeing so we do in fact see that there's an increase in search behavior
00:02:47
after a TV ad is shown but that's manifesting itself primarily on smartphones so the smaller device the
00:02:54
more likely someone is to respond directly after a TV ad digitally and we also were interested in because we have
00:03:03
very granular level search data not just in you know whether people are searching
00:03:10
more but as I mentioned this interaction with the sponsored search ads and then finally we wanted to look at how the TV
00:03:19
ads impacted different users in various ways so for instance we're interested in
00:03:25
heterogeneous effects on demographics so age and gender so do certain genders respond differently to a particular
00:03:32
creative that's shown in a particular television show similarly we looked at device and that's
00:03:39
how we were able to discern that the response was coming primarily from the mobile phone so what were some of the
00:03:47
findings that were most surprising to you I know one thing that I kind of stood out to me is this idea that you
00:03:51
found that really when people are when this increase in searching on your phone is going on it's basically only amounts
00:03:57
to about three minutes that's right so you hit the nail on the head in terms of
00:04:01
that three surprising I mean well the surprising findings so there were two that I think are obvious in hindsight
00:04:08
but we didn't necessarily anticipate so the first one was one that we already
00:04:12
talked about by disaggregating the data and looking at different cohorts of people searching from smart phones
00:04:18
versus tablets versus PCs we were able to see that the significant effect in terms of the bounce and searches after
00:04:25
TV ad was happening only on mobile phone right so that's the first thing that was
00:04:30
surprising to us although in hindsight it makes sense right if you're sitting
00:04:33
in front of the television you're not going to bring your desktop - you know
00:04:36
probably watch a television right and then the second one was because we're looking at very fine-grained windows so
00:04:42
really for the first time are we doing this sort of granular response to TV ads minute by minute we were able to see the
00:04:49
dynamic change in how people search after TV ad and as you mentioned we found that really we're seeing it either
00:04:56
in the first second or third minute after the TV on and at first were like wow we expected this thing to sort of
00:05:02
slow down but maybe tell off and the reason is we suspect is that TV ad segments are almost always exactly three
00:05:09
minutes and so people are probably switching their attention back to the television show after the TV and TV ads
00:05:16
are aired so if I am an advertiser you're looking at you have this three minute window people are on their phones
00:05:22
they're looking at these ads what can i what can I do with this information how
00:05:26
can I use this and maybe synchronize first of all are people even synchronizing now with in this way like
00:05:32
trying to make sure if someone sees an ad on TV they may also see it on mobile and if not what can I do to kind of
00:05:37
capitalize on this information that you find right so the implications of our work I think are many right so the first
00:05:45
one is that because we're finding that the search response to television ads is
00:05:50
manifesting itself primarily on mobile phones and from prior research not ours we know that people are more likely to
00:05:58
click on the first ad only on a mobile phone when compared to say a PC or desktop and that's primarily because of
00:06:06
the footprint right so you only see the first ad so what that suggests is if people really are moving to mobile phone
00:06:12
when they're watching television that if you're an advertiser and you really want
00:06:16
to keep their attention you should spend the money to make sure you're the first
00:06:20
ad that shows up for the advertiser so that's the first one but then I think
00:06:25
the work has even broader implications so because we can see who is responding right so let's just say let's take two
00:06:35
examples let's say you have only one ad creative like one TV ad one commercial
00:06:40
and you want to know sort of for this let's say it's a new product who's responding
00:06:46
you can launch that TV on and basically you know look at the response in the way
00:06:51
that we have and see which types of customers are responding and we're like which geographies are responding and
00:06:57
that can help you sort of optimize your other your other advertising efforts to do more here or less there depending
00:07:05
upon what you find the other example I wanted to point out is if instead you have many ad creatives you don't know in
00:07:12
advance like maybe you've done some focus test and you know which one small groups like but you don't know in
00:07:17
advance what the broader audience will respond best to you can launch all four of those or however many ads and see who
00:07:28
is responding the best and then you know adjust how you present those ads over time so what this approach allows for is
00:07:36
to do near real time optimization of ads with very aggregate level data now you asked a question of like what are people
00:07:44
doing now right so for the most part people are measuring advertising effectiveness in the ways that I
00:07:50
mentioned when we first started so asking you know people via surveys did they see the ad or looking at sales data
00:07:57
but the future is quite different in that now there are solutions for TV networks and and even sort of solutions
00:08:07
that sit outside of TV networks that allow people to buy advertisement programmatically so right now we're not
00:08:14
all the way there so there's gonna be programmatic buys that more advertisers
00:08:20
will do as well as something called addressable TV where people can actually advertise to individual households that
00:08:26
they know information right so if you could do that then looking at this aggregate level data kind of is
00:08:32
unnecessary but until we get there this way is a good way to begin to optimize campaigns this sort of is a beginning of
00:08:39
saying that my Pretty Little Liars crowd is maybe a little different than my scandal crowd which is maybe different
00:08:44
than my Monday Night Football exactly now speaking of that what were there some interesting things that you found
00:08:49
in terms of demographic differences and how people reacted to this we did and that was one thing that we thought would
00:08:55
be interested to advertisers so because you can see who's responding with respect to
00:09:01
we looked at really just age and gender but still that's enough to give an advertiser insights we could match the
00:09:09
demographics of the TV show for instance if you look at sporting events those tend to skew male and then ask when an
00:09:16
advertisement is shown in a sporting event who what audience members are most likely to respond and we found perhaps
00:09:24
obviously in hindsight that when a TV ad is shown in a sporting event men are much more likely to respond to it then
00:09:32
when an ad is shown in a sporting event women really don't respond more than
00:09:37
they would otherwise so the idea is maybe that your ads the the audience you might want to go after with these ads
00:09:43
are the people that are already watching anyway which I assume they know but then
00:09:47
also that it transfers over to online searches as well that's right and so and
00:09:52
you can just check right so two things right so one you can check that the people that you're targeting are
00:09:57
actually the ones responding so that's kind of like a validity check that your
00:10:01
strategy is a good one but then in addition to that if you have two types of shows that let's just say
00:10:08
men because we use that example are likely to watch you can compare and see like which type of show when an ad is
00:10:15
placed in it are men most likely to respond to that ad because there could be all kinds of things going on perhaps
00:10:21
in some shows people are more engaged with the show and are less likely to turn away from the commercials for
00:10:26
instance or you know get up and maybe they're a longer show and they get up and do other things so you can look at
00:10:34
the match between the type of show and the audience that's responding and it
00:10:38
has two implications now just this research play into also the idea that more and more people are maybe turning
00:10:44
away from broadcast TV and going more towards streaming for example because I mean I know when I watch Hulu for
00:10:50
example because I have the kind where you do get ads with it is that it's asking me do I want the experience of
00:10:55
this or do I want here or do I want to learn about do I want to travel video about California or do I want something
00:11:00
about a cleaner I mean can this also be applied to other like be on broadcast TV
00:11:05
so it can be applied to other advertising strategies where you have a specific time stamp associated
00:11:15
with the event so that could be you know sort of placing a billboard in a particular location and then taking it
00:11:22
away it could be a you have a radio advertisement and it has a certain time so the type of methodology that we
00:11:28
actually advocate for is one that allows us to tease out the causality between an
00:11:33
event that has a specific time and behavior that happens you know after that event by comparing it not only to
00:11:40
what happened before but also to some control group that we come up with but so any event based advertising it would
00:11:48
work but to answer your question about Hulu and let's say Netflix those those
00:11:53
solutions for sort of media consumption or a little bit different in that they know who you are right like they have
00:12:00
your information so what they can do is closer to the addressable television example that I mentioned earlier where
00:12:09
people can sort of now advertise directly to individual household so companies like Hulu and Netflix have the
00:12:17
ability already to do sort of one-to-one advertising and they can use your your behavior either on their own site or by
00:12:23
matching their data with third parties to target to you directly now what is their guess what sets this
00:12:31
research apart from other research that's been conducted on this topic so there are a few things that set it apart
00:12:36
right so the one thing is the granularity of the data so because of the scale of the data we were able to
00:12:42
look at minute by minute response by different locations so that's one thing
00:12:48
the second thing is that for these searchers we also have as I mentioned demographics very crude high level
00:12:55
demographics but we were able to then look at these heterogeneous treatment effects at four demographics and then
00:13:01
also by device type which no one's done before and then finally this combination
00:13:06
between not just looking at search response but looking at the clicks so conditioned on a search looking at the
00:13:13
sponsored ad clicks is something that's also novel so looking at how firms might
00:13:17
begin to coordinate their advertising efforts something that hasn't yet been done
00:13:23
before when looking at response to TV ads and what's next for this research I
00:13:27
know you've done a lot with social TV in the past couple years but where were you
00:13:30
gonna go with this next so there are a number of sort of obvious extensions so we want to well and we are have already
00:13:36
started to bring in just organic search so we focused in the first paper on sponsored search and we can look at well
00:13:43
you know when an organic search response actually is for the brands or not like does that make a difference in their
00:13:50
likelihood to click after an ad and we're also looking at other types of digital responses not just search so
00:13:59
we're looking at clicks on webpages associated with the brand and also looking at where people are coming in
00:14:06
from when they make those clicks asking questions around which specific advertising platforms might be most
00:14:13
effective right after a TV ad campaign and now after sort of those things that we're already working on what we plan to
00:14:22
look at are assigning people to different categories so instead of thinking about it instead of assigning
00:14:30
them to demographics like male or female we assign them to a place on the purchase funnel so are they ready to buy
00:14:36
are they just seeking information are they doing comparison shopping and we want to know whether the TV ad is more
00:14:43
or less effective depending upon where people are in that purchase funnel other things that we'd like to do but we'll
00:14:53
you know need some convincing for a partner or we have used observational data techniques and we feel pretty
00:15:00
strongly that our method for teasing out the causality the relation the causal relationship between the TV ads and the
00:15:08
search response is pretty solid however what we'd really like to do is run an
00:15:13
experiment while TV ads are running to make sure that what we're finding for
00:15:19
this with the sponsored search results are it's really true that in fact there
00:15:23
is the impact on sponsored search results that we're seeing and I guess the pie in the sky you know kind of
00:15:30
future work would be to actually run experiments using addressable TV right so our work I think will last
00:15:37
for quite a while because although addressable TV exists today they're not that many advertisers that have adopted
00:15:46
right away but what we want to see again is whether these different combinations
00:15:52
of advertising lead to sort of more or less spend more or less clicks more or less search for information and in using
00:16:02
addressable TV solutions in combination with experiment on sponsored research you can get precisely at that answer
00:16:11
Chandra thanks so much for being with us today thanks Rito you [Music]