Download Super Returns to Super Bowl Ads?

Super Returns to Super Bowl Ads?∗
Seth Stephens-Davidowitz
[email protected]
Hal Varian
[email protected]
Michael D. Smith
[email protected]
February 20, 2015
Abstract
1
This paper uses a natural experiment—the Super Bowl—to study the
causal effect of advertising on demand for movies. Identification of
the causal effect rests on two points: 1) Super Bowl ads are purchased
before advertisers know which teams will play; 2) home cities of the
teams that are playing will have proportionally more viewers that
other cities. We find that the movies in our sample experience on
average incremental opening weekend ticket sales of about $8.4 million
from a $3 million Super Bowl advertisement.
2
3
4
5
6
7
8
9
∗
This paper benefited greatly from discussions with Randall Lewis, David Reiley, Bo
Cowgill, Lawrence Katz, and Lawrence Summers. We also thank participants at IO Fest
at Berkeley and the NBER Summer Institute for helpful comments.
1
10
The United States spends roughly 2% of its GDP on advertising (Galbi
11
[2008]). Not surprisingly, whether, when, and why advertising increases prod-
12
uct demand is of considerable interest to economists. However, empirically
13
measuring the impact of advertising is notoriously difficult. Products that
14
are heavily advertised tend to sell more, but this in itself does not prove
15
causation. (Sherman and Tollison [1971], Comanor and Wilson [1971]). A
16
particular product often sees an increase in sales after increasing its ad ex-
17
penditures, but here too the causation could run the other way. (Heyse and
18
Wei [1985], Ackerberg [2003]). For example, flower companies increase ad ex-
19
penditures in the weeks leading up to Valentine’s Day and see increased sales
20
around Valentine’s Day. But it is not easy to determine the causal impact
21
of that ad expenditure since many of the same factors that affect consumer
22
demand may also affect advertising purchase decisions (Schmalensee [1978],
23
Lee et al. [1996].)
24
Testing for causal effects requires an exogenous shock to ad exposures.
25
The gold standard, as always, is a randomized experiment. For this rea-
26
son, field experiments have become increasingly popular among economists
27
studying advertising (Simester et al. [2009], Bertrand et al. [2010], Lewis and
28
Rao [2012].) However, these experiments tend to be expensive and require
29
access to proprietary data. Moreover, they tend to have low power, usually
30
do not produce statistically significant effects, and have not led to consensus
31
on advertising effectiveness (Hu et al. [2007], Lewis and Reiley [2008], Lewis
32
and Rao [2012].)
2
33
Further, field experiments tend to involve a particular subset of ads: those
34
that a firm is uncertain enough about to agree to conduct an experiment.
35
These ads may be quite different from ads that are routinely purchased by
36
firms. By contrast the differential viewership associated with the the Super
37
Bowl and other sports events yields a natural experiments that can be used
38
to estimate advertising effectiveness.
39
Two weeks prior to the Super Bowl, the NFC and AFC Championship
40
games are played. Controlling for the point spread, the winners of these
41
games are essentially random. On average, the Super Bowl will be watched by
42
an additional eight percentage points, or roughly 20 percent more households,
43
in the home city of the team that play in the game. There is a similar increase
44
in viewership for the host city.
45
Super Bowl ads are sold out months before these Championship games, so
46
firms have to decide whether to purchase ads long before knowing who will be
47
featured in the Super Bowl. Hence the Championship Games are essentially
48
random shocks to the number of viewers of Super Bowl ads in the home
49
and host cities of the performing teams. The increased sales of advertised
50
products in cities of qualifying teams, compared to sales in home/host cities
51
of near-qualifying teams, can thus be attributed to advertisements.
52
We study 70 movies advertised during the 2004–2012 Super Bowls and
53
70 placebo movies that are similar to movies that chose to advertise but did
54
not do so.
55
There are three attractive features to studying movies. First, movie ad3
56
vertisements are common for Super Bowls, with an average of 8 per game in
57
our sample. Second, different movies advertise each year. Third Super Bowl
58
ad expenditure represent a large fraction of a movie’s expected revenue. For
59
a Pepsi ad to be profitable, it only needs to move sales by a very small
60
percent. As Lewis and Rao [2012] show, in their Super Bowl Impossibility
61
Theorem, for products like Pepsi, it can be virtually impossible to detect
62
even profitable effects. The cost of Super Bowl ads, on the other hand, can
63
represent a meaningful fraction of a movie’s revenue.
64
There are however, two notable disadvantages to studying movies. First,
65
city-specific, movie sales data are difficult to obtain. However, we were able
66
to obtain this data for a limited sample of cities. We also have an additional
67
proxy for movie demand—Google searches after the Super Bowl —for the
68
full sample of cities. Second, movies do not have a standard measure of ex-
69
pected demand prior to the broadcast of the ads. Having such a measure can
70
drastically reduce noise. We construct this measure using Google searches
71
prior to the Super Bowl as a proxy for interest and show that searches are a
72
predictor of eventual sales.
73
Overall, we find strong evidence of large effects of advertising on movie
74
demand. Our results suggest that a 100 ratings point increase due to addi-
75
tional Super Bowl ad exposures increases opening weekend movies revenue
76
by about 50 percent. For the average movie in our sample, this translates
77
into an incremental return of $8.4 million in opening weekend ticket sales
78
associated with a $3 million Super Bowl advertisement.
4
79
A similar methodology was independently used by Hartmann and Klapper
80
[2014], who study the effects of Super Bowl ads in the beer and soda category,
81
and find positive effects on product sales following the game.
82
We believe that researchers can this methodology for other types of ad-
83
vertising. Sports events such as the World Series, basketball playoffs, college
84
bowls, the Olympics, and the World Cup create many large, essentially ran-
85
dom shocks to viewership of ads shown during these events that can serve as
86
natural experiments to measure ad impact.
87
1
Empirical specification
We use the following notation.
88
89
ymct = outcome for movie m in city c at time t
(1)
xmcs = ad views for movie m in city c at time s (Super Bowl)
(2)
fck = fans of team k in city c
(3)
zks = 1 if team k plays in the Super Bowl at time s, 0 otherwise
(4)
The “outcome” is the measure of ad performance which specify in two
ways:
90
1. ymct = log of Google queries on movie m in city c at time t
91
2. ymct = log of opening weekend box office receipts for movie m in city c
92
at time t
5
The “fans” variable can be constructed in two ways. The simplest specification is that it is just a dummy variable for the home city, as in Equation
(5). However, some major cities do not have an NFL team, but football fans
in those cities may identify with teams from other cities. In the next section,
we describe how to use Google searches for a given team that emanate from
a particular city to measure the local interest in that team.
fck = 1 if c is home city of team k, 0 otherwise
(5)
Google searches from city c for team k
Google searches from city c for all teams
(6)
fck =
93
In this paper, we report results based on the first definition of fans, since it is
94
the simplest measure. However, we provide the data for the second measure
95
as well since it may be of interest in other applications.
Our baseline model is
ycmt = α0 + α1 xcms + other predictors + mct
X
zks fck + other predictors + δcms
xcms = β
(7)
(8)
k∈N F L
96
Equation (7) says that the outcome, ycmt , depends on prior ad exposure,
97
xcms , and other predictors. We would not expect that estimating this single
98
equation by OLSQ would produce a good estimate of the causal effect of
99
advertising, since xmcs is likely correlated with mct .
100
To see what can go wrong, consider a movie about surfing. This might
6
101
have a small advertising campaign in Fargo, North Dakota and a large cam-
102
paign in Honolulu, Hawaii. It is also likely that a surfing movie would also
103
have a much larger audience in Honolulu than in Fargo. But of course, it
104
does not follow that increasing the Fargo ad spend to the Honolulu level
105
would dramatically increase the Fargo audience.
106
Marketing experts choose advertising expenditure based on factors we do
107
not observe, and these advertising choices will typically be correlated with
108
the factors that affect outcomes. In order to estimate the causal impact of
109
ad views on outcomes, we need an instrument—a variable that perturbs ad
110
expenditures exogenously.
111
Equation (8) contains such an instrument, zks . This equation says that
112
ad views in a city depend on whether the viewers are fans of the teams
113
playing in the Super Bowl. We know from prior experience, and will verify
114
below, that this instrument is a strong predictor of ad views. Furthermore,
115
this instrument should be independent of mct since advertising expenditures
116
have to be chosen far before it is known which teams will play in the Super
117
Bowl.
118
It is conceivable that fans of the (essentially random) qualifier for the
119
Super Bowl would be more likely to watch additional movies in the weeks
120
following the Super Bowl. But it is hard to see how this could affect movie
121
viewing in general. Nevertheless, for completeness, we include an analysis of
122
placebo movies—movies that were not advertised in the Super Bowl—and
123
find that there is no effect.
7
124
2
Data
125
2.1
126
We measured ad views using Nielsen ratings for the 2004-2012 Super Bowls,
127
for 56 designated media markets (cities) from Street & Smith’s Sports Busi-
128
ness Daily Global Journal.
129
2.2
130
Movies that advertised for the Super Bowl were obtained from the USA
131
Today’s AdMeter, which lists commercials and viewer ratings for all com-
132
mercials after every Super Bowl. Release dates, distributor, budget, and
133
national sales by week for every movie were found at the-numbers.com.
Ad views
Movies
134
For each movie that advertised, we also obtained a similar placebo movie
135
that did not advertise. To do this, for each year, we used the complete sample
136
of movies, from 2004-2013, from thenumbers.com along with the R package
137
Matchit.1 Advertising movies are shown in Table 1 and placebo movies are
138
shown in Table 2.
139
2.3
140
The simplest proxy for fans of a team in a city is just a dummy variable that
141
equals 1 if the team plays in the home city and 0 otherwise. However, this
1
Fans
gking.harvard.edu/matchit
8
New.York.Giants
San.Francisco.49ers
value
16
value
30
12
20
8
10
4
Figure 1: Fans for two NFL teams
142
proxy misses some important variation in fans by city. Some cities do not
143
have an NFL team and their residents may follow different teams, so we also
144
the more refined definition of fans given in equation (6).
145
We calculate Google searches for a team using Google’s entity classifica-
146
tion system and then applied equation (6). Figure 1 shows the distribution of
147
fans by DMA for two NFL teams which corresponds well with our intuition.
148
We only report results for the dummy variable fans measure, since the
149
results using the Google data are similar.
150
3
151
This section tests for the effects of advertising on product sales. As discussed
152
above, we compare movies that advertise in a Super Bowl to similar movies
Results
9
153
that did not advertise in the Super Bowl, and instrument Super Bowl ratings
154
based on fans in the city for the team that qualified.
155
3.1
156
Table 3 shows that the proposed instrument is a strong one: Super Bowl
157
ratings are significantly higher in a city of a team that is playing. As the
158
regression indicates, about 8 percentage points more households will watch
159
the Super Bowl in the home city of qualifying teams. This is about a 20
160
percent increase in ratings for the average ratings in the sample. We also note
161
that fans of near-qualifying teams (the teams that lost the Championship
162
Game) is not a significant predictor of Super Bowl ratings.
First stage
163
Column (2) of Table 3 shows that ratings for a Super Bowl, in a given
164
year and city, are well-predicted by how many fans in that city there are
165
for the teams that are playing; whether the city is hosting; and year and
166
city fixed effects. Together, these variables explain more than 70 percent of
167
variation in Super Bowl ratings.
168
3.2
169
If advertisers choose their subsequent ad spend on a movie based on the
170
associated Super Bowl ratings, our instrument would not be valid. Table
171
4 shows a regression of a city’s total advertising per capita for a movie on
172
Super Bowl ratings for that city. The estimated coefficient is essentially zero,
Reliability of the instrument
10
Figure 2: Impact of Super Bowl ads on searches, nationwide
173
suggesting that subsequent ad spend does not depend on whether or not a
174
movie is featured in the Super Bowl.2
175
4
176
As mentioned above, we have two measures of outcome: Google searches on
177
the movie title and opening weekend revenue.
178
4.1
179
Figure 1 shows the nationwide queries on movie titles advertised in the Super
180
Bowl.
Impact of ad views on outcome
Google searches
181
It is clear that movies advertised in the Super Bowl see a bump in inter-
182
est. However, after that bump, it is not apparent how much of that initial
183
interest translates into box office revenue. That question is what our model
184
is designed to answer.
2
Advertising spend is from Kantar Media.
11
185
Table 5, Column (1), shows that, for movies that advertised in the Super
186
Bowl, Google searches on release week are far higher in places with higher
187
Super Bowl ratings than in other cities. Column (2) uses the instrument
188
based on home city discussed earlier and finds similar results.
189
4.2
190
The movie sales data we have is only available for a subset of cities. In
191
particular, we only have data for (1) movies that advertised in the Super
192
Bowl, not the placebo movies; (2) for cities that were the home cities for
193
teams that qualified for a Super Bowl or were the runners-up.
Opening weekend box office revenue
194
Despite the smaller sample, there is evidence of a significant positive effect
195
of Super Bowl ratings on movie sales as shown in Table 5, Columns 3 and
196
4. Note, though, that the effect on ticket sales is smaller than the effect on
197
Google searches. This is true even limiting the Google search data to only
198
the sample of cities for which we have box office data.
199
4.3
200
Table 6 uses the same model as in Table 5 but uses placebo movies. Columns
201
(1) and (2) reproduce the results for advertising movies for comparison. Ta-
202
bles (3) and (4) use the identical model with placebo movies.
Placebo movies
203
Unlike the advertised movies, there is typically a small or even negative
204
influence of ratings on the searches for placebo movies. This could be due
12
205
to moviegoers substituting away from these movies in favor of the movies
206
advertised on the Super Bowl.
207
4.4
208
The results suggest that a 100 ratings points increase due to additional ad
209
exposures increases release week ticket sales for a movie advertised on the
210
Super Bowl by about 50 percent. Since the Super Bowl has, country-wide,
211
about 42 ratings points, this implies that a Super Bowl ad increases release-
212
week ticket sales by about 21 percent. The average movie in our sample took
213
in $40 million on the opening weekend. Thus the incremental ticket revenue
214
from the Super Bowl ad were roughly $8.4 million on average. Since a Super
215
Bowl ad cost about $3 million, this means a return of 2.8 to 1.
Interpretation
216
We want to emphasize four caveats in interpreting these results.
217
First, note this ignores future revenue streams such as additional ticket
218
sales and other media licensing. There are subsequent theater receipts, home
219
movie purchases, TV licensing, and so on. Some of this additional revenue
220
stream may be attributable to the Super Bowl ad impressions as well, though
221
we have no easy way to measure this. We also do not know how the incre-
222
mental revenue is divided among the various parties—how much goes to the
223
studios, local theaters, stars, and so on. Similarly, we don’t know how the
224
costs of the TV ad are divided among the relevant parties.
225
Second, in calculating the return to advertising, we are assuming that the
226
incremental viewers of the Super Bowl have the same response to ads as those
13
227
who would watch the Super Bowl anyway. It is possible that the committed
228
fans pay more attention to the game and less to ads. Or perhaps they are
229
much more engaged with the entire experience and so pay more attention
230
to ads than the incremental viewers. It is possible that the incremental fans
231
have substantially different tastes in movies than the fans you would get
232
simply by purchasing more ad slots.
233
Third, it is hard to know we don’t know how these results extend to other
234
settings, as the Super Bowl has unique qualities. There are other similar
235
events such as the World Series, basketball playoffs, the Summer and Winter
236
Olympics, and so on. These natural experiments are not quite as clean-cut
237
as the Super Bowl, but are certainly worthy of future study.
238
Fourth, how should we interpret the estimated 2.8 to 1 return? If the NFL
239
is setting the price of its ad slots to maximize revenue, we would expect that
240
the marginal ad would just break even. It could easily be that the average
241
return is significantly larger.
242
Finally, we want to clarify how these results fit with the Super Bowl
243
Impossibility Theorem (Lewis and Rao [2012]). They argue that it is nearly
244
impossible for a firm to test the effects of an individual ad campaign, even if
245
it randomly assigned DMAs during a Super Bowl.
246
How, then, can we find such highly statistically significant results? The
247
answer is that the Super Bowl Impossibility Theorem refers to the question
248
of measuring the effectiveness of a single campaign. But here, we study the
249
average effect of 70 campaigns. The noise level is too high to say anything
14
250
about the effects of a particular advertisement, but the average performance
251
of all movies in our sample can be estimated reasonably precisely.
252
5
253
We use a natural experiment—the Super Bowl—to study the causal effect
254
of advertising on movie demand. Our identification strategy relies on the
255
fact that Super Bowl ads are purchased before advertisers know which teams
256
will play in the Super Bowl and that cities where there are many fans of the
257
qualifying teams have substantially larger viewership than other cities do.
Conclusion
258
Within this setting we study 70 movies that were advertised during the
259
2004-2012 Super Bowls. We compare product purchase patterns for adver-
260
tised movies in cities with fans from the qualifying teams to cities with fans
261
of near-qualifying teams. We find a substantial increase in opening weekend
262
revenue due to Super Bowl advertisements. On average, the movies in our
263
sample experience an incremental increase of $8.4 million in opening weekend
264
box office revenue from a $3 million Super Bowl advertisement.
265
We suggest that our methodology can be generalized to a variety of sports
266
settings where the nature of qualifying creates a large random shock to ad
267
viewership in a particular area, and that this methodology has notable ad-
268
vantages compared to the more common approach of using field experiments
269
to determine the causal impact of advertising. The best identification comes
270
from sporting events such as the Super Bowl in which the teams that will play
15
271
are unknown at the time companies purchase advertising spot. However, even
272
if the home cities are known it seems to us unlikely that advertisers would
273
take this information into account when choosing its ad expenditure. So the
274
methodology could well be applicable for a broader set of media broadcasts
275
with differential appeal across geographies.
16
17
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
16 blocks
50 first dates
act of valor
alamo
alice in wonderland
angels and demons
avengers
batman begins
battleship
be cool
benchwarmers
captain america first avenger
cars
chronicles of narnia prince caspian
constantine
cowboys and aliens
fast and furious
fast and furious 5
fast and furious 6
gi joe retaliation
hidalgo
hitch
iron man
iron man 3
john carter
Movie
Warner Bros.
Sony Pictures
Relativity
Walt Disney
Walt Disney
Sony Pictures
Walt Disney
Warner Bros.
Universal
MGM
Sony Pictures
Paramount Pictures
Walt Disney
Walt Disney
Warner Bros.
Universal
Universal
Universal
Universal
Paramount Pictures
Walt Disney
Sony Pictures
Paramount Pictures
Walt Disney
Walt Disney
Distributor
continued on next page
2006-03-03
2004-02-13
2012-02-24
2004-04-09
2010-03-05
2009-05-15
2012-05-04
2005-06-15
2012-05-18
2005-03-04
2006-04-07
2011-07-22
2006-06-09
2008-05-16
2005-02-18
2011-07-29
2009-04-03
2011-04-29
2013-05-24
2013-03-27
2004-03-05
2005-02-11
2008-05-02
2013-05-03
2012-03-09
Release Date
Table 1: Advertisers: Movies that Advertised in Super Bowl
PG-13
PG-13
R
PG-13
PG
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
G
PG
R
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
Rating
45
75
12
92
200
150
225
150
209
75
35
140
70
225
75
163
85
125
160
140
78
55
186
200
275
Budget
($Mil.)
18
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
jumper
ladykillers
land of lost
leatherheads
limitless
longest yard
lorax
meet robinsons
mission impossible iii
monsters vs aliens
need for speed
oz great and powerful
pacifier
pirates of caribbean dead man’s chest
pirates of caribbean on stranger tides
poseidon
prince of persia sands of time
race to witch mountain
rango
rio
robin hood
robots
running scared
secret window
shaggy dog
shutter island
Movie
20th Century Fox
Walt Disney
Universal
Universal
Relativity
Paramount Pictures
Universal
Walt Disney
Paramount Pictures
Paramount Pictures
Walt Disney
Walt Disney
Walt Disney
Walt Disney
Walt Disney
Warner Bros.
Walt Disney
Walt Disney
Paramount Pictures
20th Century Fox
Universal
20th Century Fox
New Line
Sony Pictures
Walt Disney
Paramount Pictures
Distributor
continued on next page
2008-02-14
2004-03-26
2009-06-05
2008-04-04
2011-03-18
2005-05-27
2012-03-02
2007-03-30
2006-05-05
2009-03-27
2014-03-14
2013-03-08
2005-03-04
2006-07-07
2011-05-20
2006-05-12
2010-05-28
2009-03-13
2011-03-04
2011-04-15
2010-05-14
2005-03-11
2006-02-24
2004-03-12
2006-03-10
2010-02-19
Release Date
Table 1: Advertisers – continued from previous page
PG-13
R
PG-13
PG-13
PG-13
PG-13
PG
G
PG-13
PG
PG-13
PG
PG
PG-13
PG-13
PG-13
PG-13
PG
PG
G
PG-13
PG
R
PG-13
PG
R
Rating
82.5
35
100
58
27
82
67.5
195
150
175
66
200
56
225
250
160
200
50
135
90
210
80
17
40
60
80
Budget
($Mil.)
19
277
276
spider-man 2
star trek
star trek into darkness
starsky and hutch
super 8
thor
transformers age of extinction
transformers dark of moon
transformers revenge of fallen
troy
up
v for vandetta
van helsing
wanted
war of worlds
wild hogs
wolfman
year one
you don’t mess with zohan
2004-06-30
2009-05-08
2013-05-15
2004-03-05
2011-06-10
2011-05-06
2014-06-27
2011-06-29
2009-06-24
2004-05-14
2009-05-29
2006-03-17
2004-05-07
2008-06-27
2005-06-29
2007-03-02
2010-02-12
2009-06-19
2008-06-06
Release Date
Sony Pictures
Paramount Pictures
Paramount Pictures
Warner Bros.
Paramount Pictures
Paramount Pictures
Paramount Pictures
Paramount Pictures
Paramount Pictures
Warner Bros.
Walt Disney
Warner Bros.
Universal
Universal
Paramount Pictures
Walt Disney
Universal
Sony Pictures
Sony Pictures
Distributor
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
R
PG
R
PG-13
R
PG-13
PG-13
R
PG-13
PG-13
Rating
200
140
190
60
50
150
210
195
210
150
175
50
170
75
132
60
150
60
90
Budget
($Mil.)
Notes: These are all movies that advertised in the Super Bowl and were not released prior to the Super Bowl or the February
in which the Super Bowl was played. Data sources are discussed in more detail in Section 2.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
Movie
Table 1: Advertisers – continued from previous page
20
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
10,000 bc
3 days to kill
adjustment bureau
around world in 80 days
basic instinct 2
battle los angeles
be kind rewind
break up
captain america winter soldier
cars 2
chronicles of riddick
clash of titans
da vinci code
doomsday
drillbit taylor
duplicity
eight below
failure to launch
fast and furious tokyo drift
ghost rider
ghost rider spirit of vengeance
green lantern
guess who
hangover
harry potter and prisoner of azkaban
Movie
Warner Bros.
Relativity
Universal
Walt Disney
Sony Pictures
Sony Pictures
New Line
Universal
Walt Disney
Walt Disney
Universal
Warner Bros.
Sony Pictures
Universal
Paramount Pictures
Universal
Walt Disney
Paramount Pictures
Universal
Sony Pictures
Sony Pictures
Warner Bros.
Sony Pictures
Warner Bros.
Warner Bros.
Distributor
continued on next page
2008-03-07
2014-02-21
2011-03-04
2004-06-16
2006-03-31
2011-03-11
2008-02-22
2006-06-02
2014-04-04
2011-06-24
2004-06-11
2010-04-01
2006-05-19
2008-03-14
2008-03-21
2009-03-20
2006-02-17
2006-03-10
2006-06-16
2007-02-16
2012-02-17
2011-06-17
2005-03-25
2009-06-05
2004-06-04
Release Date
Table 2: Placebos: Movies that Did Not Advertise in Super Bowl
PG-13
PG-13
PG-13
PG
R
PG-13
PG-13
PG-13
PG-13
G
PG-13
PG-13
PG-13
R
PG-13
PG-13
PG
PG-13
PG-13
PG-13
PG-13
PG-13
PG-13
R
PG
Rating
105
28
50.2
110
70
70
20
52
170
200
120
125
125
33
40
60
40
50
85
120
57
200
35
35
130
Budget
($Mil.)
21
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
hellboy
home on range
honeymooners
how to train your dragon
i am number four
i love you, man
ice princess
incredible hulk
indiana jones and kingdom of crystal skull
inside man
international
interpreter
iron man 2
just go with it
lone ranger
madagascar 3 europe’s most wanted
man of house
mars needs moms
miami vice
mirror mirror
mr and mrs smith
paul
pirates of caribbean at world’s end
proposal
raising helen
robocop
Movie
Sony Pictures
Walt Disney
Paramount Pictures
Paramount Pictures
Walt Disney
Paramount Pictures
Walt Disney
Universal
Paramount Pictures
Universal
Sony Pictures
Universal
Paramount Pictures
Sony Pictures
Walt Disney
Paramount Pictures
Sony Pictures
Walt Disney
Universal
Relativity
20th Century Fox
Universal
Walt Disney
Walt Disney
Walt Disney
Sony Pictures
Distributor
continued on next page
2004-04-02
2004-04-02
2005-06-10
2010-03-26
2011-02-18
2009-03-20
2005-03-18
2008-06-13
2008-05-22
2006-03-24
2009-02-13
2005-04-22
2010-05-07
2011-02-11
2013-07-02
2012-06-08
2005-02-25
2011-03-11
2006-07-28
2012-03-30
2005-06-10
2011-03-18
2007-05-24
2009-06-19
2004-05-28
2014-02-12
Release Date
Table 2: Placebos – continued from previous page
PG-13
PG
PG-13
PG
PG-13
R
G
PG-13
PG-13
R
R
PG-13
PG-13
PG-13
PG-13
PG
PG-13
PG
R
PG
PG-13
R
PG-13
PG-13
PG-13
PG-13
Rating
60
110
27
165
50
40
25
137.5
185
50
50
90
170
80
275
145
50
150
135
85
110
40
300
40
50
120
Budget
($Mil.)
22
279
278
sahara
she’s man
shrek forever after
snow white and huntsman
son of mask
spiderwick chronicles
stay alive
stepford wives
street fighter legend of chun-li
sucker punch
take me home tonight
taking of pelham 123
terminator salvation future begins
twisted
ultraviolet
wall-e
watchmen
wild
world war z
2005-04-08
2006-03-17
2010-05-21
2012-06-01
2005-02-18
2008-02-14
2006-03-24
2004-06-11
2009-02-27
2011-03-25
2011-03-04
2009-06-12
2009-05-21
2004-02-27
2006-03-03
2008-06-27
2009-03-06
2006-04-14
2013-06-21
Release Date
Paramount Pictures
Paramount Pictures
Paramount Pictures
Universal
New Line
Paramount Pictures
Walt Disney
Paramount Pictures
20th Century Fox
Warner Bros.
Relativity
Sony Pictures
Warner Bros.
Paramount Pictures
Sony Pictures
Walt Disney
Warner Bros.
Walt Disney
Paramount Pictures
Distributor
PG-13
PG-13
PG
PG-13
PG
PG
PG-13
PG-13
PG-13
R
R
R
PG-13
R
PG-13
G
R
G
PG-13
Rating
145
25
165
170
100
92.5
20
100
50
75
19
110
200
50
30
180
138
80
190
Budget
($Mil.)
Notes: These are placebo movies that were deemed most similar to advertising movies but did not advertise in the Super
Bowl. They were calculated by the methodology discussed in Section 2.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
Movie
Table 2: Placebos – continued from previous page
Table 3: First Stage: Super Bowl Ratings and Fans of Teams
Nielsen Ratings
(1)
(2)
City of AFC Championship Game Winner
City of NFC Championship Game Winner
City of AFC Championship Game Runner-Up
City of NFC Championship Game Runner-Up
Super Bowl Host City
Adjusted R-squared
Observations
Fixed Effects
0.105***
0.088***
(0.016)
(0.012)
0.072***
0.082***
(0.016)
(0.010)
0.003
0.013
(0.016)
(0.010)
0.017
0.010
(0.017)
(0.010)
0.051***
0.069***
(0.017)
(0.010)
0.13
441
None
0.71
441
Year and City
* p < 0.1; ** p < 0.05; *** p < 0.01
Notes: Robust standard errors are shown in parentheses. Super Bowl ratings are Nielsen
ratings, corresponding to percent of households watching the Super Bowl, in an average
half hour. Home City is a dummy variable that takes the value 1 if a team plays in a
city; 0 otherwise. The Green Bay Packers’ Home City is Milwaukee, since we do not have
ratings data on Green Bay. Data sources are discussed in more detail in Section 2.
23
24
0.79
769
OLS
Notes: Robust standard errors clustered at the city-year level, are shown in parentheses. Super Bowl ratings are Nielsen
ratings, corresponding to percent of households watching the Super Bowl, in an average half hour. Data sources are discussed
in more detail in Section 2.
0.79
769
2SLS
0.000
(0.000)
0.000
(0.009)
(0.003)
(0.000)
0.005
-0.001
* p < 0.1; ** p < 0.05; *** p < 0.01
Adjusted R-squared
Observations
Specification
log(Pre Super Bowl Search Volume)
Nielsen Super Bowl Ratings
Table 4: Advertising Spending
Ad Spend Per Capita
(1)
(2)
25
* p < 0.1; ** p < 0.05; *** p < 0.01
0.87
2,260
2SLS
(0.019)
(0.019)
0.87
2,260
OLS
0.057***
(0.318)
(0.269)
0.056***
0.840***
0.446*
0.96
984
OLS
(0.013)
0.033**
(0.234)
0.452*
0.96
984
2SLS
(0.013)
0.033***
(0.365)
0.697*
log(Tickets Sold Per Capita)
(3)
(4)
Notes: Robust standard errors clustered at the city-year level, are shown in parentheses. Super Bowl ratings are Nielsen
ratings, corresponding to percent of households watching the Super Bowl, in an average half hour. Data sources are discussed
in more detail in Section 2.
Adjusted R-squared
Observations
Specification
log(Pre Super Bowl Search Volume)
Nielsen Super Bowl Ratings
log(Google Searches on Release Week)
(1)
(2)
Table 5: Effects of Advertising
26
* p < 0.1; ** p < 0.05; *** p < 0.01
0.87
2,260
2SLS
(0.019)
(0.019)
0.87
2,260
OLS
0.057***
(0.318)
(0.269)
0.056***
0.840***
0.446*
0.86
2,165
OLS
(0.021)
0.085***
(0.262)
-0.536**
0.86
2,165
2SLS
(0.021)
0.085***
(0.410)
-0.082
Placebo Movies
(3)
(4)
Notes: Robust standard errors clustered at the city-year level, are shown in parentheses. Super Bowl ratings are Nielsen
ratings, corresponding to percent of households watching the Super Bowl, in an average half hour. Data sources are discussed
in more detail in Section 2.
Adjusted R-squared
Observations
Specification
log(Pre Super Bowl Search Volume)
Nielsen Super Bowl Ratings
Table 6: Placebo Movies
Advertising Movies
(1)
(2)
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
References
Daniel A. Ackerberg. Advertising, Learning, and Consumer Choice in Experience Good Markets: an Empirical Examination. International Economic Review, 44(3):1007–1040, August 2003. ISSN 0020-6598. doi:
10.1111/1468-2354.t01-2-00098. URL http://doi.wiley.com/10.1111/
1468-2354.t01-2-00098.
Marianne Bertrand, Dean Karlan, Sendhil Mullainathan, Eldar Shafir,
and Jonathan Zinman. What’s Advertising Content Worth? Evidence from a Consumer Credit Marketing Field Experiment.
The Quarterly Journal of Economics, 125(1):44, 2010.
URL
http://www.ingentaconnect.com.ezp-prod1.hul.harvard.edu/
content/oup/qje/2010/00000125/00000001/art00007.
William S Comanor and Thomas A Wilson.
On Advertising and
Profitability. The Review of Economics and Statistics, 53(4):408–10,
1971. URL http://econpapers.repec.org/RePEc:tpr:restat:v:53:
y:1971:i:4:p:408-10.
Douglas Galbi. U.S. Annual Advertising Spending Since 1919, 2008. URL
http://www.galbithink.org/ad-spending.htm.
Wesley R. Hartmann and Daniel Klapper. Super bowl ads. Technical report, Stanford Graduate School of Business, 2014. INFORMS Marketing
Science, 2012.
Joseph F. Heyse and William W. S. Wei. Modelling the Advertising-Sales
Relationship through Use of Multiple Time Series Techniques. Journal
of Forecasting, 4(2):165–181, 1985. ISSN 02776693. doi: 10.1002/for.
3980040206. URL http://doi.wiley.com/10.1002/for.3980040206.
Ye Hu, Leonard M. Lodish, and Abba A. Krieger. An Analysis of Real World
TV Advertising Tests: a 15-Year Update. Journal of Advertising Research,
47(3):341–353, 2007.
Junsoo Lee, B. S. Shin, and In Chung. Causality Between Advertising and
Sales: New Evidence from Cointegration. Applied Economics Letters, 3(5):
299–301, 1996. ISSN 1350-4851. URL http://econpapers.repec.org/
RePEc:taf:apeclt:v:3:y:1996:i:5:p:299-301.
27
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
Randall A. Lewis and Justin M. Rao. On the Near Impossibility of Measuring
Advertising Effectiveness. Working Paper, 2012.
Randall Aaron Lewis and David H. Reiley. Does Retail Advertising Work?
Measuring the Effects of Advertising on Sales Via a Controlled Experiment
on Yahoo! SSRN Electronic Journal, June 2008. ISSN 1556-5068. doi:
10.2139/ssrn.1865943. URL http://papers.ssrn.com.ezp-prod1.hul.
harvard.edu/abstract=1865943.
Richard Schmalensee. A Model of Advertising and Product Quality. Journal
of Political Economy, 86(3):485–503, 1978. URL http://ideas.repec.
org/a/ucp/jpolec/v86y1978i3p485-503.html.
Roger Sherman and Robert D. Tollison.
Advertising and Profitability.
The Review of Economics and Statistics, 53(4):397–407,
1971. URL http://econpapers.repec.org/RePEc:tpr:restat:v:53:
y:1971:i:4:p:397-407.
Duncan Simester, Yu Hu, Erik Brynjolfsson, and Eric T. Anderson. Dynamics of Retail Advertising: Evidence from a Field Experiment. Economic
Inquiry, 47(3):482–499, 2009. ISSN 0095-2583. URL http://econpapers.
repec.org/RePEc:bla:ecinqu:v:47:y:2009:i:3:p:482-499.
28