Download The Economic and Cognitive Costs of Annoying Ads

The Economic and
Cognitive Costs of
Annoying Ads
Daniel G. Goldstein
Siddharth Suri
Fernando Diaz
•  Microsoft Research, New York City
R. Preston McAfee
•  Microsoft Corporation
Matthew Ekstrand-Abueg
•  Northeastern University
Questions of the day
•  How can online labor markets be used for
field studies?
•  What are the economic and cognitive costs
of annoying ads?
Field studies in online
labor markets
•  Mechanical Turk
•  Widely used as subject pool (much like the early ones of
Reips, Birnbaum, Baron, Göritz, Goldstein, etc.)
•  However, MTurk was created as a bona fide labour market
•  Work as DV. Dropout as a good thing.
•  Compensating differential
•  Amount needed to pay someone to compensate for risk or
unpleasantness
•  Innovative method of Toomim et al for testing user interfaces
Publisher: New York Times
Citibank paid
New York Times
Display Ad
Advertiser:
Citibank
Display advertising
•  Online advertising about as large as broadcast TV
•  Display: $13 billion USD in 2013
•  33% of online advertising revenue
•  Display expected to overtake sponsored search ads on
mobile devices
Good Ads
Bad Ads
Mobile annoying ads
Tension
"   Advertisers pay publishers to display annoying ads
"   Annoying ads cost publishers page views through user
abandonment
Impacted parties
" Publishers
"   Lost traffic
"   Apparent desperation: Warning of service quality
" Users
"   Interfere with content consumption. 200 million AdBlock
downloads on Firefox alone.
" Advertisers
"   Appear disreputable
"   Undermines money-to-burn signal
Research questions
"   What makes ads annoying?
"   What are the economic costs of annoying ads?
"   What are the cognitive costs of annoying ads?
Related Work
" Yoo & Kim 2005, Burke et al 2005
"   Moderate animation increases brand recognition and attitude
"   Heavy animation decreases brand recognition and attitude
"   Goldfarb & Tucker 2011
"   Ads that are intrusive or match content increase intent to purchase
"   Ads that are both decrease attention decrease intent to purchase
"   Calculating compensating differentials, Toomim et al 2011
Preparatory study:
Identifying annoying ads
Ranking Ads by Annoyingness
"   72 animated ads, 72 static ads
generated from their final frame
"   Users randomly shown either
the static or animated version
"   First users previewed all ads
Ranking Ads by Annoyingness
"   72 animated ads, 72 static ads
generated from their final frame
"   Users randomly shown either
the static or animated version
"   First users previewed all ads
"   Then rated each on a 1-5 scale
of annoyingness
Ranking Ads by Annoyingness
"   72 animated ads, 72 static ads
generated from their final frame
"   Users randomly shown either
the static or animated version
"   First users previewed all ads
"   Then rated each on a 1-5 scale
of annoyingness
"   Finally, commented on why
Preparatory study
participants
"   Conducted on Amazon
Mechanical Turk
"   $0.25 flat rate, $0.02
per ad rated
"   141 participants in 18
hours
"   Restricted to US
workers with 95%
approval rating
Identifying Good & Bad Ads
“bad
ads”
“good
ads”
"   Animated ads are most annoying, static are least
"   Used 10 most/least annoying ads in study 2
The Impact of Animation
"   Animation caused ads to be rated as substantially more
annoying
What Makes Ads Annoying?
Study 1: The economic
cost of annoying ads"
"
Measuring the "
Compensating Differential
"   Task: users classify email, can stop
any time
"   Randomly assigned into
"   3 pay conditions: .2, .4, .6 cents
per email
"   3 ad conditions: no ads, good ads,
bad ads
"   Treatment not revealed until
acceptance
"   Circumvented ad blockers
"   Ads were randomly
selected from the
set of good or bad
ads
"   New ads with each
email
"   Emails from the
Enron data set
Study 1 participants
"   Conducted on Amazon
Mechanical Turk
"   $0.25 flat rate, 0.2,
0.4, 0.6 cents/email
"   1223 workers in two
weeks
"   Restricted to workers
with 90% approval
rating
Compensating Differential: "
Bad Ads to No Ads
68
"   Bad ads caused 12.7
fewer impressions
compared to no ads
"   .2 cents per impression
52
yields an additional ~16.6
impressions
"   To get an additional 12.7
impressions need to pay
0.2 * 12.7/16.6 = .153
40
50
65
85
Calculation
•  If you get 16.6 additional impressions for every .2 cents you
pay, how much does it cost to make up the 12.7
impressions lost to annoying ads?
𝑋 (​16.6 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠/.2 𝑐𝑒𝑛𝑡𝑠 )= 12.7 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠
𝑋 =12.7 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠(​.2 𝑐𝑒𝑛𝑡𝑠/16.6 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠 )
𝑋 =​.153 𝑐𝑒𝑛𝑡𝑠/ 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛 𝑋 =​$1.53 / 1𝑘 𝑖𝑚𝑝𝑟𝑒𝑠𝑠𝑖𝑜𝑛𝑠 =$1.53 𝐶𝑃𝑀
Compensating Differential: "
Bad Ads to Good Ads
68
"   Bad ads caused 9.5 fewer
impressions compared to
good ads
"   .2 cents per impression
52 62
yields an additional ~16.6
impressions
"   To get an additional 9.5
impressions need to pay
0.2 * 9.5/16.6 = .115
"   $1.15 CPM
40 47
81
Robustness
"   Data are skewed:
"   Many people categorized a few
emails
"   Few people categorized a lot
"   Means drop as observations are
excluded reflecting skew,
annoyingness effect remains
"   Good ads and no ads yield
same number of impressions
Accuracy
"   Enron data set has emails categorized as spam or ham
Treatment
Accuracy
Bad Ads
Good Ads
No Ads
90% ±1
92% ±1
93% ±1
"   Statistically significant difference between bad ads and
good ads/no ads
Study 2: The cognitive cost
of annoying ads"
"
Mousetracking
•  Mousetracking correlates with eye tracking
•  Chen, Anderson, & Sohn, 2001; Guo & Agichtein, 2010; Huang,
White & Buscher, 2012 •  Mousetracking proxies for user interest
•  Steiger & Reips, 2010; Willemsen & Johnson, 2010;
Navalpakkam, & Churchill, 2012
Study 2 participants
"   Conducted on Amazon
Mechanical Turk
"   $0.50 flat rate, $0.10
per question
answered
"   2,840 participants
Task
Good or Bad
Ad
Manipulation check
•  42% listed ad as annoying in “bad”
condition
•  5% did so in “good” condition
Good ad fixations
Bad Ad Fixations
Mousetracking metrics
•  Fixations of the mouse in an area
•  Distance mouse travels in an area
•  Number of entrances of the mouse into an
area
•  Time mouse spends in an area
Mousetracking metrics:"
Ad area
Measure
Bad Ads
Good Ads P-Val (log OLS)
Fixations
4.45 (0.67) 1.57 (0.26) <.001
Distance
182.6 (7.9) 157.9 (8.0) .003
Entrances 1.31 (0.05) 1.13 (0.05) .004
Time (ms) 1873 (321) 1101 (186) .001
Mousetracking metrics:"
Text area
Measure
Bad Ads
Good Ads P-Val (log OLS)
Fixations
Distance
Entrances
Time (ms)
135.7 (9.07)
128.0 (9.05)
.047
1,492 (55.20)
1,570 (66.60)
.677
1.51 (.06)
1.50 (.07)
.322
38,268 (1,207.00)
36,367 (1,085.00)
.596
Mouse reading
•  Rated by 3 judges
•  About 7% incidence of mouse reading in good and bad
conditions, no significant difference
Accuracy
•  Accuracy was significantly lower in “bad” condition
compared to the good ad condition (2.5 percentage point
delta, 6 percentage points for answer at bottom of page)
Summary of mousetracking studies
•  Annoying ads do capture user attention
•  Annoying ads have negative affect on accuracy
•  However, absence of evidence that people read differently
when annoying ads present. Why?
•  Could be depletion, however, in Experiment 1, the accuracy
difference between the bad and good ad conditions held
constant.
•  Our hypothesis is that annoying ads make people care less
about what they’re reading or the questions they are
answering
Conclusions &
Future Work"
"
Conclusions
"   Animation has a causal effect on annoyance
"   Annoying ads hamper accuracy in cognitive tasks
"   Annoying ads cause dropout
"   Compensating differential of bad ads to no ads/good ads is
at least $1 CPM, which would price some ads out of the
market
"   Good ads lead to about the same number of impressions
as no ads
"   We would lose money by running annoying ads
Conclusions
"   Mousetracking shows that people attend to annoying ads
and they the impact accuracy
"   However, they don’t appear to affect reading behavior and
may impact accuracy for motivational reasons
Research Agenda
"   Towards a psychologically informed display ad pricing
scheme
"   Display ads should be priced in terms of time in view instead
of impressions (Economics and Computation, 2011)
"   Two short ads are better than one long ad (Economics and
Computation, 2012)
"   What is the cost of annoying ads? (Journal of Marketing
Research, 2014)
Thank you!"
"