Download Measuring Ad Effectiveness

Measuring Ad
Effectiveness
Lecture 13
Introduction
Advertising: the eighth Wonder of the World.
Advertising
5/23/2017
Advertising & Public Relations - Lecture
13
Effectiveness Measurement Tends

Measuring ad effectiveness is a key concern in the advertising
industry today

Unprecedented focus on ROI and accountability

Maturing of the advertising industry; pressure on agencies to
produce

Cross-industry move to non-commissionable promotion (i.e.
Heineken’s abandons tv ads in favor of sales promotion)

Still, most companies don’t measure, or measure very little
5/23/2017
Advertising & Public Relations - Lecture
13
Why measure effectiveness?

Optimization of current advertising efforts


Optimization of future advertising efforts


Attain best performance on key variables of interest
Learn what works and what doesn’t
Advertising is extremely expensive




Budget Issues
Accountability issues
Changes are more expensive the later they occur
High Cost of Mistakes



Lost Sales
Company Reputation and Standing
Though never easy, measurement is easier than ever before


Advances in information and tracking technology
Development of advanced statistical techniques
5/23/2017
Advertising & Public Relations - Lecture
13
Why to not measure ad effectiveness?








Creative “history” doesn’t often repeat itself
Cost-benefit analysis is prohibitive
No agreement on what to test
Pre-testing stifles creativity
Results don’t generalize to actual market conditions
Tests can’t account for wearout/repetition effects
Consumer preference/market volatility issues
No clear way to measure the link between cause
and effect (did ad really affect sales?)
5/23/2017
Advertising & Public Relations - Lecture
13
Stages of Effectiveness Measurement

Ad effectiveness can be measured at any stage
of execution by various methods
 Before
Execution (Pre-testing)
 During Execution (Concurrent testing)
 After Execution (Post-testing)

Allocation of effort is typically determined by
cost-benefit analysis or resource constraints.
5/23/2017
Advertising & Public Relations - Lecture
13
Testing Methods

Before Execution (Pre-testing)








Concept Tests (“Big Idea”)
Messaging Tests
Copy Tests
Source Tests
Media Tests (or Vehicle Tests)
Components Analysis
Emotional Reactions Tests
Physiological Arousal/Reaction Tests





5/23/2017
Galvanic Skin Response
Perspiration Tracking
Eye Tracking
EEG (Electroencephalographic meaures of Alpha Waves)
fMRI and other brain imaging techniques
Advertising & Public Relations - Lecture
13
Testing Methods

Before Execution (Pre-testing)
 Consumer
Juries
 Portfolio Tests
 Theater Tests
 Readability Tests (Flesch Formula for
readability)
 Rough Art (“Thumbnails”) Tests
 Comprehension Tests
5/23/2017
Advertising & Public Relations - Lecture
13
Main Pre-test Metrics
Recall
 Recognition
 Readability
 Comprehension
 Preference Change for Brand
 Rankings

5/23/2017
Advertising & Public Relations - Lecture
13
Measurement Methods

During Execution (Concurrent testing)
 Purchase
Cohort Tracking Studies
 Pantry Checks & Consumer Diaries
 Day After Recall (DAR)
 Test Marketing
 Split Run Tests
 Wave Analysis (Modification Tests)
 Single-Source Tracking
5/23/2017
Advertising & Public Relations - Lecture
13
Measurement Methods

After Execution (Post-testing)
 Memory
Tests
Recognition
 Recall (Aided vs. Unaided)

 Persuasion
Tests
 Likeability Tests
 Call-to-Action/Inquiry Tests
 Scanner Panel Data
5/23/2017
Advertising & Public Relations - Lecture
13
Measurement Methods

Data Collection Vehicles








Surveys (mail, internet, phone, person to person)
Interviews
Direct Observation/Ethnographic Studies
Mall Intercepts
Focus Groups
Sales Data
Primary and Secondary Data Source Compilations
Where to Test


Home
Field
 Lab
5/23/2017
Advertising & Public Relations - Lecture
13
Measurement Method Limitations

What are the major limitations of these
testing methods?
5/23/2017
Advertising & Public Relations - Lecture
13
Sales as a Post-Testing Measure -Limitations
Consider this scenario:

Gateway runs a series of television advertisements for its new
personal computer and sees a 15% increase in sales.
Can we safely attribute the 15% increase in sales to
Gateway’s tv ads?
5/23/2017
Advertising & Public Relations - Lecture
13
Sales as a Post-Testing Measure -Limitations

Other factors affect sales besides ads






Consumer inertia
Other concurrent promotions
Past promotions (lag effects)
Competitor efforts
Seasonality effects
Sales measures may neglect “non-consumatory” or
longer-term effects of ads
Sales are a highly imperfect way to measure ad effectiveness
5/23/2017
Advertising & Public Relations - Lecture
13
Solutions to the “Sales Problem”

Use sales measures, but control for confounding
variables with
 Sound
experimental (control-test) methodology
 Analytical techniques

Use Surrogate Measures
5/23/2017
Advertising & Public Relations - Lecture
13
Surrogate Measurement Methods















Ad Recall
Attention Draw
Attention Hold
Brand Association
Main Idea Recall
Likeability
Relevance
Attitude Change
Recognition
Brand Switch
Credibility
Attribute Recall
Emotional Response
Purchase Intention
Action/Behavioral
5/23/2017
“What do you remember about this ad?”
“Did this ad capture your attention?”
“Did this ad hold your attention?”
“What brand was advertised in this ad?”
“What was this ad about?”
“Did you like this ad?” “Did you like the message?”
“How important is this ad’s message to you?”
“Did this ad change your attitude towards…”
“Do you recognize having seen this ad?”
“Would you switch brands after seeing this ad?”
“Do you believe the ad’s claims about the brand?”
“What aspects of the product do you remember from
the ad?”
“How did the ad make you feel?”
“After seeing this ad, how likely are you to buy?”
Click-thrus, calls, website visits, etc.
Advertising & Public Relations - Lecture
13
General Principles for Measuring Effectiveness




Use a “portfolio” approach where possible
Test at various stages (pre-, concurrent, post-execution)
Testing conditions should mimic reality to the extent possible
Use good experimental design


Representative samples
Well controlled
 Reliability and validity Issues



Use independent observers where possible
Take multiple measures of effectiveness, but focus on key variables
of interest
Use appropriate effectiveness benchmarks
5/23/2017
Advertising & Public Relations - Lecture
13
Advertising Return on Investment
(ROI)

A key issue for advertisers and agencies today

Driven by need for financially prudent decision-making
and accountability

Used to compare various campaign alternatives or judge
a campaign against a benchmark

Much better than Cost or Response per Thousand
measures (CPM or RPM)
5/23/2017
Advertising & Public Relations - Lecture
13
ROI Calculation

Formula:
ROI % = (Incremental Contribution from Ad – Ad
Cost)/Ad Cost x 100
ROI % = (ICFA – ADCOST)/ADCOST x 100
Contribution = Sales – Variable Costs

Note: Many companies just look at Incremental Sales, even though
conceptually incorrect
5/23/2017
Advertising & Public Relations - Lecture
13
Essentials of Calculating Ad ROI
Choose a timeframe for analysis




Immediate/Short-term
Medium Term
Lifetime
Determine the relevant unit or total contribution margin



May depend on timeframe chosen
May involve some pre-test assumptions about product sales mix
Determine the relevant ad cost



Should be fully loaded (include corporate allocations)
Includes pre-production, production, post-production and media buy costs
Analyze or predict sales data




5/23/2017
New buyers (triers, new customers, etc.)
Incremental demand from repeat buyers
Must be consistent with analysis timeframe
Advertising & Public Relations - Lecture
13
Simple ROI Calculation Example

Safeway advertises a 10% off coupon for Tropicana Orange Juice in the
local newspaper. The total ad project cost is $600. The modified
contribution margin per unit of Tropicana OJ given the promotion is
$1.50. The analysis timeframe is two weeks from the release of the
coupon ad. Suppose Safeway’s Tropicana OJ sales for the two week
period total 1,000 units. Scanner data shows that 40% of the Tropicana
OJ buys are inertial, 50% are new buyers and the remaining 10% is
increased demand from loyal users. Assume no cannibalization of
same or other brands. What is Safeway’s ROI for the Tropicana coupon
ad?

Answer: [(50% + 10%) x 1,000 units x $1.50/unit) - $600] / $600 = 50%
5/23/2017
Advertising & Public Relations - Lecture
13
Class Activity
You are pre-testing ad copy for one of the following ad campaigns.
1) McDonald’s tv ad introducing a new line of salads. McDonald’s wishes to stimulate demand for
these new additions to their menu.
2) A Warner Brothers tv ad for a new action-adventure movie. Warner Brothers wants as large an
audience for its new picture as possible.
3) The American Heart Association introduces a “Stop Smoking” ad campaign in television and print
aimed at teenagers. The AHA wants to curb teen smoking.
4) Williams Sonoma sends out its newest catalog to opt-in consumers, displaying all the items it
carries in its stores. WS wants to increase sales of these items as much as possible.
5) NBC runs an ad for the final exciting episode of its new show “American Inventor”, hoping to gain
as large an audience as possible.
6) Zetia, a new cholesterol-lowering drug, launches a tv ad campaign to stimulate interest in the drug.
7) Fiesta runs a 30 minute informercial about its new flagship outdoor grill.
1. Given the ad campaign’s particular goals, which three surrogate measures do you believe
are most important in gauging ad effectiveness? Justify your answer.
2. Describe in general terms how you would measure post-execution ROI for the ad
campaign. Focus on (1) the timeframe chosen and (2) the relevant measure of
incremental contribution and (3) on how you would go about distinguishing incremental
“sales” from inertial (baseline) “sales”.
5/23/2017
Advertising & Public Relations - Lecture
13
Effectiveness Testing Companies

Millward-Brown, Ipsos-ASI: Copy testing

IRI, Ipsos-ASI, MSW, Neilsen: On-air testing

Starch Ad Readership Report: Print Ad Effectiveness
(post-tests)

IRI, MarketSource, Nielsen: Single-Source Tracking
Systems
5/23/2017
Advertising & Public Relations - Lecture
13