Download Experiments Part 2 - NMSU College of Business

Experiments: Part 2
Slide 1
In part two of this lecture, I’ll talk about several different examples of experiments in marketing
and then talk about test marketing.
Slide 2 (No Audio)
Slide 3
This is an example of a before-and-after experiment without control. This is not a true
experiment because it lacks a control group. In this case, the manager or the researcher was
interested in the affect of brand labeling on beer preferences. The way this experiment was run,
test subjects first received a six-pack of beer with no labeling, and then were asked to evaluate
it. The next week they were given beer with labels and asked to evaluate it. The summary
indicates the labeling tended to influence evaluations; the labeled beer was preferred it to the
unlabeled beer.
Slide 4
Here’s an example of a true experiment, in which the study unit is sales territory rather than
people. The issue was to determine the effect on profits and dropping unprofitable products
from a total product line. As it says here, “Two sales territories were randomly selected as
experimental and controlled territories. In the experimental territory, 592 unprofitable items were
eliminated from the original product line of 875 items. (We’re talking about severely truncating
the product line). No product changes were made in the control territory. The profit contribution
of each territory was measured for 18 months before and after eliminating the unprofitable
items.” Unprofitable items were dropped from the experimental territories and nothing was
changed in the control territories. The results: the control territories saw no change in profit
contribution, and the experimental territories saw an almost 24% increase in profit contribution.
Therefore, the best course of action for this company—if it’s to increase profits—is to prune
unprofitable items from the product line.
Slide 5
The purpose of the experiment described here is to measure the efficacy of two alternative TV
commercials. This is a true experiment because there is a control group. In this experiment, one
group of participants didn’t see an ad but their sales were tracked; the other two groups saw
one of two different ads and their sales were tracked afterwards. As the table indicates, the
group that saw the second ad tended to purchase at a much higher rate than study participants
in either the first test group or the control group. Thus, the second ad seems to be the most
effective in stimulating sales.
Slide 6
This slide summarizes the results of an experiment to assess the relationship between people’s
perceptions of product quality and the stated price for a product. In this case, the product is a
soft drink concentrate. Students (the subjects) were told seven different price points or given no
price point (the control group), tasted the concentrate, and then were asked to rate it’s quality.
The results are non-linear, with intermediate-priced concentrate perceived as having the highest
quality. (The concentrate was identical on all treatment conditions, so the results are due only to
what students were told). Students told the intermediate prices judged the concentrate of the
Page | 1
highest quality. Students told either the lowest or highest price point judged the concentrate of
lower quality. Interestingly, the students given no price judged the concentrate of the lowest
quality. The results of this experiment suggest there’s a curvilinear or non-linear relationship
between price and perceived quality.
Slide 7
This experiment attempts to address the issue of whether or not sales are related to shelf space
at the supermarket. The results of this experiment were branded products and the summary on
this slide is limited to salt only. Seemingly, the amount of shelf space dedicated to the branded
product did not tend to influence sales.
Slide 8
The goal of this experiment was to measure the effect of alternative promotions on the sale of
pears. The five promotional treatments were (1) special point of purchase displays, (2) store
demonstrations, (3) dealer contests, (4) media advertising program, or (5) nothing (the control
group). Average sales of pears were measured in five cities, with fifteen supermarkets selected
as test units in each city. Treatments were assigned randomly to these different supermarkets,
as order effects were thought important. The results of the experiment indicate that store
demonstrations and dealer contests were the most effective for increasing pear sales.
Slide 9
This slide shows a quasi-experimental rather than true experimental design. (The former is the
design for natural experiments.) In the months leading to the introduction of a new competitive
diet chewing gum, the existing diet chewing gum company measured brand awareness,
consumer perceptions, and usage. The new gum was introduced between June and July. In
subsequent months, the existing gum company again measured brand awareness, consumer
perceptions, and usage. If the new diet gum was stealing sales from the existing diet gum, then
that would be revealed by comparing pre-exposure measures to post-exposure measures. The
problem with this design is the lack of a control group. Although the post-exposure measures
could all be negative relative to the pre-exposure measures, perhaps this shift could be due
issues unrelated to the introduction of a new gum. For example, a Consumer Reports cover
story about the deleterious effects of diet chewing gum might have caused usage to drop; thus,
reduced sales were unrelated to the introduction of the new gum.
Slide 10 (No Audio)
Slide 11
A test market is a type of field experiment. It’s formally defined as a field test of a new product or
marketing mix elements of a current product using experimental or quasi-experimental designs.
Slide 12
Here are four of the many uses and objectives for test markets.

To estimate market shares and volumes, relative to changes in the marketing mix, for a
current product

To estimate cannibalization rates on existing products in a product line after introduction
of a new product in that line.
Page | 2

To collect classification data on potential customers as well as current customers.

To analyze competitors’ reactions to changes in the marketing mix of our product.
Slide 13
The benefits and disadvantages of test markets are as follows. Under benefits, there’s the
opportunity to estimate sales potential under realistic conditions. A test market is a type of field
experiment, so external validity is relatively high. Test markets also are useful for finding and
correcting marketing mix weaknesses. When Pampers was initially introduced by Proctor and
Gamble, it was relatively unsuccessful. When it went into test market, it was discovered that the
problem was an excessively high price point. When P&G lowered the price per diaper, Pampers
became successful. Disadvantages of test markets include their cost and—given the time
needed to complete one—tipping competitors that a marketing mix modification or new product
introduction is being contemplated. As a result, competitors have time to develop a counterstrategy.
Slide 14
Here are four issues that a marketing manager should consider prior to running a test market:

The cost versus information trade off. Conducting a test market is not cheap; there are
major costs. Consider this issue from a Bayesian perspective. What can be learned,
what’s the risk of failure, and what’s the quality of the incremental information gained by
running a test market? If the costs exceed the benefits, then test marketing should be
avoided.

The speed of competitive response. Test markets take time and can tip competitors in a
way that gives them months to respond to marketing mix modifications or a new brand
introduction.

The cost of introducing the product into the test market. Limited production runs to
produce the small volume of product required for a test market may be cost prohibitive.

Test markets can fail, and to some extent the failure of a product in a test market can
negatively affect a company’s reputation. If P&G conducted a test market and the
product failed miserably, then people who purchased that test product might generalize
negative attitudes about that product to other P&G products.
Slide 15
I’m certain these cost figures are dated and somewhat conservative. Assume that test that
includes only two markets is a very conservative test. Regardless, even this conservative test
would cost millions of dollars once direct and indirect costs are included. Unfortunately, many
people forget indirect costs like management time, diversion of resources from current products,
negative internal and external impact of test failures, and the like.
Slide 16
The first step in conducting a test market is to define your objectives. What’s to be learned?
Test market studies can be used to estimate market shares, purchase frequencies, repeat
purchase rates, and cannibalization rates. You must decide which of these things to measure.
The second step is to decide which testing approach to use. The standard test market approach
Page | 3
entails multiple cities and comparing sales results over several months to assess the efficacy of
alternative marketing mixes. Alternatively, a controlled test market (recall my discussion of
single-source data suppliers) may take only a few weeks and cost far less. An intermediate
alternative is a simulated test market. These are research set ups inside traditional malls, in
which people are asked to walk through a simulated store and purchase products with the
money they’ve been given. Although a simulated test sounds hokey, the results of simulated
test markets are highly predictive of ultimate market performance. Because only the
predictiveness of these tests is critical, their artificiality is irrelevant.
Slide 17
The third step is relatively clear and somewhat independent of test marketing: to develop a
marketing plan. What’s the marketing mix to be? Once that’s decided, the fourth step is to
decide on the test markets. The test markets should reflect regional differences if they’re
important. The different test markets should have different demographics and little media
spillover to or from other markets. They should be as similar as possible so that differences
should be due to marketing mix differences rather than market differences. Over-tested and
idiosyncratic markets should be avoided; the former should be avoided because once
consumers become wise to being tested frequently, then they’ll start to respond differently
merely because they’re frequently involved in testing.
Slide 18
This slide shows a detailed list of criteria for selecting test markets:

Large enough to produce meaningful projections and contain at least 2% of the actual
population.

Representative demographically and with respect to product consumption, behavior,
media usage, and competition. A market with vastly different competitors makes crossmarket comparison problematic.

Relatively isolated to avoid media and physical distribution from other markets.

Normal historical development in the product class.

Auditing and marketing research services should be available.

It shouldn’t be over-tested.
Slide 19
These last two steps are fairly obvious.

Run the test market and then analyze the results. Notice the typical test market takes
between 6-12 months because the initial estimates of repeat purchase rates tend to be
high. Trial rates are very different from repeat purchase rates. When a test market is run
over less than six months, trial and repeat purchases often are confounded. A product’s
long term success depends on repeat purchase, and that ultimately is what test markets
try to estimate. Thus, test markets tend to run from 6 to 12 months.

Analyze the results and check the purchase data to be certain that trial and repeat
purchases haven’t been conflated. To fully use the test market study, awareness data,
Page | 4
competitive responses, sales of similar products, and the like would be collected and
analyzed.
Slide 20
To recap this lecture on experimentation and marketing, I covered the three basic criteria for
establishing causality. Then I briefly defined some key terms, such as dependent and
independent variable. After that, I covered the importance of control and control groups for
establishing causality. Next, I compared and contrasted field versus laboratory experiments.
Then I discussed internal validity and threats to it. Finally, I closed with several examples of
marketing experiments and a discussion of test markets.
Page | 5