Download Esking JAR 1975

In test marketing a new food product,
low price and high advertising led to greatest sales,
but high price and low advertising gained
as much immediate profit.
A Case for Test Market Experiments
Gerald J. Eskin
Test markets of new products typically
use a single marketing plan which does
not provide information on how marketing variables, such as pricing and advertising, affect performance of the new
product. The single plan approach implicitly assumes that the new product
decision-maker knows how marketing
variables affect sales and that he has
already taken this into account.
The knowledge to make such decisions is clearly not forthcoming from
published sources. It is in neither textbooks nor journals. A review of the
Journal of Marketing and the Journal of
Advertising Research for the five-year
period ending December 1973 revealed
no marketplace studies for new products
in which the tools of the marketer were
experimentally manipulated in order to
learn their effects. Knowledge from experiments is, of course, not the only
possible source of information, but it is
one of the most defmitive ones.
The basis for new product marketing
decisions must then rest either on
proprietary information or on the
judgment and qualitative experience of
decision-makers. Informal discussions
with research suppliers and market
research directors lead me to believe
that although a number of companies
have done some quantitative work on the
role of marketing variables in new
product introduction, most of it does not
involve well-controlled experimentation.
Only a small number of companies have
tried experimental methods in a test
market situation to learn about major
components of the marketing plan. Of
those companies that have done
"variable tests" {i.e., experiments), most
have done so only once or twice and have
tested only one variable in each such test.
Given this apparent lack of information about these relationships and
the obvious fact that such information is
needed for proper decision-making, one
may question the popularity of the predetermined single plan mode of research
and wish to assess the desirability of an
alternative strategy, such as the con-
trolled variable test, to obtain information about various marketing alternatives.
The controlled variable test appears
to be an established alternative. Procedures for experimenting with different marketing variables have been well
developed for years. Banks wrote an
applied book on the subject in 1965.
Becknell and Mclsaac (1963) produced a
sound study of Teflon's response to advertising pressure in the early Sixties,
and the agricultural economists have
been systematically measuring the effects of such variables for at least 15
years {see Hoofnagle, 1963, for a review
of this early work). Lipstein (1965) even
suggested prototype designs appropriate
for the new product situation.
What then are some of the reasons for
non-use? The following have been
suggested:
1. All aspects of the marketing plan to
be utilized in subsequent national introduction have been preselected by
27
Journal ofAdvertising Research
the time the test market starts.
Therefore alternatives need not be
tested.
vertising and price, affected product performance.
2. High quality information is required
concerning the one plan to be used
nationally. Introduction of major
variables, like advertising expenditure level, into the test will only dilute sample size on each variable,
hence reducing the quality of the
reading.
The Newfood Test Market
3. Variable testing constitutes an
unreal situation. What is desired is a
real world reading, not an artificial
one that cannot be translated.
A new convenience food suitable for
use as a snack or a nutritionally balanced meal substitute was being contemplated for national introduction.
Test marketing was undertaken with
several objectives in mind. Sales data
were sought to allow extrapolation
through time and over markets in order
to obtain first-year national performance
4. Such tests have not worked in the
past because they have often gone
out of control due to uncooperative
retailers, distribution problems, or
unforseen changes in the competitive
environment.
5. Results ofsuch tests are unreadable.
Either they show no differences between treatments, are statistically
not significant, or are counter-intuitive.
Clearly, many of these reasons have
merit. Experiments do cost extra money,
they don't always go as planned, and
results are sometimes hard to interpret.
But just as clearly, the extra cost may be
well worth it, many problems can be anticipated and dealt with in advance, and
imperfect data often are still informative.
In a practical sense the issues remain:
Is it true that our facts or judgments
specify enough knowledge for marketing
new products? Are the five objections to
experimentation sound? Are alternative
experiences possible?
What follows is a report on a controlled-variable test conducted during
the test marketing of a new product in
which attempts were made to determine
how two major marketing tools, ad28
Gerald J. Eskin is an associate professor
of marketing at the University of Iowa.
Formerly he taught at the Graduate
School of Business, Stanford University,
and, prior to that, he was manager
of marketing methods and of mathematical and behavioral research at the
Pillsbur; Company in Minneapolis. He
received his Ph.D. in economics and
econometrics from the University of
Minnesota. Dr. Eskin has published
various articles on new product test
marketing, forecasting, and pianning in
the Journal of Marketing Research and
eisewhere. He is a co-author of two books
on Cases in Computer and Model
Assisted Marketing. He is currentiy on
the editorial board of the Journal of
Marketing Research.
projections. Information was also sought
concerning possible differential responses to several aspects of the introductory
marketing plan, the most important of
which were variations in price and level
of advertising expenditure.
Design and Preliminary Analysis
A controlled store test was conducted
for six months in four test markets.
Three levels of price were tested: a base
price below 50 cents, a price 10 cents
above the base, and a price 20 cents
above it. A sample of 30 stores within
each of the test cities was split into three
matched panels of 10 stores. Each of
these matched panels received one of the
three price treatments.
Variations in advertising expenditures were tested across the four cities,
with two cities receiving a low weight
plan and two receiving a high weight
plan. The high weight treatment was
to involve double the estimated gross
impressions per 1,000 households as did
the cities which received the low weight
treatment. (However, later analysis revealed that the actual impressions generated differed by only 70 per cent.)
The proportion of total expenditures
allocated to each media class was the
same in ail four test markets. All
schedules were dominated by television.
Advertising expenditure allocations over
time were also matched, all involving
front loading of expenditures in time
with one-half of all test expenditures occurring within the first two months of the
test market period. Figure 1 shows the
overall research design.
The principal dependent measure of
interest was unit sales per store per
month. This measure was selected in
preference to market share because the
boundaries of the market for this
new product were not known. Various
measures on extraneous factors which
could conceivably affect test out-comes
were also taken. These included several
Volume 15, Number 2. April 1975
Figure 1
Low Weight
Adwrtising:
Vfarftcf 2
\iarket t
•*rice:
Base
N=
(10)
+10c.
(10)
High weight
+20c.
(10)
Base
(10)
+lOe.
(10)
Market _
+20c.
(10)
Base
(10)
+IOc.
(10)
Market
+20c.
(10)
Base
(10)
+10c.
(10)
+20c.
(10)
Month
1
2
3
4
5
b
Total No. of observations = 720
dep measure = toial sales/per store/by month
store size measures, measures characterizing the trading area each store
served, and monitoring of various instore conditions. These control measures
and procedures indicated that all factors
were well balanced and, hence, not included in the subsequent analysis, except
the following: all commodity volume
(ACV—the dollar sales for all items
sold in each store); average family income of the trading areas served by each
store (I); and number of facings (F), a
measure of the amount of shelf space
given to the test product. These variables
were all significantly correlated with the
dependent variable or with the experimental treatment; thus they were
potentially important covariates.
Table I presents the correlations
among the experimental treatments, the
covariates, and the dependent variable.
Note first that the correlation between
actual price and advertising is very small
(.012). This suggests that the original
design specification requiring orthogonal (independent) treatments was
largely achieved—i.e., out of design conditions were minimal.
Row 1 of the table displays the
correlations between the sales of the new
product, the experimental treatments,
and the covariates. In the subsequent
analyses, the effects of the covariates on
the dependent variable are adjusted out
using standard regression procedures.
Inspection of Row 3 of the table
reveals an important and unexpected
problem. The experimental advertising
treatment is strongly correlated with two
of the covariates. Because of this, it is impossible to interpret its relationship to
sales. It is not clear whether advertising
has no effect on sales (as suggested by
the —.005 correlation in Row 1) or
whether it does have an effect, but it is
being hidden by the influence of the
covariates. Subsequent analysis suggested that an imbalance in sample
selection had occurred. Large stores in
high income neighborhoods tended to be
located in the geographic areas receiving
the lower advertising treatment while the
small stores in the lower income neigh-
borhood tended to be located in areas
which received the high advertising
treatment. The fact that large stores
(high ACV) tended to be found in high
income areas (r = .35) compounded the
problem.
The effects of price, on the other
hand, are not greatly confounded by the
effects of the covariates (see Row 2). Considering these covariates, in this case,
may improve the estimates of the effects
of price, but it will not change the interpretation of how price is related to
sales.
Before analysis can proceed, a
technique is needed which eliminates the
dependence between the advertising
treatment and the covariates without
disturbing the independence of the price
treatment. Recall that, in constructing
the initial design procedures, it was not
possible to control, a priori, for the differential effects of market share because
of the difficulty in defining the relevant
market. In this study, it turned out that
All Commodity Volume provided an acceptable substitute for this variable,
given that the primary dependent
variable was total unit sales per store per
month. When this original variable is
adjusted for the appropriate ACV, a new
dependent variable (Sales/ACV) is obtained. This variable seems to allow for a
reasonable interpretation of the effects
of the advertising treatment since it is
not affected by differences in store size
Table 1
Correlations Between Treatments, Covariatra*
and Dependent Measures
Sales
Price
Treatments
Price
Adv
-.397
-.005
.012
Adv
Covariates
Facings
.029
.006
-.157
Facing
ACV
.586
-.043
-.271
-.005
ACV
Sales/AVC
-.419
.234
-.014
-.070
Income
.262
-.059
-.462
.057
.351
.043
29
Journal ofAdvertising Research
(ACV) which exist between the advertising treatments. It is adjusted in
sueh a way that the effects of these differences are eliminated. The last row of
Table 1 indirates the correlation of the
treatments and covariates with this new
dependent variable, Sales/ACV. Note
that the correlation with price is changed
very little but that the correlation with
advertising is now significant and
positive (.234). Sales/ACV also shows
relatively low intercorrelation with the
various covariates.
In the analysis which follows,
Sales/ACV is the dependent variable to
be explained by the two experimental
variables and the two remaining
covariates.
Principal Regression Results
In the regression work each of the 720
store-period combinations is taken as an
observation point. Sales divided by the
store size measure (ACV) is treated as
the dependent variable. This variable is
regressed against a set of measures of the
independent variables, each of which is
treated as a continuous variable. The
regression procedures allow estimation
and statistical testing of (1) the main effects of the price and advertising treatments, (2) interactions between these
treatments, (3) Interactions between the
treatments and time, (4) the effects of the
covariates, and (5) seasonal and trend
factors.
Table 2 displays the average Sales/
AVC values corresponding to the various treatments averaged over the entire six months, adjusted for the control
factors (income and number of facings).
Entries in the table are obtained by first
averaging the raw data by cell, adjusting
for control factor imbalance by use of the
regression coefficients, and then normalizing by dividing all entries by the
average for the base price-low advertising plan. This final normalization
has the dual effect of protecting the
proprietary nature of the raw data and
30
Table 2
Average Sales/ACV by Experimental Condition
Adjusted for Income and Facing Differences
Across Trading Areas
^""•^..^^/l dvertising
^
\
Price
Base Price
+10<t.
+204.
Average over
aii prices
Low-weight
plan
High-weight
plan
Average over both
advertising plans
100
79
64
150
106
71
125
93
81
109
^^--^
allowing users to easily determine the per
cent deviations from the low price-low
advertising base plan.
The marginal totals indicate the main
effects of the treatments. The high advertising weight plan produces sales of
109 compared to a low weight plan of 81.
Literally interpreted, a highly significant
(t = 9.1) 35 per cent increase is attributable to the advertising weight differential. Price increase, on the other
hand, tends to substantially reduce
Sales/ACV. The dependent measure
falls by 32 points when the price changes
from the base to 10 cents higher, and by
an additional 25 points from this price to
the highest price. Again the linear effect
is significant (t = 3.25).
Although significant main effects are
present, they do not adequately describe
the structure of the situation. Examination of the individual cells in
Table 2 shows a pattern where the effectiveness of advertising to attract
buyers depends on the price charged. At
the base price, the high advertising plan
produces sales 50 per cent above the low
plan. At the middle price this percentage
slips to 34 per cent and at the highest
price the high weight plan is ahead by
only 11 per cent. Conversely, looking at
the same data in terms of responsiveness
to price, we see substantially more sensitivity to price under the high advertising plan than under the low one.
The structure of this interaction is
68
illustrated in Figure 2, "*" represents
the means from Table 2 and the solid
line the regression fits. As can be seen,
the linear model fits the data well
although the deviations suggest a slight
tendency toward the type of curvilinear
line that would be associated with a constant elasticity model. In the terms of a
constant elasticity model (i.e., per cent
change in sales/per cent change in price)
the price elasticity at the low weight advertising plan is -1.0, while under the
high weight plan the price elasticity is
-2.5. This interaction effect is statistically significant (t = 6.65).
Table 3 shows the impact of pricing
and advertising over time. Part A of the
table displays the ratio of Sales/A VC under the high advertising weight plan to
Sales/ACV under the low weight plan for
each time period (months). Again the
data have been adjusted by the covariates. This affects the levels of the
ratios but not the trends. Here we see a
clear tendency for the ratios to be larger
in the early months of the test and substantially smaller in later months. The
linear component of this trend, as
measured by the regression model, is
statistically significant (t = 4.19).
Ratios of Sales/ACV at the two
higher prices to Sales/ACV at the base
price are exhibited in Part B of Table 3.
Again, covariate adjustments have been
made that affect levels but not trends.
Although there is a slight tendency for
Volume 15, Number 2. April 1975
the later ratios to be smaller than the
earlier ones, indicating growing price
sensitivity, the linear component of this
trend is not statistically significant (t =
.29).
Discussion
Conclusion for the Product Tested.
Based on the data analysis, both advertising and price were important determinants of the sales levels achieved for
the new food product, at least for their
first six months in markets. However,
sales differentials due to price sustained
themselves over time while the advertising-induced differentials did not. Finally, the effectiveness of advertising as a
sales-inducing factor depended on the
price charged, and conversely.
Although the preceding statements
all follow from the analysis, they vary in
the degree of credence that should be
associated with each. Interpretive ambiguity also varies. The results concerning price are by far the strongest
being based on a design in which all
prices were tested within the same city
and replicated over four cities. The
various store characteristic imbalances
are not large with respect to the price
variable, and the statistical errors (standard error of coefficients) are small compared to coefficient sizes. These facts
lead us to place high credence in the
assertion of large stable differentials due
to price. Increasing price from the lowest
to the highest was estimated to reduce
sales by some 36 per cent under the low
weight advertising plan, and by 53 per
cent under the high weight plan.
The validity of the conclusions concerning advertising are closely tied to
the appropriateness of sample imbalance adjustment procedures via the
(Sales/AVC) normalization and its relations to the covariance analysis procedure. Although adjusting for store
size differences seems appropriate, it
must be noted that raw sales data did not
Table 3
Part A
Ratios of Sales Response Under High Advertising
Weight to Sales Response Under Low Weight Plan by Month
(Adjusted for ACV, Income and Facing Differences)
1
2
3
4
5
1.43
1.48
1.48
1.27
1.17
Month
Ratio
6
1.19
PartB
Ratio of Sales Response at 10c. and 20c. Price
Inereases to Base Price by Month
(Adjusted for ACV, Income and Facing Differences)
Month
1
2
3
4
5
Ratio at+10<t.
.80
.75
.81
.76
.76
Ratio at +2(H.
.59
.53
.61
.54
.52
exhibit an advertising effect. Nevertheless, there seems to be support for the
assertion that the high weight advertising program resulted in sales increases of about 35 per cent over the entire six-month period and that this differential diminished with time. Under
the highest price conditions, the differential has entirely disappeared by the
end of the test. However, the con-
6
.69
.47
servative reader may legitimately object
to this conclusion.
Interpretations of the advertisingtime interaction is especially difficult
because the distribution of spending for
advertising over the test period was not
uniform. The time effect could be attributable either to wearout—that is,
advertising becoming less effective over
time—or to the decrease in advertising
Figure 2
Demand at Various Combinations of Advertising Spending and Price Levels
1.5
1.4
Mean Response
Regression Line
1.3
1.2
1.1
1.0
.9
With low ADV
.7
.6
Base Price
+204. Prices
31
Journal of Advertising Research
expenditures during the later periods.
These alternatives appear to be observationally equivalent, hence all that
can be said is that a front-loaded
program of the type used here has the effect of producing large sales differentials
in the early portion of the test markets
and smaller ones later on.
With respect to the evidence for a
negative interaction between price and
advertising, it is strong if one grants
the appropriateness of the Sales/ACV
measure. TTiis result is, of course, inconsistent with the conventional wisdom
of classical economics in which the role
of advertising is to reduce price elasticity, hence allowing higher prices to
be charged. On the other hand, many
marketers have always implicitly assumed the market works in the manner specified here. Grocery retailers who
advertise only when there is a special
price must believe in a negative priceadvertising interaction.
For the new food product, this interaction has particular importance
when the sales results are translated into
profits, as shown in Figure 3. Here we see
that about the same peak profits are
possible with high advertising as with
low. In the case of the low spending plan,
a high price is required to generate
profits through margin effects. In the
case of high advertising expenditures, a
low price will interact with the advertising to produce larger volumes that
are profitable even at the lower unit
margins.
It should be noted in passing that
before this test was conducted the best
candidates for a national plan were
based on the belief that the role of high
prices is to create margins to allow for
advertising. Thus, the promising alternatives were thought to be either high
price-high advertising, or low pricelow advertising. These research results
suggest quite different alternatives,
namely: low price-high advertising to
build a market or high price-low advertising to build margin.
Figure 3
ProHt Curves as a Function of Price Conditional on Various Advertising Weight Plans
Profit Index
1.0
Base Price
Implications for Test-Market Planning. The objectors to experimental
manipulation of marketing variables in
test markets claim that (a) the technique
is not needed, or (b) it does not work.
In the case of Newfood, the technique
was clearly needed. It was not needed as
viewed before the test when there was a
feeling that the best marketing plans
were known, but as viewed after the test
when empirical results established that
new alternatives were superior to those
originally contemplated. For example, if
the conventional test market had been
conducted using the high price-high
advertising plan, expected sales results
would have been about 53 per cent below
the sales maximizing results obtainable
under the low price-high advertising
strategy. The conventional test alternative would not have generated expected revenues sufficient to warrant
national introduction of the product.
Without knowledge of how marketing
variables work, test market failure would
probably have been attributed to lack of
product appeal or poor copy.
in planning for the subsequent introduction of Newfood, the possibility of
utilization of a price-advertising plan
different from any of those tested arose,
ln the eventual introduction, such a plan
was, in fact, utilized. The evaluation of
these non-tested alternatives constituted
a problem in extrapolation requiring
utilization of such curves as those in
Figure 2 and 3. Such procedures are
risky, but at least possible, given a
properly executed experimental variable
test. In the conventional single plan test,
no information is forthcoming on this
issue at all.
Turning to the question. Do such
variable tests work?, in the Newfood
case, the test worked. It worked in the
sense that no fatal flaw was apparent in
the design or execution of the test and
subsequent analysis produced plausible
and statistically significant results. It
also worked in the sense that a
management team accepted the outcome
of the project and utilized it, along with
other information, in making a "go/nogo" decision concerning product introduction and in developing the eventual marketing program.
Having produced a successful example, one is tempted to conjecture about
why this was so and the conditions under
which the experience can be replicated.
Volume 15. Number 2. April 1975
In my view, the answer is pedantic and
obvious. The successfulness of such experiments is most closely related to the
degree to which the tests satisfy the wellknown basic principles of experimental
design. These include:
* Selection of measurably different
treatments. Here advertising programs
are being tested that are similar in all
respects but one—the differences in
planned weight is two to one, not a
realistic 10 or 20 per cent. Such large differences are necessary to produce
noticeably different results and to offset
any convergence which might occur
when the research is actually executed in
the field. (In the case of Newfood, the actual difference in advertising levels, was
more like 1.7 to 1 rather than the planned two to one.)
* Replication is used. Each treatment condition is tested in many stores
and in at least two cities. This allows
both measurement of error and reduction of error. Still more cities would be
desirable.
* Measurement of treatments and
extraneous factors. In test marketing it
Services
• Focused group moderating
• Individual depth interviewing
• Executive interviewing
• One-way mirror facility rental
• Respondent recruiting
• Testimonial commercial recruiting
• Custom studies
• Analysis
• Local/nationwide coverage
• Complete package or any component
Areas of Research Specialization
• Basic motivational, behavioral,
life-style and attitudinal research
• Product category idea generation—
problems, benefits, attributes,
associations—for creative insight or to
provide consumer language for
quantitative studies
• Determining the problems of target
consumers and defming opportunities
must be assumed that something will go
wrong. Tests go "out of design" and outside factors (like competition) create imbalances; this test was no exception. In
the present instance a measurement
system existed to identify and quantify
such problems. (See discussion of the
preliminary analysis.) This made adjustment at the analysis stage possible.
• Use of designs that fit the problem.
If advertising performance depends on
price, then the design must allow
measurement of this, such as through
the orthogonal design described here. If
time matters, tests must be run long
enough for all trends to develop or
special designs built for this purpose
must be used.
• Appropriate analysis methods.
Perfectly executed designs are amenable
to simple analysis but imperfect tests are
in fact the rule. In such cases, advanced
analysis procedures are required to
remove those extraneous effects.
Finally, the role of the experience factor must be noted. As in the marketing process, experience counts in the
successful
utilization
of
research
methodology. The Newfood test market
was not the first variable test conducted
by the makers of Newfood. It is a
reasonable expectation that the probability of obtaining useful results increases with the number of such tests
previously completed, and with one's
willingness to inquire aboDt and take into account the experience of others.
References
Banks, Seymour. Experimentation in
Marketing. New York: McGraw-Hill,
1965.
Becknell, James C. and Robert W.
Mclsaac. Testing Marketing Cookware
Coated with Teflon. Journal of Advertising Research. Vol. 3, No. 3, pp. 2-8.
Hoofnagle, William S. The Effectiveness
of Advertising for Farm Products. Journal of Advertising Research. Vol. 3, No.
4, pp. 2-6.
Lipstein, Benjamin. The Design of Test
Marketing Experiments. Journal of Advertising Research, Vol. 5, No. 4, pp. 2-7.
• Quiet central air conditioning
• Experienced staff and operators to meet
your research needs
Respondent Recruiting Service
• Low or high incidence/income
• Blue collar/professionals
Unique Group Session Facility
• Ethnic groups
• Convenient mid-Manhattan location
• Fresh or highly verbal respondents
• Relaxed in-home atmosphere
• Urban/suburban—local/nationwide
• Inconspicuous one-way mirrors
Real People Search
• Conducive surroundings for
• Locating, screening and interviewing
post-session discussions
for testimonial commercials
• Full kitchen facilities for taste tests and • Complete service nationwide/
refreshments
international/local
• Custom engineered sound and
recording equipment
SID SHAPIRO
• Videotaping and all audio/visual
TOM
DALE
equipment available
I(.OEa>l 4Stli Street
• Observer-moderator intercom
New York. Now York 10017
• Separate entrances and facilities for
21: PLa/;tSS765
clients and respondents
for meaningful and unpreempted
advertising strategy solutions
• Advertising/communication evaluation
at early or fmished stages of execution
• New product/concept/name
development/screening/evaluation
PENTHOUSE U
33