Download Optimal Chapter 6 - Cal State LA

Optimal Database Marketing Drozdenko & Drake, © 2002
1
Chapter 6
The Analysis Sample
Optimal Database Marketing Drozdenko & Drake, © 2002
2
Chapter Objectives
• Why We Sample
• Sampling Methods
• Creation of the Analysis Sample
• Methods of Saving Point-in-Time Sample Data
• Analysis and Validation Samples
• Application of Analysis Findings
Optimal Database Marketing Drozdenko & Drake, © 2002
3
Why We Sample
Testing is the foundation upon which direct marketing is built.
correct test planning, a direct marketer can:
With
• Evaluate new product offerings.
• Gauge the reaction to price changes by measuring the associated
increase or decrease in response rates.
• Determine the impact of a new promotional format change on
response, payment or conversion rates.
• Identify the target market for a new product test.
• Gain insight about specific customer groups or segments.
Optimal Database Marketing Drozdenko & Drake, © 2002
4
Sampling Methods
For samples to be meaningful and unbiased, they
must be selected randomly and representatively
from the universe of interest.
Optimal Database Marketing Drozdenko & Drake, © 2002
5
Sampling Methods (Cont.)
Representative Samples - A representative sample is a sample truly
reflecting the population of interest from which the direct marketer draws
inferences.
For a sample to be representative, no members of the population of
interest are purposely excluded from the sample.
Some direct marketers overlook this very important concept and
assume they can apply test results from one population to another. This
may work in some cases but not always. Be careful!
Optimal Database Marketing Drozdenko & Drake, © 2002
6
Sampling Methods (Cont.)
Typically, the only names that should be eliminated from testing are
names eliminated in roll-outs such as:
• DMA do-not-promotes
• Frauds
• Credit risk accounts
When testing new promotions, some direct marketers also consider
eliminating:
• Names recently promoted for other marketing tests
• States or cities such as Washington D. C. known to have strict
promotional restrictions.
Optimal Database Marketing Drozdenko & Drake, © 2002
7
Sampling Methods (Cont.)
Random Samples - A random sample is one in which every member of
the sample is equally likely to be chosen, ensuring a composition similar
to that of the population.
Pulling names one after another from the beginning of a geographically
sequenced customer database will result in a geographically biased
sample.
To ensure random samples, many direct marketers utilize what is called
“nth selects.”
Optimal Database Marketing Drozdenko & Drake, © 2002
8
Sampling Methods (Cont.)
For example, ACME Direct, a direct marketer of books, music, videos
and magazines is interested in testing a new book concept.
The universe of interest is the segment of most active book buyers.
Therefore, a random and representative sample will be created by
taking an nth select from the universe of concern with no exclusions.
ACME Database
(10,000,000 names)
Most Active Book
Buyer Segment
(3,000,000 names)
Optimal Database Marketing Drozdenko & Drake, © 2002
Least Active Book
Buyer Segment
(2,500,000 names)
Non-Book Buyer
Segment
(4,500,000 names)
9
Creation of the Analysis Sample
In order to properly determine the characteristics that define responders
vs. non-responders you must base it on a sample in which the customer
characteristics were “frozen” at the point-in-time of the promotion. You
cannot pull customers from the database today and examine such
characteristics for a promotion that occurred, for example,six months
prior.
If you are going to properly determine what caused a customer to either
respond or not respond to your promotion, then the characteristics that
you examine must be reflective of what the customer looked like at the
time you promoted them.
Think of this file as a “snap shot” of each customer’s record prior to
sending the promotion. This “snap shot” of the customer’s records is
also referred to as a “frozen file.”
Optimal Database Marketing Drozdenko & Drake, © 2002
10
Creation of the Analysis Sample (Cont.)
To illustrate why “point-in-time” customer data is important in making
sound marketing decisions, consider the following example:
One year ago you sent an offer to 10,000 names from the database a
special Club Med vacation package known to be only of interest to
young married couples without children. If you examine the sample
today (one year later) to determine the characteristics that uniquely
separated responders from non-responders and use the customer
characteristics as of today, you may be misled as to what a responder
looks like. Some responders to the test promotion may have had a
baby since the time of the original test (one year ago). As a result, you
will erroneously conclude that people with babies also are interested in
this vacation package. This is false since at the time of the promotion
when the person made their decision to order the vacation package or
not, they were childless.
Optimal Database Marketing Drozdenko & Drake, © 2002
11
Creation of the Analysis Sample (Cont.)
The process to create a frozen file for analysis purposes is as follows:
• Select names (including all customer data) to be test promoted
• On the database, using a unique key code (Chapter 4), mark the names that
were selected for this test promotion
• Create a file of the selected names with their address and customer ID
information only and send to the lettershop for promotion
• Create a file of the selected names with all customer data for later analysis
(the frozen file)
• Once customer responses come in:
• Update the customer records on the database with response information
using the unique key code
• Update the frozen file with response information
You are now ready to conduct the analysis to determine the characteristics that
separate responders from non-responders based on the frozen file updated with
response information.
Optimal Database Marketing Drozdenko & Drake, © 2002
12
Methods of Saving Point-in-Time Data
• Saving a “snap shot” of all customer data for every test sample
promoted is an expensive and costly proposition.
• Depending on the size of the direct marketer’s testing program, this
can quickly create major computer storage capacity issues.
• To circumvent this problem, some smaller direct marketing companies
save an entire copy of their marketing database on regular basis (e.g.,
quarterly).
• When analysis of a product promotion is required, an analysis sample
is created “on the fly” by identifying the names promoted for the test
(and who responded) via a unique key code residing on the database.
• Once the names are identified on the database, customer data from
the frozen customer database closest, but prior, to the actual promotion
date is merged and an analysis sample created.
Optimal Database Marketing Drozdenko & Drake, © 2002
13
Methods of Saving Point-in-Time Data (Cont.)
This process is illustrated below.
Available Frozen
Files of the Entire
Customer Database
01/1/97
04/1/97
07/1/97
10/1/97
01/1/98
04/1/98
07/1/98
10/1/98
01/1/99
10,000 names
promoted for a new
book product offering
on 3/1/98 identified
via a unique key code
on the database.
10,000 names
promoted for a new
book product offering
on 3/1/98 merged with
saved customer data
as of 1/1/98.
Perform Analysis
The danger with this method is the customer data to be appended to the
test names may not represent the customer’s status near the time of the
promotion. In the above figure we note the appended data is two months
prior to the actual promotion date. Ideally, the customer data should
represent the customer’s status as near the actual promotion date as
possible. As a result, saving a snap shot of the customer data with each
test sample is preferred.
Optimal Database Marketing Drozdenko & Drake, © 2002
14
Methods of Saving Point-in-Time Data (Cont.)
If your company cannot do this due to data storage capacity issues, this
is the preferred alternative.
Just remember, do not merge a promotion file with a version of the
customer database reflecting the customer’s status after the date of the
promotion.
Why?
Optimal Database Marketing Drozdenko & Drake, © 2002
15
Analysis and Validation Samples
Before analysis of any sample is performed, the sample is typically spilt
into two.
The analysis is often performed on two-thirds of the sample and the
results validated or calibrated on the remaining one-third of the sample.
Optimal Database Marketing Drozdenko & Drake, © 2002
16
Analysis and Validation Samples (Cont.)
Why?
Remember, a sample is just that, a sample; and samples have a certain
level of error associated with them.
The validation sample is used to ensure the analyst does not make
erroneous conclusions based on the error variance associated with the
sample.
Creating an analysis and validation sample is a very important step in
the analysis process flow. The importance of the validation sample will
be addressed fully in Chapter 11.
Optimal Database Marketing Drozdenko & Drake, © 2002
17
Application of Analysis Findings
Once the target market is defined based on the analysis of a frozen file,
you are ready to select names from the entire database meeting your
criteria.
When preparing to select names from the entire database based on an
analysis of a frozen file, ensure the data residing on the customer
database is current.
Make sure you are selecting the correct names by keeping the
information residing on the database fresh and relevant.
Optimal Database Marketing Drozdenko & Drake, © 2002
18
Database Modeling Analysis Process Flow Chart
Customer Database
Names pulled and saved from
the database
Define test segment
Names sent test promotion
“The Frozen Analysis File”
Sample split 2/3 for analysis
Analysis of responders vs.
non-responders
Responses
matched to
create the
frozen
analysis file
Sample split 1/3 for validation
Validate findings and
refine results
Application of analysis findings to the database for roll-out
Optimal Database Marketing Drozdenko & Drake, © 2002
19