Download Marketing database essentials

Chapter 4.1 : Marketing database essentials
Chapter 4.1
Marketing database essentials
This chapter includes:
J
What is a marketing database?
J
Types of marketing database
J
Changing demands on the marketing database
J
The essential elements of a marketing database
J
Organising the data
J
Sources of data
J
Validating, loading and cleansing data
J
Deduplication
J
Getting reports from a database
J
Data manipulation – working with the data
J
Data analysis – understanding the data
J
Techniques to improve database performance
J
B2B considerations
J
Dos and don’ts of building a marketing database
About this chapter
W
ith the growing recognition in the last few years that insight and
knowledge about customers has tremendous value for organisations
there has been continued investment in the development of marketing
databases. Marketing databases are being adopted by organisations in a growing
range of industry sectors and by organisations of all sizes.
Marketing databases are becoming even more valuable as the media landscape
continues to evolve, enabling marketers not only to exploit the opportunities
presented by new media and channels, but also providing a central means of coordinating a multi-channel strategy. It is therefore increasingly important for
marketers to understand in general terms how a marketing database operates.
Author/Consultant: Brian Wyatt
4.1 – 1
Chapter 4.1 : Marketing database essentials
As with any subject where technology is involved, there is a lot of jargon and
apparent complexity associated with marketing databases. The aim of this chapter
is to lay out the key elements and capabilities of marketing databases so that a
marketer can feel comfortable with the main concepts when working with more
specialist personnel who would be directly involved in the design, implementation
or running of a marketing database.
Brian Wyatt, Head of Consulting, Acxiom Ltd
Counting House
Tooley Street
London SE1 2QN
020 7526 5100
[email protected]
Brian has worked in marketing and marketing
technology for over fifteen years. He has been
with Acxiom since 1998, working with clients in
the financial services, telecommunications,
automotive, technology and publishing sectors.
include MSN, GM, Capital One, Time Life and O2.
A key aspect of most of these projects has been
helping clients identify and quantify the revenue
and cost-saving benefits of improving database
marketing efficiency and effectiveness.
Prior to joining Acxiom Brian worked for Lex
Service and Volvo Cars. While at Volvo he led an
international team that designed a customer
database for use in worldwide markets, and
implemented what was at the time the UK’s most
sophisticated internet used car locator.
Brian’s background means that he looks at the
technical aspects of database marketing from a
business perspective, and can therefore ensure
that the data and technology are really aligned
with marketers’ objectives. He is also able to
offer expertise across the spectrum from initial
strategic planning right through to campaign
delivery.
Since joining Acxiom’s consulting organisation in
2001 Brian has carried out a large number of
assignments spanning data quality, data strategy,
database management, customer insight and
contact strategy. Clients he has worked with
Brian has spoken at industry seminars in the UK
and Europe and makes regular contributions to
the specialist direct marketing press.
Chapter 4.1
Marketing database essentials
What is a marketing database?
A
ccording to The New Oxford Dictionary of English (2001), a database is ‘a
structured set of data held in a computer, especially one that is
accessible in various ways’.
The term marketing database deserves a narrower definition. For the purposes of
this chapter, we will define it as:
4.1 – 2
Chapter 4.1 : Marketing database essentials
‘A comprehensive collection of inter-related customer and prospect data that allows the
timely and accurate retrieval, use or manipulation of that data to support the marketing
objectives of the enterprise’.
Our aim with a marketing database is to bring together all that we know about
individual customers so we can make the best possible decisions about how to
market to them, and how to service them. This is why you will often hear a
marketing database described as a customer-centric or customer-focused
database, or that it provides a ‘single view’ of the customer.
The diagram below shows a fairly typical layout for a marketing database. Starting
on the left, information from various data sources is taken into the database,
having been cleansed and validated. Two software tools, one for analysis and one
for running campaigns, take information from the database and allow marketing
users to do their own analysis and carry out campaign counts and selections.
More detail on the elements shown in this diagram is provided in this chapter.
Figure 4.1.1
Marketing database elements in overview
$QDO\VLV
7RRO
/HJDF\
/HJDF\
6\VWHP
6\VWHP
'DWD
'DWD
([WHUQDO
([WHUQDO
'DWD
'DWD
&DPSDLJQ
&DPSDLJQ
5HVSRQVH
5HVSRQVH
'DWD
'DWD
/RDGLQJ
/RDGLQJ
9DOLGDWLQJ
9DOLGDWLQJ
&OHDQVLQJ
&OHDQVLQJ
3URFHVVHV
3URFHVVHV
$QDO\VLV
$QDO\VLV
0RGHOV
0RGHOV
5HSRUWV
5HSRUWV
0DUNHWLQJ
0DUNHWLQJ
'DWDEDVH
'DWDEDVH
6XUYH\
6XUYH\
'DWD
'DWD
&DPSDLJQ
7RRO
&DPSDLJQ
&DPSDLJQ
RU:KDWLI
RU:KDWLI
&RXQWV
&RXQWV
&DPSDLJQ
&DPSDLJQ
6HOHFWLRQV
6HOHFWLRQV
As well as ‘database’, it is quite likely that you will hear the terms ‘datamarts’ and
‘data warehouses’ referred to.
G
Data warehouses are typically very large collections of data that have been
pulled together from a variety of different databases. They are more often
used to support analysis and reporting needs than supporting marketing
campaigns, although this can be the case in larger organisations. Often, they
are not customer-centric, as they have to support a wide range of business
functions and therefore may not provide the type of information required by
marketers.
G
Datamarts tend to be smaller databases that are pulled together to support
one area of the business, and for one particular purpose. They are often
4.1 – 3
Chapter 4.1 : Marketing database essentials
selections from larger databases or data warehouses. For example, a
company may have one datamart for carrying out campaigns and another
for carrying out analysis and modelling. The data in each of these marts will
refer to the same customers, but the marts will be constructed in such a
way as to make them most suitable for a particular type of work.
More than just a direct marketing engine
The marketing database is a fundamental building block of the successful
enterprise. It’s often said that data, used effectively, is the most valuable asset that
an organisation has. Or, more colloquially, knowledge is power!
As well as being a platform for marketing communications, the applications of a
marketing database can include, but are not limited to:
A source of information for all outbound customer contact
Market research and analysis
Identification of markets for new products
Identification of new markets for existing products
Product development
Business development
Central control of communications with customers and prospects
Types of marketing database
Marketing databases tend to be optimised towards a particular stage in the
customer life cycle, primarily:
G
A customer database, optimised to support marketing programmes aimed
at customer development and retention
G
A prospect database, optimised to support marketing programmes aimed at
new customer acquisition
The requirements of a database for acquisition do tend to be somewhat different
to those for a database supporting customer development and retention.
Customer database
For a customer database, you need something that will give you a single view of
each customer, complete with all associated history and information, so that you
can get a clear idea of the value of each individual customer. You need to be able
to access individual spending patterns and seasonal variations. You want to be
able to see whether that customer’s purchases are growing or declining. And you
want to know which products or services are due for renewal (e.g. loans,
insurance, car upgrades and warranty agreements etc.) This will also help you
identify important opportunities for cross-selling activity. Finally, make sure it
4.1 – 4
Chapter 4.1 : Marketing database essentials
allows you to enrich your records with the addition of external data such as
lifestyle, life stage and geodemographic information.
Prospect database
A prospect database should make it easy to identify your best prospects. One way
of doing this is by comparing the profiles of your prospects to the profiles of your
top customers. Similar characteristics, be it location, status, industry or
whatever, will identify which prospects are most likely to become customers and
therefore, which should be targeted.
It should also be easy to update (with bought-in lists from external brokers, for
example) and easy to clean, deduplicate and manage. Also, it is beneficial to be
able to integrate it with your customer database so that, once a prospect becomes
a customer, the record can be flagged as such and then migrated – complete with
all transactional activity – from one database to the other. This will help keep your
customer records clean and deduplicated.
Business-to-business
Business-to-business databases, for either acquisition or customer development
and retention, have specific requirements. This is particularly important when it
comes to outbound communications (i.e. do you want everyone in the
organisation to be contacted? Or one contact per company? Or one per
department?) A B2B database should also be flexible enough to tell you everything
you need to know about the spending behaviour, purchasing responsibility and
history of those of your customers who may have many branches and subsidiaries
(even if they have different names) by accruing transactional information from all
sites.
As business-to-business databases have a number of very specific requirements
we’ve included a summary of the main factors involved later in the chapter. (See
also chapter 4.3.)
Changing demands on the marketing database
At one time a marketing database would be primarily, or possibly exclusively,
used to support direct mail. While direct mail continues to be an important
channel, it is no longer necessarily the dominant one for many organisations. The
rise in the use of contact centres and the internet has meant that the marketing
database has had to adapt in order to be able to properly support the needs of
these channels. This process is bound to continue as new communications media
and devices mature.
What has been the effect of these ongoing changes in the channel mix? At a basic
level, the type of data that the database has to store is changing. Take for example
contact history. As well as having to store information about direct mail campaign
inclusion and response, a database today should also be able to cope with the
more detailed campaign activity data generated by telemarketing or email
campaigns. It should also be able to log inbound contacts if these are relevant to
decisions about which proposition a customer should be offered next – if any. As
well as being able to store more complex contact history, gathered across a
number of touchpoints, the database needs to provide marketers with the
capability to plan, execute and track campaigns across a variety of channels.
While the actual campaign fulfilment will still probably be carried on outside of
4.1 – 5
Chapter 4.1 : Marketing database essentials
the database, marketers need to be able to provide the data required for all
channels in the right format and with the required frequency.
As well as being able to simply select and export the data required for campaigns
across channels, the database increasingly needs to be able to manage the contact
strategy within or across channels. At the most basic level the database should
provide marketers with the ability to ensure that customers and prospects receive
appropriate offers via a channel that they prefer, and that they are not overcontacted. The optimum contact frequency will vary from sector to sector, but
once it has been identified the database needs to support rules to ensure that it is
not exceeded, to prevent ‘burnout’ of the names in the database, particularly
important when considering telemarketing, a channel increasingly challenged by
consumer opt-outs.
Some organisations have built separate databases for each channel of
communication, generally for good, but short-term, operational reasons. If
possible this should be avoided, as aside from the duplication of costs, it
means that it will not be possible to co-ordinate communications or to
measure the impact of different combinations of outbound contacts on
customers.
The essential elements of a marketing database
A marketing database needs to offer marketers a number of aspects and
capabilities if it is to meet their requirements. It needs to offer:
G
Relevant data
G
Processes for validating, cleansing and loading data
G
Deduplication capabilities to create a true single view of the customer
G
A method of working with the data (manipulation)
G
Some means of understanding the data (analysis)
Types of data
In our working definition of a database we described a ‘comprehensive
collection…of data’. But what is meant by comprehensive? There are four types of
data that you are likely to store:
4.1 – 6
Primary data
Examples include: a customer’s name and address,
definitions of your products or services, pricing, campaign
definitions and channels of distribution.
Secondary data
Secondary data is used to qualify primary data. Examples
include: demographics, lifestyle information, geographical
profiles or levels of penetration. This is also often referred
to as descriptive data.
Performance data
Performance data records how your customers have
responded, what they have bought, how much they have
Chapter 4.1 : Marketing database essentials
spent and which campaigns they reacted to. This is
sometimes referred to as behavioural data.
External data
External data covers everything from rented or bought-in
lists to the data that is available from various agencies to
augment, qualify or enhance your base data.
Data structure in databases – some basics
In order to understand some of the different ways that a marketing database can
be structured, you first have to be aware of some basic concepts around how
information is held in a database. In order for a database to be able to allow you
to store, retrieve and analyse information it needs to hold that information in a
consistent way. It does this by breaking information down into data elements,
fields and records.
G
A data element is a single piece of information, e.g. a date of birth, forename
or surname
G
A field can be a single data element or a logical collection of data elements,
e.g. the field ‘name’ could be made up of the data elements title, forename
and surname
G
A record is a logical collection of fields, e.g. the name and address of a
prospect
G
A table is a collection of records, e.g. the names and addresses of all
prospects
G
A database contains one or more tables, and these tables have to be
organised and linked to each other in a way that ensures data can be added,
updated, deleted or used quickly and reliably
Databases need to break down and organise information in this way in order to
fulfil their purpose – this is the ‘structured’ part of the definition. To take an
example, if we look at names, then we could collect and store consumers’ names
in one field as they have been presented to us:
‘Mr John Smith’
‘J Smith’
‘John Smith’
This looks fine, however if we wanted to carry out some simple selections we
would have a problem. For example, if we wanted to find out how many people
were on the database with the initial ‘J’, the computer would not be able to give us
the correct answer as it would not know where in the field to look for an initial.
As human beings, we can look at names and work it out, but computers are not
intelligent, and have to be fed information in a very consistent and basic fashion if
they are to be able to answer our questions accurately. This is why it makes sense
to break down name, address and other types of information into their
constituent data elements; in the example above, by having an ‘initial’ field, and
ensuring that all name information entering the database is split up into its
elements correctly, the database would be able to provide you with an accurate
answer to the question. To take a more real-world example, imagine how difficult
it would be to extract details of all customers living in Oxford from your database
if your address information was not broken down into address lines, post town,
county and postcode.
4.1 – 7
Chapter 4.1 : Marketing database essentials
Whenever you’re looking at collecting a new piece of information, for example
in a call centre, make sure that you are able to capture and store the
information in a structured way. What sometimes happens in this type of
situation is that data ends up in a free text ‘comments’ field, where it can
prove difficult or impossible to analyse or use.
Jargon buster: As technology evolves, so does its attendant jargon, and as well as
‘field’ and ‘record’ you may hear the terms ‘column’ and ‘row’ used. To all intents
and purposes a column is the same as a field and a row is the same as a record.
Organising the data
As mentioned above, the different tables of information in a database need to be
linked to each other. There are a variety of different approaches to this, each one
of which has its advantages and drawbacks. The right choice for your database
will depend on your specific requirements; this is a rather technical subject and is
best left to IT specialists, but it can be helpful to be aware of the main types of
database design and some of their characteristics. Database models you may
come across include:
G
Flat – as the name suggests this is one table that contains all the
information on a customer or prospect. This enables rapid access to data
but is inflexible and also has a lot of redundancy, i.e. the same data will be
stored again and again on different records; if your database has Mr and
Mrs Smith living at 1 The High Street, then 1 The High Street will be on
your database twice. This model can be quick for answering queries, but the
redundancy can cause problems such as slower responses to queries and
difficulties with being able to support new types of data as the volume and
complexity of data increases.
G
Relational – in this model tables are related to each other, with each table
containing only one type of data. In this type of database a customer’s name
and address could be stored in different tables. The advantage of this
compared to the flat model is that it allows a customer to have multiple
addresses, either at the same time or over time – something that is hard to
manage with the flat model. It also means that if two customers share the
same address you only have to store the address information once in the
database, so it has less redundancy. The downside of relational databases is
that they are less efficient at answering queries as the computer has to link
(the term used is ‘join’) data from different tables, and this can make them
slow to answer the types of count and selection queries that marketers
typically need.
G
Dimensional – this model is a variation on the relational one, differing in the
way in which the data is organised. The key difference is that a dimensional
database (sometimes referred to as a ‘star schema’) is optimised to answer
frequently asked queries. This can make this type of database much faster
in answering queries, but there is a trade-off in flexibility.
As a marketer, you shouldn’t have to worry about what type of database model
your database employs, and as you can imagine, there is a lot of technical detail
behind the brief descriptions above. What you do have to do is to be clear on your
requirements for your database so that the technical specialists can determine
what the most appropriate model would be.
4.1 – 8
Chapter 4.1 : Marketing database essentials
In marketing, the only constant is change, so when looking at your
requirements for a marketing database you need to stress that flexibility is a
key requirement; what you don’t want to end up with is a database that will
require expensive changes just months after it has been implemented; the
ability to cope with new types of data or processing without incurring large
costs can be designed into a database.
Sources of data
The whole point of a marketing database is to bring together as much relevant
information about a given customer as you can, so that you can make informed
decisions about the timing and content of your communication. So, as well as
taking clean, accurate data as your starting point, you need to be able to add data
to increase and improve your pool of information.
Where does data come from?
There are a huge variety of data sources available to you. As well as using external
suppliers to supplement and enrich your data, there are opportunities to gather
and record information wherever your company touches a customer. Possible
sources include:
Sales
Warranty registrations
Marketing research and surveys
Enquiries/helpline
Accounts
Complaints
Third parties and marketing
partners
Sales promotions
Branches and channels
Prize draws
Servicing
Competitions
Direct response
PR events
External lists
Website
E-commerce transactions
Where to find the data in your company
Almost every organisation runs a number of systems to support its various
business functions and many of these systems hold valuable customer
information. Account systems (in banking and financial services), customer
service, accounting and finance, sales support, branch networks and many other
departments or functions are rich sources for customer data, even if there is little
or no communication between them.
As you plan the construction of your database, search your company for these
potential sources of data, and obtain copies of the details held in each source or
4.1 – 9
Chapter 4.1 : Marketing database essentials
legacy system. By combining the relevant customer data from all these sources
into your database, you will be able to build up a comprehensive, centralised
repository of information on your customers, which you can use to profile and
segment them for targeted marketing. You must, however, arrange for your
database to receive updates from all the systems on a regular basis, with a
frequency dependent upon the use you make of it. This will in turn be defined by
the analysis of requirements (your business requirements) that you undertake
when you embark upon your marketing database project.
Data is expensive!
It costs money to collect. It costs money to store. And it costs money to
clean. The rule is only collect what you can use and use everything you
collect.
This has become particularly important with the increasing availability of online
data – online systems can generate enormous amounts of data about customer
usage of websites. It is likely that you will need this data summarised into more
actionable information before you load it into your marketing database. Similarly,
in some sectors, e.g. retail and telecommunications, lots of data is generated as a
by-product of customer purchases or usage. While some of this data may have
value at a low level of detail, some of it may be more usable if summarised, e.g.
amount spent on each category of product each month will sometimes be more
usable than a detailed history of every product ever bought by each customer.
While there may be a lot of value in the raw data that can be unlocked by
advanced data mining techniques, there is a real danger that a marketing database
can be swamped by loading huge quantities of transactional data that has to have
a lot of processing done within the database before any usable insight is
generated. Even with the advances in hardware and database technology that
continue, the more data is held in a database, the more time it will take for the
database to update or answer queries. If getting campaigns out increasingly
quickly is a major priority, the last thing you need to be doing is loading lots of
detailed data into the database ‘in case we need it’.
Adding value with thorough data enhancement
Data enhancement increases the value and relevance of your data by enriching it
with additional information.
A simple example of this would be adding correct postal codes to your mailing
lists. But at the other end of the spectrum, you can add phone numbers, credit
ratings, responsivity scores and a wide range of geodemographic and lifestyle
information. Much of this information is available from external sources if you
cannot obtain it from internal systems.
The benefit of this is that you gain not just a better understanding of your
customers’ behaviour, but the ability to target your campaigns more accurately
and with better results.
4.1 – 10
Chapter 4.1 : Marketing database essentials
Suppression data
Another type of external data that should be considered for a marketing database
is information about individuals and businesses that have changed address,
individuals that have died, businesses that have ceased trading, and any relevant
‘do not contact’ files. Collectively, this type of information is generally referred to
as ‘suppressions’.
Depending on the volume and frequency of your marketing campaigns you may
want to store these files on your database under a licensing arrangement. In this
scenario, customer or prospect records that match to the suppression files will be
flagged as deceased/gone away/not mailable etc. As well as being more costeffective for large volumes, using this approach has the benefit that flagged
records can be excluded from counts, so marketers can see exactly how many
customers or prospects are marketable for a particular campaign.
With smaller mailing volumes it is often more cost-efficient to match the campaign
files to these files after they have been output from the database, and pay for each
record that is matched (the so-called ‘per hit’ basis). The drawback with this
approach is that you do not know how many records will be dropped until after
data has been extracted from the database and matched to the suppression files,
which can make campaign volumes a little unpredictable.
Validating, loading and cleaning data
It is important to understand that wherever and whenever you perform an update,
you create the possibility of corrupting the very data that you are trying to
enhance. Fortunately, modern databases come with a variety of tools to make the
process easier.
Jargon buster: The overall process of taking data from a range of sources, making it
consistent and creating a single customer view to put into the database is increasingly
referred to as customer data integration or CDI.
Loading and updating
Naturally, the size of your marketing database will grow as your business evolves,
so it should cater for this by allowing you to add, delete and update records as
frequently as you desire. It should also be able to handle imported data from a
broad range of disparate sources.
As well as letting you refresh the database, the load and update function should
include a process for automatic name and address validation and deduplication.
Jargon buster: When talking to your colleagues in IT about the data that you need from
legacy systems and the process of loading it you may hear the acronym ETL; this stands for
extract, transform and load, which covers the processes involved in getting the data out of
the source or legacy systems and into your marketing database.
4.1 – 11
Chapter 4.1 : Marketing database essentials
Process, verifying and correcting
A marketing database should incorporate ‘batch control’ and ‘job scheduling’
functions that let you automate the process of performing regular updates (e.g.
monthly/weekly), reducing the time and effort it takes to refresh your data as well
as helping to prevent manual input errors. They also allow ‘hands-free’ processing
of any updates that you may receive electronically and via the internet.
Name and address handling
“(Name and address handling) is the most intellectually demanding and difficult task that
data processing has to undertake anywhere.”
Robin Fairlie
Names and address data is the very core of a customer database, whether
business-to- consumer (B2C) or business-to-business (B2B). It’s important not to
underestimate their importance in data terms, no matter how straightforward it
may appear. Consider the purposes for which you hold and use name and address
data:
Addressing letters and envelopes (printout)
A key function of any marketing database! In order to present a professional
result, your database needs, as a minimum, to hold (or create) initials, surname,
company name (B2B), address and correct postcode. It must also ‘know’ how to
lay these out (i.e. either hold separate lines or line separators).
Personalised communications
If you are going to personalise communications to your customers you will need
to add and manage titles and honorifics (e.g. Doctor, Colonel and Lady), forename
and sometimes marital status (for joint accounts, for example). The database
must also be able to provide data in the forms required to support other
outbound communications media, e.g. telemarketing and email.
Name and address hygiene
A key part of name and address handling is making sure that the presentation of
the names and addresses is to the highest possible standard. This processing is
generally referred to as name and address hygiene or name and address
cleansing.
Given the wide variety of ways in which names can be spelled, automated
checking and correction of names can be risky. However, your database should at
least be able to identify names that are unusable or contain profanities and ensure
that these are reported on and not loaded onto the database.
A key step in this process is parsing, the process by which the system takes a set
of input data (e.g. Mr John Smith) and divides it into its different elements. In
this example an intelligent name parser could identify the three separate elements
in the supplied data as a title of Mr, a forename of John and a surname of Smith.
Parsing is more complex than it might appear, principally because of
inconsistencies in the data. These inconsistencies can include blank or missing
4.1 – 12
Chapter 4.1 : Marketing database essentials
fields, extraneous additional data, and atypical or foreign words that the system
may either not recognise, or worse, misinterpret.
With addresses, there is more scope for improving the quality of the information
that has been supplied, as there is often a reference file of correct addresses that
can be obtained from the local postal authority – in the case of the UK this is the
Postcode Address File or PAF, available from Royal Mail. Address hygiene software
can ensure that the address data is consistent and corresponds to the format of
the country of origin (e.g. the postcode or zip code matches the county or state). It
may change elements where it can, or drop parts of data that it identifies as
incorrect.
Software systems build a ‘match key’ using parts of the data, and use this to
attempt to match the file to the Post Office/PTT standard reference file. Exact
duplicates may have postcodes added if required. Probable matches are handled
in different ways by different systems and, generally, the threshold or confidence
level for matching can be set by the user. In some systems, the user may also
select whether to use the customer’s preferred or given address, or the Post
Office/PTT standard address. From the marketing point of view, the former is
preferable – Rose Cottage, Any Lane for example, rather than 66 Any Lane, if the
customer has submitted it that way. Any anomalies outside of the threshold set
are usually highlighted in an exception report so that further action can be taken.
Name and address information can be further refined by standardisation. This
type of process applies a consistent and appropriate format to the customer data,
in accordance with predefined rules (e.g. the abbreviation of ‘company’ to ‘co.’).
It is these processes of parsing, correction and standardisation that make it more
likely that the system will catch duplicate records. With your data in a consistent
format, the system can compare parts of the new record to corresponding fields of
similar records that already exist, such as the postcode, postal town, surname
and so on, to establish a match.
Deduplication
Duplicate records, i.e. multiple records referring to the same individual or
business, cause bias and skew, which prevent meaningful data analysis. Worse,
they lead to wasted communications, irritated customers and a ‘don’t care’
impression. To illustrate this, let’s take the example of an insurance firm.
Mr Smith applies for a new car policy. The details he supplies include the
following information:
Mr John Smith, Kings House, Bond Street, Bristol, BS1.
Question:
Is this a new customer or is this a new policy for a customer already in the
database?
To answer this manually, we would probably have to look up all the Smiths in
Bristol. We would try to identify which addresses looked the same and then make
a decision based on experience about whether it was the right one or not. We
would be applying human intelligence and intuition to solve the problem.
Unfortunately, the time taken to do this, coupled with the size of the database and
4.1 – 13
Chapter 4.1 : Marketing database essentials
the volume of updates, makes this approach impractical on a record-by-record
basis, which is why we have to use a computer to do this for us.
But computers are ‘stupid’. They have neither intelligence nor intuition and they
cannot attach value to data. All they can do is follow a series of predefined rules.
How deduplication is used
Deduplication tools are typically used in one or more of the following ways:
G
An online deduplication tool that reduces the number of duplicates
that get onto the database in the first place
G
A housekeeping process that runs in batch mode to check all or part
of the database on a regular basis, or as part of an overall database
cleaning exercise
G
An automatic update process with predefined rules which checks to
see if any new records being entered already exist on the database
The importance of accurate deduplication cannot be overemphasised – you should
be doing whatever it takes to avoid annoying your customers with duplicate
communications. What’s more, because your objective is to build up an accurate
picture of your customer relationships, you need to know that Mr and Mrs Jones
have bought products A and B together, not think that Mr Jones has product A,
while a Ms Janes has Product B!
The converse of this example equally illustrates the risks of over-reliance on autocorrection. Where a match is not close enough for reliable automatic
deduplication, a report is usually generated that lists the exceptions that can then
be manually evaluated and resubmitted. Recent developments in reference or
knowledge-based matching where records are compared to an external database
containing information about consumers with different presentations of their
names (e.g., married and maiden) and addresses over time have made it easier for
marketers to manage the problem of duplicates; these techniques reduce the
number of duplicates and the number of ambiguous matches, thereby reducing
administrative effort.
4.1 – 14
Chapter 4.1 : Marketing database essentials
Figure 4.1.2
,QSXWGDWD
0U56PLWK
+LJK6WUHHW
3266
,QSXWGDWD
5REHUW6PLWK
+LJK6WUHHW
1HZSRUW
,2:
,QSXWGDWD
Database load processing in overview
3URFHVVLQJ
1DPH
DGGUHVV
DQDO\VLV
'HGXSOLFDWLRQ
'DWDEDVH
+LJK6WUHHW
1HZSRUW
,VOHRI:LJKW
3266
0U5REHUW6PLWK
0LVV-DQH6PLWK
0LVV-DQH6PLWK
+LJK6W
1HZSRUW
32
Getting reports from a database
A good marketing database will keep you fully informed about your data – and
your business – by generating a wide range of standard and customisable reports.
At the initial load stage a data audit report is usually generated with details about
the quality of the data that has been loaded. This includes information about:
G
Number of records supplied
G
Number of records rejected
G
Number of records amended
G
Number of records now loaded
Further reports should be able to be generated as the database changes and on
demand.
Monthly/weekly load reports
The monthly/weekly load reports follow the same format as above and provide an
audit trail so that you can see exactly what has been loaded or rejected. One of the
functions of these reports is to reveal anomalies (caused, for example, by the
unsuccessful transmission of a file over the internet) so that you can take the
necessary corrective steps.
4.1 – 15
Chapter 4.1 : Marketing database essentials
Data manipulation – working with the data
Now that you have your data freshly scrubbed and deduplicated and sitting in
your database, you want to do something useful with it!
Campaign management is the most common type of functionality that marketers
seek from their marketing databases – the ability to carry out counts and
selections for campaigns, and manage the process from planning through
execution and back-end analysis. Campaign management tools can offer a widely
varying degree of capability. Entry-level tools allow a marketer to carry out simple
counts and selections on their PC. This can be very useful for carrying out ‘whatif’ analyses, or answering questions like “How many people have not responded to
three or more campaigns?” or “How many customers do we have who have been
on holiday with us to France and Spain?”
More sophisticated tools allow marketers to plan, execute and track the entire
campaign process, from initial counts and selections through data extraction,
fulfilment and response analysis. The kind of tool that is appropriate for your
needs will typically be determined by the quantity and complexity of the
campaigns you carry out. For example, if you are carrying out quarterly direct
mail campaigns with a relatively small number of cells in each mailing, an entrylevel tool will probably provide the most cost-effective option. If you are carrying
out weekly campaigns across a number of channels, or with a large number of
cells, you will probably need something a little more sophisticated.
Only buy the campaign management tool functionality that you will actually
need. There are plenty of examples of campaign management tools being
installed where the client only uses 20 per cent of what the tool can do. As
well as representing a waste of money (20 per cent used, 100 per cent paid
for!), more sophisticated tools often demand more of the user in terms of
training and the tasks involved in setting up and carrying out a campaign; if
the most is being made of the tool’s capabilities, this is a good trade-off, but
if not it represents an avoidable drain on marketing users’ time.
Campaign management functionality is available from a variety of vendors,
including:
G
Alterian
G
Chordiant
G
SmartFOCUS
G
SAS
G
Unica
Marketers may also need lead management and marketing resource management
tools to be provided with their database. Lead management tools are of interest to
marketers who go to market via multiple channels and need to distribute and
track progress on leads that have been generated by marketing programmes. This
type of software allows marketers to control and monitor this process, enabling
them to apply learnings that will increase the number or quality of leads
generated in future programmes or to identify which channel partners are not
performing as well as they could.
4.1 – 16
Chapter 4.1 : Marketing database essentials
Marketing resource management tools enable marketers to automate many of the
aspects of marketing programme design and execution. For example, some tools
allow marketers to manage the complete treatment design process, handling the
flow of draft creative and copy between a client and their agency, including signoffs. These tools also allow marketers to budget and track marketing campaigns
with a high degree of sophistication. These tools are useful where large-scale
campaigns are involved, or if there are a number of departments or parties
involved in campaigns that need to be co-ordinated.
Data analysis – understanding the data
Increasingly, marketers are using analytical techniques to improve the
performance of their campaigns. These were once exclusively the province of
statisticians, but as with many other aspects of technology, tools are emerging
that enable non-specialists to use sophisticated techniques. The kinds of things
that marketers can now do using marketing analytics software ranges from simple
profiling through creating targeting models to quite complex segmentations. While
not requiring a detailed knowledge of statistics, marketers do need a reasonable
understanding of how statistical techniques can be applied to marketing in order
to get the most out of this kind of software, so again, marketers should be
confident they have the right kind of skills available inhouse before investing in
this kind of tool for their marketing database.
The campaign management tool vendors above also offer analytical tools that are
suitable for use by marketers. If you require very sophisticated statistical analysis,
then more specialised tools are available, but these are really only suitable for
users with training in statistics.
For reports that are required week in, week out, or that report on standard
processes, a specialised reporting tool that allows non-technical users to create
reports may be suitable if no similar facility is provided with the database itself.
Techniques used to improve database performance
Summarisation
Let’s consider a customer who has made three purchases. With a normalised
database structure, the customer information would be stored in one table and
the details of each purchasing transaction in another.
Any query that contains a question about the purchases will involve the system
doing a ‘join’ between the tables. It will fetch all the information from the
purchase table, associate it with the customer three times and then answer the
query. However, some of the questions may actually be about the customer rather
than the individual purchases: e.g. “How many purchases has this person made?”
and “What was the total purchase value?”
An excellent technique here is to carry out summarisation on the database. For
example, a new field could be created on the customer record to show the number
of associated purchases, with another field used to show the total value of the
purchases. Both of these fields would be automatically updated each time a new
purchase is recorded.
4.1 – 17
Chapter 4.1 : Marketing database essentials
The benefit of this approach is that you still have full information about the
purchases when you need it but some of your queries can be answered just from
the summarised information on the customer record. Using this technique is not
only faster and less processing-intensive, but it is also much easier to set up the
query in the first place.
Database views
A ‘view’ is a way of showing tables on the database so that they look like
something else. It’s like looking into the same room through different windows.
The contents of the room remain unchanged, but what you see will change
according to your vantage point – either you will see only some of the items, or
you find that you are looking at the same items from a different perspective.
If, for example, you have a table with both customers and prospects, you could
create two views; one of which showed only the prospects and the other the
customers. The user would now see two separate tables and there would be no
confusion over what was being selected.
Setting up views can be a useful way of hiding complexities of a database and
making queries easier, but they should be applied with care. Badly set up views
can significantly impact on performance.
Lookup tables
A lookup table is usually a list of codes and descriptions – a form of ‘shorthand’.
An example might be a list of dealer codes and descriptions in a car company.
B23
H11
R14
Bristol South Motors
High Wycombe Valley Cars
Reading Autos
The idea of using lookup tables is simple but extremely useful. They add several
advantages:
Online data entry is quick and, just as important, accurate
The free text entry problem is resolved (free text is unsuitable for selection
and analysis purposes)
Loaded data can be accurately validated
Storage costs are reduced because only the code is actually stored on each
record
B2B considerations
As mentioned earlier in this chapter, B2B marketing databases present their own
challenges. If you are involved with a B2B database, this section will give you an
initial concept of the considerations that this involves; chapter 4.3 will expand on
these. While there are fewer organisations than there are individuals, there is
much greater complexity associated with data pertaining to businesses, and a B2B
database must provide marketers with capabilities that reflect this complexity and
allow them to target organisations and decision makers accurately.
4.1 – 18
Chapter 4.1 : Marketing database essentials
Some key differences with B2B that a marketing database must accommodate
include:
G
Organisations have names! Sounds obvious, but a B2C database will not
have a special field to hold this data. What often happens in these cases is
that the company name ends up in the first address line, which can not only
make it impossible to accurately identify organisations, but can also cause
problems matching the address to the PAF.
G
Organisations have variations on their names. For example, all of these
could be the same company:
‘The Ultimate Widget Company’
‘TUWC Ltd’
‘Widgets R Us T/A The Ultimate Widget Company’
Your B2B marketing database needs to have specialised deduplication capabilities
that enable you to accurately identify the varied ways in which an organisation’s
name can be presented.
G
Organisations have multiple sites. Depending on your product and service
you may be dealing with the same organisation at a number of its sites or
branches. You may need to be able to recognise that these different locations
belong to the same organisation in order to make the appropriate offer, e.g.
some kind of bulk discount. Your marketing database will therefore need to
be able to link these different locations together.
G
Similarly, organisations can own other organisations with no apparent
similarity in the organisation names. If you are a supplier of car
components you need to know that Land Rover belongs to Ford and Saab
belongs to General Motors, and your database needs to be able to make
these linkages – and maintain them easily, as organisations are constantly
acquiring, merging and divesting.
G
B2B marketing often involves intermediaries or partners. A B2B marketing
database needs to be able to cope with the complexities that this involves.
To take an automotive example, car manufacturers sell a lot of vehicles to
organisations via leasing companies. A car manufacturer’s database
therefore needs to be able to track which organisations are buying cars via
which leasing companies, and to ensure that the right offers are made to
end-user companies based on both the total number of cars bought in a
period and which leasing company may be involved in a particular potential
deal.
G
B2B marketing obviously involves targeting and communicating with
individuals at companies, and therefore the database must be able to
manage this layer of information as well. In addition to the basic contact
information about individuals employed by organisations, the database
must be able to identify (as a minimum) their job title, their role or job
function and their place in the decision-making process for buying a
product or service, e.g. influencer or decision maker.
G
Broadly speaking, there is less external information available about
organisations than there is regarding consumers. However, depending on
the product or service being marketed, there could be a depth of very
specific information required that the database must be able to support. For
example, if your company markets software to organisations you may need
to be able to store detailed information about the kinds of computers
4.1 – 19
Chapter 4.1 : Marketing database essentials
organisations they use, the type of operating systems they use, other
software products they own that may be complementary or competitive to
your products and the number of users they have etc.
G
A B2B database must also be able to associate information at the right level.
For example, contact history should usually be associated with individuals,
whereas a SIC code or annual revenue should be related to the organisation.
Clarity is needed regarding what levels of complexity you have to deal with
in your marketing. For example, if you have a wide range of products or
services that are marketed to multinationals your database might have to
cope with:
O
Ultimate Head Office
O
Regional organisations, e.g. EMEA and Asia Pacific
O
Cluster organisations, e.g. Benelux and UKIE
O
Country organisations, e.g. UK and Ireland
O
Business units, e.g. life and pensions and general insurance
O
Sites or locations
And all of these organisational layers will have individuals associated with them,
and specific information about activities and budgets etc.
Thankfully, most B2B marketing scenarios are less complicated than this. Even
where there is less complexity, it is still possible to come up with database
requirements that are very involved and therefore expensive to implement.
Whenever you’re looking at marketing database requirements it pays to be
pragmatic, and this is particularly the case where B2B marketing is involved.
Trying to accommodate all of the potential complexity can be very expensive,
and also make the database extremely difficult to use. Take a reality check –
how much complexity do you really need? While you may in principle want
to be able to track a number of organisations back to a parent company, does
this add enough value to your marketing to justify the cost of acquiring and
maintaining that data and designing a database structure that can reflect
these links?
Dos and don’ts of building a marketing database
The secret of success with databases lies in planning, consultation and careful
thought. The following checklist will give you a good starting point:
4.1 – 20
Do find out which sources of data are already available within your
organisation before you start. The chances are that you already have a great
deal of data to work with – it’s a question of finding out which department is
sitting on it and how to use it.
Don’t do your planning in isolation. A well-designed marketing database can
be a valuable asset to many different parts of your organisation if their
needs are considered in the planning stage.
Chapter 4.1 : Marketing database essentials
Do be conservative with data collection. Data is expensive so remember the
rule: ‘only collect what you can use – and use all you collect’.
Don’t rush! Careful research and planning in the design stage will deliver a
database that will support your company’s needs for years into the future.
Don’t make false economies. The more you use your database, the faster it
will grow. A scant saving in relatively inexpensive hardware today can have a
disastrous impact on the performance of your database tomorrow.
Do take the time to investigate the many tools and add-ons that can make
your life – and the lives of your colleagues – much easier.
Don’t allow your data to become corrupted. Make sure that your company
agrees, implements and strictly maintains a documented process for
collecting data, updating your database and maintaining data integrity. This
applies to data that is already in your company and data that you may
acquire from external sources.
Do make use of the wealth of experience that is widely available to you.
Read around the subject. Talk to suppliers. Compare notes with your peers
in other companies. Hire consultants.
4.1 – 21
Chapter 4.1 : Marketing database essentials
4.1 – 22