Download Assessing Marketing Performance: Reasons for Metrics

Journal of Marketing Management, 2004, 20,475-498
Assessing Marketing Performance: Reasons for Metrics Selection
Tim Ambler, London Business School 1
Flora Kokkinaki, University of Patras
and Stefano Puntoni, London Business School
In recent years both practitioners and academics have shown an increasing interest in the assessment of
marketing -performance. This paper explores the metrics that firms select and some reasons for those
choices. Our data are drawn from two UK studies. The first reports practitioner usage by the main
metrics categories (consumer behaviour and intermediate, trade customer, competitor, accounting and
innovativeness). The second considers which individual metrics are seen as the most important and
whether that differs by sector. The role of brand equity in performance assessment and top
London Business School* management orientations are key considerations. We found consistency
between orientation and University of Patras^ metrics. Within these categories we identified 19 metrics
that could be regarded as primary and could therefore serve as a short-list for initial selection. However,
the sector importantly moderates that selection, not least because competitive benchmarking requires
similar metrics to be available. Control, orientation and institutional theories appeared to influence
metrics selection and the absence of agency theory is probably due to the research method of this paper.
We concluded with some propositions formally to test the basis of metrics selection.
Keywords: marketing metrics; performance assessment; brand equity; UK firms.
Introduction
Practitioners and academics have shown increasing
interest in the assessment of marketing
performance (Clark 1999; Marketing Week 2001;
Schultz 2000; Shaw and Mazur 1997). The
Marketing Science Institute has raised marketing
metrics to become its leading capital research
project (MSI 2002).
The marketing performance literature has been
criticized for its limited diagnostic power (Day
and Wensley 1988), its focus on the short term
(Dekimpe and Hanssens 1995, 1999), the
excessive number of different measures and the
related difficulty of comparison (Ambler and
Kokkinaki 1997; Clark 1999); the dependence of
the perceived performance on the set of
indicators chosen (Murphy, Trailer and Hill
1996); and the lack of attention to shareholder
value creation (Doyle 2000). "Perhaps no other
concept in marketing's short history has proven
as stubbornly resistant to conceptualization,
definition, or application as that of marketing
performance" (Bonoma and Clark 1988, p. 1).
1
Corresponding author: Tim Ambler, London Business
School, London NWl 4SA. Email: [email protected],
ph/fax: 020 7262 5050/77241145
ISSN0267-257X/2004/3-4/00475 + 23 £8.00/0 ©Westburn
Publishers Ltd.
This paper explores the usage of marketing
metrics in the UK. Marketing is broadly defined
here as what the whole company does to achieve
customer preference and, thereby, its own goals
(Webster 1992). Accordingly, every business has
some interest in assessing marketing in this
sense. Although the usage of marketing metrics
has been increasingly reported (e.g. Shaw 1998,
Ambler 2000), this paper focuses on the
categories of marketing metrics and some
reasons why metrics are chosen. The theoretic
aspects of this research area are not, as yet,
developed and this paper represents a step in that
direction.
First we discuss four theoretical perspectives:
control, agency, institutional and orientation
theories. While the literature is largely based on
US experience, some crossover has taken place
(e.g. The Marketing Leadership Council 2001)
and transnational, or intra-national come to that,
differences seem unlikely to obscure the
fundamental issues of metrics selection. One
relatively new factor has emerged in the last
decade, namely brand equity (Aaker 1991,
1996). The emergence of this intangible market
asset from the shadows has created the need to
measure it and seems likely to be a factor in
metrics selection. After summarizing the
theoretical background, we present a framework
for categorising metrics. This is used in two
empirical studies of metrics usage, the first for
those categories and the second for individual
metrics.
After discussion, limitations and propositions for
future research, we draw some final conclusions.
Theoretical Perspectives
Control Theory
Monitoring
performance
provides
one
informational means to help "planned marketing
activities produce desired results", as stated by
Jaworski (1988, p.24) in his definition of
marketing control. Control theory assumes that
management has a strategy and a known set of
intermediary stages (plans) with which actual
performance can be compared. Metrics selection
is an essentially rational process by which
"marketing managers can learn to improve
performance by altering the utility levels
associated with marketing control variables"
(Fraser and Hite 1988, p.97).
Merchant (1998) defines control as being both
reactive (like a cybernetic feedback loop) and
proactive in anticipating problems before they
can damage performance: "Controls, then,
include all the devices managers use to ensure
that the behaviors and decisions of people in the
organization
are
consistent
with
the
organization's objectives and strategies." (p.2).
This broadens the concept in an interesting
fashion and implies that the costs of control,
including the behavioural effects, need to be
balanced against the benefits. At the same time,
it does not materially change the theory on
control briefly summarised here.
Kotler (2003) lists four types of marketing
controls (table 22.1, p.685): annual-plan,
profitability, efficiency and strategic. These
distinguish whether the company is selecting the
right goals (strategic), whether they are being
achieved (effectiveness or annual-plan), where
the company is making or losing money
(profitability) and the return on each marketing
expenditure (efficiency).
Thus control theory assumes that management
establishes goals of whatever type. Having done
that the metrics necessary to compare goals with
performance are defined.
Agency Theory
A rational-actor perspective is also taken by
agency theory (Jensen and Meckling 1976). In
this case, the focus is on the principal-agent
contractual relationship where the principal has
delegated work to the agent. Agency theory
takes an economic perspective of how
information will be transmitted vertically within
the organization: information that is positive for
the agent will be communicated to the principal
to the extent that the gain obtained from its
disclosure does not exceed the costs of obtaining
and disseminating it.
This is related to control theory in that "agency
theory looks at the relative merits of behaviorbased contracts (...) vis-a-vis outcome-based
contracts (...) as a means of efficiently ensuring
the fidelity of the agents" (Nilakant and Rao
1994, p.653). The greater the difficulty of
effectively measuring marketing performance,
the greater should be the efficiency of behaviorbased forms of control compared to outcomebased forms of control (Fisenhardt 1985). This
implies that when it is more difficult to evaluate
marketing results, more reliance is likely to be
placed on marketing expenditure controls. It also
implies that specialist marketers are likely to
propose metrics that will justify prospective
expenditure (budgets) and past activities.
Institutional Theory
Institutional theory (Meyer and Rowan 1977)
suggests that organizational action is mainly
driven by both the cultural values and the history
of the specific company, as well as by those of
its industry sector. Accordingly, marketing
information disclosure to top management can
be predicted from "perceptions of legitimate
behavior derived from cultural values, industry
tradition, firm history, popular management
folklore and the like" (Fisenhardt 1988, p.492).
The set of marketing metrics selected by a
company therefore is likely to reflect the
intended subjective performance (the indicators
the Board is used to seeing) rather than measures
independent
observers
might
consider
appropriate. Since corporate cultures evolve with
time, we can expect metrics to similarly adapt as
distinct from being created from scratch, e.g. by
a consultancy project.
Success measures can be classified broadly as
either accounting or non-accounting (Frazier and
2 Howell 1982; Buckley et al. 1988). Early work
on firm-level measurement of marketing
performance focused on accounting measures:
profit, sales and cash flow (Sevin 1965, Feder
1965, Day and Fahey 1988). Many authors
however highlighted the problem with using
only accounting indicators in determining
marketing
performance
(e.g., Bhargava,
Dubelaar and Ramaswani 1994; Chakravarthy
1986; Doyle 2000; Eccles 1991). For example,
Chakravarthy (1986) argues that: "accountingmeasure-of-performance record only the history
of a firm. Monitoring a firm's strategy requires
measures that can also capture its potential for
performance in the future" (p.444). The US
Institute of Management Accountants reported
the growing use of non-financial measures (IMA
1993, 1995, 1996).
Clark (1999) showed how traditional accounting
measures (profit, sales, cash flow) expanded to
include of non-accounting (market share,
quality, customer satisfaction, loyalty, brand
equity) measures, as well as wider
considerations covering marketing audit,
implementation and orientation. Clark posited
that the set of selected metrics evolved
incrementally, as suggested by institutional
theory. In recent years the number and variety of
measures available to firms has risen
significantly (Meyer 1998). A literature search in
five leading marketing journals yielded 19
different measures of marketing "success", the
most frequent being sales, market share, profit
contribution and purchase intention (Ambler and
Kokkinaki 1997).
Market Orientation
The literature on market orientation and
corporate culture takes a similar view in that the
concept of marketing adopted within an
organization influences the measurement system
implemented for determining performance
(Moorman 1995; Jaworski 1988; Webster 1992).
The extent to which top management is
interested in assessing marketing, or market
performance, depends on the extent to which
they are market-oriented (Day 1994; Jaworski
and Kohli 1993; Kohli and Jaworski 1990;
Narver and Slater 1990) because market-driven
firms need to gather and disseminate market
intelligence within the organization (Kohli and
Jaworski 1990; Morgan, Katsikeas and AppiahAdu 1998; Slater and Narver 1995). As a
consequence, one of the main features of a
market-oriented organizational culture is the
presence of organization-wide norms for market
orientation (Homburg and Pflesser 2000). These
norms will shape in turn the dynamics of
information disclosure to the top management as
well as the content of such information.
Institutional theory and the concept of market
orientation are related because, as argued by
Dobni and Luffman (2000), "organizations with
similar market orientations have a tendency or
aptitude to engage in similar strategies when in
the same industry, and the types of strategy
chosen are related to the operational behaviors
manifesting a market orientation" (p.9O9).
Brand Equity
Brand equity (Aaker 1991; 1996) is a widely
used term for the intangible marketing asset.
Srivastava and Shocker (1991) define brand
equity as "a set of associations and behaviors on
the part of a brand's customers, channel
members and parent corporation that permits the
brand to earn greater volume or greater margins
than it could without the brand name and that
gives a strong, sustainable and differential
advantage" (p.5).
Brand equity may be measured financially (cf.
Egan and Guilding 1994; Simon and Sullivan
1993) and/or non-financially (cf., e.g., Agarwal
and Rao 1996; Keller 1993, 1998). We treat
financial
measures
synonymously
with
accounting measures, i.e. they are expressed in
currency or as ratios of currency values. On the
other hand, we distinguish between brand equity
(the intangible asset) and brand valuation (the
financial worth of that asset). As brands are
autonomous units for marketing measurement
purposes, multi-brand companies, if they are
assessing their brand equities at all, would need
to assess each brand separately.
For the purposes of this paper, we envisage the
increasing recognition of brand equity as
creating the need for measures of those assets.
Framework for Categorising Metrics
Control theory looks to encouraging behaviour
which leads to the achievement of goals and
these include, but are not limited to, the financial
bottom line. The simplest framework would
simply include a category for the marketing
actions and expenditures (inputs) and the profits
and cash flow (outputs). In practice, those links
3 are not always clear and marketing plans will
have two stages in between: the "intermediate"
measures and consumer behaviours, such as
purchases and loyalty. Intermediate measures are
of customer memories be they attitudes,
intentions, awareness or other cognitive or
affective
or
experiential
brand-linked
characteristics. Thus control theory works
backwards: if the links to financial results are
unclear, consider the links with behaviours. If
those are also unclear, consider the links with
intermediate measures and then those with
behaviours and then those with financial results.
Market-oriented companies will consider the
links in the reverse order: consumers first and
then financial results but the categories are the
same except that competition will be more
carefully considered. Simmonds (1986b) pointed
out that traditional financial accounting fails to
give attention to competitive factors and
proposes that the competitors be tracked on
comparable measures such as sales and profits.
Figure 1 shows how these four categories of
metrics link together. Brand equity consists of
the elements from inputs onwards which have
not yet materialized as sales revenue. Ambler
(2000) describes brand equity as the reservoir of
cash flow that has been earned by good
marketing but has yet to materialise as sales or
profits. Although brand equity, as defined above,
arguably lies in the heads of customers and other
stakeholders, the difficulty of measuring neural
effects leads academics and practitioners to use
proxies, such as inputs and behaviour, to infer
what lies between. Figure 1 shows only one
generic "brain" but in practice brand equity is
measured for each segment separately
considered by management and typically
distinguishes immediate trade customers from
end users.
Although competition, for conceptual purposes,
is shown as an input, competitive metrics arise in
all categories and are usually expressed as
relative measures, e.g. market share, share of
category requirements (loyalty).
Thus Figure 1 provides the framework for the
categories of metrics considered in the first
study:


Own inputs (marketing activities).
Intermediate measures of memory (e.g.
awareness and usage satisfaction, and



attitudes).
Behaviours.
Competitive measures.
Financial outcomes.
Inputs
Intermediate
Behaviour
(eg awareness and commitment)
Competition
Purchases
Loyaity
Word of mouth
Own
Financiai
outcomes
Figure 1. Metrics Categories
Finally we need to consider performance. The
paper assumes that performance, in some sense,
influences the selection of metrics ("what you
measure is what you get"). Swartz, Hardie, Gray
son and Ambler (1996) concluded that marketing
activities broadly achieved planned performance
but the return on marketing investment could not
be compared across companies because their
performance intentions differed. In the two
studies below, performance was first based on
how they were seen by competitors and selfrating of success relative others in their sector.
The second study also used self-ratings but
across four variables: relative to plan, to
previous year's sales, to competitors and overall.
These variables emerged from the empirical
work.
Empirical Analysis of Current Practice
The exploration of how practitioners viewed
metrics occupied two studies. The first grouped
metrics into categories and asked the basis for
comparison, e.g. plan (control theory) or
competition (market orientation). Then we
explored how widely brand equity was used as a
concept and how it was measured. Finally we
looked at whether orientation would be
associated with the frequency of use or
importance of competitor or customer metrics.
The second study switched focus from the
categories, inevitably a somewhat arbitrary
grouping, to the metrics themselves. We sought
the metrics perceived as most important and how
their selection varied by business sector.
4 Study 1
Method
Forty-four in-depth interviews were conducted
with senior marketing and finance managers
from 24 UK firms in order not to restrict the
perspective to the marketers (Homburg et al.
1999). The issues addressed included: the type of
measures collected, the level of review of these
measures (e.g. marketing department. Board),
the assessment of the marketing asset, planning
and benchmarking, practitioners' satisfaction
with their measurement processes and their
views on measurement aspects that call for
improvement, and firm orientation. Information
on firm characteristics, such as size and sector,
was also obtained. Data were collected face-toface and each interview was recorded for
subsequent analysis. After the first 10 interviews
minor changes were made to the interview guide
for clarification and to reduce interview length.
Some respondents took the opportunity to brief
us widely about their companies.
The pilot broadly confirmed the validity of the
five categories above although respondents
preferred to put all financial/accounting and
competitive measures into separate categories,
i.e. not just competitive inputs and financial
outputs. In practice, since most of the inputs
were financial metrics, this removed the "own
inputs" category. Non-retail respondents also
distinguished immediate (trade) customers from
end-users or consumers. They used a separate
category to monitor innovation. Thus we were
left with six categories: consumer intermediate
and behaviour, trade customer, competitive,
innovativeness and accounting (inputs and
outputs). These changes became obvious after
the first 10 interviews and, after making the
changes, the categories were not challenged in
the subsequent 34. The categories may not seem
strictly logical to an outsider but we were
seeking to understand how practitioners grouped
their metrics.
The pilot stage was also used to refine the survey
instrument that was sent to 1014 marketing and
1180 finance senior executives in the UK,
recruited through two professional bodies (The
Marketing Society and the Institute of Chartered
Accountants in England and Wales). A total of
531 questionnaires were returned (367 from
marketers and 164 from finance officers,
response rates 36 percent and 14 percent,
respectively). Table 1 presents a description of
the sample. To encourage response, given the
sensitive nature of this information, the returns
were anonymous so we were unable to check
back for missing values. The majority of the
"other" category is probably explained by large
companies operating in more than one sector,
e.g. high street banks are both retail and B2B.
Table 1. Respondents by Business Size and Sector (Study 1)
(# employees)
Small (<110)
Medium (<500)
Large (>50G)
Missing values
Total
Retail
8
8
51
67
Consumer
goods
7
13
111
131
Consumer
services
14
6
38
58
Respondents were asked to indicate the
importance
attached
to
the
different
measurement categories by top management on a
7-point scale anchored by very important / very
unimportant. They were also asked to report how
regularly data are collected for each measure
category and the benchmark against which each
measure category is compared (previous year,
marketing/business plan, total category data,
specific competitors, other units in the group).
B2B goods
6
7
30
43
B2B services
44
21
38
103
Other
32
12
11
121
Total
111
67
345
8
531
Respondents were then asked whether they have
a term for the main intangible asset built by the
firm's marketing efforts and whether this asset is
formally and regularly tracked, through financial
valuation or other measures. Customer and
competitor orientation were measured with eight
7point scales drawn from Narver and Slater
(1990). Separate single indices of customer and
competitor orientation were computed as the
mean of responses to these items (Cronbach's
alpha .81 and .69, respectively).
5 Results
As shown by Table 2, accounting measures were
reported as being seen by top management as
significantly more important than all other
categories. The t-tests comparing the importance
of accounting measures with other categories
were all significant (p< 0.001). However, the
differences between customer and competitive
measures were small.
Apart from a slightly greater concern with
innovation by marketers, there were no
significant differences between the importance
attributed to each metric category by marketers
and finance respondents (note that both
marketers and finance respondents were asked to
report the importance attached by top
management). Moreover, there were no
significant differences in measurement category
importance between different business sectors.
Table 2.
Importance of Measure Categories for Assessing Performance (Overall)
Financial
Direct customer
Competitive
Consumer intermediate
Consumer behaviour
Innovativeness
Mean
6.51
5.53
5.42
5.42
5.38
5.04
t
df
Sig.
-14.90
-16.78
-15.60
-15.60
20.13
499
523
515
522
524
.000
.000
.000
.000
.000
Note. t- tests refer to the comparisons between financial measures and each of the other categories.
Irrespective of the importance attached to
different indicators, accounting measures were
more frequently collected than any other
category (74.9 percent of the sample reported
that accounting measures were collected at least
on a monthly basis). In 33.5 percent of the cases,
consumer intermediate measures were collected
only rarely/ad hoc. Innovativeness, which some
see as the lifeblood of marketing (e.g. Simmonds
1986a), is least regularly measured (55 percent
of firms measure innovativeness rarely or never).
Larger firms, as might be expected, measure
most categories more frequently than smaller
firms
(p<0.01
for
categories
except
innovativeness). Similarly, business sector was
found to have a significant effect on the
frequency of data collection, with the exception
of innovativeness. Closer inspection of mean
frequency per business sector, however, did not
reveal any systematic differences across sectors.
On average, irrespective of measure category,
consumer goods and retail firms tend to collect
data more frequently than other sectors (F (5,
512) = 11.81, p < .001).
Table 3. Frequency of Benchmarks Used (valid percent) where measure category used)
Accounting measures
Competitive
Consumer behaviour
Consumer intermediate
Direct customer
Innovativeness
Previous
year
80.4
51.4
47,1
36.7
40.3
21.3
Marketing
Business Plan
85.1
51.0
42.0
30.3
37.7
33.7
Metrics Comparisons
Plans provided the most frequent benchmarks of
accounting and innovativeness measures, where
such measures are used (Table 3). Competitive
measures, however, were compared with market
research rather than forecast in plans. Consumer
and direct customer measures were most
typically compared with previous year results.
Market share apart, it appears that internal (plan)
Total
category data
17.5
35.8
27.1
22.0
17.3
10.9
Specific
competitor(s)
23.0
55.7
31.6
27.7
22.8
20.7
Other units in
the Group
22.0
6.6
6.4
5.1
7.3
6.6
and external benchmarks were routinely used
only by the minority of respondent firms. The
modal frequency for each row is highlighted in
the table.
Brand Equity
Moving now to the marketing asset, 62.2 percent
of the respondents reported the use of some term
to describe the concept. The most common terms
were brand equity (32.5 percent of those who
6 reported a term), reputation (19.6 percent), brand
strength (8.8 percent), brand value (8.2 percent)
and brand health (6.9 percent). Twenty percent
of those who used a term reported one or more
of 65 different terms, such as brand integrity,
customer loyalty, global image, quality, contact
base and trademark value.
24.9 percent of the total sample regularly (yearly
or more) valued the marketing asset financially,
and 41 percent regularly quantified it in other
ways, e.g. through customer/consumer based
measures (see Table 4). Less than 15 percent of
the total sample did both.
Table 4. Regularity of Tracking the Marketing Asset (valid percent of the total sample)
Financial valuation
Other measures
Never
51.4
36.8
Rarely/ Ad hoc
23.6
22.2
Regularly Yearly/ Quarterly
16.9
28.7
Our theoretical discussion suggested that the
marketing performance assessment system can
be tested against three criteria: benchmarking
against internal expectations (plan) and external
(competitor) performance, adjusted by changes
in brand equity. Of the 196 respondents who
reported quantifying their marketing assets
(either financially and/or in other ways), 24
Monthly or more
8.0
12.3
percent also quantify consumer, competitive or
direct customer measures in their business plan
(internal) and use market or competitive
benchmarks. Thus, on this survey, less than one
quarter of firms could meet all three criteria. Of
course, these data merely indicate that they have
the measures available for such comparisons, not
that they necessarily make such comparisons.
Table 5. Regression of Regularity of Tracking on Customer and Competitor Orientation
Accounting
Competitive
Consumer behaviour
Consumer intermediate
Direct customer
Innovativeness
***
F
210.40***
39.17***
11.14***
12.07***
8.53***
9.01***
R
.27
.48
.28
.31
.26
.26
R2
.07
.23
.08
.09
.07
.07
Customer Orientation
beta
t
.03
.83
.03
.81
.18
3.78***
.23
4.88***
.16
3.29***
.24
5.05***
Competitor Orientation
beta
t
.05
1.13
.22
5.31***
.07
1.50
.00
.16
.00
.04
.03
.71
p < .001 Market Orientation
In order to examine whether customer and
competitor orientation have an effect on
performance assessment practices, regularity of
tracking and importance attached to different
measures were regressed simultaneously on
these constructs, after partialling out the effect of
firm size and sector. As can be seen in Table 5,
customer orientation was strongly associated
only with the regularity of collection of
consumer, direct customer and innovativeness
measures. As might be expected, more customeroriented firms tended to collect data on such
measures more frequently than firms less so
oriented. Customer orientation does not seem to
be related to the regularity of tracking of
accounting and competitive measures, whereas
the regularity of collection of competitor
measures was found to be related to the level of
competitor orientation.
Table 6. Regression of Measure Category Importance on Customer and Competitor Orientation
Accounting
Competitive
Consumer behaviour
Consumer intermediate
Direct customer
Innovativeness
F
4.86***
27.88***
19.05***
21.45***
9.40***
9.80***
R
.19
.42
.36
.38
.26
.26
R2
.00
.18
.13
.14
.07
.07
Customer Orientation
beta
t
2.19*
.10
1.30
.05
.25
5.64***
.30
6.68***
.22
4.67***
.20
4 29***
Competitor Orientation
beta
t
.03
.71
.26
6.03***
.16
3.57***
.09
2.18*
.07
1.62
.10
2.32*
7 The relation between orientation and measure
importance is less clear (Table 6). Customer
orientation was positively related with the
importance attached to most measures except
competitive measures, although the correlation
with accounting measures was relatively weak.
Competitor orientation strongly correlated with
the importance of competitor and consumer
behaviour measures and less robustly with
intermediate and innovative measures.
Both customer and competitor orientation were
significantly associated with assessment
practice. No moderating effects by sector were
observed, but firm size appeared to moderate the
importance attached to measures.
The results indicate that customer orientation is a
stronger predictor of the importance of
competitive measures for large firms, compared
to small firms {beta of the interaction term = .50,
t = 2.60, p < .01). However, competitor
orientation was found to be a stronger predictor
of importance for small firms, compared to large
firms {beta of the interaction term = -.46, t = 2.57, p < .01). We have no simple explanation
for these results. It is possible that larger UK
firms need to be more customer-oriented in order
to be effective, perhaps because they already are
competitor oriented. In contrast, in the case of
smaller UK firms, competitor orientation seems
to be more important, perhaps because they
already are customer oriented.
Study 2
Up to this point, we have explored practitioner
usage based on the metric categories first
identified in the framework and revised in the
pilot stage of Study 1. The next step was
therefore to perform a study specifically aimed
at designing a list of most frequently used
metrics. Providing useful indications to
managers and stimulating further academic
research, such endeavour was thought to be
important for a complete understanding of
current practice in the area of marketing
performance assessment. In particular, while we
expected to find wide variation by sector and
firm, we explored the extent to which certain
metrics stood out as more valuable, or at least
more widely used, than others.
Method
An initial survey instrument with 54 metrics was
developed, using the relevant literature, and
piloted to establish any additional measures and
to eliminate those measures that were not used or
were redundant. No additions were made but 16
were eliminated. The resulting 38 measures were
classified into the six categories described
above.
A telephone survey was then conducted with 200
UK marketing or finance senior executives
drawn from the lists supplied by The Marketing
Society and The Institute of Chartered
Accountants in England and Wales. In both
cases, only senior practitioners were selected.
The acceptance level for the telephone
interviews, i.e. response level, was 50.1 percent.
This excludes wrong numbers and other
technical blockages. Since the survey instrument
was not materially altered by the pilot stage (the
eliminated metrics had been left blank) we added
the responses of the 31 executives who
participated in the Study 2 pilot.
Respondents were asked to indicate the
importance of each measure for assessing the
overall marketing performance of the business
on a 5-point scale. They were also asked to
indicate the highest level of routine review of
this metric within the firm, on a scale ranging
from the [group's] top board level (5) through
junior marketing (1) to not used at all (0).
Respondents were also asked to add any relevant
measures not listed.
Contextual data were also collected in order to
determine the impact of environmental factors
such as firm size, business sector, organization
structure, and age of business. Table 7 shows a
broad spread across firm size and sector.
Table 7. Respondents by Structure and Sector (Study 2)
# Employees
One unit without marketing dept
One unit with marketing dept
Subsidiaries with one board
More complex
Missing values
Total
Retail
4
5
7
6
22
Consumer
goods
4
7
8
13
32
Consumer
Services
17
6
5
10
23
B2B
goods
4
11
11
15
41
B2B
Services
23
15
13
15
66
Other
9
6
5
25
2
47
Total
46
50
49
84
2
231
8 The great majority of the firms had been in
business for more than five years and therefore
have reporting systems that have evolved beyond
the start-up phase and to that extent become
established. As was the case in Study 1, the
principle reason for the companies not attributed
to any sector is due to large firms that trade in
more than one sector.
Results
Table 8 ranks the top 15 (> 62 percent usage)
metrics by frequency of use compared with the
frequency that it was rated as "very important"
and the frequency that it reached the top level of
management.
Table 8. Ranking of Marketing Metrics
Metric
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
% claiming
to use
measure
% firms rating
as very
important
% claimed
to reach top
level
Pearson
Correlation
between Level
and Importance
92
91
81
78
78
73
70
69
80
71
66
28
37
18
36
45
71
65
58
29
34
19
33
31
.729**
.758**
.827**
.732**
.727**
.859**
.735**
.802**
68
66
66
65
64
64
63.
48
18
24
39
37
47
39
37
11
23
46
32
34
30
.815**
.900**
.812**
.849**
.783**
.830**
.814**
Profit/Profitability
Sales, Value and/or Volume
Gross Margin
Awareness
Market Share (Volume or Value)
Number of New Products
Relative Price (SOMValue/Volume)
Number of Consumer Complaints
(Level of dissatisfaction)
Consumer Satisfaction
Distribution/ Availability
Total Number of Customers
Marketing Spend
Perceived Quality/esteem
Loyalty/Retention
Relative perceived quality
n = 231, ** p < .01
The last column of Table 8 highlights the have a significant effect on the usage of specific
expected correlation between the measure being items, particularly consumer intermediate,
seen as very important and its review by the top competitive and accounting measures. Table 9
management level. Business sector was found to shows the 15 most significant differences of 38.
Table 9. ANOVA for Significant Metric Variations by Business Sector
Level of Importance
df
F
Other attitudes e.g. liking
Image/ personality/ identity
Penetration
Salience
Commitment/purchase intent
Distribution/ availability
Awareness
Relevance to consumer
Marketing spend
Market share
Share of voice
Brand/product knowledge
Conversions
Margin of new products
Purchasing on Promotion
228
226
225
225
228
218
227
226
228
226
225
228
225
225
225
8.88***
7.91***
7.67***
3.81
6.88***
6.83***
5.84***
5.34***
5.12***
5.02***
4.93***
4.54**
2.55**
3.95**
11.87***
Level of Review
df
F
229
229
229
229
229
228
229
228
229
229
229
229
228
228
229
4.66***
4.36**
3.07**
3.15**
3.45**
6.14***
3.05**
2.61**
2.55**
2.63**
4.82**
2.62**
2.96**
2.60**
5.88***
n = 231, ** p < .05, *** p < .001
9 Distinctions between sectors were greater for
level of importance measures than level of
review. As would be expected, consumer
oriented items were found to be more important
for consumer sectors. Perhaps the most
surprising result is the variation in importance
ascribed to market share across business sectors.
As might be expected, consumer goods firms
consistently rated metrics as more important and
reviewed at more senior levels than business-tobusiness services.
Developing Primary Metrics
Finally we analysed which metrics are
candidates for being seen as the most valuable or
primary, irrespective of sector or size. Content
validity (Churchill 1979) using a 50 percent cutoff (Cronbach and Meehl 1955) left 30 metrics
that were then subjected to scale purification
procedures. Construct validity was assessed with
the guidelines outlined by Churchill (1979) and
Gerbing and Anderson (1987). We examined
item-to-total correlations and the factor structure
(through principal components) for each scale.
The decision criterion for item deletion was an
improvement in corresponding alpha values to
the point at which all items retained had
corrected item-total correlations greater than 0.5.
Fight items were eliminated, varying slightly as
to whether the level of review or level of
importance was considered.
Both level of review and importance are shown
for comparison in Table 10. 19 items match and
could be considered as the primary general
metrics:
Awareness,
Perceived
quality,
Consumer satisfaction, Relevance to consumer,
Perceived
differentiation,
Brand/product
knowledge, Number of new customers,
Loyalty/retention,
Conversions,
[Trade]
Customer satisfaction, Number of complaints,
Relative consumer satisfaction, Perceived
quality, Number of new products, Revenue of
new products, Margin of new products, Sales,
Gross margins, Profitability.
Table 10. Primary Metrics
If level of review is measured
Construct
Alpha
Items
If level of importance is measured
Alpha
Items
Awareness
Perceived quality
Consumer satisfaction
Relevance to consumer
Image/personality
Perceived differentiation
Brand/product knowledge
Total number of consumers
Number of new consumers
Loyalty/retention
Conversions
Number of consumer complaints
Customer satisfaction
Number of complaints
.84
Awareness
Perceived quality
Consumer satisfaction
Relevance to consumer
Perceived differentiation
Brand/product knowledge
.83
Number of new consumers
Loyalty
Leads generated
Conversions
.79
.79
Relative consumer satisfaction
Perceived quality
.80
Innovation
.84
Number of new products
Revenue of new products
Margin of new products
.81
Distribution/ availability
Customer satisfaction
Number of customer complaints
Relative consumer satisfaction
Perceived quality
Share of voice
Number of new products
Revenue of new products
Margin of new products
Accounting
.81
Sales
Gross margins
Profitability
.77
Consumer Attitudes
.85
Consumer Behaviour
.78
Trade Customer
.80
Relative to Competitor
TOTAL : 22 items
Sales
Gross margins
Profitability
TOTAL: 22 items
n = 231
10 Discussion
The primacy of accounting metrics, both in
terms of importance attributed by the
respondents and of regularity of assessment, is
consistent with the literature (e.g. Clark 1999).
We were surprised by the relatively low levels
reported for basics such as sales and
profitability. Every board must see these figures
as part of their financial accounts but the
respondents here were reporting on what they
perceive to be marketing. The role of
"marketing" varies widely across UK companies
(Ambler 2000).
In our analysis, brand equity can provide the
bridge between short- and the long-term effects
as regarded important by Dekimpe and Hanssens
(1995). Although a substantial proportion of our
respondents were measuring brand equity
financially or non-financially, it seems likely
that a formal (control theory) process rarely
meets all three marketing performance
assessment criteria above, i.e. internal and
external benchmarking adjusted by any change
in brand equity. Less than 25 percent of our
respondents had the data to do so.
Orientation was consistent with metrics usage as
shown for regularity and importance (Tables 5
and 6 respectively).
Thus we found some support for control,
institutional (sector differences) and orientation
theories in the selection of metrics but little
support for agency theory. This last would
require analysis of the interactions between the
Board and junior levels of the firm, notably in
respect of budget approvals. From the literature
and the two studies, we can advance some
propositions about how metrics are adopted with
some implications for the way they should be.
P1: Strongly control oriented top management
will review those metrics projected in the
marketing plan. Apart from accounting
measures, we found that between a third
and one half of measures were compared to
plan but we expect that to be moderated by
control orientation.
P2: Following institutional theory, metrics will
evolve a-rationally in conformity with
sector norms. We do not expect that firms
can provide rational explanations for the
metrics they adopt.
P3:
Following orientation theory, metrics
selection will reflect the primary interests of
top management, e.g. customer, competitor
or internal accounting measures.
P4: Agency theory will provide explanation of
metrics selection only in the context of
budget negotiation and subsequent interlevel evaluation of performance.
Limitations and Future Research
Our findings are limited in several respects. First
the research, as with other survey-based
methods, does not capture causality nor the
dynamics of the development of measurement,
orientation and performance. In future research
linking metrics selection with performance, it
may be important to distinguish the types of
performance sought by management rather than
build a signal performance construct, i.e. do
companies, in fact, get what they measure?
We examined the potential effects of size and
sector, but we did not consider external
environmental effects. Market turbulence, for
example, may moderate metrics selection and
their value (Greenley 1995, Harris 2001).
Although we used the literature from elsewhere,
notably the US, the empirical work was
conducted in the UK. It is likely that there are
variations internationally in metrics selection,
not least because the metrics available from
suppliers will differ in other countries. It would
be more interesting to explore what, if any,
theoretical differences exist. Finally, the whole
area of linking non-financial market measures to
financial outcomes, such as shareholder value,
has barely been explored.
Conclusions
This paper has contributed two exploratory UK
studies to the increasing interest in the
assessment
of
marketing
performance.
Accounting remains the dominant metrics
category relative to consumer behaviour and
intermediate, trade customer, competitor, and
innovativeness. Brand equity is widely measured
but rarely integrated into a formal assessment
system. We found consistency between
orientation and metrics. Within these categories
we identified 19 metrics that could be regarded
as primary and could therefore serve as a shortlist for initial selection. However, the sector
importantly moderates that selection, not least
11 because competitive benchmarking requires
similar metrics to be available. Control,
orientation and institutional theories appeared to
influence metrics selection and the absence of
agency theory is probably due to the research
method of this paper. We concluded with some
propositions formally to test the basis of metrics
selection.
Dekimpe, Marruk G. and Hanssens, Dominique M. (1995), "The
Persistence of Marketing Effects on Sales", Marketing
Science, Vol.14, No. 1, pp.1-21
Dekimpe, Marnik G. and Hanssens, Dominique M. (1999),
"Sustained Spending and Persistent Response: A New Look
at Long-Term Marketing Profitability", Journal of
Marketing Research, Vol.36, No.4, pp.397-412
Dobni, C. Brooke and . Luffman, George (2000), "Implementing
Marketing Strategy Through Market Orientation", Journal
of Marketing Management, Vol.16, pp.895-916
Doyle, Peter (2000), Value-Based Marketing, Chichester, UK,
Wiley
Acknowledgements
We are grateful to Debra Riley who conducted much of the
earlier research and to The Marketing Society, The Marketing
Council, Institute for Practitioners in Advertising, Sales
Promotions Consultants Association, London Business School
and Marketing Science Institute for sponsoring this research.
References
Aaker, David A. (1991), Managing Brand Equity, New York,
Free Press
Aaker, David A. (1996), Building Strong Brands, New York, Free
Press
Agarwal, Manoj K. and Rao, Vithala R. (1996), "An Empirical
Comparison of Consumer-Based Measures of Brand
Equity", Marketing Letters, Vol.7, No.3, pp.237-247
Ambler, Tim (2000), Marketing and the Bottom Line: The New
Metrics of Corporate Wealth, London, Prentice
Hall/Financial Times
Ambler, Tim and Kokkinaki, Flora (1997), "Measures of
Marketing Success", Journal of Marketing Management,
Vol.13, pp.665-678
Bhargava, Mukesh, Dubelaar, Chris and Ramaswami, Sridhar
(1994), "Reconciling Diverse Measures of Performance: A
Conceptual Framework and Test of a Methodology",
Journal of Business Research, Vol.31, pp.235-246
Bonoma, Thomas V. and Clark, Bruce H. (1988), Marketing
Performance Assessment, Boston, Harvard Business School
Press
Buckley, Ronald, Hall, Sid, Benson, Philip G. and Buckley, M.
(1988), "The Impact of Rating Scale Format on Rater
Accuracy: An Evaluation", Journal of Management,
Vol.14, No.3, pp.415-423
Chakravarthy, Balaji S. (1986), "Measuring Strategic
Performance", Strategic Management Journal, Vol.7,
pp.437-458
Churchill, Gilbert A. Jr. (1979), "A Paradigm for Developing
Better Measures of Marketing Constructs", Journal of
Marketing Research, Vol.16, No. 1, pp.64-73.
Eccles, Robert G. (1991), "The Performance Measurement
Manifesto", Harvard Business Review, Vol.69, No. 1,
pp.131-137
Egan, Colin and Guilding, Christopher (1994), "Dimensions of
Brand Performance: Challenges for Marketing
Management and Managerial Accountancy", Journal of
Marketing Management, Vol.10, pp.449-472
Eisenhardt, Kathleen M. (1985), "Control: Organizational and
Economic Approaches", Management Science, Vol.31,
No.2, pp.134-149
Eisenhardt, Kathleen M. (1988), "Agency- and InstitutionalTheory Explanations: The Case of Retail Sales
Compensation", Academy of Management Journal, Vol.31,
No.3, pp.488-511
Feder, Robert A. (1965), "How to Measure Marketing
Performance", Harvard Business Review, Vol.43, No.3,
pp.132-142
Fraser, Cynthia and Hite, Robert F. (1988), "An Adaptive Utility
Approach for Improved Use of Marketing Models",
Journal of Marketing, Vol.52, No.4, pp.96-103
Frazier, Gary L. and Howell, Roy D. (1982), "Intra-industry
Marketing Strategy Effects on the Analysis of Firm
Performance", Journal of Business Research, Vol.10, No.4,
pp.431-443
Gerbing, David W. and Anderson, James (1988), "An Updated
Paradigm for Scale Development Incorporating
Unidimensionality and its Assessment", Journal of
Marketing Research, Vol.25, No.2, pp.186-92.
Greenley, Gordon F. (1995), "Market Orientation and Company
Performance: Empirical Evidence From UK Companies",
British Journal of Management, Vol.6, pp.1-13
Harris, Lloyd C. (2001), "Market Orientation and Performance:
Objective and Subjective Empirical Evidence From UK
Companies", Journal of Management Studies, VoL38 No.
1, pp.17-43
Homburg, Christian, Workman, John P. Jr. and Krohmer, Harley
(1999), "Marketing's Influence Within the Firm", Journal
of Marketing, Vol.63, No.2, pp.1-17
Clark, Bruce H. (1999), "Marketing Performance Measures:
History and Interrelationships", Journal of Marketing
Management, Vol.15, pp.711-732
Homburg, Christian and Pflesser, Christian (2000), "A MultipleLayer model of Market-Oriented Organizational Culture:
Measurement Issues and Performance Outcomes", Journal
of Marketing, Vol.37, No.4, pp.449-462
Cronbach, L.J. and Meehl, P.F. (1955), "Construct Validity in
Psychological Tests", Psychological Bulletin, Vol.52,
pp.281-302
IMA (1992), Cost Management Update, October 1992 Vol.32, US
Institute of Management Accountants, Cost Management
Group
Day, George S. (1994), "The Capabilities of Market Driven
Organizations", Journal of Marketing, 58, No.4, pp.37-52
IMA (1994), Cost Management Update, March 1994 Vol.49, US
Institute of Management Accountants, Cost Management
Group
Day, George S. and Fahey, Liam (1988), "Valuing Market
Strategies", Journal of Marketing, Vol.52 No.3, pp.45-57
Day, George S. and Wensley, Robin (1988), "Assessing
Advantage: A Framework for Diagnosing Competitive
Superiority", Journal of Marketing, Vol.52, No.2, pp.1-20
IMA (1996), Cost Management Update, June 1996 Vol.69, US
Institute of Management Accountants, Cost Management
Group
12 Jaworski, Bernard J. (1988), "Toward a Theory of Marketing
Control: Environmental Context, Control Types, and
Consequences", Journal of Marketing, Vol.52, No.3, pp.2339
Jaworski, Bernard J. and Kohli, Ajay K. (1993), "Market
Orientation: Antecedents and Consequences", Journal of
Marketing, Vol.57, No.3, pp.5370
Jensen, Michael C. and Meckling, William H. (1976), "Theory of
the Firm: Managerial Behavior, Agency Costs and
Ownership Structure", Journal of Financial Economics,
Vol.3, pp.305-360
Keller, Kevin L. (1993), "Conceptualizing, Measuring, and
Managing Customer-Based Brand Equity", Journal of
Marketing, Vol.57, No. 1, pp.1-22
Keller, Kevin L. (1998), Strategic Brand Management: Building,
Measuring and Managing Brand Equity, Upper Saddle
River, NJ, Prentice-Hall
Kohli, Ajay K. and Jaworski, Bernard J. (1990), "Market
Orientation: The Construct, Research Proposition, and
Managerial Implications", Journal of Marketing, Vol.54,
No.2, pp.1-18
Kotler, Philip (2003), Marketing Management, (11th edition).
Upper Saddle River, NJ: Prentice Hall
Marketing Week (2001), "Assessing Marketers' Worth", February
2001 No. 1, pp. 42-43
Merchant, Kenneth A. (1998), Modern Management Control
Systems, Upper Saddle River, NJ: Prentice Hall
Simmonds, Kenneth (1986a), "Marketing as Innovation. The
Eighth Paradigm", Journal of Management Studies, Vol.23,
No.3, pp.479-500
Simmonds, Kenneth (1986b), "The Accounting Assessment of
Competitive Position", European Journal of Marketing,
Vol.20, No. 1, pp.16-31
Simon, Carol J. and Sullivan, Mary W. (1993), "The Measurement
and Determinants of Brand Equity: A Financial Approach",
Marketing Science, Vol.12, No. 1, pp.28-51
Slater, Stanley F. and Narver, J.C. (1994), "Does Competitive
Environment Moderate the Market OrientationPerformance Relationship?", Journal of Marketing, Vol.58,
No. 1, pp.46-55
Slater, Stanley F. and Narver, John C. (1995), "Market Orientation
and the Learning Organization", Journal of Marketing,
Vol.59, No.3, pp.63-74
Srivastava, Rajendra K. and Shocker, Allan D. (1991), "Brand
Equity: A Perspective on its Meaning and Measurement",
Working Paper No. 91-124, Cambridge, MA, Marketing
Science Institute
Swartz, G., Hardie Bruce, Grayson, Kent and Ambler, Tim
(1996), "Value for money? The relationships between
marketing expenditure and business performance in the UK
financial services industry", Cookham, Berkshire, UK,
Chartered Institute of Marketing, April
Webster, Frederick E. Jr. (1992), "The Changing Role of
Marketing in the Corporation", Journal of Marketing,
Vol.56, No.4, 1-17
Meyer, John W. and Rowan, Brian (1977), "Institutionalized
Organizations: Formal Structure as Myth and Ceremony",
American Journal of Sociology, Vol.83, pp.340-363
Meyer, Marshall W. (1998), "Finding Performance: The new
discipline in management". Performance MeasurementTheory and Practice, Vol. 1, Cambridge, UK, Centre for
Business Performance, xiv-xxi
Moorman, Christine (1995), "Organizational Market Information
Processes: Cultural Antecedents and New Product
Outcomes", Journal of Marketing Research, Vol.32, No.3,
pp.318-335
Morgan, Robert F., Katsikeas, Constantine S. and Appiah-Adu,
Kwaku (1998), "Market Orientation and Organizational
Learning Capabilities", Journal of Marketing Management,
Vol.14, pp.353-381
MSI (2002), 2002-2004 Research Priorities: A Guide to MSI
Research Programs and Procedures, Cambridge, MA,
Marketing Science Institute
Murphy, Gregory B., Trailer, Jeff W. and Hill, Robert C. (1996),
"Measuring Performance in Entrepreneurship Research",
Journal of Business Research, Vol.36, pp.15-23
Narver, John C. and Slater, Stanley F. (1990), "The Effect of a
Market Orientation on Business Profitability", Journal of
Marketing, Vol.54, No.4, pp.20-35
Nikalant, V. and Rao, Hayagreeva (1994), "Agency Theory and
Uncertainty in Organizations: An Evaluation",
Organization Studies, Vol.15, No.5, pp.649-672
Sevin, Charles H. (1965), Marketing Productivity Analysis, New
York, McGraw-Hill Schultz, Don E. (2000),
"Understanding and Measuring Brand Equity", Marketing
Management, Spring 2000, pp.8-9 Shaw, Robert (1998),
Improving Marketing Effectiveness, London, The
Economist Profile Books
Shaw, Robert and Mazur, Laura. (1997), Marketing
Accountability: Improving Business Performance, London,
Financial Times, Retail and Consumer Publishing
About the Authors
Tim Ambler is Senior Fellow at London Business School. His
primary research interests concern international marketing and the
measurement of marketing and advertising performance. More
recently he has broadened these to include the quantification of
government waste (bureaucracy and unnecessary regulation and
red tape). His books include Marketing and the Bottom Line: The
New Metrics of Corporate Wealth (2000), Doing Business in
China (2000), The SILK Road to International Marketing (2000)
and Marketing from Advertising to Zen (1996). He has published
in the Journal of Marketing, Journal of Marketing Research,
International Journal of Research in Marketing, Journal of
Advertising Research and International Journal of Advertising.
He was previously Joint Managing Director of International
Distillers and Vintners which is now part of Diageo plc.
Flora Kokkinaki is a lecturer in Social Psychology at the
University of Patras. After the completion of her PhD in
Psychology at University College London, she joined London
Business School as Research Fellow working on the assessment
of marketing effectiveness. After appointments as lecturer at
University College London and the London School of Economics
and Political Science, she joined the University of Patras. Her
research revolves around the areas of Social Cognition, Consumer
Behaviour and Economic Psychology. Her publications include
papers in the British Journal of Social Psychology, the Journal of
Economic Psychology and the Journal of Marketing Management.
Stefano Puntoni is a doctoral student in marketing at London
Business School. He holds a Master in Statistics from the
University of Padua (Italy). Stefano has published in the areas of
marketing metrics, consumer behaviour, marketing
communications and pricing and participated to various marketing
conferences and symposia. He is currently working on a research
project devoted to a systematic analysis of the antecedents of
advertising polysemy.
13