* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Assessing Marketing Performance: Reasons for Metrics
Customer experience wikipedia , lookup
Brand awareness wikipedia , lookup
Customer relationship management wikipedia , lookup
Social media marketing wikipedia , lookup
Market segmentation wikipedia , lookup
Brand ambassador wikipedia , lookup
Bayesian inference in marketing wikipedia , lookup
Brand loyalty wikipedia , lookup
Sales process engineering wikipedia , lookup
Consumer behaviour wikipedia , lookup
Affiliate marketing wikipedia , lookup
Food marketing wikipedia , lookup
Product planning wikipedia , lookup
Brand equity wikipedia , lookup
Customer satisfaction wikipedia , lookup
Marketing communications wikipedia , lookup
Target audience wikipedia , lookup
Sports marketing wikipedia , lookup
Customer engagement wikipedia , lookup
Ambush marketing wikipedia , lookup
Neuromarketing wikipedia , lookup
Marketing channel wikipedia , lookup
Multi-level marketing wikipedia , lookup
Guerrilla marketing wikipedia , lookup
Marketing research wikipedia , lookup
Digital marketing wikipedia , lookup
Integrated marketing communications wikipedia , lookup
Viral marketing wikipedia , lookup
Youth marketing wikipedia , lookup
Target market wikipedia , lookup
Marketing strategy wikipedia , lookup
Direct marketing wikipedia , lookup
Advertising campaign wikipedia , lookup
Marketing plan wikipedia , lookup
Multicultural marketing wikipedia , lookup
Street marketing wikipedia , lookup
Green marketing wikipedia , lookup
Marketing mix modeling wikipedia , lookup
Journal of Marketing Management, 2004, 20,475-498 Assessing Marketing Performance: Reasons for Metrics Selection Tim Ambler, London Business School 1 Flora Kokkinaki, University of Patras and Stefano Puntoni, London Business School In recent years both practitioners and academics have shown an increasing interest in the assessment of marketing -performance. This paper explores the metrics that firms select and some reasons for those choices. Our data are drawn from two UK studies. The first reports practitioner usage by the main metrics categories (consumer behaviour and intermediate, trade customer, competitor, accounting and innovativeness). The second considers which individual metrics are seen as the most important and whether that differs by sector. The role of brand equity in performance assessment and top London Business School* management orientations are key considerations. We found consistency between orientation and University of Patras^ metrics. Within these categories we identified 19 metrics that could be regarded as primary and could therefore serve as a short-list for initial selection. However, the sector importantly moderates that selection, not least because competitive benchmarking requires similar metrics to be available. Control, orientation and institutional theories appeared to influence metrics selection and the absence of agency theory is probably due to the research method of this paper. We concluded with some propositions formally to test the basis of metrics selection. Keywords: marketing metrics; performance assessment; brand equity; UK firms. Introduction Practitioners and academics have shown increasing interest in the assessment of marketing performance (Clark 1999; Marketing Week 2001; Schultz 2000; Shaw and Mazur 1997). The Marketing Science Institute has raised marketing metrics to become its leading capital research project (MSI 2002). The marketing performance literature has been criticized for its limited diagnostic power (Day and Wensley 1988), its focus on the short term (Dekimpe and Hanssens 1995, 1999), the excessive number of different measures and the related difficulty of comparison (Ambler and Kokkinaki 1997; Clark 1999); the dependence of the perceived performance on the set of indicators chosen (Murphy, Trailer and Hill 1996); and the lack of attention to shareholder value creation (Doyle 2000). "Perhaps no other concept in marketing's short history has proven as stubbornly resistant to conceptualization, definition, or application as that of marketing performance" (Bonoma and Clark 1988, p. 1). 1 Corresponding author: Tim Ambler, London Business School, London NWl 4SA. Email: [email protected], ph/fax: 020 7262 5050/77241145 ISSN0267-257X/2004/3-4/00475 + 23 £8.00/0 ©Westburn Publishers Ltd. This paper explores the usage of marketing metrics in the UK. Marketing is broadly defined here as what the whole company does to achieve customer preference and, thereby, its own goals (Webster 1992). Accordingly, every business has some interest in assessing marketing in this sense. Although the usage of marketing metrics has been increasingly reported (e.g. Shaw 1998, Ambler 2000), this paper focuses on the categories of marketing metrics and some reasons why metrics are chosen. The theoretic aspects of this research area are not, as yet, developed and this paper represents a step in that direction. First we discuss four theoretical perspectives: control, agency, institutional and orientation theories. While the literature is largely based on US experience, some crossover has taken place (e.g. The Marketing Leadership Council 2001) and transnational, or intra-national come to that, differences seem unlikely to obscure the fundamental issues of metrics selection. One relatively new factor has emerged in the last decade, namely brand equity (Aaker 1991, 1996). The emergence of this intangible market asset from the shadows has created the need to measure it and seems likely to be a factor in metrics selection. After summarizing the theoretical background, we present a framework for categorising metrics. This is used in two empirical studies of metrics usage, the first for those categories and the second for individual metrics. After discussion, limitations and propositions for future research, we draw some final conclusions. Theoretical Perspectives Control Theory Monitoring performance provides one informational means to help "planned marketing activities produce desired results", as stated by Jaworski (1988, p.24) in his definition of marketing control. Control theory assumes that management has a strategy and a known set of intermediary stages (plans) with which actual performance can be compared. Metrics selection is an essentially rational process by which "marketing managers can learn to improve performance by altering the utility levels associated with marketing control variables" (Fraser and Hite 1988, p.97). Merchant (1998) defines control as being both reactive (like a cybernetic feedback loop) and proactive in anticipating problems before they can damage performance: "Controls, then, include all the devices managers use to ensure that the behaviors and decisions of people in the organization are consistent with the organization's objectives and strategies." (p.2). This broadens the concept in an interesting fashion and implies that the costs of control, including the behavioural effects, need to be balanced against the benefits. At the same time, it does not materially change the theory on control briefly summarised here. Kotler (2003) lists four types of marketing controls (table 22.1, p.685): annual-plan, profitability, efficiency and strategic. These distinguish whether the company is selecting the right goals (strategic), whether they are being achieved (effectiveness or annual-plan), where the company is making or losing money (profitability) and the return on each marketing expenditure (efficiency). Thus control theory assumes that management establishes goals of whatever type. Having done that the metrics necessary to compare goals with performance are defined. Agency Theory A rational-actor perspective is also taken by agency theory (Jensen and Meckling 1976). In this case, the focus is on the principal-agent contractual relationship where the principal has delegated work to the agent. Agency theory takes an economic perspective of how information will be transmitted vertically within the organization: information that is positive for the agent will be communicated to the principal to the extent that the gain obtained from its disclosure does not exceed the costs of obtaining and disseminating it. This is related to control theory in that "agency theory looks at the relative merits of behaviorbased contracts (...) vis-a-vis outcome-based contracts (...) as a means of efficiently ensuring the fidelity of the agents" (Nilakant and Rao 1994, p.653). The greater the difficulty of effectively measuring marketing performance, the greater should be the efficiency of behaviorbased forms of control compared to outcomebased forms of control (Fisenhardt 1985). This implies that when it is more difficult to evaluate marketing results, more reliance is likely to be placed on marketing expenditure controls. It also implies that specialist marketers are likely to propose metrics that will justify prospective expenditure (budgets) and past activities. Institutional Theory Institutional theory (Meyer and Rowan 1977) suggests that organizational action is mainly driven by both the cultural values and the history of the specific company, as well as by those of its industry sector. Accordingly, marketing information disclosure to top management can be predicted from "perceptions of legitimate behavior derived from cultural values, industry tradition, firm history, popular management folklore and the like" (Fisenhardt 1988, p.492). The set of marketing metrics selected by a company therefore is likely to reflect the intended subjective performance (the indicators the Board is used to seeing) rather than measures independent observers might consider appropriate. Since corporate cultures evolve with time, we can expect metrics to similarly adapt as distinct from being created from scratch, e.g. by a consultancy project. Success measures can be classified broadly as either accounting or non-accounting (Frazier and 2 Howell 1982; Buckley et al. 1988). Early work on firm-level measurement of marketing performance focused on accounting measures: profit, sales and cash flow (Sevin 1965, Feder 1965, Day and Fahey 1988). Many authors however highlighted the problem with using only accounting indicators in determining marketing performance (e.g., Bhargava, Dubelaar and Ramaswani 1994; Chakravarthy 1986; Doyle 2000; Eccles 1991). For example, Chakravarthy (1986) argues that: "accountingmeasure-of-performance record only the history of a firm. Monitoring a firm's strategy requires measures that can also capture its potential for performance in the future" (p.444). The US Institute of Management Accountants reported the growing use of non-financial measures (IMA 1993, 1995, 1996). Clark (1999) showed how traditional accounting measures (profit, sales, cash flow) expanded to include of non-accounting (market share, quality, customer satisfaction, loyalty, brand equity) measures, as well as wider considerations covering marketing audit, implementation and orientation. Clark posited that the set of selected metrics evolved incrementally, as suggested by institutional theory. In recent years the number and variety of measures available to firms has risen significantly (Meyer 1998). A literature search in five leading marketing journals yielded 19 different measures of marketing "success", the most frequent being sales, market share, profit contribution and purchase intention (Ambler and Kokkinaki 1997). Market Orientation The literature on market orientation and corporate culture takes a similar view in that the concept of marketing adopted within an organization influences the measurement system implemented for determining performance (Moorman 1995; Jaworski 1988; Webster 1992). The extent to which top management is interested in assessing marketing, or market performance, depends on the extent to which they are market-oriented (Day 1994; Jaworski and Kohli 1993; Kohli and Jaworski 1990; Narver and Slater 1990) because market-driven firms need to gather and disseminate market intelligence within the organization (Kohli and Jaworski 1990; Morgan, Katsikeas and AppiahAdu 1998; Slater and Narver 1995). As a consequence, one of the main features of a market-oriented organizational culture is the presence of organization-wide norms for market orientation (Homburg and Pflesser 2000). These norms will shape in turn the dynamics of information disclosure to the top management as well as the content of such information. Institutional theory and the concept of market orientation are related because, as argued by Dobni and Luffman (2000), "organizations with similar market orientations have a tendency or aptitude to engage in similar strategies when in the same industry, and the types of strategy chosen are related to the operational behaviors manifesting a market orientation" (p.9O9). Brand Equity Brand equity (Aaker 1991; 1996) is a widely used term for the intangible marketing asset. Srivastava and Shocker (1991) define brand equity as "a set of associations and behaviors on the part of a brand's customers, channel members and parent corporation that permits the brand to earn greater volume or greater margins than it could without the brand name and that gives a strong, sustainable and differential advantage" (p.5). Brand equity may be measured financially (cf. Egan and Guilding 1994; Simon and Sullivan 1993) and/or non-financially (cf., e.g., Agarwal and Rao 1996; Keller 1993, 1998). We treat financial measures synonymously with accounting measures, i.e. they are expressed in currency or as ratios of currency values. On the other hand, we distinguish between brand equity (the intangible asset) and brand valuation (the financial worth of that asset). As brands are autonomous units for marketing measurement purposes, multi-brand companies, if they are assessing their brand equities at all, would need to assess each brand separately. For the purposes of this paper, we envisage the increasing recognition of brand equity as creating the need for measures of those assets. Framework for Categorising Metrics Control theory looks to encouraging behaviour which leads to the achievement of goals and these include, but are not limited to, the financial bottom line. The simplest framework would simply include a category for the marketing actions and expenditures (inputs) and the profits and cash flow (outputs). In practice, those links 3 are not always clear and marketing plans will have two stages in between: the "intermediate" measures and consumer behaviours, such as purchases and loyalty. Intermediate measures are of customer memories be they attitudes, intentions, awareness or other cognitive or affective or experiential brand-linked characteristics. Thus control theory works backwards: if the links to financial results are unclear, consider the links with behaviours. If those are also unclear, consider the links with intermediate measures and then those with behaviours and then those with financial results. Market-oriented companies will consider the links in the reverse order: consumers first and then financial results but the categories are the same except that competition will be more carefully considered. Simmonds (1986b) pointed out that traditional financial accounting fails to give attention to competitive factors and proposes that the competitors be tracked on comparable measures such as sales and profits. Figure 1 shows how these four categories of metrics link together. Brand equity consists of the elements from inputs onwards which have not yet materialized as sales revenue. Ambler (2000) describes brand equity as the reservoir of cash flow that has been earned by good marketing but has yet to materialise as sales or profits. Although brand equity, as defined above, arguably lies in the heads of customers and other stakeholders, the difficulty of measuring neural effects leads academics and practitioners to use proxies, such as inputs and behaviour, to infer what lies between. Figure 1 shows only one generic "brain" but in practice brand equity is measured for each segment separately considered by management and typically distinguishes immediate trade customers from end users. Although competition, for conceptual purposes, is shown as an input, competitive metrics arise in all categories and are usually expressed as relative measures, e.g. market share, share of category requirements (loyalty). Thus Figure 1 provides the framework for the categories of metrics considered in the first study: Own inputs (marketing activities). Intermediate measures of memory (e.g. awareness and usage satisfaction, and attitudes). Behaviours. Competitive measures. Financial outcomes. Inputs Intermediate Behaviour (eg awareness and commitment) Competition Purchases Loyaity Word of mouth Own Financiai outcomes Figure 1. Metrics Categories Finally we need to consider performance. The paper assumes that performance, in some sense, influences the selection of metrics ("what you measure is what you get"). Swartz, Hardie, Gray son and Ambler (1996) concluded that marketing activities broadly achieved planned performance but the return on marketing investment could not be compared across companies because their performance intentions differed. In the two studies below, performance was first based on how they were seen by competitors and selfrating of success relative others in their sector. The second study also used self-ratings but across four variables: relative to plan, to previous year's sales, to competitors and overall. These variables emerged from the empirical work. Empirical Analysis of Current Practice The exploration of how practitioners viewed metrics occupied two studies. The first grouped metrics into categories and asked the basis for comparison, e.g. plan (control theory) or competition (market orientation). Then we explored how widely brand equity was used as a concept and how it was measured. Finally we looked at whether orientation would be associated with the frequency of use or importance of competitor or customer metrics. The second study switched focus from the categories, inevitably a somewhat arbitrary grouping, to the metrics themselves. We sought the metrics perceived as most important and how their selection varied by business sector. 4 Study 1 Method Forty-four in-depth interviews were conducted with senior marketing and finance managers from 24 UK firms in order not to restrict the perspective to the marketers (Homburg et al. 1999). The issues addressed included: the type of measures collected, the level of review of these measures (e.g. marketing department. Board), the assessment of the marketing asset, planning and benchmarking, practitioners' satisfaction with their measurement processes and their views on measurement aspects that call for improvement, and firm orientation. Information on firm characteristics, such as size and sector, was also obtained. Data were collected face-toface and each interview was recorded for subsequent analysis. After the first 10 interviews minor changes were made to the interview guide for clarification and to reduce interview length. Some respondents took the opportunity to brief us widely about their companies. The pilot broadly confirmed the validity of the five categories above although respondents preferred to put all financial/accounting and competitive measures into separate categories, i.e. not just competitive inputs and financial outputs. In practice, since most of the inputs were financial metrics, this removed the "own inputs" category. Non-retail respondents also distinguished immediate (trade) customers from end-users or consumers. They used a separate category to monitor innovation. Thus we were left with six categories: consumer intermediate and behaviour, trade customer, competitive, innovativeness and accounting (inputs and outputs). These changes became obvious after the first 10 interviews and, after making the changes, the categories were not challenged in the subsequent 34. The categories may not seem strictly logical to an outsider but we were seeking to understand how practitioners grouped their metrics. The pilot stage was also used to refine the survey instrument that was sent to 1014 marketing and 1180 finance senior executives in the UK, recruited through two professional bodies (The Marketing Society and the Institute of Chartered Accountants in England and Wales). A total of 531 questionnaires were returned (367 from marketers and 164 from finance officers, response rates 36 percent and 14 percent, respectively). Table 1 presents a description of the sample. To encourage response, given the sensitive nature of this information, the returns were anonymous so we were unable to check back for missing values. The majority of the "other" category is probably explained by large companies operating in more than one sector, e.g. high street banks are both retail and B2B. Table 1. Respondents by Business Size and Sector (Study 1) (# employees) Small (<110) Medium (<500) Large (>50G) Missing values Total Retail 8 8 51 67 Consumer goods 7 13 111 131 Consumer services 14 6 38 58 Respondents were asked to indicate the importance attached to the different measurement categories by top management on a 7-point scale anchored by very important / very unimportant. They were also asked to report how regularly data are collected for each measure category and the benchmark against which each measure category is compared (previous year, marketing/business plan, total category data, specific competitors, other units in the group). B2B goods 6 7 30 43 B2B services 44 21 38 103 Other 32 12 11 121 Total 111 67 345 8 531 Respondents were then asked whether they have a term for the main intangible asset built by the firm's marketing efforts and whether this asset is formally and regularly tracked, through financial valuation or other measures. Customer and competitor orientation were measured with eight 7point scales drawn from Narver and Slater (1990). Separate single indices of customer and competitor orientation were computed as the mean of responses to these items (Cronbach's alpha .81 and .69, respectively). 5 Results As shown by Table 2, accounting measures were reported as being seen by top management as significantly more important than all other categories. The t-tests comparing the importance of accounting measures with other categories were all significant (p< 0.001). However, the differences between customer and competitive measures were small. Apart from a slightly greater concern with innovation by marketers, there were no significant differences between the importance attributed to each metric category by marketers and finance respondents (note that both marketers and finance respondents were asked to report the importance attached by top management). Moreover, there were no significant differences in measurement category importance between different business sectors. Table 2. Importance of Measure Categories for Assessing Performance (Overall) Financial Direct customer Competitive Consumer intermediate Consumer behaviour Innovativeness Mean 6.51 5.53 5.42 5.42 5.38 5.04 t df Sig. -14.90 -16.78 -15.60 -15.60 20.13 499 523 515 522 524 .000 .000 .000 .000 .000 Note. t- tests refer to the comparisons between financial measures and each of the other categories. Irrespective of the importance attached to different indicators, accounting measures were more frequently collected than any other category (74.9 percent of the sample reported that accounting measures were collected at least on a monthly basis). In 33.5 percent of the cases, consumer intermediate measures were collected only rarely/ad hoc. Innovativeness, which some see as the lifeblood of marketing (e.g. Simmonds 1986a), is least regularly measured (55 percent of firms measure innovativeness rarely or never). Larger firms, as might be expected, measure most categories more frequently than smaller firms (p<0.01 for categories except innovativeness). Similarly, business sector was found to have a significant effect on the frequency of data collection, with the exception of innovativeness. Closer inspection of mean frequency per business sector, however, did not reveal any systematic differences across sectors. On average, irrespective of measure category, consumer goods and retail firms tend to collect data more frequently than other sectors (F (5, 512) = 11.81, p < .001). Table 3. Frequency of Benchmarks Used (valid percent) where measure category used) Accounting measures Competitive Consumer behaviour Consumer intermediate Direct customer Innovativeness Previous year 80.4 51.4 47,1 36.7 40.3 21.3 Marketing Business Plan 85.1 51.0 42.0 30.3 37.7 33.7 Metrics Comparisons Plans provided the most frequent benchmarks of accounting and innovativeness measures, where such measures are used (Table 3). Competitive measures, however, were compared with market research rather than forecast in plans. Consumer and direct customer measures were most typically compared with previous year results. Market share apart, it appears that internal (plan) Total category data 17.5 35.8 27.1 22.0 17.3 10.9 Specific competitor(s) 23.0 55.7 31.6 27.7 22.8 20.7 Other units in the Group 22.0 6.6 6.4 5.1 7.3 6.6 and external benchmarks were routinely used only by the minority of respondent firms. The modal frequency for each row is highlighted in the table. Brand Equity Moving now to the marketing asset, 62.2 percent of the respondents reported the use of some term to describe the concept. The most common terms were brand equity (32.5 percent of those who 6 reported a term), reputation (19.6 percent), brand strength (8.8 percent), brand value (8.2 percent) and brand health (6.9 percent). Twenty percent of those who used a term reported one or more of 65 different terms, such as brand integrity, customer loyalty, global image, quality, contact base and trademark value. 24.9 percent of the total sample regularly (yearly or more) valued the marketing asset financially, and 41 percent regularly quantified it in other ways, e.g. through customer/consumer based measures (see Table 4). Less than 15 percent of the total sample did both. Table 4. Regularity of Tracking the Marketing Asset (valid percent of the total sample) Financial valuation Other measures Never 51.4 36.8 Rarely/ Ad hoc 23.6 22.2 Regularly Yearly/ Quarterly 16.9 28.7 Our theoretical discussion suggested that the marketing performance assessment system can be tested against three criteria: benchmarking against internal expectations (plan) and external (competitor) performance, adjusted by changes in brand equity. Of the 196 respondents who reported quantifying their marketing assets (either financially and/or in other ways), 24 Monthly or more 8.0 12.3 percent also quantify consumer, competitive or direct customer measures in their business plan (internal) and use market or competitive benchmarks. Thus, on this survey, less than one quarter of firms could meet all three criteria. Of course, these data merely indicate that they have the measures available for such comparisons, not that they necessarily make such comparisons. Table 5. Regression of Regularity of Tracking on Customer and Competitor Orientation Accounting Competitive Consumer behaviour Consumer intermediate Direct customer Innovativeness *** F 210.40*** 39.17*** 11.14*** 12.07*** 8.53*** 9.01*** R .27 .48 .28 .31 .26 .26 R2 .07 .23 .08 .09 .07 .07 Customer Orientation beta t .03 .83 .03 .81 .18 3.78*** .23 4.88*** .16 3.29*** .24 5.05*** Competitor Orientation beta t .05 1.13 .22 5.31*** .07 1.50 .00 .16 .00 .04 .03 .71 p < .001 Market Orientation In order to examine whether customer and competitor orientation have an effect on performance assessment practices, regularity of tracking and importance attached to different measures were regressed simultaneously on these constructs, after partialling out the effect of firm size and sector. As can be seen in Table 5, customer orientation was strongly associated only with the regularity of collection of consumer, direct customer and innovativeness measures. As might be expected, more customeroriented firms tended to collect data on such measures more frequently than firms less so oriented. Customer orientation does not seem to be related to the regularity of tracking of accounting and competitive measures, whereas the regularity of collection of competitor measures was found to be related to the level of competitor orientation. Table 6. Regression of Measure Category Importance on Customer and Competitor Orientation Accounting Competitive Consumer behaviour Consumer intermediate Direct customer Innovativeness F 4.86*** 27.88*** 19.05*** 21.45*** 9.40*** 9.80*** R .19 .42 .36 .38 .26 .26 R2 .00 .18 .13 .14 .07 .07 Customer Orientation beta t 2.19* .10 1.30 .05 .25 5.64*** .30 6.68*** .22 4.67*** .20 4 29*** Competitor Orientation beta t .03 .71 .26 6.03*** .16 3.57*** .09 2.18* .07 1.62 .10 2.32* 7 The relation between orientation and measure importance is less clear (Table 6). Customer orientation was positively related with the importance attached to most measures except competitive measures, although the correlation with accounting measures was relatively weak. Competitor orientation strongly correlated with the importance of competitor and consumer behaviour measures and less robustly with intermediate and innovative measures. Both customer and competitor orientation were significantly associated with assessment practice. No moderating effects by sector were observed, but firm size appeared to moderate the importance attached to measures. The results indicate that customer orientation is a stronger predictor of the importance of competitive measures for large firms, compared to small firms {beta of the interaction term = .50, t = 2.60, p < .01). However, competitor orientation was found to be a stronger predictor of importance for small firms, compared to large firms {beta of the interaction term = -.46, t = 2.57, p < .01). We have no simple explanation for these results. It is possible that larger UK firms need to be more customer-oriented in order to be effective, perhaps because they already are competitor oriented. In contrast, in the case of smaller UK firms, competitor orientation seems to be more important, perhaps because they already are customer oriented. Study 2 Up to this point, we have explored practitioner usage based on the metric categories first identified in the framework and revised in the pilot stage of Study 1. The next step was therefore to perform a study specifically aimed at designing a list of most frequently used metrics. Providing useful indications to managers and stimulating further academic research, such endeavour was thought to be important for a complete understanding of current practice in the area of marketing performance assessment. In particular, while we expected to find wide variation by sector and firm, we explored the extent to which certain metrics stood out as more valuable, or at least more widely used, than others. Method An initial survey instrument with 54 metrics was developed, using the relevant literature, and piloted to establish any additional measures and to eliminate those measures that were not used or were redundant. No additions were made but 16 were eliminated. The resulting 38 measures were classified into the six categories described above. A telephone survey was then conducted with 200 UK marketing or finance senior executives drawn from the lists supplied by The Marketing Society and The Institute of Chartered Accountants in England and Wales. In both cases, only senior practitioners were selected. The acceptance level for the telephone interviews, i.e. response level, was 50.1 percent. This excludes wrong numbers and other technical blockages. Since the survey instrument was not materially altered by the pilot stage (the eliminated metrics had been left blank) we added the responses of the 31 executives who participated in the Study 2 pilot. Respondents were asked to indicate the importance of each measure for assessing the overall marketing performance of the business on a 5-point scale. They were also asked to indicate the highest level of routine review of this metric within the firm, on a scale ranging from the [group's] top board level (5) through junior marketing (1) to not used at all (0). Respondents were also asked to add any relevant measures not listed. Contextual data were also collected in order to determine the impact of environmental factors such as firm size, business sector, organization structure, and age of business. Table 7 shows a broad spread across firm size and sector. Table 7. Respondents by Structure and Sector (Study 2) # Employees One unit without marketing dept One unit with marketing dept Subsidiaries with one board More complex Missing values Total Retail 4 5 7 6 22 Consumer goods 4 7 8 13 32 Consumer Services 17 6 5 10 23 B2B goods 4 11 11 15 41 B2B Services 23 15 13 15 66 Other 9 6 5 25 2 47 Total 46 50 49 84 2 231 8 The great majority of the firms had been in business for more than five years and therefore have reporting systems that have evolved beyond the start-up phase and to that extent become established. As was the case in Study 1, the principle reason for the companies not attributed to any sector is due to large firms that trade in more than one sector. Results Table 8 ranks the top 15 (> 62 percent usage) metrics by frequency of use compared with the frequency that it was rated as "very important" and the frequency that it reached the top level of management. Table 8. Ranking of Marketing Metrics Metric 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. % claiming to use measure % firms rating as very important % claimed to reach top level Pearson Correlation between Level and Importance 92 91 81 78 78 73 70 69 80 71 66 28 37 18 36 45 71 65 58 29 34 19 33 31 .729** .758** .827** .732** .727** .859** .735** .802** 68 66 66 65 64 64 63. 48 18 24 39 37 47 39 37 11 23 46 32 34 30 .815** .900** .812** .849** .783** .830** .814** Profit/Profitability Sales, Value and/or Volume Gross Margin Awareness Market Share (Volume or Value) Number of New Products Relative Price (SOMValue/Volume) Number of Consumer Complaints (Level of dissatisfaction) Consumer Satisfaction Distribution/ Availability Total Number of Customers Marketing Spend Perceived Quality/esteem Loyalty/Retention Relative perceived quality n = 231, ** p < .01 The last column of Table 8 highlights the have a significant effect on the usage of specific expected correlation between the measure being items, particularly consumer intermediate, seen as very important and its review by the top competitive and accounting measures. Table 9 management level. Business sector was found to shows the 15 most significant differences of 38. Table 9. ANOVA for Significant Metric Variations by Business Sector Level of Importance df F Other attitudes e.g. liking Image/ personality/ identity Penetration Salience Commitment/purchase intent Distribution/ availability Awareness Relevance to consumer Marketing spend Market share Share of voice Brand/product knowledge Conversions Margin of new products Purchasing on Promotion 228 226 225 225 228 218 227 226 228 226 225 228 225 225 225 8.88*** 7.91*** 7.67*** 3.81 6.88*** 6.83*** 5.84*** 5.34*** 5.12*** 5.02*** 4.93*** 4.54** 2.55** 3.95** 11.87*** Level of Review df F 229 229 229 229 229 228 229 228 229 229 229 229 228 228 229 4.66*** 4.36** 3.07** 3.15** 3.45** 6.14*** 3.05** 2.61** 2.55** 2.63** 4.82** 2.62** 2.96** 2.60** 5.88*** n = 231, ** p < .05, *** p < .001 9 Distinctions between sectors were greater for level of importance measures than level of review. As would be expected, consumer oriented items were found to be more important for consumer sectors. Perhaps the most surprising result is the variation in importance ascribed to market share across business sectors. As might be expected, consumer goods firms consistently rated metrics as more important and reviewed at more senior levels than business-tobusiness services. Developing Primary Metrics Finally we analysed which metrics are candidates for being seen as the most valuable or primary, irrespective of sector or size. Content validity (Churchill 1979) using a 50 percent cutoff (Cronbach and Meehl 1955) left 30 metrics that were then subjected to scale purification procedures. Construct validity was assessed with the guidelines outlined by Churchill (1979) and Gerbing and Anderson (1987). We examined item-to-total correlations and the factor structure (through principal components) for each scale. The decision criterion for item deletion was an improvement in corresponding alpha values to the point at which all items retained had corrected item-total correlations greater than 0.5. Fight items were eliminated, varying slightly as to whether the level of review or level of importance was considered. Both level of review and importance are shown for comparison in Table 10. 19 items match and could be considered as the primary general metrics: Awareness, Perceived quality, Consumer satisfaction, Relevance to consumer, Perceived differentiation, Brand/product knowledge, Number of new customers, Loyalty/retention, Conversions, [Trade] Customer satisfaction, Number of complaints, Relative consumer satisfaction, Perceived quality, Number of new products, Revenue of new products, Margin of new products, Sales, Gross margins, Profitability. Table 10. Primary Metrics If level of review is measured Construct Alpha Items If level of importance is measured Alpha Items Awareness Perceived quality Consumer satisfaction Relevance to consumer Image/personality Perceived differentiation Brand/product knowledge Total number of consumers Number of new consumers Loyalty/retention Conversions Number of consumer complaints Customer satisfaction Number of complaints .84 Awareness Perceived quality Consumer satisfaction Relevance to consumer Perceived differentiation Brand/product knowledge .83 Number of new consumers Loyalty Leads generated Conversions .79 .79 Relative consumer satisfaction Perceived quality .80 Innovation .84 Number of new products Revenue of new products Margin of new products .81 Distribution/ availability Customer satisfaction Number of customer complaints Relative consumer satisfaction Perceived quality Share of voice Number of new products Revenue of new products Margin of new products Accounting .81 Sales Gross margins Profitability .77 Consumer Attitudes .85 Consumer Behaviour .78 Trade Customer .80 Relative to Competitor TOTAL : 22 items Sales Gross margins Profitability TOTAL: 22 items n = 231 10 Discussion The primacy of accounting metrics, both in terms of importance attributed by the respondents and of regularity of assessment, is consistent with the literature (e.g. Clark 1999). We were surprised by the relatively low levels reported for basics such as sales and profitability. Every board must see these figures as part of their financial accounts but the respondents here were reporting on what they perceive to be marketing. The role of "marketing" varies widely across UK companies (Ambler 2000). In our analysis, brand equity can provide the bridge between short- and the long-term effects as regarded important by Dekimpe and Hanssens (1995). Although a substantial proportion of our respondents were measuring brand equity financially or non-financially, it seems likely that a formal (control theory) process rarely meets all three marketing performance assessment criteria above, i.e. internal and external benchmarking adjusted by any change in brand equity. Less than 25 percent of our respondents had the data to do so. Orientation was consistent with metrics usage as shown for regularity and importance (Tables 5 and 6 respectively). Thus we found some support for control, institutional (sector differences) and orientation theories in the selection of metrics but little support for agency theory. This last would require analysis of the interactions between the Board and junior levels of the firm, notably in respect of budget approvals. From the literature and the two studies, we can advance some propositions about how metrics are adopted with some implications for the way they should be. P1: Strongly control oriented top management will review those metrics projected in the marketing plan. Apart from accounting measures, we found that between a third and one half of measures were compared to plan but we expect that to be moderated by control orientation. P2: Following institutional theory, metrics will evolve a-rationally in conformity with sector norms. We do not expect that firms can provide rational explanations for the metrics they adopt. P3: Following orientation theory, metrics selection will reflect the primary interests of top management, e.g. customer, competitor or internal accounting measures. P4: Agency theory will provide explanation of metrics selection only in the context of budget negotiation and subsequent interlevel evaluation of performance. Limitations and Future Research Our findings are limited in several respects. First the research, as with other survey-based methods, does not capture causality nor the dynamics of the development of measurement, orientation and performance. In future research linking metrics selection with performance, it may be important to distinguish the types of performance sought by management rather than build a signal performance construct, i.e. do companies, in fact, get what they measure? We examined the potential effects of size and sector, but we did not consider external environmental effects. Market turbulence, for example, may moderate metrics selection and their value (Greenley 1995, Harris 2001). Although we used the literature from elsewhere, notably the US, the empirical work was conducted in the UK. It is likely that there are variations internationally in metrics selection, not least because the metrics available from suppliers will differ in other countries. It would be more interesting to explore what, if any, theoretical differences exist. Finally, the whole area of linking non-financial market measures to financial outcomes, such as shareholder value, has barely been explored. Conclusions This paper has contributed two exploratory UK studies to the increasing interest in the assessment of marketing performance. Accounting remains the dominant metrics category relative to consumer behaviour and intermediate, trade customer, competitor, and innovativeness. Brand equity is widely measured but rarely integrated into a formal assessment system. We found consistency between orientation and metrics. Within these categories we identified 19 metrics that could be regarded as primary and could therefore serve as a shortlist for initial selection. However, the sector importantly moderates that selection, not least 11 because competitive benchmarking requires similar metrics to be available. Control, orientation and institutional theories appeared to influence metrics selection and the absence of agency theory is probably due to the research method of this paper. We concluded with some propositions formally to test the basis of metrics selection. Dekimpe, Marruk G. and Hanssens, Dominique M. (1995), "The Persistence of Marketing Effects on Sales", Marketing Science, Vol.14, No. 1, pp.1-21 Dekimpe, Marnik G. and Hanssens, Dominique M. (1999), "Sustained Spending and Persistent Response: A New Look at Long-Term Marketing Profitability", Journal of Marketing Research, Vol.36, No.4, pp.397-412 Dobni, C. Brooke and . Luffman, George (2000), "Implementing Marketing Strategy Through Market Orientation", Journal of Marketing Management, Vol.16, pp.895-916 Doyle, Peter (2000), Value-Based Marketing, Chichester, UK, Wiley Acknowledgements We are grateful to Debra Riley who conducted much of the earlier research and to The Marketing Society, The Marketing Council, Institute for Practitioners in Advertising, Sales Promotions Consultants Association, London Business School and Marketing Science Institute for sponsoring this research. References Aaker, David A. (1991), Managing Brand Equity, New York, Free Press Aaker, David A. (1996), Building Strong Brands, New York, Free Press Agarwal, Manoj K. and Rao, Vithala R. (1996), "An Empirical Comparison of Consumer-Based Measures of Brand Equity", Marketing Letters, Vol.7, No.3, pp.237-247 Ambler, Tim (2000), Marketing and the Bottom Line: The New Metrics of Corporate Wealth, London, Prentice Hall/Financial Times Ambler, Tim and Kokkinaki, Flora (1997), "Measures of Marketing Success", Journal of Marketing Management, Vol.13, pp.665-678 Bhargava, Mukesh, Dubelaar, Chris and Ramaswami, Sridhar (1994), "Reconciling Diverse Measures of Performance: A Conceptual Framework and Test of a Methodology", Journal of Business Research, Vol.31, pp.235-246 Bonoma, Thomas V. and Clark, Bruce H. (1988), Marketing Performance Assessment, Boston, Harvard Business School Press Buckley, Ronald, Hall, Sid, Benson, Philip G. and Buckley, M. (1988), "The Impact of Rating Scale Format on Rater Accuracy: An Evaluation", Journal of Management, Vol.14, No.3, pp.415-423 Chakravarthy, Balaji S. (1986), "Measuring Strategic Performance", Strategic Management Journal, Vol.7, pp.437-458 Churchill, Gilbert A. Jr. (1979), "A Paradigm for Developing Better Measures of Marketing Constructs", Journal of Marketing Research, Vol.16, No. 1, pp.64-73. Eccles, Robert G. (1991), "The Performance Measurement Manifesto", Harvard Business Review, Vol.69, No. 1, pp.131-137 Egan, Colin and Guilding, Christopher (1994), "Dimensions of Brand Performance: Challenges for Marketing Management and Managerial Accountancy", Journal of Marketing Management, Vol.10, pp.449-472 Eisenhardt, Kathleen M. (1985), "Control: Organizational and Economic Approaches", Management Science, Vol.31, No.2, pp.134-149 Eisenhardt, Kathleen M. (1988), "Agency- and InstitutionalTheory Explanations: The Case of Retail Sales Compensation", Academy of Management Journal, Vol.31, No.3, pp.488-511 Feder, Robert A. (1965), "How to Measure Marketing Performance", Harvard Business Review, Vol.43, No.3, pp.132-142 Fraser, Cynthia and Hite, Robert F. (1988), "An Adaptive Utility Approach for Improved Use of Marketing Models", Journal of Marketing, Vol.52, No.4, pp.96-103 Frazier, Gary L. and Howell, Roy D. (1982), "Intra-industry Marketing Strategy Effects on the Analysis of Firm Performance", Journal of Business Research, Vol.10, No.4, pp.431-443 Gerbing, David W. and Anderson, James (1988), "An Updated Paradigm for Scale Development Incorporating Unidimensionality and its Assessment", Journal of Marketing Research, Vol.25, No.2, pp.186-92. Greenley, Gordon F. (1995), "Market Orientation and Company Performance: Empirical Evidence From UK Companies", British Journal of Management, Vol.6, pp.1-13 Harris, Lloyd C. (2001), "Market Orientation and Performance: Objective and Subjective Empirical Evidence From UK Companies", Journal of Management Studies, VoL38 No. 1, pp.17-43 Homburg, Christian, Workman, John P. Jr. and Krohmer, Harley (1999), "Marketing's Influence Within the Firm", Journal of Marketing, Vol.63, No.2, pp.1-17 Clark, Bruce H. (1999), "Marketing Performance Measures: History and Interrelationships", Journal of Marketing Management, Vol.15, pp.711-732 Homburg, Christian and Pflesser, Christian (2000), "A MultipleLayer model of Market-Oriented Organizational Culture: Measurement Issues and Performance Outcomes", Journal of Marketing, Vol.37, No.4, pp.449-462 Cronbach, L.J. and Meehl, P.F. (1955), "Construct Validity in Psychological Tests", Psychological Bulletin, Vol.52, pp.281-302 IMA (1992), Cost Management Update, October 1992 Vol.32, US Institute of Management Accountants, Cost Management Group Day, George S. (1994), "The Capabilities of Market Driven Organizations", Journal of Marketing, 58, No.4, pp.37-52 IMA (1994), Cost Management Update, March 1994 Vol.49, US Institute of Management Accountants, Cost Management Group Day, George S. and Fahey, Liam (1988), "Valuing Market Strategies", Journal of Marketing, Vol.52 No.3, pp.45-57 Day, George S. and Wensley, Robin (1988), "Assessing Advantage: A Framework for Diagnosing Competitive Superiority", Journal of Marketing, Vol.52, No.2, pp.1-20 IMA (1996), Cost Management Update, June 1996 Vol.69, US Institute of Management Accountants, Cost Management Group 12 Jaworski, Bernard J. (1988), "Toward a Theory of Marketing Control: Environmental Context, Control Types, and Consequences", Journal of Marketing, Vol.52, No.3, pp.2339 Jaworski, Bernard J. and Kohli, Ajay K. (1993), "Market Orientation: Antecedents and Consequences", Journal of Marketing, Vol.57, No.3, pp.5370 Jensen, Michael C. and Meckling, William H. (1976), "Theory of the Firm: Managerial Behavior, Agency Costs and Ownership Structure", Journal of Financial Economics, Vol.3, pp.305-360 Keller, Kevin L. (1993), "Conceptualizing, Measuring, and Managing Customer-Based Brand Equity", Journal of Marketing, Vol.57, No. 1, pp.1-22 Keller, Kevin L. (1998), Strategic Brand Management: Building, Measuring and Managing Brand Equity, Upper Saddle River, NJ, Prentice-Hall Kohli, Ajay K. and Jaworski, Bernard J. (1990), "Market Orientation: The Construct, Research Proposition, and Managerial Implications", Journal of Marketing, Vol.54, No.2, pp.1-18 Kotler, Philip (2003), Marketing Management, (11th edition). Upper Saddle River, NJ: Prentice Hall Marketing Week (2001), "Assessing Marketers' Worth", February 2001 No. 1, pp. 42-43 Merchant, Kenneth A. (1998), Modern Management Control Systems, Upper Saddle River, NJ: Prentice Hall Simmonds, Kenneth (1986a), "Marketing as Innovation. The Eighth Paradigm", Journal of Management Studies, Vol.23, No.3, pp.479-500 Simmonds, Kenneth (1986b), "The Accounting Assessment of Competitive Position", European Journal of Marketing, Vol.20, No. 1, pp.16-31 Simon, Carol J. and Sullivan, Mary W. (1993), "The Measurement and Determinants of Brand Equity: A Financial Approach", Marketing Science, Vol.12, No. 1, pp.28-51 Slater, Stanley F. and Narver, J.C. (1994), "Does Competitive Environment Moderate the Market OrientationPerformance Relationship?", Journal of Marketing, Vol.58, No. 1, pp.46-55 Slater, Stanley F. and Narver, John C. (1995), "Market Orientation and the Learning Organization", Journal of Marketing, Vol.59, No.3, pp.63-74 Srivastava, Rajendra K. and Shocker, Allan D. (1991), "Brand Equity: A Perspective on its Meaning and Measurement", Working Paper No. 91-124, Cambridge, MA, Marketing Science Institute Swartz, G., Hardie Bruce, Grayson, Kent and Ambler, Tim (1996), "Value for money? The relationships between marketing expenditure and business performance in the UK financial services industry", Cookham, Berkshire, UK, Chartered Institute of Marketing, April Webster, Frederick E. Jr. (1992), "The Changing Role of Marketing in the Corporation", Journal of Marketing, Vol.56, No.4, 1-17 Meyer, John W. and Rowan, Brian (1977), "Institutionalized Organizations: Formal Structure as Myth and Ceremony", American Journal of Sociology, Vol.83, pp.340-363 Meyer, Marshall W. (1998), "Finding Performance: The new discipline in management". Performance MeasurementTheory and Practice, Vol. 1, Cambridge, UK, Centre for Business Performance, xiv-xxi Moorman, Christine (1995), "Organizational Market Information Processes: Cultural Antecedents and New Product Outcomes", Journal of Marketing Research, Vol.32, No.3, pp.318-335 Morgan, Robert F., Katsikeas, Constantine S. and Appiah-Adu, Kwaku (1998), "Market Orientation and Organizational Learning Capabilities", Journal of Marketing Management, Vol.14, pp.353-381 MSI (2002), 2002-2004 Research Priorities: A Guide to MSI Research Programs and Procedures, Cambridge, MA, Marketing Science Institute Murphy, Gregory B., Trailer, Jeff W. and Hill, Robert C. (1996), "Measuring Performance in Entrepreneurship Research", Journal of Business Research, Vol.36, pp.15-23 Narver, John C. and Slater, Stanley F. (1990), "The Effect of a Market Orientation on Business Profitability", Journal of Marketing, Vol.54, No.4, pp.20-35 Nikalant, V. and Rao, Hayagreeva (1994), "Agency Theory and Uncertainty in Organizations: An Evaluation", Organization Studies, Vol.15, No.5, pp.649-672 Sevin, Charles H. (1965), Marketing Productivity Analysis, New York, McGraw-Hill Schultz, Don E. (2000), "Understanding and Measuring Brand Equity", Marketing Management, Spring 2000, pp.8-9 Shaw, Robert (1998), Improving Marketing Effectiveness, London, The Economist Profile Books Shaw, Robert and Mazur, Laura. (1997), Marketing Accountability: Improving Business Performance, London, Financial Times, Retail and Consumer Publishing About the Authors Tim Ambler is Senior Fellow at London Business School. His primary research interests concern international marketing and the measurement of marketing and advertising performance. More recently he has broadened these to include the quantification of government waste (bureaucracy and unnecessary regulation and red tape). His books include Marketing and the Bottom Line: The New Metrics of Corporate Wealth (2000), Doing Business in China (2000), The SILK Road to International Marketing (2000) and Marketing from Advertising to Zen (1996). He has published in the Journal of Marketing, Journal of Marketing Research, International Journal of Research in Marketing, Journal of Advertising Research and International Journal of Advertising. He was previously Joint Managing Director of International Distillers and Vintners which is now part of Diageo plc. Flora Kokkinaki is a lecturer in Social Psychology at the University of Patras. After the completion of her PhD in Psychology at University College London, she joined London Business School as Research Fellow working on the assessment of marketing effectiveness. After appointments as lecturer at University College London and the London School of Economics and Political Science, she joined the University of Patras. Her research revolves around the areas of Social Cognition, Consumer Behaviour and Economic Psychology. Her publications include papers in the British Journal of Social Psychology, the Journal of Economic Psychology and the Journal of Marketing Management. Stefano Puntoni is a doctoral student in marketing at London Business School. He holds a Master in Statistics from the University of Padua (Italy). Stefano has published in the areas of marketing metrics, consumer behaviour, marketing communications and pricing and participated to various marketing conferences and symposia. He is currently working on a research project devoted to a systematic analysis of the antecedents of advertising polysemy. 13