Download Roberts_IM3e_IM_ch14 - Dr. Robert Davis (Ph.D) FCIM (UK)

Chapter 14
Measuring and Evaluating Web Marketing Programs
Learning Objectives:
By the time students complete this chapter they should be able to:
 Discuss the various types of Internet marketing metrics that are available.
 Explain the importance of usability testing.
 Identify the reasons why traffic and audience measurement are a central issue in Internet
marketing.
 Understand the process of collecting data from server request log files, tagged pages, and
user panels.
 Define key traffic, audience, and campaign metrics and the purpose of each.
 Discuss the role of objectives and key performance indicators in the identification and
use of metrics.
 Discuss the importance of segmentation to an understanding of visitor activity.
 Identify issues involved in obtaining a complete picture of the customer buying process,
both on and off the web.
Chapter Perspective
The importance of this chapter cannot be overstated. Web metrics are at the heart of the
interactivity, trackability and measurability of the Internet. Usability testing is relatively easy for
students to identify with, particularly after you point out that it has many of the elements of focus
group research in the physical world. Internet metrics are a new animal to most students. Many
undergraduate students still have a marketing research perspective, and it can be very difficult to
get them to discard that and genuinely understand Internet metrics. MBA students, on the other
hand, tend to grasp the concept of measurability quickly even if they lack mastery of the details.
In graduate classes one or more students may have access to online marketing reports and be able
to demonstrate them in class. Or if your school uses a commercial metrics program, perhaps one
of the web masters or a user like the head of Enrollment Management, might demonstrate uses.
The real time nature of these demonstrations tends to make an impression. However, in their
absence virtually all of the web metrics companies have great demos that are a great help in
illustrating these concepts.
I would encourage you to spend at least two hour or hour-plus class sessions on this material.
Usability can take up most of a session. I find that even after two or three hours on metrics a lot
of students are still quite fuzzy on even the basics. One thing might be to play one demo that
you really like and assign another one to be accesses and viewed at home like the Google
Product Tour, Interactive Exercise 14.2. Hopefully students will be impressed by the fact that a
free metrics program offers robust metrics. If students ask, a reasonable piece of advice is to use
a free platform until comfortable with it and then only change when the size/scope of the website
demands it.
My personal belief, after many years of experience, is that you simply need to take students
through these examples, through the numbers. Hopefully they will find it fascinating to realize
the depth of understanding that can be gained about how customers behave. Even if they find it
gruesome, they need to understand the role and general outlines of metrics.
Two things may help. One is to emphasize the availability and importance of metrics throughout
the course. Second is to emphasize the extreme shortage of metrics specialists and the excellent
career opportunities available to young people who have enough knowledge to discuss the
subject intelligently in an interview and can demonstrate a willingness to learn.
Marketing Effectiveness and the Importance of Marketing Metrics
The two case studies—Papa Gino’s Pizza and Sybase—are meant to focus students on the
business reasons for using metrics—basically to understand what is happening and how to
improve effectiveness. The uses in B2B and B2C are similar. The issue of the cases is what
metrics to focus on to get the greatest increase in effectiveness.
If you need to add nonprofits to the mix, this is a good post that actually shows nonprofit use
being much the same as for-profits: http://www.lunametrics.com/blog/2012/02/28/nonprofitgoogle-analytics. Convio has case studies, which include web metrics in addition to email. This
one from the Childrens Defense Fund focuses on the website: http://www.convio.com/ourclients/case-studies/childrens-defense-fund.html. Marketing Sherpa has case histories from
nonprofits that focus on a variety of issues: http://www.marketingsherpa.com/non-profitfundraising-category.html.
The following section basically says that good metrics are both important and difficult. Notice
especially the fact some of the most important things for marketers are also the most difficult;
various measures of 360° views of customers and of channels. Combining metrics from different
sources in order to understand the whole picture is difficult.
Ways to Evaluate Website Effectiveness
Figure 14.2 provides an overview of metrics. Usability metrics provide the customer’s
perspective. The business view has two components. Site performance, as discussed in Chapter
12 is the purview of the web master whose job it is to ensure that the site is working properly at
all times. Marketing should care deeply about site performance, but achieving it is the job of IT
personnel. Marketing’s job is to use the traffic, audience and campaign metrics produced on the
Internet itself as an evaluation of current programs and as a way to improve future programs. It
sounds simple, but it’s not. Consequently, students need to have this overview firmly in mind
before they become swamped with the details, which is all too easy to do.
Usability Testing
Visitors to a website need to be able to navigate it smoothly, to find the products or information
they need, and to be pleased—not frustrated—by their experience on the website. All those
things are true. The fundamental basis of web usability, however, is user task performance.
Visitors come to the website motivated to accomplish some goal, to perform some task. Usability
testing is designed to ensure that task performance is not only possible, but hopefully efficient
and entirely satisfactory, a good customer experience.
The stages are:



Testing site design concepts to see if they make sense. This is primarily site structure,
although design concepts can and often should be tested also. The shot of Nielsen’s card
sorting exercise (see first link in Interactive Exercise 14.1) is an example of how
marketers can develop or test concepts in the very early stages of site development. The
second link in the exercise is a modern version. It is good to use both in class.
Testing prototypes to see if they fit the manner in which users expect the site to be
organized and laid out in order for them to complete tasks in an orderly fashion. This is
the stage at which there can be exploration of the terms that will provide the best signals
to users. For example, “Corporate Information” and “About Us.” Do they mean the
same thing; would users expect to find the same information categories under both
headings or would they have different expectations?
Testing the full usability of the site when it is functionally complete and most if not all
of the content is there. This stage is meant to put the site through its paces in a rigorous
manner. Students should be clear that the site is not on the WWW at this point. It is on a
server and can be accessed from a usability laboratory or at a distance. Much of the value
will be lost, however, if a running commentary during testing and a final debrief after
testing are not collected by a trained facilitator. This is described with images on the
Nielsen link. You are likely to be struck by how much this sounds like directions for a
focus group and students need to recognize the similarity.
You probably need to point out that this example comes from the early days of the web. We
should be grateful that Dr. Neilsen preserved it because few enterprises will now release this
kind of information because of its competitive significance. The good news is that the structure
of usability testing hasn’t changed much. There is now software to track each mouse click that is
probably more sophisticated than what Dr. Neilsen used in his early tests. Some usability labs
incorporate eye-tracking capability into their set-up. However, the focus group origins of the
process remain intact. Dr. Neilsen’s early and simple set-up can be used to emphasize to
students that this is something they can do for their companies themselves. A test can be set up in
a conference room, quickly and without great expense. It can be outsourced to a firm that has
specialized competence in the area if resources are available. There is simply no excuse for
businesses not to test the usability of their sites before going online and from time to time
thereafter.
Alternatively you could assign Interactive Exercise 14.1 for class preparation. Notice that
Usability Sciences is recruiting for a test panel and provide information at the bottom of the
home page: http://www.usabilitysciences.com. They also have a lab tour page that shows a setup
only slightly updated from Dr. Neilsen’s early tests.
The Pareto curve makes a powerful argument for the affordability of usability testing. Over 75%
of the problems can be identified with 5 user tests. Only 15 are needed to find 100% of the
usability problems with the site. The general concept of a Pareto curve is important and it might
be worthwhile to take a few minutes and talk about the general importance of the 80/20 rule in
many areas of marketing.
Enterprise Metrics for Evaluating Websites
This section is introduced with the example of Fresh Direct, a food delivery service. In the U.S.
the landscape is littered with failed food service delivery companies; they are more popular in
other countries. Is one of the reasons trust in the quality of products and the service of the
company? Probably. Fresh Direct needed to know which product categories were more popular
and which promotions were working best.
Specifics included looking carefully at metrics for the recommendation engine, which showed
that recommended (complimentary) items sold well. They were offering a new prime meat
product and were able to target emails only to customers who had previously purchased meat.
They also did a Valentine’s Day email promotion that worked well; this data could have come
from their email platform as well as from an integrated metrics platform. Metrics on keyword
searched that direct visitors to the Fresh Direct site can provide a number of valuable insights. So
does monitoring the metrics specifically related to their shopping cart.
All this is available as metrics. Only when they got to the question “which customers have
iPhones” did they have to resort to marketing research. They might have done a pop-up survey
(questionable sample) or send an email survey. Google has a new tool for consumer surveys that
has interesting methodology: http://googlesmb.blogspot.com/2012/03/fast-accurate-andaffordable-way-to-do.html. It’s not clear to me whether it can be attached to a piece of content on
your own blog, but it seems likely.
This series of marketing activities were all online. It is important to keep reminding students that
offline promotions and other marketing activities should be incorporated into campaigns as
needed and that metrics platforms can often capture the results of the offline activities.
The subject of website performance is treated briefly. The aim is to let students be exposed to
the “big picture.” There are two basic types of site metrics: business metrics, our focus—and
technical metrics, used by the IT professionals who run the site to monitor and fine-tune the
performance of the site. Figure 14.5 is an example of a performance report. It deals with links,
both internal and external, and gives a number of data items that indicate which links are
working properly and which are not. Students should focus on it only long enough to recognize
that it provides no actionable marketing insights. Marketers should visit their own site frequently
and make their judgments about performance as a customer does.
This is also a point at which you might bring in a guest speaker, perhaps the web master of your
school’s site, to explain how they use site metrics and to show some in real time. Or, again, you
might want to wait until you have completed the chapter so a guest speaker will be illustrating
concepts you have already covered. Agency people are often happy to talk about how they use
metrics to serve their clients. Corporate speakers usually don’t like to divulge much detailed
information; good use of metrics is important to their success and closely held.
There are three basic types of behavioral data that students need to understand before getting into
the details. As described in the text they are:

Traffic data describes activity on the site. It includes metrics such as number of visitors,
sessions, and page views.
 Audience data describes both the behavior of people on the site—where they come from,
what paths they take through the site, and whether they take desired actions—and the
people themselves using both anonymous and identified profile data as described in
Chapter 4.
 Campaign data describes the results of a specific marketing program like an online
display ad or an email program.
It is admittedly hard to draw a clear line between “traffic” and “audience.”
Figure 14.6 shows a report from a site that used to be a simple hit counter. It has grown into a
more robust metrics platform. There is a link on the PPT slide so you can substitute a simple hit
counter home page if you wish.
If some of your students have personal websites, you might ask if they have a hit counter and, if
so, what they use it for. The likely answer is, “Not much.” The data does little except satisfy
curiosity on a personal website. In fact, this type of high-level data doesn’t do much for any
website.
Students need to understand what a server request log really is. They are really nasty things that
no marketer wants to look at, but marketers need to have a general sense of where their data
comes from. The main entries, by IP address, are:







The IP address of the requesting computer
Date and time of request
Code indicating whether request was successful or not
Number of bytes of data transferred
Referring site
Type and version of web browser making request
Operating system of computer making request
The data is collected for each hit (file), and the spreadsheet analogy helps to make the enormity
of the resulting database seem real to the students.
A web master at your school could give you a copy of a page from a server request log—that
makes the point quite effectively! Students can move on from that with gratitude for web
metrics programs that make data accessible!
Tagged web pages are necessary in order to track visitors through the site. Otherwise you have
data on page views, but not what pages one individual visitor looked at and in which order,
which is obviously important to the marketer. These little pieces of code have been given
numerous names. The semantic confusion is heightened when you realize there are completely
different uses for tags—tagging images on Flickr or Pinterest, for example. This may raise
privacy questions in the minds of your students. If so, it is a good time to encourage them to start
reading privacy policies to find out what, how and if marketers are telling visitors about what
they do—and why they do it. It is also a good question to ask of the metrics person at your
school. Table 14.1 summarizes some of the data that can be provided for a single visitor to the
site using tagged pages.
Cookies—session or persistent, first-party or third-party, are necessary to the process of tracking
a visitor through the site over multiple visits. The description of cookies here is rather brief. The
point is made in Figure 14.7 which shows that cookies and tagged pages work together to
provide behavioral data at the individual level.
This may be too much for your students, but you should be aware that the page view has been a
key metric, probably because it is much like “exposure” in traditional advertising. However, the
page view is losing currency because of Ajax technology (Old Navy and many retail sites are
good examples). Basically, many pages are loaded as thumbnails. Looking at only that “page
view” does not show which of the thumbnails were clicked on. That can be handled by metrics
platforms, but it has pretty much done away with the original idea of page view, since the
meaning has become inconsistent between Ajax pages and regular product pages.
This may also be a point where student’s personal concerns about the privacy of their data
becomes evident. You can engage in an extended discussion, but at this point students will lack
the information and the perspectives from Chapter 15. I encourage them to focus solely on the
marketing uses of the data at this point. The consumer/privacy concerns will be covered in the
next chapter.
Server logs collect data without the active participation or permission of the user and often
without their knowledge. Panels of Internet users are recruited who permit tracking software to
be placed on their computers as indicated in the text. Neither Nielsen nor comScore are
particularly transparent, at least to the public user, about the exact composition of their panels
and how they are recruited.
http://www.nielsen-online.com/downloads/us/NetView_US.pdf
http://www.comscore.com/About_comScore/Methodology
These data have the usual advantages of panel data including identified respondents and the
ability to collect demographic and lifestyle data about them. Longitudinal data is also an output
of panel research. Firms that maintain panels are eager to do custom work for individual clients
as well as to sell their standard research products.
Measuring Website Traffic, Audiences, and Campaigns
This section attempts to give a reasoned explanation of traffic and audience measures.
Fortunately, it is clear what we mean by campaign data. As pointed out above, the line between
traffic and audience is blurry. It is important to keep emphasizing huge volume of data that is
available.
The text lists some of the key metrics in each category. On the PPT they have been summarized
in a table to try to clarify when is a confusing picture, if for no other reason that the almost
limitless number of specific variables that are available. By combining the possible time frame
with only a few of the basic metrics, the magnitude of the metrics challenge becomes evident.
The engaged student should already be thinking, “So how do you choose which ones?”
I have only one answer. You have to have good (SMART) marketing objectives and map metrics
to your objectives. It’s hard to find straightforward examples of objectives at the campaign level;
they tend to be more at the annual marketing plan level. This one is pretty good
http://www.steinarknutsen.com/setting-smart-goals-for-your-2012-internet-marketing-strategy
and it’s very clear about what SMART is and why it is important. Figure 14.8 gives examples
without the emphasis on SMART.
I strongly recommend that you play the Google Analytics Product Tour before you cover this
section or you assign it for outside viewing before you cover the material in class (Interactive
Exercise 14.2).
The example is a real one—for my blog. It’s important for students to know that each blog post
has an individual URL and therefore can be tracked as a web page.
If you matched to Figure 14.9 you’d find that most of the peaks are when I posted. However, it
maintains a steady, lower level of traffic between postings. The reason is that I write for search,
using keywords in titles, content and tagging all posts. Now all my blog posts go through
automatically to Twitter. Not to Facebook, because I reserve that page for personal use. The
Twitter postings drive traffic to the blog and gain me a small but steady stream of followers,
although it’s not something I work on.
The blowup of the map from the dashboard (Figure 14.10) is interesting and it can have
commercial relevance. For me, it just indicates that a number of former students and a few
international bloggers follow the blog.
I’ve tried to mimic some of my use of Google Analytics in Figure 14.11. I tried to tell an
interesting story and keep it as simple as I could. If you are using your own analytics or having
someone come in to demonstrate live, think about the story aspect. It’s more meaningful to
students that just showing the pages that exist. Their eyes glaze over after just a few pages.
In this case I did notice a continuing stream of visitors to an old post “Do Facebook Ads Work.”
I knew I hadn’t done anything to publicize this particular post, so I was puzzled, to put it mildly.
You can follow my odyssey in as much or as little detail as possible. I find considerable detail is
necessary to give students even a rudimentary understanding, hence the content boxes beside
each screen capture.
At this point some student will invariably ask if they can see Google Analytics reports for sites
they are interested in. The answer is emphatically “NO.” Using Google’s Blogger platform and
their analytics is pretty easy. Google, however, is good about making sure the user owns a piece
of content before allowing many types of actions, including placing Google Analytics on it.
The answer to another frequent question is that Google Analytics offers virtually all of the
advanced metrics from paid platforms that are illustrated in the remainder of the chapter.
Students should see some different examples and I don’t want to be a shill for Google, much as I
find their tools usable.
Segmentation and Conversion Analysis
The example of segmentation analysis in Figure 14.12a was chosen because it was analyzing an
email campaign, not a website. The link takes you to the product PDF. It is hard to reproduce
these captures so they are readable, so you could go to it live. Figure 14.12b goes back to a web
page, showing click-through overlays. It’s nice, after looking at all the numbers to look at a
graph. Most platforms allow you to toggle back and forth between numbers, which the marketer
needs to study, and graphics, which managers are generally happier to view.
Figure 14.13 shows a conversion path graphically (the numerical version is mind numbing).
If you want to spend some time with it you can ask students to interpret the drop off at each step.
The number who don’t go on to the next step is pretty straightforward. The answer to “why” is
key. The page from which they left may offer hints. If one page shows a lot of exits, it may
suggest a need for usability analysis. Or it may take marketing research to answer the “why;” a
product that is priced too high, for example.
It is probably more realistic to just try to get students to understand the nature and importance of
the funnel. A full, detailed path will be a lot more complicated then this summary graphic, but it
can provide a lot of useful guidance.
Keep reminding students that these are only a few of the reports available.
Metrics Based on Panel Data
This section reemphasizes the usefulness of panel data. The metrics based on server logs can
only produce data for the marketing programs of the business itself. Students need to be clear on
that. I keep being asked, “Can I put Google Analytics on a site I’m interested in?” NO! That can
only be done by the site owner.
In order to obtain competitive data (as well as customer-identified longitudinal data) a panel is
required. This page talks about products http://www.nielsenonline.com/solutions.jsp?section=sol_4&amp%3Bnav=4 and you can find examples
http://www.nielsenonline.com/downloads/nz/NZ_Online_Travel_Report_%202010%20Promo.pdf. No individual
site can collect this data itself.
Both Quantcast and Compete used to offer comparative data for free. Now they only offer data
for one site at a time. Both use panels to collect the data. Theirs seem to be convenience samples
while Nielsen and comScore both recruit statistically valid panels.
The next section goes a step further and shows an excellent explanation of how metrics, which
measure single marketing variables, are aggregated into KPIs which measure higher-level
business objectives. Kaushik is Google’s analytics evangelist and his blog is a good place to
search for specific analytics topics. This particular chart comes from a student in his metrics
certification program, a sure sign of how important this Internet marketing topic has become.
The topics of social media metrics and mobile metrics needed to be mentioned here for
completeness although they are discussed in their separate chapters. There is good news and bad
news. The bad news—social media metrics are specific to the social media space and still mostly
available only through the platform, although the recent purchase of Radian6 by salesforce.com
shows that integration is beginning to occur. The good news is that mobile websites can be
evaluated using essentially the same set of metrics used by traditional sites. Consequently, they
are much easier to integrate into the existing metrics platforms.
Marketers’ Confidence in Their Metrics
One thing that never seems to change is that marketing managers are not confident that they have
the data they need about the effectiveness of their Internet marketing activities. In Figure 14.16
few rate their capabilities as excellent or even good –20%. A lot of them are working on it, but it
doesn’t seem to be a destination with a clear goal. The Internet, its platforms and its metrics are
changing so rapidly that most marketers are severely challenged to keep up. eMarketer
frequently reports on studies that bear on the subject, or just try searching for recent reports. You
will usually find newer ones because of the importance and the difficulty of the measurement of
Internet marketing effectiveness.
Discussion Questions
1. The term “metrics” is commonly used by Internet marketers. Explain your
understanding of the meaning of the term.
The term metrics is a piece of marketing jargon, but it seems appropriate for its intended
meaning. Metrics are numbers, pure and simple.
Wherever it came from, it is used throughout marketing today, where its importance continues to
increase, propelled by the measurability of the Internet. The Internet is, so far at least, the
ultimately trackable medium. As such, it provides the marketer with unparallel ability to
measure and evaluate the effectiveness of Internet marketing programs of all kinds. This
includes not only the usability and effectiveness of websites but of outbound marketing programs
like email promotions. Students need to be reminded frequently of the importance of trackability
and measurability if they are to clearly understand the fundamental difference between mass
media and the Internet.
They also should be reminded that the importance of these issues is increasingly being
recognized by mass media marketers.
2. True or False: Internet marketers need to decide whether they will conduct
measurement from a visitor usability perspective or from a traffic and audience
measurement perspective. Defend your answer.
The issue is not “either or.” It is “both.” This question tests the students’ understanding of
testing website usability from the perspective of providing satisfactory user experience versus
traffic and audience measures (behavioral) measures that are the output of the systems used to
run the Internet. They are both critically important and should be used by all Internet marketers
in pursuit of their states goals.
3. At what point in the site development or redevelopment process should usability testing
be done?
Usability testing should start in the prototype stage. There are many reasons to argue for many
small prototype usability tests (find it, fix it) than one or two large ones when mistakes have
already been made and will take substantial effort to correct. Full Beta testing should be carried
out with usability in mind.
It is essential that usability testing be an ongoing part of site management. It can, in fact must be,
done when metrics indicate there may be a usability problem. It can also be done on a regular
basis, sort of like maintaining your home. Good, regular maintenance is a useful perspective.
4. The term “hit counter” implies that all this technology can do is to measure the number
of people who enter a site. Based on the discussion in this chapter, is that an accurate
perception? Why or why not?
Students may be familiar with hit counters, may have even used one on a personal site they have
built. The simple idea of a hit counter that increments each time a page is accessed is a useful
foundation for understanding more complex metrics. However, that is just about all they do, and
the more complex metrics obtained in one way or another from server request logs provide the
information that drives commercial marketing programs. It is interesting to know how many
people have visited your site, but that information alone is of limited usefulness in determining
how to improve the site or marketing programs associated with it.
5. Describe the similarities and differences between server request logs and panel data.
Which do you think is most useful?


Server request logs, are the operational records kept by any file server. They provide
essential data for operating the system. In the hands of a competent services provider the
data can be transformed into data items that are useful to marketers.
Panel data are collected by placing special-purpose software on the user’s computer, with
the user’s permission. This software monitors the clickstream, but since it was developed
for the purpose of collecting marketing data, it is reasonable to assume that it does so in a
more direct way than do server logs.
Each technique collects data in a different manner. The actual data items and marketing
variables produced are essentially the same, although all variables may not be available from all
sources. It is also true that logs are “server side” data.
Panel data has quite different characteristics, primarily the demographic and lifestyle background
(survey) data that is available on identified users, and the ability to produce longitudinal data, the
ability to access competitive comparisons, and access to customized panel research projects.
6. True or False: The web master and the marketer use the same metrics when assessing
website effectiveness.
The web master is tasked with site performance and needs that data. The marketer is tasked with
marketing effectiveness and needs that data. They are separate data sets. However, it cannot be
stressed too often that the marketer must care about site performance that creates satisfactory
customer experience. That is best observed—all marketers should spend time on their sites,
acting like customers and finding out how the site performs for them.
7. What are the uses of server log data, cookies, and tagged pages in the collection of
website effectiveness data?
That is the point of Figure 14.7. Server log data, tagged pages, and cookies all work together to
provide a tsunami of data that needs to be handled on a metrics platform—a free one or a paid
platform.
8. What are some of the specific metrics that measure Internet traffic, audiences, and
campaigns? Which ones do you think are most important?
The text lists the following under each category:


Traffic measures document site activity:
 Hits: the number of files requested
 Impressions: the number of times an ad banner is requested by a browser
 Page views or deliveries: the number of times a web page is requested. Remember
that page views are becoming less revealing of what is really going on.
 Sessions: the amount activity on a site during a specified period of time
 Click-throughs: the number of times visitors come to the site by clicking on an ad
Audience measures provide data about the people who visit the site:
 Visitors: the number of people who visit a site
o Total (includes multiple visits) or unique (different people) during a
specified time frame
o Unidentified (anonymous) or identified (registered or customer)
 Unique visitors: have visited at least once during a specified period of time. Not
double-counted during that time.


Behavior on the site
o Number of page views
o Session time
o Path through the site
o Shopping cart abandonment
o Entry page (many visitors do not enter through the home page)
Campaign Measures provide data about the effectiveness of marketing efforts
o By communications channel; email, mail, online banners, etc.
o By offer; free shipping versus 25% off, for example
o Search effectiveness by keyword.
Each metric can be obtained for a variety of specific time frames.
If you have gone live to one of the metrics sites, or have assigned students to do so on their own,
they should have others. In any event, it is important to keep reminding them of the “almost
infinite” issue.
9. What are some of the special advantages of user panel data?
The advantages are basically those we learned in marketing research. Panel data allows the
marketer to collect a lot of profile data and merge that with the behavioral data from the web.
That allows for longitudinal data about an individual or segments. It also allows the marketer to
purchase similar data for her competitors or request research to meet specific purposes—the
travel industry in New Zealand example given above, for example. The custom research may be
simply a reconfiguration of existing data or it can include collection of survey data from panel
members.
10. How should the marketer go about choosing the right metrics to measure website or
campaign effectiveness?
Another thing that cannot be said too often is that the only way is to match metrics to SMART
objectives. Here’s a good post from Avanish Kaushik, who talks about the whole process
http://www.kaushik.net/avinash/digital-marketing-and-measurement-model with some graphics
you could use and an earlier post with excellent definitions:
http://www.kaushik.net/avinash/web-analytics-101-definitions-goals-metrics-kpis-dimensionstargets.
11. What art SMART objectives?
Kaushik uses DUMB:
Doable
Understandable
Manageable
Beneficial (http://www.kaushik.net/avinash/digital-marketing-and-measurement-model)
I still have a preference for SMART and link to the quality management literature:
Specific
Measurable
Achievable
Realistic
Time bounded
I’ll admit that achievable and realistic overlap a bit. Students need to remember one or the other.
12. What is the meaning and importance of the term “KPI”?
A Key Performance Indicator is a metric that has a direct relationship to business success.
(http://www.kaushik.net/avinash/web-analytics-101-definitions-goals-metrics-kpis-dimensionstargets) It can be a single metric. It can be a composite metric composed of several individual
metrics. The point is that it’s not something nice like “engagement.” It is something you can take
to the bank (or to your boss) like “ecommerce sales last month.”
13. What are some of the main challenges facing the web metrics discipline at present?
The challenge all alone—and it only gets worse—is the huge amount of data (now “big data”)
that is produced. It must be aggregated, analyzed, transformed into decision-relevant marketing
variables, decisions must be made and programs executed, more data collected—and then the
cycle starts again. There is just more data—particularly social and mobile at present.
Students who have struggled with understanding the complexity of metrics may have recognized
that their managers have not had this training and know little or nothing specifically about
Internet metrics. Is that at least part of the reason that managers are so unsure of the correctness
of their metrics, the ROI on their Internet marketing objectives? Part of it may well be that the
right metrics and KPIs are not being used. Part of it may also be that managers don’t entirely
understand what they are being given. “Manage up.”
14. Do you believe that most commercial websites are using metrics to improve customer
experience and enhance marketing effectiveness? Do you have any personal experiences
that support your view?
The reality appears to me that most websites, even startups, usually put metrics on the site—
Google Analytics being the starter platform of choice. At the other end, large sites like Staples
may not only have sophisticated metrics platforms and a staff of sophisticated analysts but may
also have their own usability lab like Staples does. In between there is a huge range of effective
use of metrics. I’d put most sites, especially the small and mid-sized ones on the low end. Data
throughout the chapter supports the skepticism of a majority of managers about the quality of the
metrics they are given. Data from 2012 can be found here:
http://www.marketingcharts.com/direct/digital-marketing-tools-not-yet-providing-key-strategicinsights-21662.
Internet Exercises
1. Internet Career Builder Exercise.
2. Conduct a partial usability analysis of your school’s website. Describe two tasks that
you believe the target audience wants to perform on the site. Go to the site, and attempt
to perform both tasks. Record your experience for each, noting the time it took to
perform each task. What recommendations would you make to the web master of the
site based on your test?
This is a good activity to do live in the classroom or in a lab. Alternatively, you can assign
students to do it as an outside assignment. If so, they could be required to come up with a clear
set of instructions, just as if they were running a usability test.
Encourage students to choose commonplace tasks—locate a course, find out how to get an
application for admission, find the email address of a faculty member, and so forth. Have them
think about tasks they have to perform that can reasonably be performed online and that students
would expect to be possible on the school’s website. Then have them document their efforts and
discuss not only the specifics, but the pattern. Do some functions appear to be handled well
while others are not? Do some departments, schools or colleges have good sites while others do
not? What about student organizations that typically build their own sites?
If your school or some unit has done actual usability testing, it would be interesting to invite the
person who directed it to explain what they did, what they found, and what they did as a result.
The changes can be striking. If students were present when the redesigned site was launched,
they may have noticed the improvement in usability—hopefully!
3. Using one of the sites you are tracking, prepare a careful summary of the objectives you
believe that site has. In doing so, keep in mind that the objectives of the site may be a
subset of the marketing and business objectives of the organization. When you are
satisfied that the set of objectives is complete, list the corresponding metric or metrics
that would be needed to measure each objective. Present this information is a table that
shows objectives and the corresponding metrics.
This exercise can have two positive results—a better understanding of marketing objectives as
well as a more in-depth understanding of metrics. Encourage students to look at annual reports,
press releases and trade literature to find out what they can about the nature of the objectives for
the enterprise, SBU if relevant, and site. They may find objectives, usually general in nature, but
they can often imply objectives. The results that are being reported come from metrics of some
kind and their collection is based on objectives. They should recognize these as hints from which
they can reconstruct objectives. The other major learning is in the attempt to connect objectives
and metrics. This is an important marketing skill, the necessity for which is stressed in the
chapter.
4. Using a website with which you are familiar, develop a scenario that tells the story of
how one user segment would behave on a visit to the site. When you are satisfied that
your scenario represents behavior of one segment adequately, identify the metrics that
would be useful in ascertaining if the segment’s needs are being met and if the customer
experience for this segment is satisfactory.
Sitting down and thinking about how a member of an identified target segment goes about using
a site is useful. Writing it down as a logical, carefully-crafted scenario is even more useful and
good practice.
Here’s a how-to post with links to articles about scenarios used by Best Buy and Adidas:
http://www.user.com/personas.htm.
Key Terms
benchmark an industry standard against which a business can measure its performance.
clickstream the complete data record, made up of mouse clicks, of consumer activity on the
Internet during a specified period, usually the duration of a visit to a single website.
concept testing research performed on the idea behind a product or a communications program.
cookie a few lines of code that a website or advertising network places on a user’s computer to
store data about the user’s activities on the site.
dashboard customizable display of summary data.
DNS (Domain Name System) the process for converting the name of a website into its IP
address.
focus group the qualitative marketing research technique in which a group, usually six to twelve
people who fit a particular profile, are brought together to discuss an issue under the guidance of
a skilled moderator.
hit counter a piece of software inserted onto a website that measures the number of visits to the
site.
metric a number used for evaluation or assessment of business success.
multichannel using more than one channel of distribution or communications to reach the
customer.
page view a page actually seen by a visitor; generally measured as a page being delivered to the
visitor, which is not exactly the same thing.
Pareto curve a plot of number of occurrences against percent of total; the source of the 80/20
rule.
pixel short for picture element; a single point on a display screen or in a graphic image.
prototype a preliminary model.
server a computer from which other computers request files.
server request log record kept at the server level that records each file requested from a website.
tag a piece of code that identifies a page or an element on it.
tagged web page a technique in which a small image is placed on a web page; used in
conjunction with a cookie on the user’s computer, the image returns data about user activity on
the web page.
touchpoint marketing jargon for each channel available for customer interaction.
unique visitor an identifiably distinct, although not necessarily identifiable, visitor to a website
within a specified period.
usability testing ensuring that it is easy for visitors to navigate and, in general, to find desired
content quickly and efficiently on a website.