Download How Explanatory Analytics Enables Marketing Leadership

How
Explanatory
Analytics
Enables
Marketing
Leadership
BY J O SH R E YNO L DS
HE AD O F MAR K E TING & CL IE NT CO NSULTING
Q UANTIF IND
1
©2016 Quantifind, All rights reserved.
How Explanatory Analytics Enables
Marketing Leadership
The role of the Chief Marketing Officer has always been somewhat nebulous.
CMOs wear multiple hats. No matter the role, they are held accountable to
drive increasingly hard-to-control outcomes – and the recent proliferation
of new technologies, new consumer behaviors, and unpredictable business
pressures have only further complicated things. What’s more, this identity crisis
is exacerbated by the fact that CMOs don’t feel in control of driving bottom-line
results, and are clamoring for meaningful digital insights to help them.
As a result, marketing leaders are asking themselves:
• Which CMO hat do I need to wear today?
• Which roles feel most natural to me?
• How can I enhance my skill sets across all the various roles I need to play?
The answer lies in recent advances in marketing analytics. Until now, most
analytics approaches haven’t helped CMOs set their own agenda, primarily
because none of them correlated to growth. Social media listening measures
buzz and sentiment, but that doesn’t map to revenue. Predictive analytics may tell
you what’s likely to happen if you don’t change course, but that doesn’t tell you
who’s driving revenue, why, or how to change the outcome. And traditional brand
health metrics, while sometimes helpful, are often time-intensive, backwardslooking, and rarely correlated to sales. Perhaps that’s why all these approaches
are facing increasing scrutiny and skepticism in the boardroom.
Think about it: In order to be a strategic asset to the company, a CMO needs to
explain why something is happening and how to change it.
And that’s where explanatory analytics comes in. Explanatory analytics looks
not only at which results marketing is driving, but also why, with whom, and
how to change them. That knowledge allows marketers to shape their own roles,
reinvigorate their relationships with their CEOs, and bridge internal silos across
departments.
What’s more, explanatory analytics helps to unite the quantitative tribes of
research, data, and analysis with the creative tribes of brand, campaigns, and
messaging. Perhaps most importantly, this new approach keeps humans in the
loop as big data computing continues to proliferate throughout the marketing
discipline. It helps marketers use numbers to shape narrative. And it allows
marketing executives to explore, understand, and change their impacts on
revenue by intuitively searching through data visualizations correlated to KPIs
CEOs already care about.
In other words, explanatory analytics shifts marketers from playing defense to
offense. It changes the question from “So what?” to “Now what?” And that’s a
game-changer.
2
©2016 Quantifind, All rights reserved.
Introduction
And now, for the first time, CMOs can use data to shape their own objectives,
drive growth with surgical precision, and determine what kind of marketing leader
they want to be.
Let’s begin with a look at the science behind explanatory analytics, and then we’ll
examine how it applies to five CMO roles as identified and categorized by Deloitte.1
Explanatory Analytics At-a-Glance
The trouble with most forms of marketing data is that so much of it is noise and
static. On average, only about 20% of social data has any explanatory value when
it comes to revenue. But once you find that 20%, all kinds of new growth-driving
possibilities emerge.
The graph below represents real data from an extremely popular fast-food
restaurant illustrating the difference between noise and signal. The shaded
gray area represents “buzz;” the green line represents sales, and the blue line
represents social data tracking to sales. Within that blue line are all kinds of
additional data points — verbatims, affinity groups, demographics, regions, etc. —
that explain why revenue is moving the way it is, and how to change it.
83
160
78
140
73
120
68
100
63
80
58
60
53
40
48
20
43
Brand Mentions (thousands)
Sales ($ millions)
Explanatory analytics
systematically filters out
everything that … doesn’t
correlate to sales — and measures
how strongly the remainder does
correlates to sales.
0
Jan 20, 2014
●
Mar 20
Brand Buzz
●
May 20
Sales
●
Jul 20
Sep 20
Nov 20 Jan 20, 2015
Mar 20
May 20
Jul 20
Buyers
1
Diana O’Brien, The multiple roles of the 21st century CMO, February 1, 2016, Deloitte, http://cmo.deloitte.com/the-
multiple-roles-of-the-21st-century-cmo/, accessed May 5, 2016. Used with permission.
3
©2016 Quantifind, All rights reserved.
HOW EX PLANATORY ANALYTICS ENABLES MAR KETING LEAD ER S H IP
How do you find that magical blue line? Explanatory analytics systematically
filters out everything that isn’t about the brand, everything that isn’t from a real
human, and everything that doesn’t correlate to sales — and measures how
strongly the remainder does correlates to sales. Then the data is categorized into
searchable dimensions that let marketers explore the who, what, where, when
and why behind their revenue.
raw data
brand buzz: statement about the brand, including marketing messages
and Tweets.
brand buzz
organic data: statements by people offering their unaided views.
organic
buyer data: statements made by people whose views correlate with KPI
movement.
buyer
topics: themes of conversation about the brand, including marketing
messages and Tweets.
topics
drivers: brand conversations that most align with buyer decision-making.
Now, let’s take a look at five CMO roles, and examine how explanatory analytics
unlocks the full potential of marketing leaders.
The Deloitte Model
At its Next Generation CMO Academy in 2016, Deloitte delineated its five roles
Chief Marketing Officers play. It's a great model for understanding the modes
with which you need to engage your colleagues, the different skill sets and
perspectives you need to tap into, and for identifying where a fresh approach to
data and insights can help boost your effectiveness as a marketing leader.
Almost every senior marketer plays each one of these roles at one time or
another, and marketers must be able to move from role to role at the appropriate
time in order to lead effectively:
• Customer Champion
• Capability Builder
• Innovation Catalyst
• Growth Driver
• Chief Storyteller
Here’s how explanatory analytics empowers each one…
4
©2016 Quantifind, All rights reserved.
DELO I TTE D EFINITION
Customer Champion
/'kəs-tə-mər 'cham-pē-ən/
Deloitte asserts that CMOs acting
as Customer Champions align the
organization around customer
centricity. They utilize data insights
and analytics to deliver superior,
personalized customer experiences
and measurable business results.
DEFAULT DATA SOLU T ION S
Consumer insights, focus groups,
advisory councils, customer journey
analysis
Customer Champions have
leaned heavily on focus groups
and other forms of customer
intelligence to help them win
cases. But those approaches are
too easy for skeptical executives
to question.
When wearing this hat, a CMO champions the preferences and priorities of the
customer across the C-Suite. The success metrics most commonly associated
with a Customer Champion are customer satisfaction scores, Net Promoter
Scores, customer loyalty/customer churn, share of wallet, and brand engagement.
Their job is to know how close the company is to its customers, and figure out
how to get closer.
The challenge is that all these metrics, while helpful in many regards, are one
or more steps removed from the most important metric C-suite peers are
tracking — revenue. That’s why Customer Champions have typically faced an
uphill battle when it comes to internal credibility. With seemingly “soft” metrics,
it’s not particularly easy to go ask a COO to adjust operations, or a CFO to
change pricing models, or a head of product strategy to change the UX. Those
kinds of strategic bets are only made after serious due diligence.
And yet Customer Champions must act as trusted advisors to other senior
executives if they ever hope to be more than a passive conveyer of customer
sentiment. If they truly want to become change agents and successfully advocate
for a better relationship with customers, they need to drive new outcomes, and
that requires quantitative evidence.
Traditionally, Customer Champions have leaned heavily on focus groups
and other forms of customer intelligence to help them win cases. But those
approaches are too easy for skeptical executives to question. The sample sizes
are too small. The responses aren’t unaided or organic. We haven’t accounted for
various statistical biases. We’re just hearing what we want to hear.
This is where explanatory analytics can shift the burden of proof for Customer
Champions. By filtering out all the noise and static, Customer Champions
are empowered with a world of organic and unaided customer comments
quantitatively connected with revenue.
In an explanatory analytics model, the Customer Champion can not only predetermine which topics, demographics and issues to explore, they can also let
data whisper to them. Organically-emerging topics and data clusters may suggest
new demographics to consider, unexpected affinities to leverage, and hidden
opportunities to explore.
For example, a fast-food restaurant wanted to increase breakfast sales. They
had detailed day-part sales data that told them what they were selling, where,
and when. And they had a good running hypothesis that the best way to grow
breakfast sales was by selling more breakfast to teens. The food options
and the creative tested well with teens, yet sales didn't grow. But by taking
an explanatory analytics approach, and exploring the data that mattered, the
5
©2016 Quantifind, All rights reserved.
HOW EX PLANATORY ANALYTICS ENABLES MAR KETING LEAD ER S H IP
company discovered that moms didn’t like the restaurant’s coffee all that much,
and the teens were relying on a ride from mom. So the answer to the question
“How do we sell more breakfast to teens?” was “Give moms better coffee.”
That kind of insight gives a Customer Champion persuading power. That kind of
insight helps the Customer Champion speak boldly on behalf of the people, with
quantifiable evidence to back up assertions. That kind of insight is what gives
a Customer Champion the ability to go to their CEO with a decisive agenda for
driving revenue.
DELO I TTE D EFINITION
Capability Builder
/̩kā-pə-'bi-lə-tē 'bil-dər/
Deloitte describes CMOs as Capability
Builders when they develop robust
marketing capabilities (e.g., technology
fluency, digital expertise, customer
data and insights, data analytics) to
help the organization compete in the
future.
DEFAULT DATA SOLU T ION S
Predictive analytics, ROI calculations,
after-the-fact KPI tracking
When a CMO is acting in Capability Builder mode, they’re focused on unlocking
the value of existing investments. Capability Builders focus on what functions,
skill sets, and expertise the company needs to succeed. They’re typically
operational marketing executives, focused on infrastructure and process.
They’re constantly tracking things like KPIs, workflow efficiencies, and reporting
structures. They’re constantly on the lookout for signs of trouble, proof of
success, and insights that help their organization compete now and in the future.
So the Capability Builder understands the power of math. They want to quantify
the effectiveness of individuals and teams. They want to know the CPM and CPL
of every ad buy. They’re constantly monitoring the lifetime value of a customer.
They want to know how they stack up against the competition, quantitatively,
along multiple vectors. And they want to be able to predict success and make
the rights bets. In other words, the Capability Builder needs to know exactly how
much revenue a dollar of marketing spend can bring back to the bottom line.
As a result, marketers operating as Capability Builders often find it easier to earn
the respect and credibility of their C-Suite peers, particularly the CFO and COO.
In a few cases, the CMO may actually be a former CFO. But what they don’t
always have is the trust of the CEO or the head of sales who doubts they have
the insights needed to do more than just track outcomes.
That’s because most metrics Capability Builders use tell them what’s already
happened, not what to do next. ROI is inherently backwards-looking. Nobody
wants to be relegated to marketing in the rear-view mirror.
In recent times, some Capability Builders have turned to predictive analytics,
hoping to get a sense of ROI before the investment is made. But even predictive
models fall short of the mark, as they only predict the likely outcome if you don’t
make any changes. They don’t tell you which changes to make or why.
And yet the Capability Builder knows, intuitively, the enormous value of the
data sets at their disposal. The question is how to tap into the math that matters.
6
©2016 Quantifind, All rights reserved.
Explanatory analytics lets the
Capability Builder feel assured
they’re advocating for the right
capabilities for the right reasons.
And by taking an investigative approach that uses explanatory analytics, the
Capability Builder can become the future-proofer of growth for the organization.
By tapping into the 20% of the external data sets that actually matters, the
Capability Builder can discover the correlations to their internal performance data
and explore the why behind the what.
Let’s say a car company is looking for a way to sell more cars, and they’ve
already identified millennials as the target market offering the most potential for
growth. On the one hand, it takes time to retool an assembly line and materially
change what goes into the drive train, engine or chassis, so it’s a real challenge
to use customer data to change the capabilities of the organization.
But with explanatory analytics, that car company can discover what really drives
sales among millennials. And as it turns out, what’s under the hood doesn’t
matter as much as what’s in the interior. That car company can pinpoint which
aspects of the interior most impact sales and strategically decide how to prioritize
the drivers’ experience. Seat heaters, displays, leather seats, sound systems,
Bluetooth capabilities — all these have the theoretical potential to lift sales with
millennials. But why guess? Explanatory analytics lets the Capability Builder feel
assured they’re advocating for the right capabilities for the right reasons.
Most importantly, explanatory analytics gives marketers wearing the Capability
Builder hat a metric that matters — one that explains not only where their brand is
winning, but also why and how to accelerate that growth.
DELO I TTE D EFINITION
Innovation Catalyst
/̩i-nə-'vā-shən 'ka-tə-ləst/
Deloitte asserts CMOs in the
Innovation Catalyst role create
breakthrough customer offerings and
advance marketing by experimenting
with new technology platforms,
alternative media, new tools and
techniques to transform the customer’s
experience while simultaneously
improving internal processes.
DEFAULT DATA SOLU T ION S
Social Listening platforms, web/
mobile/application analytics, customer
engagement metrics
We all have friends more technologically advanced than the rest of us. There’s
always that early adopter at the cocktail party sporting a pair of augmented
reality glasses. During the holidays they give friends gifts made with their
3D printers. They’re already over SnapChat because it’s “so 2015.” They’re
technological trendsetters who like discovering something first. There are CMOs
with similar technological proclivities who enjoy disrupting their categories.
Deloitte calls these people in this role Innovation Catalysts.
This leader wearing this hat is always looking for new platforms and possibilities.
Marketing automation, new advertising technologies, customer experience
advances, emotional and gesture recognition, advances in programmatic
approaches — Innovation Catalysts view all these technologies through the lens of
the entrepreneur looking for a game-changing advantage. Their default mode is
experimentation.
The marketing leader performing this role is like an athlete always on the lookout
for the latest breakthroughs in sports science. They use performance-enhancing
data to monitor progress and outpace the competition. That’s why Innovation
7
©2016 Quantifind, All rights reserved.
HOW EX PLANATORY ANALYTICS ENABLES MAR KETING LEAD ER S H IP
Without a strong understanding
of what is and isn’t driving
revenue before the
experimentation begins, the
Innovation Catalyst could get
labeled a cost center.
Catalysts are attracted to the latest advances in social media listening and
mobile, web and application analytics. They’re probably the first in their vertical
to experiment with emotion-recognition technologies and eye tracking. They’re
looking for clues into how their latest innovations are resonating with customers.
The challenge here, however, is that the reporting on the effectiveness of these
new platforms always lags behind results. Experimentation, while bold and
innovative, is often like shooting in the dark. And while there may indeed be a
first-mover advantage, that advantage is quickly lost unless the movement is in
the right direction.
In other words, Innovation Catalysts are often their own worst enemy.
Particularly in uncertain economic times, the appetite of the CEO or CFO for
“experimentation” is limited, at best. Without a strong understanding of what is
and isn’t driving revenue before the experimentation begins, the Innovation
Catalyst could get labeled a cost center. Or worse yet, when they do discover the
winning innovation formula, without hard metrics to show it was an intentional
success, the CMO may not get a fair share of the credit.
With explanatory analytics, the Innovation Catalyst knows what problem to solve
for. They know whom they need to reach, and how. Most importantly, they have a
real-time view into what’s working, what’s not, and why.
For example, a financial institution recently wanted to expand its reach into
existing customers. They started with the reasonable hypothesis that targeting
younger consumers with mobile banking solutions would be a smart bet. But
when they engaged with explanatory analytics, they soon discovered that in
fact older customers were more drawn to the mobile banking experience, while
younger customers cared more about in-branch experiences.
This kind of insight allows an Innovation Catalyst to focus on the right innovations
that matter most to the right target audiences. What’s more it gives them the
ability to take a smarter business case to internal stakeholders by showing
insights correlated to real KPIs. And as entrepreneurs who value “failing fast”
and accelerated learning curves, Innovation Catalysts can lean on explanatory
analytics to show them quickly which bets are paying off, which ones aren’t, and
how to adjust course. What’s more, they can bring their CEO and CFO a business
case in need of an innovative solution — and not the other way around.
8
©2016 Quantifind, All rights reserved.
Growth Driver
/'grōth 'drī-vər/
Deloitte describes CMOs in the Growth
Driver role as leaders who create and
manage a plan to achieve sustainable,
profitable growth.
DEFAULT DATA SOLU T ION S
Revenue models, sales analyses and
predictive analytics
When CMOs are primarily focused on revenue they are Growth Drivers. They
have explicitly embraced the mandate to drive growth. Ultimately, all CMOs are
held to this goal, but the Growth Driver leans into it. They speak the language of
sales. They look for ways to sell more to existing customers and more ways to
sell to new customers. They’re constantly tracking results and are most likely to
report how they’re doing to the CEO and CFO before the CEO and CFO have to
ask. Brand health metrics, buzz, sentiment and vanity metrics mean nothing to
the Growth Driver; they’ve already discovered the profound disconnect between
brand studies and the bottom line.
In other words, this CMO role looks the most like a Chief Revenue Officer, and
is the least likely to be displaced or disrupted by one. They live and die by the
revenue data.
The challenge for Growth Drivers is that although they may know what the
revenue is doing, they may not know why. Think about it. If a CMO goes into a
board meeting with great revenue results, but can’t explain why they achieved
them or how to improve upon them, how big of a win is that? But if a CMO walks
in with mixed results and can explain why, and can show exactly how to improve
results next quarter, that’s a great meeting. That’s a CMO who can drive growth.
And that’s what explanatory analytics does for the Growth Driver — it solves
revenue mysteries. Nobody likes a KPI line he or she can’t explain, even if it’s
a positive one. For CFOs and shareholders, predictability is as important as
profitability. And the successful Growth Driver is the one who can zoom in on the
“why” axis of any revenue chart and explain it.
Growth Drivers are the ones
who benefit most directly from
using explanatory analytics to
identify new opportunities and
competitive strategies. And by
filtering out all the external data
that doesn’t correlate to revenue,
explanatory analytics opens up a
whole new world of insights that
actually matter.
Of all the marketing leader roles, Growth Drivers are the ones who benefit
most directly from using explanatory analytics to identify new opportunities and
competitive strategies. And by filtering out all the external data that doesn’t
correlate to revenue, explanatory analytics opens up a whole new world of
insights that actually matter. Now they can use explanatory analytics platforms to
understand, explore, and change their impact on revenue.
Let’s say a beer company needs to find a way to grow sales. Let’s say they know
the demographics they appeal to most, and they have a strong sense of brand
identity. But what if they didn’t know how to steal market share from aggressive
competitors in a declining category?
With an explanatory analytics approach, a Growth Driver is able to benchmark the
company’s performance against the overall category, and specific competitors.
It lets them systematically target specific demographics, topics, and issues that
9
©2016 Quantifind, All rights reserved.
HOW EX PLANATORY ANALYTICS ENABLES MAR KETING LEAD ER S H IP
give them the best chance to beat the competition. It gives them the ability
to surgically identify distribution strategies, creative campaigns, sponsorships
and promotions that boost sales. And it helps them define the terms of the
engagement against competitors and pick and choose the battles they engage
in — the ones most likely to lead to victory.
DELO I TTE D EFINITION
Chief Storyteller
/'chēf 'stȯr-ē-̩te-lər/
Deloitte asserts that CMOs, as Chief
Storytellers, act as both architect
and steward of the brand by creating
and telling brand stories and inviting
consumers to participate in the
narrative. Develop, preserve, and
promote the brand’s relevance and
consistency while evolving its position,
meaning, and messaging.
DEFAULT DATA SOLU T ION S
Focus groups, creative testing, brand
metrics, consumer intelligence
This marketing leader in this role tests and tweaks brand narratives. Chief
Storytellers dig deep to make sure the brand is not just top of mind in the
marketplace, but also pragmatically positioned to drive sales. They know that
a product is more than a sum of its constituent parts. They know that myth and
story around a product creates value, opens the door to price premiums, and
drives customer loyalty.
But of all the CMO roles, the Chief Storyteller is arguably the farthest removed
from hard ROI and has the toughest case to make when describing contributions
to revenue. After all, how do you measure the ROI of a storyline? At best,
marketers can demonstrate a coincidental relationship between brilliant creative
strategy and movements in revenue.
And that’s the challenge. Chief Storytellers understand what drives revenue
on an intuitive level. And unfortunately for them, the business world has yet
to recognize intuition for the brilliant data algorithm it really is. It doesn’t help
that “storytelling” has become one of the most over-used, and therefore
least meaningful, terms in all of marketing. So we impose all kinds of other
mathematical equations on top of human intuition and creativity in a retroactive
measurement of its effectiveness.
But despite all the hype around storytelling, there is in fact a genuine and welldocumented methodology behind persuasive storytelling. Most of it is based
on the work of Joseph Campbell and his Hero’s Journey model. And thanks to
explanatory analytics, the science of corporate storytelling has evolved beyond
merely generating buzz and sentiment; it can now be used to motivate and
measure behavioral change.
Explanatory analytics opens up a new world of creative possibilities for
storytellers. Social media data, voice-of-customer (VOC) data, spending patterns.
and all kinds of external data points can be filtered and correlated to revenue.
What results is a treasure trove of topics, themes and associations that provide
before-the-fact inspiration for storytellers, not after-the-fact judgments. What’s
more, these findings can be used across a variety of storytelling methodologies,
all of which share a few common characteristics.
10
©2016 Quantifind, All rights reserved.
All too often, Storytellers try to
concoct some surprise element of
their own. But sometimes — not all
the time, but sometimes — organic
patterns bubble up in the data to
suggest new and unanticipated
creative strategies to pursue.
The second job of the storyteller is to connect on an emotional level. Again, the
trap here is to pay too much attention to sentiment analysis and be steered in the
wrong direction by the wrong audience. By correlating social data to revenue and
other KPIs, explanatory analytics gives you insights into which voices to listen to,
and which emotional hooks to leverage in your narrative.
Another critical component to any good story is a plot twist or an unexpected
discovery. This is perhaps where explanatory analytics provides the most distinct
value. All too often, Storytellers try to concoct some surprise element of their own.
But sometimes — not all the time, but sometimes — organic patterns bubble up in
the data to suggest new and unanticipated creative strategies to pursue. The data
itself can often provide the element of surprise many storytellers are looking for.
In other words, numbers can be used to shape narrative.
All kinds of data flows back and forth across the cerebral cortex when people
hear a story — it’s just being processed in a black-box organic operating system
that doesn’t translate to a CMO dashboard as neatly as Google Analytics.
Here’s an example. A movie studio was looking to promote a buddy cop movie
featuring a well-known comic actor and a lesser-known hunk. Naturally, the ads
focused on promoting the comedian. And naturally, they targeted guys. And as a
result, they focused on drug humor and the kind of lowbrow comedy that appeals
broadly to guys. But the movie was projected to open at levels far below what the
studio was hoping for.
Explanatory analytics revealed that the biggest opportunity to drive more people
into the theater wasn’t with guys, it was with females. And they were more
interested in the hunk than the funny man. What’s more, they were turned off
by the drug humor and instead were hoping to see another hunk make a muchanticipated cameo in the movie. So the studio changed the creative storytelling
in their movie trailers, shifted their focus to the female audience – and boom the
opening box office numbers jumped significantly.
That’s the power of explanatory analytics. It helps Chief Storytellers tell the stories
that change behavior. After all, by their very nature, Chief Storytellers are full of
stories. With explanatory analytics, their creative genius and informed intuition
can be targeted intelligently at the right audiences, incorporate the right story
elements, and inspire the right behavioral changes to lead to revenue growth.
11
©2016 Quantifind, All rights reserved.
HOW EX PLANATORY ANALYTICS ENABLES MAR KETING LEAD ER S H IP
Three Takeaways:
1
2
3
Understanding your strengths as a marketing executive can help
bring clarity to your day-to-day activities. The Deloitte Model is a
great place to begin the personal inventory.
Explanatory analytics is a great tool for unlocking the full potential
behind the kind of CMO role that helps you excel.
What’s more, learning to use explanatory analytics is a great way
for CMOs to set the strategic agenda with their CEO and become
the kind of CMO they most want to be — and the one that’s most
valuable to their organization.
Quantifind
Quantifind's mission is to empower people to make better decisions that
combine human intuition with the voice of intelligent data. Our on-demand
insights platform, Signum, uses explanatory analytics to help marketers explore,
understand, and change their impact on revenue. Quantifind’s clients are some
of the world’s most recognized brands, spanning industries such as automotive,
consumer packaged goods, entertainment, financial services, pharmaceutical,
retail, and telecom. Quantifind is proud to be backed by investors such as AME
Cloud Ventures, Andreessen Horowitz, Cathay Innovation, Comcast Ventures, Iris
Capital, Redpoint Ventures, and U.S. Venture Partners. Founded by two PhD's in
atomic physics, the company now has offices in Silicon Valley and New York, NY.
Quantifind | 8 Homewood Place, Menlo Park, California 94025 | 650.561.4937
General Email: [email protected] | Media & Press: [email protected]
www.quantifind.com
12
©2016 Quantifind, All rights reserved.