Download AIE - Hubbard Decision Research

Intro to
AIE
Risk/
Return
Analysis
Process
Applied Information
Economics: Kickoff for Risk
Return Analysis
Workshops
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Risk/
Return
Analysis
Process
What is AIE?
Applied Information Economics is the practical application of
scientific and mathematical methods to quantify the value of ITenabled business investments
Economics
Operations
Research
Workshops
Modern Portfolio
Theory
Decision/Game
Theory
Applied
Information
Economics
Information
Engineering
Software
Metrics
Decision Research
Hubbard
The Applied Information Economics Company
Empirical
Decision Theory
Statistics
Information Theory
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
Real Solutions to…
•
•
•
•
…the economics of information
…the economics of IT infrastructure
…the economics of risk
…the economics of labor reduction when
headcount is not reduced
• Bottom Line: AIE assesses and prioritizes IT
investments based on quantitative and
economically rational methods
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
What Do the Critics Say?
• “Quantifying the risk and comparing its risk/return with other
investments sets AIE apart from other methodologies. It can
substantially assist in financially justifying a project -especially projects that promise significant intangible
benefits.” The Gartner Group
• “AIE represents a rigorous, quantitative approach to
improving IT investment decision making…..this investment
will return multiples by enabling much better decision
making. Giga recommends that IT executives learn more
about AIE and begin to adopt its tools and methodologies,
especially for large IT projects.” Giga Information Group
• “AIE-like methods must become the standard way to make
(IT) investment decisions.” Forrester Research, Inc.
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Risk/
Return
Analysis
Process
Basic Risk/Return Analysis
Organization
Procedure
2. Clarify
Decision
Model
1. Describe &
Classify
Tools
Workshops
3. Make
Measurements
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
AIE
Deliverables
AIE
Decision Research
Hubbard
The Applied Information Economics Company
6. Make
Recommendations
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
Describe, Classify & Plan
1. Classify &
Plan
2. Clarify
Intangibles &
CBA
3. Conduct
Measurements
4. Conduct
VIA
5. Conduct
Risk & Return
Analysis
6. Make
Recommendations
• Purpose: Agree on the specific investment
to be analyzed, Agree on the specific
question to be answered, Plan the rest of
the analysis
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Clarify The Decision Model
Risk/
Return
Analysis
Process
Workshops
1. Describe &
Classify
2. Clarify
Decision
Model
• During
Clarification we
translate the
“Intangibles” into
measurable units
• These are
ultimately modeled
in a spreadsheet
Decision Research
Hubbard
The Applied Information Economics Company
3. Make
Measurements
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
6. Make
Recommendations
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
Understanding Measurement
(The Measurement.com approach)
• Gilb’s Law “Anything can be measured in
a way which is superior to not measuring it
at all”
• The perceived impossibility of
measurement is an illusion caused by not
understanding:
– the Concept of measurement
– the Object of measurement
– the Methods of measurement
• See my “Everything is Measurable” article
in CIO Magazine (got to “articles” link on
www.hubbardresearch.com
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
Conduct Measurements
1. Describe &
Classify
2. Clarify
Decision
Model
3. Make
Measurements
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
6. Make
Recommendations
• We use the variety of measurement methods
previously discussed
• We usually start with what we know now (i.e.
calibrated estimates)
• More elaborate measurements (large controlled
experiments or surveys) are only taken if we can
show they are economically justified
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Risk/
Return
Analysis
Process
Calibrated Estimates
•
•
Workshops
•
Measuring your own
uncertainty about a quantity
is a general skill that can be
taught with a measurable
improvement
Studies show that most
managers are statistically
“overconfident” when
assessing their own
uncertainty
Training can “calibrate”
people so that when they
provide a 90% confidence
interval, it still has a 90%
chance of being right (even
though it is subjective)
Decision Research
Hubbard
The Applied Information Economics Company
When asked to provide a subjective 90%
confidence interval, most managers provide
a range that only has about a 40%-50%
chance of being right
Perceived 90%
Confidence Interval
Actual 90%
Confidence Interval
Intro to
AIE
Calculate the Value of
Information
Risk/
Return
Analysis
Process
1. Describe &
Classify
•
•
Workshops
•
2. Clarify
Decision
Model
3. Make
Measurements
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
6. Make
Recommendations
The value of additional information can be calculated for each
uncertain variable in the analysis
Measurement efforts will be more productive by focusing on
variables that matter the most (results are often surprising)
This method is based on the probability of a change in a
decision with additional information and the difference in the
value of the decision
$
Decision Research
Hubbard
The Applied Information Economics Company
$$$
Intro to
AIE
The Economic Value of
Information
Risk/
Return
Analysis
Process
The Decision Theory Formula:
z
z
 z

EVI   p(ri ) max V1, j p( j | ri ),V2 , j p( j | ri ),... Vl , j p( j | ri ),  EV *
i 1
j 1
j 1
 j 1

k
Workshops
What it means:
 Information reduces uncertainty
 Reduced uncertainty improves decisions
 Improved decisions satisfy business objectives
(by definition)
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
Conduct Risk/Return Analysis
1. Describe &
Classify
2. Clarify
Decision
Model
3. Make
Measurements
4. Conduct
VIA
5. Conduct
Risk/Return
Analysis
Administrative Cost
Reduction
5%
10%
% Improvement in
Customer Retention
10%
20%
6. Make
Recommendations
15%
30%
Total Project Cost
$2 million
ROI
Decision Research
Hubbard
The Applied Information Economics Company
0%
$4 million
50%
$6 million
100%
Intro to
AIE
Risk/
Return
Analysis
Process
Make Recommendations
1. Describe &
Classify
2. Clarify
Decision
Model
3. Make
Measurements
4. Conduct
VIA
• The recommendations include:
Workshops
– To accept or reject the investment
– Possible modifications to the
proposed investment
– Various risk management tactics
• Deliverables include
– The written report
– The spreadsheet
– The presentation
Decision Research
Hubbard
The Applied Information Economics Company
5. Conduct
Risk/Return
Analysis
6. Make
Recommendations
Intro to
AIE
Risk/
Return
Analysis
Process
Overview of RRA Analysis
Classification
Value of Info.
Measurements
Intangibles
5%
“Customer Satisfaction”
“Strategic Alignment”
“Technology Risk”
“Information Quality” etc.
10%
$
15%
$$$
10%
20%
30%
$$
Measurables
Workshops
$2 mill
Errors in Decision X
Change to Strategic Measure M
Productivity in Activity Y
Chance of cancellation, etc.
Calculate Risk/Return Position
0%
Risk
$6 mill
Organization's
investment limit
"expected" ROI
Probability
of a negative
ROI
-50%
$4 mill
Probability of a
positive ROI
50%
100%
150%
Decision Research
Hubbard
The Applied Information Economics Company
200%
250%
Acceptable region of
investment
Return
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
Workshops
• Much of the initial data gathering is
from a series of workshops
• We need to schedule 5-6 workshops
for the following activities:
– Define & Classify the investment
– Clarify Decision Model
– Measurement (initial)
• Calibration
• Estimation
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
Defining the Investment
• What is the objective of this investment? (A onesentence description of why)
• What costs are unique to this investment?
• What benefits are unique to this investment?
• What are the risks of the investment?
• What “decision dimensions” are important besides
just an accept/reject recommendation?
• Is all of the investment optional?
• The decision is analyzed on behalf of which
investor?
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
The Concept Of Measurement
Risk/
Return
Analysis
Process
Workshops
• Sometimes one believes that a
thing is “immeasurable” only
because they do not actually
understand the concept of
measurement
• The “Measurement Theory”
definition of measurement: “A
measurement is an observation
that results in information
(reduction of uncertainty) about
a quantity.”
• Any “reduction of uncertainty”
about a quantity can be of value
Decision Research
Hubbard
The Applied Information Economics Company
?
Intro to
AIE
Risk/
Return
Analysis
Process
Real-world Measurements vs.
Ideal Values
Ideal Values: Point
Real-world Meas.
Normal Distribution
Uniform Distribution
Workshops
Lognormal Distribution
Hybrid
Threshold confidence
Decision Research
Hubbard
The Applied Information Economics Company
15%
85%
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
The Object of Measurement
• If a thing seems like and
immeasurable
“intangible” it may just
be ill-defined
• Often, if we can define
what we mean by a
certain “intangible” we
find ways to measure it
Decision Research
Hubbard
The Applied Information Economics Company
?
Intro to
AIE
The Clarification Chain
Risk/
Return
Analysis
Process
•
•
Workshops
•
•
AIE assumes that if a benefit or cost is
defined unambiguously, then it is
measurable.
If it is “Better” it is different in some
relevant way...
If it is relevantly different then it is
observable...
If it is observable then it is observable in
some amount...
If we can observe it in some amount
then it is measurable.
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
The “Thought Experiment”
• Imagine that you are a scientist
capable of making clones of entire
companies and that you have a
cloned pair of your company
• Change one of the companies so
that one has the stated
“intangible” and the other does
not
• Ask what would you actually
observe that would be different
between the two companies
Decision Research
Hubbard
The Applied Information Economics Company
Intro to
AIE
Risk/
Return
Analysis
Process
Workshops
Examples of Clarification
The “Intangible” Possible Meanings After Clarification
“Employee
Empowerment”
• Less management overhead
• Certain decisions are more accurate and faster
“Information
Availability”
• Time and cost of searching is reduced
• Certain costly errors are less frequent
“Customer
Relationship”
• Increased repeat business
•
•
Tools like “The Clarification Chain” are used to identify unit-ofmeasure variables hidden beneath the “intangible” label
I offer a challenge that given any intangible, I can clarify it and
identify a method of measurement within 15 minutes (I’ve never
lost)
Decision Research
Hubbard
The Applied Information Economics Company