Download Lempert_Blossom_Talk_new

New Methods to Bridge
Long-Term Policy Analysis
and Robust Decision Making
Robert Lempert
Director
RAND Frederick S. Pardee Center
for Longer Range Global Policy
and the Future Human Condition
BLOSSOM Workshop
European Environmental Agency
April 30, 2008
Long-Term Decisions
Present Difficult Challenges
• Long-term decisions occur
– When reflecting on potential events decades or more in the future
– Causes decision makers to choose near-term actions different
than those they would otherwise pursue
• Long-term policy analysis often fails to
persuade because
– Conditions of deep uncertainty prevail
– Short-term needs loom large
– Views of the future often anchored in the
present
2
4-30-08
Long-Term Decisions
Present Difficult Challenges
• Long-term decisions occur
– When reflecting on potential events decades or more in the future
– Causes decision makers to choose near-term actions different
than those they would otherwise pursue
• Long-term policy analysis often fails to
persuade because
– Conditions of deep uncertainty prevail
– Short-term needs loom large
– Views of the future often anchored in the
present
“Missions to Mars” at
Disneyland’s Tomorrowland ca 1955
3
4-30-08
Scenarios Attractive for Long-Term Analysis, but
Have Weakness in Contentious Public Debates
• At their best, scenarios can help decision makers
– Reduce overconfidence
– Challenge their mental models
– Overcome organizational and psychological barriers to
considering threatening or inconvenient futures
4
4-30-08
Scenarios Capture the Key Concept
That a Multiplicity of Plausible Futures
May Be as Close as We Get to the Truth
Rabbi Eliezer Ashkenazi (1580) chose to interpret the
Tower of Babel story not as a challenge to divine power to
which the Lord's response was to divide the human race
but rather the opposite.
He saw the story as an attempt to establish a universal
religious regime which God "was obliged to separate
…since the proliferation of doctrines aids and stimulates
the investigator to attain the desired truths.”
5
4-30-08
Scenarios Attractive for Long-Term Analysis, but
Have Weakness in Contentious Public Debates
• At their best, scenarios can help decision makers
– Reduce overconfidence
– Challenge their mental models
– Overcome organizational and psychological barriers to
considering threatening or inconvenient futures
• But in contentious public debates, scenario
methods can have difficulty
– Engaging mental models of diverse stakeholders
– Systematically informing decisions under uncertainty
– Addressing surprise and discontinuities
6
4-30-08
Outline
• Robust decision making (RDM) provides
framework for effective long-term analysis
• RDM’s “Scenario Discovery” approach offers
useful scenario concept for public debates
• Recent work measures the impacts of these
approaches with decision makers
7
4-30-08
RDM Views Scenarios As Part of Process Identifying
and Building Consensus for Robust Strategies
Key Robust Decision Making Concepts:
• Construct ensemble of long-term scenarios that
highlight key tradeoffs among near-term policy choices
• Consider near-term choices as one step in a sequence
of decisions that evolve over time
• Use robustness criteria to compare alternative
strategies
– A robust strategy performs well compared to the
alternatives over a wide range of plausible futures
8
4-30-08
New Technology Allows Computer to Serve As
“Prosthesis for the Imagination”
• Robust Decision Making (RDM) is a quantitative decision analytic
approach that
– Characterizes uncertainty with multiple, rather than single, views of the future
– Evaluates alternative decision options with a robustness, rather than
optimality, criterion
9
4-30-08
New Technology Allows Computer to Serve As
“Prosthesis for the Imagination”
• Robust Decision Making (RDM) is a quantitative decision analytic
approach that
– Characterizes uncertainty with multiple, rather than single, views of the future
– Evaluates alternative decision options with a robustness, rather than
optimality, criterion
– Iteratively identifies vulnerabilities of plans and evaluates potential responses
Candidate
strategy
Identify
vulnerabilities
Assess alternatives
for ameliorating
vulnerabilities
• RDM combines key advantages of scenario planning and quantitative
decision analysis in ways that
– Decision makers find credible
– Contribute usefully to contentious debates
10
4-30-08
Stylized Sustainability Example
Summarizes RDM Approach
What near-term actions can help ensure economic growth and
environmental quality over the 21st century?
Analysis suggests testing alternative strategies over this range of futures
Decoupling
rate (%)
5.0
China
since
1960
4.0
(Rate at which 3.0
technology,
without
2.0
regulation,
reduces
1.0
emissions per
GDP)
0
U.S.
since 1950
Russia
since
1993
U.S.
in 20th century
Brazil since 1980
U.S. 1890-1930
–1.0
India since 1960
0
1.0
2.0
3.0
Economic growth rate (%)
4.0
11
4-30-08
Compare “Fixed” Near-Term Strategies
Across Scenarios
Assume near-term policy continues until
changed by future generations
Near term
Future
Choose policies
Future decisionmakers recognize
and correct our
mistakes
12
RAND MR-1626-RPC
4-30-08
Initial Scan Suggests No Fixed
Emission Reduction Target Is Robust
Decoupling
rate (%)
Stay the Course
1.
5.0
2.
4.0
Conventional
world
U.S.
scenario
since 1950
3.0
2.0
U.S.
in 20th century
1.0
Run simulation thousands
of times
Display “scenario maps”
showing deviation of
proposed strategy from
optimality over many
futures
Strategy
0
Vulnerabilities
Alternatives
–1.0
0
1.0
2.0
3.0
4.0
Economic growth rate (%)
Strategy’s
Performance
No regret
Mild
A lot
Overwhelming
13
4-30-08
Initial Scan Suggests No Fixed
Emission Reduction Target Is Robust
Decoupling
rate (%)
Stay the Course
Crash Effort
5.0
5.0
4.0
4.0
Conventional
world
U.S.
scenario
since 1950
3.0
Conventional
world
scenario
3.0
2.0
U.S. since 1950
2.0
U.S.
in 20th century
1.0
0
0
–1.0
–1.0
0
1.0
2.0
3.0
U.S. in 20th
century
1.0
4.0
0
1.0
2.0
3.0
4.0
Economic growth rate (%)
Strategy’s
Performance
No regret
Mild
A lot
Overwhelming
14
4-30-08
Craft Near-Term Adaptive Strategy That Aims to
Balance Environmental and Economic Goals
Present
Future
Select near-term
milestone
Does the carrying
capacity change?
Determine best policy
to meet milestone
NO
YES
Implement policy
Is milestone
achievable with
current approach? YES
Choose policies to
maximize utility
NO
Relax milestone
15
RAND MR-1626-RPC
4-30-08
Robust Strategy Reduces Uncertainty By Performing
Well No Matter What Future Comes to Pass
Adaptive Strategy
Decoupling
rate (%)
5.0
4.0
Strategy
3.0
Vulnerabilities
Alternatives
U.S. since 1950
2.0
1.0 U.S. in 19th
century
0
U.S. in 20th
century
–1.0
0
1.0
2.0
3.0
4.0
Economic growth rate (%)
No regret
Mild
A lot
Overwhelming
16
4-30-08
Outline
• Robust decision making (RDM) provides
framework for effective long-term analysis
• RDM’s “Scenario Discovery” approach offers
useful scenario concept for public debates
• Recent work measures the impacts of these
approaches with decision makers
17
4-30-08
Long-Term RDM Scenarios Highlight
Trade-offs Among Near-Term Decisions
1. Run simulation model for many different combinations of
uncertain input parameters
2. Identify those clusters of cases that highlight tradeoffs
among near-term candidate strategies
Candidate
strategy
Identify
vulnerabilities
Assess alternatives
for ameliorating
vulnerabilities
• Example future conditions highlighting near-term tradeoffs:
– 2007 Congressional Reauthorization of Terrorism Risk
Insurance Act: In what situations would ending TRIA cost
the taxpayer more than retaining the program?
– California water planning: Under what conditions would
future climate change impacts suggest modifying current
long-range water management plans?
4-30-08
18
Scenario Discovery Implements This Concept for
Computer-Assisted Scenario Development
1. Indicate policy-relevant cases in
database of simulation results
.
.
.
.
.
.
. . .. .
19
4-30-08
Scenario Discovery Implements This Concept for
Computer-Assisted Scenario Development
1. Indicate policy-relevant cases in
database of simulation results
2. Statistical analysis finds lowdimensional clusters with high
density of these cases
Uncertain
input
variable 2
.
.
.
.
.
.
. . .. .
Uncertain input variable 1
20
4-30-08
Scenario Discovery Implements This Concept for
Computer-Assisted Scenario Development
1. Indicate policy-relevant cases in
database of simulation results
2. Statistical analysis finds lowdimensional clusters with high
density of these cases
Uncertain
input
variable 2
.
.
.
.
.
.
. . .. .
Uncertain input variable 1
3. Clusters represent scenarios and
driving forces of interest to
decision makers
21
4-30-08
Scenario Discovery Implements This Concept for
Computer-Assisted Scenario Development
Approach provides measures
of merit for scenario quality
Density:
•
How many cases inside the
scenario are policy-relevant?
(e.g. 75%)
Coverage:
•
How many of all the policyrelevant cases do the
scenarios include? (e.g. 82%)
1. Indicate policy-relevant cases in
database of simulation results
2. Statistical analysis finds lowdimensional clusters with high
density of these cases
Uncertain
input
variable 2
.
.
.
.
.
.
. . .. .
Interpretability:
•
Is the number of scenarios
and driving forces sufficiently
small to understand? (e.g. 1
scenario with two driving
forces)
Uncertain input variable 1
3. Clusters represent scenarios and
driving forces of interest to
decision makers
22
4-30-08
Scenario Discovery May Improve Impact of
Scenarios in Contentious Public Debates
• For instance, recent scenario discovery work on
U.S. Federal terrorism insurance program was
– Based on a scenario not considered in the official
budgetary analysis by government agencies
– Quoted on the floor of the United States Senate by a
program supporter
– Criticized as “insidious” by program opponents
• But neither side in the debate could gain
traction by quarrelling with our choice of
scenario and its key driving forces
23
4-30-08
Outline
• Robust decision making (RDM) provides
framework for effective long-term analysis
• RDM’s “Scenario Discovery” approach offers
useful scenario concept for public debates
• Recent work measures the impacts of these
approaches with decision makers
24
4-30-08
How Does Climate Change Affect California’s
Inland Empire Utilities Agency (IEUA)?
IEUA currently serves 800,000
people
– May add 300,000 by 2025
Water presents a significant
challenge
25
4-30-08
How Does Climate Change Affect California’s
Inland Empire Utilities Agency (IEUA)?
IEUA currently serves 800,000
people
– May add 300,000 by 2025
Water presents a significant
challenge
IEUA’s 2005 long-range Urban Water
Management Plan (UWMP)
– Aims to meet needs of growing
population, but
– Did not address climate change
26
4-30-08
How Does Climate Change Affect California’s
Inland Empire Utilities Agency (IEUA)?
IEUA currently serves 800,000
people
– May add 300,000 by 2025
Water presents a significant
challenge
We conducted several analyses to
help IEUA assess impact of climate
change on their 2005 UWMP
– Traditional scenarios
– Probabilistic risk analysis
– Scenario Discovery
27
4-30-08
Conducted Workshops to Measure Impact
of Alternative Analyses on IEUA
– Four IEUA workshops presented modeling results to
participants including:
• Agency professional managers and technical staff
• Local elected officials
• Community stakeholders
– “Real-time” surveys measured participants’
• Understanding of concepts
• Willingness to adjust policy choices based on
information presented
• Views on RDM, traditional scenarios, and probabilistic
risk analysis
28
4-30-08
Participants Ranked Scenario Discovery More
Useful, But More Difficult to Understand
Questionnaire item from
first 3 workshops
Traditional
Scenarios
Scenario
Discovery
Provides results that can
be used in planning
Agree
somewhat
Agree
strongly
Provides information on
how to improve plan
Agree
somewhat
Agree
somewhat
Is easy to explain to
decisionmakers
Agree
somewhat
Disagree
strongly
– Traditional scenarios
• Gave IEUA much of the information they needed
• Emphasized the importance of achieving goals in IEUA’s plan
– Scenario discovery
• Provided more useful information for evaluating alternatives to plan
• Sparked discussion of adaptive strategies
29
4-30-08
Surveys Suggest RDM Analysis
Changed Participants’ Views
Participants provided:
– Information on most effective RDM visualizations
After the workshop:
– 35% said consequences of bad climate change now appeared
“more serious” than before
– 75% though the ability of IEUA planner to plan for and manage
effects was “greater” than before
Overall, analysis:
– Increased support for near-term modifications to current IEUA plan
– Suggests that participants’ willingness to acknowledge a serious
climate change threat increased after they felt more confident they
could address the threat
30
4-30-08
Key Concepts
Choose scenarios to highlight tradeoffs among
near-term decisions
Otherwise number of potentially interesting scenarios remains
unlimited
Use analytics to facilitate human creativity in
designing policies robust across many futures
Measure scenarios’ impacts on decision makers
to help improve process and methods
Designing measurements makes purpose clear
Can use framework for general thinking about
long-term policy under deep uncertainty
Not just as basis for a modeling exercise
31
4-30-08
For More Information
http://www.rand.org/international_programs/pardee/
Thank you!
32
4-30-08