Download Values-informed Mental Models: A New Tool for

Values-informed Mental Models: A New Tool for Climate Risk Management
Douglas L. Bessette a*, Bryan Cwik b, Lauren A. Mayer c, Klaus Keller a,d,e, Robert
Lempert c, and Nancy Tuana f,g
a
Earth and Environmental Systems Institute, Penn State University
b
Department of Philosophy, Indiana University of Pennsylvania
c
RAND Corporation
d
Department of Geosciences, Penn State University
e
Department of Engineering and Public Policy, Carnegie Mellon University
f
Department of Philosophy, Penn State University
g
Rock Ethics Institute, Penn State University
* Corresponding Author: Email: [email protected]; 2217F Earth and Engineering Science,
The Pennsylvania State University, University Park, PA, USA, 16802-6813 Phone: 814865-9912
ABSTRACT.
Climate risk management (CRM) is both critically important and extremely complicated.
Not only do climate change risks manifest over long time periods and across vast spatial
scales, but managing them requires making decisions under conditions of deep
uncertainty. Deep uncertainty occurs, for example, when decision makers do not agree or
do not know the probabilities associated with future states of the word. In addition to
these analytical difficulties, scientists and risk analysts must account for the wide range
of conflicting values present when modeling climate change risk, developing CRM
strategies, and incorporating stakeholders. These analytic and value complexities pose
challenges to current risk communication methods like Mental Models, as they assume
that experts agree and that individuals’ values are stable across risk objects and decisions.
Because both experts and stakeholders rely on values to inform their understanding of
deeply uncertain risks, and because individuals themselves deploy their values differently
across CRM decisions, mental models should also map values. As such, this paper
presents “Values-informed Mental Models” (ViMM) as a means of improved CRM. It
also presents results from three case studies in which ViMM have been successfully
deployed.
2
I. INTRODUCTION
Designing effective climate risk management (CRM) strategies is neither a simple nor
straightforward undertaking. Climate change and its impacts manifest over long time
periods [2] and across vast spatial scales [3]. They are difficult to translate across these
scales, [4] and may show subtle yet abrupt transitions and hysteresis responses [5]. How
these impacts will affect human society depends a great deal on multi-decadal to century
scale projections regarding changes in human culture, places and social values [9]. As a
result, CRM requires decisionmaking that accounts for deep uncertainty and is especially
sensitive to assumptions, priors and parameters [11].
In addition scientists and decision analysts must account for a wide range of ethical
values (e.g., equality, distributive and intergenerational justice) and epistemic values (e.g.,
simplicity and completeness) in CRM. These values compete both within and between
all stages of CRM, from the modeling of climate change and its impacts [14-16], to
selecting portfolios of management strategies such as adaptation, mitigation or
geoengineering, and evaluating their effectiveness [17, 18], to meaningfully incorporating
stakeholders into the decision-making process [19]. Research in the decision sciences
shows that individuals often struggle to make trade-offs between their own competing
values (and the values of others) [20, 21], and individuals may deploy their own values
differently during decisions [19]. Such factors are likely magnified when the values in
question are considered protected [22] or sacred [23].
3
These analytical and value complexities combine to make CRM especially complicated.
Both analysts and the public must lean on values to inform or even supplant factual
knowledge [24]. Whereas more straightforward and better understood risks may rely on
a practical separation of facts and values, no clear separation can be assumed with respect
to climate change risk [25, 26]. Instead values are endogenous to, or coupled with,
individuals’ understanding of climate change and its specific elements [27, 28]. These
values deeply affect the process by which people "model the world,” and thus individuals’
mental models cannot be considered "value neutral.” Values are not exogenous factors
affecting how individuals rank outcomes of actions, but instead affect the way in which
individuals model decision situations. Additionally, if individuals indeed deploy their
own values differently throughout a decision [19], then eliciting values separately from
individuals’ knowledge, objectives or decisions can lead to significant gaps. For example,
both experts and stakeholders may define certain climate change impacts or risks
differently depending on how they value the specific outcomes at stake [24] (see Pidgeon
& Fischhoff for a relevant example and insightful discussion [20]).
As a result, common risk communication methods like decision-analysis based mental
models [29], which often assume expert consensus and a consistent deployment of shared
values and objectives [30], may struggle to inform CRM. Mental models excel in
providing expert instructions, presenting quantitative summaries, and/or framing risks
[31]. However, they may suffer, respectively, when there is little agreement about what
to do, little agreement about the probabilities of hazards, consequences and future states
of the world, and no motivated audience sharing similar values and objectives [32-34].
4
The task of CRM (just like the management of nanomaterials, transgenic entities, or
nuclear waste disposal), is characterized by these deep uncertainties and value tradeoffs
[35]. CRM cannot simply educate the populace or correct misperceptions, as the grounds
for doing both assume a “correct” initial condition and a prevalence of facts [36]. Instead
of individuals’ risk perceptions being corrected (e.g. [37, 38]), climate change risk must
be understood, which requires recognition of differences in risk tolerances and risk
aversion. This in turn requires a dialogue between stakeholders and experts [39], one that
acknowledges and emphasizes the role that values play in both experts and stakeholders’
risk assessments, perceptions and behavior.
The method of Value Informed Mental Models, or ViMM, is thus based on two
assumptions: a) experts and stakeholders rely heavily on values to inform their
understanding of CRM [24], and b) those values are deployed differently by an individual
across risk objects, states of the world and decisions [19]. ViMM expand on decisionanalytic mental models to allow for a mapping of individuals’ values onto these objects
and their patterns of influence. As such, ViMM go beyond simply acknowledging that
individuals’ internal definitions of risk, or perceptions, influence their external, or
analytical, assessments and instead endeavor to make these influences explicit [24, 25].
In addition to mapping values within experts and stakeholders’ mental models, ViMM
also work to expand and improve elicitation of CRM-relevant values by focusing on
epistemic and ethical values. These are in addition to the moral values and decision
5
objectives emphasized by value-focused approaches [40] and multi-attribute methods like
structured decision-making [41]. ViMM provide a rigorous lens for eliciting and
defining these additional values via open-ended interviews created and conducted by an
interdisciplinary team of subject-matter experts. Given the interactions between
epistemic and ethical questions, this team often requires inputs from scientists as well as
applied philosophers, the latter of whom can assist in both identifying and coding for
values and moral judgments [42]. This interdisciplinary focus leads to the development
of a coupled scientific-ethical research agenda [28], or coupled ethical-epistemic analysis.
Such analysis speaks to recent calls urging greater contributions to climate change
research and communication by the social sciences and humanities [10, 43, 44].
This interdisciplinary analysis also helps to identify critical gaps in the ethical systems
relied upon by experts and stakeholders. For instance, our ViMM work has already
shown that climate scientists often represent a consequentialist ethical system in their
models, while many stakeholders rely on a system of virtue ethics to make decisions or
assess tradeoffs. The former system, which aggregates impacts, may rely upon
assumptions and generate results that are opposed to the latter, in which individuals place
a high value on human life and demand some degree of distributive justice.
Beyond identifying ethical systems, epistemic values such as simplicity and completeness
infuse CRM scientists’ assumptions and decisions. These decisions likely have
significant ethical implications, yet their coupling is rarely analyzed as it is in ViMM.
6
Examining this coupling and these values qualitatively, and ensuring that CRM provides
similarly qualitative representations of impacts and outcomes is also a key goal of ViMM.
ViMM, like coupled ethical-epistemic analysis more broadly, focus especially on
improving the relevance of CRM [28]. CRM often relies on system models which
examine strategies and outcomes using a set of probability distributions over uncertain
input parameters [45]. These models can be unfamiliar to stakeholders as well as to
researchers outside climate science [20]. ViMM also incorporate an iterative structure
for testing, evaluating and deploying CRM decision support based on these simulation
models. ViMM expand on the third step of Morgan’s et al [31] Mental Model Approach
(MMA), i.e., large and representative confirmatory surveys to validate mental model
beliefs, to include pilot-testing of a decision support tool or its elements. These pilot tests
also provide critically for more dialogue between experts and stakeholders [39]. Due to
the deep uncertainty and value flexibility of CRM, ViMM focuses less on correcting
factual misperceptions and much more on recognizing and shrinking the gaps between
scientists, analysts and stakeholders’ values and perceptions.
In the next section, we describe how ViMM augment MMA and delineate how ViMM
can lead to more values-informed and values-relevant CRM. We provide short
illustrative examples from a case study in which local climate change risk perceptions
and CRM strategies for the city of New Orleans are currently being investigated using
ViMM. In the final section, we discuss this project in more detail, along with two other
NSF-sponsored case studies using ViMM.
7
II. ViMM Operationalized
The ViMM method expands upon the MMA’s five steps, elucidated by Morgan et al [31],
or four according to Bruine de Bruin and Bostrom [30], to allow for the development of
simulation-based CRM decision support. To address the deep uncertainty associated with
climate change impacts, ViMM were originally purposed to support robust-decision
making (RDM), an increasingly used tool in CRM [45] that uses computational, multiscenario simulation modeling to identify strategies that perform well across a range of
assumptions and objectives.
ViMM consist of five steps (Figure 1). We pay considerably more attention here to the
first three steps as they focus on constructing the expert and stakeholder ViMM and the
analysis of their gaps. The latter two steps focus on ViMM’s role in the development of
CRM decision support, the details of which are outside the purview of this paper.
8
Figure 1. ViMM Steps: Numbers represent the chronological order of steps; arrows represent
interactions and iterations. Step 2 typically follows and is influenced by Step 1, but can be
constructed independently to inform decision support (represented by dotted arrow)
ViMM begin with the construction of the expert model. This step involves mapping both
experts’ beliefs and values with respect to different risks (e.g. storm surge) and
management strategies (e.g., investing in structures, resilience, etc.), their specific
components and causal mechanisms, how they can be manipulated, and the outcomes of
different processes. The confluence of facts and values with respect to CRM requires
elicitation of those values that mediate specific beliefs. As opposed to identifying values
a posteriori [31] in the assessment of outcomes or providing an accounting of all relevant
values and tradeoffs in a single box in a diagram (e.g., [46]), ViMM incorporate values
9
individually as a third type of node in an influence diagram. This is in addition to the two
traditional nodes representing states of the world (or uncertainties) and choices made by
decision-makers; the latter are represented as rectangles and ovals, respectively; values
are represented as diamonds; see Figure 2.)
Figure 2. ViMM Nodes. Boxes represent states of the world or uncertainties; ovals represent
decisions or actions; and diamonds represent the particular values emphasized when the former
interacts with the latter, and vice-versa.
Similar to the MMA, construction of the Expert ViMM can rely on literature analyses to
either a) construct a map of experts’ beliefs and values, or preferably, as these documents
often ignore or obscure values, b) inform a series of more value-focused semi-structured,
open-ended interviews to be conducted with individual experts. For our study of New
Orleans CRM, we began our expert ViMM with a content analysis of the State of
Louisiana’s Comprehensive Master Plan for a Sustainable Coast (2012), followed by
phone interviews with regional climate change experts. These interviews take
approximately one to two hours and can vary between exploratory, when very little is
known about experts’ beliefs and values, and directed, when much more is known about
both.
10
The interviews are transcribed and individual states of the worlds, decisions and values
(see above) are coded for, or categorized for easier analysis, by multiple raters. Special
attention is paid to the influence of values on decisions and interpretations of states of the
world and uncertainties, evidenced by the frequency of mentions and co-occurrences of
values with the latter two nodes.
Step 2 involves construction of the Stakeholder ViMM. This mapping relies on
stakeholder interviews, semi-structured open-ended interviews similar to, and preferably
informed by, the expert ViMM, as well as by literature analyses. The latter are currently
less helpful in CRM due to the very thin (but growing) literature on local or regional
adaptation and values. Interviews are transcribed and nodes are coded again with special
attention paid to the values that co-occur with specific risks, states of the world and
decisions.
The next step, Step 3, analyzes the Expert and Stakeholder ViMM to identify gaps in both
experts and stakeholders’ knowledge and values. Here again, the focus is on identifying
unique values or value systems related to specific risks, states of the world and decision
nodes or their influence. An example of two opposing value systems, the first, a
consequentialist system often used in models to represent aggregate impacts, costs and
benefits, and the second, a virtue ethics system, which more accurately describes how
stakeholders view the world, was introduced in the previous section. Similar to other
value-focused approaches like SDM, this gap analysis provides an accounting of the
relevant values. However unique from SDM, ViMM focuses on the (possibly) diverse
11
representations of those values by stakeholders and experts. Deliberation (with or
between each party) can then be focused on representing those values in a way that
acknowledges both parties’ concerns. This does not simply mean incorporating specific
values or objectives as focal points of CRM or as metrics to be used in decision support.
Instead, Step 3 requires evaluating the often-epistemic assumptions incorporated into
CRM, its strategies, and its decision-support tools. For instance, climate scientists are
often encouraged to enhance the skill and completeness of their models – in other words
to “get the science right.” They also often incorporate complex visualization techniques.
Yet the stakeholders we interviewed from New Orleans desired relatively simple
representations of uncertainty and CRM strategy recommendations. These individuals
deemphasized empirical accuracy and stressed the importance of receiving the “right
science.”
This type of analysis informs an initial selection of the elements that can then be tested in
Step #4: Iterative Development of Decision Support. This step incorporates scientists,
analysts, applied philosophers and stakeholders in the iterative development and
evaluation of the data input, materials and methods to be used (e.g., facilitated versus
online decision support), performance metrics, representations of initial conditions (e.g.,
parameters chosen), and the type of information and visualizations presented to
stakeholders, etc. Special attention is paid to ensuring (via pilot-tests) that the decisionsupport elements and process used by scientists and analysts represent stakeholders’
values and concerns accurately.
12
Once scientists and stakeholders approve these elements, the latter are incorporated into
the CRM decision-support process (Step #5), often along with far more complicated
simulation modeling (in our case a multi-objective RDM model). Evaluation of the
process by stakeholders and analysts continues throughout its deployment and data
collection.
III. CASE STUDIES
We deployed ViMM Steps 1 through 3 across three NSF-sponsored projects, each project
helpful in illustrating ViMM’s ability to improve CRM. The first project was an in-depth
study of scientists’ decision-making under uncertainty and their conception of tradeoffs
between climate change mitigation, adaptation and geo-engineering strategies. We
conducted interviews with eleven scientists from across the U.S. to determine the ethical
and epistemic values influencing their CRM decisions. These interviews led to the
development of an expert ViMM (Step 1) that showed scientists deploy different values
differently, i.e. an individual’s values are often emphasized or deemphasized at different
stages of CRM and model development.
For instance, scientists appear to apply distributive and intergenerational justice values
and value predictive power when defining their objectives and outcomes, or during the
“model structuring phase.” However, when interpreting and evaluating judgments, or
during the “model evaluation phase,” scientists emphasize the value of “democratized”
decision-making, or ensuring key constituencies have a voice. They also seek
consistency and explanatory power. These results suggest that mapping values alongside
13
states of the world and choices is key to understanding scientists’ decision-making under
uncertainty.
The second project examined climate resiliency amongst business executives,
sustainability officers, health professionals and other public and private interests of the
West Michigan Sustainable Business Forum at the Michigan Climate Resiliency
Conference in October of 2014. ViMM interviews were conducted with key stakeholders
and used to develop a decision-support instrument intended to facilitate cross-disciplinary
dialogue [47]. This was the first use of the stakeholder ViMM method (Step 2).
Results of this ViMM-based dialogue demonstrate shifts in participants’ responses to a
post-workshop survey targeting difficult value tradeoffs amidst CRM decisions. For
instance, participants acknowledged a greater need for emergency managers to take
account of uncertainties regarding climate change and advised that storm water plans
should acknowledge and slow the redistribution of West Michigan flood risk to neighbors.
Whereas these two studies focused on individual steps of the ViMM process, Steps 1 and
2, respectively, the third example project is intended to deploy all five steps on the way to
generating a full values-informed RDM-based decision-support tool. Specifically, this
project, centered in New Orleans, examines expert and stakeholder climate change risk
perceptions, beliefs and values. It also identifies CRM strategies that are robust across
both individuals’ values and future possible states of the world.
14
This project began with an expert model constructed as described in Section II above
(Figure 3 shows a portion of the New Orleans Expert ViMM). This was followed by 20
one-hour phone and in-person interviews with stakeholders from across New Orleans and
the Southern Louisiana coast.
Figure 3. Portion of Expert ViMM: Non-structural protection & storm surge. Rectangles
represent states of the world (uncertainties); ovals represent decisions; diamonds represent values.
The values Equity and Culture & Tradition are emphasized at different points in the decisionmaking process. Equity represents concerns that the distribution of costs and benefits of strategies
is equitable; this value is stressed (i.e., it co-occurs) when experts discuss the elevation of homes
(both of the land and the home) and the costs of raising them. The latter value, Culture &
Tradition, represents the importance of community-supporting values and institutions and cultural
practices linking generations; this value is emphasized when city government officials and
tourism boards assess non-structural protection proposals.
We conducted a gap analysis of the expert and stakeholder interviews to determine how
the groups deploy specific values differently across climate change impacts and CRM
strategies. For instance, qualitative analysis showed that stakeholders (as opposed to
experts) might map Individual Welfare, or the effect of CRM strategies on the enjoyment
and possession of important goods (like health, income, political stability), not Equity
(see Figure 3) onto specific CRM strategies such as elevating homes (i.e., non-structural
protection):
15
“If I elevated my house six-ten feet in the air, how shall I get to my front door, you know, if I’m
not able to afford the elevator, you know, or if I’m not able to afford the lift, you know. And if I
am able to afford the elevator or the lift to get to my front door because I can’t walk stairs because
I’ve got both of my knees replaced, what happens if that lift or that elevator is broken at the time?”
-Participant X
…while mapping Fairness concerns, or concerns that the distribution of costs and
benefits does not place an undue burden on certain portions of the population (such as the
disadvantaged), onto building levees:
“it’s the only solution that we have given the fact that we continue to mismanage the Mississippi
River. The decisions at which that is made are not decisions in which the population of Louisiana
has had any say about. So the people of Louisiana in New Orleans and in these small communities
are being held accountable to a risk that they did not cause.” -Participant Y
These value distinctions are important to recognize when experts begin to represent each
impact, or provide metrics, during communication or decision support. We are currently
developing and testing decision-support elements for New Orleans (Step 4) based on
these gap-analysis results, and a full CRM-RDM-based decision-support tool is
scheduled for facilitated deployment in the spring of 2016.
IV. CONCLUSIONS
Value-relevant CRM and decision support takes time and resources. It requires more
than just describing individuals’ knowledge of a system or risk and eliciting their
preferences a posteriori. It requires identifying key distinctions in the way individuals
encounter uncertainty and deploy values within their mental model. This type of
interdisciplinary, coupled ethical-epistemic analysis requires sustained and iterative
engagement with both experts and stakeholders that can often require months or even
years [48]. This effort is necessary however to manage deeply uncertain risks both
ethically and effectively.
16
AUTHOR CONTRIBUTIONS
B.C., L.M., K.K., R.L. & N.T. devised the ViMM methodology. B.C., L.M., and N.T.
initiated the ViMM work and analyzed the results from the Grand Rapids and Scientist
Decision-making studies. D.L.B., B.C, L.M., K.K., R.L., & N.T. initiated the New
Orleans ViMM work. D.L.B. conducted the New Orleans ViMM work and D.L.B. and
B.C analyzed the data. D.L.B., L.M., K.K., R.L. & N.T. are currently developing New
Orleans CRM decision support. D.L.B. is lead author of this manuscript; however, all
co-authors provided comments and revisions and approve of the manuscript and its
contents.
ACKNOWLEDGEMENTS
The authors wish to thank Chad Gonnerman and Katie Loa for their work on the Grand
Rapids and Scientist Decision-making ViMM projects, respectively. The authors also
would like to thank the West Michigan Sustainable Business Forum, as well as Gary
Cecchine and the RAND Gulf States Policy Institute in New Orleans. This work was
partially supported by the National Science Foundation through the Network for
Sustainable Climate Risk Management (SCRiM) under NSF cooperative agreement
GEO-1240507 as well as the Penn State Center for Climate Risk Management. We thank
Greg Garner and Chris Forest for insightful discussions. All opinions are the authors’
own.
17
Sources
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
Solomon, S., Climate change 2007-the physical science basis: Working group I
contribution to the fourth assessment report of the IPCC. Vol. 4. 2007:
Cambridge University Press.
Solomon, S., et al., Persistence of climate changes due to a range of greenhouse
gases. Proceedings of the National Academy of Sciences, 2010. 107(43): p.
18354-18359.
Adger, W.N., N.W. Arnell, and E.L. Tompkins, Successful adaptation to climate
change across scales. Global environmental change, 2005. 15(2): p. 77-86.
Wilby, R.L. and S. Dessai, Robust adaptation to climate change. Weather, 2010.
65(7): p. 180-185.
Keller, K., G. Yohe, and M. Schlesinger, Managing the risks of climate
thresholds: uncertainties and information needs. Climatic Change, 2008. 91(1): p.
5-10.
Lempert, R.J. and M.E. Schlesinger, Robust strategies for abating climate change.
Climatic Change, 2000. 45(3): p. 387-401.
Katzav, J. and W.S. Parker, The future of climate modeling. Climatic Change,
2015: p. 1-13.
Dessai, S., et al., Do we need better predictions to adapt to a changing climate?
Eos, Transactions American Geophysical Union, 2009. 90(13): p. 111-112.
Adger, W.N., et al., Are there social limits to adaptation to climate change?
Climatic change, 2009. 93(3-4): p. 335-354.
Hulme, M., Cosmopolitan Climates Hybridity, Foresight and Meaning. Theory,
Culture & Society, 2010. 27(2-3): p. 267-276.
Lempert, R.J., A New Decision Sciences for Complex Systems. Proceedings of the
National Academy of Sciences of the United States of America, 2002: p. 73097313.
Ackerman, F., et al., Limitations of integrated assessment models of climate
change. Climatic change, 2009. 95(3-4): p. 297-315.
Stern, N., The structure of economic modeling of the potential impacts of climate
change: grafting gross underestimation of risk onto already narrow science
models. Journal of Economic Literature, 2013. 51(3): p. 838-859.
Schneider, S.H., Integrated assessment modeling of global climate change:
Transparent rational tool for policy making or opaque screen hiding value-­‐‑laden
assumptions? Environmental Modeling & Assessment, 1997. 2(4): p. 229-249.
Weitzman, M.L., On modeling and interpreting the economics of catastrophic
climate change. The Review of Economics and Statistics, 2009. 91(1): p. 1-19.
Wilkerson, J.T., et al., Comparison of integrated assessment models: Carbon
price impacts on US energy. Energy Policy, 2015. 76: p. 18-31.
Drouet, L., V. Bosetti, and M. Tavoni, Selection of climate policies under the
uncertainties in the Fifth Assessment Report of the IPCC. Nature Climate Change,
2015.
18
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
Dasgupta, P., Discounting climate change. Journal of risk and uncertainty, 2008.
37(2-3): p. 141-169.
Dietz, T., Bringing values and deliberation to science communication.
Proceedings of the National Academy of Sciences, 2013. 110(Supplement 3): p.
14081-14087.
Pidgeon, N. and B. Fischhoff, The role of social and decision sciences in
communicating uncertain climate risks. Nature Clim. Change, 2011. 1(1): p. 3541.
Slovic, P., The feeling of risk: New perspectives on risk perception. 2010:
Routledge.
Baron, J. and S. Leshner, How serious are expressions of protected values?
Journal of Experimental Psychology: Applied, 2000. 6(3): p. 183.
Tetlock, P.E., Thinking the unthinkable: Sacred values and taboo cognitions.
Trends in cognitive sciences, 2003. 7(7): p. 320-324.
Lowe, T.D. and I. Lorenzoni, Danger is all around: Eliciting expert perceptions
for managing climate change through a mental models approach. Global
Environmental Change, 2007. 17(1): p. 131-146.
Dessai, S., et al., Defining and experiencing dangerous climate change. Climatic
Change, 2004. 64(1-2): p. 11-25.
Kasperson, R.E., et al., The social amplification of risk: A conceptual framework.
1988.
Lorenzoni, I. and N.F. Pidgeon, Public views on climate change: European and
USA perspectives. Climatic change, 2006. 77(1-2): p. 73-95.
Tuana, N., et al., Towards integrated ethical and scientific analysis of
geoengineering: a research agenda. Ethics, Policy & Environment, 2012. 15(2): p.
136-157.
Wood, M.D., et al., Cognitive mapping tools: review and risk management needs.
Risk Analysis, 2012. 32(8): p. 1333-1348.
de Bruin, W.B. and A. Bostrom, Assessing what to address in science
communication. Proceedings of the National Academy of Sciences, 2013.
110(Supplement 3): p. 14062-14068.
Morgan, M.G., et al., Risk Communication: A Mental Models Approach. 2002,
New York: Cambridge University Press.
Fezzi, C., et al., The environmental impact of climate change adaptation. Nature
Climate Change, 2015.
Van Den Berg, M., et al., Impact of the choice of emission metric on greenhouse
gas abatement and costs. Environmental Research Letters, 2015. 10(2): p. 024001.
Kunreuther, H., et al., Risk management and climate change. Nature Climate
Change, 2013. 3(5): p. 447-450.
Whitfield, S.C., et al., The future of nuclear power: Value orientations and risk
perception. Risk Analysis, 2009. 29(3): p. 425-437.
Sarewitz, D., Does climate change knowledge really matter? Wiley
Interdisciplinary Reviews: Climate Change, 2011. 2(4): p. 475-481.
Read, D., et al., What do people know about global climate change? 2. Survey
studies of educated laypeople. Risk Analysis, 1994. 14(6): p. 971-982.
19
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
Reynolds, T.W., et al., Now what do people know about global climate change?
Survey studies of educated laypeople. Risk Analysis, 2010. 30(10): p. 1520-1538.
Árvai, J., The end of risk communication as we know it. Journal of Risk Research,
2014. 17(10): p. 1245-1249.
Keeney, R.L., Value-focused thinking. A path to creative decision making. 1992,
Cambridge, MA: Harvard University Press.
Gregory, R., et al., Structured Decision Making: A Practical Guide to
Environmental Management Choices. 2012, West Sussex, UK: Wiley-Blackwell.
Tuana, N., Embedding philosophers in the practices of science: bringing
humanities to the sciences. Synthese, 2013. 190(11): p. 1955-1973.
Corbera, E., et al., Patterns of authorship in the IPCC Working Group III report.
Nature Climate Change, 2015.
Beck, U., et al., Cosmopolitan communities of climate risk: Conceptual and
empirical suggestions for a new research agenda. Global Networks, 2013. 13(1):
p. 1-21.
Lempert, R.J., et al., A general, analytic method for generating robust strategies
and narrative scenarios. Management science, 2006. 52(4): p. 514-528.
Bridges, T.S., et al., Climate change risk management: a Mental Modeling
application. Environment Systems and Decisions, 2013. 33(3): p. 376-390.
O’Rourke, M. and S.J. Crowley, Philosophical intervention and crossdisciplinary science: The story of the Toolbox Project. Synthese, 2013. 190(11): p.
1937-1954.
Mind meld: Interdisciplinary science must break down barriers between fields to
build common ground. Nature, 2015. 525: p. 289-290.
20