Download Evaluating Climate Visualization

Evaluating Climate Visualization: An Information Visualization Approach
Jimmy Johansson∗ , Tina-Simone Schmid Neset† and Björn-Ola Linnér†
∗
C-Research, Linköping University, Sweden
†
Centre for Climate Science and Policy Research, Linköping University, Sweden
[email protected], [email protected], [email protected]
Abstract
To meet the growing demand of communicating climate
science and policy research, the interdisciplinary field of
climate visualization has increasingly extended its traditional use of 2D representations and techniques from the
field of scientific visualization to include information visualization for the creation of highly interactive tools for both
spatial and abstract data.
This paper provides an initial discussion on the need
and design of evaluations for climate visualization. We
report on previous experiences and identify how evaluation methods commonly used in information visualization
can be used in climate visualization to increase our understanding of visualization techniques and tools.
Keywords— Evaluation, climate visualization, information
visualization
1
Introduction
Climate change has been broadly discussed in a large
variety of media over the last years. There is thus an evident need for communicating scientific data to all ages,
social, cultural and professional groups. The most recent
conflicts of trust towards data referred to in the reports of
the International Panel for Climate Change (IPCC) [9] has
further increased the need for clear and uncompromised
communication of uncertainties, global implications and
effects on a regional to global scale.
Visualization is a powerful tool, as it is persuasive and
can provide intuitive understanding of complex data. Climate visualization refers to tools for analysis and communication of climate change issues and research results, and
provides decision making support for planners and policy
makers [15], see figure 1. The field of climate visualization has a tradition of representing data using both static
2D representations and visualization techniques originating from scientific visualization and engineering [17]. The
complexity of climate research and its inherent need for
communication within scientific fields, to decision makers
as well as to a broader audience, however, benefits from
interactive information visualization techniques [11].
Analysis
Communication
Decision Making
Figure 1: The three interlinked research areas of climate
visualization.
Due to the power of visualization, all choices related
to data selection, transformation and representation need
to be carefully reflected upon and their effectiveness and
efficiency [8] evaluated. This calls for a close collaboration between climate, visualization and human-computer
interaction researchers to define key communication challenges and to design appropriate evaluation tools for different groups of audiences.
The aim of this paper is to initiate a discussion on how to
perform scientific evaluations of climate visualization tools
originating from the field of information visualization. We
argue the need for advanced interactive information visualization tools for climate visualization and the need for
appropriate evaluation techniques. A number of survey
evaluations have so far provided a general understanding
of the requirements and aims of our audiences and users
but more detailed evaluations are necessary to get a more
in-depth understanding of the effectiveness and efficiency
of climate visualization. This paper will discuss challenges
that arise with the use of visual representations of climate
data sets and how evaluation methods applied within the
Figure 3: Example of a web-based application—the Interactive Climate Visualization Environment (ICE [7]).
Figure 2: An inflatable dome, with space for an audience
of 25 visitors. In this immersive environment, the WorldView project [19] has presented a programme on climate
visualization at several venues during 2009.
field of information visualization can support the development and interaction between climate and visualization researchers. We will exemplify evaluations of visual presentations created for immersive environments (figure 2) for a
broad audience where the focus is on the cause, effect and
future alternatives related to climate changes, as well as for
analysis tools (figure 3) for a specific user group.
2
Background
Climate visualization as a general concept of interdisciplinary research over the fields of visualization and climate research ‘refers to interactive research platforms,
which use computer graphics to create visual images of
causes and effects of climate change as well as mitigation and adaptation options’ [15]. Key challenges are the
complexity of societal and environmental systems and the
vast amount of heterogeneous data on spatial and temporal
scales.
Within the climate system and impact research communities standard visualization techniques have been used
for communicating results between scientists themselves
as well as to a broader public. The rapid development of
computational power and computer graphics has, however,
created a broad spectrum of opportunities to visually represent vast amounts of data on climate processes and its
effects as well as the associated complexities and uncertainties interactively, using consumer computer hardware.
Further, information visualization techniques enable the
interactive analysis of interrelations of abstract data to convey synergies within social and environmental systems. A
number of attempts have been made to use information visualization techniques for climate visualization, see for ex-
ample [17, 16, 11]. Using information visualization techniques within climate visualization is today receiving increasing attention but discussions on how to perform scientific evaluations of these tools are not as prominent.
The information visualization community has today
recognized the importance of usability and evaluation is
one of the key research challenges [1, 2]. Evaluation can
be qualitative or quantitative and be performed at different levels, ranging from evaluating small, individual features to entire systems. For a review of currently available
methods and further discussions, see [4, 20, 18]. In [18]
Plaisant derived four different categories of evaluation obtained from a survey of literature of about fifty different
user studies conducted within the area of information visualization: (1) controlled experiments comparing design
elements, (2) (qualitative) usability evaluation of a tool,
(3) controlled experiments comparing several tools, and
(4) case studies of tools in realistic settings. Among these
categories, case studies of tools in realistic settings (4) is
the one least occurring in literature. The most frequently
reported evaluations concern controlled experiments and
comparisons between different tools (1 and 3). Controlled
experiments are typically used to test the effectiveness or
efficiency [8] of an isolated feature. For example, in [10]
common patterns in parallel coordinates were examined
when distorted by different levels of noise in order to determine thresholds for visual identification. Comparisons
of several tools can be found in, for example, [12, 3].
To meet the specific requirements of climate visualization tools originating from information visualization, evaluation techniques already used within this community can
be applied and adopted.
3
Climate Visualization
One of the main challenges within climate visualization is the creation of nuanced representations of climate
change related issues. Within climate communication sev-
eral challenges have been identified [14], and a responsible
approach has to be taken by climate scientists regarding
the selection of data, their visual representations and the
creation of narratives.
Information on climate change is of a particularly complex nature. Studies of causes, effects and means to take
action include several interlinked areas of studies. Climate science includes areas such as atmospheric chemistry, oceanography, biology, palaeoclimatology, physics,
biology, glaciology as well as natural and human geography, political science, economics and sociology. One of
the great challenges is how to capture the numerous intricate linkages between these areas in order to understand
the consequences of inaction as well as of the various action alternatives to mitigate greenhouse gas emissions or
adapt to a changing climate. In addition, information on
climate change involves large data sets and complex modelling. The character of climate change information poses
a particularly wicked problem to convey to public and decision makers.
The intricate interaction between humans and nature
is often conveyed in narratives. The storylines of climate change narratives typically follow a temporal structure (what has happened, what is happening and how will
this develop in the future). Narratives in climate visualization often imply the specific storyline of a sequence of
visuals and the link between areas of content. It is however equally important to relate the narrative to the selection of presented scenarios, alternative futures and how a
future society will adapt to climate change. Visions of
alternative futures and future societies are frequently employed in climate science and multilateral negotiations, for
instance, the storylines developed by the IPCC. Indicators that are represented by visualization range from economic development, greenhouse gas emission and energy
use to regional scenario representations including effects
of water scarcity and sea level rise. The selection of these
narratives are however strongly related to the conception
of (un)sustainable futures. Whichever future scenario we
choose to represent, this implies a choice of a brighter or
darker future. As climate debate gains a massive medial
and political breakthrough, the risk of exaggerating worstcase scenarios has been substantial, creating a general anxiety over the unavoidable doomsday [5, 6].
The climate debate has been characterized by rhetorical apocalypse, i.e. warnings of catastrophic consequences
if we do not change our course of action. However, such
rhetorical narratives run the risk of being contra productive, making the public, in particular young people, disheartened and in the end passive. Further, if the worstcase scenarios do not come true, the history of environmental debate reveals that opponents of stricter environmental
policies use the failed forecasts or exaggerated scenarios to
discredit the environmental organizations and policy making [13]. For each selected data set and narrative, we thus
have to thoroughly evaluate their impact, both in terms of
content and representation, on the audience.
3.1
The WorldView Project
Since 2008, the WorldView project [19] is the basis of a cooperation between climate and visualization researchers at Linköping University, the Norrköping Visualization Centre and the Swedish Meteorological and Hydrological Institute. Within the project we have developed
a number of narratives and data sets focused on the cause
and effect of climate change as well as on a number of action alternatives aimed for approximately 30 minutes presentations in an immersive dome environment (illustrated
in figure 2). During 2009 several presentations were held
for mostly heterogeneous audiences where survey evaluations were conducted [14], see section 3.2.
The narrative of the WorldView presentation included
both effects of climate change to regional change in temperature, changes in arctic sea ice as well as regional and
global sea level rise. Principles for responsibility and effort sharing related to green house gas emissions were presented in form of volume bar charts for each country (as
illustrated in figure 4) and discussed with the audience.
Both categorical and numerical data sets were produced
by the Swedish Meteorological and Hydrological Institute
and the Centre for Climate Science and Policy Research.
The general aim of the project is to provide a nuanced picture based on scientifically validated data to communicate
the complexity of climate change as well as to open up to
joint discussions with the audience. Presentations were always held by climate experts to facilitate interaction and
dialogue with the audience.
3.2
Previously Performed Evaluations
Our experience of evaluating climate visualization in
the WorldView project has so far been limited to surveys.
In six different studies, some ongoing, we have focused on
five specific areas: data, representation, narrative, narrator
and presentation environments.
All survey evaluations have been performed as paper or
web-based surveys and included grading scales as well as
free-text answers. Evaluations were conducted with heterogeneous groups and most respondents indicated that the
content and form of the presentations were relevant. Most
survey questions were related to the understanding of data
sets and on how these are related to increased engagement,
own lifestyle choices, general knowledge and understanding for climate related issues and the audience’s professional interest. When the participants were asked to iden-
tify which audience would have the largest gaining of this
kind of climate visualization presentation, most pointed towards education [14].
Questions that were related specifically to the visual
representation were posed in three areas: novelty, aesthetics and understanding of the visual representations. Questions related to these areas were posed as positive claims
e.g. ‘it was clear that emission levels were symbolized by
the height of the bars in the bar chart, not the volume’ and
the respondents answered using a rating scale ranging from
disagreement to agreement.
We received positive response to the aesthetics and novelty of the climate visualization presentation as well as in
respect to the immersive environment as ‘a new way of presenting this data’. The aesthetics of the visuals was also
by most respondents considered as supporting their general understanding of the data. Further, the response to
the visual representation as volume bar chart (see figure 4)
was positive and the colouring was by most respondents
considered as helpful when interpreting the data.
These surveys are, however, limited in detail and indepth communication of the responses due to the fact that
the questions were directly posed after a brief presentation
and the format of using rating scale answers. Further, generalized surveys for heterogeneous audiences pose a challenge and may cause mis-interpretation of both questions
and responses. While even brief comments on open-ended
questions generate an insight into the argumentation and
motivation of the respondent, these were not sufficient in
order to reflect upon the general understanding and experience of the visual representation. The performed evaluations provide us thus with a general knowledge on the
audience’s background and general rating of the presentation, but lack a deeper insight both in terms of climate
communication as well as of the visual representations.
4
(a)
(b)
Future Directions
The understanding and perception of visual representations needs to be evaluated beyond the realms of a general audience survey. In particular since climate visualization covers three distinct but overlapping areas: science
communication, data analysis and decision making support
(figure 1), evaluation of visualization tools and visual representations need to focus on features relevant to one or
more of these three dimensions. To capture in depth knowledge on the effectiveness and efficiency of climate visualization, we have identified the below listed areas, following
the categorization by Plaisant [18].
Controlled experiments comparing elements. This category refers to testing the usability of small, individual features. Specific areas for climate visualization
include:
(c)
Figure 4: Illustrations of climate visualization, using extruded regions with different colour maps to convey information on a regional and global scale using example data
sets [7].
i) the evaluation of colour maps both directed towards
climate researchers from different fields and their understanding and connotation of colour choice as well
as for a laymen audience. For example, what does
a colour choice deviating from a white-red scale for
temperature increase imply, or how does the colouring of a high-emission country in bright red influence
your perception of the underlying data?
ii) evaluation of how effective and efficient different
graphical primitives (points, lines, polygons) convey
significant structures in data. How are, for example,
multivariate parameters best represented on a local or
global scale? Are multiple simple 2D glyphs a better
choice than fewer complex 3D ones?
iii) assessment of interaction techniques and their
efficiency in revealing information. For example,
how are parameters selected and how do pre-defined
choices of adaptation and mitigation parameters influence the users’ perception of the inter-linkages in the
data set.
iv) evaluation of how uncertainties and variations in
research data are conveyed by the visual representations. How do, for instance, parallel Earth representations opposed to a single representation effect the
understanding of future climate scenarios for greenhouse gas emissions and temperature changes?
v) mapping of data values to opacity using linear and
non-linear transfer functions. For data sets containing
large ranges of values, a linear mapping might not always be sufficient in order to represent all differences
within the data. Non-linear mappings allow the user
to put emphasis on selected value ranges but at the
cost of distorting the underlying structures in the climate data.
vi) effectiveness and efficiency of advanced illumination models; e.g. how does the gaining of attractive
advanced illumination models reflect upon engagement and interest for the presentation itself opposed
to the accuracy and reliability of the presented climate
data.
(Qualitative) usability evaluation of a tool. Specific application areas for climate visualization tools might
range from:
i) web-based platforms and their applicability for uploading, displaying and sharing various data sets for
a number of research areas (e.g. climate temperature
models, land use data, water scarcity data)
ii) tools for dome presentations, their interactivity and
compatibility for different data formats. In these settings the visualization tool is ideally accompanied by
a climate expert, guiding the audience through the
narrative and providing additional information and the
possibility for increased interaction and response to
spontaneous questions.
Controlled experiments comparing two or more tools.
Presentations using climate visualization in immersive environments are becoming more frequent as
the number of visualization domes (both portable as
well as large dome theatres) increases throughout the
world. Hence, it is vital that the visualization tool
is flexible (supporting numerous data formats, interaction techniques and graphical representations) and
intuitive to use for the presenter. Controlled experiments, using the same environment with identical settings, should thus be used to compare visualization
tools currently under development with what is stateof-the-art to advance the insights on user perspectives.
Case studies of tools in realistic settings. While presentations to a larger audience have certain requirements on the visualization tool, climate visualization
for analysis and decision making support is an equally
important area with a specific setup of demands. A
particular need of flexibility, interactivity and ease of
use is posed by climate researchers handling both categorical and numerical data and the inherent need of
their studies to provide decision support to planners
and policy makers on a local to global scale. The
evaluation process should involve new applications
functioning as visualization platforms, that include
the development, storage and possibility to display
and share relevant data sets in an interactive session
with for instance regional decision makers. As the
direct use of the visualization tool is of significant importance for the development and improvement of the
application, climate researchers and decision makers
need to be observed in their interaction with the tool
in a realistic decision environment.
5
Conclusions
Communicating climate change is a global challenge,
both in terms of the large amount and complexity of data,
scenarios and the mixture of categorical and numerical data
sets. Communication and analysis tools therefore need to
be adapted to meet the specific goals and tasks at hand—
be it to provide a scientific understanding of complex phenomena or to tell a nuanced story to a school class. Information visualization provides climate researchers with
such interactive tools to meet the demands of science communication, analysis, planning and decision making.
In this paper we have reported on previous experiences
and identified specific areas of climate visualization where
evaluation methods commonly used in information visualization can be applied to develop and improve existing
tools as well as gain a better understanding of the effects
and efficiency of developed visual representations of climate change related issues. These evaluation methods need
to be further adapted and designed towards a large variety
of audiences and user groups.
Acknowledgements
This work was partly supported by the Swedish Research Council in the Linnaeus Centre CADICS.
References
[11] F. Ladstädter, A. K. Steiner, B. C. Lackner,
B. Pirscher, G. Kirchengast, J. Kehrer, H. Hauser,
P. Muigg, and H. Doleisch. Exploration of climate
data using interactive visualization. Journal of Atmospheric and Oceanic Technology, 2010. Advance online Publication: doi:10.1175/2009JTECHA1374.1.
[12] M. Lanzenberger, S. Miksch, and M. Pohl. Exploring
highly structured data: A comparative study of stardinates and parallel coordinates. In International Conference on Information Visualisation, IV05, pages
312–320, 2005.
[1] C. Chen. Top 10 unsolved information visualization problems. Computer Graphics and Applications,
25(4):12–16, 2005.
[13] B.-O. Linnér. The Return of Malthus: Environmentalism and Postwar Population-Resource Crises. White
Horse Press, Cambridge, UK, 2003.
[2] C. Chen and M. P. Czerwinski. Empirical evaluation of information visualizations: an introduction.
International Journal of Human-Computer Studies,
53(5):631–635, 2000.
[14] T. S. Neset, V. Wibeck, O. Uhrqvist, and J. Johansson. Visualizing climate change: the potential of
dome presentations as a tool for climate communication. In 31st Annual Conference of the European
Association for Computer Graphics.
[3] C. Forsell and J. Johansson. Task-based evaluation
of multi-relational 3D and standard 2D parallel coordinates. In Proceedings of SPIE-IS&T Electronic
Imaging, volume 6495, pages 64950C–1–12, 2007.
[4] C. Forsell and J. Johansson. An heuristic set for evaluation in information visualization. To appear in Advanced Visual Interfaces, 2010.
[5] J. Hedrén and B.-O. Linnér. Utopian thought and
the politics of sustainable development. Futures,
41(4):210–219, 2009.
[15] T-S. S. Neset, J. Johansson, and B.-O. Linnér, editors. State of climate visualization. CSPR Report
N:o 09:04, Centre for Climate Science and Policy Research, Sweden, 2009.
[16] T. Nocke, M. Flechsig, and U. Böhm. Visual exploration and evaluation of climate-related simulation
data. In WSC ’07: Proceedings of the 39th conference
on Winter simulation, pages 703–711, 2007.
[7] ICE: www.visualiseringscenter.se/1/1.0.1.0/255/2.
[17] T. Nocke, T. Sterzel, M. Böttinger, and M Wrobel.
Visualization of climate and climate change data: an
overview. In Digital Earth Summit on Geoinformatics
2008: Tools for Global Change Research, pages 226–
232, 2008.
[8] ISO 9241-11. Ergonomic requirements for office
work with visual display terminals, part 11: Guidance on usability, 1998. http://www.iso.ch/iso/en.
[18] C. Plaisant. The challenge of information visualization evaluation. In Advanced Visual Interfaces, pages
109–116, 2004.
[9] IPCC. Climate change 2007: Synthesis report. contribution of working groups i, ii and iii to the fourth
assessment report of the intergovernmental panel on
climate change. IPCC, 2007.
[19] The WorldView project: http://www.cspr.se/vcc.
[6] M. Hjerpe and B.-O. Linnér. Utopian and dystopian
thought in climate change science and policy. Futures, 41(4):234–245, 2009.
[10] J. Johansson, C. Forsell, M. Lind, and M. Cooper.
Perceiving patterns in parallel coordinates: Determining thresholds for identification of relationships.
Information Visualization, 7(2):152–162, 2008.
[20] T. Zuk, L. Schleiser, and P. Neuman. Heuristics for
information visualization evaluation. In BEyond time
and errors: novel evaLuation methods for Information Visualization (BELIV’06), pages 1–6, 2006.