Download Climate and Carbon Cycle Prediction: C3P

The SciDAC2 CCSM
Consortium Project
2007 - 2012
Presented by John Drake
Atmospheric Model Working Group Meeting
January 31, 2007, NCAR, Boulder
The SciDAC2 CCSM Consortium Project:
A Scalable and Extensible Earth System Model
Lead PI: John B. Drake, Oak Ridge National Laboratory, Co-Lead PI: Phil Jones, Los Alamos National Laboratory
Argonne National Laboratory (ANL) Robert Jacob, Ray Loy, Brookhaven National Laboratory (BNL) Robert McGraw; Lawrence Berkeley National Laboratory (LBNL) Michael Wehner,Lawrence Livermore National Laboratory (LLNL) Phillip
Cameron-Smith, Arthur Mirin; Los Alamos National Laboratory (LANL) Scott Elliot, Philip Jones, William Lipscomb, Mat Maltrud; National Center for Atmospheric Research (NCAR) Peter Gent, William Collins, Tony Craig, Jean-Francois
Lamarque, Mariana Vertenstein, Warren Washington; Oak Ridge National Laboratory (ORNL) Marcia Branstetter, John Drake, David Erickson, Forrest Hoffman,W. M. Post, Trey White, Patrick Worley; Pacific Northwest National
Laboratory (PNNL) Steven Ghan, Xiaohong Liu, Richard Easter, Rahul Zaveri, Sandia National Laboratories (SNL) Mark Taylor
Scientific Challenge
To determine the range of possible climate changes
over the 21st century and beyond through simulations
using a more accurate climate system model that
includes the full range of human and natural climate
feedbacks with increased realism and spatial resolution
Objective
Goals and Deliverables
Over the next five years, we will pursue this objective
through four integrated goals:
1. Extend the capabilities of the Community Climate System
Model (CCSM) to include representations of biological,
ecological, chemical, and aerosol processes that will allow
scientists and policy-makers to simulate climate and climate
change using a comprehensive Earth system model,
2. Provide the necessary software and modeling expertise to rapidly
integrate new methods and model improvements,
3. Pursue the development and evaluation of innovative methods in
the coupled context of the CCSM, and
4. Improve the performance, portability and scalability of the
CCSM on available and future computing architectures for use in
national and international assessments of climate change.
Current Activities (cont.)
Biogeochemical Coupling:
Carbon Land Model Intercomparison (C-LAMP):
•Determine the best set of terrestrial biogeochemistry
process representations to include a future release of CCSM
•Three models coupled to CCSM3: CLM3-CASA', CLM3CN, and LSX-IBIS
•Experiments are being performed with changing climate,
CO2, nitrogen, and land use
•Comparison with high frequency, high quality flux tower
measurements
Atmospheric CO2 resulting from
ecosystem respiration from the
CLM3-CASA’
Aerosols and Atmospheric Chemistry
The primary objective of this proposal is to develop, test, and
exploit a first generation of Earth system models based upon
the CCSM. We will bring to the community a new wellvalidated version of CCSM that will run efficiently on
thousands of processors and include significant model
enhancements.
Improvements to the representation of carbon and chemical
processes for treatment of greenhouse gas emissions and
aerosol feedbacks will be performed in collaboration with the
DOE Atmospheric Science Program, DOE Atmospheric
Radiation Measurement Program, and DOE Terrestrial Carbon
Program. This project will enable climate change simulations
required for scheduled national and international climate
change assessments to which the CCSM is committed as part of
the National Climate Change Science Program (CCSP)
strategy.
Collaborations
Scientific Applications (SAs) and
Partnerships(SAPs)
Brookhaven National Laboratory Robert
McGraw
Oak Ridge National Laboratory Patrick Worley
Argonne National Laboratory Kotamarthi Rao
Centers for Enabling Technology
Collaborations
Earth System Grid (ESG) - Dean Williams
Performance Engineering (PER) – Bob Lucas
VACET – Wes Bethel ; SDM - Ari Shoshani
TOPS – David Keyes; ITAPS - Lori Diachin
APDEC - Phil Colella
Sea surface temperatures
simulated on 0.1o displaced pole
grid using the POP2 code
•Aerosols alter cloud properties and the distribution of
radiation in the atmosphere
•Development of a faster chemistry package for the
troposphere and lower stratosphere
•Introduction of modal and moment representations of
aerosol dynamics modeling aerosol size distribution
•Modeling the surface emissions of pollutants changes
atmospheric radiation
Tasks and Organization
Ice sheet model:
•Largest missing piece of physical climate model and
necessary for sea level rise prediction
•Variable resolution for melting at margins of ice sheet
Ocean vertical grids:
•Arbitrary Lagrangian-Eulerian vertical grid for deep ocean
circulation with good resolution in the surface mixed layer
Ocean horizontal grids:
•Eddy-resolving simulations required for adequate simulation
of ocean circulation
•Unstructured grids provide ability to focus grids in eddyactive regions at reduced cost
Terrestrial carbon cycle and dynamic vegetation
Atmospheric chemistry and aerosol dynamics
Ocean ecosystems and biogeochemical coupling
Feedbacks between atmospheric composition and biogenic
emissions
Model Integration and Evaluation
Integration and unit testing
New icesheet and ocean models
New atmospheric dynamics: finite volume (cubed sphere),
discontinuous Galerkin, others(icosahedral)
Frameworks for model evaluation
Computational Performance
Scalablity to thousands of processors, load balance, (fault recovery)
Computational Climate
End Station for Climate
Change Science
- Warren Washington (PI)
High Resolution Ocean and Cryosphere:
Earth System Model
Current Activities
Model Integration and
Evaluation:
Software enabling integration:
• Model Coupling Toolkit (MCT 2.3.0) released (1/10/07)
• Optimized coupling strategies
• New methods for model evaluation (w/ Rao)
CVT Delaunay tessellation of the
North Atlantic with 85K
triangles - Gunzburger and Ju
Scalability and Performance:
Software enabling scalability:
• Collecting performance data to quantify and identify
scalability limiters. Modifying code to improve
scalability:
• Enabling different degrees of parallelism in physics
and dynamics of atmosphere model.
• Identifying and eliminating nonscalable memory
allocations.
• Working with Pnetcdf developers and CCSM
software engineers to implement a single parallel I/O
interface across all CCSM components.
• Validating software solutions on large Cray XT3 and
IBM BlueGene systems.
Contact Info: [email protected], [email protected]
Project Web Page: www.scidac.gov/climate/earth.html
AMWG Related Activities

Atm chemistry and aerosol dynamics (J.-F. Lamarque)
–
–
–
–

Aerosol Indirect effect - Steve Ghan, Bill Collins
Moment methods - Bob McGraw
Atm chem fast mechanisms - Philip Cameron-Smith
Sulfur cycle - Dave Erickson
Model Integration and Evaluation (M. Vertenstein)
– Integration and unit testing (Rob Jacobs, Tony Craig)
– FV (cubed sphere), DG, others (Mark Taylor, Art Mirin, Pat Worley,
J. Drake)
– Evaluation Frameworks (K. Rao et al.)

Computational Performance (Pat Worley, Art Mirin, Ray Loy)
– Scalability, load balance, (fault recovery)
Supercomputers at NCCS
50 TF Cray XT3
• 5212 dual core Opterons
• 21 Terabytes memory
• 100TB scratch
18 TF Cray X1 /
X1E
1000 TF
Cray
TBD
60 TF Cray XT4
250 TF
100 TF
• 6296 dual core Opterons
25 TF 50 TF
• New interconnect
18.5 TF
• Acceptance testing
• Will not expand
• 1024 processors
• Vector processing for
sustained performance
2004 2005
2006
Leadership Computing Facility
(also BG/L at Argonne)
2007 2008
2009
Computer Allocation:
Computational End Station for Climate
Change Science
PI: Warren Washington (NCAR), partners: CCSM, COSIM, PCMDI, SciDAC,
NASA-GSFC, PNNL, CCRI(Universities)
– Extensible community models available for computational
science
– Coordination of effort among agencies and institutions (talk to
Lawrence Buja, J. Drake or one of the partner representatives)
What does DOE want?



Relation to Aerosol Science Program and Terrestrial Carbon
Program
Coordinated enterprise, relation to CETs and SAPs
Reporting
–
–
–
–
Impacts and revised scope statement
30 days: 4-6 slides for Dr. Orbach
60 days: Management Plan, website, performance baseline
6months: progress reports, highlights
DOE OBER Performance Targets
•2006 Deliver new measurements of clouds where observations are missing
•2007 Include realistic cloud simulations in a climate model
•2008 Measure ecosystem responses to climate change
•2010 Develop/validate new models predicting effect of aerosols on climate forcing
•2010 Provide climate model that links the Earth climate system with Earth’s biological
systems
•2013 Reduce differences between observed temperature & model simulations at sub
continental scales using several decades of recent data
•2015 Deliver improved climate data & models for policy makers to determine safe levels
of greenhouse gasses.