Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Project overview
Document related concepts
Transcript
ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 1 ACRI-ST/LOV, UoP, NIVA, BC, DLR, ICESS consortium ESA DUE GlobColour Global Ocean Colour for Carbon Cycle Research SeaBASS Final Report Reference: GC-MA-ACR-FR-01 Version 1.0 February 13, 2008 All rights reserved ACRI-ST 2007 ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 2 Document Signature Table Author Name Function Company O. Fanton d’Andon GlobCOLOUR ACRI-ST 12 February 2008 13 February 2008 Project manager Verification S. Lavender Management controller UoP Approval S. Pinnock ESA Project Manager ESA Signature Date Change record Issue Date 1.0 13/02/08 Description Final Report All rights reserved ACRI-ST 2007 Change pages Initial version ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 3 Distribution List Organisation To Nb. of copies ESA ESRIN Simon Pinnock 2 GlobColour Partners All partners 1 GlobColour Users Cyril Moulin (IOCCP) Venetia Stuart (IOCCG) Rosa Barciela (MetO) 1 1 1 All rights reserved ACRI-ST 2007 ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 4 Table of content 1 INTRODUCTION...............................................................................................................7 1.1. 1.2. 1.3. 1.4. 1.5. 1.6. Overview ....................................................................................................................7 Organisation of the document....................................................................................7 Reference Document .................................................................................................7 Acronyms ...................................................................................................................8 Consortium...............................................................................................................11 List of GlobColour presentations/publications..........................................................12 2 EXECUTIVE SUMMARY ................................................................................................13 3 THE GLOBCOLOUR SYSTEM ......................................................................................15 3.1. Overview ..................................................................................................................15 3.1.1 Requirements Baseline............................................................................................15 3.1.2 Design Justification File ...........................................................................................16 3.1.3 Technical Specification ............................................................................................16 3.1.4 Design Definition File...............................................................................................17 3.1.5 GlobColour data processing system........................................................................17 3.1.6 Acceptance Review Report .....................................................................................17 3.2. GlobColour System..................................................................................................17 3.2.1 Overview..................................................................................................................17 3.2.2 The GlobColour processor architecture...................................................................19 3.2.3 Functionalities of the GlobColour processor............................................................19 3.2.4 Input Products Overview..........................................................................................21 3.2.5 Output Products Overview.......................................................................................21 3.3. Diagnostic Data Set and DDS Tools........................................................................22 3.4. GlobColour Tools .....................................................................................................22 4 GLOBCOLOUR DATA SET ...........................................................................................24 4.1. 4.2. 5 Full Product Set .......................................................................................................24 Diagnostic Data Set .................................................................................................25 VALIDATION ..................................................................................................................26 5.1. Validation Protocol ...................................................................................................26 5.2. Validation Report .....................................................................................................27 5.2.1 Open ocean water conclusions................................................................................27 5.2.2 Coastal case 2 water conclusions ...........................................................................32 6 USER ASSESSMENT ....................................................................................................34 6.1. First user consultation..............................................................................................34 6.1.1 Merging recommendations ......................................................................................35 6.1.2 Meeting summary ....................................................................................................35 6.2. Second user consultation.........................................................................................36 6.3. Potential extensions.................................................................................................37 7 FAQ.................................................................................................................................38 [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 5 List of Figures Figure 1: Example of GlobColour product – Chla monthly mean October 2002 ...........................14 Figure 2: GlobColour System overview.........................................................................................18 Figure 3: The GlobColour processor high-level description ..........................................................20 Figure 4: Global composite of L412 for May 2006 ........................................................................28 Figure 5: Global composite of L555 for May 2006 ........................................................................28 Figure 6: Global composite of CHL1 (weighted average) for May 2006 .......................................29 Figure 7: Global composite of CHL1 (GSM) for May 2006............................................................29 Figure 8: First GlobColour User consultation – Villefranche – 4-6 December 2006......................34 Figure 9: Second GlobColour User consultation – Oslo – 20-22 November 2007........................36 List of Tables Table 1: GlobColour consortium....................................................................................................11 Table 2: GlobColour presentations/publications............................................................................12 Table 3 : Current core GlobColour User Group.............................................................................15 Table 4 : Input product overview ...................................................................................................21 Table 5 : Output product overview ................................................................................................22 Table 6: Available tools .................................................................................................................23 Table 7: List of the DDS sites........................................................................................................25 Table 8: Match-up statistics Performance of GlobColour data set ................................................31 Table 9: User involvement key rendezvous’..................................................................................34 [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Page: 6 Final Report 1 INTRODUCTION 1.1. Overview This document is the Final Report of the GlobColour EO service. It contains a summary of the Requirements Baseline (RB), Design Justification File (DJF), Technical Specification (TS), Full Product Set (FPS), Full Validation Report (FVR) and Service Assessment Report (SAR). This document is publicly available. 1.2. Organisation of the document Section 1 includes a description of the consortium and a list of presentations/publications. Section 2 is an executive summary of the GlobColour Project achievements. Section 3 provides a description of the GlobColour system, established from the Requirements Baseline, up to the delivery of the processor. Section 4 presents the GlobColour data set as available at the end of Phase 2 while Section 5 documents the results of the consortium validation of the FPS. Section 6 provides an overview of the user assessment, the two workshops and their outcomes. Section 6 attempts to provide answers to frequently asked questions. 1.3. Reference Document N° [1] File Reference EOEP-DUEP-EOPS-SW-05-0003 [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] 079-P360 FPS-v1.1 GC-PL-ACR-MA-01 GC-PL-BC-SDP-01 GC-RS-UOP-RB-01 GC-RS-UOP-DJF-01 GC-RS-ACR-TS-01 GC-PL-NIVA-VP-01 GC-RS-BC-DDF-01 GC-SW-BC-ATD-01 [12] GC-RS-UOP-PVAR-01 [12] [13] [14] [15] GC-PL-NIVA-FVR-01 GC-RS-UOP-SAR-01 GC-UM-ACR-PUG-01 5_Mangin_Merging_Benefit_v1.0. ppt.ppt Title/Description Statement of Work DUE GlobColour AO/1-4807/05/I-LG ACRI-ST Proposal Full Product Set Management plan Software Development Plan Requirements Baseline Design Justification file Technical specification Validation Protocol Design Definition file Acceptance Test Document Preliminary Validation and Assessment Report GlobColour Validation Report Service Assessment Report Product User Guide GlobColour/Medspiration Workshop - Session 4 Version June 7, 2005 September 9, 2005 November 2007 November 15, 2005 December 22, 2005 February 27, 2006 November 21, 2006 August 31, 2006 December 6, 2006 July 4, 2006 December 11, 2007 January 12, 2007 December 14, 2007 February 13, 2008 October 8, 2007 November 2007 [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 7 1.4. Acronyms AD ADEOS AR ATD AATSR AMESD AVHRR BC BEAM BOUSSOLE CDOM CDR CoP CF CFI CNES COTS CZCS DDF DDS DJF DLR DPM DRD DUE ECSS EEA EO EOSDIS ESL FAQ FP FP6 FPS FR FTP FVR Applicable Document Advanced Earth Observation Satellite Acceptance Review Acceptance Test Document Advanced Along Track Scanning Radiometer African Monitoring of the Environment for Sustainable Development Advanced Very High Resolution Radiometer Brockmann Consult Basic ERS and Envisat (A)ATSR and MERIS Toolbox Bouée pour l’acquisition de Séries Optiques à Long Terme Coloured dissolved organic matter Critical Design Review Conference of the Parties Climate and Forecast Customer-furnished item Centre National d'Etudes Spatiales Commercial Off-The-Shelf software Coastal Zone Color Scanner Design Definition File Diagnostic Data Set Design Justification File Deutsches Zentrum für Luft- und Raumfahrt Detailed Processing Model Document Requirement Definition Data User Element of the ESA Earth Observation Envelope Programme II European Cooperation for Space Standardization European Environment Agency Earth observation Earth Observing System Data and Information System Expert Support Laboratories Frequently Asked Questions Final Presentation EC Framework Programme 6 Full Product Set Full Resolution (300 m for MERIS) File Transfer Protocol Full Validation Report [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 8 GAC Global Area Coverage (4 km sub-sampled SeaWiFS product) GC-merging group GlobColour merging group GIS Geographic Information System GMES Global Monitoring for Environment and Security GOMOS Global Ozone Monitoring by Occultation of Stars ICESS Institute for Computational and Earth Systems Science IDDS Initial Diagnostic Data Set ITT Invitation to tender IOCCG International Ocean Colour Coordinating Group IOCCP International Ocean Carbon Coordination Project IOP Inherent Optical Property IODD Input Output Data Definition JPEG Joint Picture Experts Group JRC Joint Research Center KO Project kick-off LAC Local Area Coverage (1 km SeaWiFS product) LMD Laboratoire de Météorologie Dynamique LOV Laboratoire Océanologique de Villefranche-sur-mer MERIS Medium Resolution Imaging Spectrometer MERSEA Marine Environment and Security for the European Area – Integrated Project of the EC Framework Programme 6 MetO Met Office MM5 Meteorological Mesoscale Model from NCAR University MOBY Marine Optical Buoy MODIS Moderate Resolution Imaging Spectrometer netCDF Network Common Data Format NIVA Norwegian Institute for Water Research NOMAD NASA Bio-Optical Marine Algorithm Data Set NRT Near-real time OCTS Ocean Color and Temperature Scanner PC Personal computer PDF Adobe portable document format PDL Parameters Data List PDR Preliminary Design Review PLYMBODY Plymouth Marine Bio-Optical Data Buoy PM Progress meeting PMP Project Management Plan POLDER Polarization and Directionality of the Earth's Reflectances PP Primary Production PPS Preliminary Product Set [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP PVAR QR RB RD REASoN RID RMS SAR SDP SeaWiFS SeaBASS SeaDAS SIMBIOS SPR SRR SYS TO TS UNFCCC UoP VP WFD WIM WKS WWW Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 9 Preliminary Validation and Assessment Report Qualification Review Requirements Baseline document Reference Document NASA Research, Education and Applications Solution Network project Review Item Discrepancy Root mean square Service Assessment Report Software Development Plan Sea-Viewing Wide Field of View Sensor SeaWiFS Bio-Optical Archive and Storage System SeaWiFS Data Analysis System Sensor Intercomparison for Marine Biological and Interdisciplinary Ocean Studies Software Problem Report System Requirements Review GlobColour data processing system deliverable Technical Officer Technical Specification United Nations Framework Convention on Climate Change University of Plymouth (UK) Validation Protocol Water Framework Directive Windows Image Manager Workshop GlobColour web site deliverable [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 10 1.5. Consortium Organisation ESA/ESRIN (Italy) Role Technical officer Contact Simon Pinnock [email protected] ACRI-ST (France) Prime contractor Management GlobColour Processor Production User requirements follow up/ Design justification / Merging Odile Fanton d’Andon [email protected] ARGANS LIMITED (UK) NIVA (Norway) New products / Evolution /Merging Samantha Lavender [email protected] Validation (DDS) Dominique Durand [email protected] Brockman Consult (Germany) Tools development Web server Carsten Brockmann [email protected] ICESS (USA) Scientific support Stéphane Maritorena [email protected] DLR (Germany) Scientific support Andreas Neumann [email protected] LOV (France) Scientific support David Antoine [email protected] University of Plymouth (UK) Logo Samantha Lavender [email protected] Table 1: GlobColour consortium [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Page: 11 Final Report 1.6. List of GlobColour presentations/publications GlobColour Presentations Place Date At the Medspiration User consultation workshop, by P. Bardey IFREMER, Brest December 13, 2005 At the IOCCG meeting, by P. Regner Korea January, 2006 At the NASA Ocean Colour Team meeting, by S. Lavender Newport, USA 11-13 2006 NASA workshop invited talk at the Ocean Optics Conference, by A. Morel Montreal, Canada 9-13 October 2006 GlobColour Workshop, by Team Villefranche Citadel, France 4-6 December 2006 GlobColour project, by S. Lavender at 12th IOCCG Meeting – Invited talk Swakopmund, Namibia 16-18 January 2007 April ESA GlobColour validation results addressing MODIS, MERIS and merged IVM, The 15 products at Marcoast Validation workshop Netherlands 2007 By Kai Sörensen (NIVA) et al. March GlobColour: European Service for Ocean Colour, Invited talk at NASA Seattle Ocean Color Team Meeting, by Sam Lavender 2007 April 09-13 Invited talk at the ENVISAT symposium, by O. Fanton d’Andon 24 April 2007 Montreux, Switzerland GlobColour: Implementation as a GHRSST European RDAC, GHRSST Melbourne Science Meeting, by Sam Lavender 2007 14th18th May Invited talk at IGARSS Invited Session: ENVISAT MERIS/AATSR Barcelona, applications Spain by O. Fanton d’Andon 23-27 2007 Developing a European ocean colour service supporting global carbon- Amsterdam, cycle research and operational oceanography at Joint 2007 The EUMETSAT/AMS Conference Netherlands by O. Fanton d’Andon http://www.eumetsat.int/Home/Main/Publications/ Conference_and_Workshop_Proceedings/SP_1196354659081?l=en 24-28 September 2007 GlobColour Workshop, by Team Oslo, Norway 20-22 November 2007 November ESA Bulletin article http://www.esa.int/esaCP/SEMVVV19R9F_index_0.html ESRIN November 2007 July Merchant, C. J., M. J. Filipiak, P. Le Borgne, H. Roquet, E. Autret, Scientific 2008 J.-F. Piollé, and S. Lavender (2008), Diurnal warm-layer events in the Publication: western Mediterranean and European shelf seas, Geophys. Res. Lett., 35, GlobColour used to assess L04601, doi:10.1029/2007GL033071. the biological influence Table 2: GlobColour presentations/publications [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report 2 Page: 12 EXECUTIVE SUMMARY In 2005, the International Ocean Colour Coordinating Group (IOCCG) convened a working group to examine the state of the art in ocean colour data merging, which showed that the research techniques had matured sufficiently for creating long multi-sensor datasets (IOCCG, 2007). As a result, ESA initiated and funded the DUE GlobColour project (http://www.globcolour.info/) to develop a satellite based ocean colour data set to support global carbon-cycle research. It aims to satisfy the scientific requirement for a long (10+ year) time-series of consistently calibrated global ocean colour information with the best possible spatial coverage. This has been achieved by merging data from the three most capable sensors: SeaWiFS on GeoEye’s Orbview-2 mission, MODIS on NASA’s Aqua mission and MERIS on ESA’s ENVISAT mission. In setting up the GlobColour project, three user organisations were invited to help. Their roles are to specify the detailed user requirements, act as a channel to the broader end user community and to provide feedback and assessment of the results. The International Ocean Carbon Coordination Project (IOCCP) based at UNESCO in Paris provides direct access to the carbon cycle modelling community's requirements and to the modellers themselves who will use the final products. The UK Met Office's National Centre for Ocean Forecasting (NCOF) in Exeter, UK, provides an understanding of the requirements of oceanography users, and the IOCCG bring their understanding of the global user needs and valuable advice on best practice within the ocean colour science community. The three year project kicked-off in November 2005 under the leadership of ACRI-ST (France). The first year was a feasibility demonstration phase that was successfully concluded at a user consultation workshop organised by the Laboratoire d'Océanographie de Villefranche, France, in December 2006. Error statistics and inter-sensor biases were quantified by comparison with insitu measurements from moored optical buoys and ship based campaigns, and used as an input to the merging. The second year was dedicated to the production of the time series. In total, more than 25 Tb of input (level 2) data have been ingested and 14 Tb of intermediate and output products created, with 4 Tb of data distributed to the user community. Quality control (QC) is provided through the Diagnostic Data Sets (DDS), which are extracted sub-areas covering locations of in-situ data collection or interesting oceanographic phenomena. The Full Product Set (FPS) covers global daily merged ocean colour products in the time period 1997-2006 and is freely available for use by the worldwide science community at http://www.globcolour.info/data_access_full_prod_set.html [RD3]. The GlobColour service distributes global daily, 8-day and monthly data sets at 4.6 km resolution for, chlorophyll-a concentration, normalised water-leaving radiances (412, 443, 490, 510, 531, 555 and 620 nm, 670, 681 and 709 nm), diffuse attenuation coefficient, coloured dissolved and detrital organic materials, total suspended matter or particulate backscattering coefficient, turbidity index, cloud fraction and quality indicators. Error statistics from the initial sensor characterisation are used as an input to the merging methods and propagate through the merging process to provide error estimates for the output merged products. These error estimates are a key component of GlobColour as they are invaluable to the users; particularly the modellers who [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 13 need them in order to assimilate the ocean colour data into ocean simulations. See the Product User Guide (PUG) for further details on the GlobColour products http://www.globcolour.info/CDR_Docs/GlobCOLOUR_PUG.pdf [RD14]. An intensive phase of validation has been undertaken to assess the quality of the data set. In addition, inter-comparisons between the different merged datasets will help in further refining the techniques used. Both the final products and the quality assessment were presented at a second user consultation in Oslo on 20-22 November 2007 organised by the Norwegian Institute for Water Research (NIVA); presentations are available on the GlobColour WWW site. In 2008, the project continues by merging MERIS and MODIS ocean colour data, with a global daily delivery in NRT to primarily support operational oceanography. In the future this will feed into the European Community funded Marine Core Service that will start to provide, in 2009, a suite of services to support Europe's decision makers. GlobColour's merged ocean colour dataset will be provided by the Ocean Colour Thematic Assembly Centre (OC TAC) whose main objective is to bridge the gap between space agencies providing ocean colour data and GMES marine applications. The OC TAC will deliver core ocean colour products, annotated with pixel level quality control flags and reliable error estimates, at global to regional European scales consolidating European efforts and maximising their impact. Future availability of MERIS ocean colour data will be assured with the launch of the first Sentinel-3 satellite in 2012. Figure 1: Example of GlobColour product – Chla monthly mean October 2002 [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report 3 Page: 14 THE GLOBCOLOUR SYSTEM 3.1. Overview 3.1.1 Requirements Baseline A consolidated set of product and system requirements has been derived from the user requirements identified in the Statement of Work [RD1]. This formed the Requirements Baseline (RB) [RD6] for the GlobColour EO service and was used as an input to the User Requirements Engineering. The final version of the Requirements Baseline was obtained through a further consultation process with the users and other relevant sources. A questionnaire listing all the issues that arose from the first understanding of the requirements was distributed and discussed at the first user consultation at Unesco on January 4-5, 2006. User feedback was captured and the limitations that have to be understood in association with each requirement and proposed solution were synthesised. The following organisations form the core GlobColour user group. IOCCG IOCCP Met Office http://www.ioccg.org International Ocean-Colour Coordinating Group "The International Ocean-Colour Coordinating Group (IOCCG) was established during 1996 under the auspices of the Intergovernmental Oceanographic Commission (IOC), following a resolution endorsed by the Committee on Earth Observation Satellites (CEOS), to act as a liaison and communication channel between users, managers and agencies in the Ocean Colour arena." http://ioc.unesco.org/ioccp International Ocean Carbon Coordination Project "The IOCCP will work with national, regional, and international programs and data centres to provide a global view of ocean carbon by: (i) developing a compilation and synthesis of ocean carbon activities and plans; (ii) working with international research programs to fully integrate carbon studies into planning activities; (iii) standardizing methods, quality control/quality assurance procedures, data formats, and use of certified reference materials; and (iv) supporting regional synthesis groups to develop regional and global databases." Operational oceanography http://www.metoffice.gov.uk National Centre for Ocean Forecasting (NCOF) Table 3 : Current core GlobColour User Group [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 15 3.1.2 Design Justification File The Design Justification File (DJF) [RD7] documents the justification for all design decisions for the GlobColour EO service. It contains: • an analysis of the Earth Observation (EO), in-situ and ancillary data available as inputs; • a data procurement plan; • a justification of the candidate methods planned to be used to derive the required products; • a preliminary design of the processing system; • the full pre-merger sensor characterisation; • a review of the potential impact on candidate merging algorithms. Following the ECSS standard, the DJF was updated during the life of the project. Two versions of the DJF were delivered, one at SRR and one at CDR, with an additional final update in November 2006. As a result of the analysis performed, three merging techniques were identified as the basis for further testing (see the PUG for further details): • Simple averaging (AV); • Weighted averaging (AVW) with weightings based on the sensor/product characterisation; • GSM model based on Maritorena and Siegel (2005, Remote Sensing of Environment) paper. 3.1.3 Technical Specification The Technical Specification document [RD8] defines the technical performances of the processor (GlobColour software, DDS tools and web site), in terms of input data description, precise processing description (DPM) and output data specification, content and format (IODD). For each component, the Technical Specification defines: o Functional and performance expectations o Product quality requirements o Data definition and database functionality o Software quality requirements The software requirements including the processing system requirements, the DDS tools requirements and the web site and services requirements have been derived from all requirements expressed in the RB [RD6]. The data definition covers the input level 2 data, the output level 3 data (binned and mapped products), the level 3 DDS products, the in-situ data and the auxiliary data. The logical model of the processing system (GlobColour software) provides the processing system overview and the detailed processing model (ocean colour property retrieval, binning method algorithmic baseline and merging method algorithmic baseline). The logical model of the DDS tools includes a DDS tools overview and the detailed processing model of the tools. A traceability matrix with respect to the RB [RD6] and VP [RD9] requirements completes the Technical Specification. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 16 This document is the technical reference that was used to build the architectural design of the data processing system. 3.1.4 Design Definition File The Design Definition File (DDF) [RD10] for the GlobColour EO service contains the architectural design of the data processing system to be developed, which responds to the software requirements documented in the Technical Specification (TS) [RD8]. It was updated during the life of the project, so that it documents the detailed design of the software. DDF version 2 was issued at Critical Design Review (KO + 7 months). The software design document is structured in accordance with the ECSS-E-40 document Part 2B, and is a constituent of the GlobColour design definition file, addressing both the GlobColour processor and the DDS tools. The DDF also includes all software configuration items, which have been produced during the detailed design phase. These are: source code, test code, detailed design documentation, build scripts, README and CHANGELOGs. Since these items are part of the software distributions they underlie an independent versioning. The DDF simply lists the developed configuration items. The actual files are found in the latest software distribution. The definition of the production hardware and of the delivered hardware are also provided. 3.1.5 GlobColour data processing system The GlobColour processor and the DDS tools have been developed on the basis of the TS [RD8]. The GlobColour data processing system was delivered to ESA on December 10, 2007. The delivered hardware system is a DELL Intel Core2 Duo (E6400 @ 2.13GHz) computer including 2 Gb of DDR2-SDRAM and running the latest stable (“Etch” 4.0) GNU/Linux Debian operating system. 3.1.6 Acceptance Review Report The acceptance review was performed at ESRIN over the period 8-15 Jan 2008 by Simon Pinnock (ESA T.O.) with remote support from ACRI-ST. During this Acceptance Review, the software tests defined in the GlobColour Acceptance Test Document (ATD) v2.0 (11/12/2007) [RD11] were performed. The ATD provides a description of acceptance tests for the GlobColour software, which consists of the GlobColour processor, the DDS tools and the GlobColour web services. The ATD is part of the Design Justification File of the GlobColour project [RD7]. All acceptance tests have been performed successfully, and therefore the Acceptance Review is also successfully completed. 3.2. GlobColour System 3.2.1 Overview The following diagram shows the dataflow from the level-2 and in-situ input data to the end products (DDS, PPS, FPS merged by the production system) and the distribution to the end users through the GlobColour web server: [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 17 Figure 2: GlobColour System overview The level-2 data acquisition system can be any means to acquire the latest version of the input data and to store it into the GlobColour file repository. This can be either data distributed via the ESA Data Dissemination System, a web-service client for the automatic download of data using the ENVISAT catalogue interface, an FTP client driven by a UNIX daemon which downloads data from rolling archives, MERIS level 2 products generated from level 0 products received via DDS or even a manual download of data using an internet browser. The level-2 data is ingested into the processing system through either automatic or manual procedures. In NRT, the processing system is capable of providing daily level-3 merged products within 24h. The daily merged products are used to derive the 8-day average products, and to derive the monthly average products, all of them being archived by the system. The inventory is realised as a relational database within the open-source database management system mySQL. The processing system also creates DDS that are extraction of level-2 products on a pre-defined set of geographical areas. Further to their extraction, these DDS are resampled on a 1km grid [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 18 (using a nearest neighbour technique) as well as on a 4.6km Plate-Carré grid (using the Sutherland-Hodgeman area clipping technique). The purpose of these two grids are that: ¾ Results at 1km resolution are used in conjunction with the in-situ measurements for validation and characterisation of the level-2 input data, ¾ Results at 4.6km resolution are used to characterise, for each sensor, the impact on data characterisation due to the change in spatial resolution and binning. Future GlobColour products could aim for a final higher resolution as computer processing and storage improve in the future. BEAM/VISAT serves as a platform for the validation tools that were developed as VISAT plug-ins. Additionally a BEAM product reader allows for direct ingestion of the GlobColour level-3 products. 3.2.2 The GlobColour processor architecture The GlobColour processor is designed to operate as a stand-alone system. It neither requires a special processing infrastructure nor imposes any constraints or special needs on the hardware used for the processing. The minimum target system is PC based hardware under the GNU/Linux operating system. The final performance of the processing system scales with the number of parallel hardware processing nodes. The design of the processor furthermore considers different merging methods at any time without having the need to change the existing data flow and system logic. Therefore, it is possible to easily exchange the merging methods. Two different families of merging strategies are currently implemented: 1. Merging of bio-optical properties obtained from different sensors. 2. Merging of normalised water-leaving radiances obtained from different sensors and then the application of bio-optical models for calculating ocean-colour products. Special care was taken in order to achieve the runtime performance of the processing system with a minimum number of parallel processing nodes so that it meets the requirements stated in the SoW, namely: • automatically processing and merging of at least one full month of input data from all four sensors, within five days of real time • availability of a merged global ocean colour product (all fields) within 24 hours of satellite acquisition via the GlobColour web site 3.2.3 Functionalities of the GlobColour processor The GlobColour processor is the computation element of the GlobColour processing system. Its responsibility is the transformation of EO level-2 products (or level-3 products) from independent instrument/missions into a single merged level-3 product. The GlobColour processor is mainly composed of 4 separate modules, namely: 1. a pre-processor module 2. a spatial binning module 3. a merging module [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 19 4. a temporal binning module The logical model of the GlobColour processing system is illustrated in Figure 3. Figure 3: The GlobColour processor high-level description For each sensor, a pre-processing is applied just after extraction of the level-2 data. This pre processor serves in the case of MERIS, and wherever requested, to transform reflectance into normalised water leaving radiance. It is also used to apply a cross calibration LUT so that equivalent data can be merged according to the outcomes of the cross-characterisation exercise (as determined by the results of DJF [RD7]). The complete binning scheme proposed for production the GlobColour ocean colour products is a three step approach comprising of spatial binning, data merging and temporal binning. These three steps follow the specific requirements expressed in the RB [RD7] as summarised: • Spatial binning of level-2 data onto a global integerised sinusoidal (ISIN) grid by employing a flux-conserving algorithm. The algorithm uses the fast Sutherland-Hodgeman area clipping. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP • • • 3.2.4 Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Page: 20 Final Report Merging of spatially binned level-2 data into daily, weekly (eight days) and monthly averaged level-3 data Production of diagnostic data sets (DDS) Generation of quick look (QL) images Input Products Overview The following level-2 ocean colour products are used as inputs to the processing system: Level-2 input product Envisat MERIS Reduced Resolution MODIS Aqua Ocean Color SeaWiFS Ocean Color GAC SeaWiFS Ocean Color LAC Resolution 1 km 1 km 4 km 1 km Period 2002 - 2006 2002 - 2006 1997 – 2006 Data Volume 16 TB 25 TB 5 TB Table 4 : Input product overview Note that these volumes are given for uncompressed products. The products were downloaded under their compressed form (bzip2) where the compression gain is equal to 85% for MODIS and 80% for SeaWiFS. The MERIS level-2 products are available on NAS and are not compressed. 3.2.5 Output Products Overview The product format for the level-3 global ocean colour data products is netCDF 3. The structure and contents follow the Climate and Forecast Metadata Conventions [http://www.cgd.ucar.edu/cms/eaton/cf-metadata/CF-1.0.html]. The geophysical data layers include: a. Chlorophyll-a concentration b. Coloured dissolved organic matter c. Total suspended matter d. Diffuse attenuation coefficient (in-water) e. Fully normalised water leaving radiances (available bands) f. Data quality flags including aerosol optical thickness and cloud fraction [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Time Period Frequency Title GlobColour level-3 Product GlobColour level-3 LowRes Meteorology Product GlobColour level-3 Product Page: 21 Final Report 1997–2006 Daily, weekly (8-days), monthly Grid 1/24° (4.63 km) equal area, integerised sinusoidal projection (ISIN) Equal angle, Plate-Carré projection (PC) Equal angle, Plate-Carré projection (PC) Resolution Format 8640 pixels at equator, 3 at poles netCDF 3 e.g. 1440 x 720 pixels for 0.25° resolution netCDF 3 e.g. 1440 x 720 pixels for 0.25° resolution JPEG Table 5 : Output product overview 3.3. Diagnostic Data Set and DDS Tools The processing system creates DDS granules for a minimum of 20 predefined diagnostic sites. The list of diagnostic sites and their coordinates is made available to the processor as an auxiliary table. DDS granules are extracted from level-2 input products and level-3 output products: • Level-2: for each level-2 input product, a DDS granule is written for all diagnostic sites intersecting the area of each level-2 scene. DDS files from the level-2 input products are extracted and resampled on a local 1km equal area PC grid. The resampling uses a nearest neighbour technique (so that the radiance values remain unchanged) and keeps properties such as sensor geometry (so they can be utilised within the bio-optical modelling approaches). They are also available in the final geographical coordinate system (4.63km sinusoidal grid) and in a local 4.63 PC grid for direct visualisation. • 20 DDS files are always created for each daily merged product and for each of the weekly and monthly averaged products. Level-3 DDS files are extracted after merging and averaging the data so that they are provided in the final geographical coordinate system (4.63km sinusoidal grid) and in a local 4.63 PC grid for direct visualisation. A DDS granule covers approximately 100km x 100km and comprises all layers of information from the input or output products. The granules are provided in the netCDF format following the Climate and Forecast Metadata Conventions. 3.4. GlobColour Tools In order to validate the GlobColour data products, a set of tools was developed which allows for visualising and analysing the DDS granules and for inter-comparison of the DDS data with in-situ measurements from dedicated diagnostic sites. Tools to analyse the DDS are created via a BEAM plugin. Users can download the latest version of the tools from the GlobColour project home page. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 22 The software components developed for the DDS tools include: • Reader for the DDS granules in netCDF format • Product reader for the final netCDF temporally binned products • Reader for the in-situ measurement data sets • Tool for plotting correlations between L2/L3 DDS and in-situ measurements • Tool for calculating error statistics from DDS and in-situ measurement • Tool for exporting the plots, statistics and analytic results, e.g. to MS Excel Tools to browse and view the content of the GlobColour products are available on the web site: http://www.globcolour.info/ "Data tools" DDS tools Export Pin-Pixels The DDS Tools support the validation of the data products generated in GlobColour. The current sets of tools are developed as plug-ins for BEAM 3.6. This plug-in allows to export the pixel data of a rectangular region around all or the selected pin BEAM is the Basic ERS & Envisat (A)ATSR and MERIS Toolbox and is a collection of executable tools and an application programming interface (API) which have been developed to facilitate the use, viewing and processing of data of various sensors. BEAM is used for the validation of the GlobColour data in conjunction with the DDS Tools Table 6: Available tools In addition to these tools developed in the frame of the GlobColour project, and as GlobColour products become more and more popular, a reader has recently been added to the WIM tool (Windows Image Manager). WIM is a general-purpose image display and analysis program for the Microsoft® Windows™ operating systems with special features for analyzing satellite images. http://www.wimsoft.com/whats_new.htm 1. Added functions to deal with GlobColour binned and mapped files. GlobColour is an ESA project (http://www.globcolour.info) that produces global and local ocean color datasets from individual sensors or merged from multiple sensors. Section 20 in the WIM manual Wim.pdf has examples on using GlobColour global (FPS = Full Product Set) data that can be downloaded from (http://www.globcolour.info/data_access_full_prod_set.html. Local datasets (DDS) for many “diagnostic” data sites are available from http://www.globcolour.info/data_access_dds_list2.html. 2. Added a new WAM program wam_count that calculates a time series of the number of pixels with values either above or below a certain threshold. The time series can be produced for either whole images or for up to 255 specified areas of interest (masks). [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report 4 Page: 23 GLOBCOLOUR DATA SET 4.1. Full Product Set GlobColour products (1997-2007) distributed to the users are freely available at: http://www.globcolour.info/data_access_full_prod_set.html Global ocean colour data set at 4.6km resolution covering 1997-2007 daily, weekly, monthly products: • • • • • • • • • • • • Chlorophyll concentration (Chla) Diffuse attenuation coefficient @ 490nm (Kd490) Total Suspended Matter CDM absorption (aCDM443) Particle backscattering coefficient (bbp443) Aerosol Optical Thickness (T865) Exact normalised water-leaving radiance @ 412, 443, 490, 510, 531, 555, 620nm Water-leaving radiance @ 670, 681, 709nm Data quality flags Cloud fraction Excess of radiance at ~ 555 nm (turbidity index) (EL555) Error estimates per pixel for each layer MODIS-only, MERIS-only The products are derived as follows (see the PUG for further details): • The fully normalized water-leaving radiances (Lxxx) are derived using the weighted average. • The Chlorophyll (CHL1) is derived from GSM. The linked products, also output from the GSM model, are Coloured dissolved and detrital organic materials (CDM) and particulate back-scattering coefficient (bbp). • The diffuse attenuation coefficient at 490 nm, Kd(490), is derived from the merged chlorophyll. • The “excess of radiance” at 555 nm, EL555, is derived from the merged chlorophyll and the merged fully normalized water-leaving radiances. • The aerosol optical thickness at 865 nm, T865, is derived by simple averaging of the aerosol optical thickness provided by individual sensors at the same wavelength. • Total Suspended Matter (TSM), CDM and Case II Chlorophyll (CHL2) are also products from just the MERIS data. • Cloud Fraction (CF) based on classification and statistical methods. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 24 4.2. Diagnostic Data Set The following table shows the default GlobColour list of DDS where GlobColour products are systematically generated. Site-ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 Site-Name MOBY BOUSSOLE VeniceTower BATS CARIACO CALCOFI GulfOfMaine LEO15 Benguela Helgoland Channel Sopot Palmer RapaNui Concepcion TaiwanStr YellowSea AbuAlBukhoos GustavDalenTower HelsinkiLighthouse MVCO COVE NIVAFerryBox Azores CapeVerde Reserved Barbados Tahiti Nauru Okinawa MidwayIsland DongshaIsland Goa AmsterdamIsland Location Hawaii Ligurian Sea Adriatic Sea Sargasso S. Carib.Sea California USA-Canada New Jersey South Africa North Sea English Ch. Baltic Sea Antarctic S. Pacific Chile China China Persian (Arabian) Gulf Baltic Sea Gulf of Finland Cape Cod (or the Vineyard) Chesapeake Lighthouse-Virginia Skagerrak N. Atlantic N. Atlantic … N. Atlantic S. Pacific Eq Pacific N. Pacific N. Pacific S. China Sea Arabian Sea S. Indian Latitude 20.8 43.37 45.31 32 11 35 43 39 -32.5 54 50 55.2 -65 -23 -36.5 22.5 35 25.5 58.59 59.95 41.3 36.9 58.5 (1) 38.53 16.73 … 13.17 -17.57 -0.52 26.35 28.21 20.71 15.45 -37.81 Longitude -157.2 7.9 12.51 -64.5 -65 -125 -69 -74 17.4 7.5 -3 19 -65 -118 -73 118 122 53.15 17.47 24.93 -70.55 -75.71 10.5 (1) -28.63 -22.94 … -59.5 -149.61 166.92 127.77 -177.38 116.72 73.81 77.57 Table 7: List of the DDS sites GlobColour products are also systematically generated at the NIVAFerryBox site. On user request, other sites could become reference “DDS” for GlobColour. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Page: 25 Final Report 5 VALIDATION Validation is the process of determining the spatial and temporal error fields of a given biological or geophysical data product and includes the development of comparison or match-up data sets, i.e. field observations and satellite data coincident in time and location (Mueller et al., 1998). Validation of satellite data against in-situ measurements is generally known to be fraught with difficulties, and has to be addressed in a structured manner to obtain the right answers. One of the key questions when comparing satellite and in-situ observations is whether the conclusions drawn might be affected by the mismatches in the time and space scales of both the satellite and in-situ observations. Another issue to address is the systematic biases that have been observed when the same biological variables are estimated using different in-situ techniques or performed by different validation teams. The validation process that was undertaken within GlobColour was structured to facilitate identification of sources of uncertainties in retrieval algorithms. It was also designed to identify regional differences in the performance of algorithms, as well as seasonal differences. 5.1. Validation Protocol The Validation Protocol [RD9] aimed at giving the necessary background and methods for conducting validation of GlobColour level-3 products in an efficient and controlled manner while keeping high quality standards. It describes the planned validation protocol for the ocean colour (OC) products used and/or generated by the GlobColour project. The Validation Protocol reviews the standard practices employed by the ocean colour scientific community for validation of level-2 and level-3 products. An emphasis is given to the SIMBIOS validation protocol, which has already addressed and discussed many issues related to the validation of merged ocean colour products generated from SeaWiFS, MOS and MODIS-aqua.An extension of the SIMBIOS protocol to MERIS and tentatively PARASOL is presented while specific issues are discussed. Specific validation exercises inherent to the GlobColour project are presented, i.e., validation of the subset called the Preliminary product Set (PPS) and the Full Product Set (FPS). The document also discusses the reuirements requested by the GlobColour user group i.e. IOCCG, IOCCP and the operational oceanography community federated through the marine component of GMES. To summarise, the validation protocol addresses the following items: 1. Review SIMBIOS protocols and data quality standards: Review the SIMBIOS protocol and propose modifications or improvements if appropriate to fit the requirement of the GlobColour user groups. 2. Define limits of ocean regions: Review oceanic regions used in other ocean colour merging projects and propose adjustment or alternatives if appropriate. 3. Refinement of exclusion/selection criteria for match-up data: Propose exclusion/selection criteria for the match-up data set based upon experience, quality control, knowledge on variability in time and space of validation parameters. Eliminate – as far as possible- those in-situ data that were used to calibrate any of the input data sources. 4. Statistical framework: Define procedures to ensure a robust and unbiased statistical analysis for validation of the PPS and FPS. In particular, a plan was set up to prepare two independent validation datasets to be respectively used for the preliminary and the full validation phases, while keeping enough data to ensure the statistical robustness of the analysis. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 26 5. Identify possible problems, such as lack of sufficient validation data, asses the impacts and identify practical solutions 5.2. Validation Report The Full Validation Report [RD12] documents the validation of the quality of the Full Product Set (composed of 10 years of merged ocean colour data from SeaWiFS, MODIS and MERIS) according to the procedure specified in the Validation Protocol [RD9] and tested on the Preliminary Product Set [RD12]. The Full Validation Report also includes a comparison of GlobColour products against merged products developed within the NASA-REASoN project. 5.2.1 Open ocean water conclusions The objectives/questions addressed by the validation activities are: ¾ Are the overall geographical distributions valid in the merged data set? (e.g. any artificial boundaries?) ¾ Are the statistics derived from the match up analysis of the FPS with field data at least not worst (and hopefully better) than the individual-sensors’ statistics? ¾ Is the data set usable for delivery of operational services (GMES-MCS) and for “carbon cycle research”? ¾ Recommendations for the next steps We don’t discuss all parameters here. We selected: - the normalized water-leaving radiance at 412 nm, which is the most challenging to accurately retrieve with any ocean colour sensor - the normalized water-leaving radiance at 555 nm, which has in principle a well-known average value for clear waters (Gordon and Clark, 1981) - the chlorophyll concentration (GSM and weighted average) - the particulate backscattering coefficient at 443 nm and the CDM absorption, which are two new parameters not generally routinely produced by ocean colour missions (so they represent a true added value of GlobColour). The results are illustrated and discussed for the month of May 2006, when data from all three sensors are available.The first observation is the excellent coverage of the monthly composites. The “observable area”, i.e. part where the sun zenith angle remains within the acceptable limit for the processing to perform well, is nearly entirely covered when the monthly temporal scale is considered. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 27 Figure 4: Global composite of L412 for May 2006 The L412 composite (Figure 4) clearly shows the oligotrophic gyres, in particular in the southeast and northwest Pacific, with the highest values. The lowest values correspond to coastal seas where the influence of CDOM is important (e.g. the Baltic Sea). Equatorial zones have moderate values, corresponding to moderate phytoplankton concentrations. Coastal upwelling areas also clearly appear. The overall distribution is, therefore, as expected. Figure 5: Global composite of L555 for May 2006 The L555 composite (Figure 5) shows a very homogeneous distribution, which is expected because this wavelength is close to the “hinge point” of the ocean reflectance (Clarke et al., 1970); around 510-520 nm and is the part of the e.m. spectrum where the reflectance is nearly constant whatever the chlorophyll concentration (at least varying minimally). Exceptions to this are observed where coccolithophore blooms are known to occur, such as the north Atlantic and [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 28 the Bering Sea (e.g. Brown, 1995). Other areas of high L555 correspond to turbid Case 2 coastal waters (Black Sea, North Sea and the English Channel, north of Australia etc). Figure 6: Global composite of CHL1 (weighted average) for May 2006 The CHL1 distribution (Figure 6) clearly matches the expectations and doesn’t exhibit any obvious artefact (except, again, in coastal waters where the used algorithms are not expected to provide accurate results). Figure 7: Global composite of CHL1 (GSM) for May 2006 [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 29 The comments made for the CHL1 from the weighted average can be repeated for the CHL1 from the GSM method (Figure 7). The difference lies in the overall smaller concentrations, in particular in regions of high chlorophyll concentration (e.g. coastal upwellings). This is inherent to the GSM model. In conclusion, the examination of global composites (other products that are not shown here have also been scrutinised) didn’t reveal any obvious flaws or artefacts. The range of values is within expectations and the distributions are coherent with the general oceanographic knowledge. The spatial coverage at the monthly temporal scale is excellent. The match-ups with the in-situ data base provide the quantitative evaluation of the GlobColour FPS. The full statistics are provided in [RD12], and only the main observations from these statistics are provided here: • Correlation coefficients are usually > 0.7, except for CDM and AOT865. • The median percent differences for the normalized-water leaving radiances are between 12% (490 nm) and 21% (412 nm). • The RMS is increasing for the normalized water-leaving radiances from the green to the blue. This is expected considering that atmospheric correction errors increase in the same direction (at least with the algorithms in use for all present sensors). • The biases are small for the normalized water-leaving radiances, and systematically negative (median ratios slightly below 1). • In the blue, the smallest intercept is observed at 412 nm. One important question here is whether or not the GlobColour FPS performs better than the data sets of the three sensors included in the merging process. To answer this question, we have compared the match-up statistics of the three individual sensors and the match-up statistics for the GlobColour FPS. The changes from the “individual statistics” to the “merged statistics” are summarized in the table below: ++ Means that the statistics for the parameter in question (slope, intercept etc…) are better for the FPS than for any of the 3 individual-sensor statistics. + Means that the FPS statistics are better than at least two of the 3 individual-sensor statistics, and similar for the third one = Means that there is no significant difference between the FPS statistics and the three individual-sensor statistics - Means that the FPS statistics are worst than at least 2 of the individual-sensor statistics. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Page: 30 Final Report Slope Intercept R2 Mean ratio Median ratio Mean % diff Median % diff Bias RMS L(412) ++ ++ ++ + + ++ + + + L(443) ++ ++ ++ + + + ++ + ++ L(490) ++ + ++ + + + + = = L(555) ++ + + + - ++ + ++ ++ L(670) - + - + - + - = - Chl (AVW) + + + - - + + = + Kd(490) ++ ++ ++ + = ++ ++ = ++ AOT(865) - - - = + + + = - Table 8: Match-up statistics Performance of GlobColour data set From this table we can say that: - Overall, the statistics for the GlobColour data set are better than for the three individual sensors data sets. - This observation is particularly true for the blue bands (412 to 490 nm), while the result for the red band (670 nm) doesn’t show any improvement. - The results are degraded, however, for the aerosol optical thickness. In this is a peculiar case as all three sensors overestimate the AOT, so that the merged data set also overestimates AOT1. The results in this table clearly show that the merging process has improved the overall statistical fit with in situ data. However, the exact reason for this is not totally clear. A small part of the improvement may simply come from the higher number of match-up points when validating the merged data set; however, this cannot be a major cause of the improvement because the number of match-up points is already large when deriving the three individual sensors statistics. Another reason is the compensation of overestimations with underestimations by the three sensors. For instance, at 412 nm, MERIS is overestimating the normalized water-leaving radiance whereas SeaWiFS and MODIS-A underestimate it (MODIS-A in particular). Combining the three data sets nearly offsets the bias and leads to a slope close to 1. These observations raise the issue of the quality of the individual sensors data sets that enter into the merging process. The good results obtained here should not mask the obvious: the individual sensors data sets are still not at the desired level of uncertainty and accuracy. 1 It should be noted that producing a merged AOT product is not the focus of GlobColour and the validation approach was based on in-situ rather than atmospheric products. The AOT information is provided solely for use as ancillary data quality information. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 31 The FPS has been validated with a global data set of field data (nLw’s and Chl). The statistics for the nLw blue bands (412 and 443 nm) are improved compared to the individual sensors’ data sets. At 490 nm, nLw is by far the most homogeneous product among the 3 sensors and so confidence in the merged product is higher for this waveband. The MODIS-A L555 is often smaller than the L555 for the two other sensors. For Chl, the statistics for the GSM Chl are a little better than those for the weighted average product. The time series of the chlorophyll anomalies, however, are closer to previously published data sets (i.e. Behrenfeld et al, 2006) for the weighted average chlorophyll. Overall, the merged product has not degraded the situation as compared to the single sensors’ data sets. However, good results can result from compensating effects, in particular between MODIS-A and MERIS, and the FPS is often close to the SeaWiFS data set alone. Therefore, the FPS is definitely qualified and usable for e.g. assimilation into global models (there is a pixel-bypixel error bar delivered with the GSM Chl)2. It is not yet qualified to perform temporal analysis over the period 1998-2007 i.e. it doesn’t yet meet the standards for being qualified as a “climate quality data record”. However, qualification of trends/cycles has already been initiated by the GlobColour team and first results were shown at the second GlobColour User Workshop (see [RD15]). 5.2.2 Coastal case 2 water conclusions The main questions the validation is supposed to answer are: o Are the GlobColour products valid/usable in coastal waters o Are the statistics derived from the match up analysis of the FPS with in-situ data at least not worst (and hopefully better) than the individual-sensors’ statistics? o Does the merging process improve the estimation of chlorophyll content in coastal case 2 waters? The best results are from the Ferrybox data set. Although not originally designed for Case II waters, the validation shows that the GlobColour GSM01 merging algorithm seems to be fairly robust when applied to coastal waters. Therefore, GlobColour merged products have potential for investigating seasonal to inter-annual relative variability in the coastal zone. However, validation of merged case I products in coastal Case II waters presents the same limitations as with non-merged products, and therefore large errors. In addition, the GlobColour FPS was not designed for coastal waters i.e. it has a relatively coarse resolution and the application of Case I techniques. Improvements also need a better in-situ dataset for coastal case II waters. - We need several in situ long-term time series in order to add the temporal dimension to our statistical analyses (match ups). - An international effort is still needed to standardise and improve the vicarious calibration methodologies. We are still not at the desired level of confidence to answer “climate questions” (see, e.g., Ohring et al., EOS Trans AGU, 88(11), 13 March 2007) 2 The quality of the data set is sufficient to meet the needs of operational users, but actual implementation would depend on negotiating suitable data access agreements with the data providers. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 32 - Establish a collaborative frame between space Agencies (ESA, NASA and others), so that vicarious calibration and related issues (atmospheric correction) can be standardised. This may need a specific body where these issues are discussed and the methods are implemented (see, e.g., the Global High-Resolution Sea Surface Temperature (SST) Pilot Project, GHRSST-PP). - Incorporate new approaches, for instance where the TOA level-1 observations of all instruments are processed the same way (same algorithm), which also means that they are all vicariously calibrated against the same standard. It might become obvious at some point that this approach is mandatory if one wants to get to CDRs (“climate quality data records”). [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Page: 33 Final Report 6 USER ASSESSMENT Users have always been involved during the project for consultation, advice and final assessment. The key rendezvous’ are listed in the table below. KO+2: Jan. 2006 Generation and review of Requirements Baseline Document, Design Justification File, and Validation Protocol KO+8: July 2006 Participation in the Critical Design Review KO+13: Dec. 2006 User assessment of the Preliminary Product Sets (radiance and biophysical) and selection of final product Participation in the end of phase 1 review (advisors to ESA) and GlobColour User Workshop 1 (LOV) KO+16: Mar. 2007 User meeting at UK Met Office KO+20: July 2007 User meeting at University of East Anglia KO+23: Oct. 2007 User assessment of the Final Product Set (10 years) KO+24: Nov. 2007 Participation in the GlobColour User Workshop 2 (NIVA) KO+36: Nov. 2008 Participation in the GlobColour User Workshop 3 (ESRIN) Table 9: User involvement key rendezvous’ 6.1. First user consultation The GlobColour precursor service was presented to the combined GlobColour / Medspiration community at a meeting in Villefranche (France) on the 4-6 December 2006. Figure 8: First GlobColour User consultation – Villefranche – 4-6 December 2006 [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 34 6.1.1 Merging recommendations The merging recommendations that arose from the workshop are: Normalized water-leaving radiances: - Statistics are slightly better when using the weighted average than the simple average - Use of the weighted average for the nLw’s Chlorophyll: - GSM01 provides the best fit to in-situ chlorophyll - It has the advantage of providing other products - Pixel-by-pixel error bars can be provided in the future - Produce also weighted average Chlorophyll 6.1.2 Meeting summary - Need to strengthen/broaden the ocean colour community. - Synergies with Medspiration on DDS, formats and metadata. - Need for an ocean colour equivalent of GHRSST? - New user requirements, e.g. PAR, Case 2, warming depth, Secchi disk, to be monitored and new (useful) products to be included as research provides suitable validated methods. - Utility of GlobColour merged nLw for PFTs. - Intercomparison with other merging methods. - New dissemination methods (Google-Earth). - Rigour, uniformity and honesty in error statistics – need to develop an operational & centralised quality control approach? - Need for coordinated validation efforts within Europe - Development of multi-disciplinary integrated (SST + OC + …) data sets for model assimilation, seasonal forecasting, etc - Need to prepare for a probable gap between ENVISAT and Sentinel-3 data supply, and how to manage the impact of this gap on the users. [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 35 6.2. Second user consultation The GlobColour FPS and project as a whole was presented to the combined GlobColour / Medspiration community at a meeting in Oslo (Norway) on the 20-22 November 2007. Figure 9: Second GlobColour User consultation – Oslo – 20-22 November 2007 Positive feedback was received from the combined user community as summarised below. 1. GlobColour has produced a 10 year set of merged products for MERIS, MODIS and SeaWiFS. The dataset contains an extensive list of products using various merging techniques. IOCCG feedback: We still lack sufficiently long satellite time series to sort out differences between cycles and trends (i.e. some ocean cycles, e.g. the Pacific Decadal Oscillation, are longer than a decade); we need to sustain an international effort to make sure we can link one satellite data set to another to build the long time series that we need to distinguish change from cycles; and GlobColour is definitely a significant step in that perspective. IOCCP comment: This 10-year dataset is going to be very useful for carbon studies and global modelling. 2. From the Phase 2 validation and an intercomparison of merged products we believe that the results are as good as comparable products. IOCCP comment: Yes, and the error bars are also very useful for modellers. 3. The products are available to download via the WWW and a near-real time (NRT) service is starting; the NRT service has been running for the DIVERSITY project. IOCCG comment: IOCCG supports efforts to make ocean color data and imagery easily available via the web and FTP, and we are very pleased to see the GlobColour products easily accessible. UKMO comment: NRT delivery, due to start in 2008, would be a good test for the timeliness and reliability of the merged product. The Met Office is in a unique position to test this aspect of GlobColour given the right funding is in place. 1. A coastal water extension and additional products are under consideration. IOCCG comment: IOCCG recognizes the importance of coastal water sensing and the difficulty of doing so as discussed in IOCCG Reports #3 and #5. A global coastal water product which included 300-m resolution MERIS data (when available) perhaps embedded within merged 1km [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 36 MERIS, SeaWiFS and MODIS imagery would be a unique and useful data set for coastal research and applications and well received by the international community. IOCCP comment: Phytoplankton type might become an additional product. This is interesting for carbon studies because the various types do not have the same efficiency with respect to carbon fixation and export. 6.3. Potential extensions The following extensions have been discussed at the workshop: 1. Coastal zones a. Addition of global coastal 1km products such as Case 2 Chl, CDOM, etc - perhaps only based on MERIS products b. Global Case 2 CHL, CDM, TSM, … product for assessing role of coastal zones in carbon flux c. Use of MERIS-FRS for operational applications (i.e. incl. NRT Level-2 availability) d. NOAA IOOS (and CoastWatch) would like a coastal GlobColour (NASA & ICESS to be involved!) e. AMESD users..? & LME’s in Africa. 2. Assimilation of GlobColour products into models. 3. Inclusion of PARASOL as 4th sensor, + ISRO sensors 4. Need for cloud-free analysed Chl product (DIVERSITY, are there others?) 5. Need for new products: Primary productivity, pCO2, PAR, Secchi disk depth, heated layer depth, phytoplankton Functional Types … 6. Support IOCCP Ocean CO2 from Space workshop 7. Distribute ocean colour products to African users via ESA/DDS 8. Additional hardware and consumables for wider distribution of the GlobColour products and web access [email protected] www.globcolour.info ACRI-ST LOV BC DLR ICESS NIVA UoP Ref: GC-MA-ACR-FR-01 GlobCOLOUR : Date: February 13, 2008 An EO based service supporting global ocean carbon cycle research Issue: 1 Rev. 0 Final Report Page: 37 7 FAQ 1. Which chlorophyll algorithm does GlobColour use? We use the standard algorithms as inputs, but these are converted to a “MERIS-like” algorithm in the merged averaged weighted product. However, conversions are provided in the Product Guide so that this can be converted to a “SeaWiFS-like” or “MODIS-like” product. For the GSM merged products the nLw’s are inputs and the GSM inverse IOP model chlorophyll is the output. 2. The chlorophyll and Kd products will not be accurate in coastal waters, how is GlobColour dealing with this? In turbid Case 2 waters the EL555 flag will flag these pixels and the GSM merging should be able to cope, at least partially, with the influence of CDOM. In GlobColour these potentially less accurate pixels are not removed, but flagged and given a larger error bar. The validation has shown that the GSM approach is performing better than the standard chlorophyll product in coastal waters and so provides a step in the right direction. 3. Has SeaDAS been taken into account when designing the GlobColour tools? BEAM was selected as the viewer for the GlobColour products. Therefore, SeaDAS doesn’t have the capability to read GlobColour data, but this could be a future addition if there was a sufficient user requirement. In addition, routines for reading GlobColour products will be made available on the GlobColour WWW site. 4. Is GlobColour’s metadata compliant with climate metadata standards? Particular attention should be paid to ISO19115. GlobColour has followed the Medspiration netCDF format and consulted with data centres so as to be as compliant as possible. 5. What are the main differences between the GlobColour and REASoN datasets? Within GlobColour the GSM merging accounts for nLw input uncertainties and is produced at 4.63km resolution, while in REASoN it is not weighted and at 9km resolution. The uncertainities are produced by a forward version that is implemented after merging so that the nLws can be regenerated and checked against the original nLws for quality control (residual monitoring). 6. How to access GlobColour data? Load the following link to any browser and click on "Data Access" http://www.globcolour.info/ To access either the Full Product Set or the Diagnostic Data Sets. Please refer to the PUG [RD14] for all details on how to download, read and display products. If you are still uncertain please contact the GlobColour project for assistance: [email protected] www.globcolour.info [email protected] www.globcolour.info
