Download Presentation - National e

European Grid of Solar Observations
EGSO
A Virtual Observatory for solar and
heliospheric physics
Rob Bentley (UCL/MSSL), Isabelle Scholl (ISU)
and the EGSO Team
Super Computing 2004
Pittsburgh, 7-12 November 2004
http://www.egso.org
EGSO – European Grid of Solar Observations
European Grid of Solar Observations
EGSO is a Grid test-bed related to a particular application
Addresses the generic problem of linking a distributed
heterogeneous data set, and serving a scattered user community
Designed to improve access to solar data for the solar physics and
other communities by establishing a virtual observatory
Funded under the Information Society Technologies (IST)
thematic priority of the EC’s Fifth Framework Program (FP5)
Objectives include:
Building enhanced search capability for solar data
Support of user community scattered around the world
Provide enhanced access to data archives and observatories
around the world
o
Where possible, provide ability to process data at source
The Solar Virtual Observatory family
European Grid of Solar Observations
Partners and collaborators provide expertise in solar physics and IT
UK
University College London: UCL-MSSL (PI) & UCL-CS,
France
Institut d’Astrophysique Spatiale (Orsay),
Obs. de Paris-Meudon, International Space Univ. (Strasbourg)
Istituto Nazionale di Astrofisica, Politecnico di Torino
Italy
Switzerland
US
Rutherford Appleton Laboratory, University of Bradford
INAf includes Obs. of Turin, Trieste, Florence and Naples
University of Applied Sciences (Aargau)
Solar Data Analysis Center at NASA-GSFC (Greenbelt, MD),
National Solar Observatory (Tuscon, AZ)
SDAC and NSO are also part of the US VSO
Belgium
Netherlands
Royal Observatory of Belgium
ESA-ESTEC – Solar Group
US
VSO:
Stanford University, Montana State University
CoSEC: Lockheed-Martin
VSPO:
LEP at NASA-GSFC
(Lab. Extraterrestrial Physics)
16JAN03
Use of solar observations
European Grid of Solar Observations
The appearance of the Sun changes
dramatically with wavelength
31JAN03
Heliosphere
Emissions originate from different
layers in the atmosphere and different
physical phenomena
2x106 K
For a complete picture we need to
use as wide a range of observations
as possible
Mixture of multi-wavelength
observations from space- and
ground-based platforms
Identifying observations that match
some User search criteria and then
retrieving them are major problems
8x104 K
6x103 K
Corona
Chromosphere-TR
Photosphere
Surface Magnetic Field
Linking solar data into a wider context
European Grid of Solar Observations
Increasing desire to use solar
data in study problems that
span communities
Space weather
o
heliosphere, magnetosphere,
ionosphere…
Climate physics
Planetary physics
Astrophysics
Need to find ways of tying the
data from different communities
together
Single data model covering all
solar system not practical
Intersecting data models in a
general pool should be possible
European Grid of Solar Observations
Space Weather affects human activities
Electrical power distribution
Long-line telephone
communications
Pipeline operations
Geophysical exploration
Navigation
HF radio Communications
High-altitude flights
Satellite operations
Space Shuttle and Space Station
activities
The Generic Query
Identify suitable observations
(many serendipitous)
Want to access as many different types of data as are available
European Grid of Solar Observations
o
o
Identification should be possible without accessing the data
Data volumes are increasing rapidly - SDO will produce 2 TB/day
User only wants to know if data addressing a problem exists
Locate the data
Data scattered, with differing means of access
o
(some proprietary)
Large and small data providers, with varying resources
Process the data
Involves extraction and calibration of a subset of raw data
o
Often only need a subset of each data set
Return results to the User
Compare results from different instruments
SolarSoft (IDL) provides a standard platform for analysis
EGSO is placing a lot of emphasis on the first item!
The EGSO Search Engine
EGSO is improving the quality and availability of metadata
European Grid of Solar Observations
Enhanced cataloguing describes the data more fully
Standardized versions of observing catalogues (UOC) tie together
the heterogeneous data sets
Search Registry, an abstraction of entries in the UOC, enables
narrowing of the search in initial stages
New types of catalogue allow searches on events, features
and phenomena, not just date & time, pointing, etc…
Solar Event Catalogue (SEC) - derived from published lists
Solar Feature Catalogue (SFC) - generated by feature recognition
Ancillary data used to assist the User in the search
QLK Server provides Phone book access to images, etc.
DSO Server gives Yellow Page information on instruments
Similar hierarchical cataloguing techniques are required in
other Grid projects handling diverse data sets
Catalogue relationships
European Grid of Solar Observations
SR
DSO
Time coverages
- instrument
- observing date start
- observing date end
- observing parameter name
- observing parameter value
- data source
Built from
Objective of the improved metadata,
etc. is to be able to pose questions like:
Database of solar observations
- instrument
- observatory
- EGSO available?
- observing location
- observing interval
- description ….
Identify events when a filament eruption
UOC
occurred within 30° of the north-west
Solar Observations
Manually
- date start
limb andBuilt
there were good observations
- date end
- wavelengths
in H, EUV and soft X-rays
- coordinates
SEC/SFC
- ….many more
QKL
relevant characteristics
Event/Feature Catalogs
-
catalog name
event name
observing date
description ….
Data
Archives
Built from
needed for searches
Simplified Architecture
Results
ARCHIVES
European Grid of Solar Observations
EGSO GRID
Consumer
supports GUI
and GUI
API access
Cat.
Provider
Consumer
Broker
Provider
GUI
Consumer
GUI
API
Broker
Brokers manage the
metadata and decides
and allocates
resources
Provider
Architecture defined in terms of three roles:
Consumer, Broker and Provider
Archive access
can be by
FTP, HTTP,
Web Services,
cgi-bin…
through adaptor
modules
SEC, SFC,
UOC, DSO,
QLK, CoSEC
Special Providers
After R. Linsolas, IAS
The UOC and Search Registry
European Grid of Solar Observations
Unified Observing Catalogue (UOC)
Unified form of observing catalogues that tie together the heterogeneous
data, leaving the data unchanged
o Increase interoperability by using “standard” coordinate formats
Self describing, with dependencies on ancillary data or proprietary
software removed (and with any errors corrected)
The
logic behind the UOC addresses a
Standards defined for future data sets (e.g. STEREO, ILWS, Solar-B)
universal problem - it allows the observations
Search
Registry
is an abstraction
of entries
in the UOC way
to be
described
in more
interoperable
It reduces interactions with Providers without data matching the search
The Registry allows
identify
instruments
that:
Similarities
in the
theBroker
wayto we
use
data means
o have data properties matching the search - Static SR
that
the static Search Registry can be used to
o probably have observations during search time interval - Dynamic
tie solar
and heliospheric data together.
SR
Planetary
data iscould
easilyaccess
be added…
Static Search Registry
able to support
to different types of data
from solar and heliospheric observations
o Describes instrument capabilities & observing objectives
o First step in search - later steps handled br dSR and within other VOs
o EGSO sSR includes instruments on Ulysses, ACE, Cassini, SDO, STEREO…
Locating the data
Data could be located anywhere in the world
European Grid of Solar Observations
User only needs to know observations exist, not where located
System should isolate the user from the intricacies of access
System should be able to optimize use of sources
Choice of source - most capable, closest, least used, etc.
Handles replicated data and aggregated sources
Must respect any use policies and ensure data integrity
Burden on data providers minimized to encourage participation
In EGSO, data sources are interfaced by the Provider Role
Broker uses Data Registry to formulate requests to data sources
o
o
Which instrument data sets are hosted by each data archive
Which data archives interfaced to each Provider Role
Provider Role uses adaptor modules to manage access protocols
o
o
Standardizes way data source interface appears to the system and
simplifies addition of new data sources
Also allows access to data “hosted” by other VOs
Query Work Flow
European Grid of Solar Observations
User specifies query through the GUI or API
Static Search Registry narrows the search, based on the criteria
specified in the query
o
o
Identifies instruments that make the desired type of observations
Search can includes solar, heliospheric, etc… instruments
Dynamic Search Registry determines (at some granularity) which
were actually observing
o
Includes pointing, observatory location…
List of instruments identified as possibly having data returned to
the User who refines selection
Data Registry used to locate archive holding the instrument data
and make data request
List of files returned that can be retrieved directly, used to
generate data products, etc.
Convert to processed products if required
Summary
European Grid of Solar Observations
EGSO has become a principal player in the virtual
observatories covering solar and heliospheric physics
Already able to exchange of information and resources
between the solar VO projects
Approaches of the solar VOs very complementary
Links into the space plasma VOs developing rapidly
The layered approach to metadata greatly simplifies
interoperability with other VOs
Similar techniques applicable to similar Data Grid
Demonstration of capabilities at the EGSO stand
http://www.egso.org
European Grid of Solar Observations
QuickTime™ and a
Sorenson Video decompressor
are needed to see this picture.