Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
European Grid of Solar Observations EGSO A Virtual Observatory for solar and heliospheric physics Rob Bentley (UCL/MSSL), Isabelle Scholl (ISU) and the EGSO Team Super Computing 2004 Pittsburgh, 7-12 November 2004 http://www.egso.org EGSO – European Grid of Solar Observations European Grid of Solar Observations EGSO is a Grid test-bed related to a particular application Addresses the generic problem of linking a distributed heterogeneous data set, and serving a scattered user community Designed to improve access to solar data for the solar physics and other communities by establishing a virtual observatory Funded under the Information Society Technologies (IST) thematic priority of the EC’s Fifth Framework Program (FP5) Objectives include: Building enhanced search capability for solar data Support of user community scattered around the world Provide enhanced access to data archives and observatories around the world o Where possible, provide ability to process data at source The Solar Virtual Observatory family European Grid of Solar Observations Partners and collaborators provide expertise in solar physics and IT UK University College London: UCL-MSSL (PI) & UCL-CS, France Institut d’Astrophysique Spatiale (Orsay), Obs. de Paris-Meudon, International Space Univ. (Strasbourg) Istituto Nazionale di Astrofisica, Politecnico di Torino Italy Switzerland US Rutherford Appleton Laboratory, University of Bradford INAf includes Obs. of Turin, Trieste, Florence and Naples University of Applied Sciences (Aargau) Solar Data Analysis Center at NASA-GSFC (Greenbelt, MD), National Solar Observatory (Tuscon, AZ) SDAC and NSO are also part of the US VSO Belgium Netherlands Royal Observatory of Belgium ESA-ESTEC – Solar Group US VSO: Stanford University, Montana State University CoSEC: Lockheed-Martin VSPO: LEP at NASA-GSFC (Lab. Extraterrestrial Physics) 16JAN03 Use of solar observations European Grid of Solar Observations The appearance of the Sun changes dramatically with wavelength 31JAN03 Heliosphere Emissions originate from different layers in the atmosphere and different physical phenomena 2x106 K For a complete picture we need to use as wide a range of observations as possible Mixture of multi-wavelength observations from space- and ground-based platforms Identifying observations that match some User search criteria and then retrieving them are major problems 8x104 K 6x103 K Corona Chromosphere-TR Photosphere Surface Magnetic Field Linking solar data into a wider context European Grid of Solar Observations Increasing desire to use solar data in study problems that span communities Space weather o heliosphere, magnetosphere, ionosphere… Climate physics Planetary physics Astrophysics Need to find ways of tying the data from different communities together Single data model covering all solar system not practical Intersecting data models in a general pool should be possible European Grid of Solar Observations Space Weather affects human activities Electrical power distribution Long-line telephone communications Pipeline operations Geophysical exploration Navigation HF radio Communications High-altitude flights Satellite operations Space Shuttle and Space Station activities The Generic Query Identify suitable observations (many serendipitous) Want to access as many different types of data as are available European Grid of Solar Observations o o Identification should be possible without accessing the data Data volumes are increasing rapidly - SDO will produce 2 TB/day User only wants to know if data addressing a problem exists Locate the data Data scattered, with differing means of access o (some proprietary) Large and small data providers, with varying resources Process the data Involves extraction and calibration of a subset of raw data o Often only need a subset of each data set Return results to the User Compare results from different instruments SolarSoft (IDL) provides a standard platform for analysis EGSO is placing a lot of emphasis on the first item! The EGSO Search Engine EGSO is improving the quality and availability of metadata European Grid of Solar Observations Enhanced cataloguing describes the data more fully Standardized versions of observing catalogues (UOC) tie together the heterogeneous data sets Search Registry, an abstraction of entries in the UOC, enables narrowing of the search in initial stages New types of catalogue allow searches on events, features and phenomena, not just date & time, pointing, etc… Solar Event Catalogue (SEC) - derived from published lists Solar Feature Catalogue (SFC) - generated by feature recognition Ancillary data used to assist the User in the search QLK Server provides Phone book access to images, etc. DSO Server gives Yellow Page information on instruments Similar hierarchical cataloguing techniques are required in other Grid projects handling diverse data sets Catalogue relationships European Grid of Solar Observations SR DSO Time coverages - instrument - observing date start - observing date end - observing parameter name - observing parameter value - data source Built from Objective of the improved metadata, etc. is to be able to pose questions like: Database of solar observations - instrument - observatory - EGSO available? - observing location - observing interval - description …. Identify events when a filament eruption UOC occurred within 30° of the north-west Solar Observations Manually - date start limb andBuilt there were good observations - date end - wavelengths in H, EUV and soft X-rays - coordinates SEC/SFC - ….many more QKL relevant characteristics Event/Feature Catalogs - catalog name event name observing date description …. Data Archives Built from needed for searches Simplified Architecture Results ARCHIVES European Grid of Solar Observations EGSO GRID Consumer supports GUI and GUI API access Cat. Provider Consumer Broker Provider GUI Consumer GUI API Broker Brokers manage the metadata and decides and allocates resources Provider Architecture defined in terms of three roles: Consumer, Broker and Provider Archive access can be by FTP, HTTP, Web Services, cgi-bin… through adaptor modules SEC, SFC, UOC, DSO, QLK, CoSEC Special Providers After R. Linsolas, IAS The UOC and Search Registry European Grid of Solar Observations Unified Observing Catalogue (UOC) Unified form of observing catalogues that tie together the heterogeneous data, leaving the data unchanged o Increase interoperability by using “standard” coordinate formats Self describing, with dependencies on ancillary data or proprietary software removed (and with any errors corrected) The logic behind the UOC addresses a Standards defined for future data sets (e.g. STEREO, ILWS, Solar-B) universal problem - it allows the observations Search Registry is an abstraction of entries in the UOC way to be described in more interoperable It reduces interactions with Providers without data matching the search The Registry allows identify instruments that: Similarities in the theBroker wayto we use data means o have data properties matching the search - Static SR that the static Search Registry can be used to o probably have observations during search time interval - Dynamic tie solar and heliospheric data together. SR Planetary data iscould easilyaccess be added… Static Search Registry able to support to different types of data from solar and heliospheric observations o Describes instrument capabilities & observing objectives o First step in search - later steps handled br dSR and within other VOs o EGSO sSR includes instruments on Ulysses, ACE, Cassini, SDO, STEREO… Locating the data Data could be located anywhere in the world European Grid of Solar Observations User only needs to know observations exist, not where located System should isolate the user from the intricacies of access System should be able to optimize use of sources Choice of source - most capable, closest, least used, etc. Handles replicated data and aggregated sources Must respect any use policies and ensure data integrity Burden on data providers minimized to encourage participation In EGSO, data sources are interfaced by the Provider Role Broker uses Data Registry to formulate requests to data sources o o Which instrument data sets are hosted by each data archive Which data archives interfaced to each Provider Role Provider Role uses adaptor modules to manage access protocols o o Standardizes way data source interface appears to the system and simplifies addition of new data sources Also allows access to data “hosted” by other VOs Query Work Flow European Grid of Solar Observations User specifies query through the GUI or API Static Search Registry narrows the search, based on the criteria specified in the query o o Identifies instruments that make the desired type of observations Search can includes solar, heliospheric, etc… instruments Dynamic Search Registry determines (at some granularity) which were actually observing o Includes pointing, observatory location… List of instruments identified as possibly having data returned to the User who refines selection Data Registry used to locate archive holding the instrument data and make data request List of files returned that can be retrieved directly, used to generate data products, etc. Convert to processed products if required Summary European Grid of Solar Observations EGSO has become a principal player in the virtual observatories covering solar and heliospheric physics Already able to exchange of information and resources between the solar VO projects Approaches of the solar VOs very complementary Links into the space plasma VOs developing rapidly The layered approach to metadata greatly simplifies interoperability with other VOs Similar techniques applicable to similar Data Grid Demonstration of capabilities at the EGSO stand http://www.egso.org European Grid of Solar Observations QuickTime™ and a Sorenson Video decompressor are needed to see this picture.