Download User Access to Solar-B Data - National e

Solar-B Global DataGrid
21st September 2006
UK e-Science All Hands Meeting 2006
RAL
Tim Folkes
Jens Jensen
Matthew Wild
Elizabeth Auden
MSSL
Elizabeth Auden
Paul Lamb
Matthew Whillock
Len Culhane
21 September 2006
e-Science AHM 2006
Overview
The Solar-B mission will be launched tomorrow, Friday 22 September
2006, from the Uchinoura Space Center in Japan.
Mission duration: 3 years nominal, +10 years lifetime expected
Scientists and software engineers at MSSL and RAL have designed the
Solar-B global data grid to provide mission data to solar research
communities in the UK and abroad.
•
•
•
•
•
Solar-B science objectives
Data transfer and processing
Data storage
User access to data
Data security
Elizabeth Auden
21 September 2006
e-Science AHM 2006
Solar-B Mission Overview
EIS - MSSL/NRL EUV Imaging Spectrometer
SOT - ISAS/NAOJ Solar Optical Telescope
XRT - SAO/ISAS X-ray Telescope
FPP - Lockheed/NAOJ Focal Plane Package
Elizabeth Auden
21 September 2006
e-Science AHM 2006
Solar-B Science
Key Science Questions
1. How is the solar corona
heated? Are active regions
heated in a different way to
the quiet Sun?
2. What causes solar flares
and coronal mass
ejections?
3. How is energy transferred
in the quiet Sun? How are
solar magnetic fields
affected?
Solar events like flares and coronal mass ejections can have a profound affect on the Earth’s
magnetosphere. A geomagnetic storm in 1989 triggered a blackout of the Hydro Quebec power system.
Elizabeth Auden
21 September 2006
e-Science AHM 2006
Transfer of Solar-B Data to ADS
Data Migration Facility
Manages NFS area
ISAS
Japan
Satellite data
LL data
ADS
RAL
LL data
processing
LL
HL data
MSSL
HL
Solar-B data transferred via GridFTP
from ISAS and MSSL to an NFS
mounted disk at ADS.
Elizabeth Auden
21 September 2006
• Produced by SGI
• User transparent file
migration
• Licence for 500Tbytes
storage
• Backend: STK SL8500
– 10,000 tape slots
– STK 9940B drives
(200GB 30MB/sec)
– STK T10000 drives
(500GB 120MB/sec)
e-Science AHM 2006
ADS Data Storage
The DMF monitors the Solar-B NFS
area and removes disk copies of data to
keep free space at required level.
ADS
Interface
NFS
DMF STK SL8500 robot
Hardware
• Based on Altix 350
• Numaflex architecture
• 40 Tbytes raid disk
Network
• Local server: 1Gbit network
• Part of 10Gbit back-bone
• Coming soon: SJ5 10Gbit link
Elizabeth Auden
Tape
Small
Larger
files
fileshave
haveprimary
primarycopy
on
copy
disk.
onhave
(Speeds
tape.secondary
up
All
files
read/writes.)
copy on tape. These are
periodically moved to the
Firesafe
firesafe.
21 September 2006
e-Science AHM 2006
User Access to Solar-B Data
Instrument Data
SOT: Solar Optical Telescope
Magnetic field images
Optical images
Magnetic field velocity
measurements
DMF
XRT: X-Ray Telescope
X-ray coronal images
EIS: EUV Imaging Spectrometer
Coronal line strengths and
profiles
Plasma diagnostics
Files via URLs and DMF system at RAL’s UKSSDC
Users access data as URLs through the AstroGrid
workbench, STAP web services, browsers, or wget.
Elizabeth Auden
21 September 2006
e-Science AHM 2006
Accessing Solar-B data through AstroGrid
AstroGrid
workbench
INPUT: ADQL queries
AstroGrid
DSA
OUTPUT: FITS keywords, URLs
DSA: DataSet Access
ADQL to SQL
AstroGrid
HelioScope,
STAP enabled
tools
AstroGRID
PLASTIC tools,
Solar-B website,
wget, browser
Elizabeth Auden
INPUT: start / stop dates
OUTPUT: URLs
AstroGrid
STAP service
MySQL
FITS keywords,
Data URLs
STAP: Simple Time
Access Protocol
INPUT: URLs
OUTPUT: Solar-B files
21 September 2006
ADS
Web-visible
file system
e-Science AHM 2006
AstroGrid User Components
AstroGrid workbench: STAP service access through
HelioScope, DSA access through the Task Launcher.
Task Launcher: locate DSAs, submit ADQL queries
Workbench:
http://www2.astrogrid.org/desktop/index
STAP: Simple Time Access Protocol
http://personalpages.manchester.ac.uk/staff/s.dalla/astrogrid/strap.html
DSA: DataSet Access
http://www.astrogrid.org/maven/docs/HEAD/pal/index.html
ADQL: Astronomical Data Query Language
HelioScope: solar data results from STAP services.
Elizabeth Auden
http://www.ivoa.net/Documents/latest/ADQL.html
21 September 2006
e-Science AHM 2006
Mission Data Flow
Elizabeth Auden
21 September 2006
e-Science AHM 2006
Solar-B Studies / Observation Planning
•
•
•
Study Proposal
– All study proposals are tested on the Flight Spare model
before acceptance into the master Study Database
– The Study Database contains information about the science
(target, pointing, rationale) as well as meta-data (author, joint
ops id, etc)
Database Distribution
– This is done using the automatic Solarsoft update mechanism
through Goddard Space Flight Centre
Daily Planning
– Performed at ISAS using the Study Database
– Studies are entered into the Timeline (As-planned) Database
Elizabeth Auden
21 September 2006
e-Science AHM 2006
FITS File Reformatting
•
•
Fits Reformatting (1)
– Level-0 performed at ISAS
– Accesses As-Planned Database
– Populates As-Run Database
– Transfers files to ADS and ISAS web system
– Accesses other instruments’ Fits files and transfers them to the
ADS for storage
Fits Reformatting (2)
– EIS Level-0 Fits files are used to generate Level-2 Fits files at
MSSL
– Updates the data catalogue
– Transfers Level-2 Fits files back to the ADS
Elizabeth Auden
21 September 2006
e-Science AHM 2006
Data Security
GRIDFTP
• Secure transfers
GridFTP used for file transfers
• 3rd party transfers
• Good performance
ADS
(RAL)
• Certificates from UK eScience CA and AIST
(Japan)
GridFTP
GridFTP
ADS
(RAL)
HTTP
USERS
STORAGE
• 10 TB per year
ADS
(RAL)
• 30 TB over nominal 3
year duration of mission
GridFTP
GridFTP: Clients / hosts authenticate with X509 certificates
Elizabeth Auden
21 September 2006
e-Science AHM 2006
Conclusions
• Solar-B launch tomorrow. 22 September 2006
• Data transferred via GridFTP to ADS at RAL
• ADS facility uses DMF to manage files on disk and tape
• Data searchable through AstroGrid DSA and STAP web service
• Solar community accesses data through AstroGrid workbench,
HelioScope or a web browser
LINKS:
AstroGrid: http://www.astrogrid.org
Solar-B countdown: http://www.mssl.ucl.ac.uk/www_solar/
Questions?
Elizabeth Auden
21 September 2006
e-Science AHM 2006