Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
AstroGrid http://www.astrogrid.org • • • • who are we ? what is a VO ? what is a Grid ? how do we get there ? Andy Lawrence 2002 S.P.I.E. Hawaii Aug AstroGrid http://www.astrogrid.org Belfast Cambridge Edinburgh Jodrell Leicester MSSL RAL Andy Lawrence 2002 S.P.I.E. Hawaii Aug AstroGrid http://www.astrogrid.org Optical Infrared X-ray Radio Solar Space Plasma Andy Lawrence 2002 S.P.I.E. Hawaii Aug people • PL = • PS • PM Andy Lawrence = Nic Walton = Tony Linde slogans • • • • the archive is the sky (stolen from US-NVO) everybody can be a power user shift the results not the data a supercomputer on your desk the project • PPARC funded project Sept 2001-2004 – approx $6M • Grid technology development programme – links to GridPP, MyGrid (Bio), OGSA (Globus) • Stepping stone to Virtual Observatory – partner in AVO – working links with US-VO • using Unified Process • open collaborative web sites status • elaboration phase two-thirds finished – – – – – science problems / use cases / architecture technology assessment reports sw demos : portal, auth/auth, ontology; AVO science demos collaborative web pages set up Phase A study report to be published November • construction phase begins ~end 2002 collectivisation and democratisation • thirty year trend towards communal organisation – – – – – facility class (common-user) instruments facility class data reduction s/w calibrated archives with simple tools information services (Vizier, ADS, NED) large consortium projects (MACHO, 2dF, SDSS, UKIDSS, VISTA...) • next steps – inter-operable archives (joint queries) – automated resource discovery (registry) – facility-class exploration and analysis tools (data mining) the Virtual Observatory concept • Aim to make all archives speak the same language – – – – – all searchable and analysable by the same tools all data sources accessible through a uniform interface all data held in distributed databases that appear as one archives form the Digital Sky eventual interface to real observatories the archive is the sky the Grid concept • shared managed distributed resources – documents + data + software + storage + cycles + expertise • • • • • • network : ability to pass messages web : transparent document system a supercomputer computational grid : transparent CPU on your desktop datagrid : transparent data access and services information grid, knowledge grid ... ? everybody can be a power user Virtual Organisations ? what do we need ? • sociology – need agreed formats for data, metadata, provenance – need standardised semantics ("ontology") • internet technology – – – – need protocols for publishing and exchanging data need registry for publishing service availability and semantics need method of transmitting authentication/authorisation need methods for managing distributed resources bottlenecks • i/o bottleneck – need database supercomputers – need innovative search and analysis algorithms • network bottleneck – data centres must provide analysis service – facility class analysis code needed shift the results not the data grid geometry needed • • • • not a centralised superarchive not P2P (like Napster) not a hierarchy (like LHC grid) service providers + users (like most commerce) – some unplanned open use – some registered use – variety of access rights two rivers • academic / Globus – remote log on – identity/authentication/authorisation – resource management • commercial / W3C – exchange of data (B2B) – service description and publication – "Web services" = XML + SOAP + WSDL (cf GLU) Globus problems • only half works • data transfer primitive – flat files only • not a services-user structure Web service problems • • • • • one-to-one bulky no general auth/auth solution no accepted service registry solution no ontology solution Grid Services ? • web services + auth/auth + lifetime management • to be developed by OGSA project – OGSA-DAI integrates database access – AstroGrid and MyGrid "early adopters" who does what • the VO projects will : – evolve agreed data standards – build infrastructural toolkit for data exchange – build registry of available data services • Data Centres will : – maintain data in whatever format they like – use toolkit to write standard data services – publish services to registry • AstroGrid also will : – provide first example data services for key UK databases – provide example "point of entry" user interface – provide some resource : warehouse, CPU, "MySpace". who does what part(II) • observatories / instrument builders need to : – output data to new standards – link observing request to VO research – decide whether to provide data services or farm out to data centres FIN Web DB access today DB engine SQL data front end CGI request html browser web page user Web service native data web service XML data application DB engine SQL XML request anything user