Download ppt - CDS, Strasbourg

Survey
yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project

Document related concepts

Entity–attribute–value model wikipedia , lookup

Big data wikipedia , lookup

Clusterpoint wikipedia , lookup

Data Protection Act, 2012 wikipedia , lookup

Data model wikipedia , lookup

Data center wikipedia , lookup

Forecasting wikipedia , lookup

Data analysis wikipedia , lookup

Database model wikipedia , lookup

3D optical data storage wikipedia , lookup

Data vault modeling wikipedia , lookup

Information privacy law wikipedia , lookup

Business intelligence wikipedia , lookup

Transcript
Hello!
International Virtual Observatory
Alliance
Ajit Kembhavi, IUCAA, Pune
Data Storage and Retrieval
The Astronomer
Vermeer
1632-1675
The Library of Alexandria
3rd Century BC
Star Positions
Stars in the Milky Way
The Hertzsprung-Russell Diagram
The Data Avalanche
• An increasing number of telescopes and large
area detectors are producing immense amounts of
data through imaging and spectroscopic surveys.
• Terabytes of data are now available, and Petabytes
will soon be available from frequent all sky
imaging.
• Vast databases are also being produced through
simulations.
Wavelength Coverage
• The data spans the electromagnetic spectrum
from the radio to the gamma-ray region.
• Obtaining, analysing and interpreting the
data in different wavebands involves highly
specialised instruments and techniques.
• The astronomer needs new tools for using
this wealth of data in multiwavelength
studies.
Virtual Observatories
• Manage vast data resources and provide
these on-line to astronomers and other
users.
• Provide tools for data analysis, visualization
and mining.
• Develop interoperability concepts to make
different databases seamless.
• Empower astronomers by providing
sophisticated query and computational
tools, and computing grids for producing
new science.
VO Schema
Interactions
The Alliance
Members of the IVOA
IVOA Technology Initiatives
The IVOA has identified six major
technical initiatives to fulfill the scientific
goal of the VO concept.
IVOA-LISTS
• REGISTRIES: These collect metadata about
about data resources and information
services into a queryable database. The
registry is distributed. A variety of industry
standards are being investigated.
• DATA MODELS: This initiative aims to
define the common elements of astronomical
data structures and to provide a framework to
describe their relationships.
• UNIFORM CONTENT DESCRIPTORS:
These will provide the common language for
for metadata definitions for the VO.
• DATA ACCESS LAYER: This provides a
standardized access mechanisms to
distributed data objects. Initial prototypes are
a Cone Search Protocol and a simple Image
Access Protocol.
• VO QUERY LANGUAGE: This will provide
a standard query language which will go
beyond the limitations of SQL.
• VOTable: This is an XML mark-up standard
or astronomical tables.
VOTable
• This is a new data exchange standard
produced through efforts led by
Francois Ochsenbien of CDS,
Strasbourg and Roy Williams of
Caltech.
• VOTable is in XML format. Physical
quantities come with sophisticated
semantic information.
VOTable
• The format enables computers to easily parse
the information and communicate it to other
computers.
• Federation and joining of information become
possible and Grid computing is easier.
• VOTable parsers have been developed in Perl,
Java and C++.
• Enhancements and extensions are being
considered.
VOTable Data
The data part in a VOTable may be represented using one of
three different formats:
– FITS : VOTable can be used either to encapsulate FITS
files, or to re-encode the metadata.
– BINARY : Supported for efficiency and ease of
programming, no FITS library is required, and the
streaming paradigm is supported.
– TABLEDATA : Pure XML format for small tables.
Science Initiatives
• Many IVOA projects have active Science Working
Groups consisting of astronomers from a broad
cross-section of the community representing all
wavelengths.
• The focus here is to develop a clear perception of
the scientific requirements of a VO.
• Projects within the working groups will develop
new capabilities for VO based analysis.
• This will enable the community to create new
research programs and to publish their data and
research in a more pervasive and scientifically
useful manner.
Caltech, Fermilab, JHU, NASA/HEARC,
Microsoft, NCSA/UIUC, NOAO, NRAO,
Raytheon ITS, SDSC/UCSD, SAO/CXC,
STScI, UPenn, UPitts/CMU, UWis, USC,
USNO, USRA, CVO
NVO-People
Registry and DIS
Terapix
Jodrell Bank
AVO Prototype
Demo
Astrogrid:
Astronomy
Catalogue
Extractor
AVO:
Aladin+SED
VO-India:VOPlot
CVO Collaborations
• There are three major projects at the CVO involving
collaborations with other VO.
• CVO is collaborating with the German Astrophysical VO
to incorporate ROSAT X-ray data and catalogues into the
CVO system.
• CVO is collaborating with the Australian VO.to
incorporate 2Qz and 2DF galaxy spectra into the CVO
database.
• CVO is an associate member of NVO and is have put in
place some components of the NVO galaxy morphology
demo.
Australian –VO Collaborations
• The distributed volume renderer (dvr) software, is a
tool for rendering large volumetric data sets using the
combined memory and processing resources of
Beowulf like clusters.
• A collaboration between the Melbourne site of Aus-VO
and AstroGrid aims to develop the existing dvr
software into a grid-based volume rendering service.
• Users will be able to select FITS-format cubes from a
number of "Data Centres",have the data transferred to
a chosen rendering cluster, and then proceed to
visualise the volume of data remotely (See Demo).
Virtual Observatory - India
Persistent Systems
IUCAA
VO-India Projects
•
•
•
•
•
•
•
•
VOTable Parser in C++; Streaming data
Web based FITS browser
Visualizer for VIZIER
Statistical packages for VIZIER and beyond
GALics data bases
Data mirror sites
Scientific data mining
Applications beyond astronomy
The VOPLOT Collaboration
The VOPlot Collaboration
• The idea for VOPlot was developed through a
discussion between VO-I and CDS.
• VOPlot was developed by PSPL and IUCAA and
was integrated into Vizier with the help of CDS.
• It was then decided to integrate VOPlot with
Aladin, an interface was agreed on, and the
integration is now complete.
• The collaboration was carried through a few short
visits, phone calls and E-mail.
• Main persons: Sonali Kale et al (PSPL), Pierre
Fernique and Francois Ochsenbien. (CDS).