* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Greene - STScI
Oracle Database wikipedia , lookup
Microsoft Access wikipedia , lookup
Open Database Connectivity wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Microsoft SQL Server wikipedia , lookup
Concurrency control wikipedia , lookup
Relational model wikipedia , lookup
Functional Database Model wikipedia , lookup
Microsoft Jet Database Engine wikipedia , lookup
Versant Object Database wikipedia , lookup
Clusterpoint wikipedia , lookup
The COMPASS (Catalogs of Objects and Measure Parameters for All Sky Surveys) Database Overview Gretchen Greene, Brian McLean, David Wolfe, and Charles Loomis Space Telescope Science Institute Antonio Volpicelli, Roberto Morbidelli and Allesandro Spagna Osservatorio Astronomico di Torino October 22, 2001 Introduction: Database Functionality Key Component in the GSC-2 catalog production – Online access to processed plate data Hierarchical Triangulated Mesh Level 3 – Identification of unique sources – Cross-indexing of multiple observations • Overlapping photographic plates • Multiple epochs – Global recalibration capability – GSC 2 is “exported” from COMPASS Successful demonstation of Object-Oriented technology and the selection of commercial software, Objectivity/DB One of the first large-scale astronomical archives – Estimated 2 billion objects, 10 billion sources/observations – 4 TB catalog derived from 8 TB image archive! Collaborative project with OATo and SDSS JHU Science Archive – OATo assisted with schema development, QA (CIQA), platematching and much more! – SDSS/STScI joint development of the HTM Indexing Leafnode S213 Infrastructure: How is the Database designed? Object model or design is based on hierarchical systems for efficiency – Federation of Databases and Containers Plate data information stored in 2 forms for access – MetaData and Observations within each HTM region HTM Region Database for each of 32768 regions RegionInfoContainer Plate Database PlateInfoContainer GSH RegionCatalogContainers Plate Source Objects for each plate section within region Page = Federated Database SourceIndex Section of reference catalog within region GSPC2 = Database = Container = Class/Basic Object Tycho2 [ COMPASS Federated Database ] for each plate Highlights: What happened in 2001 • Completion of Loading and Cross-Matching Observations: All-Sky, 2 bands • Reference catalogs: Successfully loaded GSPC2 and Tycho2 into COMPASS, utilize real-time object-matching based on GSC1 • Fully Online RAID System: Has been upgraded to 4TB using NT client/server configuration. • Export of GSC 2.2 Public Release: The export of ~1 billion objects from the database with best observation position selection, magnitude and classification filtering for accuracy. • Steady State Production Operations: Catalog production pipeline, recalibration, and administration of multi-Terabyte data volume • HST Bright Object Protection (BOP): GSC-2.2 incorporation into Observation Planning Tools and target acquisition development (APT) – special release • ShowSky JAVA Visualization Tool Public Release •Multiple Catalog viewing, including SDSS, 2MASS, Tycho, DSS, GSC…. •Parameter Filtering •HTM grid field view with level GSC2.2 Export Catalog Fields: what will change with 2.3? Field/Column Number Parameter 1 GSC ID2 2 Right Ascension 3 Declination 4 Mean Epoch 5 RA Error 6 Dec Error 7 RA Proper Motion 8 Dec Proper Motion 9 RA Proper Motion Error 10 Dec Proper Motion Error 11 F Magnitude 12 F Magnitude Error 13 J Magnitude 14 J Magnitude Error 15 V Magnitude 16 V Magnitude Error 17 N Magnitude 18 N Magnitude Error 19 Classification 20 Semi Major Axis 21 Eccentricity 22 Position Angle 23 Status Flag • Define new requirements and evaluate current parameters early enough to avoid schedule slips •need time to develop modifications to export task •Coordination of DB administation with science goals • How do we want to query the catalog? •move toward mining. Export Catalog: Current configuration and moving toward GSC 2.3 Source identification – names from GSC2.2 release should be preserved now that the catalog has been incorporated into telescope operations, is this always possible? Catalog column parameters, new ones? Changes to existing? – Position currently is based on observation closest to plate center for the same bandpass • Astrometric residuals were verified to be a factor of 2 improved. – Reclassification: further definition of non-star into galaxy, blend, etc. Plate identification – epoch is non-unique distinction of plate source Multi-plate calibrated parameters, proper motion: – API development will benefit from experience of previous catalog pipeline tasks, e.g. Plate loading and object/source cross matching. – Minimize the large scale transaction time, perform dynamically if possible. • Will this parameter be changing Multi-Archive at STScI: CASB coordination with MAST • MAST develops and maintains the public data release WWW user interface for SDSS and the STScI WWW pages for GSC-2 – individual development sites provide more project specific information -CASB -FNAL -JHU • Unified view of STScI archives –Growth in the large scale catalog archives –Cross correlation of mission data now possible, development of large scale correlations between ground and space missions in progress. Moving toward Multi-catalog archive: SDSS and GSC-2 joint operation at STScI • Data Archive Server is mirrored parallel to GSC-2 on hybrid LINUX/NT dual server. • New Develop of MAST/GALEX archive based on GSC-2/SDSS archive interoperation • SDSS population into COMPASS for high speed cross-matching utilizing HTM -no spatial remapping • New Database Engine?? There is a possibility that SDSS will migrate from Objectivity to another commerical system, e.g. SQL server. Future Developments: GSC-2 enhancements to support access for science and operations 2MASS population into COMPASS – GSC-2 + 2-MASS will be input catalog of NGST guidance system – 2MASS also utilizes HTM for their file system SDSS population into COMPASS Direct public access to COMPASS observations extended for higher speed access – HTMIndex container using sorted level10 source indexing as in export catalog similar to the export catalog infrastructure Correct Internal database page file limitations GSC2Query Enhancements GSC-2, SDSS, GALEX, 2MASS =>>>> NVO (National Virtual Observatory) – Still very early in conceptual stage, but nevertheless it is real, 10M NSF grant is currently issued for collaboration between STScI, Caltech, JHU + others.