Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Entity–attribute–value model wikipedia , lookup
Extensible Storage Engine wikipedia , lookup
Relational model wikipedia , lookup
Clusterpoint wikipedia , lookup
Functional Database Model wikipedia , lookup
Object-relational impedance mismatch wikipedia , lookup
Near Real Time Ocean Observations Online the Detailed Escape of SEACOOS (Southeastern Atlantic Coastal Ocean Observing System) Data Management and Visualization Secrets Software : engines Mapserver Open source from University of Minnesota USC runs a mixture of versions from 3.6 to 4.2 PostgreSQL Open source from postgresql.org USC runs version 7.4.1 PostGIS Open source from Refractions.net USC runs version 0.8.1 Near Real Time Ocean Observations Online : SEACOOS Software : additional PHP Open source from php.org USC runs version 4.3.2 Perl Open source from perl.org USC runs version 5.8.0 Miscellaneous ANiS and gifsicle Imagemagick Near Real Time Ocean Observations Online : SEACOOS Hardware data scout In-situ RS application server Apache 2.x MapServer Perl, PHP, misc. database A In-situ RS Near Real Time Ocean Observations Online : SEACOOS database B In-situ model output Directory structure Data that sits on USC filesystems RS images cached images All files have strict naming convention that includes timestamp Near Real Time Ocean Observations Online : SEACOOS System administration Databases backed up nightly Databases cleaned up and optimized nightly requires some downtime (~ 2 hours) significant overhead ~ 4 hours Some backend data massaging (mainly for model output aggregation) Near Real Time Ocean Observations Online : SEACOOS Database structure One table category per data type e.g. in-situ winds e.g. QuikSCAT winds more complicated since requires aggregation and normalization e.g. OI SST wind_prod contains all wind data wind_map contains wind data appropriate for maps table containing pointers to data files on disk Ancillary tables for specific data types e.g. OI SST table containing RGB to SST lookup values (for querying purposes) Near Real Time Ocean Observations Online : SEACOOS Data processing overview Data scout (netCDF) Perl code flattens incoming netCDF into arrays which are turned into SQL INSERT statemtents. Triggers update the new records suitable for normalization as well as Mapserver display elements. Procedures run to optimize tables for display. RS images (HDF to PNG) begin as HDF but are availed to USC as PNG’s Images have standard naming convention and agreed upon extents as well as predefined RGB to value, e.g. RGB to SST, pairs. Incoming files cause postgreSQL table to be updated with the new file and timestamp. Near Real Time Ocean Observations Online : SEACOOS