Download Data Browsing/Mining/Metadata

Data Browsing/Mining/Metadata
OTGA – UEM TRAINING COURSE –
FUNDAMENTALS OF OCEACOGRAPHIC
DATA MANAGEMENT , 23-27 JANUARY
2017, MAPUTO, MOZAMBIQUE
Prepared by: Harrison Ong’anda,
National Coordinator,
Kenya National Oceanographic Data Center (KeNODC)
[email protected], [email protected]
Data Browsing/Mining
• During the past 3 years many state-of-the-art systems have been
developed to bridge gaps between users and data resources,
completely replacing the old download-reformat-display paradigm.
• A leader in the development of innovative approaches has been the
Open Geospatial Consortium (OGC) which focuses on melding
environmental datasets into mapping systems.
• Because self-describing formats (HDF, GRIB, NetCDF) are such
important formats in these recent developments, general methods
for their use are also included here
THREDDS servers
• Thematic Realtime Environmental Distributed Data Services
• Middleware to bridge the gap between data providers and data users
• Catalogs are the heart of the THREDDS concept
• They are XML documents that describe on-line datasets
• The current focus of THREDDS development is the THREDDS Data
Server (TDS),
• Actually serves the contents of the datasets, in addition to providing
catalogs and metadata for them
• The TDS uses the Common Data Model to read datasets in various
formats, and serves them through OPeNDAP, OGC Web Coverage
Service, NetCDF subset, and bulk HTTP file transfer services.“
• http://www.unidata.ucar.edu/software/thredds/current/tds/
OPeNDAP Server
• Open-source Project for a Network Data Access Protocol
(OPeNDAP)• https://www.opendap.org/
• framework that simplifies all aspects of scientific data networking.
• provides software which makes local data accessible to remote locations
regardless of local storage format.
• also provides tools for transforming existing applications into OPeNDAP
clients (i.e., enabling them to remotely access OPeNDAP served data).
LAS Server
• Live Access Server
• http://ferret.pmel.noaa.gov/LAS/home
Enables the web user to:
•
•
•
•
visualize data with on-the-fly graphics
request custom subsets of variables in a choice of file formats
access background reference material about the data (metadata)
compare (difference) variables from distributed locations
SOS servers
• Sensor Observation Service (SOS) Servers
• http://www.opengeospatial.org/standards/sos
• Uncertainty surrounds the future of SOS, which is not being widely
used yet.
• Many initial test sites are apparently now inoperable
Web Mapping Service
• http://www.opengeospatial.org/standards/wms
• The OpenGIS® Web Map Service Interface Standard (WMS) provides a
simple HTTP interface for requesting geo-registered map images from
one or more distributed geospatial databases.
• A WMS request defines the geographic layer(s) and area of interest to
be processed.
• The response to the request is one or more geo-registered map
images (returned as JPEG, PNG, etc) that can be displayed in a
browser application.
Managing Sea Surface Temperature (SST)
• https://data.nodc.noaa.gov/las/getUI.do
• Managing Sea Surface Temperature (SST) Time-Series from Long-Term or
Classic Climatologies: US National Oceanographic Data Center (NODC) Live
Access Server (LAS)
• The NODC is the final depository for a number of satellite imagery products
from other programs, and they have boldly taken on the job of organizing
huge amounts of data into an easily usable LAS.
• Using this LAS or any of the many LAS installations available on the net, you
can revisit global conditions for specific times in the past (various intervals;
sometimes very distant), or (for sites with "classic" climatologies) you can
see the usual conditions (averaged over many years) for particular days,
weeks, months or seasons
Selected LAS Servers
• AVISO (Archiving, Validation and Interpretation of Satellite Oceanographic data)
• Indian National Centre for Ocean Information Services (INCOIS) - Attempts to visualize at-depth
Argo and other GTSPP operational data; unfortunately often offline
• My NASA Data (NASA data for educators; 5 possible choices at this HTML address)
• NASA PODAAC - Mixture of near-real-time and older satellite data
• NVODS (National Virtual Oceanographic Data System)
• OSMC (NOAA's Ocean Observing System Monitoring Center, with custom user interface; seems to
be unfinished, providing location information but not data)
• PacIOOS (Pacific Islands Ocean Observing System)
• PFEL (Pacific Fisheries Environmental Laboratory; 6 possible choices at this HTML address)
• Unified Access Framework GEOIDE - New US NOAA project to make THREDDS catalog publishing
easier and more comprehensive through all NOAA datasets
• US Global Data Assimilation Experiment (USGODAE) - Global LAS with incomplete demo versions
of Argo display capabilities similar to INCOIS