Download OkeanProgn10

NSO8055 Okeanograafiline prognoos
Jüri Elken [email protected]
Andmete haldamise küsimusi
(väga lai temaatika, ühtne kontseptsioon puudub, siiski: http://www.iode.org)
ICES ajalooline andmehõive
perfokaardi formaat (andmed)
ROSCOP vorm (meta-andmed)
geograafiline kodeering (Mardseni kvadraadid)
Andmete haldamise tendentsid
MyOcean, Sea-SEARCH & SeaDataNet
Meta-andmed: EDIOS
Perfokaart
1881, Herman Hollerith
IBM 80 column punch card format
kasutusel alates 1928 kuni ca 1980
andmestruktuur kanti üle magnetlintidele
numbrid 1 auk, tähed mitu auku
kasutati ka “overpunch”
FORTRAN reeglid põhinevad perfokaardile
Card from a Fortran program: Z(1) = Y + W(1)
ICES ajalooline andmehõive
ICES Punch Card format (Data) 1968
ROSCOP Cruise Report format (MetaData)
ICES perfokaardid
Hydro Master Card
iga jaama kohta 1, sisaldab meteoandmeid
Hydrochemistry Card
iga sügavuse kohta 1 kaart, sh riigi, laeva, koordinaatide, aja andmed
Hydrography Card
iga sügavuse kohta 1 kaart, kohandatud CTD-le (rohkem tüvenumbreid)
andmete lahutus
asukoht minut
sügavus m
T, S sajandik
biogeenide suured kontsentratsioonid “overpunch” abil
NB! puudub info metoodika ja kvaliteedikontrolli kohta
ICES andmehõive kaasaegne juhend
(ka HELCOM seire kasutab seda)
Submitters of data should, if at all possible, use either of the formats described here.
Receivers of data from ICES will receive it in only the first described format, the ICES
Oceanographic 'punch card' format but software is normally supplied with
any request to help the user read data sets prepared in this format,
including an export facility to data bases and spreadsheets. The user
should note that this format has been modified from that published by ICES in 1979 in
several important respects. In particular provision was made to include position
information to .01 of a degree, and time to the nearest minute. Other changes include a
re-definition of the > (greater than) overpunch in the nutrient fields (type '56' chemistry
record was replaced by type '76') and record type 'P6' was introduced to accommodate
the very high nutrient levels reported from some coastal regions. In both of these record
types chlorophyll 'a' is stored to only one decimal place (the '56' record type was 2
decimal places).
From early 1994, additional features were added to the format (03 record) to
accommodate extra decimal places common in CTD records. This affected only columns
previously used for derived quantities (sigma-t, dynamic depth).
For data received after ca 1997 parameters not supported by the above record types
were included by the inclusion of the '0Z' record (Additional Parameter Record). This
record type allows for any number of parameters, so long as it is specifified in the
BODC/JGOFS data dictionary.
http://www.ices.dk/Ocean/formats/
(Position 79-80 = "76" or *Position 79-80 = "P6" or Position 79-80 = "56")
ICES perfokaardi näited:
Hydro Master ja
Hydrochemistry
Position 79-80 = "0J"
PositionParameter
01-02 Country
03-04 Ship
05-08 Station No.
Description
Coded according to IOC country codes.
Coded according to IOC ship codes.
Station number (within a given year
start counting one at the 1 Jan 0000
Hrs UTC is recommended).
09-12 Latitude
Geographical latitude in degrees and
minutes (decimals see below).
13-17 Longitude
Geographical longitude in degrees and
minutes (decimals see below).
18
Quadrant
Indicator of quadrant on globe:
0 = Latitude North Longitude East
1 = Latitude North Longitude West
2 = Latitude South Longitude East
3 = Latitude South Longitude West
0° Latitude is defined as being North
0° Longitude is defined as being East
180° Longitude is defined as being West
North, South relative to the equator
East, West relative to Greenwich
meridian
19-21 Year
Last 3 digits of year.
22-23 Month
Number of the month within a year
24-25 Day
Number of the day within a month
26-27 Time
Starting time of hydrographic station
in UTC (minutes given later).
28-31 Depth
Corrected depth to bottom in meters
32-45 None
Reserved (usually specifies origin of
data)
46-64 None
Weather information (rarely used in
recent data)
65-66 Latitude ct'd Decimals of Latitude minutes
67-68 Longitude ct'd Decimals of Longitude minutes
69-70 Time ct'd
Minutes of time
71-74 None
Reserved
75-77 Secchi
Secchi Disk Depth (metre, 1 implied decimal)
78
None
79
Indicator
Always "0" (Zero)
80
Record Type
Always "J" (Juliett)
PositionParameter
01-27 Parameter
Description
Copy of contents of positions 01-27 in
the Hydromaster record.
28-31 Depth/Pressure Pressure (decibars) or Depth (meters) no implied
decimals
Assumes unit as in "03" record above.
32-35 Temperature
Temperature Given in degrees Celsius (°C)
2 implied decimal places
Negative temperatures are indicated by
"}" (Closing Brace) in field 32
36-39 Salinity
Salinity given in PSS-78 scale for post-1978 data.
2 implied decimal places
40-42 Oxygen
Oxygen contents. Given in cm3 02 / dm3
water at STP. 2 implied decimals
(If position 78 = "K" then /kg)
43-45 Phosphate
Phosphate phosphorus. Given in µmol/l.
(if position 78 = "K" then µmol/kg)
2 implied decimals (* 1 implied decimal)
46-48 Tot.Phosphorus Total Phosphorus contents. Given in µmol/l.
(if position 78 = "K" then µmol/kg)
2 implied decimals (* 1 implied decimal)
49-51 Silicate
Silicate contents (Silicate Silicon).
Given in µmol/l.
(if position 78 = "K" then µmol/kg)
1 implied decimal (* 0 implied decimal)
52-54 Nitrate
Nitrate contents (Nitrate Nitrogen).
Given in µmol/l.
(if position 78 = "K" then µmol/kg)
1 implied decimal (* 0 implied decimal)
55-57 Nitrite
Nitrite contents (Nitrite Nitrogen).
Given in µmol/l.
(if position 78 = "K" then µmol/kg)
2 implied decimals (* 1 implied decimal)
58-60 Ammonium
Ammonium contents (Ammonium Nitrogen)
Given in µmol/l.
(if position 78 = "K" then µmol/kg)
1 implied decimal (* 0 implied decimal)
61-63 Tot. Nitrogen Total nitrogen contents. Given in µmol/l.
(if position 78 = "K" then µmol/kg)
1 implied decimal (* 0 implied decimal)
64-66 Hydrog. Sulph. Hydrogen Sulphide contents (Sulphide
Sulphur).Given in µmol/l.
(if position 78 = "K" then µmol/kg)
1 implied decimal (* 0 implied decimal)
67-69 pH
Hydrogen ion concentration in situ 2
implied decimals
70-73 Alkalinity
Alkalinity. Given in milliequivalents(millival)/dm3
water
at 20°C.( if position 78 = "K" then meq/kg)
3 implied decimal places
74-76 Chlorophyll a Chlorophyll a. Given in ug/dm3 water at
20°C.(if position 78 = "K" then ug/kg)
1 implied decimal (if 56 record, 2 implied decimals)
77 None
Reserved
78 Unit Indicator
Not K = All Chemistry units in /l (per volume)
K
= All Chemistry units in /kg (per mass)
79 Indicator
Always "7" (Seven) (**or "P" in which
case all nutrients and H2S X10, ie decimals-1)
80 Record type
Always "6" (Six)
ROSCOP (Cruise Summary Report)
ROSCOP (Report of Observations/Samples collected by Oceanographic
Programmes) kinnitati IOC poolt 1960-ndate lõpus
Ekspeditsioonide lühiinfo
mida millega mõõdeti
kus (Marsden kvadraadid)
kelle käest küsida mõõtmisandmeid
kaasajal analoog: meta-andmed
Kirjelduste grupid:
Meteorology (6)
Physical Oceanography (18)
Chemical Oceanography (16)
Marine Contaminants/Pollution (8)
Marine Biology/Fisheries (27)
Marine Geology/Geophysics (16)
Other (>30)
Näide
Data Tools and Models
(SeaDataNet)
Data. We make distinction between information and data. Information is describing a status or
situation (e.g. today is cold); data quantify the status or situation (e.g. temperature is
10°C). We define data broadly to include experimental or in situ observations, model outputs
and images.
Visualization. The first significant examples of Marine Information Systems were developed
during the '60s. Until the '80s the data management was composed of many independent
processes: data collection; pre-processing; storage in files. Graphical representation and
dissemination were part of the scientific study of the ocean dynamics. Today an efficient
information strategy includes visual representation of data (graphs, maps, ...), as a tool for
dissemination of data among users and the public.
Quality control. There are areas were the use of term 'data' is often controversial: a)
processed versus raw measurements, b) model outputs versus observations, c) images
versus digital underpinnings. Scientist strive to fully characterise their data to enable a better
understanding of its limitations. Use of data can be limited by the lack of certain attributes
such as: procedures for collection, conditions during collection, instrumentation,
temporal and spatial referencing, error or uncertainty, indications on quality assurance
procedures.
Analysis. Working with data provides opportunities for quantitative analysis and reasoning,
broad discussion and debate to evolve scientific understanding.
Models. We define as model an idealisation that embodies certain aspects of the 'real ocean'.
Models provide an experimental apparatus for the scientific rationalisation of the ocean
phenomena. In the presentation of ocean model fundamentals, it is useful to start with a
discussion on fluid kinematics.
Andmete haldamise tendentsid
Kui andureid ja platvorme oli vähe, koguti andmed suurtesse
keskustesse
Kaasajal hoitakse andmed enamasti mõõtja juures, formaadid jms on
kirjeldatud, ligipääs üle veebi, rahvuslikud andmekeskused on siiski
tugevad:
Meta-andmed kirjeldavad andmeid (parameetri definitsioon, metoodika,
platvorm, kvaliteedikontroll jne)
Andmete otsimine läbi meta-andmete kataloogi
Andmeülekande protokollid (ftp, OpeNDAP jne)
Operatiivne okeanograafia: mõõdetavaid parameetreid vähe, lihtsam
suuri “süsteemide süsteeme” kokku panna
Interdistsiplinaarne mereteadus: parameetreid tohutult, juba
defineerimine keerukas
Operatiivne süsteem
Operatiivne süsteem
Andmete “otsimise” projekt
http://www.sea-search.net/
(juba lõppenud)
http://www.seadatanet.org
SEADATANET has developed an efficient
distributed Marine Data Management Infrastructure
for the management of large and diverse sets of
data deriving from in situ and remote observation
of the seas and oceans.
The on-line access to in-situ and remote sensing data, meta-data and
products is provided through a unique portal interconnecting the
interoperable node platforms constituted by the SeaDataNet data
centres.
The development and adoption of common communication standards
and adapted technology ensure the platforms interoperability. The
quality, compatibility and coherence of the data issuing from so many
sources, is assured by the adoption of standardized methodologies for
data checking, by dedicating part of the activities to training and
preparation of synthesised regional and global statistical products from
the most comprehensive in-situ and remote sensing data sets made
available by the SeaDataNet partners.
The marine observing system is highly fragmented: more than 600
scientific data collecting laboratories from governmental organizations
and private industry have been identified. They collect data by using
various sensors on board of research vessels, submarines, fixed and
drifting platforms, airplanes and satellites, to measure physical,
geophysical, geological, biological and chemical parameters, biological
species etc. The collected data are neither easily accessible, nor
standardized. They are not always validated and their security and
availability have to be insure in the future.
SeaDataNet üldine andmete haldamise kontseptsioon
CDI
Otsing 2005-2008
Andmeid on
veel vähe!
väljavõte
2010
http://www.seadatanet.org
SeaDataNet produktid
Soolsuse klimatoloogia 1975-2005
Jaanuar
Juuli
Aprill
Oktoober
Interpolatsioon: 4D Data-Interpolating Variational Analysis
Soft: DIVA GHER (University of Liege) http://modb.oce.ulg.ac.be/projects/1/diva
http://odv.awi.de/en/home/
• vabalt kasutatav
• Windows (7, Vista, XP, 9x, Me, NT,
2000), Mac OS X, Linux, and UNIX
(Solaris, Irix, AIX)
• oma andmete formaat, kuid loeb ka
NetCDF
• rannajoon, sügavused
Milliseid andmeid tuleb käsitleda?
BODC = British Oceanographic Data Centre
The BODC Parameter Dictionary
In the 1980s, when we first started managing oceanographic data our dictionary
contained less than twenty parameters.
The BODC Parameter Dictionary is a collection of controlled vocabularies for
parameter management. The BODC Parameter Usage Vocabulary (8 MB)
contains almost 19,000 terms that are designed to label data values. These
have been systematically constructed using a semantic model.
Navigation through such a large number of parameters is a daunting task. To help
with this, a 3-layer hierarchy of discovery keywords is provided. The top level
is the SeaDataNet Parameter Disciplines, followed by the SeaDataNet Agreed
Parameter Groups and the BODC Parameter Discovery Vocabulary.
XML formaadis, kasutatav vastava tarkvaraga
http://www.bodc.ac.uk/data/codes_and_formats/parameter_codes/
SeaDataNet Parameter Disciplines
Term
definition
Administration and
dimensions
Parameters related to spatial and temporal coordinates, entity referencing (eg record
numbering and keys) and access control
Atmosphere
The atmospheric sciences domain
Biological oceanography
The biological oceanographic science domain
Chemical oceanography
The chemical oceanographic science domain
Cross-discipline
No specific association with an identified domain
Cryosphere
The cryosphere science domain, including ice on
both land and sea
Marine geology
The marine geological science domain
Physical oceanography
The physical oceanographic science domain
Terrestrial
The terrestrial science domain
asendatud veebisõnastikuga
http://seadatanet.maris2.nl/v_bodc_vocab/welcome.aspx/
SeaDataNet Agreed Parameter Groups
term
term
Acoustics
Optical properties
Administration and dimensions
Other biological measurements
Amino acids
Other inorganic chemical measurements
Atmospheric chemistry
Other organic chemical measurements
Bacteria and viruses
Other physical oceanographic measurements
Biota composition
PCBs and organic micropollutants
Birds, mammals and reptiles
Phytoplankton
Carbon, nitrogen and phosphorus
Pigments
Carbonate system
Rate measurements (including production,
excretion and grazing)
Cryosphere
Rock and sediment age and dating
Currents
Rock and sediment biota
Dissolved gases
Rock and sediment chemistry
Fatty acids
Rock and sediment lithology and mineralogy
Fish
Rock and sediment physical properties
Fluxes
Sea level
Gravity, magnetics and bathymetry
Sediment pore water chemistry
Halocarbons (including freons)
Sedimentation and erosion processes
Hydrocarbons
Sonar and seismics
Isotopes
Suspended particulate matter
Metal concentrations
Terrestrial
Meteorology
Water column temperature and salinity
asendatud
Microzooplankton
Waves
veebisõnastikuga
Nutrients
Zooplankton
http://seadatanet.maris2.nl/v_bodc_vocab/welcome.aspx/
EDIOS Meta-andmete formaat
(MIF)
(1)
asendatud veebivormiga http://seadatanet.maris2.nl/v_edios/search.asp
EDIOS Meta-andmete formaat
(MIF)
(2)
asendatud veebivormiga http://seadatanet.maris2.nl/v_edios/search.asp
Uued arengud (1)
Marine Strategy Framework Directive
foresees adoption of methodological standards for the assessment of the
status of the marine environment, monitoring, environmental targets and the
adoption of technical formats for the purposes of transmission and processing
of data in line with INSPIRE Directive. In respect of each marine region or
subregion, Member States shall make an initial assessment of their marine
waters, taking account of existing data where available. Member States
sharing a marine region or subregion shall draw up monitoring programmes
and shall, in the interest of coherence and coordination, endeavour to ensure
that:
(a) monitoring methods are consistent across the marine region or subregion
so as to facilitate comparability of monitoring results;
(b) relevant transboundary impacts and transboundary features are taken into
account.
Uued arengud (2)
EMODNET
European Marine Observation and Data NETwork
EMODNET will improve availability of high quality data.
EMODNET will provide data on scales defined by the regions and
subregions of the Marine Strategy Framework Directive. The
parameters to be collated are chosen to fit in with the requirements
of the Directive.
Four service contracts were launched for creating pilot components:
• Lot 1 – Hydrographic data
• Lot 2 – Marine geological data
• Lot 3 – Chemical data
• Lot 4 – Biological data
EMODNET vision: components
Läänemere andmeid veebis
Operatiivsed andmed (ainult näha) BOOS
Ajaloolised andmebaasid
ICES
vaba
BED
vaba
BALTEX
parooliga
FMI ja SYKE (endine FIMR)
SMHI (SHARK)
www.boos.org
http://www.ices.dk/ocean/
http://nest.su.se/models/bed.htm
http://www.gkss.de/baltex/data/index.html