Download New Technologies for Storage and Display of Meteorolgical Data

New Technologies for
Storage and Display of
Meteorological Data
Dale Paynter
Operations Management Group
May, 2002
NUMUG Conference
Benefits of Technology?
 Move to modern technology path that will be developed
and supported
 Make initial installation, maintenance & upgrades easier
Change is the name of the game
External requirements & standards
Infrastructure changes
Organizational re-alignment
Meet rising expectations
Simple systems interfaces
Capitalize on staff knowledge & skills
 Let Meteorologists manage, analyze and use the data
10 May 2002
OMG
2
New Technologies
What are they?
Data Driven System Architecture
Relational Databases
Web Applications
10 May 2002
OMG
3
Data Driven Architecture
What is it?
Write code to be independent of specifics
Make system configurable to define:
Input stream
Parameters for internal calculations
Reports
Interfaces to other systems
10 May 2002
OMG
4
Data Driven Architecture
What are the benefits?
Respond to new requirements and standards
Reduce dependency on vendors & IT
Get out of program change queue
Reduce change orders & site visits
Give users control through configuring system
10 May 2002
OMG
5
What does this let you do
easily?
Add new sensors with no code changes
Create new list reports
Change corporate information
Revise alarm / flag limits
Add or decommission sampling locations
Interface to different systems
10 May 2002
OMG
6
So how do we do this?
Parameters, parameters, parameters...
Make data base ‘self-descriptive’
Store ‘what’ is being stored as well as ‘values’
Eliminate need to reorganize / reload data to
accommodate changes
Make variables for all ‘constants’
Tower names
Time zones
Daylight saving time
Measurement codes
10 May 2002
Report titles & headings
Data to display / order
Report field formats
Conversion formulae
OMG
7
Adding a new sensor
 Traditional “Programming” Approach  Data Driven Architecture Approach
 User configures or reprograms
datalogger
 Programmer adds new field to file
formats. Then dumps existing data,
reloads and verifies
 Programmer changes data loading to
process new input layout & update
new field
 Programmer changes updating to
display / edit new field
 Programmer changes reporting to
display new field
 System administrator installs final
programs on all affected computers
10 May 2002
OMG




User configures datalogger
User defines sensor in database
User tests and verifies results
System administrator copies
configurations to production system
8
Relational Databases
What are they?
Data is stored in Tables made up of rows of
fields (columns)
Data is referenced through name of the field
Subsets of data may be selected
Tables may be joined to retrieve related records
10 May 2002
OMG
9
Relational Databases
Usage comparison example
Sequential file - 30’ wind speed is in columns 26
to 31 (format F6.2)
Typical relational database - 30’ wind speed is
stored in a field named WS30
Relational database using data driven
architecture - 30’ wind speed is stored in a field
named Value where the field MeasCode
contains ‘WS’ and Height contains ‘30’
10 May 2002
OMG
10
Relational Databases
What are the the general benefits?
Built-in security
Supports simultaneous accesses
Built-in data recovery techniques
Built-in data retrieval algorithms & optimization
Standard database interaction (SQL)
In-house expertise can be leveraged
10 May 2002
OMG
11
Relational Databases
And... for meteorological data?
Concurrent loading, updating and reporting
Permits non-sequential loading for legacy data
and data recovery
Handles data retention and archiving
requirements
Ad hoc queries
Time-stamping data changes
Keep original values as well as edited values
10 May 2002
OMG
12
Web Applications
In general...
 User requests report and submits parameters through
browser
 Web application generates report on server and sends to
browser
 User views / prints report through the desktop browser
Benefits?
 Removes need to run / maintain specific report programs
on desktop
 Just ask your LAN Administrator…
10 May 2002
OMG
13
Web Server Configuration
Meteorologist’s
Workstation
running
Updater
Meteorological Monitoring System
Server
Web Server
MMS
Database
Sensors
Zenos
Zenos
InterceptTM
&
Loader
Zeno®s
10 May 2002
OMG
Reporter
Interface
Files
Meteorologist’s
& Users’
Workstations
running
Web Browsers
Other
Servers
14
Web Technology
Internet or Intranet Server
Data manipulation and report generation
performed on server
Only results sent to browser
Automatic report refresh option
Additional security levels possible
10 May 2002
OMG
15
How does it perform?
Even running on a notebook:
Current reports are instantaneous
Historical reports take a few seconds
Frequency Distribution produces a 1 year
report in 12 seconds
Data Capture Summary produces a 1 year
report in 55 seconds
10 May 2002
OMG
16
Benefits of Technology
Move to modern technology path that will be
developed and supported
Get out of the program change queue
Flexibility to respond to change
Let Meteorologists manage, analyze and
use the data
10 May 2002
OMG
17