Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Development of a Community Hydrologic Information System David Maidment (PI), David Tarboton, Dan Ames, Jon Goodall, Richard Hooper, and Ilya Zaslavsky CUAHSI HIS http://his.cuahsi.org/ Sharing hydrologic data Support EAR 0622374 The way that data is organized can enhance or inhibit the analysis that can be done I have your information right here … Picture from: http://initsspace.com/ Data Models (x1,y1) What are the basic attributes to be associated with hydrologic data values and how can these best be organized? Streamflow Precipitation & Climate Water Quality Groundwater levels Soil moisture data Flux tower data What is HIS • The CUAHSI Hydrologic Information System (HIS) provides web services, tools, standards and procedures that enhance access to more and better data for hydrologic analysis. http://his.cuahsi.org Water quantity Water quality Precipitation Meteorology Consortium of Universities for the Advancement of Hydrologic Science, Inc. • 110 US University members • 6 affiliate members • 12 International affiliate members (as of March 2009) An organization representing more than one hundred United States universities, receives support from the National Science Foundation to develop infrastructure and services for the advancement of hydrologic science and education in the U.S. http://www.cuahsi.org/ HIS Team and Collaborators • University of Texas at Austin – David Maidment (PI), Tim Whiteaker, Ernest To, Bryan Enslein, Kate Marney • San Diego Supercomputer Center – Ilya Zaslavsky, David Valentine, Tom Whitenack • Utah State University – David Tarboton, Jeff Horsburgh, Kim Schreuders, Justin Berger • Drexel University – Michael Piasecki • University of South Carolina – Jon Goodall, Tony Castronova • Idaho State University – Dan Ames, Jiří Kadlec, Ted Dunsford, Teva Veluppillai • CUAHSI Program Office – Rick Hooper, David Kirschtel, Conrad Matiuk, Yoori Choi • WATERS Network – Testbed Data Managers • HIS Standing Committee • USGS – Bob Hirsch, David Briar, Scott McFarlane • NCDC – Rich Baldwin Data Searching – What we used to have to do Searching each data source separately NWIS request return request return request return NAWQA request return NAM-12 request return request return request return request return NARR Michael Piasecki Drexel University What HIS enables Searching all data sources collectively GetValues GetValues NWIS GetValues generic request GetValues GetValues GetValues NAWQA Michael Piasecki Drexel University GetValues GetValues NARR ODM Services-Oriented Architecture for Water Data HIS Central Data Discovery and Integration platform Data Services Data Publication platform HIS Server Water Data Services Spatial Data Services Data Synthesis and Research platform HydroDesktop CUAHSI Water Data Services System Analysis Query, Visualize, and Edit data using ODM Tools Streaming Data Loader Base Station Computer(s) Telemetry Network Sensors Hydroseek GIS Matlab Splus R IDL Java C++ VB WaterML ODM Database Excel, text ODM Data Loader Discovery Access USGS NWIS EPA STORET GetSites GetSiteInfo GetVariableInfo GetValues Hydro Desktop HydroExcel Service Registry Hydrotagger Harvester WaterOneFlow Web Service NCDC HIS Central Water Metadata Catalog Others WaterML and WaterOneFlow WaterML is an XML language for communicating water data WaterOneFlow is a set of web services based on WaterML • Set of query functions • Returns data in WaterML Slide from David Valentine GetVariables GetValues qualifier ISO Time value Direct analysis from your favorite analysis environment. e.g. Matlab % create NWIS Class and an instance of the class createClassFromWsdl('http://river.sdsc.edu/wateroneflow /NWIS/DailyValues.asmx?WSDL'); WS = NWISDailyValues; % GetValues to get the data siteid='NWIS:02087500'; bdate='2002-09-30T00:00:00'; edate='2006-10-16T00:00:00'; variable='NWIS:00060'; valuesxml=GetValues(WS,siteid,variable,bdate,edate,''); 2.5 x 10 4 Daily Discharge NEUSE RIVER NEAR CLAYTON, NC 2 cfs 1.5 1 0.5 0 1920 1930 1940 1950 1960 1970 1980 1990 2000 2010 What are the basic attributes to be associated with each single data value and how can these best be organized? “When” Time, T t A data value vi (s,t) s “Where” Space, S Vi “What” Variables, V Streamflow Precipitation & Climate Water Quality Groundwater levels Soil moisture data Flux tower data Observations Data Model (ODM) • A relational database at the single observation level (atomic model) • Stores observation data made at points • Metadata for unambiguous interpretation • Traceable heritage from raw measurements to usable information • Standard format for data sharing • Cross dimension retrieval and analysis CUAHSI Observations Data Model http://his.cuahsi.org/odmdatabases.html Horsburgh, J. S., D. G. Tarboton, D. R. Maidment and I. Zaslavsky, (2008), A Relational Model for Environmental and Water Resources Data, Water Resour. Res., 44: W05406, doi:10.1029/2007WR006392. Stage and Streamflow Example Lets explore the system http://his.cuahsi.org Hydroseek Bora Beran, Drexel http://www.hydroseek.org Supports search by location and type of data across multiple observation networks including NWIS and Storet HydroExcel http://his.cuahsi.org/hydroexcel.html HIS Server • Internet Map Server built using ArcGIS • Web browser client • Combine spatial data and observational data • Self contained fully documented with local control of data • Launch data visualization tools e.g. http://icewater.usu.edu/map and http://littlebearriver.usu.edu/ Loading data into ODM ODM Data Loader • Interactive ODM Data Loader – Loads data from spreadsheets and comma separated tables in simple format • Streaming Data Loader (SDL) SDL – Loads data from datalogger files on a prescribed schedule – Interactive configuration • SQL Server Integration Services (SSIS) – Microsoft application accompanying SQL Server useful for programming complex loading or data management functions SSIS Reducing Semantic Heterogeneity • ODM Controlled Vocabulary System – ODM CV central database – Online submission and editing of CV terms – Web services for broadcasting CVs Variable Name Investigator 1: Investigator 2: Investigator 3: Investigator 4: “Temperature, water” “Water Temperature” “Temperature” “Temp.” ODM VariableNameCV Term … Sunshine duration Temperature Turbidity … From Jeff Horsburgh Dynamic controlled vocabulary moderation system ODM Data Manager ODM Website ODM Tools XML Local ODM Database Local Server ODM Controlled Vocabulary Moderator ODM Controlled Vocabulary Web Services http://his.cuahsi.org/mastercvreg.html Master ODM Controlled Vocabulary From Jeff Horsburgh HydroDesktop Harvesting and analyzing data from web services GIS Hydrology • Add shapefiles to map • Change symbology and labels • Print and export map • GIS toolbox • • • • Search for data Download data Display time series Export data http://hydrodesktop.codeplex.com HydroDesktop Building blocks HydroDesktop Main Application Web Service API Metadata Cache Database Database Access Layer Actual Data Dababase MapWindow GIS Components HIS Server Web Services HIS Central Web Service Plug-in Interface Metadata plug-in Graph View plug-in Search plug-in 26 Advancement of water science is critically dependent on integration of water information Models Databases: Structured data sets to facilitate data integrity and effective sharing and analysis. - Standards ODM - Metadata - Unambiguous interpretation Analysis: Tools to provide windows into the database to support visualization, queries, analysis, and data driven discovery. Web Services Databases Analysis Models: Numerical implementations of hydrologic theory to integrate process understanding, test hypotheses and provide hydrologic forecasts. Conclusions • Data Storage in an Observations Data Model (ODM) • Data Delivery through internet-based Water Data Services using a consistent data language, called WaterML • Data Discovery through federated thematic keyword search services and a National Water Metadata Catalog • Data Access and Analysis through a Client Application (HydroDesktop) The combination of these capabilities creates a common window on water observations data for the United States unlike any that has existed before. Reports • 2008 status report summarizes the conceptual framework, methodology, and application tools for HIS • Shows how to develop and publish a CUAHSI Water Data Service • 2009 status report details the expansions and updates http://his.cuahsi.org/documents/HISOverview2009.pdf http://his.cuahsi.org