Download Ling_Qiu_UCARE.v1.2 - DigitalCommons@University of

Toward Mining Massive and Multi-dimensional Data for Extreme
Hydrometeorological and Climate Event Analyses
Ling Qiu1,2,3, Carlos M. Carrillo3, and Francisco Munoz-Arriola,3,4,5
1Electrical and Computer Engineering Department, 2UCARE Fellow; 3Biological Systems Engineering; 4School of Natural Resources, 5Affiliated to Robert Daugherty Water for Food Institute
University of Nebraska-Lincoln
Missouri River Basin(MRB)
• MRB has 117 million acres in
cropland.
• Produce 46%, 22% and 34% US’
wheat, corn, and cattle,
respectively.
• Economic contribution of agricultural activity can
be jeopardized by extreme hydrometeorological
and climate events (EHCEs).
Why do we study precipitation and temperature?
• Precipitation and temperature are two of the most important
variables to evaluate extreme events.
• It is unclear if precipitation and Minimum and Maximum
Temperatures’ patterns present a historical persistence in
their occurrence.
Precipitation Analysis
DATASETS
Introduction
Anomalies
• A spatially comprehensive hydrometeorological
dataset for Mexico, the U.S., and southern Canada for
the period of 1950-2013 (Livneh et al., 2015).
• Includes observed and gridded daily precipitation,
maximum and minimum temperatures.
• The resolution of the dataset is 1/16 degree(~6km)
• ~40K grids in the MRB and 20K time steps by grid.
Summer
Summer
EOF1
Dry
Dry
Summer
Summer
29.06%
Wet
Wet
Winter
Winter
PROGRAMING TOOLS
EOF1
Dry
Dry
Matplotlib
Winter
Winter
HCL Color
•
A
python
2D
plotting
Research Question – Objectives - Hypothesis library.
What are the main modes of spatial distribution and temporal
• Make plotting easier in
variability of extreme events (extreme warm/cold, dry/wet) in
python.
•
MRB?
• Capable of plotting
various kinds of
• Extract spatial patterns and temporal variability EHCEs.
common plots.
• Develop and implement data mining techniques for EHCEs
• Users are able to
•
control
most
of
the
analyses using Python.
details in a figure.
MRB modes of spatiotemporal variability can be characterized as
areas of historical water deficits/surpluses and warm/cold
conditions at the basin scale.
31.15%
Wet
Wet
HCL(Hue-ChromaLuminance) is based to how
the human perception
works.
Users can directly control
the three indexes in HCL.
Temperature Analysis
Anomalies
EOFs
Summer
EOF1
Hot
Summer
Conclusions
Empirical Orthogonal Functions (EOFs)
•
• Climate is characterized by nonlinearity and high
dimensionality.
• EOFs analyses are widely used to extract important
spatiotemporal patterns of variability in atmospheric
sciences (Wilks, 2006).
• Python uses a method based on singular value
decomposition (SVD) in EOF analysis.
• The input to EOF analysis is a spatial-temporal anomaly
matrix of a variable.
• EOFs is our basic analyses for Data Mining of
multidimensional data.
EOFs
We found that precipitation and temperature are the ideal meteorological
variable to test spatiotemporal variability of extreme events using simple
(composite analysis) and sophisticated (EOF) statistical techniques.
The EOF data mining technique is able to portray the dry and wet events of
the MRB climatic variability of the last 50 years. Wet and dry events in the
interannual scale of variability are captured as the statistically dominant
mode of EOF for summer and winter.
Python codes for plotting and analysis used in this study were found to be
very efficient to work on massive multidimensional climate datasets.
•
•
56.74%
Cold
Winter
EOF1
Hot
Winter
References
•
•
•
Wilks, D. S. (2006). Statistical Methods in the Atomspheric Sciences (2nd ed., Vol. 91,
International Geophysics Series). Burlington, MA: Elsevier.
Livneh, B., Bohn, T. J., Pierce, D. W., Munoz-Arriola, F., Nijssen, B., Vose, R., . . . Brekke, L. (2015).
A spatially comprehensive, hydrometeorological data set for Mexico, the U.S., and Southern
Canada 1950–2013. Scientific Data. Retrieved March 04, 2016.
Why HCL. (n.d.). Retrieved April 06, 2016, from http://hclwizard.org/why-hcl/
66.94%
Cold