Download Document

DATA ANALYSIS USING
PYTHON-I
Baburao Kamble and Ayse Kilic
University of Nebraska-Lincoln
University of Nebraska-Lincoln
GIS in Water Resources Lecture, 2014
5/24/2017
Objectives
To be able to understand and write Python scripts for
Numerical, Text (Weather) & Geospatial (Environmental)
data manipulation
5/24/2017
Agenda
Part I: Basic Python Programming
• Python Syntax, Strings, Array, Conditional and Control flow,
Functions
• File Input/Output and Text Data Processing
Part II: Geospatial Data Analysis
• Installation of GDAL bindings on windows operating system
• Reading Raster (Landsat and DEM) data
• Using GDAL function from command line (interpolation, translation)
• Raster Processing.
• NDVI calculation
• DN2Rad2Ref
5/24/2017
What is Python?
5/24/2017
Why Python?
• Natural Language ToolKit
• Ease of use; interpreter
• Processing: Symbolic
• Python’s built-in data-types for strings, lists, and more.
• Java or C++ require the use of special classes for this.
• Processing: Statistical
• Python has strong numeric processing capabilities:
matrix operations, etc.
• Suitable for probability and machine learning code.
5/24/2017
Writing a Program in Python
• A program is a collection of Python statements, functions, etc.
• A script is a list of commands which are run by a program
written in a specific language such as Python
• Scripts can be edited by text editors or preferably by a
specialized editor that also allows the script to run
• We can use Python command line in Windows
• However, the use of the command line is limited (it does not support
testing scripts)
• So, instead we use the Python script editor (an IDE) that has a better
interface than the command line mode
5/24/2017
Installing Python
• Python is installed on Lab Computers.
• Python for Win/Mac/Unix/Linux is available from
www.python.org.
• Generally an easy install.
• On Mac, already part of OS X.
• For GDAL you need Python 2.6 or 2.7.
• Python 3.X is little-bit different
• GUI development environment: IDLE (Integrated
DeveLopment Environment).
5/24/2017
IDLE
• We will also continue using the IDLE environment for
Python scripting
• IDLE is available with Python download
• Using the interactive Python interpreter is great for writing
small code
• However, it is not meant to be saved!
• For larger, more complex multi-line code, it is better to
write it as a script (rather than using the interactive Python
interpreter, e.g., IDLE) or configure ECLIPSE for Python
5/24/2017
Elements of Programs
• Simpler expressions can be combined using operators.
• +, -, *, /, **
• Spaces are irrelevant within an expression.
• The normal mathematical precedence applies.
• 2/4
• 2*3
5/24/2017
Variable




“Variables” in python are really object references.
Can contain letters, numbers, and underscores
Must begin with a letter
pi = 3.1415926
Cannot be one of the reserved Python HW = "Hello, world"
i = 2+7
print type(pi)
print type(HW)
print type(i)
Output:
<type 'float'>
<type 'str'>
<type 'int'>
5/24/2017
Operators
5/24/2017
Simple Sample of Python code
Sample.py
5/24/2017
Array & Lists
• How to Define array
myArray=numpy.array([2,3,4,5,6,7])
myArray
myArray[0]
myArray[4]
• How to define list
myList=[2,3,4,5,6,7]
myList
myList[1]
myList[4]=99
myList
5/24/2017
Modules (Library) & Packages in Python
A module is a file containing Python definitions and
statements.
• A module is a single file (or files) that are imported under one
import and used. e.g.
import my_module
import numpy
• A package is a collection of modules in directories that give a
package hierarchy.
from my_package.timing.danger.internets import my_module
5/24/2017
Using the numpy Library
• Besides (+, -, *, /, //, **, %, abs), we have lots
of other math functions available in a math
library.
• A library is a module with some useful
definitions/functions.
• To use a library, we need to make sure this
line is in our program:
import numpy
• Offers Matlab-ish capabilities within Python
5/24/2017
Functions
• A function is like a subprogram, a small program inside of
a program.
• The basic idea – we write a sequence of statements and
then give that sequence a name. We can then execute
this sequence at any time by referring to the name.
• The part of the program that creates a function is called a
function definition.
• When the function is used in a program, we say the
definition is called or invoked.
5/24/2017
Functions



Define them in the file above the point they're used
Body of the function should be indented consistently (4
spaces is typical in Python)
Example: square.py

def square(n):
return n*n
print "The square of 10 is ",
print square(10)
Output:
The square of 3 is 9
5/24/2017
Functions That Return Values
• Sometimes a function needs to return one or more values.
5/24/2017
Let’s Consider Celsius to Fahrenheit
temperature conversion program
def main():
celsius = input("Input Celsius temperature:")
fahrenheit = 9/5 * celsius + 32
print "The temperature is", fahrenheit, "degrees Fahrenheit."
main()
ConvertTemp.py
5/24/2017
Python Program:
Quadratic equation/Numerical Calculations
• Let’s write a program to compute the roots of a quadratic
equation!
b  b2  4ac
x
2a
• The only part of this we don’t know how to do is find a
square root… but it’s in the mathematical library!
• Lets get mathematical library inside python.
quadratic.py
5/24/2017
Let’s Consider Celsius to Fahrenheit
temperature conversion program
• Let’s say we want to modify that program to print a
warning when the weather is extreme.
• Any temperature over 90 degrees Fahrenheit and lower
than 30 degrees Fahrenheit will cause a hot and cold
weather warning, respectively.
5/24/2017
Three major control constructs of
programming
• Sequential: Simply do steps one after the other in
order they are listed.
• Conditional: Decide which statement to do next
based on some true/false test.
• Iterative: A set of statements is repeated over and
over until some condition is met.
5/24/2017
Multi-Way Decisions
• if <condition1>:
<case1 statements>
elif <condition2>:
<case2 statements>
elif <condition3>:
<case3 statements>
…
else:
<default statements>
5/24/2017
Forming Simple Conditions
Python
Mathematics Meaning
<
<
Less than
<=
≤
Less than or equal to
==
=
Equal to
>=
≥
Greater than or equal to
>
>
Greater than
!=
≠
Not equal to
ControlStruct_Demo1-3.py
5/24/2017
Text Data Processing
file = open("test.txt")
# Read everything into single string:
content = file.read()
# file.read(20) reads (at most) 20 bytes
print (len(content))
print content
# close file
file.close()
#CSV file:
#import csv library
import csv
#open csv file
myfile = open('weather.csv', "r")
#read content of file
reader = csv.reader(myfile)
# iterate over the rows
for row in reader:
print row,
myfile.close()
File_IO.py
5/24/2017
CSV file read/write/manipulation
• Estimating daily evapotranspiration (ET) using the
California Irrigation Management Information System
(CIMIS) Weather Data
CIMIS_ETr.py
Please refer ASCE manual for Evapotranspiration equations
http://www.kimberly.uidaho.edu/water/asceewri/ascestzdetmain2005.pdf (Allen et al, 2005)