Download notes

Python – May 11
• Briefing
• Course overview
• Introduction to the language
• Lab
Python
• Why computer programming?
– Computer is a general-purpose machine, born to be
programmed
– We want to solve many types of problems
• Our goals
– Become multi-lingual
– Practice with design & implementation
– Learn by doing: start simple
• Help available
– Textbook, python.org, tutorials
Background
• First released, 1991
– Guido van Rossum, CWI, Netherlands
– Dissatisfaction with existing languages
• High-level, multi-purpose language
–
–
–
–
–
–
–
Relatively easy to learn & use (we hope!)
Procedural, object oriented or functional
Interpreted
Interface to OS, file system, Web
Some data structures built into language
Standard library
(More details in sections 1.3 and 1.5)
Initial To-Do
Essential skills; special features of language
• Routine stuff
– Source file name ends in .py
– How to run interpreter (command line or IDE)
– Program structure
• I/O
– Interactive, file; formatting
• Variables, identifiers
• Doing math
• Comments, tokens, whitespace, style conventions
continued
• Control structures
–
–
–
–
Asking questions, repeating stuff
nesting
How to make comparisons
Boolean operations: and/or/not
• Functions/procedures: parameters, returns
• Data structures
– String, dynamically-sized list, etc.
• Standard (i.e. run-time) library
– Random numbers, etc.
In the end…
• How can you tell if you’ve mastered a language?
–
–
–
–
Write a game
Given a spec or algorithm, can you implement it?
You may prefer to do something in Python!
You never need to know everything. Know where to
find stuff quickly
• This course lies between first 2 prog. courses
– You’ll see more operations & applications than in the
1st course. Not as theoretical as 2nd course.
Beginning
• Versions 2.x and 3.x
• Lab: let’s use IDE such as ActivePython
– Free, simple, includes debugger 
• Code can by entered:
– directly in shell (good for experimenting!)
– in a file, which we can save and reuse later
Initial nuts & bolts
•
•
•
•
•
Comments begin with #, go to end of line
Statements don’t end in semicolon
Don’t arbitrarily indent code
Put \ at end of line to continue statement
Variables are not declared
• Interactive output
– Version 2.6: print statement
– Version 3.0: print function
print name
print (name)
continued
• Examples
print “Hello, world”
print 7+4*3
• Interactive input
– input( ) for getting a number
– raw_input( ) for getting a string
name = raw_input(“What is your name”)
What’s next?
• Lab
• Please read chapter 2
• Practice with
–
–
–
–
–
Basic I/O
Mathematical operations
Strings
Simple if statement
Simple loop
Python – May 12
• Recap lab
• Chapter 2
–
–
–
–
operators
Strings
Lists
Control structures
Review
• Some things we saw in lab
– String slice is like Java substring, e.g. s[2:7]
– Range is a function that returns a list, and we can
iterate over this list:
for j in range(5, 10): …
– Formatting real numbers
print “Answer is {0:4.1f} feet.”.format(ans)
– Syntax error: cursor takes you location
– Initialize your variables: Interpreter remembers value
from previous run! (Even if you earlier ran a different
program)
Chapter 2
• ** and // operators
• We have += but not ++
• Boolean operators are words
• List : dynamic array.
– Commonly used functions are:
append, insert, remove
– Can use [ ] to access/change single element.
– See handout
Strings
• Some operations
–
–
–
–
Length: use len(stringName)  integer
indexOf: use stringName.find(pattern)  integer
charAt: just use brackets
Substring: just use brackets
Note that you can leave one of the arguments empty:
s[4:] means to go to the end
– String compare: can use ==, > and <.

File I/O
• File output
outFile = open(“output.txt”, “w”)
outFile.write(“help me\n”)
outFile.close()
• File input
(see CountLines.py)
inFile = open(“input.txt”, “r”)
for line in inFile:
...
inFile.close()
# Good idea to close file, as interpreter may
# remember where you left off next time!
Python – May 13
• Recap lab
• More about loops / repetition
• Functions!
Review
• Examples
–
–
–
–
Useful list functions: index, count
elif
Range with negative stride: range(6,0,-2) = [6,4,2]
Printing without automatic newline
for i in range(1,6):
print i*5,
– Yes, you can insert anywhere in a list:
L = [1,2,3]
L.insert(1,7)
 L becomes [1,7,2,3]
cont’d
• Can convert type: int(), float(), str()
– type() tells you what you currently have
Typical response: <type ‘int’>
val = input(“Enter a number”)
if str(type(val)) == “<type ‘float’>”:
# handle float case
else:
# handle int case
List comprehension
• Special case of a “loop” inside an expression.
Often we want to do something to every item in a list.
L = [ 1, 2, 3, 4 ]
M = [-x for x in L]
# brackets required
• Combining 2 lists!
A = [ 1, 2, 3]
B = [ 7, 8, 9]
[x+y for x in A for y in B]
Produces this: [8, 9, 10, 9, 10, 11, 10, 11, 12]
A = [ "parl", "jou", "pens" ]
B = [ "e", "es", "ons" "ez", "ent" ]
The same formula now gives 15 verb conjugations 
More on loops
• while True is allowed
• break & continue statements
• Important skill: error checking of input
needInput = True
while needInput:
value = input("Please enter positive number")
if value > 0:
needInput = False
else:
print("Sorry, bad input")
Functions
• Declare early
– A function must be declared before it is called. The
interpreter only reads your program once.
• Syntax
def functionName(parameters):
code
optional return statement
• Example: fun.py
Python – May 16
• Recap lab
• Simple string tokenizing
• Random numbers
• Tomorrow:
– multidimensional array (list of list)
– Exceptions
Review
• To see both code & I/O: Window  Tile
• If your program has functions, how does the interpreter
know where your program starts?
• When you write a loop with range:
for i in range (1, 10)
– i retains its last value after the loop (9)
• Look up something?
Documentation: Python Standard Library, see “index”
Link at top right of page.
Tokenizing
• Important skill in reading text input
– Input file may be in specific format
• Two techniques
– String’s split( ) function
– Python’s regular expression library  more powerful
String split( )
• Simple to use
• Good when there is a certain pattern of
characters you want as a delimiter
s = “moo--goo---gai--pan“
s.split(“-”)
[‘moo’,’’,’goo’,’’,’’,’gai’,’’,’pan’]
s.split(“--“)
[‘moo’,’goo’,’-gai’,’pan’]
s.split(“---”)
[‘moo--goo‘,’gai--pan’]
Using re
• More often, we want to handle multiple possible
delimiters.
– Like Java’s StringTokenizer or C’s strtok( )
– Extracting words by ignoring any and all punctuation
• See handout: tokenize.py
 takes your set of
delimiters and creates a “regular expression object”
i.e. tokenizer
Don’t forget the brackets around delimiters.
– tok.split(my_string)
 gives list of tokens 
– tok = re.compile( )
Random #s
• Python has a built-in library (module) called
random
At top of program, say: import random
• Two most important functions
– random.uniform(a, b) returns a random real
number between a and b inclusive
– random.randint(a, b)
does the same for int
Python – May 17
• Recap lab
• Multidimensional list
• Exceptions
• Commitment: quiz tomorrow
Lab notes
• Put delimiters inside (“[
]”)
• When you sort a list, there is no return value
L.sort()
# rather than L = L.sort()
• In lottery problem, how did you make sure all
winning numbers distinct?
– Technique also helpful when “dealing” cards.
• Can check syntax before running: next to run
button there is a “check”. Status bar shows
result, and cursor taken to error, if any
Multi-D
• Intuitive, just like other languages.
• List inside a list
[ [ 1, 2 ] , [ 8, 6 ] ]
• Access element with multiple indices: list[3][2]
• Very often associated with nested loops
• Applications?
– List of lines; list of words on the line
– Board game
Examples
• Sudoku and Reversi
1 7 5 2 9 4 8 3 6
6 2 3 1
8 9
7
4 5
5 6 3 2 7 1
5 1 9 7 3 2 4 6
3 4
8 5
1 2 9
8 6 9 4 1 7 5 3
9 3
4
4 2 5 6
7
1 3 7 9 5 8 2
7 5 2
1 8 3 9
Exceptions
• Chapter 10 covers in great detail
• Similar to Java
• Find out what could go wrong with your code
• try-except block
try :
possibly exceptional code
except Name_of_exception:
what to do in case of problem
Exceptions
• Common types
– ValueError
– IOError
– IndexError
• You can raise an exception if you discover
something wrong:
if value < 0:
raise ValueError
• Another use of “raise”:
out of your program.
raise SystemExit
is a way
Example
while True:
try:
value = input("Enter a positive number")
if value <= 0:
raise ValueError
else:
break
except ValueError:
print "Please follow directions"
continue
print value
Python – May 18
• Quiz
• Sorting
• Relatives of the list:
– Tuple
– Dictionary
– Set
– We’ll do more with dictionaries later
Lab notes
• Interactive run – hard to tell when program’s
output begins and ends – what can you do?
• Review Battleship
– How would you change to handle real game?
Sorting
• Often we want to sort data in a list, but our list
does not contain atoms like single numbers.
• Need to tell Python how to compare elements
• Analogous to comparators in Java.
• Steps:
– Create function, taking 2 arbitrary elements from your
list. Return positive / negative / zero.
– Call:
list.sort(comparatorFunction)
– See example handout (compare.py)
Tuple
• Similar to list: use ( ) instead of [ ]
• Good for identifying a point in some space, like
an ordered pair, triple
• Often an anonymous object
• Immutable – not meant to be updated, just throw
away
– We just have count & index functions
• Syntax is straightforward: multiple assignment
– Can re-use variables later
(a, b, c) = (10, 8, 2)
Dictionary
• A nice array
• Index can be anything that Python can easily
evaluate, such as a single int, float or string.
• Typical procedure:
– Initialize as { }
– Add element by assignment, e.g. d[“USA”] = 308
– Can traverse dictionary elegantly
for i in d:
print i, d[ i ]
Notes
• In a dictionary, like a set, the order of the data is
irrelevant. The “key” is already an index.
• Example:
{ “Finland”: 5, “India”: 1150, “USA”, 308, “France” : 61 }
• Don’t rely on Finland being at the “beginning” of the
dictionary. The value 5 is obtained by d[“Finland”], not
d[0] !
• Python can quickly find your data in a dictionary 
Illustration
d = { }
d[“Tina”] = 3
d[“Kevin”] = 2
“Kevin” in d  returns True
“James” in d  returns False
d [“James”] gives a KeyError, so when in doubt, check to
see that key actually exists!
Applications
• Good for sparse array: only store the values you
actually use.
1920
1951
1960
1986
2000
2010
90
150
650
6656
18768
37728
230
700
750
3176
5144
9572
• Here, the value at each key can be a list:
f [1920] = [90, 230]
f [1920][0] = 90
Applications (2)
• Excellent for maintaining data
– E.g. Stock portfolio: reading a list of buy/sell
transactions. At any point in time we may want to
know total holdings
– Can remove an element from a dictionary using del:
del portfolio[“IBM”]
del enemy[“Soviet Union”]
Set
• Essentially, a set is a list in which the elements do not
repeat. (See chapter 7)
– E.g. Useful when you need a lot of boolean values
• Can convert a list to a set by using set( ).
s = set([1, 3, 5, 7, 9])
• If you call set( ) on a string, you get a set of its
characters!
• Operations
in
not in
&
|
^
Note that you can’t use ~
Python – May 19
• Review
– What is the difference between: list, tuple, set,
dictionary?
– When is it appropriate to use each?
• Creating our own data types: classes/objects
• Reminder: “meeting” program due tomorrow
Paradigms
• Note that there are 3 ways to approach
programming
• Procedural – follow definition of program: list of
operations to perform: verb focus
• Object oriented – noun focus: define a type with
its attributes and operations
• Functional – everything is a function call; loops
written as recursion
OO
• A different way to approach problem solving –
think about the nouns first.
• Python supports OO design.
– We can define attributes & operations that belong to
instances of the class.
• Everything is public – no information hiding.
Example
class Rectangle:
length = 4
width = 3
# Static & default values
def area(self):
return self.length * self.width
# -----------------------------------------r = Rectangle()
print r.length
print r.area()
Example (2)
• More realistic to allow instances to be different! To
initialize attributes, use special function called __init__
class Triangle:
def __init__(self, side1, side2, side3):
self.a = side1
self.b = side2
self.c = side3
def perimeter(self):
return self.a + self.b + self.c
t = Triangle(3,4,5)
print t.perimeter()
Notes
• Python not the best language for true OO.
• Unfortunately, can’t have more than 1
“constructor”
• When you call an instance method, you literally
pass it with 1 fewer parameter than in the
declaration – leave off the “self”.
definition:
def giveRaise(self, percent): …
call:
bob.giveRaise(4)
Careful!
class Rectangle:
length = 4
width = 3
def __init__(self, L, W):
self.length = L
self.width = W
# --------------------------------r1 = Rectangle(10, 8)
print Rectangle.length
# equals 4
print r1.length
# equals 10
Moral – if you have some static values you want to share, don’t confuse
yourself by using same name as attribute. Now you see why we
always use “self”.
Python – May 20
• Reading data from the Web
– Example: web.py
• Practice with large amount of I/O
– See lab sheet
Web input
• Just like Java, Python has an easy way to
automatically read a Web page
import urllib2
page = urllib2.urlopen(“http://--------”)
• Can continue as if “page” is an input file
for line in page:
• Example: web.py reads a stock quote.
Large input
• Algorithm needs to be practical, in addition to
correct. Beware of nested loops
• Comments about the data; “density”
• #3 - County population, area, density
– Assume that all blocks in county are listed together.
• #5 – Creating tract-level data
– Need to look up tract in dictionary search for tract,
because blocks in input file are not sorted by tract
– Once you are finished with a county, you can output
all its tracts and empty the dictionary.
Python – May 25
• Quiz
• Python’s imaging library
• Lab
– Creating images
– Pretty parenthesizer problem
• Questions about homework?
PIL
• Python imaging library
– First, download if necessary
– Defines “Image” and many useful functions
– We’ll create images!
• Practical note:
– Need to add PIL to Python’s search path
– “import sys” and print sys.path to see what is
ordinarily included
– Since sys.path is a list, you can simply append the
PIL folder to it…. if necessary
3 steps
• First, create image object
image = Image.new(“RGB”, (100,100))
2 parameters are mode and dimensions
• Next, paint your pixels
usually in a nested loop of x & y values
image.putpixel((x,y),(red,green,blue))
• Finally, write image to file
image.save(“myimage.jpg”)
Does not have to be jpg.
See
handout
Reference
http://www.pythonware.com/library/pil/handbook
• These sections are especially helpful
– Concepts
– Image: Easy to use
– ImageDraw
A “draw” object can create geometric shapes like
circles, rectangles and polygons.
• Can also be used to read or modify existing
image 
image.getpixel((x, y))  returns RGB tuple
Python – May 26
• Persistent objects
• Experimenting with time
Persistence
• You may find it helpful to write a large object to a
binary file.
import cPickle
file = open(“outputfile”, “wb”)
cPickle.dump(aLotOfData, file)
file.close()
file2 = open(“outputfile”, “rb”)
newlist = cPickle.load(file2)
file2.close()
Time module
import time
• time.localtime()
returns a list containing:
– Year, month, day, hour, min, sec, day#, Julian day
• time.clock()
– Returns 0 the first time you call it
– Otherwise returns a float # of seconds since first call.
– Good to subtract 2 consecutive calls to time a
segment of code.
• time.sleep(# of seconds)
Timing quick event
• How long does it take to add an element to a
list?
t1 = time.clock()
# loop to add 100 elements
t2 = time.clock()
print (t2-t1)/100
What’s wrong with this approach?
Python – May 27
• How to use modules in Python
• Examples
– Printing large digits one at a time
– Defining a class in another file
– Careful if module has a function with same name
• Lab
• Last homework: map of population change
Modules
• Typical design: put functions in a separate file,
freeing up “main” program to do the important
action.
• Not required, but convenient if same functions
used in multiple programs!
• At top of source file:
import module_name
reload (module_name)
# required
# convenient!
Examples
• Suppose other file is called bank.py
import bank
x = bank.computeInterest(75, 3, 2)
# Saying “bank.” is not required unless we have a
# name conflict.
• If you want to use a class defined in another
source file, you have to name both the module
and the class
import shapes
r1 = shapes.rectangle(10, 8)
print r1.area()
Python – May 31
• Multi-threaded applications
• Outline for final
• Lab & surveys
Thread intro
• Also called “lightweight process”
• One process may have multiple threads of execution
• Allows a process to do 2+ things concurrently 
– Games
– Simulations
• Even better: if you have 2+ CPU’s, you can execute in
parallel
• Multicore architecture  demand for multithreaded
applications for speedup
• More efficient than using several concurrent processes
Thread steps
• Import the “threading” module
• Create a class
– Extends the existing threading.Thread class
– Constructor: initialize each thread with some unique
information
– Implement run( ) function: this is the work you want
the thread to do
• Main program:
– Create threads 1 at a time, put them in list
– “start” them: this invokes their run( ) functions
– “join” them: i.e. wait for them to finish
Final
Two parts
• Written part, up to 1 hour, 50%
– 12-15 questions
• Programming part, remaining 2 hours, 50%
–
–
–
–
–
5 questions
Once you start, you cannot go back to written part
Open book, open note
You may use ActivePython’s copious documentation
Do not use Internet
Images
import Image
• Create √
– Initialize mode and dimensions of image
– image.putpixel( )
– Save
• Read 
– image.getpixel((x, y))  returns RGB tuple
Python – May 19
• Discuss lab
• Bitwise operators
Lab notes
– the statement that does nothing
copy.deepcopy() – make a true copy of an object,
avoid aliasing error
• pass
•
import copy
list = [4,6]
list2 = copy.deepcopy(list)
list[0] = 9
print list2
Bitwise
• Python includes bitwise operations
– Extremely fast… efficient in time, energy, space.
– Applications: lots of boolean values, saving space if
you know you have a value requiring few bits.
– Two kinds of operators: logical and shift.
• Logical Operators
& means “and”
| means “or”
^ means “exclusive or”
~ means “not” – this is a unary operator
How they work
• Truth table (for each bit position)
X
Y
& (and)
| (or)
^ (xor)
1
1
1
1
0
1
0
0
1
1
0
1
0
1
1
0
0
0
0
0
• Most important facts:
– Anything “and-ed” with 0 becomes 0
– Anything “or-ed” with 1 becomes 1
– Anything “xor-ed” with 1 becomes inverted
Example
• Integers usually hold 32 bits. For simplicity, let’s
just show 8.
x = 1010 1101
y = 1111 0000
-------------------x & y =
x | y =
x ^ y =
Answer
• Integers usually hold 32 bits. For simplicity, let’s
just show 8.
x = 1010 1101
y = 1111 0000
-------------------x & y = 1010 0000
x | y = 1111 1101
x ^ y = 0101 1101
Binary number
• Based on powers of two:
…
256
128
64
32
16
8
4
2
1
0
1
0
1
0
1
1
0
1
This number equals 128+32+8+4+1 = 173
• Interesting formula for ~
~n = – (n + 1)
Shorthand
• Occasionally we need to write a large number in
binary. Better to do so as “hexadecimal” – a
shorthand that compresses 4 bits in one symbol.
• Notation: begin with “0x” followed by, usually, 8
digits.
• Each digit is: 0-9, a-f
a = ten = “1010”
b = eleven = “1011”
c = twelve = “1100”
d = thirteen = “1101”
e = fourteen = “1110”
f = fifteen = “1111”
• Example: What does 0x7ffc0 look like in binary?
Shift operations
• Two directions
– Shift left (<<) basically means to multiply by a power
of 2.
– Shift right (>>) essentially divides by a power of 2 and
eliminates the remainder.
• For example
1 << 4 = 2**4 = 16
5 << 3 = 5 * 2**3 = 40
13 >> 1 = 13 / 2 = 6
Practical use
• Let’s suppose we want to use an integer to hold
31 boolean values, representing the days of a
month. We’ll use bit positions 1-31 only.
• Start with may = 0, clearing all bits.
• To turn on a certain bit, we need to “or with 1”
may = may | (1 << 19)
• To turn off a bit, we need to “and with 0”
may = may & ~(1 << 19)
• Multiple bits?
Python – May 20
• Recap lab:
– Answers to question #3?
• Finish bitwise operators
• Set data type
Bitwise
• Note – Python integers can have > 32 bits 
• Operators
• Masks
– Used to determine exactly which bits we want to turn
on, turn off or invert
• Applications
– Bit vectors: use integer as “array” of booleans
– Bit fields: pack more than one value
How to…
Useful operations for a bit vector:
•
•
•
•
•
Turn on a bit
Turn off a bit
Invert a bit
Determine if a bit is 1
Determine if a bit is 0
Mask
• Often we want to modify more than 1 bit, so we
need to create an operand that looks like:
– Ones in middle: 0000000000111110000000
– Zeros in middle: 1111111000001111111111
• How? Two ways
– Can subtract powers of 2
– Use only bitwise operations
Examples
•
•
•
•
~0
~0 << 5
~(~0 << 5)
~(~0 << 5) << 4
… 11111111 11111111
… 11111111 11100000
… 00000000 00011111
… 00000001 11110000
• And you can invert one more time if needed.
• How could we write this as a difference of
powers of 2 ?
Bit field
• Store more than one value in a variable, to save
space.
• Example: date as month-day-year
– How many bits do we need for each?
date
date
date
date
Month
Day
Year
4
5
11
= 0
|= month << 16
|= day << 11
|= year
Set
• The big problem with bitwise operators is that
they are low level. Rely too much on data
representation, which may distract from our
algorithm.
• We can achieve similar functionality with the set
data type
• Create, add/delete elements, operations (Ch. 7)
Python – May 21
• Recap lab
• OOP
Review
• Sometimes we check a bit to see if it’s set.
result = myValue & (1 << 21)
if result != 0: …
Don’t ask if
result == 1.
• What is the algorithm to look for longest
sequence of something?
• Default parameter
• Set (useful functions p.283)
Python – May 22
• Recap lab
• Dictionary data type
Review
• How do you like OO in Python?
• Very long string literal: delimited by 3 single
quotes and can extend for multiple lines without
the \
Python – May 26
• Tuple data type
• Image processing
Python – May 27
• Homework questions?
• “Persistent” data
• Time
Python – May 28
• Recap lab
• Modules
• Notes about final
Lab review
• How were you able to distinguish the images of
dice?
• Speed of –
– Sorting
– Accessing elements of list vs. dictionary