Download List Comprehensions and Simulations

List Comprehensions and Simulations
1
List Comprehensions
examples in the Python shell
zipping, filtering, and reducing
2
Monte Carlo Simulations
testing the normal distribution
the Mean Time Between Failures (MTBF) problem
MCS 507 Lecture 5
Mathematical, Statistical and Scientific Software
Jan Verschelde, 6 September 2013
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
1 / 24
List Comprehensions and Simulations
1
List Comprehensions
examples in the Python shell
zipping, filtering, and reducing
2
Monte Carlo Simulations
testing the normal distribution
the Mean Time Between Failures (MTBF) problem
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
2 / 24
using for in lists
Computing the square of some numbers:
>>>
>>>
[0,
>>>
[0,
L = range(5)
L
1, 2, 3, 4]
[x**2 for x in L]
1, 4, 9, 16]
The general syntax of a list comprehension is
[ expression(x) for x in somelist ]
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
3 / 24
range versus xrange
With range we create a list. So:
>>> L = [x**2 for x in range(0,5)]
1
creates the list [0, 1, 2, 3, 4]
2
lets x run through that list.
If we are interested only in the end result,
then generating [0, 1, 2, 3, 4] is wasteful.
With xrange, the element in the range is generated only when needed
and the entire range is not stored.
>>> L = [x**2 for x in xrange(0,5)]
>>> L
[0, 1, 4, 9, 16]
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
4 / 24
listing all characters
Every character is encoded by one byte:
>>> ord(’a’)
97
>>> chr(97)
’a’
To list all characters:
>>> a = ord(’a’)
>> z = ord(’z’)
>>> [chr(i) for i in xrange(a,z+1)]
[’a’, ’b’, ’c’, .. , ’y’, ’z’]
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
5 / 24
string representations
The str() function is defined on a list:
>>> L = range(5)
>>> s = str(L)
>>> L
[0, 1, 2, 3, 4]
>>> s
’[0, 1, 2, 3, 4]’
>>> eval(s)
[0, 1, 2, 3, 4]
We may prompt for a list:
>>> K = input(’enter a list : ’)
enter a list : [1,2,3]
>>> K
[1, 2, 3]
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
6 / 24
List Comprehensions and Simulations
1
List Comprehensions
examples in the Python shell
zipping, filtering, and reducing
2
Monte Carlo Simulations
testing the normal distribution
the Mean Time Between Failures (MTBF) problem
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
7 / 24
zipping lists
>>> L = range(5)
>>> a = ord(’a’); a
97
>>> chr(a)
’a’
The name a refers to the string ’a’,
which is encoded by the byte with value 97.
>>> K = [chr(i) for i in xrange(a,a+5)]
>>> K
[’a’, ’b’, ’c’, ’d’, ’e’]
>>> L
[0, 1, 2, 3, 4]
>>> Z = [[a,b] for a, b in zip(L,K)]
>>> Z
[[0, ’a’], [1, ’b’], [2, ’c’], [3, ’d’], [4, ’e’]]
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
8 / 24
lambda expressions
We can make short functions with lambda:
>>> f = lambda x: x**2
>>> type(f)
<type ’function’>
>>> f(3)
9
Lambda expressions are used in other functions,
for example: numerical integration,
and are in list processing functions.
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
9 / 24
filtering lists
Find all numbers between 0 and 99
that are divisible by 3; and
that have 2 as a last digit.
First we define a filter function:
>>> f = lambda x: x % 3 == 0 and x % 10 == 2
>>> f(12)
True
>>> f(11)
False
Then we apply the filter f to a list:
>>> L = range(100)
>>> F = filter(f,L)
>>> F
[12, 42, 72]
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
10 / 24
reducing lists
To compute the sum of the elements in a list L: sum(L).
The factorial of 10 is 1 × 2 × · · · × 10:
>>> L = range(1,11)
>>> reduce(lambda x,y: x*y, L)
3628800
The reduce() calculated (..((1 ⋆ 2) ⋆ 3)..) ⋆ 10
The function given as argument to reduce() must
take two elements on input,
return one single element.
reduce() repeatedly replaces the first two elements of the list by the
result of the function, applied to those first two elements, until only one
element in the list is left.
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
11 / 24
the Horner form
The Horner form of a polynomial is
2x 2 − 8x + 3 = ((2x − 8)x + 3.
To evaluate a dense polynomial of degree d
we need d additions and d multiplications.
We apply reduce() to a polynomial with integer coefficients stored in
a list:
>>> c = [2, -8, 3]
>>> h = lambda x,y: ’(’ + str(x) + ’*x’ \
... + (’%+d’ % y) + ’)’
>>> h(2,-8)
’(2*x-8)’
>>> h(2,8)
’(2*x+8)’
>>> reduce(h,c)
’((2*x-8)*x+3)’
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
12 / 24
List Comprehensions and Simulations
1
List Comprehensions
examples in the Python shell
zipping, filtering, and reducing
2
Monte Carlo Simulations
testing the normal distribution
the Mean Time Between Failures (MTBF) problem
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
13 / 24
testing random.gauss
The normal distribution is provided by the function
gauss in the builtin module random of Python.
To test gauss, given mean µ and standard deviation σ,
we generate a number n of samples, and
for the n samples,
we compute the mean µ
b and standard deviation σ
b.
For large enough n, the computed µ
b ≈ µ and σ
b ≈ σ.
We count the number of samples within the standard deviation of
the mean, that is: inside [σ − µ, σ + µ].
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
14 / 24
running testgauss.py
$ python testgauss.py
testing the normal distribution
give the mean : 5
give the standard deviation : 2
give the number of samples : 10000
average of samples : 5.00381821545
standard deviation : 1.99589251836
#samples in [3.00,7.00] : 6856
smallest sample : -3.03178366925
largest sample : 12.9686083287
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
15 / 24
the script testgauss.py
from random import gauss
from math import sqrt
print ’testing the normal distribution’
MU = input(’give the mean : ’)
SIGMA = input(’give the standard deviation : ’)
DIM = input(’give the number of samples : ’)
L = [gauss(MU, SIGMA) for i in xrange(DIM)]
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
16 / 24
testgauss.py continued
AVG = sum(L)/DIM
SQR = [(e - AVG)**2 for e in L]
STD = sqrt(sum(SQR)/DIM)
A = MU - SIGMA
B = MU + SIGMA
F = filter(lambda x: A < x < B, L)
print ’average of samples : ’, AVG
print ’standard deviation : ’, STD
print ’#samples in [%.2f, %.2f] : %d’ % (A, B, len(F))
print ’
smallest sample : ’, min(L)
print ’
largest sample : ’, max(L)
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
17 / 24
deprecated-lambda in map/filter
pylint warns about
F = filter(lambda x: A < x < B, L)
In particular:
W: 26, 4: Used builtin function ’filter’ (bad-builtin)
W: 26, 4: map/filter on lambda could be replaced
by comprehension (deprecated-lambda)
A better construction (if only interested in len(F)) is
TEST = lambda x: A < x < B
F = [TEST(e) for e in L]
print sum(F)
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
18 / 24
List Comprehensions and Simulations
1
List Comprehensions
examples in the Python shell
zipping, filtering, and reducing
2
Monte Carlo Simulations
testing the normal distribution
the Mean Time Between Failures (MTBF) problem
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
19 / 24
mean time between failures
The Mean Time Between Failures (MTBF) problem asks for the
expected life span of a product made of components.
Every component is critical. The multi-component product fails as soon
as one of its components fails.
For every component we assume that the life span follows
a known normal distribution, given by µ and σ.
For example, consider 3 components with respective means 11, 12, 13
and corresponding standard deviations 1, 2, 3.
Running 10,000 simulations, we compute the average life span of the
composite product.
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
20 / 24
the script mtbf.py
from random import gauss
from math import sqrt
(MEAN1, SIGMA1) = (11, 1)
(MEAN2, SIGMA2) = (12, 2)
(MEAN3, SIGMA3) = (13, 3)
DIM = 10000
L1 = [gauss(MEAN1, SIGMA1) for i in xrange(DIM)]
L2 = [gauss(MEAN2, SIGMA2) for i in xrange(DIM)]
L3 = [gauss(MEAN3, SIGMA3) for i in xrange(DIM)]
L = [min(x, y, z) for x, y, z in zip(L1, L2, L3)]
AVG = sum(L)/DIM
SQR = [(e - AVG)**2 for e in L]
STD = sqrt(sum(SQR)/DIM)
print ’average of samples : ’, AVG
print ’standard deviation : ’, STD
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
21 / 24
running mtbf.py
$ python mtbf.py
average of samples :
standard deviation :
$
10.1287785075
1.35543878815
Some extensions to the example:
product fails if two of its components fail; or
product fails if component one and two both fail
or if component three fails.
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
22 / 24
Summary + Exercises
We covered more of chapter 2 of the text book.
Exercises:
1
Make a list of all the digits of math.pi.
2
A sieve method to find all primes in a range 2 to n removes all
multiples of 2, 3, 5, etc. Apply the filter function to sieve for all
primes between 2 and 100.
3
Use list comprehensions to generate points (x, y) uniformly
distributed on the circle: x 2 + y 2 = 1.
Use random.uniform(0,2*math.pi) for random angles t:
x = cos(t), y = sin(t), cos2 (t) + sin2 (t) = 1.
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
23 / 24
more exercises
4
5
Generate a sequence of cubes x 3 , for x ranging over all natural
numbers between 1 and 70. To determine which cubes are
squares of natural numbers, take the square root of those cubes
and select those numbers which have a natural number as their
square root.
Extend mtbf.py so that it works for any number of components,
prompting the user for a list of means and a list of corresponding
standard deviations.
The first homework collection is on Monday 9 September, at 9AM.
Bring to class your answers to exercise 3 of Lecture 1;
exercises 1, 2 of Lecture 2; and exercises 1, 3 of Lecture 3.
Along with a paper version, also email me your scripts.
Scientific Software (MCS 507 L-5)
Lists Comprehensions and Simulations
6 September 2013
24 / 24