Download Basics: random variables, probabilities and distributions

Random numbers and Monte Carlo
João R. T. de Mello Neto
IF - UFRJ
Inmetro, 8/7/2003
References
• The Nature of Mathematical Modeling, N.
Gershenfelder, Cambridge, 1999;
• Numerical Recipes in C, Second Edition, W.H
Press et al., Cambridge, 1992;
• Statistical Data Analysis, G. Cowan, Oxford,
1998
• Computational Physics, Dean Karlen (online),
1998
Random
O acaso não existe.
car sticker
Random numbers
Important task: generate random variables from known
probability distributions.
random numbers: produced by the computer in a strictly
deterministic way – pseudorandom (→ Monte Carlo Method)
random number generators:
x j 1  f ( x j , x j 1,..., x1 )
linear congruential generators: x j 1  (ax j  c ) mod m
ex: c=3, a=5, m=16
x0  0 x1  (5  0  3) mod16  3
x1  (5  3  3) mod16  2
0,2,3,13,4,7,6,1,8,11,10,5,12,15,14,9,0,2….
x0  x1    xm1  x0
Generators
Arguments from number theory: good values for a, c and m.
Good generators: • longest possible period
• individual elements within a period should
follow each other “randomly”
Ex. 1 RANDU
a = 65539, m=231, c=0
xn 1  65539xn mod 231
xt  2  (6 xt  9 xt 1 ) mod 231
Generators
Generators
Generators
Generators
“Random numbers fall mainly in the planes”
Marsaglia, Proc. Acad. Sci. 61, 25, 1968
What is seen in RANDU is present in any multiplicative
congruential generator.
In 32 bit machines: maximum number of hyperplanes in
the space of d-dimensions is:
d=3 2953
d=4
566
d=6
120
d=10
41
RANDU has much less than the maximum!
Generators
Ex. 2 Minimal standard generator (Num. Recp. ran0)
a = 75 =16807, m=231-1
RAN1 and RAN2, given in
the first edition, are “at best
mediocre”
Generators
cernlib (mathlib V115, F.James )
Ex. 3 RANLUX A portable high-quality random generator for
lattice field theory simulation, M. Lüscher,
period ≥ 10165
Comp. Phys. Comm. 79, 100, 1994
Functional definition: an algorithm that generates uniform
numbers in acceptable if it is not rejected by a set of tests.
Lots of literature about random generators testing. See a
set of programs (Die Hard)
http://stat.fsu.edu/~geo/diehard.html
Recommendations:
1. Do some simple tests.
2. Check the results with other generator
Inverse transform
dx 0  x  1
p( x )dx 
0 otherwise
p(x) uniform probability distribution
x: uniform deviate
g( y )dy  p( x )dx
y
x
/
/
g
(
y
)
dy

Generate y according to g(y).
dx
dx
 g (y )
g ( y )  p( x )
dy
dy
x  G(y )
y  G 1(x )
0
1
G(y)
uniform
deviate in x
0
g(y)
out
y
analytically
or numerically.
Inverse transform
Ex. 4 discrete case
f(x=0)=0.3
f(x=1)=0.2
f(x=2)=0.5
u

F(x)
1
0.5
0
0.0 ≤ u ≤ 0.3 x=0
0.3 ≤ u ≤ 0.5 x=1
0.5 ≤ u ≤ 1.0 x=2
1
2
3
x
for 2000 deviates:
x=0 0.2880
X=1 0.2075
X=2 0.5045
Inverse transform
Exponential
• amount of time until a specific event occurs;
• time between independent events (time until
a part fails);
f ( x;  )  e x , x  0,   0
u  F (x )
e
 x
x
 1 u
1
log(1  u )

1
x   log u

F(x)  1  e
 x
Acceptance-rejection method
x  ( xmax  xmin )u  xmin
ymax
y  y maxu
f(x)
0
xmin
xmax
if (x,y) under the curve, accept
the point, else, discard.
A  ( xmax  xmin )y max
xmax
Ef 
I
I
 f ( x )dx
xmin
A

1
N

nacc
p
N
1 p 


 p 
nacc
I
A I 
N
I
I

1
N
A
Np(1  p )
N
Acceptance-rejection method
Importance sampling
g(x): random numbers are
easy to generate
g(x)
f(x)
•generate random x
according to g(x)
• generate u uniformily
between 0 and g(x);
• if u < f(x), accept x, if
not, reject.
Multivariate Normal
Very useful
Starts with a d-dimensional vector of standard normal
Random numbers;
Transformed to the desired distribution using:

z


R


T
x R z
dX1 vector of standard normal random numbers
dX1 vector representing the mean
dXd matrix such that
R R
T
covariance
matrix desired
(Cholesky factorization of the cov. matrix)
Multivariate Normal
   2 3
 1 0.7


0
.
7
1


Random walk
10 simulations
1K points each