Download Generating pseudo

Generating pseudo- random numbers
What are pseudo-random numbers?
Numbers exhibiting statistical randomness while being generated by a deterministic
process.
Easier to generate than true random numbers?
What is a good pseudo-random number?
From the correct distribution (especially the tails!)
Long periodicity
Independence
Fast to generate
Most standard computing languages have packages or functions that generate either
U (0, 1) random numbers or integers on U (0, 232 − 1).
rand or unifrnd in MATLAB
rand in C/C++
– p. 1/??
Linear congruential generator (LCG)
The linear congruential generator is a simple, fast, and popular way of generating
random numbers,
xi = (axi−1 + c)
mod m.
This will generate integer random numbers on [0, m − 1] which are mapped to U (0, 1)
after division by m. The period of the generator is m if
c and m are relative prime
a − 1 is divisible by all prime factors of m
a − 1 is divisible by 4 if m is divisible by 4.
The LCG has problems with high serial correlation, and is sensitive to the choice of
parameters.
– p. 2/??
Mersenne twister
Developed in 1997 by Makoto Matsumoto and Takuji Nishimura.
Called Mersenne twister since it uses Mersenne prime numbers 2p − 1.
The most commonly used version is the MT19937 algorithm.
Standard generator in MATLAB (from ver. 7.4), Maple, R, and GNU Scientific Library.
Period of the generated numbers is 219937 − 1.
Equidistributed in high dimensions (LCG has problems with d > 5).
– p. 3/??
Simulation algorithms
After having generated a sample of pseudo-random numbers u1 , . . . , un ∈ U (0, 1), we
can simulate data from a specified distribution function by either:
inversion and transformation methods
rejection sampling
conditional methods
– p. 4/??
Inversion method
Let U ∈ U (0, 1) be a uniform random variable.
We want random variable X ∈ R from a continuous distribution F .
Does a simple transform x = h(u) that is increasing with an inverse
h−1 : R → [0, 1] exist?
X = F −1 (U ) is a r.v with distribution F .
For discrete distributions F −1 (u) does not exist. Take instead inf{x; F (x) ≥ u}
Limited to cases where:
The inverse of the distribution function exists and is easy to evaluate.
We want univariate random numbers.
– p. 5/??
Exponential distribution- an example
To simulate from the exponential distribution F (x) = 1 − exp(−λx), so
F −1 (u) = −λ−1 log(1 − u).
1
500
0.9
450
0.8
400
0.7
350
0.6
300
frequency
u
u=rand(1,n);
x=-(log(1-u))/lambda;
0.5
250
0.4
200
0.3
150
0.2
100
0.1
50
0
0
1
2
3
4
x
5
6
7
8
0
0
1
2
3
4
5
6
7
x
Left : cdf for Exp(1). Right: histogram of 1000 unit exponential
– p. 6/??
Poisson(2) distribution - an example
The distribution function of the Poisson(2) distribution is
x
F(x)
x
F(x)
0
0.1353353
5
0.9834364
1
0.4060058
6
0.9954662
2
0.6766764
7
0.9989033
3
0.8571235
8
0.9997626
4
0.9473470
9
0.9999535
10
0.9999917
Generate a seq of standard uniforms (s.u) u1 , . . . , un
∀ui get xi ∈ P o(2): F (xi−1 ) < ui ≤ F (xi )
For u1 = 0.7352, we get x1 = 3 and for u2 = 0.4256, we get x2 = 2 and so on...
– p. 7/??
Transformation method
The inversion method is a special case of a more general class of methods called:
transformation method.
Theorem 1: Let X have a continuous density f and let h be a differentiable function with a
differentiable inverse g = h−1 . Then the random variable Z = h(X) has density
f (g(z))|g 0 (z)|.
Can be generalized to multivariate densities and cases where the inverse of h does not
exist.
Examples: Γ(n, λ), N (µ, σ 2 ), χ2 (1)
Theorem 2: Let X and Y be independent random variables that take values in R with
densities fx and fy , then the density of Z = X + Y is
Z
fz (z) =
fx (z − t)fy (t)dt.
Examples: Γ(k, 1), χ2 (n), Bin(n, p)
– p. 8/??
Rejection sampling
Suppose we can simulate from some density g. we can use this as the basis for
simulating from the continuous density f by simulating Y from g and then accepting this
simulated value with a prob proportional to f (Y )/g(Y ). Specifically, let c be a constant
s.t
f (y)
≤ c, ∀y
g(y)
Algorithm:
1. Simulate Y from g and simulate a random number U .
2. If U ≤ f (Y )/cG(Y ), set X = Y . Otherwise return to step 1.
Remarks:
The way in which we "accept the value Y with prob f (Y )/cg(Y )" is by generating a
r.number U and then accepting Y if U ≤ f (Y )/cg(Y )
Since each iteration will, independently, result in an accepted value with prob
P (U ≤ f (Y )/cg(Y )) = K = 1/c, the number of iterations follows a geometric
distribution with mean c.
Examples: Γ(k, 1), χ2 (n), Bin(n, p)
– p. 9/??
An example
Want to simulate from f (x) ∝ x2 e−x ; 0 ≤ x ≤ 1, a truncated gamma distribution
1
25
0.9
0.8
20
0.7
exp(−x)
0.6
15
0.5
0.4
10
0.3
0.2
5
0.1
0
0
0.5
1
1.5
2
2.5
x
3
3.5
4
4.5
5
0
0.2
0.3
0.4
0.5
0.6
x
0.7
0.8
0.9
1
Left: Scaled density and envelope Right: Histogram of simulated data.
– p. 10/??
Conditional Methods
If we can decompose a multivariate density into conditional parts, the problem of
sampling from a multivariate density reduces to sampling from several univariate
densities.
f (x1 , . . . , xn )f1 (x1 )f2 (x2 |x1 ) · · · fn (xn |xn−1 , . . . , x1 ).
The problem is that usually is hard to find the conditional densities since this involves:
Z
Z
f1 (x1 ) =
. . . f (x1 , . . . , xn )dx2 . . . dxn
– p. 11/??