Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Generating pseudo- random numbers What are pseudo-random numbers? Numbers exhibiting statistical randomness while being generated by a deterministic process. Easier to generate than true random numbers? What is a good pseudo-random number? From the correct distribution (especially the tails!) Long periodicity Independence Fast to generate Most standard computing languages have packages or functions that generate either U (0, 1) random numbers or integers on U (0, 232 − 1). rand or unifrnd in MATLAB rand in C/C++ – p. 1/?? Linear congruential generator (LCG) The linear congruential generator is a simple, fast, and popular way of generating random numbers, xi = (axi−1 + c) mod m. This will generate integer random numbers on [0, m − 1] which are mapped to U (0, 1) after division by m. The period of the generator is m if c and m are relative prime a − 1 is divisible by all prime factors of m a − 1 is divisible by 4 if m is divisible by 4. The LCG has problems with high serial correlation, and is sensitive to the choice of parameters. – p. 2/?? Mersenne twister Developed in 1997 by Makoto Matsumoto and Takuji Nishimura. Called Mersenne twister since it uses Mersenne prime numbers 2p − 1. The most commonly used version is the MT19937 algorithm. Standard generator in MATLAB (from ver. 7.4), Maple, R, and GNU Scientific Library. Period of the generated numbers is 219937 − 1. Equidistributed in high dimensions (LCG has problems with d > 5). – p. 3/?? Simulation algorithms After having generated a sample of pseudo-random numbers u1 , . . . , un ∈ U (0, 1), we can simulate data from a specified distribution function by either: inversion and transformation methods rejection sampling conditional methods – p. 4/?? Inversion method Let U ∈ U (0, 1) be a uniform random variable. We want random variable X ∈ R from a continuous distribution F . Does a simple transform x = h(u) that is increasing with an inverse h−1 : R → [0, 1] exist? X = F −1 (U ) is a r.v with distribution F . For discrete distributions F −1 (u) does not exist. Take instead inf{x; F (x) ≥ u} Limited to cases where: The inverse of the distribution function exists and is easy to evaluate. We want univariate random numbers. – p. 5/?? Exponential distribution- an example To simulate from the exponential distribution F (x) = 1 − exp(−λx), so F −1 (u) = −λ−1 log(1 − u). 1 500 0.9 450 0.8 400 0.7 350 0.6 300 frequency u u=rand(1,n); x=-(log(1-u))/lambda; 0.5 250 0.4 200 0.3 150 0.2 100 0.1 50 0 0 1 2 3 4 x 5 6 7 8 0 0 1 2 3 4 5 6 7 x Left : cdf for Exp(1). Right: histogram of 1000 unit exponential – p. 6/?? Poisson(2) distribution - an example The distribution function of the Poisson(2) distribution is x F(x) x F(x) 0 0.1353353 5 0.9834364 1 0.4060058 6 0.9954662 2 0.6766764 7 0.9989033 3 0.8571235 8 0.9997626 4 0.9473470 9 0.9999535 10 0.9999917 Generate a seq of standard uniforms (s.u) u1 , . . . , un ∀ui get xi ∈ P o(2): F (xi−1 ) < ui ≤ F (xi ) For u1 = 0.7352, we get x1 = 3 and for u2 = 0.4256, we get x2 = 2 and so on... – p. 7/?? Transformation method The inversion method is a special case of a more general class of methods called: transformation method. Theorem 1: Let X have a continuous density f and let h be a differentiable function with a differentiable inverse g = h−1 . Then the random variable Z = h(X) has density f (g(z))|g 0 (z)|. Can be generalized to multivariate densities and cases where the inverse of h does not exist. Examples: Γ(n, λ), N (µ, σ 2 ), χ2 (1) Theorem 2: Let X and Y be independent random variables that take values in R with densities fx and fy , then the density of Z = X + Y is Z fz (z) = fx (z − t)fy (t)dt. Examples: Γ(k, 1), χ2 (n), Bin(n, p) – p. 8/?? Rejection sampling Suppose we can simulate from some density g. we can use this as the basis for simulating from the continuous density f by simulating Y from g and then accepting this simulated value with a prob proportional to f (Y )/g(Y ). Specifically, let c be a constant s.t f (y) ≤ c, ∀y g(y) Algorithm: 1. Simulate Y from g and simulate a random number U . 2. If U ≤ f (Y )/cG(Y ), set X = Y . Otherwise return to step 1. Remarks: The way in which we "accept the value Y with prob f (Y )/cg(Y )" is by generating a r.number U and then accepting Y if U ≤ f (Y )/cg(Y ) Since each iteration will, independently, result in an accepted value with prob P (U ≤ f (Y )/cg(Y )) = K = 1/c, the number of iterations follows a geometric distribution with mean c. Examples: Γ(k, 1), χ2 (n), Bin(n, p) – p. 9/?? An example Want to simulate from f (x) ∝ x2 e−x ; 0 ≤ x ≤ 1, a truncated gamma distribution 1 25 0.9 0.8 20 0.7 exp(−x) 0.6 15 0.5 0.4 10 0.3 0.2 5 0.1 0 0 0.5 1 1.5 2 2.5 x 3 3.5 4 4.5 5 0 0.2 0.3 0.4 0.5 0.6 x 0.7 0.8 0.9 1 Left: Scaled density and envelope Right: Histogram of simulated data. – p. 10/?? Conditional Methods If we can decompose a multivariate density into conditional parts, the problem of sampling from a multivariate density reduces to sampling from several univariate densities. f (x1 , . . . , xn )f1 (x1 )f2 (x2 |x1 ) · · · fn (xn |xn−1 , . . . , x1 ). The problem is that usually is hard to find the conditional densities since this involves: Z Z f1 (x1 ) = . . . f (x1 , . . . , xn )dx2 . . . dxn – p. 11/??