Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
History of the function concept wikipedia , lookup
Large numbers wikipedia , lookup
Mathematics of radio engineering wikipedia , lookup
Proofs of Fermat's little theorem wikipedia , lookup
Infinite monkey theorem wikipedia , lookup
Elementary mathematics wikipedia , lookup
Slides for Introduction to Stochastic Search and Optimization (ISSO) by J. C. Spall APPENDIX D RANDOM NUMBER GENERATION • Organization of chapter in ISSO* – General description and linear congruential generators • Criteria for “good” random number generator – Random variates with general distribution • Different types of random number generators *Note: These slides cover some topics not included in ISSO Uniform Random Number Generators • Want a sequence of independent, identically distributed U(0, 1) random variables • However, random number generators (RNGs) produce a deterministic and periodic sequence of numbers • What qualities should the generators have? D-2 Criteria for ‘Good’ Random Number Generators • • • • • Long period Good distribution of the points (low discrepancy) Able to pass some statistical tests Speed/efficiency Portability – can be implemented easily using different languages and computers • Repeatability – should be able to generate the same sequence over again D-3 Generating Random Numbers • Given a transition function, f, the state at step n is given by xn f ( xn1), n 1 • The output function, g, produces the outputs as un g xn • The output sequence is un , n 1 • Want the sequence period to be close to 2b, where b corresponds to the number of bits D-4 Types of Random Number Generators • Linear – most commonly used • Combined – can increase period and improve statistical properties • Non-linear – structure is less regular than linear generators but more difficult to implement D-5 Linear Congruential Generators • U(0,1) numbers via linear congruential generators (LCG) are calculated by xn axn 1 c mod m un xn / m • These are the most widely used and studied random number generators • The values a, c, and m should be carefully chosen 0 m, 0 a m, 0 c m x0 m, xk 0,1, , m 1 D-6 Linear Congruential Generators • Some values for a and m (assuming c = 0) – a = 23, m = 108+1 (original implementation) – a = 65534, m = 229 (poor because of high order correlations) – a = 515, m = 247 (long period, good distribution, but lower order bits should not be trusted) – a = 16807, m = 231 –1 (this has been discussed as the minimum standard for RNGs) D-7 0.58 Empirical Mean 0.56 0.54 m = 231 – 1, a = 4, c = 1 0.52 0.5 m = 482, a = 13, c = 14 0.48 m = 27, a = 26, c = 5 0.46 0.44 m = 9, a = 4, c = 1 0.42 0 500 1000 1500 2000 Number of Samples D-8 Lattice Structure (Exercise D.2) 30 points 96 points 1 1 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0 0 0.2 0.4 0.6 Uk 0.8 1 0 0 0.2 0.4 0.6 0.8 Uk D-9 1 Fibonacci Generators • These are generators where the current value is the sum (or difference, or XOR) or the two preceding elements • Lagged Fibonacci generators use two numbers earlier in the sequence xn xn p xn r mod m u n xn / m p, q are the lags D-10 Multiple Recursive Generators • Multiple recursive generators (MRGs)are defined by xn a1xn 1 ak xn k mod m un xn / m where the ai belong to {0,1,…,m – 1} and • For prime m and properly chosen ai’s, the maximal period is mk-1 sn xnk 1, , xn D-11 Combining Generators • Used to increase period length and improve statistical properties • Shuffling: uses the second generator to choose a random order for the numbers produced by the final generator • Bit mixing: combines the numbers in the two sequences using some logical or arithmetic operation (addition and subtraction are preferred) D-12 Nonlinear Generators • Nonlinearity can be introduced by using a linear transition function with a nonlinear output function • An example is the explicit inversive generator where xn an c zn an c m2 mod m un zn / m D-13 Random Number Generators Used in Common Software Packages • Important to understand the types of generators used in statistical software packages and their limitations • MATLAB: – Versions earlier than 5: a linear congruential generator with a 7 16807; c 0; m 2 1 2147483647 5 31 – Versions 5 & 6: a lagged Fibonacci generator combined with 1492 a shift register random integer generator with period ~ 2 • EXCEL: un = fractional part (9821×un –1 + 0.211327); period ~ 223 • SAS (v6): LCG with period ~ 231 1 D-14 Inverse-Transform Method for Generating Non-U(0,1) Random Numbers • Let F(x) be the distribution function of X • Define the inverse function of F by F 1( y ) inf x : F ( x ) y ,0 y 1. • Generate X by X F 1(U ) • Example: exponential distribution F ( x ) 1 e x 1 X F 1(U ) ln(1 U ) D-15 AcceptReject Method • Let pX(x) be the density function of X • Find a function f(x) that majorizes pX(x) – f( x ) ch( x ), c 1, q is a density function • Generate X by – Generate U from U(0,1) (*) – Generate Y from q(y), independent of U pX (Y ) – If U , then set X=Y. Otherwise, go back to (*) f (Y ) • Probability of acceptance (efficiency) = 1/c • Related to Markov chain Monte Carlo (MCMC) method (see Exercise 16.4) D-16 60 x 3 (1 x )2 pX ( x ) 0 Y ~ q( y ) U (0,1) if 0 x 1 otherwise 60Y 3 (1 Y )2 U 2.0736 2.5 f( x ) cq( x ) 2.0736 U (0,1) 2.0 pX(x) 1.5 1.0 q(x) = U(0,1) 0.5 0 0.2 0.4 0.6 0.8 1.0 1.2 D-17 U ~ U(0,1): 0.9501, 0.2311, 0.6068, 0.4860, 0.8913, Y ~ q(y) U(0,1): 0.7621, 0.4565, 0.0185, 0.8214, 0.4447, pX (Y ) : 0.7249, 0.8131, cq(Y ) X ~ PX(x): 0.7621, 0.4565, reject accept D-18 References for Further Study • L’Ecuyer, P. (1998), “Random Number Generation,”in Handbook of Simulation: Principles, Methodology, Advances, Applications, and Practice (J. Banks, ed.), Wiley, New York, Chapter 4. • Neiderreiter, H. (1992), Random Number Generation and Quasi-Monte Carlo Methods, SIAM, Philadelphia. D-19