Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Georg Cantor's first set theory article wikipedia , lookup
Large numbers wikipedia , lookup
Elementary mathematics wikipedia , lookup
Series (mathematics) wikipedia , lookup
Proofs of Fermat's little theorem wikipedia , lookup
Infinite monkey theorem wikipedia , lookup
Monte Carlo Methods • So far we have discussed Monte Carlo methods based on a uniform distribution of random numbers on the interval [0,1] • p(x) = 1 0 x1 p(x)= 0 otherwise • the hit and miss algorithm generates pairs of points (x,y) and either accepts or rejects the point • the sample mean method samples points uniformly on the interval [a,b] The above uses uniformly distributed random numbers to sample the integrand. It is desirable to sample f(x) more often in regions where f(x) is large or rapidly varying. b b f ( x) I f ( x)dx p( x) dx p ( x) a a f I p 1 N N i 1 f ( xi ) p( xi ) Choose p(x) so that f(x)/p(x) is a fairly constant function Importance sampling requires nonuniform probability distributions Nonuniform probability distributions • Consider a probability density p(x) such that p(x)dx is the probability that event x is in the interval between x and x+dx • + We have p(x) dx = 1 - • x Calculate P(x) = p(x’) dx’ = r = dr - • p(x’)dx’ = dr ==> p(x’)= dr/dx’ • r is a uniform random number on the interval [0,1] • Invert this and solve for x in terms of r Eg.1 suppose we want p(x)= 1/(b-a), a x b = 0 otherwise • x • Calculate P(x) = p(x’) dx’ • a • r = (x-a)/(b-a) • Solving for x, x= a + (b-a) r • obvious! Eg.2 p(x) = (1/) exp(-x/), [0,] = 0 x<0 • Hence r = P(x) = 1 - exp(-x/) • And x = - ln(1-r) = - ln(r) • This technique is only feasible if the inversion process can be carried out. Eg.3 p(x) = (1/ 22)1/2 exp(-x2/22) P(x) = ? • However the two-dimensional distribution • p(x,y)dxdy = (1/ 22) exp(-(x2+y2)/22)dxdy can be integrated by a change of variable • = r2/2= (x2+y2)/22 , tan =y/x • p(,)dd = (1/2) exp(-)dd • Generate uniformly on the interval [0,2] i.e. = 2 r • Generate according to the exponential distribution with = 1 i.e. = -ln r • x= (22)1/2 cos and y=(22)1/2 sin are Gaussian distributed Importance Sampling • The error estimate in Monte Carlo is proportional to the variance of the integrand: • (<f2> - <f>2 )1/2 n1/2 • How can we reduce the variance? • b Introduce a function p(x) such that p(x)dx = 1 a • And rewrite b F = [f(x)/p(x)] p(x) dx a Importance Sampling • Evaluate the integral by sampling according to p(x): • Fn = (1/n) f(xi )/p(xi ) • Choose p(x) to minimize variance of f(x)/p(x) • i.e try to make f(x)/p(x) slowly varying since a constant has zero variance eg. 1 F = exp(-x2) dx = .746824 0 Sample [0,1] uniformly n Fn n 100. 0.74613 0.19602 1000. 0.75169 0.19673 10000. 0.74812 0.19993 Choose p(x)= A exp(-x) and sample [0,1] again n Fn n 100. 0.75105 0.05076 1000. 0.74882 0.05411 10000. 0.74753 0.05420 The variance of the integrand is reduced by about a factor of 4 which means that fewer samplings are needed to obtain the same accuracy. c Program Importance Sampling n=10000 h=1. a=0. b=1. sum=0. psum=0. sum2=0. psum2=0. m=2 do 4 i=1,n ww=r250(idum) x=a+b*ww y=-log(1.-ww+ww*exp(-1.)) g=exp(-y*y) p=(1.-ww+ww*exp(-1.))/(1.-exp(-1.)) f=exp(-x*x) sum=sum+f psum=psum+g/p sum2=sum2+f*f psum2=psum2+(g*g)/(p*p) sig2=sum2/i -(sum/i)*(sum/i) sig=sqrt(sig2) psig2=psum2/i -(psum/i)*(psum/i) psig=sqrt(psig2) if((i-(i/10**m)*10**m).eq.0) then write(6,10) 1.*i,sum/i,sig,psum/i,psig m=m+1 else continue end if 10 format(1x,f10.0,3x,f10.5,3x,f10.5,3x,f10.5,3x,f10.5) 4 continue stop end The above ideas can be used to simulate many different types of physical problems • • • • • Random walks and polymers Percolation Fractal growth Complexity and neural networks Phase Transitions and Critical Phenomena Monte Carlo Methods • Random numbers generated by the computer are used to simulate naturally random processes • many previously intractable thermodynamic and quantum mechanics problems have been solved using Monte Carlo techniques • how do we know is the random numbers are really random? Random Sequences • A sequence of numbers r1,r2,… is random if there are no correlations among the numbers in the sequence • however most random number generators yield a sequence in which each number is used to find the succeeding one according to a well defined algorithm • the most widely used random number generator is based on the linear congruential method Given a seed x0, each number in the sequence is determined by the one-dimensional map xn (axn1 c) mod m • where a,c and m are integers • the notation y= z mod m means that m is subtracted from z until 0 y <m • the process is characterized by the multiplier a, the increment c and the modulus m • since m is largest integer generated by this method, the maximum possible period is m Example • • • • • • • • a=3 c=4 m=32 and x0=1 produces x1=(3 x 1 + 4) mod32 = 7 x2=(3 x 7 + 4) mod32 = 25 x3=(3 x 25 +4) mod32= 79mod32=15 and so on …. 1,7,25,15,17,23,9,31,1,7,25,…. period is 8! Rather than the maximum of 32 Random Sequences • If we choose a, c and m carefully then all numbers in the range from 0 to m-1 will appear in the sequence • to have the numbers in the range 0 r <1, the generator returns xm/m which is always < 1 • there is no necessary and sufficient test for the randomness of a finite sequence of numbers • we need to consider various tests • an obvious requirement for a random number generator is that its period be much greater than the number of random numbers needed in a specific problem Sequences • A way of visualizing the period is to consider a random walker and plot the displacement as a function of the number of steps N • when the period of the random number generator is reached the plot will begin to repeat itself • consider a=899, c=0, m=32768 with x0=12 Sequence Correlations • We can check for correlations by plotting xi+k as a function of xi • if there are any obvious patterns in the plot then there are correlations correlations Uniformity N 1 k k x xi N i 1 p( x) 1 for 0 x 1 1 N N xi i 1 1 k k dx x p ( x ) O (1/ N ) 0 1 k 1 test Correlations • One way to reduce sequential correlation and to lengthen the period is to mix or shuffle two different random number generators • statistical tests should be performed on random number generators for serious calculations Exploration xi+1= 4xi(1 - xi) • It has been claimed that the logistic map in the chaotic region is a good random number generator • test this for yourself yi 1 1 cos (1 2 xi ) • will make the sequence uniform