Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
PSES8 by Devore 5 : Probability Theory and Statistics Contents 5 .1 Probability and Statistics . . . . . . . . . . . 5 .2 Guessing the population distritution . . . . . 5 .2.1 Probability Plot (q-q plot) . . . . . . . 5 .3 Example: q-q nomal plot . . . . . . . . . . . . 5 .4 After you Guessed the population distritution 5 .4.1 Method of Moments . . . . . . . . . . 5 .4.2 Maximum Likelihood Estimation . . . 5 .5 Property of Estimator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 5 6 7 8 9 10 12 5 .1 Probability and Statistics Random variables X1 , X2 , . . . , Xn are said to be a random sample of size n from distribution F0 if 1. The Xi ’s are independent 2. Each Xi has distribution F0 . 1st run of the Experiment − > realization of X1 2nd run of the Experiment − > realization of X2 3rd run of the Experiment − > realization of X3 .. . X1 X2 , . . . , X 3 2 Example: ---Rcode--D1 D2 D3 D4 = = = = read.csv("http://gozips.uakron.edu/~nmimoto/pages/datasets/Digits%20of%20Pi.txt") read.csv("http://gozips.uakron.edu/~nmimoto/pages/datasets/Lottery.txt") read.csv("http://gozips.uakron.edu/~nmimoto/pages/datasets/Speed%20of%20Light.txt") read.csv("http://gozips.uakron.edu/~nmimoto/pages/datasets/PitCorrosion.txt") Pi=D1$Pi Lot=D2$Lot_3 Sp=D3$Speed_of_Light Dep=D4$depth hist(Pi, freq=F) 3 hist(Lot, freq=F) hist(Sp, freq=F) hist(Dep, freq=F) rnorm(1) rnorm(5) rnorm(50) x <- rnorm(100); hist(x,freq=F); lines(t,dnorm(t), col=’red’); qqnorm(x) x <- rt(100, 5); hist(x,freq=F); lines(t,dnorm(t), col=’red’); qqnorm(x) 4 5 .2 Guessing the population distritution 1. Histogram(data) vs pdf(Theoretical) 2. EDF(data) vs CDF(Theoretical) 3. Probability Plot Sample Percentiles vs Theoretical Percentiles 5 5 .2.1 Probability Plot (q-q plot) (From Chapter 4) If you guessed the population distribution is A, then you can check your guess by plotting q-q plot. q-q plot is a plot of (i − .5) ith ordered obs. vs 100 × th (theoretical) percentile from your guees of A . n If the data are indeed sample from A, then the q-q plot should look like a line. 6 5 .3 Example: q-q nomal plot ---Rcode--t <- seq(-3,3,.1); layout(matrix(c(1,2), 1,2)) x <- rnorm(100); hist(x,freq=F); lines(t,dnorm(t), col=’red’); qqnorm(x) x <- rt(100, 5); hist(x,freq=F); lines(t,dnorm(t), col=’red’); qqnorm(x) 7 5 .4 After you Guessed the population distritution Say now you are confortable assuming that your data X1 , . . . , Xn is a random sample from distribuiton A with parameter α. How can we come up with estimator for parameter α? There are two major methods for coming up with an estimator in general. 1. Method of Moments 2. Maximum Likelihood Estimation 8 5 .4.1 Method of Moments k th population moment: E(X k ) n X k th sample moment: ( Xi )/n i=1 To obtain estimator for θ by method of moments, 1. Write your θ using population moments. 2. replace your population moments with sample moments. Ex: Normal, Exponential, Binomial, Poisson, HG, 9 5 .4.2 Maximum Likelihood Estimation Let X1 , . . . , Xn be random sample from a distribution with pdf f (x; θ). Then likelihood function is defined as L(θ) = f (X1 ; θ)f (X2 ; θ) · · · f (Xn ; θ) To obtain MLE for θ, 1. Write down L(θ) 2. Take ln of L(θ) 3. Find θ that maximize ln(L(θ)) by taking d dθ of ln(L(θ)) and setting it to 0. Property of MLE: 1. If θ̂ if MLE for θ, then h(θ̂) is MLE for h(θ) for any function h(·). 2. MLE is always asymptotically unbiased. i.e. If n is large, MLE is approximately unbiased. 10 Ex: exp, poisson, normal, unif 11 5 .5 Property of Estimator 1. Unbiasedness 2. Variance 3. Distribution of Estimator 12 Unbiased Estimator Let θ̂ be a estimator for a parameter θ. θ̂ is an unbiased estimator if E(θ̂) = θ. • Variance of the estimator • Minimum Variance Unbiased Estimator 13 Examples : If X1 , . . . , Xn are R.S. from a population with mean µ and standard deviation σ. We want to use X to estimate µ, then X is an unbiased estimator for µ because E(X) = µ. Sample variance S 2 also turns out to be an unbiased estimator for σ 2 . If X are random sample of size n from the Binomial distribution with parameter p and n, then p̂ = X n is an unbiased estimator for p 14