Download 5. Probability Theory and Statistics

PSES8 by Devore
5 : Probability Theory and Statistics
Contents
5 .1 Probability and Statistics . . . . . . . . . . .
5 .2 Guessing the population distritution . . . . .
5 .2.1 Probability Plot (q-q plot) . . . . . . .
5 .3 Example: q-q nomal plot . . . . . . . . . . . .
5 .4 After you Guessed the population distritution
5 .4.1 Method of Moments . . . . . . . . . .
5 .4.2 Maximum Likelihood Estimation . . .
5 .5 Property of Estimator . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2
5
6
7
8
9
10
12
5 .1
Probability and Statistics
Random variables X1 , X2 , . . . , Xn are said to be a random sample of size n from distribution F0 if
1. The Xi ’s are independent
2. Each Xi has distribution F0 .
1st run of the Experiment − > realization of X1
2nd run of the Experiment − > realization of X2
3rd run of the Experiment − > realization of X3
..
.
X1 X2 , . . . , X 3
2
Example:
---Rcode--D1
D2
D3
D4
=
=
=
=
read.csv("http://gozips.uakron.edu/~nmimoto/pages/datasets/Digits%20of%20Pi.txt")
read.csv("http://gozips.uakron.edu/~nmimoto/pages/datasets/Lottery.txt")
read.csv("http://gozips.uakron.edu/~nmimoto/pages/datasets/Speed%20of%20Light.txt")
read.csv("http://gozips.uakron.edu/~nmimoto/pages/datasets/PitCorrosion.txt")
Pi=D1$Pi
Lot=D2$Lot_3
Sp=D3$Speed_of_Light
Dep=D4$depth
hist(Pi, freq=F)
3
hist(Lot, freq=F)
hist(Sp, freq=F)
hist(Dep, freq=F)
rnorm(1)
rnorm(5)
rnorm(50)
x <- rnorm(100); hist(x,freq=F); lines(t,dnorm(t), col=’red’); qqnorm(x)
x <- rt(100, 5); hist(x,freq=F); lines(t,dnorm(t), col=’red’); qqnorm(x)
4
5 .2
Guessing the population distritution
1. Histogram(data) vs pdf(Theoretical)
2. EDF(data) vs CDF(Theoretical)
3. Probability Plot
Sample Percentiles vs
Theoretical Percentiles
5
5 .2.1
Probability Plot (q-q plot)
(From Chapter 4)
If you guessed the population distribution is A, then you can check your guess by plotting q-q plot.
q-q plot is a plot of
(i − .5)
ith ordered obs. vs 100 ×
th (theoretical) percentile from your guees of A .
n
If the data are indeed sample from A, then the q-q plot should look like a line.
6
5 .3
Example: q-q nomal plot
---Rcode--t <- seq(-3,3,.1);
layout(matrix(c(1,2), 1,2))
x <- rnorm(100); hist(x,freq=F); lines(t,dnorm(t), col=’red’); qqnorm(x)
x <- rt(100, 5); hist(x,freq=F); lines(t,dnorm(t), col=’red’); qqnorm(x)
7
5 .4
After you Guessed the population distritution
Say now you are confortable assuming that your data X1 , . . . , Xn is a random sample from distribuiton A
with parameter α. How can we come up with estimator for parameter α?
There are two major methods for coming up with an estimator in general.
1. Method of Moments
2. Maximum Likelihood Estimation
8
5 .4.1
Method of Moments
k th population moment: E(X k )
n
X
k th sample moment: (
Xi )/n
i=1
To obtain estimator for θ by method of moments,
1. Write your θ using population moments.
2. replace your population moments with sample moments.
Ex: Normal, Exponential, Binomial, Poisson, HG,
9
5 .4.2
Maximum Likelihood Estimation
Let X1 , . . . , Xn be random sample from a distribution with pdf f (x; θ). Then likelihood function is defined
as
L(θ) = f (X1 ; θ)f (X2 ; θ) · · · f (Xn ; θ)
To obtain MLE for θ,
1. Write down L(θ)
2. Take ln of L(θ)
3. Find θ that maximize ln(L(θ)) by taking
d
dθ
of ln(L(θ)) and setting it to 0.
Property of MLE:
1. If θ̂ if MLE for θ, then h(θ̂) is MLE for h(θ) for any function h(·).
2. MLE is always asymptotically unbiased. i.e. If n is large, MLE is approximately unbiased.
10
Ex:
exp, poisson, normal, unif
11
5 .5
Property of Estimator
1. Unbiasedness
2. Variance
3. Distribution of Estimator
12
Unbiased Estimator
Let θ̂ be a estimator for a parameter θ. θ̂ is an unbiased estimator if
E(θ̂) = θ.
• Variance of the estimator
• Minimum Variance Unbiased Estimator
13
Examples :
If X1 , . . . , Xn are R.S. from a population with mean µ and standard deviation σ. We want to use X to
estimate µ, then
X is an unbiased estimator for µ because E(X) = µ.
Sample variance S 2 also turns out to be an unbiased estimator for σ 2 .
If X are random sample of size n from the Binomial distribution with parameter p and n, then
p̂ =
X
n
is an unbiased estimator for p
14