Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Survey

Document related concepts

Transcript

Statistical Model • A statistical model for some data is a set of distributions, f : one of which corresponds to the true unknown distribution that produced the data. • The statistical model corresponds to the information a statistician brings to the application about what the true distribution is or at least what he or she is willing to assume about it. • The variable θ is called the parameter of the model, and the set Ω is called the parameter space. • From the definition of a statistical model, we see that there is a unique value , such that fθ is the true distribution that generated the data. We refer to this value as the true parameter value. week 2 1 Examples • Suppose there are two manufacturing plants for machines. It is known that the life lengths of machines built by the first plant have an Exponential(1) distribution, while machines manufactured by the second plant have life lengths distributed Exponential(1.5). You have purchased five of these machines and you know that all five came from the same plant but do not know which plant. Further, you observe the life lengths of these machines, obtaining a sample (x1, …, x5) and want to make inference about the true distribution of the life lengths of these machines. • Suppose we have observations of heights in cm of individuals in a population and we feel that it is reasonable to assume that the distribution of height is the population is normal with some unknown mean and variance. The statistical model in this case is f , : , 2 where Ω = R×R+, where R+ = (0, ∞). 2 week 2 2 Point Estimate • Most statistical procedures involve estimation of the unknown value of the parameter of the statistical model. • A point estimate, ˆ ˆx1 ,..., xn , is an estimate of the parameter θ. It is a statistic based on the sample and therefore it is a random variable with a distribution function. • The standard deviation of the sampling distribution of an estimator is usually called the standard error of the estimator. • For a given statistical model with unknown parameter θ there could be more then one point estimate. • The parameter θ of a statistical model can have more then just one element. week 2 3 Properties of Point Estimators • Let ˆ be a point estimator for a parameter θ. Then ˆ is an unbiased estimator if E ˆ . • The bias of a point estimator is given by B ˆ E ˆ • The variance of a point estimator is 2 var ˆ E ˆ 2 E ˆ . Ideally we would like our estimator to have minimum variance. week 2 4 Mean Square Error of Point Estimators • The mean square error (MSE) of a point estimator is 2 MSE ˆ E ˆ . • Claim: MSE ˆ Var ˆ B ˆ . 2 week 2 5 Common Point Estimators • A natural estimate for the population mean μ is the sample mean (in any distribution). The sample mean is an unbiased estimator of the population mean. • A common estimator for the population variance is the sample variance s2. week 2 6 Claim • Let X1, X2,…, Xn be random sample of size n from a normal population. The sample variance s2 is an unbiased estimator of the population variance σ2. • Proof… week 2 7 Example • Suppose X1, X2,…, Xn is a random sample from U(0, θ) distribution. Let ˆ X n . Find the density of ˆ and its mean. Is ˆ unbiased? week 2 8 Asymptotically Unbiased Estimators Bˆ 0 • An estimator is asymptotically unbiased if lim n • Example: week 2 9 Relative Efficiency • Given two estimators ˆ1 and ˆ2 of a parameter θ with variances Var ˆ1 and Var ˆ2 respectively. The efficiency of ˆ2 relative to ˆ1 is the ratio Var ˆ`1 Var ˆ2 • Interpretation… week 2 10 Example • In the uniform example above let ˆ1 X n and ˆ2 n 1 X n . n Which point estimate is more efficient? week 2 11 Consistency p • An estimator ˆn is a consistent estimator of θ, if ˆn , i.e., if ˆn converge in probability to θ. week 2 12 Theorem • An unbiased estimator ˆn for θ, is a consistent estimator of θ if lim Var ˆn 0. n • Proof: week 2 13 Example • Suppose X1, X2,…, Xn are i.i.d Poisson(λ). Let ̂ X then… week 2 14 Important comment • Consistency is an asymptotic property so we can have a consistent estimator that is biased as long as it is asymptotically unbiased. • Example: Uniform example above. week 2 15