Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Psychometrics wikipedia , lookup
History of statistics wikipedia , lookup
Degrees of freedom (statistics) wikipedia , lookup
Foundations of statistics wikipedia , lookup
Confidence interval wikipedia , lookup
Bootstrapping (statistics) wikipedia , lookup
Taylor's law wikipedia , lookup
Regression toward the mean wikipedia , lookup
Misuse of statistics wikipedia , lookup
The t distribution • Suppose that a SRS of size n is drawn from a N(μ, σ) population. Then the one sample t statistic x t s n has a t distribution with n -1 degrees of freedom. • The t distribution has mean 0 and it is a symmetric distribution. • The is a different t distribution for each sample size. • A particular t distribution is specified by the degrees of freedom that comes from the sample standard deviation. week11 1 Tests for the population mean when is unknown • Suppose that a SRS of size n is drawn from a population having unknown mean μ and unknown stdev. . To test the hypothesis H0: μ = μ0 , we first estimate by s – the sample stdev., then compute the one-sample t statistic given by x 0 t s n • In terms of a random variable T having the t (n - 1) distribution, the P-value for the test of H0 against Ha : μ > μ 0 is Ha : μ < μ 0 is P( T ≥ t ) P( T ≤ t ) Ha : μ ≠ μ 0 is 2·P( T ≥ |t|) week11 2 Example • In a metropolitan area, the concentration of cadmium (Cd) in leaf lettuce was measured in 6 representative gardens where sewage sludge was used as fertilizer. The following measurements (in mg/kg of dry weight) were obtained. Cd 21 38 12 15 14 8 Is there strong evidence that the mean concentration of Cd is higher than 12. Descriptive Statistics Variable Cd N 6 Mean 18.00 Median 14.50 TrMean 18.00 StDev 10.68 • The hypothesis to be tested are: H0: μ = 12 vs week11 SE Mean 4.36 Ha: μ > 12. 3 • The test statistics is: t x 1812 1.38 s / n 10.68/ 6 The degrees of freedom are df = 6 – 1 = 5 Since t = 1.38 < 2.015, we cannot reject H0 at the 5% level and so there are no strong evidence. The P-value is 0.1 < P(T(5) ≥ 1.38) < 0.15 and so is greater then 0.05 indicating a non significant result. week11 4 CIs for the population mean when unknown • Suppose that a SRS of size n is drawn from a population having unknown mean μ. A C-level CI for μ when is unknown is an interval of the form s s * * , x t x t n n where t* is the value for the t (n -1) density curve with area C between –t* and t*. • Example: Give a 95% CI for the mean Cd concentration. week11 5 • MINITAB commands: Stat > Basic Statistics > 1-Sample t • MINITAB outputs for the above problem: T-Test of the Mean Test of mu = 12.00 vs mu > 12.00 Variable N Mean StDev Cd 6 18.00 10.68 SE Mean 4.36 T 1.38 P 0.11 T Confidence Intervals Variable Cd N 6 Mean 18.00 StDev 10.68 week11 SE Mean 4.36 95.0 % CI (6.79, 29.21) 6 Question 3 Final exam Dec 2000 • In order to test H0: μ = 60 vs Ha: μ ≠ 60 a random sample of 9 observations (normally distributed) is obtained, yielding x 55 and s = 5. What is the p-value of the test for this sample? a) b) c) d) e) greater than 0.10. between 0.05 and 0.10. between 0.025 and 0.05. between 0.01 and 0.025. less than 0.01. week11 7 Question A manufacturing company claims that its new floodlight will last 1000 hours. After collecting a simple random sample of size ten, you determine that a 95% confidence interval for the true mean number of hours that the floodlights will last, , is (970, 995). Which of the following are true? (Assume all tests are two-sided.) I) At any < .05, we can reject the null hypothesis that the true mean is 1000. II) If a 99% confidence interval for the mean were determined here, the numerical value 972 would certainly lie in this interval. III) If we wished to test the null hypothesis H0: = 988, we could say that the p-value must be < 0.05. week11 8 Questions 1. Alpha (level of sig. α) is a) the probability of rejecting H0 when H0 is true. b) the probability of supporting H0 when H0 is false. c) supporting H0 when H0 is true. d) rejecting H0 when H0 is false. 2. Confidence intervals can be used to do hypothesis tests for a) left tail tests. b) right tail tests c) two tailed test 3. The Type II error is supporting a null hypothesis that is false. T/F week11 9 Robustness of the t procedures • Robust procedures A statistical inference procedure is called robust if the probability calculations required are insensitive to violations of the assumptions made. • t-procedures are quite robust against nonnormality of the population except in the case of outliers or strong skewness. week11 10 Simulation study • Let’s generate 100 samples of size 10 from a moderately skewed distribution (Chi-square distribution with 5 df ) and calculate the 95% t-intervals to see how many of them contain the true mean μ = 5. • First let’s have a look at the histogram of the 1000 values generated from this distribution. 400 Frequency 300 200 100 0 0 10 20 30 C1 Variable C1 N 1000 Mean 4.9758 Median 4.2788 week11 TrMean 4.7329 StDev 3.1618 11 T Confidence Intervals Variable C1 N 10 Mean 5.21 StDev 3.89 SE Mean 1.23 95.0 % CI 2.43, 7.99) ( 10 10 10 10 10 10 4.449 5.33 3.267 4.981 3.725 4.487 1.593 4.23 2.312 2.988 1.520 2.332 0.504 1.34 0.731 0.945 0.481 0.738 ( ( ( ( ( ( 3.309, 2.31, 1.612, 2.844, 2.638, 2.819, 5.589) 8.36) 4.921)* 7.118) 4.812)* 6.155) 10 10 10 10 10 10 4.650 2.973 4.685 5.594 3.468 5.59 1.854 2.163 2.254 2.984 2.078 3.84 0.586 0.684 0.713 0.944 0.657 1.22 ( ( ( ( ( ( 3.324, 1.425, 3.072, 3.459, 1.982, 2.84, 5.977) 4.520)* 6.297) 7.728) 4.955)* 8.34) 10 10 10 5.689 3.724 4.387 3.113 1.741 2.157 0.984 0.551 0.682 ( ( ( 3.462, 2.479, 2.843, 7.916) 4.970)* 5.930) 10 10 10 7.01 3.281 4.78 3.44 2.265 3.20 1.09 0.716 1.01 ( ( ( 4.55, 1.661, 2.49, 9.47) 4.902)* 7.06) 10 10 6.52 3.614 4.24 2.198 1.34 0.695 ( ( 3.49, 2.042, 9.56) 5.186) . . . C4 C5 C6 C7 C8 C9 . . . C14 C15 C16 C26 C27 C28 . . . C62 C63 C64 . . . C87 C88 C89 . . . C99 C100 week11 The number of intervals not capturing the true mean (μ = 5) is 6/100. 12 Example • 100 samples of size 15 were drawn from a very skewed distribution (Chi-square distribution with d. f. 1) Variable C1 N Mean Median TrMean StDev 1500 0.9947 0.4766 0.8059 1.3647 Frequency 1500 1000 500 0 0 5 10 15 C1 • The 95% CIs (t-intervals) for these 100 samples are given below. week11 13 T Confidence Intervals Variable N Mean C1 15 0.773 C2 15 1.093 C3 15 0.553 C4 15 0.387 C5 15 1.239 ... C23 15 0.491 C24 15 0.582 C25 15 0.550 C26 15 0.634 C27 15 0.508 ... C51 15 1.122 C52 15 0.519 C53 15 1.666 ... C59 15 1.208 C60 15 0.644 C61 15 1.088 StDev 0.939 1.491 0.735 0.732 2.146 SE Mean 0.242 0.385 0.190 0.189 0.554 ( ( ( ( ( 95.0 % CI 0.253, 1.293) 0.268, 1.919) 0.146, 0.960)* -0.019, 0.792)* 0.051, 2.427) 0.619 1.088 0.660 0.769 0.528 0.160 0.281 0.170 0.199 0.136 ( ( ( ( ( 0.148, -0.020, 0.184, 0.208, 0.216, 0.834)* 1.184) 0.915)* 1.060) 0.800)* 1.292 0.664 2.028 0.334 0.171 0.524 ( ( ( 0.406, 0.151, 0.543, 1.837) 0.887)* 2.789) 2.297 0.525 1.122 0.593 0.136 0.290 ( ( ( -0.065, 0.353, 0.466, 2.480) 0.935)* 1.709) week11 14 T Confidence Intervals (continuation) ... C79 C80 C81 C82 C83 C84 ... C99 C100 15 15 15 15 15 15 0.895 0.391 1.038 0.952 0.2763 1.237 0.931 0.767 0.992 1.407 0.2999 1.999 0.240 0.198 0.256 0.363 0.0774 0.516 ( ( ( ( ( ( 0.379, -0.034, 0.488, 0.173, 0.1102, 0.130, 1.411) 0.816)* 1.587) 1.732) 0.4424)* 2.345) 15 15 0.921 0.813 0.865 1.437 0.223 0.371 ( ( 0.442, 0.018, 1.400) 1.609) The number of intervals not capturing the true mean (μ = 1) is 9/100. week11 15 Match Pairs t-test • In a matched pairs study, subjects are matched in pairs and the outcomes are compared within each matched pair. The experimenter can toss a coin to assign two treatment to the two subjects in each pair. Matched pairs are also common when randomization is not possible. One situation calling for match pairs is when observations are taken on the same subjects, under different conditions. • A match pairs analysis is needed when there are two measurements or observations on each individual and we want to examine the difference. • For each individual (pair), we find the difference d between the measurements from that pair. Then we treat the di as one sample and use the one sample t – statistic to test for no difference between the treatments effect. • Example: similar to exercise 7.41 on page 446 in IPS. week11 16 Data Display Row 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Student 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Pretest 30 28 31 26 20 30 34 15 28 20 30 29 31 29 34 20 26 25 31 29 Posttest 29 30 32 30 16 25 31 18 33 25 32 28 34 32 32 27 28 29 32 32 improvement -1 2 1 4 -4 -5 -3 3 5 5 2 -1 3 3 -2 7 2 4 1 3 week11 17 • One sample t-test for the improvement T-Test of the Mean Test of mu = 0.000 vs mu > 0.000 Variable N Mean StDev SE Mean improvem 20 1.450 3.203 0.716 T 2.02 P 0.029 • MINITAB commands for the paired t-test Stat > Basic Statistics > Paired t Paired T-Test and Confidence Interval Paired T for Posttest – Pretest N Mean StDev SE Mean Posttest 20 28.75 4.74 1.06 Pretest 20 27.30 5.04 1.13 Difference 20 1.450 3.203 0.716 95% CI for mean difference: (-0.049, 2.949) T-Test of mean difference=0 (vs > 0): T-Value = 2.02 P-Value = 0.029 week11 18 6 Frequency 5 4 3 2 1 0 -4 -2 0 2 4 6 8 improvement Character Stem-and-Leaf Display Stem-and-leaf of improvement Leaf Unit = 1.0 2 -0 54 4 -0 32 6 -0 11 8 0 11 (7) 0 2223333 5 0 4455 1 0 7 week11 N = 20 19 Two-sample problems • The goal of inference is to compare the response in two groups. • Each group is considered to be a sample form a distinct population. • The responses in each group are independent of those in the other group. • A two-sample problem can arise form a randomized comparative experiment or comparing random samples separately selected from two populations. • Example: A medical researcher is interested in the effect of added calcium in our diet on blood pressure. She conducted a randomized comparative experiment in which one group of subjects receive a calcium supplement and a control group gets a placebo. week11 20 Comparing two means (with two independent samples) • Here we will look at the problem of comparing two population means when the population variances are known or the sample sizes are large. Suppose that a SRS of size n1 is drawn from an N( μ1, σ1) population and that an independent SRS of size n2 is drown from an N( μ2, σ2) population. Then the two-sample z statistics for testing the null hypothesis H0: μ1 = μ2 is given by x1 x2 1 2 z 2 1 n1 22 n2 and has the standard normal N(0,1) sampling distribution. • Using the standard normal tables, the P-value for the test of H0 against Ha : μ1 > μ2 is P( Z ≥ z ) Ha : μ1 < μ2 is P( Z ≤ z ) Ha : μ1 ≠ μ 2 is 2·P(Z ≥ |z|) week11 21 Example • A regional IRS auditor runs a test on a sample of returns filed by March 15 to determine whether the average return this year is larger than last year. The sample data are shown here for a random sample of returns from each year. Last Year This Year Mean 380 410 Sample size 100 120 • Assume that the std. deviation of returns is known to be about 100 for both years. Test whether the average return is larger this year than last year. week11 22 Solution • The hypothesis to be tested are: H0: μ1 = μ2 vs • The test statistics is: Ha: μ1 < μ2. z 380 410 0 2.22 1.645 1002 1002 100 120 • The P-value = P(Z < -2.22) = 0.0139 < 0.05, therefore we can reject H0 and conclude that at the 5% significant level, the average return is larger this year than last year. • A 95% CI for the difference is given by: x x 1 2 Z* 2 2 1 2 n n , 1 2 301.96 1002 1002 30 26.5 100 120 (3.5, 56.5) week11 23 Comparing two population means (unknown std. deviations) • Suppose that a SRS of size n1 is drawn from a normal population with unknown mean 1 and that an independent SRS of size n2 is drawn from another normal population with unknown mean 2. To test the null hypothesis H0: 1 = 2, we compute the two sample t-statistic t x1 x2 1 2 s 2 1 n1 s n2 2 2 • This statistic has a t-distribution with df approximately equal to smaller of n1 – 1 and n2 - 1. We can use this distribution to compute the P-value. week11 24 Example • The weight gains for n1 = n2 = 8 rats tested on diets 1 and 2 are summarized here. Test whether diet 2 has greater mean weight gain. Use the 5% significant level. n Std dev. mean Diet 1 8 .033 3.1 Diet 2 8 0.070 3.2 • The hypotheses to be tested are: H0: μ1 = μ2 vs Ha: μ1 < μ2 . • The test statistic is 3.13.2 0 t 3.65 0.0332 0.0702 8 8 week11 25 • The P-value is P(T(7) ≤- 3.65) = P(T(7) ≥ 3.65) , from table D we have 0.005 < P-value < 0.01 and so we reject H0 and conclude that the mean weight gain from diet 2 is significantly greater than that from diet 1 (at the 5% and 1% significant level). • A C% CI for the difference between the two means is given by, x1 x2 t s12 s22 n1 n2 • For this example the 95% CI is 0.0332 0.0702 3.2 3.1 2.365 8 8 = (0.0353, 0.165) week11 26