Download Topic One: IRT, and the Rasch Model, in a nutshell

Topic 7
Making inferences about Means
General process of making inference about a statistic
(1) Establish the sampling distribution of the statistic to assess the variability of the
statistic.
For example, if we are interested in the mean reading score of students in Victoria, we
take a sample and compute the sample mean. Because this sample mean is not the
population mean, there is likely variation in the value of the sample mean if different
samples are drawn. We need to find out how large the variation is. If the variation is
large, then our estimate is probably not very accurate to represent the population
mean. If the variation is small, then our estimate is probably quite close to the
population mean.
(2) We can repeatedly sample to establish the sampling distribution of the statistic of
interest. But this is impractical as it will be too costly.
Making inferences about Mean
We can use the central limit theorem to establish the sampling distribution of the
sample mean, without doing repeated sampling. Central limit theorem says that the
mean of independently drawn observations will be approximately normally
distributed, even if the distribution from which the sample is drawn is not normal.
Further, it can be shown that mean values computed from samples of size n have a
normal distribution with mean  and standard deviation of

n
(known as the
standard error), where  is the mean of the distribution we draw our samples from,
and  is the standard deviation of the distribution we draw our samples from.
That is, if X denotes the sample mean, then
X 

has a standard normal distribution
n
with mean zero and standard deviation 1 (z-score). For a standard normal distribution,
95% of the observations lie within 1.96. With a little re-arrangement of the equation,
it can be shown that 95% of the time, or, we are 95% confident that,
X  1.96  
n
 

X  1.96  
n
(There is a 95% chance that the population mean lies within the range shown above.)
t-distribution
Of course, in practice, we do not know the population standard deviation,  . So we
use the sample standard deviation, s , instead.
X 
(where s is an estimate of  ) is no longer normally
s
n
distributed. Instead, it has a t-distribution, with n  1 degrees of freedom. This means
However, the statistic
that, we cannot use 1.96 for 95% confidence interval. We must find the 95% range
from a t-distribution.
The t-distribution approaches the normal distribution as the sample size increases.
To compute the probability values from a t-distribution, use the EXCEL function
TDIST. For example, in a cell in EXCEL, type
=TDIST(1.96,50,2)
The value 0.056 is returned. This means that 5.6% of the observations from a tdistribution (with 50 degrees of freedom) lie outside the values 1.96. In comparison,
for the normal distribution, 5% of the observations lie outside the values 1.96.
Conversely, the function
=TINV(0.05, 50)
returns the value 2.01. This means that 95% (=1-0.05) of the observations from a tdistribution with 50 degrees of freedom lie within 2.01 (instead of 1.96 as in the
normal distribution).
(See page 268, Coladarci, for a comparison between t-distribution and normal
distribution)
Example 1
In a sample of 50 randomly selected teachers, the average age is 36 with a standard
deviation of 6. What is the 95% confidence interval for the population mean age of
teachers?
(Hint: What is the standard error of the mean? The 95% confidence interval for the
population mean is
Sample mean  t(0.05, 49)×standard error, where t(0.05,
probability, with 49 degrees of freedom.)
49)
is the t-value for 95%
Example 2
The cost of 15 randomly selected university textbooks at one university are (in
dollars):
120
89
65
135
145
110
160
50
115
128
99
95
105
120
75
Estimate the mean cost of textbooks based on this sample at this university. Give 95%
confidence interval.
(1) Paste the prices in EXCEL. Compute average and standard deviation of the prices.
Compute standard error.
What is the 95% confidence interval?
(2) Paste the numbers in SPSS. Select Analyse -> Descriptive statistics -> Explore.
Compare the answers from SPSS with the results from EXCEL.
Example 2B
According to university administration, the average cost of textbooks is $90. Using
the data from Example 2, at 95% confidence level, will you reject the hypothesis that
the average cost is $90? What if you test this hypothesis at 90% confidence interval?
What if you test this at 99% confidence interval? (Use SPSS explore to work out the
confidence interval at different levels.)
Two-sample t-test
Example 3
At another university, a similar survey is carried out. The cost of 15 randomly
selected university textbooks at this university are (in dollars):
100
120
145
120
115
118
66
89
93
89
109
120
60
70
55
We want to compare whether the cost of textbooks at university 2 is, on average,
cheaper than the cost of textbooks at university 1.
Paste the prices in EXCEL. Compute average and standard deviation of the prices.
Based on the average and standard deviation, do you think the cost of textbooks at
university 2 is generally cheaper?
Did you make this judgement by comparing the average cost at university 1 and
university 2? Did you take into account of the confidence intervals based on the
estimates?
Since there are only 15 textbooks in the sample, we expect variability in average cost
from sample to sample. A formal statistical method to compare the means from two
independently drawn samples is the two-sample t-test. The test takes into account the
size of the difference between the two group averages, as well as the likely variability
in the averages due to sampling.
SPSS exercise
Import the data for textbooks into SPSS
(animated demo: TwoSample-t-test_demo.swf)
(animated demo: t-test output_demo.swf)
That is, there is about 1 in 3 chance (0.37) for us to observe a difference as large as
$9.50 (= $107.4 - $97.9) when there is no difference in the ‘true’ (population) means.
So we can say the following:
At 63% confidence level (or lower), we will reject the hypothesis that the population
means are the same. (Generally, 63% confidence level is not regarded as very
confident!)
At any confidence level higher than 63% (e.g., 90%, 95%, 99%), we cannot reject this
hypothesis.
You need to choose a confidence level before making your conclusions.
Discussion point:
First, note that different conclusions can be reached depending on the confidence
level.
Second, how does one decide on the confidence level?
Third, once a conclusion is made, beware that the conclusions is NOT the ‘truth’ (of
whether there IS a difference in the population means.). In fact, the population means
are extremely unlikely to be identical to 100 decimal places, so the ‘truth’ is almost
certainly that the mean cost at university 1 is NOT the same as the mean cost at
university 2. Given large enough sample sizes, we will almost certainly find statistical
significance, at high confidence levels, even if the population mean difference is
minute.
Effect size
In addition to statistical significant, we may want to assess whether the observed
difference matters. We might decide, for example, if the population mean difference
is less than $2, then we will conclude that there is no “important” difference, and,
regardless of statistical significance or not, we will conclude there is no difference.
A commonly used measure of effect size is to look at the size of the difference in
means in relation to the standard deviation:
X1  X 2
standard deviation (pooled)
(see page 295-298, Coladarci, et al)
Example 4
Use the data set StudentLiteracyScoresCutDown.sav to assess whether there is a
difference between the average reading scores for males and females.
Use the whole data first. This is already a sample from the population. What
conclusions have you drawn? How would you write your conclusion in a report?
Sub-sample from this data set. At about what sample size, are you likely to change
your conclusion?
If you are reporting this to the minister of education, or reporting it as a newspaper
article, how would you write it?
Paired sample t-test
In the examples about the cost of university textbooks, our survey method was not
very efficient, because it could be by chance that different textbooks are selected at
the two universities. The expected variation in the average cost may fluctuate more
widely when different textbooks are selected.
A better design is to choose the same textbooks at both universities and compare the
prices, pairwise.
The following shows prices of 15 textbooks as sold at two universities.
text book 1
text book 2
text book 3
text book 4
text book 5
text book 6
text book 7
text book 8
text book 9
text book 10
text book 11
text book 12
text book 13
text book 14
text book 15
price at uni 1
120
89
60
125
150
115
160
65
115
128
100
98
105
120
80
price at uni 2
120
85
60
120
145
118
155
60
108
125
100
100
105
120
78
To compare the difference in average prices at the two universities,
(1) Analyse as two independent sample t-test
(You have to organise the data in SPSS with one column stacked under the other, and
add a group variable with values of 1 or 2 to indicate which university it is.)
(2) Analyse as paired-samples t-test
(You have to organise the data in SPSS with two matched columns. See animated
demo: Paired Sample t-test_demo.swf).
What are the differences in results between the above two analyses?
Why are they different?
Exercise
Use the data set, TIMSS2003AUS_Cutdown.sav,
(1) Investigate whether there is a difference between students’ mathematics score and
science score (variables 42 and 43, bsmstdr and bssstdr).
Carry out analyses using (a) two sample t-test and (b) paired-sample t-test.
Can you give results regarding statistical significance, and also say something about
the importance (or otherwise) about the magnitude of the difference (concept of effect
size).
(2) Investigate whether there is any difference in mathematics achievement between
girls and boys (use variable 42).