Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
CHAPTER 6 COMPARING SAMPLES (WHEN NORMAL DISTRIBUTIONS CANNOT BE USED) • This chapter introduces: – the ways for comparing the means from different samples where distributions are not matching a normal distribution. 1 School of the Built Environment PARAMETRIC AND NONPARAMETRIC TESTS • Parametric tests assumes that the samples are drawn from predictable (and often Normal) population . • In non-parametric test, we do not assume any particular distribution and is commonly used for categorical data. • Advantages of non-parametric tests – They do not require us to make the assumption that a population is distributed in the shape of a normal curve or another specific shape – Generally easier to compute and understand 2 School of the Built Environment PARAMETRIC AND NONPARAMETRIC TESTS • Disadvantages of non-parametric tests – They ignore a certain amount of information Example: by replacing values with rankings – They are often not as efficient or “sharp” as parametric tests The estimate of an interval at the 95% confidence level using a non-parametric test may be twice as large as the estimate using a parametric test Trade off: lose sharpness in estimating intervals but gain the ability to use less information and to calculate faster. 3 School of the Built Environment INDEPENDENT SUBJECTS & REPEATED / MATCHED SUBJECTS • Before carrying out statistical tests, we need to know whether the 2 samples to be compared contain independent subjects or repeated / matched subjects. – The statistical formula for each case is different • Independent subjects – One group of cases or subjects forms one sample and a completely different group forms the second sample – Subjects are randomly allocated to each group – Eg planning applications from different local authorities 4 School of the Built Environment INDEPENDENT SUBJECTS & REPEATED / MATCHED SUBJECTS • Repeated measures – Each subject is measured twice – Example in both control and experimental condition, as before and after treatment • Matched pairs – On the basis of pre-selection, subjects are sorted into matched pairs on the variable to be measured. – Example pairs of equal ability 5 School of the Built Environment STATISTICAL TESTS FOR DIFFERENT SITUATIONS & DATA 6 School of the Built Environment TESTING FOR SIGNIFICANCE Step 1 NULL HYPOTHESIS, H0 • Assumption: 2 samples belong to the same population no real difference between them. Observed differences (if any) are random and is to be expected Step 2 • Carry out statistical test to get a value for the statistic Step 3 • Look up statistical tables for no. of subjects and find the value of the statistic (corresponding to the significance level decided to use, normally p = 5% or p = 1% level) 7 School of the Built Environment TESTING FOR SIGNIFICANCE Step 4 • If the calculated statistical value is bigger than that in the table – reject H0 (hypothesis of no difference) and – conclude that the two samples are different • If calculation value is smaller than that in the tables, – keep H0 and – conclude that the 2 samples do belong to the same population 8 School of the Built Environment THE SIGN TEST (Nominal data; Repeated Measures / Matched Pair) • Test the claim that there will be a significant difference between subjects’ scores on an arithmetic test after drinking 4 pints of beer • Arithmetic test is administered to each person with and without beer. The scores are shown in the table below. • There are: – 9 “+” – 1 “–”; and – 2 zeros • H0 : Drinking beer does not impair arithmetic performance • H1 : Drinking beer impairs arithmetic performance School of the Built Environment 9 THE SIGN TEST • Count the number of occurrences of the least frequent sign. Call this X – (only one ‘-’ X = 1) • Count the total number of signs. Label this as N – (1 ‘-’ plus 9 ‘+’ N = 10) • From table (left) find the critical value (table) for your N – (for N = 10, critical X value in the table = 1) 10 School of the Built Environment THE SIGN TEST (Nominal data; Repeated Measures / Matched Pair) • In this example, as there is only 1 “-” sign and it falls on the critical value, the null hypothesis is therefore rejected. • Conclusion: drinking beer does impair arithmetic performance 11 School of the Built Environment WILCOXON TEST (Ordinal data; Repeated Measures / Matched Pair) • It is a less crude test than the sign test but it is applied to similar data. • Same data as previous example of Sign Test: Sort from smallest to largest Ignore “+” or “-” sign School of the Built Environment (A-B) value Frequency Ranking 1 3 (1+2+3)/3 = 2 2 1 4/1 = 4 3 2 (5+6)/2 = 5.5 4 0 0 5 4 (7+8+9+10)/4 =12 8.5 WILCOXON TEST (Ordinal data; Repeated Measures / Matched Pair) B is better than A Denotes A better than B • Sum the value in ranking row for subjects doing better under A (= 53) (5.5 + 8.5 + 8.5 + 2 + 8.5 + 5.5 + 8.5 + 2 + 4 = 53) • Sum the value in ranking row for subjects doing better under B (= 2) • The test statistic is the smaller value of these 2 (smaller of 53 & 2 = 2) 13 School of the Built Environment WILCOXON TEST (Ordinal data; Repeated Measures / Matched Pair) • T = 2, N = 10, – critical value T = 8 at 5% level ; & – critical value T = 3 at 1% level • Since T = 2 is smaller than either of the critical values, the difference is significant at both 5% & 1% level • Conclusion: Reject Null hypothesis 14 School of the Built Environment MANN WHITNEY U TEST (Ordinal data; Independent Subjects) • Compare truancy days off school from families in 2 housing estates A & B. • Note that these are data from 2 independent samples How the Rank scores are derived Value Frequency Ranking 13 1 1 14 1 2 18 1 3 22 1 4 23 3 (5+6+7)/3 = 6 25 1 8 26 2 (9+10)/2 = 9.5 15 School of the Built Environment MANN WHITNEY U TEST (Ordinal data; Independent Subjects) • Rank the scores across both estates as if they are a single group. • Add up the rank score for both estates • Compute U1 and U2 Where: R = Total of ranks in smallest group (R = 66) N1 = Number of cases in smallest group (N1 = 10) N2 = Number of cases in largest group (N2 = 10) 16 School of the Built Environment MANN WHITNEY U TEST • Take the smaller of U1 and U2 – If this value is ≤ the critical values in the statistical tables, then the difference is significant • U2 = 11 < U1 = 89 and critical value = 23 • Since U2 = 11 < critical value = 23, result is significant. Truancy rates on the two estates are significantly different 17 School of the Built Environment UNRELATED T-TEST (Interval / Ratio data; Independent Subjects) • The way of assessing differences between means in chapter 5 is called the z test – Accurate only for large samples (≥ 30) • If sample size < 30, sample std dev is not reliable as estimate of population std dev • use t-test (or students t-test) – t-distribution is quite similar to normal distribution but shape is flatter & wider – There is a different t-distribution for every possible sample size – Shape approaching normal distribution as sample size increases 18 School of the Built Environment UNRELATED T-TEST (Interval / Ratio data; Independent Subjects) Note that t-test is a parametric test 19 School of the Built Environment UNRELATED T-TEST (Interval / Ratio data; Independent Subjects) Example • Does an average box of cereal contain 368 grams of cereal? A random sample of 36 boxes had a mean of 372.5 grams and a standard deviation of 12 grams. Test at the 0.05 level. H0: µ = 368 t= sample _ mean − µ n H1: µ ≠ 368 Degree of freedom = 36-1 = 35 σ t= 372.5 − 368 = 2.25 12 36 Critical t = 2.03 (from t-distribution table) Therefore reject H0. There is evidence that the population average weight is not 368 gram. 20 School of the Built Environment RELATED T-TEST (Interval / Ratio data; Repeated Measures / Matched Pairs ) • Concluding concept is similar to unrelated t-test • In most statistical packages t values will be given, with the associated probability level quoted. • If the probability value is LESS than 0.05 or 0.01 then the result is significant at the 5% or 1% level respectively 21 School of the Built Environment SPSS Exercise 5 • Hand-in Requirements: open a word file to store your analysis results for hand in. Please type your name at the top of first page. • Data file - The file name is ‘uk house price 1980 – 2005’. It is saved on VISION. Part 1: Simple analysis of interval and ratio data • To make a preliminary examination of your data, use Explore to examine the variable ‘achange1’. Analyse Descriptive Statistics Explore • Move the variable achange1 into the dependent box and click OK. 22 School of the Built Environment SPSS Exercise 5 a b c a. Valid - This refers to the non-missing cases. In this column, the N is given, which is the number of non-missing cases; and the Percent is given, which is the percent of non-missing cases. b. Missing - This refers to the missing cases. In this column, the N is given, which is the number of missing cases; and the Percent is given, which is the percent of the missing cases. c. Total - This refers to the total number cases, both non-missing and missing. In this column, the N is given, which is the total number of cases in the data set; and the Percent is given, which is the total percent of cases in the data set. 23 School of the Built Environment SPSS Exercise 5 a b c d e f g h i j k l m n o 24 School of the Built Environment SPSS Exercise 5 a. Statistic - These are the descriptive statistics. b. Std. Error - These are the standard errors for the descriptive statistics. The standard error gives some idea about the variability possible in the statistic. c. Mean - This is the arithmetic mean across the observations. It is the most widely used measure of central tendency. It is commonly called the average. The mean is sensitive to extremely large or small values. d. 95% Confidence Interval for Mean Lower Bound - This is the lower (95%) confidence limit for the mean. If we repeatedly drew samples of 200 students' writing test scores and calculated the mean for each sample, we would expect that 95% of them would fall between the lower and the upper 95% confidence limits. This gives you some idea about the variability of the estimate of the true population mean. 25 School of the Built Environment SPSS Exercise 5 e. 95% Confidence Interval for Mean Upper Bound - This is the upper (95%) confidence limit for the mean. f. 5% Trimmed Mean - This is the mean that would be obtained if the lower and upper 2.5% of values of the variable were deleted. If the value of the 5% trimmed mean is very different from the mean, this indicates that there are some outliers. However, you cannot assume that all outliers have been removed from the trimmed mean. g. Median - This is the median. The median splits the distribution such that half of all values are above this value, and half are below. h. Variance - The variance is a measure of variability. It is the sum of the squared distances of data value from the mean divided by the variance divisor. We don't generally use variance as an index of spread because it is in squared units. Instead, we use standard deviation. School of the Built Environment 26 SPSS Exercise 5 i. St. Deviation - Standard deviation is the square root of the variance. It measures the spread of a set of observations. The larger the standard deviation is, the more spread out the observations are. j. Minimum - This is the minimum, or smallest, value of the variable. k. Maximum - This is the maximum, or largest, value of the variable. l. Range - The range is a measure of the spread of a variable. It is equal to the difference between the largest and the smallest observations. It is easy to compute and easy to understand. However, it is very insensitive to variability. m. Interquartile Range - The interquartile range is the difference between the upper and the lower quartiles. It measures the spread of a data set. It is robust to extreme observations. School of the Built Environment 27 SPSS Exercise 5 n. Skewness - Skewness measures the degree and direction of asymmetry. A symmetric distribution such as a normal distribution has a skewness of 0, and a distribution that is skewed to the left, e.g. when the mean is less than the median, has a negative skewness. o. Kurtosis - Kurtosis is a measure of the heaviness of the tails of a distribution. A normal distribution has kurtosis 0. Extremely nonnormal distributions may have high positive or negative kurtosis values, while nearly normal distributions will have kurtosis values close to 0. Kurtosis is positive if the tails are "heavier" than for a normal distribution and negative if the tails are "lighter" than for a normal distribution. 28 School of the Built Environment SPSS Exercise 5 a. Frequency - This is the frequency of the leaves. b. Stem - This is the stem. It is the number in the 10s place of the value of the variable (stem width). Eg in the 2nd last line, the stem is 2 and leaves are 5, 7 & 9. The value of the variable is 25, 27 & 29. c. Leaf - This is the leaf. It is the number in the 1s place of the value of the variable. The number of leaves tells you how many of these numbers is in the variable. For example, on the 2nd line, there is two 5, two 6 and two 8 (hence, the frequency is six). 29 School of the Built Environment SPSS Exercise 5 a b c d e 30 School of the Built Environment SPSS Exercise 5 a. This is the maximum score unless there are values more than 1.5 times the inter quartile range above Q3, in which, it is the line extends to a maximum of 1.5 times the inter quartile range b. This is the third quartile (Q3), also known as the 75th percentile. c. This is the median (Q2), also known as the 50th percentile. If the median line within the box is not equidistant from the edges of the box, then the data is skewed d. This is the first quartile (Q1), also known as the 25th percentile. e. This is the minimum score unless there are values less than 1.5 times the inter quartile range below Q1, in which case, line extends to a maximum of 1.5 times the inter quartile range 31 School of the Built Environment SPSS Exercise 5 • Histogram Graph Legacy Dialogs, Histogram, Use the achange1 variable (Please tick the Display normal curve option when you run the histogram). 32 School of the Built Environment SPSS Exercise 5 • Hand in Requirement 1 Copy the histogram of achange1 you just produced to the word file, and answer these questions: a) How many percentage points does each bar represent in this diagram? 2.5% b) What are the minimum and the maximum percentage changes represented by the tallest bar? Max = 12.5%; Min = 10% c) What are the possible minimum and maximum quarterly house price changes? Min = -12.5% ; Max = 32.5% c) Is this distribution a normal distribution? Why? Not normal distribution as it does not resemble a bell curve. 33 School of the Built Environment SPSS Exercise 5 d) What are the main difference between a bar chart you produced in the previous exercise and a histogram? The bar graph is often used to show a visual comparison of discrete elements, while the histogram is used to show the frequency of nondiscrete, continuous items The items in the histogram are usually numbers that are grouped or categorized in such a way that they are considered to be ranges. In regards to bar graphs, the items are taken as separate entities A bar graph is drawn in such a way that a bar representing the frequency of an item is not touching the bar of the next item. There is a visible space between the bars. The bars in the histogram are always the touching the next one. There are no spaces in between 34 School of the Built Environment SPSS Exercise 5 • Analyse quantitative data using Report Analyse Report Case summarise Move index1, price1 and achange1 in the Variables box; and move year into the Grouping Variable(s) box. Hand in Requirement 2 – Repeat the report procedure. You could include more variables in Variables box, but do not use year this time as grouping variable, use period instead. Before you run the process, make sure you have not selected the Display Cases box. 35 School of the Built Environment SPSS Exercise 5 36 School of the Built Environment SPSS Exercise 5 37 School of the Built Environment SPSS Exercise 5 Part 2: Comparing Means: Parametric and Non-parametric Tests Independent samples t-test (Parametric) • Suppose we are interested in the comparison of housing price changes during the 1980s and the 1990s, we can use the comparing means analysis. • Our null hypothesis will be: At the 5% of level of significance, there is no significant difference between the mean quarterly house price changes during the 1980s and the 1990s • To use this parametric test, we need to confirm that the data is a normal distribution by using histogram and normal curve function. For this exercise, we presume the data confirms the normal distribution School of the Built Environment 38 SPSS Exercise 5 • Analyse data using Compare Means Analyse Compare Means Independent Sample t-test The test variable is achange1 and the Grouping Variable would be period. You also need to define groups: the value 1 (1980s) for group 1 and the value 2 (1990s) for group 2. • Interpretation of Test Results: The second table contains two set of test results: (1) Levene’s Test for Equality of Variances and (2) t-test for Equality of Means. 39 School of the Built Environment SPSS Exercise 5 • You should check, firstly, the significance level of the equality of variances of the two groups using Levene’s test for equality of variances. • If Levene’s Sig. is greater than 0.05, equal variances of the groups are assumed. • if Levene’s Sig. is smaller than 0.05, equal variances of the groups are not assumed. Equality of variance assumed 40 School of the Built Environment SPSS Exercise 5 • • • t-test for Equality of Means includes two rows of results: the first row results for the assumed equal variances, and the second row for the unequal variances. Should use the row for assumed equal variance since Sig F is > 0.05 (see previous slide) After you have decided which row of the t-test result is appropriate for your data, then you should look at the Sig. (2-tailed) level. – If this value is greater than 0.05 (5%), the mean quarterly house price changes of the two periods are not significantly different; – if this value is smaller than 0.05 (5%), the mean quarterly house price changes of the two groups are significantly different. 41 School of the Built Environment Value is < 5%; population is significantly different SPSS Exercise 5 Hand in Requirement 3 Examine the t test tables (from SPSS output window, you don’t need to copy the table over to word) and answer these questions in your word document: • Looking at the first table (Group Statistics), what can we say about house price changes over the two periods? – The average quarterly price increase for all houses has decreased to 1.6725% from 12.2475% 42 School of the Built Environment SPSS Exercise 5 • Are variances of house price changes over the two periods equal? Why? – The variances of house price changes over the 2 periods are equal as the Levene’s test for Equality of Variances cannot reject the null hypothesis. • What is our Null Hypothesis for this test? – The mean quarterly price change for the period 1980-1989 is equal to the mean quarterly price change for the period 19901999 at 5% significant level • What are the test results (values) for significant level, mean differences, standard error of differences, and 95% confident interval of difference? – Significant level = 5%; mean differences = 10.575; standard error of differences = 1.58875; 95% confident interval of difference = 7.41205 to 13.73795 43 School of the Built Environment SPSS Exercise 5 • Does the result of this parametric t-test accept or reject our null hypothesis? Why? – The t-test rejects the null hypothesis as its significant value is <5% Hand in Requirement 4 • We have assumed that our data is normally distributed for the above analysis. If our data is not normally distributed, we should use the nonparametric Mann-Whitney U test. • Analyse data using Compare Means Analyse Non parametric test Legacy Dialogs 2 Independent Sample The test variable is achange1 and the Grouping Variable would be period. You also need to define groups: the value 1 (1980s) for group 1 44 and the value 2 (1990s) for group 2. School of the Built Environment SPSS Exercise 5 The significance level (p) is the last row of the second table. If this value is greater than 0.05 (5%), the mean quarterly house price changes of the two periods are not significantly different; if this value is smaller than 0.05 (5%), the mean age of the two groups are significantly different. 45 School of the Built Environment SPSS Exercise 5 Hand in Requirement 5 • If both test variables have a normal distribution, we can use the Paired samples t-test (Parametric). • Run this test using nchange2 and ochange4. Analyse Compare Means • Discuss the result in your Word Document: 46 School of the Built Environment SPSS Exercise 5 • Looking at the first table (Paired Samples Statistics), what can we say about new and old house price changes over between 1980 and 2005? – • The average quarterly price change for older houses are greater than new houses What is our Null Hypothesis for this paired samples t test? – – The average quarterly price change for new houses are equal to the average quarterly price change for old houses ; or The difference between the average quarterly price change between old and new houses are zero. School of the Built Environment 47 SPSS Exercise 5 • What are the test results (values) for significant level, mean differences, standard error of differences, and 95% confident interval of difference? – – – Significant level = 5%; - mean differences = -1.22376 standard error of differences = 0.40366 95% confident interval = -2.02462 to -0.42291 48 School of the Built Environment SPSS Exercise 5 • Does the result of this parametric paired sample t-test accept or reject our null hypothesis? Why? – The null hypothesis should be rejected. – The value of 0.003 is the two-tailed p-value computed using the t distribution. It is the probability of observing a greater absolute value of t under the null hypothesis. The p-value is less than the pre-specified alpha level (5% here). We can conclude that mean difference is statistically significantly different from zero. 49 School of the Built Environment THE END Any questions? 50 School of the Built Environment