Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
EDU5950 SEM1 2015-16 RELIABILITY ANALYSIS -CRONBACH ALPHA TEST FOR NORMALITY STEPS IN CONDUCTING RESEARCH MEASUREMENT Consists of assigning numbers or labels to units of analysis to represent the variables/phenomenon under study. The key variables need to be translated into numbers so the researcher can analyze the data using statistics. The process of measurement consists of moving from the theoretical definition of a variable (typically called the construct definition) to the concrete mode of measuring that variable in the research study. MEASUREMENT (REFER TO TEACHER EFFICACY DATA) When designing an instrument, keep in mind the following: The conclusions drawn in a research study are only as good as the data that is collected. The data that is collected is only as good as the instrument that collects the data. A poorly designed instrument will lead to bad data, which will lead to bad conclusions. Therefore, developing a good instrument is the most important part of conducting a high quality research study. VALIDITY AND RELIABILITY Validity is the most important consideration in developing and evaluating measuring instruments. Validity is the degree to which evidence and theory support the interpretations on the meaning of the scores derived from the instrument Content Validity: “based on expert ratings of the items” in the test or measurement Construct Validity: “based on the extent of scores derived from the instrument truly reflect the theory behind the psychological construct being measured. 5 RELIABILITY Reliability refers to “how well we are measuring whatever it is that is being measured (regardless of whether or not it is the right quantity to measure).” In statistics or measurement theory, a measurement or test is considered reliable if it produces consistent results over repeated testings. -D. Rindskopf, Reliability: Measurement. In: Neil J. Smelser and Paul B. Baltes, Editor(s)-in-Chief, International Encyclopedia of the Social & Behavioral Sciences, Pergamon, Oxford, 2001, Pages 13023-13028. (http://www.sciencedirect.com/science/article/B7MRM-4MT09VJ2XN/1/083e3cc0b8b9d4e027b0ba214dcd9fa3) 6 HOW CAN RELIABILITY BE ESTABLISHED? Test-Retest reliability – administer a test or instrument to the same group of individuals on two occasions and correlate the two sets of scores Equivalent-Forms reliability – administer two equivalent forms of test or instrument to the same individuals Internal-Consistency measures of reliability – test whether all the items in the test or instrument are measuring the same thing. It is a measure of homogeneity of the items. Split-Half reliability Kuder-Richardson procedures Coefficient Alpha (Cronbach Alpha after Lee Cronbach) 7 •RELIABILITY ANALYSIS REFER TO MATHEMATICS TEACHERS’ EFFICACY DATA 1. RECODE ALL NEGATIVE ITEMS 2. CLICK ANALYZE => SCALE => RELIABILITY ANALYSIS (YOU WILL GET THE RELIABILITY DIALOG BOX) 3. TRANSFER THE MEASURED ITEMS TO THE RIGHT BOX – 14 ITEMS FOR TEACHERS’ EFFICACY SCALE 4. SELECT THE FOLLOWING ICONS => CONTINUE => OK Item Statistics Mean Std. Deviation N My teacher wants us to enjoy learning maths 3.7419 1.55696 62 My teacher understand our problems in learning maths 3.9355 1.48071 62 My teacher try to make mathematics lessons interesting 3.9839 1.53101 62 4.2097 1.48365 62 My teacher show us step by step and how to solve maths problems 4.2258 1.53023 62 My teacher listen carefully to what we say 4.1290 1.18022 62 My teacher is friendly to us 3.5323 1.50101 62 My teacher gives us time to explore new maths problems 3.7581 1.19679 62 My teacher wants us to understand the content of this maths class 4.4032 1.38445 62 My teacher explains why mathematics is important 3.9677 1.49280 62 We do a lot of group work in mathematics class 3.1129 1.31952 62 3.9677 1.51460 62 A_TF13 RECODE 4.0161 1.41990 62 A_TF14 RECODE 4.5161 1.25112 62 My teacher appreciates it when we try hard, even when our results are not so good My teacher thinks mistakes are okey as long as we are learning from them Reliability Statistics Cronbach's Alpha .891 Cronbach's N of Items Alpha Based on Standardized Items .890 14 Item-Total Statistics Scale Mean if Item Deleted My teacher wants us to enjoy learning maths My teacher understand our problems in learning maths My teacher try to make mathematics lessons interesting Scale Variance if Corrected Item- Squared Multiple Item Deleted Total Correlation Correlation Cronbach's Alpha if Item Deleted 51.7581 134.285 .753 .731 .874 51.5645 146.479 .425 .421 .890 51.5161 134.287 .768 .784 .874 51.2903 136.242 .735 .890 .875 51.2742 134.399 .765 .882 .874 51.3710 143.024 .690 .652 .879 My teacher appreciates it when we try hard, even when our results are not so good My teacher show us step by step and how to solve maths problems My teacher listen carefully to what we say 51.9677 138.097 .667 .643 .879 51.7419 142.752 .689 .643 .879 51.0968 140.056 .669 .845 .879 51.5323 137.696 .684 .623 .878 52.3871 160.536 .048 .446 .904 51.5323 143.761 .491 .615 .887 A_TF13 RECODE 51.4839 155.467 .181 .574 .900 A_TF14 RECODE 50.9839 147.983 .471 .534 .887 My teacher is friendly to us My teacher gives us time to explore new maths problems My teacher wants us to understand the content of this maths class My teacher explains why mathematics is important We do a lot of group work in mathematics class My teacher thinks mistakes are okey as long as we are learning from them REPORTING The pilot study was administered to 31 graduate students undertaking Statistics for Educational Research. The internal consistency estimates, based on Cronbach’s alpha, were satisfactory thus acceptable. Based on 14 items, the Cronbach’s alpha obtained is .891. However further analysis using 12 items (by omitting item 11 AND 13) the internal consistency increased to .904. Inter-Item Correlation Matrix - - a 4-Item Summated Scale for Measuring ORG. LOYALTY (n = 518) 1. If I were completely 2. I feel a 4. I don't have a strong free to choose, I would strong sense of 3. I often think personal desire to continue working for my loyalty to the of leaving the continue working for my current employer org I work for org. I work for current employer 1. If I were completely free to choose, I would continue working for my current employer 1.000 .690 .646 .743 2. I feel a strong sense of loyalty to the org I work for .690 .573 .674 3. I often think of leaving the org. I work for .646 1.000 .573 1.000 .711 4. I don't have a strong personal desire to continue working for my current employer .743 .674 .711 1.000 Response Options: 7-Point Scales (1= Strongly Disagree, 7 = Strongly Agree) Reliability Statistics Cronbach's Alpha .891 Cronbach's Alpha Based on Standardized Items N of Items .892 4 Item-Total Statistics Scale Mean if Item Deleted Scale Variance if Item Deleted Corrected ItemTotal Correlation Squared Multiple Correlation Cronbach's Alpha if Item Deleted 9.81 9.81 .789 .633 .849 9.71 12.803 .720 .539 .874 10.06 11.857 .721 .541 .876 9.76 11.447 .816 .668 .838 If I were completely free to choose, I would continue working for my current employer I feel a strong sense of loyalty to the org I work for I often think of leaving the org. I work for I don't have a strong personal desire to continue working for my current employer Use these for item analysis; i.e., determining quality of individual items. •Test for normality of scores TEST FOR NORMALITY TEST FOR NORMALITY TEST FOR NORMALITY Descriptives Statistic Mean 95% Confidence Interval for Mean 5% Trimmed Mean Median Variance TEACHER_E FFICACY Std. Deviation Minimum Maximum Range Interquartile Range Skewness Kurtosis Lower Bound Upper Bound 3.9643 3.7321 Std. Error .11613 4.1965 3.9507 3.7857 .836 .91443 2.21 5.71 3.50 1.52 .388 -.852 .304 .599 TEST FOR NORMALITY Tests of Normality Kolmogorov-Smirnova Statistic .110 TEACHER_EFFICACY a. Lilliefors Significance Correction df Shapiro-Wilk Sig. 62 .061 Statistic .951 df Sig. 62 .016 TEST FOR NORMALITY Extreme Values Case Number Value 23 5.71 1 33 5.71 2 5 5.64 Highest 3 27 5.64 4 11 5.57 5 TEACHER_EFFICACY 12 2.21 1 24 2.50 2 51 2.71 Lowest 3 45 2.71 4 59 2.93a 5 a. Only a partial list of cases with the value 2.93 are shown in the table of lower extremes. TEST FOR NORMALITY TEST FOR NORMALITY TEACHER_EFFICACY Stem-and-Leaf Plot Frequency Stem & Leaf 1.00 2 . 2 6.00 2 . 577999 15.00 3 . 000012222222224 14.00 3 . 55556677778899 6.00 4 . 001222 9.00 4 . 556677779 6.00 5 . 001333 5.00 5 . 56677 Stem width: 1.00 Each leaf: 1 case(s) TEST FOR NORMALITY TEST FOR NORMALITY TEST FOR NORMALITY EXPLORING DATA • Normality can be assessed in SPSS using the Explore option of the Descriptive Statistics menu. • In this example we will assess the normality distribution of Scores of Test I. We wish to assess for the sample as a whole. If we want to do separately for subgroup within our sample, moves the variables that define these subgroups into the factor list. TO ASSESSING NORMALITY Click Analyze => Descriptive Statistics => Explore. Click and move variables you require statistics, graphs and test for into Dependent List. In Display section, select Both. In the Explore Statistics dialogue, Descreptive are selected by default.Click Continue. Click on the Plot button. Select Histogram and Normality plots with test. Click on Continue. Click on Option. In the Missing value section, click Exclude cases pairwise. Click continue. Then OK. OUTPUTS OUTPUTS The descriptive statistics is shown in the tables. To obtain the 5% Trimmed Mean, SPSS removes the top and bottom 5% of your cases and recalculate a new mean value. Compare the original mean with new trimmed mean to see whether some of the more extreme scores are having a strong influence on the mean. If these two mean values are very different, you may need to investigate these data point further. OUTPUTS The Kolmogorov-Smirnov statistic assess the normality of the distribution of scores. A non-significant result ( Sig value more than .05 ) indicates normality. The actual shape of the distribution can be seen in the Histograms. For this Histograms, scores appear to be reasonably normally distributed. OUTPUTS The Normal Q-Q Plots shows the observed value for each score is plotted against the expected value from the normal distribution. A reasonably straight line suggests a normal distribution. The Detrended Normal Q-Q Plots displayed in the output are obtained by plotting the actual deviation of the scores from the straight line. There should be no real clustering of points, with most collecting around the zero line. EXAMPLE OF WRITE-UP Ho1: There is no significant difference in the mean overall test performance in the learning of the Straight Lines topic between the graphing calculator (GC) strategy group and the conventional instruction (CI) strategy group. The means and standard deviations of the overall test performance for both the GC and the CI strategy groups are provided in Table 4.4. The overall performance test ranged between 0 and 40. Mean overall test performance of the GC strategy group was 16.81 (SD=4.76) while mean overall test performance of the CI strategy group was 12.53 (SD=4.99). An independent t-test analysis showed that the difference in the means were significant, t(38)=2.78, p<.05. The results indicated that there was a significant difference in the mean overall test performance in the learning of the Straight Lines topic between the GC strategy group and the CI strategy group. The magnitude of the differences in the means was considered large based on Cohen (1988) with eta squared =.17. The guidelines proposed by Cohen (1988) for interpreting this value are: .01 = small effect, .06 = moderate effect, .14 = large effect. This finding indicated that the GC strategy group had performed significantly better than the CI strategy group. EXAMPLE WRITE-UP Table 4.3: Independent samples t-test to compare means monthly test performance before experiment in Phase I Group GC strategy n 21 M 59.00 SD 10.25 MD t df p CI strategy 19 59.26 21.19 -.263 -.049 25.41 .961 Table 4.3 shows the results of the analysis. The total monthly test performance was 100. The mean performance for the GC strategy group and the CI strategy group were 59.00 and 59.26 respectively. Levene’s test indicated that the assumption for equal variance has been violated, F=9.95, p<.05. Therefore the reading for the output for the independent t-test was based on the reading for equal variance not assumed. The results of the t-test showed that there was statistically no significant difference between the mean monthly test performance for the GC strategy group and the CI strategy group, t(25.41)=−.049, p>.05. This suggested that the students’ mathematics performance for both groups in the group did not differ significantly. H01: There is no significant difference in the mean overall performance in the learning of statistic between the PBL-Tr, PBL-Web and Conv groups. The means and standard deviations of the overall performance for the PBL-Tr, PBLWeb and the Conv strategy groups and also results of the ANOVA test. The overall performance score ranged between 0 and 100. Mean overall performance of the PBL-Tr group was 70.08 (SD=12.08) while mean overall performance of the PBLWeb group was 80.18 and (SD=17.2) and the mean of overall performance of the Conv strategy group was 56.6 (SD=20.38). An ANOVA test analysis showed that the difference in the means were significant, F(2,61)=6.35, p<.05. The results indicated that there was significant difference in the mean overall performance in the learning of statistics between the three groups. The magnitude of the differences in the means was considered large based on Cohen (1988) with eta squared (ES) = 0.172. The guidelines proposed by Cohen (1988) for interpreting this value are: .01 = small effect, .06 = moderate effect, .14 = large effect. Based on Post-Hoc test the mean of overall performance for Conv group was significantly lower than PBL-Tr and PBL-Web groups. However, PBL-Tr did differ significantly from PBL-Web group at 5% level of significance.