Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Between Groups & Within-Groups ANOVA • BG & WG ANOVA – Partitioning Variation – “making” F – “making” effect sizes • Things that “influence” F – Confounding – Inflated within-condition variability • Integrating “stats” & “methods” ANOVA ANalysis Of VAriance Variance means “variation” • Sum of Squares (SS) is the most common variation index • SS stands for, “Sum of squared deviations between each of a set of values and the mean of those values” SS = ∑ (value – mean)2 So, Analysis Of Variance translates to “partitioning of SS” In order to understand something about “how ANOVA works” we need to understand how BG and WG ANOVAs partition the SS differently and how F is constructed by each. Variance partitioning for a BG design Mean Variation among all the participants – represents variation due to “treatment effects” and “individual differences” SSTotal = Tx C 20 30 10 30 10 20 20 20 15 25 Variation between the conditions – represents variation due to “treatment effects” SSEffect + Called “error” because we can’t account for why the folks in a condition -- who were all treated the same – have different scores. Variation among participants within each condition – represents “individual differences” SSError How a BG F is constructed Mean Square is the SS converted to a “mean” dividing it by “the number of things” SSTotal = SSEffect + SSError dfeffect = k - 1 represents # conditions in design F = MSeffect MSerror = SSeffect / dfeffect SSerror / dferror dferror = ∑n - k represents # participants in study How a BG r is constructed r2 = effect / (effect + error) conceptual formula = SSeffect / ( SSeffect + SSerror ) definitional formula = F / (F + dferror) computational forumla F = MSeffect MSerror = SSeffect / dfeffect SSerror / dferror An Example … i p O S S m S u d F S a i e N e a v B 4 1 4 2 7 1 0 4 0 W 0 8 0 2 0 2 0 T 4 9 0 8 0T SStotal = SSeffect + SSerror 1757.574 = 605.574 + 1152.000 r2 = SSeffect / ( SSeffect + SSerror ) = 605.574 / ( 605.574 + 1152.000 ) = .34 r2 = = F / (F + dferror) 9.462 / ( 9.462 + 18) = .34 Variance partitioning for a WG design Mean Tx C Sum Dif 20 30 50 10 10 30 40 20 10 20 30 10 20 20 40 0 15 25 Variation among participants – estimable because “S” is a composite score (sum) SSTotal = SSEffect + SSSubj Called “error” because we can’t account for why folks who were in the same two conditions -- who were all treated the same two ways – have different difference scores. Variation among participant’s difference scores – represents “individual differences” + SSError How a WG F is constructed Mean Square is the SS converted to a “mean” dividing it by “the number of things” SSTotal = SSEffect + SSSubj + SSError dfeffect = k - 1 F = MSeffect MSerror = SSeffect / dfeffect represents # conditions in design SSerror / dferror dferror = (k-1)*(n-1) represents # data points in study How a WG r is constructed r2 = effect / (effect + error) conceptual formula = SSeffect / ( SSeffect + SSerror ) definitional formula = F / (F + dferror) computational forumla F = MSeffect MSerror = SSeffect / dfeffect SSerror / dferror An Example … Don’t ever do this with e N e a real data !!!!!! S 4 0 0 e S 2 0 0 Te sts of W ithin-Subje cts Effe cts Measure: MEASURE_1 Source fac tor1 Error(factor1) Sphericity Ass umed Sphericity Ass umed Ty pe III Sum of Squares 605.574 281.676 n df 1 9 Mean Square 605.574 31.297 F 19.349 Sig. .002 - M T I I I S q d F S u i S g f I 1 1 1 4 0 E 5 9 3 SStotal = SSeffect + SSsubj 1757.574 = 605.574 + 281.676 Professional statistician on a closed course. Do not try at home! + + SSerror 870.325 r2 = SSeffect / ( SSeffect + SSerror ) = 605.574 / ( 605.574 + 281.676 ) = .68 r2 = F / (F + dferror) = 19.349 / ( 19.349 + 9) = .68 What happened????? Same data. Same means & Std. Same total variance. Different F ??? BG ANOVA SSTotal = SSEffect + WG ANOVA SSError SSTotal = SSEffect + SSSubj + SSError The variation that is called “error” for the BG ANOVA is divided between “subject” and “error” variation in the WG ANOVA. Thus, the WG F is based on a smaller error term than the BG F and so, the WG F is generally larger than the BG F. What happened????? Same data. Same means & Std. Same total variance. Different r ??? r2 = effect / (effect + error) conceptual formula = SSeffect / ( SSeffect + SSerror ) definitional formula = F / (F + dferror) computational forumla The variation that is called “error” for the BG ANOVA is divided between “subject” and “error” variation in the WG ANOVA. Thus, the WG r is based on a smaller error term than the BG r and so, the WG r is generally larger than the BG r. Both of these models assume there are no confounds, and that the individual differences are the only source of withincondition variability BG SSTotal = SSEffect + SSError WG SSTotal = SSEffect+SSSubj+SSError A “more realistic” model of F F= SSeffect / dfeffect SSerror / dferror IndDif individual differences BG SSTotal = SSEffect + SSconfound + SSIndDif + SSwcvar WG SSTotal = SSEffect + SSconfound +SSSubj + SSIndDif + SSwcvar SSconfound between condition variability caused by anything(s) other than the IV (confounds) SSwcvar inflated within condition variability caused by anything(s) other than “natural population individual differences” Imagine an educational study that compares the effects of two types of math instruction (IV) upon performance (% - DV) Participants were randomly assigned to conditons, treated, then allowed to practice (Prac) as many problems as they wanted to before taking the DV-producing test IV • compare Ss 5&2 - 7&4 Control Grp Exper. Grp Prac DV S1 S3 5 75 Prac DV S2 10 82 5 74 S4 10 84 S5 10 78 S6 15 88 S7 10 79 S8 Confounding due to Prac • mean prac dif between cond 15 89 WG variability inflated by Prac • wg corrrelation or prac & DV Individual differences • compare Ss 1&3, 5&7, 2&4, or 6&8 The problem is that the F-formula will … • Ignore the confounding caused by differential practice between the groups and attribute all BG variation to the type of instruction (IV) overestimating the effect • Ignore the inflation in within-condition variation caused by differential practice within the groups and attribute all WG variation to individual differences overestimating the error • As a result, the F & r values won’t properly reflect the relationship between type of math instruction and performance we will make a statistical conclusion error ! • Our inability to procedurally control variables like this will lead us to statistical models that can “statistically control” them F= SSeffect / dfeffect r SSerror / dferror = F / (F + dferror) How research design impacts F integrating stats & methods! SSTotal = SSEffect+SSconfound+SSIndDif+SSwcvar F= SSeffect / dfeffect SSerror / dferror SSEffect “bigger” manipulations produce larger mean difference between the conditions larger F SSconfound between group differences – other than the IV -change mean difference changing F • if the confound “augments” the IV F will be inflated • if the confound “counters” the IV F will be underestimated SSIndDif more heterogeneous populations have larger withincondition differences smaller F SSwcvar within-group differences – other than natural individual differences smaller F • could be “procedural” differential treatment within-conditions • could be “sampling” obtain a sample that is “more heterogeneous than the target population”