Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Review • Social facilitation effect: People ride stationary bikes alone or in groups. Measure each person’s power in both conditions. Power (W) Subject Alone Group Diff A 224 245 21 B 167 160 -7 C 289 301 12 D 257 264 7 E 348 360 12 F 207 205 -2 Mdiff = 7.2 1. 2. Calculate difference scores Calculate mean difference score Effect Size 10/18 Effect Size • If there's an effect, how big is it? – How different is m from m0, or mA from mB, etc.? • Separate from reliability – Inferential statistics measure effect relative to standard error – Tiny effects can be reliable, with enough power – Danger of forgetting about practical importance • Estimation vs. inference – Inferential statistics convey confidence – Estimation conveys actual, physical values • Ways of estimating effect size – Raw difference in means – Relative to standard deviation of raw scores Direct Estimates of Effect Size • Goal: estimate difference in population means – One sample: m - m0 – Independent samples: mA – mB – Paired samples: mdiff • Solution: use M as estimate of m – One sample: M – m0 – Independent samples: MA – MB – Paired samples: Mdiff • Point vs. interval estimates – We don't know exact effect size; samples just provide an estimate – Better to report a range that reflects our uncertainty • Confidence Interval – Range of effect sizes that are consistent with the data – Values that would not be rejected as H0 • CI is range of values for m or mA – mB consistent with data – Values that, if chosen as null hypothesis, would lead to |t| < tcrit • One-sample t-test (or paired samples): – Retain H0 if t = M - m0 < tcrit i.e. SE M - m0 < tcrit × SE -4 p p 0.0 0.4 0.8 0.0 0.4 0.8 – Therefore any value of m0 within tcritSE of M would not be rejected 0.0 0.4 0.8 0.0 0.4p 0.8 Computing Confidence Intervals -4 -2 tcritSE M M m m m -2 00 -4 -400 2 -2 -22 4 00 M z – tcritSE z m 004 M +ztcritSE z 2 2 4 4 Formulas for Confidence Intervals • Mean of a single population (or of difference scores) M ± tcritSE • Difference between two means (MA – MB) ± tcritSE • Effect of sample size – Increasing n decreases standard error – Confidence interval becomes narrower – More data means more precise estimate • Always use two-tailed critical value – p(|tdf| > tcrit) = a/2 – Confidence interval has upper and lower bounds – Need a/2 probability of falling outside either end Interpretation of Confidence Interval • Pick any possible value for m (or mA – mB) • IF this were true population value – 5% chance of getting data that would lead us to falsely reject that value – 95% chance we don’t reject that value • For 95% of experiments, CI will contain true population value – "95% confidence" – – – – 0.0 0.4 0.8 p • Other levels of confidence Can calculate 90% CI, 99% CI, etc. Correspond to different alpha levels: confidence = 1 – a Leads to different tcrit: t.crit = qt(1-alpha/2,df,low=FALSE) Higher confidence requires wider intervals (tcrit increases) tcritSE • Relationship to hypothesis testing – If m0 (or 0) is not in the confidence interval, M M then we reject H0 m -4 -2 00 M – tcritSE z 2 4 M + tcritSE Standardized Effect Size • Interpreting effect size depends on variable being measured – Improving digit span by 2 more important than for IQ • Solution: measure effect size relative to variability in raw scores • Cohen's d – Effect size divided by standard deviation of raw scores – Like a z-score for means Samples d dTrue Estimated One Independent Paired m - m0 s m A - mB s m diff s M - m0 M A - MB M diff MS MS MS Meaning of Cohen's d • How many standard deviations does the mean change by? • Gives z-score of one mean within the other population – (negative) z-score of m0 within population – z-score of mA within Population B • pnorm(d) tells how many scores are above other mean (if population is Normal) -4 m -2 0 m 0 z 2 4 0.0 0.4 0.8 ds 0.0 0.4 0.8 p p 0.0 0.4 0.8 – Fraction of scores in population that are greater than m0 – Fraction of scores in Population A that are greater than mB -4 ds -4-2 m m 0 2A z z -20 B 24 4 Cohen's d vs. t Samples Statistic d t One Independent Paired M - m0 M A - MB M diff MS M - m0 MS n MS M A - MB æ ö MSç n1 + n1 ÷ è A Bø MS M diff MS n • t depends on n; d does not • Bigger n makes you more confident in the effect, but it doesn't change the size of the effect