Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Introduction to Meta-Analysis by Michael Borenstein, Larry V. Hedges , Julian P. T Higgins, and Hannah R. Rothstein PART 2 Effect Size and Precision Summary of Chapter 3: Overview Chapter 4: Effect Sizes Based on Means Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) Chapter 6: Effect Sizes Based on Correlations Sarah McCann CAMARADES: Bringing evidence to translational medicine Chapter 3: Overview Treatment Effects and Effect Sizes Terminology: Effect size – relationship between two variables e.g. difference between males and females Treatment effect – only appropriate when quantifying the impact of a deliberate intervention e.g. the difference between treated and control groups Single group summary – estimating the mean or risk or rate of a single population e.g. prevalence of a disease in a particular region or mean scores on a test CAMARADES: Bringing evidence to translational medicine Chapter 3: Overview How to Choose an Effect Size Index 1. The effect sizes from the different studies should be comparable i.e. they measure (at least approximately) the same thing. 2. Estimates of the effect size should be computable from the information in the published reports (should not require re-analysis of raw data, unless they’re known to be available). 3. Need to be able to compute variance and confidence intervals. CAMARADES: Bringing evidence to translational medicine Chapter 3: Overview Summary Data Reported by Primary Studies Appropriate Effect Size Means and standard deviations (Chapter 4) Raw difference in means Standardised difference in means Response ratio Binary data e.g. events vs. non-events (Chapter 5) Risk ratio Odds ratio Risk difference Correlation between two variables (Chapter 6) Correlation coefficient CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means RAW (UNSTANDARDIZED) MEAN DIFFERENCE D The outcome is reported on a meaningful scale and all studies in the analysis use the same scale. Meta-analysis is performed directly on the raw difference in means. CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means Let µ1 and µ2 be the true (population) means of two groups. The population mean difference is defined as We can compute an estimate, D, of the population mean using the sample means from studies that use two independent groups, paired groups or matched designs. CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means Computing D from studies that use independent groups The variance is calculated using the sample standard deviations (S) and sample sizes (n) of the two groups. CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means Computing D from studies that use matched groups or pre-post scores Pairs of participants are matched in some way (e.g. siblings or patients at the same stage of disease) or the same subject acts as their own control (e.g. scores pretreatment are compared to scores pot-treatment) CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means STANDARDIZED MEAN DIFFERENCE, d and g The scale of measurement differs across studies and it is not meaningful to combine raw mean differences. The mean difference in each study is divided by that study’s standard deviation to create an index that is comparable across studies. CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means Let µ1 and σ1 and µ2 and σ2 be the true (population) means and standard deviations of two groups. The population mean difference is defined (assuming σ1 = σ2) as: We can compute an estimate, d, of the population mean using the sample means from studies that use two independent groups, paired groups or matched designs. CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means Computing d and g from studies that use independent groups By pooling the two estimates of the standard deviation, we obtain a more accurate estimate of their common value. CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means d has a slight bias, tending to overestimate the absolute value of δ in small samples. This bias can be removed with a simple correction, resulting in an unbiased estimate sometimes called Hedges’ g. To convert from d to Hedges’ g, a correction factor called J is used: And then: CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means Computing d and g from studies that use pre-post scores or matched groups CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means Including different study designs in the same analysis A systematic review may include some studies that use independent groups and others that used matched groups. The effect size from each study can be computed using the appropriate formula and all the studies can be combined in the same analysis. Direction of effect is arbitrary, the researcher must decide on a convention and apply it to all studies, regardless of design. CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means RESPONSE RATIOS When the outcome is measured on a physical scale (e.g. length, area, mass) and is unlikely to be zero, the ratio of the two means can be used as the effect size. Computations are carried out on a log scale. CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means The response ratio is computed as: The log response ratio is computed as: CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means SUMMARY POINTS The raw mean difference (D) may be used as the effect size when the outcome scale is either inherently meaningful or well known due to widespread use. This effect size can only be used when all studies in the analysis used precisely the same scale. The standardized mean difference (d or g) transforms all effect sizes to a common metric, and thus enables us to include different outcome measures in the same synthesis. CAMARADES: Bringing evidence to translational medicine Chapter 4: Effect Sizes Based on Means SUMMARY POINTS The response ratio (R) is often used in ecology. This effect size is only meaningful when the outcome has a natural zero point, but when this condition holds, it provides a unique perspective on the effect size. It is possible to compute an effect size and variance from studies that used two independent groups, from studies that used matched groups (or pre-post designs) and from studies that used clustered groups. These effect sizes may then be included in the same meta-analysis. CAMARADES: Bringing evidence to translational medicine Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) CAMARADES: Bringing evidence to translational medicine Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) RISK RATIO The ratio of two risks. Risk of death in treated group is 5/100 and the risk of death in the control group is 10/100, so the ratio of the two risks is 0.50. CAMARADES: Bringing evidence to translational medicine Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) CAMARADES: Bringing evidence to translational medicine Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) ODDS RATIO The ratio of two odds. Odds of death in the treated group would be 5/95 or 0.0526 and the odds of death in the control group would be 10/90 or 0.1111, so the ratio of the two odds would be 0.0526/0.1111 or 0.4737. CAMARADES: Bringing evidence to translational medicine Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) Why use log transformations? The log transformation is needed to maintain symmetry in the analysis. Assume that one study reports that the risk is twice as high in Group A while another reports that it is twice as high in Group B. Assuming equal weights, these studies should balance each other, with a combined effect showing equal risks (a risk ratio of 1.0). However, on the ratio scale these correspond to risk ratios of 0.50 and 2.00, which would yield a mean of 1.25. By working with log values we can avoid this problem. In log units the two estimates are +0.693 and -0.693, which yield a mean of 0.00. We convert this back to a risk ratio of 1.00, which is the correct value for this data. CAMARADES: Bringing evidence to translational medicine Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) RISK DIFFERENCE The difference between two risks. Risk in the treated group is 0.05 and the risk in the control group is 0.10, so the risk difference is -0.05. No log transformation. CAMARADES: Bringing evidence to translational medicine Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) SUMMARY POINTS We can compute the risk of an event (such as the risk of death) in each group (for example, treated versus control). The ratio of these risks then serves as an effect size (the risk ratio). The difference in these risks can also serve as an effect size (the risk difference). We can compute the odds of an event (such as ratio of dying to living) in each group (for example, treated versus control). The ratio of these odds then serves as the odds ratio. CAMARADES: Bringing evidence to translational medicine Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) SUMMARY POINTS To work with the risk ratio or odds ratio we transform all values to log values, perform the analyses, and then convert the results back to ratio values for presentation. To work with the risk difference we work with the raw values. When choosing an effect size index, consider: The risk and odds ratios are relative measures – relatively insensitive to differences in baseline events. The risk difference is an absolute measure – very sensitive to the baseline risk. CAMARADES: Bringing evidence to translational medicine Chapter 6: Effect Sizes Based on Correlations Computing r Studies report a correlation between two continuous variables. The correlation coefficient can serve as the effect size index. Standardised to take into account the different metrics in original scales. CAMARADES: Bringing evidence to translational medicine Chapter 6: Effect Sizes Based on Correlations CAMARADES: Bringing evidence to translational medicine