Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Statistical Analysis MBS-01 Nominal Data • • • • • Dichotomous data Categorizes variables Assigns names, letters or descriptors Gender, race, yes or no No rank or mathematical relationship to each other • Have equivalent weight or value Ordinal Data • • • • Reflects an order to variables or data Does not imply magnitude Zero point is arbitrary Pain Scale 0-10 Continuous Data • Interval or Ratio Data • Can compare absolute magnitude • Implies a numeric relationship • Can undergo arithmetic operations • Data can be averaged and manipulated Measures of Disease Frequency Incidence Rate • Proportion of group initially healthy that will develop a disease within a specified period of time • Expressed in persondays, person-months, etc. • Measures only new cases Prevalence • Proportion of people in the population with disease at a given time • Measures all existing cases • Underestimates acute or rapidly occurring illnesses Relative Risk and Odds Ratio • • • • Measures of association Measures of disease frequency Expressed as a single value Describes the strength of association between the exposure and outcome • Does not imply any extent of variation • Often accompanied by a confidence interval Relative Risk • How many times more likely an outcome is for one group compared with another • Ranges from 0 to infinity – RR of 0 is no association – RR of 1 is risk of acquiring disease same for subjects with and without risk factor – Association is stronger as RR increases (>10 felt to be strong association) – RR = 0.5 then initial risk is cut in half – RR = 2 then initial risk is doubled • Used in cohort (follow-up) design Odds Ratio • Estimator of relative risk • Compares the prevalence of a disease when a specific factor is present or absent • Assumes cases & control gp representative of general population with respect to occurrence of risk factors • Assumes the frequency of disease in exposed or unexposed is small Odds Ratio • Cross-sectional & Case-control design • Uses single value to describe strength of the association between exposure and outcome • OR <1 then risk as decreased • OR = 1 no association between risk factor and disease • OR >1 then risk has increased Number Needed to Treat (NNT) • How many patients must be treated to get one good event (or prevent one bad event) • Applicable to groups of patients with similar underlying risk • Calculated from follow-up and experimental design studies Disease Present F a c t o r Absent Exposed A B Not C Exposed D Relative Risk = A /(A+B) Odds Ratio = A * D C/(C+D) B*C Number Needed to Treat _________1___________ [A/(A+B) ] - [C/(C+D)] Research vs Null Hypothesis Research Hypothesis: • The hypothesis tested by the study • Can be one tailed:a difference in only 1 direction • Can be two tailed:a difference in two directions Null Hypothesis: • Opposite of the research hypothesis • Hypothesis of no difference • Statistics are applied to the null hypothesis Interval and Ratio Data Normal Distribution Nominal and Ordinal Data Nonnormal Distribution Independent Measurements • Seen in parallel design trials • Data do not depend on each other • Data do not reflect serial measurements Measure Treatment Population Sample Outcome Variables X Treatment Measure Outcome Variables Dependent Measurements • Seen in cross-over design studies • Seen in studies using matched groups • Data depend on or reflect each other Sample X Treatment Measure Outcome Measure Outcome Measure Outcome Statistical Analysis Data Type Samples are: 2 Indep 2 Related 3 or > Indep 3 or > Related Measure of Correlation Nominal Chi-square McNemar Chi square Cochran Q Contingency coefficient Ordinal MannWhitney U or Wilcoxon Rank Sum Sign test or Wilcoxon Signed Rank Test KruskalWallis oneway analysis of variance (ANOVA) Friedman 2 way analysis of variance (ANOVA) Spearman rank correl coefficient Kendel rank correlation coefficient Interval Student’s t test MannWhitney U test Paired t test One-way analysis of variance (ANOVA) 2 way (repeated measures) analysis of variance (ANOVA) Pearson Correlation Coefficient Errors Related to Hypothesis Research Conclusion True Situation: Null Hypothesis is: True False Do Not Reject H0 (No Difference) Correct Decision Confidence Level Probability =1- Error: Type II Probability= Reject H0 (Difference) Error: Type I Significance Level Probability = Correct Decision Power of test Probability =1- Alpha • The probability of making a Type I error • Predetermined by the investigator • Usual values 0.05 or 0.1 (1 in 10 or 1 in 20 chance of Type I error) • P value: the numeric representation of Beta and Power • Power is the probability of avoiding a Type II error. • Or the chance of finding a difference if it truly exists • Power is 1- • Increase power by increasing n, increasing , or increasing the size of difference accepted Summary • Statistics tell us about the role that sampling variability plays in results • Statistics make no claim about the validity of a study • Consider the impact of Type I and II Errors • Results May be Statistically Significant but Clinically irrelevant. Error, Validity and Precision Random Error • Not constant error • Due to chance • Unknown sources of variation equally likely to affect findings in either direction • Seen as inconsistency in repeated or equivalent measurements when made on the same object or person • Increase sample size to reduce random error Systematic Error • Constant error • Due to bias • Sources of variation that affect findings in one direction • Improve study design to reduce • Investigator should include explanation of systematic error in publication • Change of sample size will not affect systematic error Reliability vs Validity of Data • Reliability: reproducibility of measurement • Validity: extent to which differences in scores reflect the true differences among individuals on the characteristic we are seeking to measure Validity Study Results Internal Validity Truth in Study Results External Validity Errors of chance and bias Truth in the Universe Threats to Internal Validity • History: naturally occurring event external to study but occurring simultaneously • Maturation: change in the study subject occurring as a function of the passage of time • Instrumentation: changes or errors in the measuring instrument or observer Threats to Internal Validity • Selection: the way in which the subjects were selected and assigned to treatment groups • Experimental Mortality: dropout, nonresponse, or death • Main testing effect: being tested may bring about a change in behavior on a second observation • Statistical regression: tendency of extremes to move toward the mean during an experiment Threats to Internal Validity Hx Mat Ins Sel Mor One-Shot Case Study - - - - - One-Gp Pre & Post Tx - - - +/- + Static Group Comp + - + - - Pre & Post Tx Control Gp + + + + + Post Tx only Control Group Design + + + + - Nonequivalent Pre-post Tx Control Group Design + + + +/- + Threats to External Validity • Interaction of Subject Variables and Tx: tx has varied effects on subgroups • Interactive Effect of Testing: pretesting may sensitize subjects to variable • Reactive Effects of Experimental Arrangements: study setting may be atypical • Multiple Treatment Interferences: same subject given several treatments; effects of earlier treatments not completely erasable • Hawthorne Effect: volunteers try to give “right” answer Measures of Central Tendency • Mean: average • Median: midpoint where ½ of observations fall above and ½ fall below the value • Mode: most frequently encountered number • In a normal, or Gaussian, distribution the mean, median, and mode are identical Precision • How closely the estimates will tend to cluster about the true value • Larger the standard deviation or standard error of the mean the less precise the data Normality 6 Mean, median, & mode Patients 5 4 3 2 1 0 0 1 2 3 4 5 6 Pain Score 7 8 9 10 Skewed Distribution 7 Mode 6 Median Mean Patients 5 4 3 2 1 0 1 2 3 4 5 6 Pain Score 7 8 9 10 Measures of Variability Standard Deviation • Measures how close the values cluster to the sample mean • Interval Data • Square root of variance • Reported as +/• If normal distribution 1 S.D. = 68% data 2 S.D. = 95% data 3 S.D. = 99% data Standard Error of the Mean • Estimates mean of population from sample meanˉх • Equals S.D./ square root of n • Smaller number than SD therefore often reported as measure of dispersion Confidence Intervals • Further defines the p value by giving a range of values to describe the data • An interval that will, with the probability of a confidence level, contain the true difference being investigated • A confidence interval which includes “0” does not permit rejection of the null Sample Size, MetaAnalysis & Evidence Based Medicine MBS-01 Sample Size • Determined before initiation of study • Re-evaluated at conclusion of study due tp dropouts or deaths • Often not included in publication • Nomograms, formulas, tables are available to assist reader with sample size calculation • Studies of inadequate sample size: pilot studies Sample Size Determinants • • • • • The outcome being evaluated Tolerable risk of error (α and β) Clinically important difference () Variability of measurement (s and s2) Ratio of experimental to control subjects Sample Size Determinants • The outcome being evaluated – Dichotomous outcome – Continuous outcome • • • • Tolerable risk of error (α and β) Clinically important difference () Variability of measurement (s and s2) Ratio of experimental to control subjects Sample Size Determinants • The outcome being evaluated • Tolerable risk of error (α and β) – Type I (α) • concludes there is a difference when in fact there is not • convention sets risk at 1-in-20 chance or an α of 0.05 – Type II (β) • concludes there is no difference when in fact one exists • Convention sets risk at 2-in-10 chance or a β of 0.2 • Clinically important difference () • Variability of measurement (s and s2) • Ratio of experimental to control subjects Sample Size Determinants • The outcome being evaluated • Tolerable risk of error (α and β) • Difference between experimental and control group () – What is clinically important – Detecting a small difference between groups requires larger sample size • Variability of measurement (s and s2) • Ratio of experimental to control subjects Sample Size Determinants • • • • The outcome being evaluated Tolerable risk of error (α and β) Clinically important difference () Variability of measurement (s and s2) – Expressed as s for continuous variables – Not specified for dichotomous variables – Smaller spread around mean requires smaller sample size • Ratio of experimental to control subjects Sample Size Determinants • • • • • The outcome being evaluated Tolerable risk of error (α and β) Clinically important difference () Variability of measurement (s and s2) Ratio of experimental to control subjects – One-to-one ratio minimizes sample size – Assumed to be one-to-one in Young Nomogram Sample Size • Equations & nomograms differ for type of outcome measured; nominal vs continuous • Z (Standard Normal) distribution is used for alpha and beta – Mean of 0 – S.D. of 1 • Can solve for any portion of equations if other factors are known • Plan for drop-out Sample Size Equation for Dichotomous Variables zα/2 2P(1 P) zβ P1(1 P1) P0(1 P0) n P1 P0 2 P: proportion of responders in both groups P1: proportion of responders in experimental group P0: proportion of responders in control groups Use 2 sided alpha of 0.05 Zα =1.96 Use one-sided beta of 0.20 Zβ = 0.84 Sample Size Equation for Continuous Variables 2σ (zα/2 zβ) n 2 δ 2 Use 2 sided alpha of 0.05 2 Zα =1.96 Use one-sided beta of 0.20 Zβ = 0.84 : difference in means for control & experimental group σ: weighted average standard deviation in the control group Nomograms for Calculation of Sample Size • Assumes parallel study (no crossover) with two groups • Assumes 2 tailed alpha; if not, overestimates needed sample size • Assumes alpha 0.05 and beta 0.2 • Approximation used retrospectively to critique another author’s conclusions Number of Patients Nomogram MBS-01 Sample Size and Study Outcome • Difference Observed – n is large enough to identify difference – Difference may be overestimated if n is smaller than would have been determined using sample size calculations – Extremely large n may find difference but clinical significance of the difference lacking • No difference Observed – n may be too small – Truly may be no difference Meta-Analysis A statistical analysis which combines or integrates the results of several independent clinical trials considered by the analyst to be combinable. Goal is to discern a more objective and generalized answer to a particular question or clinical dilemma by combining results from different studies that have similar research hypotheses. The Meta-Analysis • Tertiary literature • Applied to experimental and observational designs • Useful when previous studies inconclusive, contradicting, or sample size is too small • Can be used to increase power of a study • Exercise caution and skepticism in evaluating the conclusion! MBS-01 Quality Areas of Meta-Analysis • • • • • • Study design Ability of data to be combined Control of bias Statistical analysis Sensitivity analysis Application of results Meta-Analysis: Study Design • Must clearly define focused clinical question • Provide details of literature searches • Searches should be comprehensive – Published and unpublished data – References within publications – Foreign language literature • Blind selection of articles by >1 reviewer Meta-Analysis: Selection • Identify how articles selected for inclusion prior to project initiation • Identify variables pooled to answer question prior to project initiation • Identify quality of articles to be selected prior to project initiation • Weighting of articles? Meta-Analysis: Combinability • Determine range of variables to be included – Identify data at identical timepoints – Identify baseline and therapeutic intervention data to be included • Evaluate homogeneity of the outcome variables Meta-Analysis: Bias • • • • • • • • Publication bias Language bias Reference bias Selection bias Duplicate publication bias Data extraction bias Support bias Bias within clinical trial Meta-Analysis:Application of Results to Practice • • • • Relative Risk Ratio Absolute Risk Reduction Number Needed to Treat Consideration to trials not included Evidence Based Medicine Explicit and judicious use of current best evidence in making decisions about the care of individual patients Evidence Based Medicine • Convert information needs into a clearly defined answerable clinical question • Conduct a systematic search for the best available evidence for the problem • Evaluate validity & applicability of evidence • Prepare a synthesis or summary of the evidence for decision making and implement the decision in practice • Evaluate performance & follow-up on any areas for improvement