Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
EBM WORKSHOP: TREATMENT Oct - 2016 Abdolmehdi Baghaei MD. Resident of Internal Medicine OPTIONAL COMPONENTS PATIENT Values, Preferences Concerns, Expectations Life predicament TO BE ADDED BY THE PHYSICIAN CHARITY EBM is not a required practice (yet) HUMILITY Non-authoritarian practice EBM PHYSICIAN Training Expertise Continued Learning Demand for proof ENTHUSIASM Challenge, Variety, Change INFORMATION Clinically relevant Proven by research Current, up to date THE FIVE BASIC STEPS OF EBM 1. Clinical Question Patient-focused, problem-oriented 2. Find Best Evidence Literary Search 3. Critical Appraisal Evaluate evidence for quality and usefulness 4. Apply the Evidence Implement useful findings in clinical practice 5. Evaluate The information, intervention, and EBM process Objectives: To define characteristics of a suitable report of clinical trials To define the items of paper evaluation in clinical practice To demonstrate peer review check lists for papers that report clinical trials EBM QUESTION: Should include multiple factors (Examples) P PATIENT type of patient or population Ex: 47 yr male w/DM2 and cellulitis toe, 25 yr female w/DVT and chest pain E EXPOSURE environmental, personal, biological Ex: TB, tobacco, drug, diet, pregnancy or menopause, MRSA, allergy I INTERVENTION clinical intervention Ex: medication, procedure, test, surgery, radiation, drug, vaccine C COMPARISON compare alternative treatment Ex: other prior, new or existing therapy O OUTCOME clinical outcome of interest Ex: Reduced death rate in 5 yrs, decreased infections, fewer hospitalizations FRAMING THE QUESTION (Example: PICO) ELEMENT PROMPTS THE QUESTION: Patient Intervention Comparison Outcome How would I describe a group of patients similar to mine? What main action am I considering? What is/are the other options? What do I (or the patient) want to happen (or not happen)? Example: P: In kids under age 12 with poorly controlled asthma on metered dose inhaled steroids… I: would the addition of salmetrol to the current therapy C: compared to increasing the dose of current steroid O: lead to better control of symptoms without increasing side effects? CATEGORY OF QUESTION MAJOR CATEGORIES 1. 2. 3. 4. Diagnosis Prognosis Therapy/ Treatment Harm (iatrogenic, other) MISCELLANEOUS • Quality of care • Health economics • Office Management • Etc. PICO PEO CRITICAL APPRAISAL Interpreting the evidence • How to read a paper • How to do the math CRITICAL APPRAISAL IMPORTANT! You do NOT have to become a researcher, epidemiologist, or statistician to practice EBM. Focus on how to USE research reports – not on how to generate them! Levels of evidence To help clinicians rank quality between evidence sources David Sackett, MD, popularized the evidence-based medicine pyramid. Four levels of evidence: Most desirable at top Components of internal and external validity of controlled clinical trials Internal validity Extent to which systematic error (bias) is minimized in clinical trials Assessment Index How to Evaluate Selection bias Biased allocation to comparison groups Performance bias Unequal provision of care apart from treatment under evaluation Detection bias Biased assessment of outcome Attrition bias Biased occurrence and handling of deviations from protocol and loss to follow up Components of internal and external validity of controlled clinical trials External validity Extent to which results of trials provide a correct basis for generalization to other circumstances. Assessment Index How to Evaluate Patients Age, sex, severity of disease and risk factors, comorbidity Treatment Regimen Dosage and route of administration, type of treatment within a class of treatments, concomitant treatments Setting Level of care (primary to tertiary) and experience and specialization of care provider Modalities of outcomes Type or definition of outcomes and duration of follow up be assessed Quantifying Treatment Effects Rationale Any treatment involves tradeoffs Weigh benefits against risks/costs Benefit $$ Harm Rationale Sometimes the decision is difficult! Benefit $$ Harm Rationale How big is this box? Benefit And this one? $$ Harm Rationale Tests can help us understand who is most likely to benefit from a treatment How big is this box? Benefit And this one? $$ Harm Rationale Tests can help us understand who is most likely to benefit from a treatment Rapid strep to decide who will benefit from penicillin BNP to decide who will benefit from furosemide CRP to decide who will benefit from statins Rationale The utility of a test depends on: How beneficial the treatment is How harmful the treatment is How much the test tells us about these benefits and harms in a given individual Risk of harm from the test itself Rationale The utility of a test depends on: How beneficial the treatment is How harmful the treatment is How much the test tells us about these benefits and harms in a given individual Risk of harm from the test itself The topic for this lecture Is the intervention beneficial? Randomized trials compare an outcome in treated to untreated persons MI in 10% vs. 15% Duration of flu symptoms 3 vs. 5 days Is the intervention beneficial? Randomized trials compare an outcome in treated to untreated persons MI in 10% vs. 15% Duration of flu symptoms 3 vs. 5 days *Statistics* are used to decide if should reject the “null hypothesis” and accept that the intervention is beneficial Is the intervention beneficial? But statistics cannot help us interpret effect size Quantifying the Benefit Effect size How do we summarize and communicate this? What is really important for clinicians and policymakers? Clinical Significance Vs Statistical Significance Quantifying the Benefit Effect size How do we summarize and communicate this? What is really important for clinicians and policymakers? Example: MI in 10% vs. 15% Q: What do we do with these two numbers? Quantifying the Benefit Two simple possibilities: 10% / 15% = 0.66 15% - 10% = 5% Quantifying the Benefit Two simple possibilities: 10% / 15% = 0.66 15% - 10% = 5% Relative Risk (RR) Absolute Risk Reduction (ARR) Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? Medium MI: Big Small 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime Quantifying the Benefit Relative risk as a measure of effect size RR = 0.66 – is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime RR is NOT the best measure of effect size Quantifying the Benefit Absolute risk reduction (ARR) is better ARR = Risk difference = Risk2 – Risk1 Quantifying the Benefit Absolute risk reduction (ARR) is better RR ARR MI: 10% vs. 15% in 10 years .66 5% Death: 50% vs. 75% in 3 years .66 25% Basal Cell CA: 2% vs. 3% in lifetime .66 1% Q: What does the 34% reduction mean? Nimotop® Ad Graph 22% Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% = .66 ARR = 33% - 22% = 11% 33% Nimotop® Ad Graph 22% Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% = .66 ARR = 33% - 22% = 11% 33% What is 34%? Nimotop® Ad Graph 22% Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% = .66 ARR = 33% - 22% = 11% 33% Relative risk reduction (RRR) = 1 – RR = 1-.66 = .34 or 34% Quantifying the Benefit RRR is no better than RR RR RRR MI: 10% vs. 15% in 10 years .66 34% Death: 50% vs. 75% in 3 years .66 34% Basal Cell CA: 2% vs. 3% in lifetime .66 34% Quantifying the Benefit RRR is ALWAYS bigger than ARR (unless untreated risk is 100%) Quantifying the Benefit BEWARE of risk reduction language!! ARR or RRR? “We reduced risk by 34%” “Risk was 34% lower” Quantifying the Benefit BEWARE of risk reduction language!! ARR or RRR? “We reduced risk by 34%” “Risk was 34% lower” can’t tell can’t tell Very hard to be unambiguous! Quantifying the Benefit Another reason that ARR is better: Translate NNT it into “Number Needed to Treat” = 1/ARR Why is NNT = 1/ARR? 100 SAH patients treated 67 no stroke anyway R2 11 strokes prevented R1 22 strokes with Nimotop® 33 strokes with no treatment 22 strokes with treatment Why is NNT 1/ARR? Treat 100 SAH patients prevent 11 strokes Ratio manipulation: 100 treated = 11 prevented 1 treated = 9.1 treated .11 prevented 1 prevented Why is NNT 1/ARR? Treat 100 SAH patients prevent 11 strokes Ratio manipulation: 100 treated = 1 treated = 9.1 treated 11 prevented .11 prevented 1 prevented NNT = 1/ARR Why is NNT 1/ARR? NNT best expressed in a sentence: “Need to treat 9.1 persons with SAH using nimodipine to prevent 1 cerebral infarction” Quantifying the Benefit NNT calculation practice RR ARR NNT? MI: 10% vs. 15% in 10 years .66 5% Death: 50% vs. 75% in 3 years .66 25% Basal Cell CA: 2% vs. 3% in lifetime .66 1% Quantifying the Benefit NNT calculation practice RR ARR NNT? MI: 10% vs. 15% in 10 years .66 5% Death: 50% vs. 75% in 3 years .66 25% Basal Cell CA: 2% vs. 3% in lifetime .66 1% 20 Quantifying the Benefit NNT calculation practice RR ARR NNT? MI: 10% vs. 15% in 10 years .66 5% 20 Death: 50% vs. 75% in 3 years .66 25% 4 Basal Cell CA: 2% vs. 3% in lifetime .66 1% Quantifying the Benefit NNT calculation practice RR ARR NNT? MI: 10% vs. 15% in 10 years .66 5% 20 Death: 50% vs. 75% in 3 years .66 25% 4 Basal Cell CA: 2% vs. 3% in lifetime .66 1% 100 Quantifying the Benefit NNT expression practice RR ARR NNT? Statins MI: 10% vs. 15% in 10 years .66 5% Chemo Death: 50% vs. 75% in 3 years .66 25% 4 Basal Cell CA: 2% vs. 3% in lifetime .66 1% Sunscreen every day 20 100 Quantifying the Benefit NNT expression practice “Need to treat 20 patients with statins for 10 years to prevent 1 MI” “Need to treat 4 patients with chemo for 3 years to prevent 1 death” “Need to treat 100 patients with sunscreen every day for their whole life to prevent 1 basal cell” Population: hypertensive 60-year-olds Outcome: stroke over 5 years Depiction of Results in Control Group Ref: http://www.nntonline.net/ Population: hypertensive 60-year-olds Outcome: stroke over 5 years Depiction of Results in Treatment Group Ref: http://www.nntonline.net/ Example 1 Randomized controlled trial of the effects of hip replacement vs. screws on re-operation in elderly patients with displaced hip fractures. Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. Example 1 Re-operation No Re-operation Hip Replacement 12 217 229 Internal Fixation with Screws 90 136 226 Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155. Example 1 Re-operation No Re-operation Risk Hip Replacement 12 217 229 12/229 = 5.2% Internal Fixation with Screws 90 136 226 90/226 = 39.8% Example 1 Re-operation No Re-operation Risk Hip Replacement 12 217 229 12/229 = 5.2% Internal Fixation with Screws 90 136 226 90/226 = 39.8% RR = R1/R2 = 5.2% / 39.8% = .13 RRR = 1-RR = 1-.13 = 87% ARR = R2 – R1 = 39.8% - 5.2% = 34.6% NNT = 1/ARR = 1/.346 =3 “Need to treat 3 patients with hip replacement instead of screws to prevent 1 from needing a re-do operation” Example 2 JUPITER: Randomized controlled trial of high dose rosuvastatin in patients with LDL<130 and CRP>2.0 Ridker et al. NEJM 2008; 359:2195-207 Example 2 Ridker et al. NEJM 2008; 359:2195-207 Example 2 Ridker et al. NEJM 2008; 359:2195-207 Example 2 HR = (R1/R2) (from regression) = .56 RRR = 1-HR = 1-.56 = 44% ARR = R2 – R1 = 1.36 - 0.77 = .59 / 100py* = .0059 / py NNT = 1/ARR = 1/.0059 = 100/.59 “Need to treat 169 patients for a year to prevent 1 CVD event” Or better: “Need to treat 85 patients for 2 years to prevent 1 CVD event” (average treatment duration in trial was 1.9 years) * py = person-years = 169 pys Example 4 Warfarin vs. placebo for atrial fibrillation Warfarin Placebo Risk of major bleed (/yr) 1.2% 0.7% Ann Intern Med 1999; 131:492-501 Example 4 Warfarin vs. placebo for atrial fibrillation RR = R1/R2 = 1.2% / .7% = 1.7 RR (flipped) = R2/R1 = .7% / 1.2% = .59 RRR (flipped) = 1-RR = 1 - .59 = 41% ARR = .7% - 1.2% = -.5% = R2 – R1 “ARI” – Absolute risk increase = 0.5% NNT = 1/ARR = 1/-.5% = -200 “NNH” – Number needed to harm = -NNT = 1/ARI = 200 “If you treat 200 Afib patients with warfarin, you will cause 1 major bleed” Circling back to test utility… Tests help determine: If the RR applies Treatment for a disease doesn’t help if you don’t have the disease! Interactions (RR is higher or lower than average) Statins more effective if CRP is high? Patients with gene XYZ more likely to have a side effect Baseline risk The higher the risk, the larger the ARR, the smaller the NNT Key Concepts Test utility depends on how good the treatment is RR and p-values good for hypothesis testing/statistics ARR and NNT (and NNH) better for interpreting clinical importance ARR = risk difference NNT = 1/NNT Beware RRR and ambiguous language Where To Start Where To Start Trials are usually published in a set format: abstract; introduction; methods; results and discussion plus a conclusion. Use a check list when you read a RCT (e.g. CONSORT, CASP). But always start by asking two basic question: Where is the trial published? Is the trial sponsored? Critical appraisal starts with a well-formulated question. This typically has four parts and the mnemonic is PICO: Patient Intervention Comparator Outcome. METHODS Is the precise aim of the study described? Dose the section contains of all steps taken to avoid bias? Randomization Blindness Baseline data etc. What were the inclusion and exclusion criteria of the trial? METHODS Was the group size and duration of the study sensible? Is the new drug being compared with the gold standard? Are realistic comparative doses being used? Is the variable selected for measurement really related to the answer being sought? RESULT Statistical tests and analysis (beta error, alpha error, pvalue, etc.) Confidence intervals How are the results expressed? (relative or absolute measures; NNT) Are all patients accounted for? (intention to treat versus pre-protocol analysis) Surrogate markers versus hard clinical end point. Statistical significance versus clinical significance DICUSSION Can the results be applied to the local patient population? What is the authors’ conclusion? What are the main defects of the study? FINAL CONSIDERATION: “STEP acronym” SAFETY TOLERABILITY EFFICACY PRICE Judging the quality of research JAMA articles http://www.cche.net/user Checklists e.g. CASP NHS Critical Appraisal Skills Programme http://www.phru.nhs.uk/casp/appraisa.htm A framework for thinking about the quality of research Intent to Treat Analysis People in a trial still sometimes do what they want instead of what you told them to do Some people assigned to placebo will go to their private doctor and be treated with the experimental drug Some people assigned to the active drug will not take their medicine How do you analyze these people? Intent to Treat Analysis People should be analyzed in the groups that they were assigned to If assigned to placebo, analyze as placebo If assigned to active drug, analyze as active drug, even if you have evidence that they did not take the drug Intent to Treat Analysis: Example Coronary Drug Project Men given clofibrate vs. placebo Outcome was 5-year mortality Intent to Treat Analysis: Example Intent-to-treat analysis of 5 year mortality 20.0% in clofibrate group 20.9% in placebo group On-treatment analysis of the men who took more than 80% of the clofibrate over 5 years 15.0% 5-year mortality in clofibrate compliant Intent to Treat Analysis An individual assigned to a particular intervention group is included in that group’s outcome statistics even if he/she never receives the intervention Preserves the full value of randomization Example: Experimental Designs Coronary Primary Prevention trial, 1985 Treatment groups: men with high serum cholesterol randomized to receive either cholestyramine or an equally foul-tasting placebo Outcome was cardiac death after 7-10 years Grades of Recommendations for a Specified Level of Baseline Risk A1 RCTs, no heterogeneity, CIs all on one side of threshold NNT A2 RCTs, no heterogeneity, CIs overlap threshold NNT B1 RCTs, heterogeneity, CIs all on one side of threshold NNT B2 RCTs, heterogeneity, CIs overalp threshold NNT C1 Observational studies, CIs all on one side of threshold NNT C2 Observational studies, CIs overlap threshold NNT