Download treatment

EBM WORKSHOP:
TREATMENT
Oct - 2016
Abdolmehdi Baghaei MD.
Resident of Internal Medicine
OPTIONAL
COMPONENTS
PATIENT
Values, Preferences
Concerns, Expectations
Life predicament
TO BE ADDED BY
THE PHYSICIAN
CHARITY
EBM is not a
required practice
(yet)
HUMILITY
Non-authoritarian
practice
EBM
PHYSICIAN
Training
Expertise
Continued Learning
Demand for proof
ENTHUSIASM
Challenge, Variety,
Change
INFORMATION
Clinically relevant
Proven by research
Current, up to date
THE FIVE BASIC STEPS OF EBM
1. Clinical Question
Patient-focused, problem-oriented
2. Find Best Evidence
Literary Search
3. Critical Appraisal
Evaluate evidence for quality and usefulness
4. Apply the Evidence
Implement useful findings in clinical practice
5. Evaluate
The information, intervention, and EBM process
Objectives:



To define characteristics of a suitable report of
clinical trials
To define the items of paper evaluation in clinical
practice
To demonstrate peer review check lists for papers
that report clinical trials
EBM QUESTION: Should include multiple factors
(Examples)
P
PATIENT
type of patient or population
Ex: 47 yr male w/DM2 and cellulitis toe, 25 yr female w/DVT and chest pain
E
EXPOSURE
environmental, personal, biological
Ex: TB, tobacco, drug, diet, pregnancy or menopause, MRSA, allergy
I
INTERVENTION
clinical intervention
Ex: medication, procedure, test, surgery, radiation, drug, vaccine
C
COMPARISON compare alternative treatment
Ex: other prior, new or existing therapy
O
OUTCOME
clinical outcome of interest
Ex: Reduced death rate in 5 yrs, decreased infections, fewer
hospitalizations
FRAMING THE QUESTION (Example: PICO)
ELEMENT
PROMPTS THE QUESTION:
Patient
Intervention
Comparison
Outcome
How would I describe a group of patients similar to mine?
What main action am I considering?
What is/are the other options?
What do I (or the patient) want to happen (or not happen)?
Example:
P:
In kids under age 12 with poorly controlled asthma on metered
dose inhaled steroids…
I: would the addition of salmetrol to the current therapy
C: compared to increasing the dose of current steroid
O: lead to better control of symptoms without increasing side effects?
CATEGORY OF QUESTION
MAJOR CATEGORIES
1.
2.
3.
4.
Diagnosis
Prognosis
Therapy/ Treatment
Harm (iatrogenic, other)
MISCELLANEOUS
• Quality of care
• Health economics
• Office Management
• Etc.
PICO
PEO
CRITICAL APPRAISAL
Interpreting the evidence
• How to read a paper
• How to do the math
CRITICAL APPRAISAL
IMPORTANT!
You do NOT have to become a researcher,
epidemiologist, or statistician to practice EBM.
Focus on how to USE research
reports – not on how to generate
them!
Levels of evidence
To help clinicians rank quality between evidence sources
David Sackett, MD, popularized the evidence-based
medicine pyramid.
Four levels of evidence:
Most desirable at top
Components of internal and external validity
of controlled clinical trials
Internal validity
Extent to which systematic error (bias) is minimized in
clinical trials
Assessment Index
How to Evaluate
Selection bias
Biased allocation to comparison groups
Performance bias
Unequal provision of care apart from treatment under
evaluation
Detection bias
Biased assessment of outcome
Attrition bias
Biased occurrence and handling of deviations from
protocol and loss to follow up
Components of internal and external validity
of controlled clinical trials
External validity
Extent to which results of trials provide a correct basis for
generalization to other circumstances.
Assessment Index
How to Evaluate
Patients
Age, sex, severity of disease and risk factors, comorbidity
Treatment Regimen
Dosage and route of administration, type of treatment within a
class of treatments, concomitant treatments
Setting
Level of care (primary to tertiary) and experience and
specialization of care provider
Modalities of
outcomes
Type or definition of outcomes and duration of follow up be
assessed
Quantifying Treatment
Effects
Rationale

Any treatment involves tradeoffs
 Weigh
benefits against risks/costs
Benefit
$$
Harm
Rationale

Sometimes the decision is difficult!
Benefit
$$
Harm
Rationale
How big is this box?
Benefit
And this one?
$$
Harm
Rationale
Tests can help us understand who is most likely to
benefit from a treatment
How big is this box?
Benefit
And this one?
$$
Harm

Rationale

Tests can help us understand who is most likely to
benefit from a treatment
 Rapid
strep to decide who will benefit from penicillin
 BNP to decide who will benefit from furosemide
 CRP to decide who will benefit from statins
Rationale

The utility of a test depends on:
 How
beneficial the treatment is
 How harmful the treatment is
 How much the test tells us about these benefits and
harms in a given individual
 Risk of harm from the test itself
Rationale

The utility of a test depends on:
 How
beneficial the treatment is
 How harmful the treatment is
 How much the test tells us about these benefits and
harms in a given individual
 Risk of harm from the test itself
The topic for this lecture
Is the intervention beneficial?

Randomized trials compare an outcome in treated
to untreated persons
 MI
in 10% vs. 15%
 Duration of flu symptoms 3 vs. 5 days
Is the intervention beneficial?

Randomized trials compare an outcome in
treated to untreated persons
 MI
in 10% vs. 15%
 Duration of flu symptoms 3 vs. 5 days

*Statistics* are used to decide if should reject the
“null hypothesis” and accept that the intervention
is beneficial
Is the intervention beneficial?

But statistics cannot help us interpret effect size
Quantifying the Benefit

Effect size
How do we summarize and communicate this?
 What is really important for clinicians and policymakers?

Clinical Significance
Vs
Statistical Significance
Quantifying the Benefit

Effect size
How do we summarize and communicate this?
 What is really important for clinicians and policymakers?



Example: MI in 10% vs. 15%
Q: What do we do with these two numbers?
Quantifying the Benefit

Two simple possibilities:
 10%
/ 15% = 0.66
 15% - 10% = 5%
Quantifying the Benefit

Two simple possibilities:
 10%
/ 15% = 0.66
 15% - 10% = 5%
Relative Risk (RR)
Absolute Risk Reduction (ARR)
Quantifying the Benefit

Relative risk as a measure of effect size
 RR
= 0.66 – is this big or small?
Quantifying the Benefit

Relative risk as a measure of effect size
 RR
= 0.66 – is this big or small?
 MI:
10% vs. 15% in 10 years
 Death:
50% vs. 75% in 3 years
 Basal Cell CA: 2% vs. 3% in lifetime
Quantifying the Benefit

Relative risk as a measure of effect size
 RR
= 0.66 – is this big or small?
Medium  MI:
Big
Small
10% vs. 15% in 10 years
 Death:
50% vs. 75% in 3 years
 Basal Cell CA: 2% vs. 3% in lifetime
Quantifying the Benefit

Relative risk as a measure of effect size
 RR
= 0.66 – is this big or small?
 MI:
10% vs. 15% in 10 years
 Death:
50% vs. 75% in 3 years
 Basal Cell CA: 2% vs. 3% in lifetime
 RR
is NOT the best measure of effect size
Quantifying the Benefit

Absolute risk reduction (ARR) is better
 ARR
= Risk difference = Risk2 – Risk1
Quantifying the Benefit

Absolute risk reduction (ARR) is better
RR
ARR
MI:
10% vs. 15% in 10 years
.66
5%
Death:
50% vs. 75% in 3 years
.66
25%
Basal Cell CA:
2% vs. 3% in lifetime
.66
1%
Q: What does the 34% reduction mean?
Nimotop® Ad Graph
22%




Risk1 = 61/278 = 21.8%
Risk2 = 92/276 = 33%
RR = 22%/33% = .66
ARR = 33% - 22% = 11%
33%
Nimotop® Ad Graph
22%




Risk1 = 61/278 = 21.8%
Risk2 = 92/276 = 33%
RR = 22%/33% = .66
ARR = 33% - 22% = 11%
33%
What is 34%?
Nimotop® Ad Graph
22%




Risk1 = 61/278 = 21.8%
Risk2 = 92/276 = 33%
RR = 22%/33% = .66
ARR = 33% - 22% = 11%
33%
Relative risk reduction (RRR) =
1 – RR = 1-.66 = .34 or 34%
Quantifying the Benefit

RRR is no better than RR
RR
RRR
MI:
10% vs. 15% in 10 years
.66
34%
Death:
50% vs. 75% in 3 years
.66
34%
Basal Cell CA:
2% vs. 3% in lifetime
.66
34%
Quantifying the Benefit

RRR is ALWAYS bigger than ARR
 (unless
untreated risk is 100%)
Quantifying the Benefit

BEWARE of risk reduction language!!

ARR or RRR?
 “We
reduced risk by 34%”
 “Risk was 34% lower”
Quantifying the Benefit

BEWARE of risk reduction language!!

ARR or RRR?
 “We
reduced risk by 34%”
 “Risk was 34% lower”

can’t tell
can’t tell
Very hard to be unambiguous!
Quantifying the Benefit

Another reason that ARR is better:
 Translate
 NNT
it into “Number Needed to Treat”
= 1/ARR
Why is NNT = 1/ARR?
100 SAH patients
treated
67 no stroke anyway
R2
11 strokes prevented
R1
22 strokes with Nimotop®
33 strokes with
no treatment
22 strokes with
treatment
Why is NNT 1/ARR?
Treat 100 SAH patients  prevent 11 strokes
Ratio manipulation:
100 treated =
11 prevented
1 treated
= 9.1 treated
.11 prevented
1 prevented
Why is NNT 1/ARR?
Treat 100 SAH patients  prevent 11 strokes
Ratio manipulation:
100 treated = 1 treated
= 9.1 treated
11 prevented
.11 prevented
1 prevented
NNT
=
1/ARR
Why is NNT 1/ARR?
NNT best expressed in a sentence:
“Need to treat 9.1 persons with SAH using nimodipine
to prevent 1 cerebral infarction”
Quantifying the Benefit

NNT calculation practice
RR
ARR NNT?
MI:
10% vs. 15% in 10 years
.66
5%
Death:
50% vs. 75% in 3 years
.66
25%
Basal Cell CA:
2% vs. 3% in lifetime
.66
1%
Quantifying the Benefit

NNT calculation practice
RR
ARR NNT?
MI:
10% vs. 15% in 10 years
.66
5%
Death:
50% vs. 75% in 3 years
.66
25%
Basal Cell CA:
2% vs. 3% in lifetime
.66
1%
20
Quantifying the Benefit

NNT calculation practice
RR
ARR NNT?
MI:
10% vs. 15% in 10 years
.66
5%
20
Death:
50% vs. 75% in 3 years
.66
25% 4
Basal Cell CA:
2% vs. 3% in lifetime
.66
1%
Quantifying the Benefit

NNT calculation practice
RR
ARR NNT?
MI:
10% vs. 15% in 10 years
.66
5%
20
Death:
50% vs. 75% in 3 years
.66
25% 4
Basal Cell CA:
2% vs. 3% in lifetime
.66
1%
100
Quantifying the Benefit

NNT expression practice
RR
ARR NNT?
Statins
MI:
10% vs. 15% in 10 years
.66
5%
Chemo
Death:
50% vs. 75% in 3 years
.66
25% 4
Basal Cell CA:
2% vs. 3% in lifetime
.66
1%
Sunscreen
every day
20
100
Quantifying the Benefit

NNT expression practice
“Need to treat 20 patients with statins for 10 years to prevent 1 MI”
“Need to treat 4 patients with chemo for 3 years to prevent 1 death”
“Need to treat 100 patients with sunscreen every day for their whole life to prevent 1
basal cell”
Population: hypertensive 60-year-olds
Outcome: stroke over 5 years
Depiction of Results in Control Group
Ref: http://www.nntonline.net/
Population: hypertensive 60-year-olds
Outcome: stroke over 5 years
Depiction of Results in Treatment Group
Ref: http://www.nntonline.net/
Example 1

Randomized controlled trial of the effects of hip
replacement vs. screws on re-operation in elderly
patients with displaced hip fractures.
Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.
Example 1
Re-operation
No Re-operation
Hip Replacement
12
217
229
Internal Fixation
with Screws
90
136
226
Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.
Example 1
Re-operation
No Re-operation
Risk
Hip Replacement
12
217
229
12/229 = 5.2%
Internal Fixation
with Screws
90
136
226
90/226 = 39.8%
Example 1
Re-operation
No Re-operation
Risk
Hip Replacement
12
217
229
12/229 = 5.2%
Internal Fixation
with Screws
90
136
226
90/226 = 39.8%
RR
= R1/R2
= 5.2% / 39.8%
= .13
RRR
= 1-RR
= 1-.13
= 87%
ARR
= R2 – R1
= 39.8% - 5.2%
= 34.6%
NNT
= 1/ARR
= 1/.346
=3
“Need to treat 3 patients with hip replacement instead of screws to prevent 1
from needing a re-do operation”
Example 2

JUPITER: Randomized controlled trial of high dose
rosuvastatin in patients with LDL<130 and CRP>2.0
Ridker et al. NEJM 2008; 359:2195-207
Example 2
Ridker et al. NEJM 2008; 359:2195-207
Example 2
Ridker et al. NEJM 2008; 359:2195-207
Example 2
HR
= (R1/R2)
(from regression)
= .56
RRR
= 1-HR
= 1-.56
= 44%
ARR
= R2 – R1
= 1.36 - 0.77
= .59 / 100py*
= .0059 / py
NNT
= 1/ARR
= 1/.0059 = 100/.59
“Need to treat 169 patients for a year to prevent 1 CVD event”
Or better:
“Need to treat 85 patients for 2 years to prevent 1 CVD event”
(average treatment duration in trial was 1.9 years)
* py = person-years
= 169 pys
Example 4
Warfarin vs. placebo for atrial fibrillation
Warfarin Placebo
Risk of major bleed (/yr)
1.2%
0.7%
Ann Intern Med 1999; 131:492-501
Example 4
Warfarin vs. placebo for atrial fibrillation
RR
= R1/R2
= 1.2% / .7%
= 1.7
RR (flipped) = R2/R1
= .7% / 1.2%
= .59
RRR (flipped) = 1-RR
= 1 - .59
= 41%
ARR
= .7% - 1.2%
= -.5%
= R2 – R1
“ARI” – Absolute risk increase = 0.5%
NNT
= 1/ARR
= 1/-.5%
= -200
“NNH” – Number needed to harm = -NNT = 1/ARI = 200
“If you treat 200 Afib patients with warfarin, you will cause 1 major bleed”
Circling back to test utility…

Tests help determine:

If the RR applies


Treatment for a disease doesn’t help if you don’t have the disease!
Interactions (RR is higher or lower than average)



Statins more effective if CRP is high?
Patients with gene XYZ more likely to have a side effect
Baseline risk

The higher the risk, the larger the ARR, the smaller the NNT
Key Concepts



Test utility depends on how good the treatment is
RR and p-values good for hypothesis testing/statistics
ARR and NNT (and NNH) better for interpreting
clinical importance
ARR = risk difference
 NNT = 1/NNT


Beware RRR and ambiguous language
Where To Start
Where To Start
Trials are usually published in a set format: abstract;
introduction; methods; results and discussion plus a
conclusion.
Use a check list when you read a RCT (e.g. CONSORT,
CASP). But always start by asking two basic
question:
 Where
is the trial published?
 Is the trial sponsored?
Critical appraisal starts with a well-formulated
question. This typically has four parts and the
mnemonic is PICO:
Patient
Intervention
Comparator
Outcome.
METHODS


Is the precise aim of the study described?
Dose the section contains of all steps taken to avoid
bias?
 Randomization
 Blindness
 Baseline
data
 etc.

What were the inclusion and exclusion criteria of the
trial?
METHODS




Was the group size and duration of the study
sensible?
Is the new drug being compared with the gold
standard?
Are realistic comparative doses being used?
Is the variable selected for measurement really
related to the answer being sought?
RESULT






Statistical tests and analysis (beta error, alpha error, pvalue, etc.)
Confidence intervals
How are the results expressed? (relative or absolute
measures; NNT)
Are all patients accounted for? (intention to treat
versus pre-protocol analysis)
Surrogate markers versus hard clinical end point.
Statistical significance versus clinical significance
DICUSSION



Can the results be applied to the local patient
population?
What is the authors’ conclusion?
What are the main defects of the study?
FINAL CONSIDERATION:
“STEP acronym”




SAFETY
TOLERABILITY
EFFICACY
PRICE
Judging the quality of research
JAMA articles

http://www.cche.net/user
Checklists e.g. CASP

NHS Critical Appraisal Skills Programme
http://www.phru.nhs.uk/casp/appraisa.htm
A framework for thinking about the quality of research
Intent to Treat Analysis

People in a trial still sometimes do what they want
instead of what you told them to do
 Some
people assigned to placebo will go to their
private doctor and be treated with the experimental
drug
 Some people assigned to the active drug will not take
their medicine

How do you analyze these people?
Intent to Treat Analysis

People should be analyzed in the groups that they
were assigned to
 If
assigned to placebo, analyze as placebo
 If assigned to active drug, analyze as active drug,
even if you have evidence that they did not take the
drug
Intent to Treat Analysis: Example



Coronary Drug Project
Men given clofibrate vs. placebo
Outcome was 5-year mortality
Intent to Treat Analysis: Example

Intent-to-treat analysis of 5 year mortality
 20.0%
in clofibrate group
 20.9% in placebo group

On-treatment analysis of the men who took more
than 80% of the clofibrate over 5 years
 15.0%
5-year mortality in clofibrate compliant
Intent to Treat Analysis


An individual assigned to a particular
intervention group is included in that group’s
outcome statistics even if he/she never receives
the intervention
Preserves the full value of randomization
Example: Experimental Designs



Coronary Primary Prevention trial, 1985
Treatment groups: men with high serum cholesterol
randomized to receive either cholestyramine or an
equally foul-tasting placebo
Outcome was cardiac death after 7-10 years
Grades of Recommendations for a Specified Level of
Baseline Risk
A1
RCTs, no heterogeneity, CIs all on one side of threshold NNT
A2
RCTs, no heterogeneity, CIs overlap threshold NNT
B1
RCTs, heterogeneity, CIs all on one side of threshold NNT
B2
RCTs, heterogeneity, CIs overalp threshold NNT
C1
Observational studies, CIs all on one side of threshold NNT
C2
Observational studies, CIs overlap threshold NNT