Download 2030Lecture12 - U of L Class Index

Introduction to Biomedical
Statistics
Signal Detection Theory
• What do we actually “detect” when we
say we’ve detected something?
Signal Detection Theory
• What do we actually “detect” when we
say we’ve detected something?
• We say we’ve “detected” when a
criterion value exceeds a threshold
Signal Detection Theory
• examples:
– the onset of a light or sound
– the presence of an abnormality on x-ray
Signal Detection Theory
• There are 4 possible situations
Present
Absent
You Respond:
Target is:
Present
Absent
Signal Detection Theory
• There are 4 possible situations
Present
Absent
You Respond:
Target is:
Present
Absent
Hit
Signal Detection Theory
• There are 4 possible situations
Present
Hit
Absent
You Respond:
Target is:
Present
Absent
Miss
Signal Detection Theory
• There are 4 possible situations
Present
Hit
Absent
You Respond:
Target is:
Present
Absent
Miss
False
Alarm
Signal Detection Theory
• There are 4 possible situations
Present
Hit
False
Alarm
Absent
You Respond:
Target is:
Present
Absent
Miss
Correct
Rejection
Signal Detection Theory
• There are 4 possible situations
Present
Hit
False
Alarm
Absent
You Respond:
Target is:
Present
Absent
Miss
Correct
Rejection
This is the total # of Target Present Trials
Signal Detection Theory
• There are 4 possible situations
Present
Hit
False
Alarm
Absent
You Respond:
Target is:
Present
Absent
Miss
Correct
Rejection
This is the total # of Target Present Trials
Signal Detection Theory
• Hit Rate (H) is the proportion of target
present trials on which you respond
“present”
H

# Hits
# Hits # Misses
Signal Detection Theory
• Notice that H is a proportion, so 1 - H
gives you the “miss” rate or…
MissRate 

# Misses
# Hits # Misses
Signal Detection Theory
• False-Alarm Rate (FA) is the proportion
of target absent trials on which you
respond “present”
FA 

# FalseAlarms
# FalseAlarms # CorrectRejections
Signal Detection Theory
• Notice that FA is a proportion. 1 minus
FA gives you the correct rejections or …
CorrectRejectionRate 

# CorrectRejections
# FalseAlarms # CorrectRejections
Signal Detection Theory
• Signal Detection can be modeled as
signal + noise with some detection
threshold
Noise is normally
distributed - Target
Absent trials still contain
some stimulus
Frequency
Signal Detection Theory
Target Absent
Stimulus Intensity
Noise is normally
distributed - Target
Absent trials still contain
some stimulus
Frequency
Signal Detection Theory
Target Present trials
contain a little bit extra
intensity contributed by
the signal
Target Present
Target Absent
Stimulus Intensity
Signal Detection Theory
Noise is normally
distributed - Target
Absent trials still contain
some stimulus
Frequency
This is the signal’s contribution
Target Present trials
contain a little bit extra
intensity contributed by
the signal
Target Present
Target Absent
Stimulus Intensity
Signal Detection Theory
Frequency
• We can imagine a static criterion above which we’ll
respond “target is present”
Criterion
Target Present
Target Absent
Stimulus Intensity
Signal Detection Theory
Frequency
• Notice that H, FA, etc thus have graphical meanings
Criterion
Proportion Hits
Stimulus Intensity
Signal Detection Theory
Frequency
• Notice that H, FA, etc thus have graphical meanings
Criterion
Proportion
Misses
Stimulus Intensity
Signal Detection Theory
Frequency
• Notice that H, FA, etc thus have graphical meanings
Criterion
Proportion
False
Alarms
Stimulus Intensity
Signal Detection Theory
Frequency
• Notice that H, FA, etc thus have graphical meanings
Criterion
Proportion
Correct
Rejections
Stimulus Intensity
Signal Detection Theory
Frequency
• Notice that as H increases, FA also increases
Criterion
H
Stimulus Intensity
Signal Detection Theory
Frequency
• Notice that as H increases, FA also increases
Criterion
FA
Stimulus Intensity
Signal Detection Theory
•
d’ (pronounced d prime) is a measure of sensitivity to detect a signal
from noise and does not depend on criterion - it is the distance between
the peaks of the signal present and signal absent curves
d’ is computed by
converting from H and FA
proportions into their
corresponding Z scores
and subtracting Zinv(FA)
from Zinv(H)
Frequency
•
Stimulus Intensity
Some Common Biomedical
Statistics
•
•
•
•
•
•
•
•
Sensitivity
Specificity
Positive Predictive Value
Negative Predictive Value
Likelihood Ratio
Relative Risk
Absolute Risk
Number needed to treat/harm
Sensitivity and Specificity
• Consider a test for a condition
– e.g. Pregnancy test
– e.g. Prostate-specific Antigen (Prostate
Cancer)
– e.g. Ultrasound (Breast Cancer)
• These are all signal detection problems
Sensitivity and Specificity
• Four possible situations:
Present
Absent
Test Result:
Condition is:
Present
Absent
Sensitivity and Specificity
• Four possible situations:
Present
Absent
Test Result:
Condition is:
Present
Absent
True
Positive
Sensitivity and Specificity
• Four possible situations:
Present
True
Positive
Absent
Test Result:
Condition is:
Present
Absent
False
Negative
Sensitivity and Specificity
• Four possible situations:
Present
True
Positive
Absent
Test Result:
Condition is:
Present
Absent
False
Negative
False
Positive
Sensitivity and Specificity
• Four possible situations:
Present
True
Positive
False
Positive
Absent
Test Result:
Condition is:
Present
Absent
False
Negative
True
Negative
Sensitivity and Specificity
• Four possible situations:
Present
True
Positive
False
Positive
Absent
Test Result:
Condition is:
Present
Absent
False
Negative
True
Negative
This is Total # of Condition Present Cases
Sensitivity and Specificity
• Four possible situations:
Present
True
Positive
False
Positive
Absent
Test Result:
Condition is:
Present
Absent
False
Negative
True
Negative
This is Total # of Condition Absent Cases
Sensitivity and Specificity
• Four possible situations:
Present
True
Positive
False
Positive
Absent
Test Result:
Condition is:
Present
Absent
False
Negative
True
Negative
This is Total # of
“positive” tests
Sensitivity and Specificity
• Four possible situations:
Present
True
Positive
False
Positive
Absent
Test Result:
Condition is:
Present
Absent
False
Negative
True
Negative
This is Total # of
“negative” tests
Sensitivity and Specificity
• Sensitivity is the proportion of condition present
cases on which the test returned “positive”
• Analogous to the hit rate (H) in Signal Detection
Theory
Sensivity 

# True Positives
# True Postives + # False Negatives
Sensitivity and Specificity
• Specificity is the proportion of condition absent cases
on which the test returned “negative”
• Analogous to the Correct Rejection rate in Signal
Detection Theory
Specificity 

# True Negative
# True Negative + # False Positive
Sensitivity and Specificity
• Notice that 1 minus the Sensitivity is analogous to the
FA of Signal Detection Theory
Sensitivity and Specificity
• Notice that 1 minus the Sensitivity is analagous to the
FA of Signal Detection Theory
• Recall that in Signal Detection Theory, as criterion
were relaxed both H and FA increased and as
criterion were more stringent, H and FA decreased
Sensitivity and Specificity
• Notice that 1 minus the Sensitivity is analagous to the
FA of Signal Detection Theory
• Recall that in Signal Detection Theory, as criterion
were relaxed both H and FA increased and as
criterion were more stringent, H and FA decreased
• Sensitivity and Specificity have a similar relationship:
as a cut-off value for a test becomes more stringent
the sensitivity goes down and the specificity goes
up…and vice versa
Sensitivity and Specificity
• “For detecting any prostate cancer, PSA
cutoff values of 1.1, 2.1, 3.1, and 4.1 ng/mL
yielded sensitivities of 83.4%, 52.6%, 32.2%,
and 20.5%, and specificities of 38.9%, 72.5%,
86.7%, and 93.8%, respectively.”
JAMA. 2005 Jul 6;294(1):66-70.
Sensitivity and Specificity
• Likelihood Ratio is the ratio of True Positive
rate to False Positive rate
Likelihood Ratio 

Sensitivity
1 - Specificity
• Loosely corresponds to d’ in that Likelihood
ratio is insensitive to changes in criterion
Sensitivity and Specificity
• If a test is positive, how likely is it that the
condition is present?
• Positive Predictive Value is the proportion of
“positive” test results that are correct
PPV 

# True Positives
# True Postives + # False Positives
Sensitivity and Specificity
• Negative Predictive Value is the proportion of
“negative” test results that are correct
NPV 

# True Negatives
# True Negatives + # False Negatives
Sensitivity and Specificity
• Consider the influence of exposure to some
substance or treatment on the presence or
absence of a condition
• e.g. smoking and cancer
• e.g. aspirin and heart disease
Sensitivity and Specificity
• A similar logic can be applied
No
Yes
Exposure:
Yes
Disease:
No
A
B
C
D
Sensitivity and Specificity
• A similar logic can be applied
No
Yes
Exposure:
Yes
Disease:
No
A
B
C
D
This is total #
exposure
Sensitivity and Specificity
• A similar logic can be applied
No
Yes
Exposure:
Yes
A
C
Disease:
No
B
D
This is total
non-exposure
Sensitivity and Specificity
• We can think in terms of “Event Rates”
No Yes
C
C +D
Exposure:
Control Event Rate 
Disease:
Yes
No
A
B
C
D
e.g. the proportion of non-smokers who get lung cancer

Exposure Event Rate 
A
A +B
e.g. the proportion of smokers who get lung cancer


Sensitivity and Specificity
Exposure Event Rate 
A
A +B
Relative Risk 
Control Event Rate 
Exposure
Event Rate
A /( A  B)


Control Event Rate
C /(C  D)
• Often encountered in regard to rate of
adverse reactions to drugs
No Yes
• Relative Risk is the ratio of Exposure Events
to Non-Exposure Events
Exposure:
Disease:
Yes
No
A
B
C
D
C
C +D
Sensitivity and Specificity
• Often we are interested in whether the chance of an event
changes with exposure
• Relative Risk Reduction is the difference between event rates in
the exposure and non-exposure groups, expressed as a fraction
of the non-exposure event rate
Relative Risk Reduction 
Exposure Event Rate - Control Event Rate
Control Event Rate

Sensitivity and Specificity
• Notice that Relative Risk Reduction can be positive or negative:
that is, exposure could reduce the risk of some event (e.g.
exposure to wine reduces risk of heart disease) or increase the
risk (e.g. exposure to cigarette smoke increases risk of heart
disease)
Relative Risk Reduction 
Exposure Event Rate - Control Event Rate
Control Event Rate
Sensitivity and Specificity
• Notice also that these figures do not take into account the
absolute numbers
• e.g. control event rate = .264 and exposure event rate = .198
• e.g. control event rate = .000000264 and exposure event rate =
.000000198
• Both yield the same relative risk reduction of -25%
Sensitivity and Specificity
• Notice also that these figures do not take into account the
absolute numbers
• e.g. control event rate = .264 and exposure event rate = .198
• e.g. control event rate = .000000264 and exposure event rate =
.000000198
• Both yield the same relative risk reduction of -25%
• Doesn’t discriminate between large and small effects
Sensitivity and Specificity
• The absolute risk reduction conveys effect size
Absolute Risk Reduction  Exposure Rate - Control Rate

Sensitivity and Specificity
• The absolute risk reduction conveys effect size
Absolute Risk Reduction  Exposure Rate - Control Rate
• An intuitive version is to consider the reciprocal - the “number
needed to treat or harm”

Number Needed to Treat or Harm =

1
Absolute Risk Reduction
• Indicates the number of individuals that would have to be
exposed to the treatment in order to cause one to have the
outcome of interest