Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Introduction to Biomedical Statistics Signal Detection Theory • What do we actually “detect” when we say we’ve detected something? Signal Detection Theory • What do we actually “detect” when we say we’ve detected something? • We say we’ve “detected” when a criterion value exceeds a threshold Signal Detection Theory • examples: – the onset of a light or sound – the presence of an abnormality on x-ray Signal Detection Theory • There are 4 possible situations Present Absent You Respond: Target is: Present Absent Signal Detection Theory • There are 4 possible situations Present Absent You Respond: Target is: Present Absent Hit Signal Detection Theory • There are 4 possible situations Present Hit Absent You Respond: Target is: Present Absent Miss Signal Detection Theory • There are 4 possible situations Present Hit Absent You Respond: Target is: Present Absent Miss False Alarm Signal Detection Theory • There are 4 possible situations Present Hit False Alarm Absent You Respond: Target is: Present Absent Miss Correct Rejection Signal Detection Theory • There are 4 possible situations Present Hit False Alarm Absent You Respond: Target is: Present Absent Miss Correct Rejection This is the total # of Target Present Trials Signal Detection Theory • There are 4 possible situations Present Hit False Alarm Absent You Respond: Target is: Present Absent Miss Correct Rejection This is the total # of Target Present Trials Signal Detection Theory • Hit Rate (H) is the proportion of target present trials on which you respond “present” H # Hits # Hits # Misses Signal Detection Theory • Notice that H is a proportion, so 1 - H gives you the “miss” rate or… MissRate # Misses # Hits # Misses Signal Detection Theory • False-Alarm Rate (FA) is the proportion of target absent trials on which you respond “present” FA # FalseAlarms # FalseAlarms # CorrectRejections Signal Detection Theory • Notice that FA is a proportion. 1 minus FA gives you the correct rejections or … CorrectRejectionRate # CorrectRejections # FalseAlarms # CorrectRejections Signal Detection Theory • Signal Detection can be modeled as signal + noise with some detection threshold Noise is normally distributed - Target Absent trials still contain some stimulus Frequency Signal Detection Theory Target Absent Stimulus Intensity Noise is normally distributed - Target Absent trials still contain some stimulus Frequency Signal Detection Theory Target Present trials contain a little bit extra intensity contributed by the signal Target Present Target Absent Stimulus Intensity Signal Detection Theory Noise is normally distributed - Target Absent trials still contain some stimulus Frequency This is the signal’s contribution Target Present trials contain a little bit extra intensity contributed by the signal Target Present Target Absent Stimulus Intensity Signal Detection Theory Frequency • We can imagine a static criterion above which we’ll respond “target is present” Criterion Target Present Target Absent Stimulus Intensity Signal Detection Theory Frequency • Notice that H, FA, etc thus have graphical meanings Criterion Proportion Hits Stimulus Intensity Signal Detection Theory Frequency • Notice that H, FA, etc thus have graphical meanings Criterion Proportion Misses Stimulus Intensity Signal Detection Theory Frequency • Notice that H, FA, etc thus have graphical meanings Criterion Proportion False Alarms Stimulus Intensity Signal Detection Theory Frequency • Notice that H, FA, etc thus have graphical meanings Criterion Proportion Correct Rejections Stimulus Intensity Signal Detection Theory Frequency • Notice that as H increases, FA also increases Criterion H Stimulus Intensity Signal Detection Theory Frequency • Notice that as H increases, FA also increases Criterion FA Stimulus Intensity Signal Detection Theory • d’ (pronounced d prime) is a measure of sensitivity to detect a signal from noise and does not depend on criterion - it is the distance between the peaks of the signal present and signal absent curves d’ is computed by converting from H and FA proportions into their corresponding Z scores and subtracting Zinv(FA) from Zinv(H) Frequency • Stimulus Intensity Some Common Biomedical Statistics • • • • • • • • Sensitivity Specificity Positive Predictive Value Negative Predictive Value Likelihood Ratio Relative Risk Absolute Risk Number needed to treat/harm Sensitivity and Specificity • Consider a test for a condition – e.g. Pregnancy test – e.g. Prostate-specific Antigen (Prostate Cancer) – e.g. Ultrasound (Breast Cancer) • These are all signal detection problems Sensitivity and Specificity • Four possible situations: Present Absent Test Result: Condition is: Present Absent Sensitivity and Specificity • Four possible situations: Present Absent Test Result: Condition is: Present Absent True Positive Sensitivity and Specificity • Four possible situations: Present True Positive Absent Test Result: Condition is: Present Absent False Negative Sensitivity and Specificity • Four possible situations: Present True Positive Absent Test Result: Condition is: Present Absent False Negative False Positive Sensitivity and Specificity • Four possible situations: Present True Positive False Positive Absent Test Result: Condition is: Present Absent False Negative True Negative Sensitivity and Specificity • Four possible situations: Present True Positive False Positive Absent Test Result: Condition is: Present Absent False Negative True Negative This is Total # of Condition Present Cases Sensitivity and Specificity • Four possible situations: Present True Positive False Positive Absent Test Result: Condition is: Present Absent False Negative True Negative This is Total # of Condition Absent Cases Sensitivity and Specificity • Four possible situations: Present True Positive False Positive Absent Test Result: Condition is: Present Absent False Negative True Negative This is Total # of “positive” tests Sensitivity and Specificity • Four possible situations: Present True Positive False Positive Absent Test Result: Condition is: Present Absent False Negative True Negative This is Total # of “negative” tests Sensitivity and Specificity • Sensitivity is the proportion of condition present cases on which the test returned “positive” • Analogous to the hit rate (H) in Signal Detection Theory Sensivity # True Positives # True Postives + # False Negatives Sensitivity and Specificity • Specificity is the proportion of condition absent cases on which the test returned “negative” • Analogous to the Correct Rejection rate in Signal Detection Theory Specificity # True Negative # True Negative + # False Positive Sensitivity and Specificity • Notice that 1 minus the Sensitivity is analogous to the FA of Signal Detection Theory Sensitivity and Specificity • Notice that 1 minus the Sensitivity is analagous to the FA of Signal Detection Theory • Recall that in Signal Detection Theory, as criterion were relaxed both H and FA increased and as criterion were more stringent, H and FA decreased Sensitivity and Specificity • Notice that 1 minus the Sensitivity is analagous to the FA of Signal Detection Theory • Recall that in Signal Detection Theory, as criterion were relaxed both H and FA increased and as criterion were more stringent, H and FA decreased • Sensitivity and Specificity have a similar relationship: as a cut-off value for a test becomes more stringent the sensitivity goes down and the specificity goes up…and vice versa Sensitivity and Specificity • “For detecting any prostate cancer, PSA cutoff values of 1.1, 2.1, 3.1, and 4.1 ng/mL yielded sensitivities of 83.4%, 52.6%, 32.2%, and 20.5%, and specificities of 38.9%, 72.5%, 86.7%, and 93.8%, respectively.” JAMA. 2005 Jul 6;294(1):66-70. Sensitivity and Specificity • Likelihood Ratio is the ratio of True Positive rate to False Positive rate Likelihood Ratio Sensitivity 1 - Specificity • Loosely corresponds to d’ in that Likelihood ratio is insensitive to changes in criterion Sensitivity and Specificity • If a test is positive, how likely is it that the condition is present? • Positive Predictive Value is the proportion of “positive” test results that are correct PPV # True Positives # True Postives + # False Positives Sensitivity and Specificity • Negative Predictive Value is the proportion of “negative” test results that are correct NPV # True Negatives # True Negatives + # False Negatives Sensitivity and Specificity • Consider the influence of exposure to some substance or treatment on the presence or absence of a condition • e.g. smoking and cancer • e.g. aspirin and heart disease Sensitivity and Specificity • A similar logic can be applied No Yes Exposure: Yes Disease: No A B C D Sensitivity and Specificity • A similar logic can be applied No Yes Exposure: Yes Disease: No A B C D This is total # exposure Sensitivity and Specificity • A similar logic can be applied No Yes Exposure: Yes A C Disease: No B D This is total non-exposure Sensitivity and Specificity • We can think in terms of “Event Rates” No Yes C C +D Exposure: Control Event Rate Disease: Yes No A B C D e.g. the proportion of non-smokers who get lung cancer Exposure Event Rate A A +B e.g. the proportion of smokers who get lung cancer Sensitivity and Specificity Exposure Event Rate A A +B Relative Risk Control Event Rate Exposure Event Rate A /( A B) Control Event Rate C /(C D) • Often encountered in regard to rate of adverse reactions to drugs No Yes • Relative Risk is the ratio of Exposure Events to Non-Exposure Events Exposure: Disease: Yes No A B C D C C +D Sensitivity and Specificity • Often we are interested in whether the chance of an event changes with exposure • Relative Risk Reduction is the difference between event rates in the exposure and non-exposure groups, expressed as a fraction of the non-exposure event rate Relative Risk Reduction Exposure Event Rate - Control Event Rate Control Event Rate Sensitivity and Specificity • Notice that Relative Risk Reduction can be positive or negative: that is, exposure could reduce the risk of some event (e.g. exposure to wine reduces risk of heart disease) or increase the risk (e.g. exposure to cigarette smoke increases risk of heart disease) Relative Risk Reduction Exposure Event Rate - Control Event Rate Control Event Rate Sensitivity and Specificity • Notice also that these figures do not take into account the absolute numbers • e.g. control event rate = .264 and exposure event rate = .198 • e.g. control event rate = .000000264 and exposure event rate = .000000198 • Both yield the same relative risk reduction of -25% Sensitivity and Specificity • Notice also that these figures do not take into account the absolute numbers • e.g. control event rate = .264 and exposure event rate = .198 • e.g. control event rate = .000000264 and exposure event rate = .000000198 • Both yield the same relative risk reduction of -25% • Doesn’t discriminate between large and small effects Sensitivity and Specificity • The absolute risk reduction conveys effect size Absolute Risk Reduction Exposure Rate - Control Rate Sensitivity and Specificity • The absolute risk reduction conveys effect size Absolute Risk Reduction Exposure Rate - Control Rate • An intuitive version is to consider the reciprocal - the “number needed to treat or harm” Number Needed to Treat or Harm = 1 Absolute Risk Reduction • Indicates the number of individuals that would have to be exposed to the treatment in order to cause one to have the outcome of interest