Download Epidemrating part 2 Dr Sean Lynch 12th April 2013

Research Methodology and
Epidemiology - 2
Basic epidemiological principles in
psychiatry and psychiatric rating scales
Sean Lynch
Research Methodology and
Epidemiology -2
This is the second part of the afternoon module and today
we will look at some aspects of rating scales and theory
In psychiatry
Research Methodology and
Epidemiology -2
Self-rated Scales
•
•
•
•
•
•
•
•
•
•
Quick, save interviewer time
Lack of bias
Reliable
Can be used for case detection
Can be used as outcome measures
Only as good as the questions they ask
Problems with phrasing, ease of reading, language
“Order effects”
“Social desirability effects”
“Central tendency”
Research Methodology and
Epidemiology -2
Self-rated Scales
• Can be used for any diagnostic category, except
arguably less well for psychosis, cognitive impairment
• Can be used to study core symptom dimensions in depth
e.g. sleep, energy
Research Methodology and
Epidemiology -2
Interview - Rated Scales
• Can be used to help assess whether a mental disorder is
present or not
• Can be used to assess the severity of symptoms in a
mental disorder
• Can be used with a taxonomical system to make
diagnosis
• Can also be used to study certain dimensions of
symptoms in great depth
• Can be highly structured with pre-determined questions
• Can be more flexible and allow more “open-ended”
questions
Research Methodology and
Epidemiology -2
Interview - Rated Scales
• Can be subject to observer bias
• Different raters might disagree on assessment
• Can be subject to changes in behaviour of the same rater
over time
• Can be flexible and obtain supplementary information
• Probing questions can be used to ensure subject
understand the questions
• Some say these scales can be sensitive to “clinical
change”
Research Methodology and
Epidemiology -2
Scales need to have an “anchor point” to distinguish
between lack of pathology and an “extreme” to assess
severe pathology
Scales can be broad and have numerous intermediate
points with phrases or words to help guide scoring of
severity or degree
Scales can be narrow and dichotomous
Research Methodology and
Epidemiology -2
Scales can suffer from “ceiling” and “floor” effects i.e. if
measuring dimensions within less severe or more severe
examples of pathology e.g. depressed mood.
It can be hard to show improvement in an item where there
is not much opportunity to show change! e.g. trials in mild
severity depression
Research Methodology and
Epidemiology -2
Properties of Scales
1. Case finding
Ability to detect “cases” (true positives)
Ability to distinguish “non-cases” (true negatives)
Performance on these with minimum of misclassification
i.e. low false positive and low false negative
The fine tuning of a scale cut-off point or score can be
biased towards sensitivity (true positives/true positives and
false negatives) in other words not “missing” too many
cases, or specificity (true negatives/true negatives and
false positives) in other words having a lower number of
“non-cases” incorrectly identified as cases
Research Methodology and
Epidemiology -2
Properties of Scales
2. Assessing severity
Can less severe and more severe cases be separated by the
scale?
Would similar scales or measures show agreement on
severity?
3. Assessing change
Is a reduction or increase in score on the scale associated
with changes in the clinical picture?
Research Methodology and
Epidemiology -2
Reliability
“It does what it says on the tin most times”
Test-retest
Inter-rater
Internal Consistency
Research Methodology and
Epidemiology -2
Validity
“It really tells you what is in the tin most times”
Convergent
Face
Construct
A scale can be highly reliable but measure such a narrow
concept it is clinically meaningless i.e. not valid
Research Methodology and
Epidemiology -2
DEPRESSION RATING SCALES
Interview-rated
HAMILTON 17 and 21 item
MONTGOMERY ASBERG 10 item
Self-rated
HAD (Hospital Anxiety and Depression) 7 item
Beck (BDI) 21 and 13 item
Research Methodology and
Epidemiology -2
ANXIETY RATING SCALES
Interview-rated
HAMILTON 14 item
Self-rated
HAD (Hospital Anxiety and Depression) 7 item
Research Methodology and
Epidemiology -2
OBESSIONAL RATING SCALES
Interview-rated
Y-BOCS (Yale Brown) 10 item
Self-rated
OCI (Obsessive Compulsive Inventory) 42 item
Research Methodology and
Epidemiology -2
SCHIZOPHRENIA RATING SCALES
Interview-rated
PANSS
SANS
SAPS
BPRS
Research Methodology and
Epidemiology -2
MANIA RATING SCALES
Interview-rated
YOUNG MANIA RATING SCALE
Research Methodology and
Epidemiology -2
DIAGNOSTIC INTERVIEWS
SCAN (Wing et al ) - ICD
SCID - DSM
Research Methodology and
Epidemiology -2
Disease concepts for diagnosis
• Competing paradigms were reductionist or more
pragmatic and multidimensional
• Hierachical approach (Foulds)
• Multiaxial approach (DSM)
• Now concepts of subsyndromal disorder
Levels of psychological disturbance
- severity
• Normal distress
• Monosymptomatic - but recognisable
• Subsyndromal - collection of several
symptoms which fails to meet diagnostic
criteria
• Syndrome
Other qualifying criteria
• Frequency and persistence of symptom
• Functional impairment or disability
• Symptom duration
Factors affecting agreement on
diagnosis
1. Can have the same information but different
disease concepts
2. Can evaluate the same information in a
different way
3. Can elicit different information from the same
patient
4. Can have changes in the clinical condition of
the patient at different times
Factors affecting agreement on
diagnosis
1. Different raters have different levels of
knowledge and expertise e.g. differences
between primary and secondary care
2. Our diagnoses have a degree of inbuilt
uncertainty
3. How confident are we that our diagnosis is
right?
4. How often will our colleagues agree with us?
Agreement on diagnosis - case
example
Man of 40 who presents to GP
Insomnia for ten days
Panic attacks for three weeks
Irritability at work for two weeks
Reduced libido for ten days
Agreement on diagnosis - case
example
Man of 40 who presents to GP
Insomnia for ten days
Panic attacks for three weeks
Irritability at work for two weeks
Reduced libido for ten days
Hopelessness about future (week)
Appetite reduced (two weeks)
Reduced energy (two weeks)
Agreement on diagnosis - case
example
BUT has auditory hallucinations in third person
for three days!
Agreement on diagnosis
• What is a psychiatric case? “Gold standard” Kendell, Shepherd
• Inter-rater reliability
• Intra-rater reliability
• Diagnostic interviews and index of definition
• “Cut-offs” based on symptom severity on rating
instruments
• Computerised
Agreement on diagnosis
• Generally improved with more severe illness
• Difficulty in milder illness levels distinguishing
from the normal range
• More difficult for certain diagnostic concepts e.g.
personality disorder and new DSM V
Usefulness of diagnosis
• Categorical verus dimensional models of illness
• Hypertension is not “all-or-nothing” but spectrum
form obvious disease to normal range
• Rose “not important if he has it, the question is
how much of it he has”
• Bentall - distribution of psychotic symptoms
Prevalence
• The number of defined cases of disease in an
area
• Includes older and more recently diagnosed
cases
• Can be influenced by the chronicity of illness
more than the incidence of illness
• Cases can develop and remit (or die) and
influence prevalence
Prevalence
• Point prevalence
• Period prevalence
• “Life-time rates”
Prevalence - interpreting changes
• Can be due to true changes in incidence
• Can be due to changes in effectiveness of
interventions
• Can be due to changes in nature or course of
illness
• Can be due to changes in detection rate
Incidence
• The number of new cases of illness in an area
over a period of time
• Changes more likely to reflect influences on
causation or associated risk
• Does not in itself give information on total
number of cases in community
• Changes can be due to changes in detection
rate
• Problem of including relapses inadvertently
Incidence
• Diseases with low incidence can become
prevalent in community if chronic illnesses
• Prevalence can change without change in
incidence necessarily e.g. change in severity of
illness or effectiveness of treatment
Methods to assess incidence and
prevalence
• A case register to document all contacts over
a defined period
• “Observatory” method
• Case notification methods
• Population based studies
Methods to assess incidence and
prevalence
• These methods can have limitations in diseases
with low incidence rates and prevalence rates
• The first three methods are particularly prone to
error if there are problems (or there is not full
consensus) on the case definition
• The detection rate of cases should also be
measured to assess accuracy (against best
available standards)
Methods to assess incidence and
prevalence
•
•
•
•
Case ascertainment methods
Prodromal phases
Changes to case definition
Need to have reliable and valid raw data to
review estimates
Methods to assess incidence and
prevalence
Other problems in psychiatry:• “Diagnostic overlapping”
• Cultural influences on case ascertainment
• Co-morbidity
• Diagnostic stability over time
• Social changes (“pathoplastic”)
Methods to assess incidence and
prevalence
“Head count” methods in psychiatry:• Case notification depends on accuracy of
diagnostic assessment and health seeking
behaviour
• Case registers depend on capturing all cases
and do not cope well with migration effects and
changes to housing or centres of population
Methods to assess incidence and
prevalence
Survey methods in psychiatry:• Need to carefully define area studied
• Feasible methods of case detection
• Often expensive as need to cover large areas
and numbers
• Need to have accurate estimate of population
base
Methods to assess incidence and
prevalence
Survey methods in psychiatry:• Catchment area and “house to house” methods
• Telephone methods
• For less prevalent conditions and low incidence
conditions will need to screen a large number of
“normals”
• Need a socially acceptable screening tool
Methods to assess incidence and
prevalence
Survey methods in psychiatry:• Much more problematic for more severe illness
• Problems of selection bias e.g. “cold-spots” of
participation, wrong time of day
Methods to assess incidence and
prevalence
Other methods
• Postal questionnaire
• Two-stage screening
• Representative sample e.g. random selection as
per some marketing approaches
• Quota samples
• Convenience samples
• Consecutive attendances
Methods to assess incidence and
prevalence
Error rate in study has to be defined.
Common methods:• Reference to “gold standard”
• With reference to known reliability and stability of
case definition
• Random resampling
• Non-participation and non-completion rates
• Estimates of “double counting” or “missing”
cases
Causation
Confounding variables
“Latent” variables
Interactions
Protective factors
Causative factors
Causation
Consider the following hypothesis:The risk of lung cancer is 100 times higher in
men aged 30-35 who smoke than in non-smokers.
Smokers on average watch 50% more television than
non-smokers. Smokers watching only the average
amount of television of non-smokers could reduce their
risk of lung cancer by one third. Television might also be
a causal factor in lung cancer.
Causation
Is there an argument to support this conclusion?
Is the evidence convincing?
Can you see any flaws in this argument?
Are there alternative methods of studying the
causal relationship between smoking, television
viewing and lung cancer?
Exposure and risks
Case control method is classical method
It gives an indication of differences in the rate of
disease on exposure to a potential risk factor
Cross-sectional studies only give information on
association
Longitudinal studies give some more information
on causation
Exposure or “at risk” studies give the best quality
information
Exposure and risks
There are several assumptions:• An equal chance of exposure to a risk factor in
all the population?
• Controls are “normal”
• We can measure the level of risk or exposure
• Exposure levels might vary in intensity
• Duration of exposure might be relevant
Exposure and risks
Risk measures:These attempt to quantify the increase in the
number or proportion of cases in a population
exposed to the risk factor, compared to those not
exposed
• Odds ratio
• Relative risk ratio
• Attributable risk
Exposure and risks
• These are quite useful concepts but rely on
assessing one risk factor at a time
• In psychiatry multiple risk factors might be
expected. Sometimes it is as useful to assess
interaction between factors
• We will discuss multivariate models in later
sessions
EBM terminology
Adverse event CONTROL TREATMENT
YES
a
b
NO
c
d
pc = proportion of controls with adverse event
pc= b/ (b+d)
pt = proportion of treatment group with adverse event
pt = a/(a+c)
Relative risk of event RRe = pt/pc
Relative risk of no event or RRne =(1-pt/ 1-pc)
EBM terminology
Odds ratio (OR) - (a x d) / (b x c)
Relative risk reduction RRR = (pc-pt)/ pc + 1-RRe
Absolute risk reduction (ARR) / risk difference (RD) = pc-pt
Number needed to treat NNT
NNT (risk difference) = 1/RD
NNT (relative risk of event) = 1 / (pc x RRR)
NNT (relative risk of no event) = 1 / (1-pc) x (RRne-1)
NNT (odds ratio) = (1-(pc x (1-OR)) / (pc x (1-pc) * (1-OR))
Other important related concepts
We will discuss these more fully when
discussing rating scales
• Sensitivity
• Specificity
• Misclassification rate
• Predictive value
• Efficiency