Download discuss-r-and-v-diagnosis-ib-1

Discuss the validity and reliability of diagnosis
Certain behaviours and ways of thinking or feeling are considered to be abnormal and dysfunctional and it is
possible that an individual who displays enough of these differences for long enough may be found to fit the
criteria for any one of more than 400 different mental disorders listed in classifications systems such as the
DSM IV(TR) published by the American Psychiatric Association, the ICD 10 published by the World Health
Organisation or one of the other national classification systems such as the CCMD-3 in China.
Diagnosis is important because it means that a person may then be provided with a treatment plan or at least
given support in terms of how to manage their symptoms more effectively and the additional information that
comes with a diagnosis may help the individual and/or family to know that they are not alone and that there
are other people with the same condition and possibly support groups and so on. However, many argue that
the diagnosis of mental disorders is neither valid nor reliable and people with mental disorders too often are
misdiagnosed or not diagnosed at all. For example, bipolar patients, who suffer extreme mood swings from
depression to mania, often go years without discovering their true illness. In a 2002 survey by the Depression
and Bipolar Support Alliance, 70 percent of bipolar people said their doctors misdiagnosed them at least once,
most often with depression or schizophrenia.
If diagnosis is reliable then different practitioners should reach the same conclusions when faced with a person
presenting a specific set of symptoms. This is known as inter-rater reliability. Another way of assessing
reliability is for the same patient to be tested and then rested by the same practitioner; again they should get
the same diagnosis for the system to be reliable (test-retest). Similarly, different patients presenting the exact
same symptoms should receive the same diagnosis. Statistical techniques can be used to measure consistency
or reliability of diagnoses. For example positive predictive values or PPV is the percentage of people who
retain the same diagnosis when reassessed while Cohen’s Kappa is a correlation coefficient, a number
between 0 and 1, indicating the extent to which practitioners agree and give a group of patients the same
diagnoses over time. This latter technique is particularly sensitive as it factors out the extent to which
agreement could be expected to arise by chance alone. It is apparent from these figures that it may be easier
to diagnose some conditions more reliably than others.
The original versions of the DSM were notoriously unreliable and studies such as that conducted by Beck et al
(1962) revealed that agreement on diagnosis for 153 patients between two psychiatrists was only 54%.
Similarly, Spitzer and Fleiss (1974) completed a meta-analysis of the results of six studies looking at 18
disorders and found that the average agreement was 52%, although this ranged from 10-90%. Cultural bias
also seemed to affect the reliability of diagnosis; Cooper et al (1972) found that New York psychiatrists were
twice as likely to diagnose schizophrenia as London psychiatrists, who in turn were twice as likely to diagnose
mania or depression when shown the same videotaped clinical interviews. Given these problems, the DSM III
was designed to improve the vague criteria. This manual presented standardised checklists which could be
administered using a clincial interviews and observing patients conducting various tasks. Subsequent studies
have demonstrated much higher consistency.
This said, Di Nardo et al (1993) studied the reliability of DSM-III for anxiety disorders. Two clinicians separately
diagnosed each of 267 people seeking treatment for anxiety and stress disorders and found high reliability for
obsessive-compulsive disorder (.80) but very low reliability for assessing generalised anxiety disorder (.57)
mainly due to problems with interpreting how excessive a person’s worries were and this suggests that despite
the claim that the DSM III has all but eliminated troubles with reliability (Klerman, 1984), this appears to not
be the case for all disorders.
Furthermore, Lipton and Simon (1985) indicate the unreliability of the DSM for schizophrenia. They randomly
selected 131 patients in a hospital in New York and conducted various assessment procedures to arrive at a
diagnosis for each patient. This diagnosis was then compared with the original diagnosis. Of the original 89
diagnoses of schizophrenia, only 16 received this diagnosis on re-evaluation. Fifty were diagnosed with a mood
disorder, even though only 15 had been diagnosed as such in the first place. However it is impossible to say
whether the mood disorders had developed following misdiagnosis of schizophrenia and institutionalisation!
Lipton and Simon’s findings were not replicated by Goldberg (1988) however. She applied the checklists from
DSM III to reassess the case notes of 199 people with schizophrenia diagnoses originally made using DSM II.
She also used the single blind technique to ensure that her data was not affected by investigator effects (as
she knew who had received a diagnosis of schizophrenia in the initial assessment with the DSMII). She asked
independent judges to assess eight of her sample of 199 without knowing their original diagnosis and found a
high degree of reliability, which was in line with her conclusions about the rest of the sample.
Goldstein’s study focused on the DSM III however more recently Pontizovsky et al (2006) looked at the
reliability of the diagnosis of schizophrenia in comparison with the diagnosis of depression. They used the ICD
10 with two large samples of people with depression and other mood disorders and schizophrenia and other
psychotic disorders, who had been admitted to an Israeli psychiatric hospital. PPV and Kappa scores indicated
that the diagnosis of mood disorders was slightly more reliable than the diagnoses of psychotic disorders such
as schizophrenia. The PPV was 94% for mood disorders and 84 % for psychotic disorder; Kappa was 0.68 for
mood disorders and 0.62 for psychotic disorders. Although both sets of figures demonstrate fairly high
reliability for both disorders these figures are not perfect indicating some problems with the consistency of
diagnoses using the ICD 10.
Another study by Nicholls et al (2000) looked at how reliable the ICD 10 was in comparison with the DSM IVTR
for eating disorders in children. Neither classification system proved especially reliable; in the case of the DSM,
over 50% of the sample of 81 children did not fit the criteria despite having eating problems and reliability was
only 64%. Results with the ICD 10 were even worse with only 36% agreement between raters. A system
designed by researchers at Great Ormond Street was apparently far more reliable as raters agreed on 88% of
cases. It is possible, however, that the DSM and ICD are simply less reliable when applied to child samples and
the suggestion here is that systems need to be reviewed for younger patients particularly for conditions like
eating disorders where the age of onset is becoming increasingly younger.
Finally, Brown et al (2001) looked at the diagnosis of disorders where historically, practitioners have had
difficulty in achieving consistency, mood and anxiety disorders. This is because the symptoms of depression
and anxiety often overlap and are both found in the same person. In this study inter-rater reliability was
judged to be good-excellent using the DSM IVTR checklists, however there were some problems including
disagreement between practitioners about when a symptom was severe enough to count and whether the
person seemed to have been suffering long enough to receive the diagnosis. This is a major problem within
diagnosis leading to problems with validity and reliability and is accounted for by the subjectivity involved in
interpreting what people say about how they feel.
Despite the fact that the three studies above do seem to suggest that reliability is less of a problem now that
the classification manuals contain behavioural symptoms that are more clearly operationalised, diagnosis still
has its critics. Kirk and Kutchins (1994) argue that the studies which support the reliability of the DSM for
example may be flawed and not generalisable. They suggest that the interviewers used are not sufficiently
trained to make proper clincial judgements and that the studies took place in a research context and not in a
clinical context. However, this criticism is over-ruled given the fact that Goldstein did not even use
interviewers as she analysed existing case notes (secondary data) and also that there have been studies
conducted in normal clincial settings which have also pointed towards reliability of diagnosis.
The gradual improvements to the classification systems appear to have enhanced the reliability with which
people can be diagnosed consistently. However some critics argue that this enhanced reliability is to the
detriment of the validity of the diagnosis, which may be based upon classifications which arguably don’t exist
as discreet categories, with absolute boundaries. Validity in this instance refers to the idea that diagnostic
systems are able to differentiate between those who have various mental health disorders and those who do
not. It is important that the systems are not overly inclusive, i.e. they do not end up diagnosing people who do
not require treatment and they do not miss those that do. Many psychiatrists may well be aware of the fact
that the categories are simply concepts and only useful so far as they help people to gain the support that they
need to enjoy more productive lives but the reality is that these categories have been ‘reified’ and difficulty
here is that labels may limit opportunities afforded to people who are diagnosed due to prejudice and
discrimination as much as they may initiate support. Reification refers to the fallacy that occurs when abstract
concepts are discussed without question as though they are concrete and certifiable. Whether someone has a
specific mental health condition cannot be tested objectively by using a brain scans or blood tests for example,
although in the conclusion of this essay mention is made of current technological advances, which make this a
possibility for the future, or least in terms of predicting the probability with which someone will experience a
certain condition.
One study which illustrates that the DSM specifically may be more reliable than it is valid, is the highly
controversial study conducted by David Rosenhan (1973). He was a student of renowned proponent the antipsychiatric movement, Robert Laing. Ike Thomas Szasz, he argued that psychiatrists rarely attempt to
understand symptoms as normal reactions to life’s problems and highlighted the social, legal and political
consequences of diagnoses as a means of social control. In order to test Ling’s ideas, Rosenhan devised a
participant observation whereby 8 people presented themselves at differing psychiatric institutions across
America claiming to have heard a voice which said “thud” and “hollow” but presenting no other unusual
symptoms. In all but one case the participants, who were all perfectly sane, were diagnosed with
schizophrenia (one was diagnosed with bi-polar disorder). They were all admitted to the institutions and the
average durations of stay lasted 19 days, ranging from 7 to 52 days. All were prescribed anti-psychotic
medication which they pretended to take but in fact disposed of. All were eventually released but with the
diagnosis of schizophrenia in remission. In a follow up study Rosenhan agreed with one hospital that he would
send further pseudo-patients to them for detection. In fact he never sent any more patients however one
employee claimed that 43 of the 193 patients admitted the in the following 3 months were frauds sent by
Rosenhan, and one psychiatrist who was supported by one other member of staff claimed that 19 of the
people he saw were in fact sane pseudo-patients when they were in fact real clients presenting with real
symptoms.
This study shows that when people are labelled as having a mental health problem, i.e. why else would they
turn up at a mental health institution, they are more likely to come away (or indeed end up staying) with a
diagnosis. Also once a label has been applied any subsequent behaviours are interpreted as symptomatic of
pathology. This demonstrates the subjectivity of the process of interpreting a person’s symptoms. When a
psychiatrist judges a person’s mental state, this is influenced by top-down processing and influenced by prior
knowledge, thus adding subjectivity. This was supported by an experiment by Caetano (1973) where videos of
a male psychiatrist conducting a standardised interview with either a paid university student or a hospitalised
mental health patient were shown to 77 psychology student and 36 psychiatrists. The participants were asked
to diagnose the ‘patients’ and rate the severity of their symptoms. Pps were randomly assigned to two groups;
one group heard that the interviewees were both paid student volunteers and the others that the
interviewees were both real mental health patients. Psychiatrists were found be more likely than students to
label the interviewees in accordance with the information provided about whether they were a student or a
patient. This indicates that psychiatrists do seem to pathologise reported information in line with
preconceived ideas. Phares (1979) has referred to this as 'reading-in syndrome'.
The degree of prior knowledge and extent of his or her prejudices may also lead to differences in diagnoses
which affect reliability as well as validity of diagnosis. This will go onto affect the direction of the questions
asked or the focus of an observations made meaning that the data that they record is further based and begins
to resemble even further their initial preconceptions. Meehl (1962) and Dailey (1952) suggest that clinicians
often too much emphasis on information gleaned earlier in the interview (primacy effect) and ignore later but
significant information in their diagnosis. Similarly, they may put too much emphasis on one source of
information, for example, the family's descriptions such as paying too much attention to the parents rather
than to the child (McCoy 1976). Numerous studies point to personal biases in the clinician's interpretation of
data (Fabrega et al 1994, Jenkins-Hall and Sacco 1991, DiNardo 1975, Broverman et al 1970).
In response to the cognitive biases operating on the psychiatrist’s judgements, the person being interviewed
will be responding to the cues in the psychiatrist’s behaviour and questioning. He or she will be affected by the
way in which the clinician interacts with him or her. The rapport that is established will be critical in terms of
how much or how little is revealed. It is possible that certain psychological barriers to communication may
exist in terms of the differences in class, race, gender and the linguistic codes used by the client and clinician
and all of these will affect the quantity and quality of information exchanged, leading to inaccuracies in the
impression formed by the clinician.
Certain specific biases may also affect the process, for example , some clinicians may believe that certain
symptom are more important than others when coming to a diagnostic decision, however this is generally not
the case. This means that once a certain symptom has been recorded, the clinician may pay less attention to
other things that are said. This was supported by a study by Ahn looking at eating disorders. It appeared that
some clinicians over-estimated the importance of certain symptoms and underestimated other based on their
on idea about now the symptoms were inter-related. 21 psychologists presented with two hypothetical cases
of clients presenting with three symptoms, one which was from the DSM criteria for anorexia and two that
were not. Each ‘client’ should have had an equal chance of being diagnosed with anorexia, however, if Pps
thought the client was refusing to eat, the client was far more likely to be said to possibly be suffering from
anorexia than the ‘client’ who said she had amenorrhea (menstrual periods have ceased), despite both being
symptomatic of anorexia and both sharing equal importance in the diagnosis, according to the DSM. It seemed
that the clinician’s biases affected the way they perceived the information provided by the clients and this
then in subsequent testing it was also found that this led to selective remembering and even confabulation of
symptoms that fitted with the diagnosis that were not in fact reported. This could then affect the medical
history notes that are recorded, which are an important source of secondary data, particularly when second
opinions are sought and this maybe in fact be why second opinions frequently do not differ from the initial
assessment.
Diagnosis can therefore be seen to be a very subjective process affected by top down processing by both the
psychiatrist and the client providing information. There is a great deal of subjectivity even within the DSM
itself for example, symptoms include decreased need for sleep, distractibility and excessive talking yet to
assess these criteria a baseline is required and it is unclear whether this baseline is to the person’s own normal
behaviour, that of their own family and peer group or indeed society at large. Furthermore in axis 5 of the
DSM IV TR, assessments are supposed to be in comparison with a ‘normal person’ which begs the question
normal in whose eyes? Once the psychiatrist has at least established that what the client is reporting actually is
severe enough to be count as a symptom, the next job is to decide which diagnosis best describes the
collection of symptoms being experienced. This adds further difficulty regarding validity of diagnosis as many
differing disorders share similar patterns of symptoms and it is possible that the reason behind a behaviour
that is reported are actually more important than the behaviour itself, for example a patient may report severe
under-eating but this could be due to an underlying concern that their food has been poisoned (paranoid
schizophrenia), fear of weight gain (anorexia nervosa), a social phobia regarding fear of being observed eating
by others or indeed a specific phobia such as fear of choking. It is also the case that some disorders have so
many possible symptoms of which a certain number must be experienced in order to be diagnosed that few
patients sharing the same diagnoses would actually seem similar in terms of the case histories given. This is
certainly the case with schizophrenia where there are so many subtypes which respond differently to differing
medications that it is argued that schizophrenia is not actually one disorder at all.
A number of different types of validity have been identified and are helpful in considering the validity of the
diagnoses. A system is said to have criterion validity if it returns diagnoses which are in agreement with
diagnoses suggested by other systems, already deemed to be valid. Kahneman and Tversky (1973) indicate,
however that psychiatrists may be mislead if they think that just because a range of measure such as
interview, observation and projective tests all point to the same diagnosis, then the diagnosis must be valid
(triangulation), as research suggests that there is not a positive correlation between the number of assessment
techniques used and the accuracy of eventual diagnosis, (Golden 1964).
Another way in which validity has been assessed, is looking at whether receiving a diagnosis at one point in
time is a good predictor that the person will present signs and symptoms of the disorder which can be
detected by some other measure in the future. This is known as predictive validity. This is also tied in with
treatment; if a person has a certain disorder and is treated for it and the symptoms diminish, this is one way of
supporting the fact that the person actually had the specific disorder in the first place. However, this is very
difficult to establish as individual practitioners have their own ways of treating differing conditions and there is
only about 50% agreement based on diagnosis as to how the client will be treated, (Bannister et al, 1964). Also
treatments offered seem to be tied to characteristics of the person other than their diagnosis, for example
gender and race.
A final type of validity that relates well to diagnosis is construct validity. A diagnosis is a an construct to the
extent that it is abstract and must be inferred from what is said and done, it cannot be tested objectively in the
same way as one can see cancerous cells or a fractured bones. If a mental disorder has construct validity it
should have specific characteristics, signs and symptoms shared by sufferers, a specific cause and course of
treatment which leads to similar gains for all diagnosed with the condition. However, some disorders fit this
type of validity far better than others as we have seen form some of the arguments raised in this essay.
Numerous studies have explored the validity of the various mental disorders. For example Kim-Cohen et al
(2005) conducted a longitudinal study of 2232 five year olds. The children’s mothers were interviewed and
their teachers were asked to complete postal questionnaires regarding symptoms of conduct disorder
observed within the last 6 months. Children with three or more symptoms were diagnosed with conduct
disorder; those displaying five or more symptoms were diagnosed with moderate-severe conduct disorder. Of
the sample, 6.6% received a diagnosis and 2.5.% received the moderate-severe diagnosis. The children given
the diagnosis were significantly more likely to say that they felt that their behaviour was anti-social (a key
factor in conduct disorder) than those who did not receive a diagnosis and they were also more likely to
display antisocial behaviours whilst being observed, have been exposed to risk factors associated with the
disorder and to display behavioural and educational difficulties at age 7. This study indicates that conduct
disorder may well have criterion and predictive, however it is not possible to say that the category has
construct validity as the causes were not studied and neither were reactions to treatment options.
Another disorder delineated in the DSM IV TR is attention deficit and hyperactivity disorder (ADHD) and this
shares some symptoms with conduct disorder and also with another category, oppositional defiant
disorder. These categories are controversial with regard to validity and are seen as a product of culture and
era. Critics argue that these new labels simply pathologies naughtiness and bad behaviour. However a study
by Lee et al (2006) attempted to explore the validity of the ADHD category for Korean school children.
Questionnaires were given to 48 primary school teachers regarding symptoms including inattention,
hyperactivity and impulsivity. 1663 children were rated, including 904 boys and 759 girls. Comparisons were
drawn between the findings from the questionnaires and scores on another test of ADHD and it was
concluded that ADHD as a valid construct for use in diagnosing Korean children however the match
between the two tests was better for boys than it was for girls, demonstrating a problem with gender as a
variable which may influence the way in which others pathologies socially disruptive behaviour in girls more
than they do in boys in Western cultures where it is more expected that boys will be more noisy and
physical in their play than girls. This also indicates the problems with the cultural validity of classifications
systems designed in one culture and applied in another, (the etic approach) and this the importance of
using systems that are culturally relevant to the individual being assessed.
Over time it is possible that diagnosis could become more objective with advancements in our understanding
of the human genome and gene variants which are more common in people with varying mental disorders.
Similarly, as we become more adept at reading the results of brain scans (both structural and functional) it is
possible that we may be able to use these to aid the diagnostic process. This practice will be rife with problems
however as in many cases there will be example of individuals with the characteristics biological differences
yet they will not go onto develop the condition due to various biological and environmental factor which lead
to resilience likewise, there will doubtless be instances of people who do not have the biological characteristics
yet still show symptoms who may then not be give treatments as it is not believed that their symptoms afford
enough certainty that they have the specified condition. The idea of biological testing for mental health
disorders throws up a multitude of causes for concern ethically and politically and also with regard to whether
a person is given treatment (chemotherapy or otherwise) to protect them from developing a disorder for
which they are not currently displaying symptoms.
In conclusion, it would seem that although classification systems are becoming more reliable, the debate
regarding the validity of diagnosis is far from over. However, diagnosis is still necessary in helping people to
access support and treatment and in order to conduct research into aetiology and efficacy of treatment
programmes. This said diagnosis can also lead to labelling, prejudice and discrimination whether accurate or
otherwise and inaccurate diagnosis could potentially lead to inappropriate courses of treatments which may
exacerbate or lead to further symptoms. Supporters of diagnosis such as Akiskal (1989) hold that
"classification and diagnosis can yield valuable information that greatly advances the understanding and
treatment of people in distress. To throw away such information would be too drastic a measure, one that
would create more problems than it solves" (Comer 1995). It seems that the problems with diagnosis do not
mean that the conditions do not exist and that people do not require treatment, just that an appropriate
amount of time and attention must be invested in taking a thorough assessment of the person’s symptoms
and that practitioners need to be highly trained and experienced before their judgements are accepted
without question. Also, it is imperative that continuing professional development for practitioners includes
reflexivity and an emphasis on how cognitive biases and prejudices can affect the accuracy of the histories
taken from a client.