Download How do practising clinicians and students apply newly

Journal of Evaluation in Clinical Practice ISSN 1365-2753
How do practising clinicians and students apply newly
learned causal information about mental disorders?
jep_1781
1..6
Leontien de Kwaadsteniet PhD,1 Nancy S. Kim PhD2 and Jennelle E. Yopchick MA3
1
Lecturer in Education, Radboud University Nijmegen, Nijmegen, The Netherlands
Assistant Professor of Psychology, 3Graduate Student, Department of Psychology, Northeastern University, Boston, MA, USA
2
Keywords
causal knowledge, clinical expertise, clinical
reasoning, continuing education, diagnosis,
mental disorders
Correspondence
Dr Leontien de Kwaadsteniet
Radboud University Nijmegen
Montessorilaan 3
6525HR Nijmegen
The Netherlands
E-mail: [email protected]
Accepted for publication: 28 July 2011
doi:10.1111/j.1365-2753.2011.01781.x
Abstract
Rationale, aims and objectives New causal theories explaining the aetiology of psychiatric disorders continuously appear in the literature. How might such new information
directly impact clinical practice, to the degree that clinicians are aware of it and accept it?
We investigated whether expert clinical psychologists and students use new causal information about psychiatric disorders according to rationalist norms in their diagnostic reasoning. Specifically, philosophical and Bayesian analyses suggest that it is rational to draw
stronger inferences about the presence of a disorder when a client’s presenting symptoms
are from disparate locations in a causal theory of the disorder than when they are from
proximal locations.
Method In a controlled experiment, we presented experienced clinical psychologists and
students with recently published causal theories for different disorders; specifically, these
theories proposed how the symptoms of each disorder stem from a root cause. Participants
viewed hypothetical clients with presenting proximal or diverse symptoms, and indicated
either the likelihood that the client has the disorder, or what additional information they
would seek out to help inform a diagnostic decision.
Results Clinicians and students alike showed a strong preference for diverse evidence,
over proximal evidence, in making diagnostic judgments and in seeking additional information. They did not show this preference in the control condition, in which they gave their
own opinions prior to learning the causal information.
Conclusion These findings suggest that experienced clinical psychologists and students
are likely to use newly learned causal knowledge in a normative, rational way in diagnostic
reasoning.
Introduction
New scientific knowledge about psychiatric conditions appears in
the literature steadily and continuously. Of particular interest in the
current paper is that causal knowledge about the aetiology of
disorders is increasing at a rapid rate. Evidence suggests that that
this kind of knowledge may appeal strongly to clinicians in that
they have a marked tendency to reason causally [1]. For example,
clinicians conceptualize disorders as falling along a single continuum of aetiological causes (i.e. from biological to environmental), and they base judgments of treatment efficacy on whether
those treatments directly act upon these causes [2]. Similarly,
clinicians’ ratings of intervention effectiveness can be systematically predicted from their own reported causal models of clients’
problems [3], and clinicians weight causal symptoms more heavily
than their effects in making diagnostic decisions [4].
© 2011 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice
Although such work has shown that clinicians’ own causal
theories of disorders strongly influence their judgment and decision making, it has not addressed the degree to which new, externally presented causal information directly influences clinical
decisions. For example, when a practising clinician picks up a
research journal and reads about a new theory explaining how a
psychiatric disorder comes about, how might he or she apply that
knowledge in practice, to the degree that he or she accepts it? The
question of how clinicians use such newly acquired information in
their diagnostic reasoning processes has important implications for
both researchers and practitioners. In many cases, as we discuss
next, causal reasoning strategies can be considered strategies that
conform to rationalist norms. For example, students and experienced clinicians rate interventions that affect assumed root causes
of disorders as more effective than interventions affecting factors
with less causal impact [2,5].
1
Applying causal information
L. de Kwaadsteniet et al.
Abnormal
neural
connectivity
Decreased brain
activity in regions
for higher-order
functioning
Increased brain
activity in regions for
stimulus-driven
sensory processing
Cognitive
impairment in use of
contextual
information
Must cope with an
overflow of sensory
information
Impairment in
social
interactions
Problems with
communication
Preference for
repetitive tasks
In the same vein, the current study investigated whether the
causal structure of new information about psychiatric conditions
affects clinicians’ and students’ diagnostic reasoning. Current
theories of psychiatric conditions typically explain a fairly large
range of symptoms with a smaller number of causes (e.g. genetics,
brain and/or environment), forming what we will refer to as a
hierarchical causal structure. That is, many explanatory theories
assume that the manifestation of a given psychiatric condition can
be traced back to just one or a few broad categories of root causes
[6]. Please see Fig. 1 for an illustrative example. Our study therefore specifically asks how clinicians and students might make use
of theories of psychiatric conditions that take the form of hierarchical causal structures.
In previous research, undergraduate students were presented
with hierarchical causal structures of various medical conditions
(e.g. rheumatic fever) and asked them to judge which of two
patients was most likely to have the condition in question: a patient
presenting with two symptoms originating from the same causal
path (‘causally proximal’ symptoms, henceforth), or a patient presenting with two symptoms originating from different paths
(‘causally diverse’ symptoms) [7]. Students most often chose the
patient with diverse symptoms. The influence of hierarchical
causal information earlier on in the diagnostic reasoning process
has also been examined in undergraduate students [8]. Students
presented with a patient exhibiting a single symptom were asked
whether they would prefer to seek out information about the presence of a causally diverse versus causally proximal symptom in
determining whether the patient had the medical condition. The
results showed that students preferred to seek out information
regarding the presence of causally diverse over causally proximal
symptoms [8]. Students did so even though they also reported that
patients were more likely to have causally proximal than causally
diverse symptoms.
2
Need for
structure and
rules
Figure 1 Sample causal hierarchical structure for autism, adapted from [21,22].
The students’ preference for causally diverse evidence in these
medical reasoning tasks is in line with a broad preference for
feature diversity in category-based induction tasks in classic cognitive science research [9–11]. This preference for diverse evidence has been previously argued to be normative and rational
[12,13], and can be justified via Bayesian computation [14–16]. A
preference for causally diverse evidence can be similarly justified.
The likelihood that a specific root cause is present, given proximal or diverse evidence, is equal to the probability that the evidence occurs given the root cause (P (e|c1)) multiplied by the
probability of the root cause independent of any evidence (P(c1)),
controlling for the prior probability that the evidence occurs (i.e.
the summed probabilities that the evidence occurs with all possible causes – hypothesized root cause and alternative causes).
See Equation (1):
P (c1 e) =
P (e c1 ) P (c1 )
1− n P (e c ) P (c )
∑c
(1)
Possible causes for proximal evidence may include the possible
causes for the presence of diverse evidence, and more. Thus, the
denominator in the equation gets larger for proximal evidence
compared with diverse evidence, leading to a lower probability
that, with proximal evidence, the hypothesized root cause is
present. With fewer alternative causes, as with diverse evidence,
the denominator gets smaller, resulting in a higher probability of
the hypothesized root cause. Conversely, the presence of one
symptom increases the likelihood of the preceding causal chain in
the hierarchy, and thus the presence of the other proximal
symptom, whereas it does not similarly increase the likelihood of
the other causal chain, and thus the presence of diverse symptoms.
Proximal symptoms can thus, rationally, be more expected to
occur than diverse symptoms.
© 2011 Blackwell Publishing Ltd
L. de Kwaadsteniet et al.
Of course, it is completely unknown whether actual clinical
practitioners would show the same preference for causal diversity
in diagnostic reasoning as did students in prior research regarding
medical conditions. In category-induction tasks in cognitive
science, domain expertise has been found to lead to other reasoning strategies that can instead lead to an apparent preference for
feature proximity [17–19]. Interestingly, experts in these studies
directly applied their pre-existing, additional knowledge about the
causal relationships between features and were thereby able to
circumvent the need to draw inferences from diversity. Thus,
relying on feature diversity might be a default strategy for novices.
However, in these studies, no new information was presented; both
domain experts and novices reasoned using the prior knowledge
they had.
In the current work, our driving question was whether experienced clinicians, as well as students, show a causal diversity effect
in diagnostic reasoning with new causal information. With increasing scientific knowledge about the aetiology of psychiatric conditions, insights regarding how clinicians use such newly acquired
knowledge in their reasoning can ultimately enable us to anticipate
and improve clinicians’ diagnostic and treatment decisions by
means of prescriptive guidelines, decision aids or education. In
this study, we presented experienced clinical psychologists and
undergraduate students with causal hierarchical structures for five
psychiatric conditions. Each structure was a theory, derived from
recent, published clinical literature, explaining how symptoms of a
psychiatric condition are thought to originate from a deeper cause.
We tested whether clinicians and students would show a preference for causally diverse information in making diagnoses [7] and
in seeking information to help inform a diagnosis [8]. Because we
anticipated that individual clinicians would probably vary widely
in terms of their prior knowledge about specific psychiatric conditions, all participants served as their own controls in a timedelayed, within-subject design.
Method
Participants
Sixty-two people participated in the study; of these, 29 were
experienced clinical psychologists and 33 were undergraduate
students, all recruited by mail, phone, or e-mail. We randomly
assigned the clinicians to carry out either the diagnosis task
(n = 15, 10 United States and 5 the Netherlands) or the
information-seeking task (n = 14, 8 United States and 6 the Netherlands). Clinicians had a mean of 19.9 years (SD = 8.7) of experience in clinical practice. Fifteen were female, and their mean
age was 48.6 years (SD = 10.2). US clinician participants were
paid 40 dollars and the Netherlands’ clinician participants
received a 10 euro gift certificate. Undergraduate students from
various fields of study were recruited to participate at Northeastern University in the United States and Radboud University
Nijmegen in the Netherlands. Students were randomly assigned
to either the diagnosis task (n = 16, 8 United States and 8 the
Netherlands) or the information-seeking task (n = 17, 9 United
States and 8 the Netherlands). Their mean age was 20.5 years
(SD = 1.8), and 29 were female. Students had no clinical experience and received course credit for their participation.
© 2011 Blackwell Publishing Ltd
Applying causal information
Materials
We derived summary causal diagrams for five DSM-IV-TR [20]
psychiatric disorders from published theories. Each diagram
showed how a root cause branched out in two divergent causal
paths, each containing two intermediate steps, and ultimately
leading to two pairs of symptoms each. See Fig. 1.
The other psychiatric disorders were anorexia nervosa [23,24],
attention-deficit/hyperactivity disorder [25], panic disorder [26]
and borderline personality disorder [27]. See Appendix for the
components of these diagrams. Three additional experienced clinical psychologists who did not participate in the study itself provided feedback on the plausibility of all diagrams, and on how best
to equate the proximal versus diverse symptoms in terms of their
salience and prevalence. We made adjustments to the diagrams
accordingly, while still maintaining as much as possible the integrity of the original theories.
Procedure
All participants received two questionnaires, with a 1-week time
lag in between. All clinicians received and returned the questionnaires by mail. All undergraduate participants were tested in
person, either individually or in groups. For all participants, the
first questionnaire completed was always the control condition
questionnaire, to prevent any possibility of carry-over influence of
causal information. The second questionnaire was always the
causal condition questionnaire, in which we presented causal
information. In sum, clinician and student participants were randomly assigned to complete either the diagnosis task or the
information-seeking task.
In all causal condition questionnaires, participants were shown
the causal diagram information, each divided across four pages
(four chains per disorder) to eliminate spatial distance cues (following [7]). Each chain was labelled with the name of the disorder
and a reference to the publications used for the construction. The
four chains for one disorder were presented in pseudo-randomized
order on separate sheets, such that the same causal chain was never
seen twice consecutively. After having read the four chains for the
first disorder, participants were asked to write a summary paragraph of the information to demonstrate their understanding; for
subsequent disorders they were asked to mentally summarize the
information.
Next, participants assigned to complete the diagnosis task were
asked to imagine two clients, one presenting with two symptoms
resulting from the same causal path (proximal symptoms), and the
other presenting with two symptoms resulting from different
causal paths (diverse symptoms). The two clients always had one
symptom in common. For autism, participants were presented with
the statements: ‘Client N.W. has impairment in social interactions
and problems with communication’ and ‘Client C.F. has a need for
structure and rules and problems with communication.’ As Fig. 1
illustrates, Client N.W. has causally proximal symptoms, and
Client C. F. has causally diverse symptoms. To ensure that participants would not simply discount the presented theories when they
did not agree with their content, we asked them to determine which
of two clients would be most likely, according to the theory they
had read, to have the disorder. Continuing our example, we asked:
3
Applying causal information
L. de Kwaadsteniet et al.
Table 1 Mean percentages of diverse choices
Experts
Novices
Task
Control
Causal
Control
Causal
Diagnosis
Information seeking
47 (19)
36 (22)
77 (38)
77 (26)
29 (24)
32 (21)
59 (42)
47 (42)
Note: Standard deviations are in parentheses.
‘Which client will be more likely, according to Belmonte et al., to
have autism? Please choose one’ [21,22].
Participants assigned to complete the information-seeking task
were asked to imagine a client, presenting with one symptom (for
autism, ‘problems with communication’). We then asked them to
choose one of two symptoms to find out more about assisting in
making a diagnosis (causally proximal choice: ‘has impairment in
social interactions’; causally diverse choice: ‘has a need for structure and rules’). We asked all participants to explain their choices,
and to indicate whether they had ever heard of, read about or
studied the specific theory before.
In the control condition, which was always presented first, all
participants completed the exact same task except without the
causal information, and were asked to make the diagnostic or
information-seeking judgment on the basis of their own opinions.
We counterbalanced the sequence of disorders between participants and we counterbalanced which client, with proximal or
diverse symptoms, was presented first. At the end, we provided
references for the theories we had presented.
Results
In the diagnosis task, a choice for the client presenting with diverse
symptoms was coded as ‘1’ and a choice for the client with proximal symptoms was coded as ‘0’. Similarly, in the informationseeking task, a choice to seek out more information about the
diverse symptom was coded as ‘1’ and a choice to seek out
information about the proximal symptom was coded as ‘0’. We
collapsed the binary data across the five disorders to meet distributional requirements for analyses of variance (ANOVAs). Results
are shown in Table 1.
We conducted a 2 (Condition: causal, control) ¥ 2 (Task: diagnosis, information-seeking) ¥ (Expertise: clinicians, students)
ANOVA, entering country (the Netherlands, United States) as a
covariate, at the a = 0.05 level. We found the critical main effect of
Condition, such that our participants across the board showed a
significant preference for diversity across the diagnosis and
information-seeking tasks [F(1,57) = 10.7; P < 0.01; h2 = 0.16].
The means ran in the same direction for all five disorders. There
was also a main effect of Expertise [F(1,57) = 8.46; P < 0.01;
h2 = 0.13], such that clinicians made more diverse choices overall,
across Condition and Task. Paired sample t-tests showed that the
relative preference for diversity reached significance for both clinicians [t(28) = -5.97; P < 0.001] and students [t(32) = -2.72;
P = 0.01], across tasks. There were no significant interactions and
no other main effects (all Ps > 0.10).
Discussion
Experienced clinical psychologists and undergraduate students,
when asked to presume causal hierarchical information to be true,
4
judged causally diverse symptoms to be more indicative of a
diagnosis and more useful in making a diagnosis than causally
proximal symptoms. Specifically, they judged a client showing
causally diverse symptoms to be more likely to have the psychiatric disorder in question than a client showing causally proximal
symptoms. Also, they preferred to seek out diverse rather than
proximal evidence to better inform a diagnostic judgment. Thus,
both groups seemed to use strategies that have been argued to be
rational and scientific, according to philosophical and Bayesian
analyses.
These findings are not trivial, as many studies have shown that
experts use different strategies than novices or lay people in
closely related category induction tasks. Domain novices or lay
people may prefer causally diverse evidence because this allows
them to make inferences about the probability of a root cause,
when they have no further information or knowledge about conditional probabilities [8,15,16]. Domain experts, in contrast, may
have other core knowledge available to infer the probability of a
root cause given specific evidence [18,28]. Our study is a first look
at how clinicians who accept a new causal theory might be
expected to use that information in diagnostic reasoning. Of
course, how this information may then interact with previous
knowledge and alternative reasoning strategies, and how clinicians
might apply new causal information that they do not fully accept
(e.g. examining to what degree they may ignore it, or even reason
in opposition to it), remains to be studied more fully in further
research. Our work indicates how clinicians use causal structure,
all else held equal.
Nearly all of the participants in our study indicated that they
had not previously known about the causal theories we presented
from the clinical science literature. The justifications that participants gave for their choices (reported in [29]) also overwhelmingly indicated that participants were influenced by causal
structure. Additional work will be needed to test whether clinicians would apply such new causal information spontaneously in
practice, when causal structure is not primed as it was in our
study. As discussed earlier, however, a number of studies have
shown that clinicians’ judgments are reliably and systematically
predicted from their causal beliefs [2–4,30]. As our work currently stands, we conclude that it suggests clinicians and even
students are likely to apply theoretical work on psychiatric conditions to diagnostic reasoning in a way that is normative and
rational. To the extent that clinicians are aware of and accept
theoretical models of psychiatric conditions (and to the degree
that those theoretical models are in fact accurate), these models
may be expected to have a relatively direct, positive impact on
clinical practice.
Acknowledgements
We thank all participants. We also thank Laurence Claes, Ina van
den Berckelaer-Onnes, Nicole Krol and Nancy Snyder for their
helpful comments on our study materials; Patrick Dempsey and
Jeffrey DiNardo for their help in recruiting participants; Chantal
Berens, Shane Lloyd, Thao Nguyen and Daniel Paulus for their
help with data coding; and Amanda Kelleher for her assistance
with reference formatting.
© 2011 Blackwell Publishing Ltd
L. de Kwaadsteniet et al.
References
1. Shrout, P. E. (2011) Integrating causal analysis into psychopathology
research. In Causality and Psychopathology: Finding the Determinants
of Disorders and Their Cues (eds P. E. Shrout, K. M. Keyes & K.
Ornstein), pp. 3–24. New York: Oxford University Press.
2. Ahn, W., Proctor, C. C. & Flanagan, E. H. (2009) Mental health
clinicians’ beliefs about the biological, psychological, and environmental bases of mental disorders. Cognitive Science, 33, 147–182.
3. De Kwaadsteniet, L., Hagmayer, Y., Krol, N. & Witteman, C. (2010)
Causal client models in selecting effective interventions: a cognitive
mapping study. Psychological Assessment, 22 (3), 581–592.
4. Kim, N. S. & Ahn, W. (2002) Clinical psychologists’ theory-based
representations of mental disorders predict their diagnostic reasoning
and memory. Journal of Experimental Psychology: General, 131,
451–476.
5. Yopchick, J. E. & Kim, N. S. (2009) The influence of causal information on judgements of treatment efficacy. Memory & Cognition, 37 (1),
29–41.
6. Morton, J. (2004) Understanding Developmental Disorders: A Causal
Modelling Approach. Malden, MA: Blackwell Publishing.
7. Kim, N. S. & Keil, F. C. (2003) From symptoms to causes: diversity
effects in diagnostic reasoning. Memory & Cognition, 31, 155–165.
8. Kim, N. S., Yopchick, J. E. & de Kwaadsteniet, L. (2008) Causal
diversity effects in information seeking. Psychonomic Bulletin &
Review, 15, 81–88.
9. Osherson, D. N., Wilkie, O., Smith, E. & López, A. (1990) Categorybased induction. Psychological Review, 97, 185–200.
10. López, A. (1995) The diversity principle in the testing of arguments.
Memory & Cognition, 23, 374–382.
11. Spellman, B. A., López, A. & Smith, E. E. (1999) Hypothesis testing:
strategy selection for generalising versus limiting hypotheses. Thinking & Reasoning, 5, 67–91.
12. Heit, E., Hahn, U. & Feeney, A. (2005) Defending diversity. In Categorization Inside and Outside the Laboratory (eds W. Ahn, R. L.
Goldstone, B. C. Love, A. B. Markman & P. Wolff), pp. 87–99.
Washington, DC: American Psychological Association.
13. Popper, K. R. (1963) Conjectures and Refutations: The Growth of
Scientific Knowledge. London: Routledge.
14. Heit, E. (1998) A Bayesian analysis of some forms of inductive reasoning. In Rational Models of Cognition (eds M. Oaksford & N.
Chater), pp. 248–274. Oxford: Oxford University Press.
15. Horwich, P. (1982) Probability and Evidence. Cambridge: Cambridge
University Press.
© 2011 Blackwell Publishing Ltd
Applying causal information
16. Kincannon, A. & Spellman, B. A. (2003) The use of category and
similarity information in limiting hypotheses. Memory & Cognition,
31, 114–132.
17. Bailenson, J. N., Shum, M. S., Atran, S., Medin, D. L. & Coley, J. D.
(2002) A bird’s eye view: biological categorization and reasoning
within and across cultures. Cognition, 84, 1–53.
18. Medin, D. L., Coley, J. D., Storms, G. & Hayes, B. K. (2003) A
relevance theory of induction. Psychonomic Bulletin & Review, 10,
517–532.
19. Proffitt, J. B., Coley, J. D. & Medin, D. L. (2000) Expertise and
category-based induction. Journal of Experimental Psychology:
Learning, Memory, and Cognition, 26, 811–824.
20. American Psychiatric Association (2000) Diagnostic and Statistical
Manual of Mental Disorders, 4th Edn, Text Revision. Washington,
DC: American Psychiatric Association.
21. Belmonte, M. K., Allen, G., Beckel-Mitchener, A., Boulanger, L. M.,
Carper, R. A. & Webb, S. J. (2004) Autism and abnormal development
of brain connectivity. Journal of Neuroscience, 24 (42), 9228–9231.
22. Belmonte, M. K., Cook, E. H. Jr, Anderson, G. M., et al. (2004)
Autism as a disorder of neural information processing: directions
for research and targets for therapy. Molecular Psychiatry, 9, 646–
663.
23. Fairburn, C. G., Cooper, Z. & Shafran, R. (2003) Cognitive behaviour
therapy for eating disorders: a ‘transdiagnostic’ theory and treatment.
Behaviour Research and Therapy, 41, 509–528.
24. Fairburn, C. G., Shafran, R. & Cooper, Z. (1999) A cognitive behavioural theory of anorexia nervosa. Behaviour Research and Therapy,
37, 1–13.
25. Sonuga-Barke, E. J. S. (2002) Psychological heterogenity in AD/HD –
a dual pathway model of behaviour and cognition. Behavioural Brain
Research, 130, 29–36.
26. Ninan, T. N. & Dunlop, B. W. (2005) Neurobiology and etiology of
panic disorder. Journal of Clinical Psychiatry, 66, 3–7.
27. Mauchnik, J., Schmahl, C. & Bohus, M. (2005) New findings in the
biology of borderline personality disorder. Directions in Psychiatry,
25, 197–215.
28. Murphy, G. L. (2002) The Big Book of Concepts. Cambridge, MA:
The MIT Press.
29. De Kwaadsteniet, L. (2009) Clinicians as Mechanics? Causal Reasoning in Clinical Judgment and Decision Making. PhD dissertation,
Radboud University Nijmegen.
30. Proctor, C. & Ahn, W. (2007) The effect of causal knowledge on
judgments of the likelihood of unknown features. Psychonomic Bulletin & Review, 14 (4), 635–639.
5
Applying causal information
L. de Kwaadsteniet et al.
Appendix: Components of the psychiatric disorder stimuli
Disorder
Borderline personality
disorder
Components
Anorexia nervosa
ADHD
Panic disorder
Root cause
Too severe dietary
restrictions due to an
extreme need for
self-control
Breaking of (too) strict
dietary rules; perceived
failure
Inhibitory dysfunction due
to alterations in prefrontal
control circuits
Specific trigger activates
malfunctioning amygdala
Interaction of genetic
predisposition and
environmental stressors
Cognitive dysregulation;
poor planning and
working memory
functioning
Amygdala sends overly
strong signal to lateral
hypothalamus; overactive
sympathetic nervous
symptom
Chills; Sweating
Low serotonin activity; lack
of inhibitory influence
over limbic system
Intermediate causes 1
Terminal effects 1
Intermediate causes 2
Terminal effects 2
Patient symptoms
Presenting
Diverse
Proximal
Negative self-evaluation;
abuse of laxatives
Severely low caloric intake;
state of starvation
Failure to complete tasks;
careless in daily activities
Behavioural dysregulation;
impulsiveness
Preoccupied with food and
eating; social withdrawal
Risky behaviour; interrupts
others frequently
Pounding heart; chest pain
Abuse of laxatives
Careless in daily activities
Sweating
Social withdrawal
Negative self-evaluation
Interrupts others frequently
Failure to complete tasks
Chest pain
Chills
Amygdala misdirects signal
to the locus caeruleus;
enhanced excitability
Outbursts of anger;
Impulsivity
Increased amygdala
activation;
Neurochemically based
difficulty regulating
emotions
Fluctuating emotions;
stress-related transient
paranoia
Stress-related transient
paranoia
Outbursts of anger
Fluctuating emotions
Note. A 5th psychological disorder used in this study, autism, is depicted in Fig. 1.
ADHD, attention-deficit/hyperactivity disorder.
6
© 2011 Blackwell Publishing Ltd