Download Making sense of informant disagreement for overanxious disorder

Anxiety Disorders
19 (2005) 193–210
Making sense of informant disagreement
for overanxious disorder
Debra L. Foleya,*, M. Rutterb, A. Angoldc, A. Picklesd,
H.M. Maesa, J.L. Silberga, L.J. Eavesa
a
Department of Human Genetics, Virginia Institute for Psychiatric and Behavioral Genetics,
Virginia Commonwealth University, P.O. Box 980003, Richmond, VA 23298-0003, USA
b
Social, Genetic and Developmental Psychiatry Research Center, Institute of Psychiatry, London, UK
c
Department of Psychiatry, Duke Medical Center, Duke University, Durham, NC, USA
d
School of Epidemiology and Health Science and Center for Census and Survey Research,
University of Manchester, Manchester, UK
Received 29 September 2003; received in revised form 15 January 2004; accepted 25 January 2004
Abstract
A community sample of 2798 8–17-year-old twins and their parents completed a personal
interview about the child’s current psychiatric history on two occasions separated by an
average of 18 months. Parents also completed a personal interview about their own lifetime
psychiatric history at entry to the study. Results indicate that informant agreement for
overanxious disorder (OAD) was no better than chance, and most cases of OAD were based
on only one informant’s ratings. Disagreement about level of OAD symptoms or presence of
another disorder (mostly phobias or depression) accounted for most cases of informant
disagreement: 60% of cases based only on child interview, 67% of cases based only on
maternal interview, and 100% of cases based only on paternal interview. OAD diagnosed only
by maternal interview was also distinguished by an association with maternal alcoholism and
increasingly discrepant parental reports of marital difficulties. Given the substantial overlap
in case assignments for DSM-III-R OAD and DSM-IV GAD, these findings may identify
sources of informant disagreement that generalize to juvenile GAD.
# 2004 Published by Elsevier Inc.
Keywords: Parent report; Child report; Child anxiety; Anxiety disorders; Anxiety
*
Corresponding author. Tel.: þ1-804-828-8755; fax: þ1-804-828-1471.
E-mail address: [email protected] (D.L. Foley).
0887-6185/$ – see front matter # 2004 Published by Elsevier Inc.
doi:10.1016/j.janxdis.2004.01.006
194
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
1. Introduction
Disagreement among parents and children about presence of juvenile emotional disorders in community settings is common (Achenbach, McConaughy, &
Howell, 1987; Choudhury, Pimentel, & Kendall, 2003; Foley et al., 2004; Grills &
Ollendick, 2003; Jensen et al., 1999; Klein, 1991) and routinely dealt with by
combining different informants ratings using an or-rule at either a symptom or a
diagnostic level (Bird, Gould, & Staghezza, 1992).
The utility of an or-rule is predicated on the assumption that all positive reports
are valid, and the or-rule represents an attempt to mimic clinical decision making
by integrating all known sources of information. Using an or-rule increases the
estimated prevalence of juvenile disorders, and by implication likely service
needs in the community, but an uncritical use of an or-rule sheds no light on the
relevance of informant disagreement or the accuracy of case identification in
community settings.
There are several possible explanations for informant disagreement. Each
informant may contribute largely unique but equally valid information about the
subject’s psychiatric status. If so, the application of an or-rule is justified. Adult
informants may, however, provide biased ratings in association with their own
psychiatric histories or other characteristics (Chapman, Mannuzza, Klein, & Fyer,
1994; Kendler et al., 1991) and this may yield juvenile diagnoses that are unrelated to
the child’s true status. Multiple informants may also conceptualize a child’s behavior
differently because they apply variable thresholds, standards or time frames for
identifying a deviation from normal functioning. Discrepant reporting may also
occur if a child’s behavior is only expressed in certain situations (Achenbach
et al., 1987), or if parents are not equally exposed to their child’s behavior.
This study examined the basis for informant disagreement for overanxious
disorder (OAD) of childhood in a community setting. Anxiety, especially generalized worry, is a phenotype which is largely internally experienced, but which can
be observable if children confide in others about their worries or if their worries
impact on their functioning. The utility of the or-rule was evaluated using data
collected by personal interview with children and parents enrolled with a
population-based, twin-family study to characterize: (1) variation in the 3-month
prevalence of OAD by sex, age and pubertal status, based on interview with each
informant and using a symptom and diagnosis-based or-rule, (2) agreement
between informants about the presence of OAD, and (3) the correlates, outcomes
and distinguishing characteristics of OAD diagnosed by interview with only one
informant.
2. Method
Subjects were participants in The Virginia Twin Study for Adolescent Behavioral Development (VTSABD) (Eaves et al., 1997). Recruitment and assessment
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
195
of the 1412 twin-families who comprised Time 1 of the VTSABD was described
in detail elsewhere (Hewitt et al., 1997; Meyer et al., 1996). The VTSABD was
conceived as an epidemiological study of children and as an epidemiological
study of twins for genetically informative analysis. In this study VTSABD twins
are treated as an epidemiological sample of children, and appropriate adjustment
for the correlation between co-twins was made in the data analyses. Juvenile twin
subjects were eligible for the current study if the twin, mother or father was
interviewed about the twin’s history of OAD at Time 1. The eligible sample
comprised 2798 twin children: 1291 boys and 1507 girls. Juvenile subjects at
Time 1 were 8–17 years of age (mean ¼ 11:57, S:D: ¼ 2:98), 54% female; 100%
Caucasian and 54% lived within an urban area. Using census tract data to estimate
regional socioeconomic characteristics, subjects resided in areas where, on
average, 26% of adults were college educated, and the mean (median) family
income was $45,520.
Trained field workers interviewed twins and their parents in the family home.
Interviewers held a Master’s degree in Social Work or an equivalent professional
degree, or had extensive experience in psychiatric interviewing. Interviewer
training consisted of three weeks of residential instruction, and all field interviews
were audio taped and reviewed by senior monitors. Regular meetings were
conducted to avoid drift in use of the interview or other aspects of the protocol.
A residential meeting was held annually to ensure that the assessment protocol
was being implemented in a standardized manner (Hewitt et al., 1997).
Twin subjects were personally interviewed about their psychiatric history
using the Child and Adolescent Psychiatric Assessment-Children’s version
(CAPA-C; Angold & Costello, 2000; Angold et al., 1995) on two occasions
separated by an average of 18 months. These two occasions of measurement are
referred to as Time 1 and Time 2. On each occasion twins were randomly assigned
to different interviewers, and each twin was interviewed simultaneously in
separate areas of the family home. Immediately following the interviews with
the twins, each parent was subsequently interviewed about each twin’s psychiatric
history during the past 3 months using the Child and Adolescent Psychiatric
Assessment-Parent’s version (CAPA-P) at Time 1 and Time 2. Different field
workers interviewed each parent, and the twin about whom the parent was first
interviewed was randomly assigned. Agoraphobia (fear of open spaces, going
out in crowded spaces or using public transportation), attention deficit hyperactivity disorder (ADHD, surveyed only by the CAPA-P), conduct disorder
(CD), depression [minor (DSM-IV) or major], overanxious disorder (OAD),
oppositional-defiant disorder (ODD), and separation anxiety disorder (SAD)
were assessed following the DSM-III-R (APA, 1987). Any phobia referred to
a composite phobia diagnosis (any simple or specific phobia). The 1–11-day testretest reliability of the total number of rated CAPA symptoms was .60 for conduct
disorder, .88 for major depression, .50 for oppositional-defiant disorder, .80 for
overanxious disorder, and .63 for separation anxiety disorder (Angold & Costello,
1995).
196
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
The symptomatic criteria for OAD required at least four of the following
symptoms: (1) excessive or unrealistic worry about future events; (2) excessive or
unrealistic concern about the appropriateness of past behavior; (3) excessive or
unrealistic concern about competence in one or more areas, e.g., athletic,
academic, social; (4) somatic complaints, such as headaches or stomach aches,
for which no physical basis can be established; (5) marked self-consciousness (6)
excessive need for reassurance about a variety of concerns (7) marked feelings of
tension or inability to relax.
The CAPA assessed 15 areas of psychosocial impairment associated with
specific symptom clusters (Angold et al., 1995; Pickles et al., 2001). Each area of
disability was rated on a three-point scale (for no, partial or severe impairment),
and ratings are summed to estimate overall level of associated impairment. An
impairment score of 1 or higher was used here to define OAD with at least
minimal associated impairment.
Four CAPA-C items were used to assess pubertal status. Body hair, skin
changes, voice breaking and facial hair were used to estimate puberty in boys, and
body hair, skin changes, breast development and menstruation were used to
estimate puberty in girls. Items ratings (0: no; 1: yes, barely; 2: yes, definitely; and
3: development complete) were summed to estimate current level of pubertal
development. The mean puberty score was 2.92 (S:D: ¼ 2:89, range 0–11) in boys
and 3.78 (S:D: ¼ 3:13, range 0–10) in girls. The Pearson correlation coefficient
between age in years and puberty score was r ¼ :82 (P ¼ :0001) for boys and
r ¼ :85 (P ¼ :0001) for girls.
Parents were interviewed about their own lifetime psychiatric history after they
completed the CAPA-P at Time 1. The following parental disorders were
diagnosed following the DSM-III-R: alcoholism, anti-social personality disorder
(ASPD), generalized anxiety disorder, major depressive disorder, panic disorder,
social phobia and simple phobia. The parental interview was a modified version of
the SCID, described in detail elsewhere (Foley et al., 2001).
Parents completed a questionnaire that included the Dyadic Adjustment Scale
(DAS) immediately following the child and parent interviews. Dyadic satisfaction
during the previous year was surveyed and scored as outlined by Spanier (1976),
with higher scores reflecting a relatively higher level of satisfaction. The dyadic
satisfaction items in the DAS survey the frequency of regret associated with the
marriage, thoughts of separation/divorce, leaving the house after a fight, quarrelling or irritation with spouse, positive interactions with spouse, degree of
happiness with the marital relationship and feelings about the future of the
relationship. The mean level of dyadic satisfaction rated by mothers and fathers in
the VTSABD was 33.79 (S:D: ¼ 3:95, range ¼ 3–42, N ¼ 1165) and 34.19
(S:D: ¼ 3:52, range ¼ 17–45, N ¼ 965), respectively (Wilcoxon 2-sided Test,
P ¼ :06). Maternal ratings, paternal ratings and the absolute difference between
maternal and paternal ratings of dyadic satisfaction were used to characterize the
association between parental perceptions of marital harmony and informant
disagreement for OAD.
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
197
2.1. Data analysis
Variation in prevalence between groups was evaluated using a two-tailed
Chi-square test. Fisher’s exact test was used when the expected number of
participants in any cell of the contingency table was <5. Logistic regression
was used to estimate age effects, and pubertal effects minus age effects to test if
pubertal developmental that was atypical for (or independent of) chronological
age was associated with risk for OAD. Logistic regression was also used to
estimate the association between OAD and other diagnoses or characteristics, and
informant effects were evaluated following the method described by Fitzmaurice,
Laird, Zahner, and Daskalakis (1995). Regression analyses were performed
within PROC GENMOD in SAS version 6.12 to adjust for the correlation
between twins (SAS Institute Inc., 1995).
3. Results
3.1. The 3-month prevalence of over anxious disorder
The estimated 3-month prevalence of OAD ranged from 0.25 and 8.40%
depending on the handling of multiple informant data (Table 1). Interviews with
children or mothers identified more cases of OAD than interviews with fathers.
Using an or-rule increased the prevalence of OAD, but cases diagnosed by child
interview and confirmed by parent interview were very rare. The prevalence of
OAD dropped by approximately 25% when at least minimal impairment in one
area of functioning was required for a diagnosis, irrespective of informant. All
cases diagnosed by child interview, and confirmed by parent interview, had
associated impairment. Girls were more likely to be diagnosed with OAD than
boys, but the female to male ratio was highest for cases diagnosed by child
interview. Prevalence of OAD increased with age, but age trends were only
significant in boys. Pubertal development that was advanced for chronological age
was more strongly associated with risk for OAD in girls than age in years.
3.2. Informant agreement
Informant agreement for OAD was no better than chance (Table 2). Agreement
was not improved when OAD diagnoses were associated with impairment, and,
with few exceptions, agreement for individual symptoms was generally no better
than agreement about the presence of disorder.
3.3. Making sense of informant disagreement
Informant disagreement was investigated using children for whom three
informants completed the OAD module from the CAPA at Time 1 (Table 3).
198
Table 1
Three-month prevalence of overanxious disorder
Diagnosis
Source
Total
Boys
Girls
Age
(OR)
Puberty
(OR)
Prevalence
(%)
Age
(OR)
Puberty
(OR)
Prevalence
(%)
Age
(OR)
Puberty
(OR)
OAD
Symptom or-rule
Diagnostic or-rule
Child
Mother
Father
Child and parent
8.40
7.18
3.69b
3.47c
1.43
0.25
1.08
1.08
1.12
1.08
1.04
1.08
1.15a
1.15a
1.20a
1.08
1.20b
1.25
5.96
4.80
2.03
2.62c
1.15
0.15
1.17a
1.20a
1.37a
1.10
1.12
1.54
1.07
1.03
1.05
1.05
1.03
1.19
10.48
9.22
5.10b
4.21c
1.68
0.33
1.09
1.09
1.10
1.13
1.01
1.04
1.14a
1.14a
1.19a
1.04
1.29
1.20
1.75a
1.92a
2.51a
1.60a
1.46
2.20
OAD with impairment
Symptom or-rule
Diagnostic or-rule
Child
Mother
Father
Child and parent
6.02
4.86
2.54b
2.30c
0.94
0.25
1.08
1.09
1.06
1.14
1.15
1.08
1.15a
1.14a
1.31a
1.00
1.09
1.25
4.04
3.25
1.38
1.86
0.84
0.15
1.24a
1.23a
1.24
1.21
1.35
1.54
1.05
1.05
1.26
0.98
0.92
1.19
7.71
6.24
3.51b
2.68c
1.03
0.33
1.08
1.08
1.08
1.15
1.01
1.04
1.14a
1.13a
1.24a
0.97
1.24
1.20
1.90a
1.92a
2.54a
1.44
1.22
2.20
OR: Odds ratios estimated using a type 1 (sequential) logistic regression analysis with age entered before pubertal status; OAD: subject met criteria for DSM-III-R
OAD, with or without associated impairment in functioning; OAD with impairment: subject met criteria for DSM-III-R OAD and had at least partial disability in at
least one area of psychosocial functioning.
a
Significant sex difference or age or pubertal effect at P ¼ :05.
b
Prevalence of OAD is significantly higher based on child report versus paternal report.
c
Prevalence of OAD is significantly higher at P :05 based on interview with mother versus father.
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
Prevalence
(%)
Girl:boy
ratio
Table 2
Informant agreement for individual symptoms and a diagnosis of overanxious disorder
Child’s sex
Child and father
Mother and father
Kappa
95% CI
Kappa
95% CI
Kappa
95% CI
0.07
0.01
0.05
0.02
0.10
0.06
0.002
0.05
0.09
0.01,
0.02,
0.03,
0.05,
0.03,
0.04,
0.06,
0.05,
0.04,
0.02
0.008
0.05
0.02
0.05
0.11
0.11
0.07
0.13
0.06,
0.01,
0.04,
0.06,
0.02,
0.04,
0.01,
0.08,
0.11,
0.08
0.21
0.09
0.11
0.09
0.007
0.13
0.14
0.20
0.03, 0.18
0.04, 0.47
0.02, 0.20
0.01, 0.23
0.01, 0.17
0.01, 0.003
0.006, 0.26
0.04, 0.32
0.04, 0.45
0.08
0.01
0.03
0.19
0.11
0.10
0.03
0.07
0.07
0.0008, 0.16
0.015, 0.004
0.04, 0.10
0.07, 0.31
0.04, 0.18
0.02, 0.22
0.04, 0.10
0.03, 0.16
0.04, 0.18
0.06
0.009
0.05
0.13
0.15
0.04
0.002
0.05
0.04
0.02, 0.14
0.01, 0.004
0.02, 0.13
0.03, 0.23
0.07, 0.22
0.06, 0.14
0.06, 0.06
0.04, 0.14
0.06, 0.14
0.15
0.009
0.12
0.09
0.17
0.17
0.05
0.15
0.22
0.10
0.004
0.14
0.11
0.13
0.25
0.24
0.22
0.36
Female
Worry about future
Worry about past
Worry about competence
Self-consciousness
Excessive need for reassurance
Physical symptoms
Nervous tension
Overanxious disorder
Overanxious disorder with impairment
0.13
0.07
0.07
0.06
0.11
0.05
0.16
0.07
0.04
0.06, 0.21
0.04, 0.18
0.004, 0.15
0.01, 0.13
0.05, 0.17
0.03, 0.14
0.08, 0.25
0.001, 0.16
0.03, 0.12
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
Male
Worry about future
Worry about past
Worry about competence
Self-consciousness
Excessive need for reassurance
Physical symptoms
Nervous tension
Overanxious disorder
Overanxious disorder with impairment
Child and mother
199
200
Informant per child
Child, mother, father
Child, mother
Child, father
Mother, father
Child
Mother
Father
N
1835
622
124
54
101
52
10
Overanxious disorder diagnosed by interview with
Child only
Mother only
Father only
Child þ
mother
Child þ
father
Mother þ
father
Child þ mother
þ father
Symptom
or-rule only
N
%
N
%
N
%
N
%
N
%
N
%
N
%
N
%
52
25
3
–
7
–
–
2.83
4.02
2.42
–
6.93
–
–
51
24
–
1
–
1
–
2.78
3.86
–
1.85
–
1.92
–
18
–
3
1
–
–
0
0.98
–
2.42
1.85
–
–
0.00
4
4
–
–
–
–
–
0.22
0.65
–
–
–
–
–
2
–
1
–
–
–
–
0.11
–
0.81
–
–
–
–
2
–
–
1
–
–
–
0.11
–
–
1.85
–
–
–
1
–
–
–
–
–
–
0.05
–
–
–
–
–
–
27
7
0
0
–
–
–
1.58
1.23
0.00
0.00
–
–
–
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
Table 3
Three-month prevalence of overanxious disorder by the number of informants available per child
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
201
Table 4
Sub-threshold symptoms rated by other informants in association with one-informant only OAD
Informant
for OAD
N
Informant who
didn’t confirm
diagnosis
% children with sub-threshold symptoms rated by other
informant
0 symptoms
1 symptom
2 symptoms
3 symptoms
Child
52
Mother
Father
55.77
73.08
30.77
17.31
1.92
5.77
11.54
3.85
Mother
51
Child
Father
78.43
70.59
13.73
13.73
5.88
9.80
1.96
5.88
Father
18
Child
Mother
61.11
72.22
22.22
11.11
0.00
5.56
16.67
11.11
This sub-sample (66% of the total) was used to characterize OAD diagnosed by
interview with only one informant (77% of all OAD cases in this sub-sample),
and, where applicable, only by a symptom or-rule (17% of all OAD cases in this
sub-sample). Unless otherwise indicated, the reference group in the analyses
reported below were children without a diagnosis of OAD (by child, maternal or
paternal interview, or by a symptom or-rule).
3.4. Disagreement about the number of symptoms versus disagreement
about the presence of any symptoms
Between 56 and 78% of children diagnosed with OAD by interview with only
one informant had no symptoms of OAD according to the other two informants
(Table 4). OAD diagnosed only by interview with mothers was less likely to be
associated with a sub-threshold level of symptoms rated by the other two
informants than OAD diagnosed only by interview with children or fathers.
3.5. Disagreement about the type of disorder versus disagreement
about the presence of any disorder
Cross-informant cross-disorder odds, adjusted for agreement about the
presence of a second disorder, were estimated to distinguish diagnostic disagreement from agreement about the presence of a second disorder. For example, when
the odds of OAD diagnosed only by child interview and depression diagnosed by
maternal interview were estimated, depression cases diagnosed by maternal
but not by child interview were coded 1 (1 ¼ disagreement ¼ mother rated
depression as present, child rated depression as absent), and depression cases
diagnosed by both child and maternal interview, or by neither child and maternal
interview, were coded 0 (0 ¼ agreement ¼ mother and child agreed depression
was absent or mother and child agreed depression was present), and so on. These
analyses therefore estimated the odds of diagnostic disagreement associated with
OAD diagnosed by interview with only one informant.
202
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
OAD diagnosed only by child interview was associated with a significantly
increased odds of mother-rated depression and father-rated agoraphobia that was
not confirmed by the child (Table 5). Mother–child and father–child disagreement
about presence of OAD versus any other disorder accounted for 11.54% (6/52)
and 7.69% (4/52), respectively, of OAD cases diagnosed only by child interview.
This was not significantly different from the level of parent–child disagreement
found among children without a diagnosis of OAD (11.74%, P ¼ 1:00 and 8.10%,
P ¼ 1:00, respectively).
OAD diagnosed only by maternal interview was associated with a significantly increased odds of child-rated conduct disorder, child-rated depression and father-rated (any) phobia that was not confirmed by mothers (Table 5).
Mother–child and mother–father disagreement about the presence of OAD
versus any other disorder accounted for 29.41% (15/51) and 13.73% (7/51)
of OAD cases diagnosed only by maternal interview. This was significantly different from the level of mother–child disagreement found among
children without a diagnosis of OAD (16.21%, P ¼ :02 and 7.75%, P ¼ :12,
respectively).
OAD diagnosed only by paternal interview was associated with a significantly
increased odds of child-rated agoraphobia, child-rated depression and motherrated depression that was not confirmed by fathers (Table 5). Father–child and
father–mother disagreement about the presence of OAD versus any other disorder
accounted for 38.89% (7/18) and 22.22% (4/18) of OAD cases diagnosed only by
paternal interview. This was significantly different from the level of father–child
disagreement found among children without a diagnosis of OAD (17.10%,
P ¼ :02 and 12.51%, P ¼ :27, respectively).
The pattern of informant disagreement therefore distinguished OAD diagnosed
only by child interview, whereas both the pattern and the prevalence of informant
disagreement distinguished OAD diagnosed only by parent interview.
3.6. Parental characteristics associated with informant disagreement
Only two parental characteristics were significantly associated with informant
disagreement for OAD (Table 6). Maternal alcoholism and increasingly discrepant parental reports of marital satisfaction were both more strongly associated
with OAD diagnosed only by interview with mothers than OAD diagnosed only
by interview with children.
3.7. Implications for comorbidty
Examination of the pattern of comorbidity estimated for OAD diagnosed by
interview with only one informant, within and across time, indicated that fathers
were more likely than children to rate SAD or social phobia at Time 1 in
association with OAD at Time 1 (Table 7). Fathers were also more likely than
children to rate agoraphobia at Time 2 in association with OAD at Time 1.
Table 5
Diagnostic disagreement associated with informant disagreement for overanxious disorder
Diagnosis
Time
OAD diagnosed only by
maternal interview
OAD diagnosed only by
paternal interview
Informant for other diagnosis
that is not confirmed by child
Informant for other diagnosis
that is not confirmed by mother
Informant for other diagnosis
that is not confirmed by father
Mother (OR)
Child (OR)
Child (OR)
Agoraphobia
1
2
0.00
0.00
Attention deficit hyperactivity disorder
1
2
0.00
2.07
Conduct disorder
1
2
Minor or major depression
Father (OR)
*
10.95
0.00
Father (OR)
*
17.35
0.00
Mother (OR)
2.78
5.57
0.00
0.00
0.00
0.00
1.35
6.17*
–
–
0.00
0.00
–
–
0.00
0.00
1.91
0.00
2.04
6.56
4.96*
1.44
2.22
0.00
2.41
0.00
0.00
0.00
1
2
3.89*
0.00
0.00
5.46
3.68*
2.49
0.00
4.77
6.64*
4.23
7.64*
0.00
Oppositional defiant disorder
1
2
1.71
2.72
0.00
0.00
0.00
7.57*
0.00
4.77
0.00
0.00
0.00
0.00
Separation anxiety disorder
1
2
0.00
0.00
0.00
0.00
1.27
0.00
5.57
11.16*
1.59
0.00
5.42
0.00
Any specific or social phobia
1
2
0.26
0.78
0.43
2.38
1.86
1.55
2.81*
1.92
2.41
2.53
1.54
1.06
203
Odds are adjusted for informant agreement about the presence of a second disorder, see text for details; reference group is children with no diagnosis of OAD using a
symptom or-rule or child or parent CAPA.
*
P :05.
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
OAD diagnosed only by
child interview
204
Table 6
Bivariate odds of OAD in association with parental history of psychiatric disorder
Parental disorder
Parent
Odds of OAD diagnosed by interview with
Mother only
(N ¼ 51) (OR)
Father only
(N ¼ 18) (OR)
Symptom or-rule
only (N ¼ 27) (OR)
Informant differences
Mother > child P ¼ .04
Alcoholism
Mother
Father
0.92
0.64
4.52*
0.53
0.00
1.01
3.68*
0.95
Anti-social personality disorder
Mother
Father
0.00
0.52
8.53*
0.00
0.00
0.00
0.00
0.00
Generalized anxiety disorder
Mother
Father
0.96
1.38
1.77
1.31
1.01
3.42*
0.80
3.23*
Major depression
Mother
Father
2.07*
1.73
1.54
1.74
3.07*
3.27*
2.17*
1.65
Panic disorder
Mother
Father
2.08
1.37
1.21
0.00
0.00
0.00
1.85
0.00
Simple phobia
Mother
Father
0.56
1.04
0.97
0.16
0.78
0.40
0.86
1.98
Social phobia
Mother
Father
0.32
0.89
1.30
1.68
0.00
1.27
0.96
1.47
Marital dissatisfaction
Mother
Father
[M F]
0.97
0.99
0.93
0.91*
0.95
1.15*
0.98
0.90
0.99
0.90*
0.91*
1.25*
[M F] ¼ the absolute difference between mother’s and father’s ratings of marital satisfaction.
*
P :05.
Mother > child P ¼ .04
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
Child only
(N ¼ 52) (OR)
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
205
Table 7
Bivariate odds of comorbidity associated with one-informant only OAD—within-informant effects
Diagnosis
Time
Informant for OAD and
other diagnoses
Child
Agoraphobia
Attention deficit hyperactivity
disorder
Conduct disorder
Minor or major depression
Oppositional defiant disorder
Separation anxiety disorder
Any specific or social phobia
Overanxious disorder
*
1
2
1
2
1
2
1
2
1
2
1
2
1
2
2
Mother
*
6.56
14.57*
–
–
2.25*
0.00
10.30*
5.35*
4.37*
0.00
6.47*
5.25*
5.22*
2.75*
5.19*
*
11.36
7.91
1.94
0.00
2.64
4.59*
11.24*
6.37*
8.79*
5.86*
8.60*
4.31*
5.21*
2.07*
9.37*
Informant
differences
Father
0.00
37.69*
0.00
0.00
3.17
4.44
17.27*
6.67*
19.02*
5.46
25.75*
0.00
2.78*
6.94*
15.49*
Father > child P ¼ .09
Father > child P ¼ .04
P :05.
4. Discussion
OAD was replaced by the revised criteria for generalized anxiety disorder
(GAD) in the transition from DSM-III-R (APA, 1987) to DSM-IV (APA, 1994)
and DSM-IV-TR (APA, 2000). Like OAD, GAD is characterized by pervasive and
uncontrollable worries that may include concerns about performance, personal
relationships, physical health, or ruminations about future or past behavior
(APA, 2000). In addition to excessive worries, only one of the six symptomatic
(somatic) criteria for DSM-IV GAD is required to diagnose GAD in children. Any
four of seven symptomatic criteria were required to diagnose DSM-III-R OAD,
but three symptoms related to worries and one symptom was somatic complaints.
The criteria for childhood OAD and GAD are therefore similar, and a comparison
of case assignments in a clinical study showed no significant difference using
DSM-III-R OAD and DSM-IV GAD (Kendall & Warman, 1996). Agreement
between DSM-III-R and DSM-IV for parent diagnostic interviews was 98% and
agreement for child diagnostic interviews was 93%, supporting generalizations
from OAD research to juvenile GAD defined by the DSM-IV (Kendall &
Pimentel, 2003). Overlap between GAD and OAD was lower, however, in an
epidemiological study that examined comorbidity across time in a large sample of
youth aged between 9 and 16 years (Costello, Egger and Angold, 2003). In that
study, 34% of children with GAD did not meet criteria for OAD. Given the low
level of informant agreement for individual symptoms of GAD (Kendall &
Pimentel, 2003), and the similarity of case assignments using criteria for each
206
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
disorder, the findings of the present study may identify sources of informant
disagreement for OAD that generalize to juvenile GAD.
Most informant disagreement for OAD in the VTSABD reflected disagreement
about whether the child had sufficient symptoms to meet the diagnostic threshold
for OAD or disagreement about whether the child had OAD or another disorder.
These two sources of disagreement accounted for 60% of OAD cases diagnosed
only by child interview, 67% of OAD cases diagnosed only by maternal interview,
and 100% of OAD cases diagnosed only by paternal interview. Informants mostly
disagreed about the presence of OAD versus depression or phobia, especially
agoraphobia.
Disagreement about presence of generalized worry versus fear of one or more
phobic stimuli is consistent with limited parental exposure to a child’s anxiety, or
ignorance about the source or breadth of the child’s anxiety (generalized worry
versus fear in response to a specific stimulus). Situational specificity has often
been used to defend the validity of parent-, teacher- and child-based diagnoses,
even when informant agreement is very low (Achenbach et al., 1987). Apparent
situational specificity may, however, reflect ignorance of the generalized nature of
some children’s anxiety. Awareness of a child’s anxiety in one setting, but not
other situations in which anxiety also occurred, may logically lead to a misdiagnosis of phobia. It is therefore important to distinguish valid symptom ratings
from invalid diagnoses because the former may reflect relatively accurate but
limited knowledge whereas the latter may reflect inaccurate diagnostic inferences
based on limited knowledge. Although OAD and depression were defined by
different symptoms, it is conceivable that worry and nervous tension may be
mistaken for sadness or irritability. This is likely to be especially important to
consider in future studies of GAD because of overlap between the criteria for
DSM-IV GAD and depression (e.g., difficulty concentrating, see Joormann &
Stober, 1999).
Some forms of informant disagreement could not, of course, be interpreted by
considering the difficulty of distinguishing one emotion from another or the
impact of having only partial knowledge about the circumstances under which a
child felt anxious. For example, OAD diagnosed only by maternal interview was
associated with CD and ODD based on child interview and future CD based on
maternal interviews. Further investigation of the relationship between generalized
anxiety based only on maternal report and risk for externalizing disorders in twins
might clarify the impact on children of a maternal perception that is unconfirmed
by other informants. Twins can be used to test the effects on children of maternal
perceptions unshared by others because of predictions that can be made about the
sources of twin similarity. Twin discordance reflects the impact of unshared
environmental risk factors that make twins different from each other. Twin
concordance reflects the impact of shared (familial) risk factors that makes twins
resemble one another. OAD diagnosed only by maternal report was associated
with an increased odds of child-rated CD. If OAD diagnosed only by maternal
report predicted discordance for child-rated CD then maternal perceptions
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
207
unshared by others may act as an individual-specific environmental risk factor
(e.g., due to differential maternal treatment of twins). If OAD diagnosed only by
maternal report predicted concordance for child-rated CD then maternal perceptions unshared by others may act as a shared (familial) risk factors (e.g., because
OAD diagnosed only by maternal report was strongly associated with maternal
alcoholism and marital difficulties).
Two maternal characteristics were differentially associated with OAD diagnosed only by maternal report and OAD diagnosed only by child report: maternal
alcoholism and increasingly discrepant parental reports of marital satisfaction.
These two characteristics did not differentiate OAD diagnosed only by paternal
report and OAD diagnosed only by child report. Mothers with a history of
alcoholism, and mothers who rate their marital satisfaction differently from their
spouses, may be more sensitive to their child’s true anxiety than either the child or
the father or they may over rate their child’s anxiety as a function of their own
personal or marital difficulties. It is not clear why mothers would be more
sensitive or biased raters in association with a personal history of alcoholism or
marital difficulties, but previously reported associations between parental alcoholism and OAD diagnosed by combining child and parent (usually maternal)
ratings should be interpreted cautiously (e.g., Kuperman, Schlosser, Lidral, &
Reich, 1999). Future efforts to identify putative cases of juvenile anxiety disorder,
including DSM-IV GAD, based only on maternal screening instruments should
also be interpreted cautiously, and special care should be taken with the
differential diagnosis of generalized anxiety, depression and agoraphobia.
The findings of this study suggest that the low level of informant agreement in
many community surveys may often reflect: (1) a hierarchy of informant knowledge (e.g., agreement that a child was anxious but disagreement about how many
anxiety symptoms were present or disagreement about whether the anxiety was
best attributed to a generalized source of worries or to specific phobic triggers);
(2) reasonable misinterpretation of diagnostic status (e.g., mistaking anxiety for
depression); (3) diagnostic error (e.g., mistakenly rating OAD rather than CD); or
(4) variable maternal sensitivity or rating bias (e.g., due to maternal psychiatric
history or marital difficulties). Unscrutinised informant disagreement and an
uncritical application or an or-rule will, however, continue to impact on the
accuracy of case identification in community settings. This, in turn, will impact on
the accuracy of estimated prevalence rates, sex differences, age trends, pubertal
effects, patterns and causes of comorbidity, and patterns and causes of parent–
offspring associations. Finally, and contrary to popular wisdom, mothers may be
more biased raters than fathers.
5. Limitations
Results presented here should be interpreted in light of the following limitations. First, results are based on the CAPA and may not generalize to other
208
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
assessment instruments. Clinician-related inter-rater reliability is not currently
available for the CAPA, and we therefore cannot compare clinician and parental
agreement. Second, the study participants were all Caucasian and results may not
generalize to other ethnic or racial groups. Third, the study participants were all
twins and it is always important to replicate findings based on twins with findings
based on singletons or siblings of different ages. Our findings are nonetheless
relevant for other Caucasian twin studies that use parent and/or child ratings of
twins to estimate genetic and environmental contributions to juvenile anxiety and
associated patterns of comorbidity. Fourth, sex and age effects on informant
agreement were not examined in an effort to maximize statistical power. Fifth,
analyses focussed on parent-based characteristics that may have been associated
with a parental rating bias but child-based characteristics may have also contributed to rating errors and systematic patterns of informant disagreement.
Seventh, it is important to consider the possibility that the statistically significant
findings reported here may reflect chance associations (for a discussion of the
interpretation of P values in the context of multiple statistical tests see Brown &
Russell, 1997; Cohen, 1982; Feild & Armenakis, 1974; Goodman, 1998, 1999;
Rothman, 1990; Savitz & Olshan, 1995; Thompson, 1998). Only an independent
study can determine if any or all of our statistically significant associations reflect
chance effects.
Acknowledgments
This work was supported by grants MH-60324 (Dr. Foley), MH-45268
(Dr. Angold) and MH-57761 (Dr. Eaves) from the U.S. National Institute of
Mental Health, Bethesda, MD and by the Carman Trust for Scientific Research,
Richmond, VA (Dr. Silberg). We acknowledge the contribution of the Virginia
Twin Study for Adolescent Behavioral Development, now part of the Mid-Atlantic
Twin Registry (MATR), to ascertainment of subjects for this study. The MATR
(PI Dr. Linda Corey) has received support from the National Institutes of Health,
and the W.M. Keck, John Templeton and Robert Wood Johnson Foundations.
References
Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioural and
emotional problems: implications of cross-informant correlations for situational specificity.
Psychological Bulletin, 101, 213–232.
American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders
(rev. 3rd ed.) (DSM-III-R). Washington, DC: APA.
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders
(4th ed.) (DSM-IV). Washington, DC: APA.
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders
(4th ed. Text Revision) (DSM-IV-TR). Washington, DC: APA.
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
209
Angold, A., & Costello, E. J. (1995). A test–retest reliability study of child-reported psychiatric
symptoms and diagnoses using the Child and Adolescent Psychiatric Assessment (CAPA-C).
Psychological Medicine, 25, 755–762.
Angold, A., & Costello, E. J. (2000). The Child and Adolescent Psychiatric Assessment (CAPA).
Journal of the American Academy of Child and Adolescent Psychiatry, 39, 39–48.
Angold, A., Prendergast, M., Cox, A., Harrington, R., Simonoff, E., & Rutter, M. (1995). The Child
and Adolescent Psychiatric Assessment (CAPA). Psychological Medicine, 25, 739–753.
Bird, H. R., Gould, M. S., & Staghezza, B. (1992). Aggregating data from multiple informants in
child psychiatry epidemiological research. Journal of the American Academy of Child and
Adolescent Psychiatry, 31, 78–85.
Brown, B. W., & Russell, K. (1997). Methods correcting for multiple testing: operating
characteristics. Statistics in Medicine, 16, 2511–2528.
Chapman, T. F., Mannuzza, S., Klein, D. F., & Fyer, A. J. (1994). Effects of informant mental
disorder on psychiatric family history data. American Journal of Psychiatry, 151, 574–579.
Choudhury, M. S., Pimentel, S. S., & Kendall, P. C. (2003). Childhood anxiety disorders: parent–
child (dis)agreement using a structured interview for the DSM-IV. Journal of the American
Academy of Child and Adolescent Psychiatry, 42, 957–964.
Cohen, P. (1982). To be or not to be: the control and balancing of Type 1 and Type 2 errors in
research. Journal of Evaluation and Program Planning, 3, 247–253.
Costello, E. J., Egger, H. L., & Angold, A. (2003). The developmental epidemiology of anxiety
disorders. In: T. Ollendick & J. March (Eds.), Phobic and Anxiety Disorders in Children and
Adolescents. New York: Oxford University Press, pp.60–90.
Eaves, L. J., Silberg, J. L., Meyer, J. M., Maes, H. H., Simonoff, E., Pickles, A. et al. (1997). Genetics
and developmental psychopathology. 2. The main effects of genes and environment on behavioral
problems in the Virginia Twin of Adolescent Behavioral Development. Journal of Child
Psychology and Psychiatry and Allied Disciplines, 38, 965–980.
Feild, H. S., & Armenakis, A. A. (1974). On the use of multiple tests of significance in psychological
research. Psychological Reports, 35, 427–431.
Foley, D. L., Pickles, A., Simonoff, E., Maes, H. H., Silberg, J. L., Hewitt, J. K. et al. (2001). Parental
concordance and comorbidity for psychiatric disorder and associated risks for current psychiatric
symptoms and disorders in a community sample of juvenile twins. Journal of Child Psychology
and Psychiatry and Allied Disciplines, 42, 381–394.
Foley, D. L., Rutter, M., Pickles, A., Angold, A., Maes, H. M., Silberg, J. L., et al. (2004). Informant
disagreement for separation anxiety disorder. Journal of the American Academy of Child and
Adolescent Psychiatry, 43.
Fitzmaurice, G. M., Laird, N. M., Zahner, G. E., & Daskalakis, C. (1995). Bivariate logistic
regression analysis of childhood psychopathology ratings using multiple informants. American
Journal of Epidemiology, 142, 1194–1203.
Goodman, S. N. (1998). Multiple comparisons, explained. American Journal of Epidemiology, 147,
807–812.
Goodman, S. N. (1999). Toward evidence-based medical statistics. I. The P value fallacy. Annals of
Internal Medicine, 130, 995–1004.
Grills, A. E., & Ollendick, T. H. (2003). Multiple informant agreement and the anxiety disorders
interview schedule for parents children. Journal of the American Academy of Child and
Adolescent Psychiatry, 42, 30–40.
Hewitt, J. K., Eaves, L. J., Silberg, J. L., Rutter, M., Simonoff, E., Meyer, J. M. et al. (1997). Genetics
and developmental psychopathology. 1. Phenotypic assessment in the Virginia Twin of
Adolescent Behavioral Development. Journal of Child Psychology and Psychiatry and Allied
Disciplines, 38, 943–963.
Jensen, P. S., Rubio-Stipec, M., Canino, G., Bird, H. R., Duncan, M. K., Schwab-Stone, M. E. et al.
(1999). Parent and child contributions to diagnosis of mental disorder: are both informants always
necessary? Journal of the American Academy of Child and Adolescent Psychiatry, 38, 1569–1579.
210
D.L. Foley et al. / Anxiety Disorders 19 (2005) 193–210
Joormann, J., & Stober, J. (1999). Somatic symptoms of generalized anxiety disorder from the DSMIV: associations with pathological worry and depression symptoms in a nonclinical sample.
Journal of Anxiety Disorders, 13, 491–503.
Kendall, P. C., & Pimentel, S. S. (2003). On the physiological symptom constellation in youth with
Generalized Anxiety Disorder (GAD). Journal of Anxiety Disorders, 17, 211–221.
Kendall, P. C., & Warman, M. (1996). Anxiety disorders in youth: diagnostic consistency across
DSM-III-R and DSM-IV. Journal of Anxiety Disorders, 10, 452–546.
Kendler, K. S., Silberg, J. L., Neale, M. C., Kessler, R. C., Heath, A. C., & Eaves, L. J. (1991). The
family history method: whose psychiatric history is measured? American Journal of Psychiatry,
148, 1501–1504.
Klein, R. G. (1991). Parent–child agreement in clinical assessment of anxiety and other
psychopathology: a review. Journal of Anxiety Disorders, 5, 187–198.
Kuperman, S., Schlosser, S. S., Lidral, J., & Reich, W. (1999). Relationship of child psychopathology
to parental alcoholism and antisocial personality disorder. Journal of the American Academy of
Child and Adolescent Psychiatry, 38, 686–692.
Meyer, J. M., Silberg, J. L., Simonoff, E., & Kendler, K. S. (1996). The virginia twin-family study of
adolescent behavioral development: assessing sample biases in demographic correlates of
psychopathology. Psychological Medicine, 26, 1119–1133.
Pickles, A., Rowe, R., Simonoff, E., Foley, D., Rutter, M., & Silberg, J. (2001). Child psychiatric
symptoms and psychosocial impairment: relationship and prognostic significance. British Journal
of Psychiatry, 179, 230–235.
Rothman, K. J. (1990). No adjustments are needed for multiple comparisons. Epidemiology,
1, 43–46.
SAS Institute Inc. (1995). Logistic regression examples using the SAS system, version 6 (1st ed.).
Cary, NC: SAS Institute Inc.
Savitz, D. A., & Olshan, A. F. (1995). Multiple comparisons and related issues in the interpretation of
epidemiologic data. American Journal of Epidemiology, 142, 904–908.
Spanier, G. B. (1976). Measuring dyadic adjustment: new scales for assessing the quality of marriage
and similar dyads. Journal of Marriage and the Family, 38, 15–28.
Thompson, J. R. (1998). Invited Commentary. Re: ‘‘Multiple comparisons and related issues in the
interpretation of epidemiologic data’’. American Journal of Epidemiology, 147, 801–806.