Download content validity of the psycj3atric symptom index, ces

Psychological Reports, 1996,79, 1059-1069. Q Psychological Reports 1996
CONTENT VALIDITY OF THE PSYCJ3ATRIC SYMPTOM INDEX,
CES-DEPRESSION SCALE, AND STATE-TRAIT ANXIETY
INVENTORY FROM THE PERSPECTIVE OF DSM-IV '
ALEX OKUN, RUTH E. K. STEIN,
L A U R E J. BAUMAN
AND ELLEN JOHNSON SILVER
Deporfme~z/of Pediafrics
Albert Einstein College of Medicine/Montefiore Medical Center
lacobi Medical Center, Bronx, Neru York
Szimmary.-We compared the item content of three commonly used scales of
psychiatric symptoms [the broad-band Psychiatric Symptom Index (IlEeld) and two
narrow-band scales, the Center for Epidemiologic Studies-Depression Scale (Radloff)
and the State-Trait Anxiety Inventory (Spielberger)], with diagnostic criteria and criterion-based symptoms for Major Depressive Episode and Generahzed Anxiety Disorder as they appeared in DSM-IV. The Psychiatric Symptom Index and the Center for
Epidemiologic Studies-Depression Scale each measured 7 of 9 criterion-based symptoms of Major Depressive Episode. The Psychiatric Symptom Index and State-Trait
Anxiety Inventory each measured 5 of 8 domains for Generalized Anxiety Disorder.
The Psychiatric Symptom Index had comparable content validity to the narrow-band
measures. AU met a nlajority of DSM-IV criteria for depression and anxiety, supponing their applicability for current research.
Standardized scales of psychiatric symptoms are popular for assessing
psychopathology in mental health research. The documented rehability and
validity of such scales and the efficiency of administering them in research
settings have led to their widespread adoption in studies of mental health
problems. However, most scales in use today were developed during a period in which the constructs or domains they measure were undergoing revision. A number of commonly used scales for the self-report of psychiatric
symptoms were constructed prior to the publication of diagnostic criteria for
mental disorder. Symptom scales developed prior to the development of a
taxonomy of psychiatric disorders may not reflect current conceptuahzations.
In addition to issues regarding the scales' concordance with current definitions, scales designed to measure symptoms of the same dsorder may differ from one another in the domains they assess. Even scales designed to
measure similar psychological constructs may vary greatly in their content.
'This work was supported in art by grants from the National Institute of Mental Health,
Preventive Intervention ~esearc[ Center lor Child Health, Grant P50-MH838280 and Health
Resources and Services Administration, Health Services Research Award MH18927. Please
address corres ondence to Ruth E. K. Stein, M.D., Preventive Intervention Research Center for
Child Health, bepartment of Pediatrics, Albert Einstein College of Medicine, 1300 Morris Park
Avenue, Bronx, New York 10461 or e-mail ([email protected]).
1060
A. OKUN. ET AL
Their authors may have conceptualized similar constructs differently, formulated divergent operational definitions, or have chosen to emphasize different features of the definitions. Further, some narrow-band inventories are
designed to measure single types of psychopathology, e.g., depression, while
others measure a broader range of disturbances.
Over the past two decades, investigators in our research group have
been involved in studies of the mental health of parents caring for children
who have chronic physical health conditions. In that context, we have focused particularly on the assessment of maternal anxiety and depression because these are the two more prevalent, preventable, and treatable forms of
mental health disorder in this population. Several studies (Breslau, Staruch,
& Mortimer, 1982; Daniels, Moos, B h g s , & Miller, 1987; Engstrorn, 1991;
Kronenberger & Thompson, 1992; Mullins, Olson, Reyes, Bernardy, Huszti,
& Volk, 1991) have shown that symptoms of depression and anxiety are
more common among mothers of children with chronic illnesses and handicaps than among those of healthy children. Researchers at the Preventive Intervention Research Center for Child Health have assessed the mechanisms
by which chronic childhood illness and maternal psychological rnorbidty are
associated and have developed and evaluated intervention programs designed to alter this association. Depression and anxiety are the principal types of
maternal psychological distress targeted by our intervention programs.
Since 1977, we have used a broad band scale, Ilfeld's (1976) Psychiatric
Symptom Index, to measure psychological distress among mothers of children who have a chronic Illness. The Psychiatric Symptom Index is one of
the few short measures of mental health that assess both constructs of primary interest to us, anxiety and depression (meld, 1977, 1979). It has been
validated on a large urban sample that included a significant subset of
minority inner-city residents and has comparable psychometric properties in
minority urban and more general populations (meld, 1978). Our own research has shown that the psychometric properties of the Psychiatric Symptom Index are stable across English-speaking African-American and Hispanic subgroups and across Spanish- and English-spealung Hispanics (Stein &
Jessop, 1989). In addition, Bauman (1994) showed that a total score on the
Psychiatric Symptom Index of greater than or equal to 30 indicates a high
k e l h o o d of diagnosis of major depression on a structured psychiatric interview, the Diagnostic Interview Schedule (Robins, Helzer, Croughan, & Ratcliff, 1981).
Because the Psychiatric Symptom Index was developed in 1976, we decided to verlfy that its content is consistent with more recent definitions of
psychiatric syndromes. One set of current conceptualizations of psychiatric
dsorders is found in the Diagnostic and Statistical Manual of Mental Disorders 4th Edition (DSM-IV) (American Psychiatric Association, 1994). The
CONTENT VALIDITY: SCALES O F PSYCHIATlUC SYMPTOMS
1061
DSM-IV specifies the criteria and criterion-based symptoms that are necessary to diagnose a psychiatric disorder. We chose to use the DSM-IV as the
standard for current definitions because it is commonly accepted among
mental health professionals, it is the criterion against which research interviews such as the Diagnostic Interview Schedule (Robins, et al., 1981) and
the Schedule for Affective Disorders and Schizophrenia (Endicott & Spitzer,
1978) were validated, and it is a widely used standard for current research in
depression and anxiety. Although the DSM-TV is not an absolute standard
for defining psychiatric syndromes, it appeared to us to be a reasonable
choice for assessing the v&dlty of older research scales.
In this paper, we assessed how well the Psychiatric Symptom Index and
two commonly used narrow band inventories, the Center for Epidemiologic
Studies-Depression Scale (Radloff, 1977) and the State-Trait Anxiety Lnventory (Spielberger, 1973, 1983), performed in measuring depression and anxiety using the current criteria for Major Depressive Episode and Generalized
Anxiety Disorder as they appear in the DSM-TV. We inspected each scale's
items to compare how thoroughly and specifically the scale reflects diagnostic criteria and criterion-based symptoms listed for Major Depressive Episode and/or Generalized Anxiety Disorder.
Measures
The following provides a summary description of the construction and
original psychometric testing of the three scales used in these analyses. Their
structure and format are summarized in Table 1.
The Psychiatric Symptom Index.-The Psychiatric Symptom Index (11feld, 1976) is a shortened version of the Hopkins Symptom Distress Checklist (Parloff, Kelman, & Frank, 1954) constructed for use in a survey of
stress and coping administered to a community sample of over 2200 adults
from Chicago and Northwestern Lnhana in 1972. The Index contains 29
items or symptoms. The respondent is asked to report on a 4-point scale the
frequency with which these symptoms were experienced in the past two
weeks. The total score has an internal consistency reliability of .91 (Cronbach alpha). Four subscales, "Depression," "Anxiety," "Anger" and "Cognitive Disturbance," were identified by factor analysis (Ilfeld, 1976). Concurrent v&dity was demonstrated against three criteria, having sought professional help for emotional problems, having recently used psychoactive drugs,
and interviewers' ratings of respondents' tension. All three of these criteria
are Uely to capture individuals with high anxiety and depression.
The Center for Epidemiologic Studies-Depression Scale.-The Center for
Epidemiologic Studes-Depression Scale was constructed to measure depressive symptoms. It was included in a lengthy, structured interview of a com-
1062
A. OKUN, ET AL.
TABLE 1
STRUCCURE
AND FORMAT
OF T H E PSYCHMTRIC
SYMPTOM
INDEX,THE CENTER
FOR EPIDEMIOLOGIC
SCALE, A N D THE STATE-TRUTANXIETY INVENTORY
STUDIES-DEPRESSION
Scale
Number
of Items
Duration of
Symptoms
Psychiatric Symptom
Index
29
Past two weeks
CES-Depression Scale
20
Past week
State-Trait Anxiety Inventory
State
20
Trait
20
"Right now, at this
moment"
"Generally"
Response Categories
Never
Once in a while
Fairly often
Very often
Rarely or none of the time
(less than 1 day in the past week)
Some or a little of the time
(1-2 days in the past week)
Occasionally or a moderate amount
of the time (3-4 days in the past
week)
Most or all of the time
(5-7 days in the past week)
Not at all
Somewhat
Moderately so
Very much so
Almost never
Sometimes
Ofren
Almost always
munity sample of over 2800 adults from Kansas City, Missouri and Washington County, Maryland, during 1971-73. The respondent is asked to use a
3-point response scale to indicate how often during the past week he felt or
behaved each of 20 ways. The Center for Epidemiologic Studies-Depression
Scale was reported to be internally consistent (alpha range from .84 to .90).
It dscrirninated well between respondents who said they had "needed help
for reports of an emotional problem during the past weeks" and those who
did not and between inpatient psychiatric and samples of the general
population (Radloff, 1977).
The State-Trait Anxiety Inventory, Form Y.-The State-Trait Anxiety Inventory (Spielberger, 1973, 1983) is composed of two separate 20-item scales
constructed to measure "state" and "trait" anxiety. On the State Scale, the
respondent is asked to indicate "how [he] feels right now, that is, at this
moment" with respect to each of 20 items, using a 4-point scale. O n the
Trait Scale, the respondent is asked to indicate "how [he] generally feels"
with respect to each of 20 different items, again using a 4-point scale. The
scale was standardized and vahdated on more than 1,200 high school and
college students; 1,800 workmg adults; 1,900 mihtary recruits; 600 neuropsychiatric, medical, and surgical patients; and 200 prison inmates. It was rehable and internally consistent (alpha ranged from .86 to .95) (Spielberger,
1983). Scores correlated with expected results under stressful and nonstress-
CONTENT VALIDITY: SCALES OF PSYCHIATRIC SYMPTOMS
1063
ful experimental conditions and lscriminated neuropsychiatric patients from
community residents. Compared to earlier versions of the State-Trait Anxiety Inventory, its most recent revision, Form Y, is reported to include fewer
symptoms of depression as items and to have an improved factor structure.
Content Analysis
In Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association & National Council of Measurement in Education, 1985) "content-related evidence of
validity" is defined as "evidence that shows the extent to which the content
of a test is appropriate relative to its intended purpose. Such evidence is
used to establish that the test includes a representative or critical sample of
the relevant content domain and that it excludes content outside that domain" (p. 90). To identLfy what was "representative" of the domains of depression and anxiety and what content should be excluded, we used DSMIV criteria. The content of the Center for Epidemiologic Studies-Depression
Scale was compared to criteria for Major Depressive Episode in the DSMIV.The State Scale of the State-Trait Anxiety Inventory was compared to
criteria for Generalized Anxiety Disorder in the DSM-N. The content of
the Psychiatric Symptom Index was compared to criteria for each dsorder.
The criteria for Major Depressive Episode and Generahzed Anxiety Disorder appear in the DSM-IV as descriptive statements, symptom lists, and exclusionary criteria. Items from each scale were classified in one of three ways
as (1) matching a criterion-based symptom of Major Depressive Episode, (2)
matching a criterion or criterion-based symptom of Generaked Anxiety Disorder, or ( 3 ) a nonmatching symptom.
For an item to be classified as matching a criterion or criterion-based
symptom for one of the disorders, its wording either had to correspond to
exactly that in the DSM-IV or have no other meaningful interpretation. Ln
some instances, multiple items on the scales were classified as matching the
same criterion or criterion-based symptom. For example, "difficulty concentrating andlor mind going blank" is a single criterion-based symptom for
Generaltzed Anxiety Disorder but was measured as two separate items on
one of the scales. In other cases, the content of the items represented symptoms of behaviors exhibited commonly by depressed individuals, e.g., "feeling lonely," or by anxious individuals, e.g., "feeling afraid," but these items
were classified as "nonmatching"
because they did not reflect the DSM-N
criteria or criterion-based symptoms of the disorders. We chose to use a
relatively high standard for deciding whether a scale item matched DSM-IV
content for two reasons. First, it minimized reliance on our judgment as to
whether scale items were indeed a match. Second, it put the scales to the
most rigorous test of content vali&ty. Clearly, some of the symptoms we ul-
1064
A. OKUN, ET AL.
tirnately classified as "nonmatching" are widely accepted as typical manifestations of depression and anxiety.
Aside from Major Depressive Episode and Generalized Anxiety Disorder, there are other DSM-IV disorders that could have been used in the content analysis of these scales, as their diagnostic criteria also reflect symptoms
of depression and anxiety. More scale items might have been interpreted as
corresponding to DSM-IV diagnoses if we had included criteria for Adjustment Disorder with Depressed Mood or Adjustment Disorder with Anxious
Mood. However, adjustment disorders are described in the DSM-IV as maladaptive reaction(s) to "an identifable psychosocial stressor or stressors"
(American Psychiatric Association, 1994, p. 623). The three scales we assessed measure symptoms only; they do noc identlfy whether a psychosocial
stressor is responsible. Therefore, it was impossible to include these diagnoses in our analyses. The disorder Dysthymia also was considered as a possible source of diagnostic criteria for the analysis, but its symptoms are
chronic (minimum two years' duration) and otherwise resemble those of Major Depressive Episode.
Using this classification system, we assessed the content validty of the
scales for Major Depressive Episode, Generalized Anxiety Disorder, or both
as defined in the DSM-IV. The greater the number of items classified as corresponding to DSM-IV diagnostic criteria or criterion-based symptoms of a
disorder, the higher
we considered the content validity of that scale to be
for that disorder.
RESULTS
Item Classzj5cation and Content Validity
Depression.-There are five DSM-IV criteria for Major Depressive Episode and nine categorized criterion-based symptoms. Only the nine criterion-based symptoms are amenable to assessment by means of a checklist,
thus they form the basis for the evaluation. The results of this process of
item classification are summarized in Table 2.
On the Psychiatric Symptom Index, nine of 29 items were classified as
matching diagnostic symptoms of Major Depressive Episode (see Table 3 ) .
Seven of the nine came from the Psychiatric Symptom Index's Depression
subscale. Thirteen of 20 items from the Center for Epidemiologic StudlesDepression Scale were classified as symptoms of Major Depressive Episode.
There was greater redundancy of items in the Center for Epidemiologic
Studies-Depression Scale than in the Psychiatric Symptom Index. Five ddferent items from the Center for Epidemiologic Scudes-Depression Scale
were classified as matching the same criterion-based symptom, "depressed
mood." Two other criteria each were reflected in two items of the Center
for Epidemiologic Studes-Depression Scale. There was redundancy of Psy-
CONTENT VALIDITY: SCALES O F PSYCHIATRIC SYMPTOMS
TABLE 2
NUMBER
OF ITEMS FROM
THE PSYCHIATRIC
SYMPTOMINDEXAND CENTER
FOR
SCALECLASSIFIED
AS MATCHING
EPIDEMIOLOGIC
STUDIES-DEPRESSION
CRITERION-BASED
SYMPTOMS
OF MAJOR
DEPRESSWE
EPISODE
I N DSM-IV
Symptom List
1. Depressed mood
2. Markedly diminished interest or pleasure
3 . Significant weight loss or gain or change in appetite
4. Insomnia or hypersomnia
5. Psychomotor agitat ion or retardation
6. Fatigue or loss of cncrgy
7. Feelings of worthlessness or excessive or inappropriate
guilt
8. Diminished concentration or indecisiveness
9. Recurrent thounhts
., of death or suicide
Total number of matching items on each scale
Total number of items in each scale
Total number o€ symptoms matched by items
Number of ltems From
Psychiatric
Center for
Symptom E idemiologic
ftudies-~eIndex
pression Scale
1
2
1
1
0
1
0
2
1
9
29
7
5
0
1
1
1
2
2
1
0
13
20
7
chiatric Symptom Lndex items on only two dlrnensions. Each of the scales
tapped seven of the nine symptoms, although not the same ones. Neither
scale addressed c h c a l significance of the distress or impairment caused by
the symptoms nor any of the exclusion criteria in the DSM-IV. The broad
band Psychiatric Symptom Index included 20 items that did not match criterion-based symptoms of depression. The narrow band Center for Epidemiologic Studies-Depression Scale contained only seven items that did not
match these criteria.
Anxiety.-The DSM-IV contains six criteria (A-F) used to define Generahzed Anxiety Disorder. Criteria A and B and six symptoms listed under
Criterion C provide a total of eight domains that could be used to evaluate
the content vahdity of the Psychiatric Symptom Index and State-Trait Anxiety Inventory. Seven of the 29 Psychiatric Symptom Index items matched
symptoms from diagnostic Criterion C (see Table 3). Ln all, five of six Criterion C symptoms were tapped by the Psychiatric Symptom Lndex. Of the 20
items on the State Scale of the State-Trait Anxiety Inventory, five were dassified as matching criterion-based symptoms, three items matched diagnostic
Criterion A, and three matched Criterion B. No items from the Psychiatric
Symptom Index reflected Criterion A or B.
The Psychiatric Symptom Lndex and the State-Trait Anxiety Inventory
each reflected five of eight domains (criteria or criterion-based symptoms of
Generalized Anxiety Disorder). Some domains were covered by items on the
1066
A. OKUN, ET AL.
Psychiatric Symptom Index that were not part of its Anxiety subscale. The
State-Trait Anxiety Inventory had greater redundancy than the Psychiatric
Symptom Index. The narrow band State Scale of the State-Trait Anxiety Inventory had nine items that did not match the criteria for Generalized
Anxiety Disorder. In the broad band Psychiatric Symptom Index, 22 of the
29 items did not match these criteria.
TABLE 3
NUMBER
O F ITEMS
FROMTHE PSYCHIATRIC
SYMPTOM
INDFXAND STATE-TRAIT
A N X I EINVENTORY
~
CLASSIFIED
AS MATCHING DIAGNOSTIC
CRITERIA
OR
CRITERION-BASED
SYMPTOMS
FORGENERALIZED
ANXIETYDISORDER
L N DSM-IV
Criteria
A. Excessive anxiety and worry
B. Difficulty controlling the worry
Total classified as matching to criteria A and B
C. Symptom list (3 or more)
1. Restlessness or feehng keyed-up or on-edge
2. Being easily fatigued
3. DiHiculty concentrating or mind going blank
4. Irritability
5. Muscle tension
6. S l e e ~disturbance
Total number of items matching symptoms from criterion C
Total number of matching items on each scale
Total number of irems in each scale
Total number of domains (criteria and symptoms) matched
bv items
Number of Items From
Psvchiatric
State-Trait
S$-nptom
Anxiety
Index
Inventory
0
0
0
3
7
29
11
20
5
5
3
6
Drscuss~o~
The Psychiatric Symptom Index, Center for Epidemiologic Studes-Depression Scale, and State-Trait Anxiety Inventory were designed to measure
psychological symptom frequency in community-based populations. However, the instruments ddfer considerably in focus and in the nature of the
mental health problems that they were designed to assess. Although all three
antedate the DSM-IV, each displays considerable agreement with the current
criteria, despite the evolution of the definition of these disorders. Although
the three measures use differing referent time frames ("past two weeks,"
"past week," "nght now"), the duration of symptoms described in the
DSM-IV dlagnosr~ccriteria for Major Depressive Episode and Generalized
Anxiety Disorder also dlffer (two weeks for Major Depressive Episode, six
months for Generalized Anxiety Disorder). Only one scale, the Psychiatric
Symptom Lndex, matches one disorder, Major Depressive Episode, along
CONTENT VALIDITY: SCALES O F PSYCHIATRIC SYMPTOMS
1067
this temporal dimension (symptoms for two weeks). We d ~ dnot consider
duration of symptoms in the assessment of the content.
Using the DSM-IV as a standard against which to evaluate the content
validty of the three scales has some limitations. First, all three scales were
designed to measure symptom levels in population-based research; they are
not intended to identlfy individuals with psychiatric disorder. Thus, they
may be useful for estimates within populations, but they are not intended as
screening devices to classdy individuals as "cases." In this respect, all three
scales ddfer from the DSM-IV, which is intended as a listing of criteria for
individual diagnosis. Instruments designed to measure intensities of symptoms in general populations often contain items that are highly correlated
with the phenomenon of interest but are not a priori part of che associated
condition, and, therefore, would not be included in the diagnostic criteria.
Such items would potentially enhance the value of a scale as a research instrument, but they could devalue the scale when compared to strict dagnostic criteria for specific disorders. Thus, we may have underestimated the content val~dtyof these scales by comparing them to a definition intended for
chical purposes.
Second, the DSM-IV represents only one available contemporary definition of the complex constructs of anxlecy and depression. The DSM-IV is
used as a standard for this paper because it is frequently cited by researchers
conducting epidemiolog~cal studies of mental health. Further, authors of
many symptom chechsts have chosen one of its earlier versions as a standard agmst which to establish the validity of their scales. Had we used different contemporaneous definitions, we might
have come to different con-
elusions.
Third, we used the wording of the items from these three scales as the
basis for assessing content validity in comparison to DSM-IV criteria. Item
wording is just one determinant of a scale's validity. The context, order, and
presentation of a scale's questions and many other factors influence its effectiveness. Items worded exactly the way the criteria appear in a standard reference such as DSM-IV may not adequately measure that symptom. Reviewing the wording of the items from different psychiatric symptom scales is an
important early component of scale selection for research purposes, but it is
only one step in establishing a scale's validity.
Although there are hitations in using the DSM-IV as a standard, it is
Uely that any definition we selected would have some drawbacks. We believe that it is a useful standard for judging how well the three inventories
reflect current conceptualizations of anxiety and depression. Our analysis
leads us to conclude that each scale does a reasonable job of measuring the
construct(s) that it claims to assess. Of particular note, the broad band measure, the Psychiatric Symptom Lndex, was comparable to the narrow band
1068
A. OKUN, ET AL.
measures in its inclusion of symptoms of depression and anxiety according
to the DSM-IV criteria. It should be noted that some of the items in the Psychiatric Symptom Index that matched DSM-IV symptoms were not specifically included in either its "anxiety" or "depression" subscales.
Conclusions
The contents of three widely used self-report scales for psychiatric syrnptoms were analyzed for correspondence to the DSM-IV diagnostic criteria
for Major Depressive Episode and Generalized Anxiety Disorder. Using correspondence to the DSM-TV as the basis for comparison, the Psychiatric
Symptom Index achieves comparable content validity to the Center for Epidemiologic Studes-Depression Scale and State-Trait Anxiety Inventory for
depression and anxiety, the domains that these scales were constructed to
measure. The findings of this analysis support the continued use of all three
scales in assessing populations for symptoms of depression and anxiety.
REFERENCES
AMERICAN
EDUCATIONAL
RESEARCH
ASSOCIATION, AMERICANPSYCHOLOGICAL
ASSOCIATION A N D
N A ~ O N COUNCIL
AL
OF MWSUREMENT
I N EDUCATION.
(1985) Startdards for edztcatio~~al
and
psychological resting. Washington, DC: American Psychological Association.
AMERICAN
PSYCHIATRIC
ASSOCIATION.(1994) DSM-IV; diagnos~icand statiztical rnanual of rnerttal
disorders. (4th ed.) Washington, DC: American Psychiatric Association.
BAUMAN.
L. 1. (1994) Mental health of inner-city mothers of children with chron~cconditions.
Presented at the Ambulatory Pediatric Association annual meering, Seattle, 1YJ3shington.
BRESIAU,
N., STARUCH,
K. S., & MORTIMER,
E. A. (1982) Psychological distress ~n mothers of
disabled children. Americorr]ozrrnal ofDiseases of Childhood, 136, 682-686.
DANIELS,
D.. Moos, R. H., BILLINGS.
A. G., &MILLER.
J. J. (1987) Psychosocial risk and resistance factors among children with chronic illness, healthy siblings, and healthy controls.
]ozmral ofAbnor>nal Child Psychology, 15, 295-308.
ENDICOTT,
J.. &SPITZER.
R. L. (1978) A diagnostic interview: the Schedule for Mective Disorders and Schizophrenia. Archiues of General Psychiatry, 35, 837-844.
ENGSTROM,
I. (1991) Parental distress and social interaction in families with children with inflammatory bowel disease. Jounzal of the American Academy of Child and Adolescent Psychiatry, 30, 904-912.
ILFELD,
F. W (1976) Further vahdation of a psychiatric symptom index in a normal population. Psychological Reports, 39, 1215-1228.
ILFELD,
F. W. (1977) Current social stressors and symptoms of depression. American Journal of
Psychiatry, 134, 161-166.
ILFELD,
F. W. (1978) Psychologic status of community residents along major demographic dimensions. Archiues of General Psychiatry, 35, 716-724.
ILFELD,
E W. (1979) Persons at risk for symptoms of anxiety. In Proceedings of Clinical Anxiety/Tenston in Primary Medicine, Excerpta Medico. Pp. 24-40.
KRONENBERGER,
W. G., &THOMPSON,
R. 1. (1992) Psycholo ical adaptation of mothers of children with s ina bzida: association with dimensions o f social relationships. Journal of Ped d r i c ~syc&logy, 17, 1-11.
MULLINS.
L. L., OLSON,R. A., REYES, S., BERNARDY,
N., Huszn, H. C., &VOLK,R. J. (1991) Risk
and resistance factors in the adaptation of mothers of children with cystic fibrosis. Journal
of Pediatric Psychology, 16, 701-715.
PARLOFF,
M. B., KELW. H. C., &FRANK,
J . D. (1954) Comfort, effectiveness, and self-awareness as criteria of improvement in psychotherapy. American Journal of Psychiatry, 111,
343-351.
CONTENT VALIDITY: SCALES O F PSYCHIATRIC SYMPTOMS
1069
F ~ D L O F FL., S. (1977) The CES-D Scale: a self-repon depression scale for research in the general population. Applied Psychological Measurer?zenf, 1, 385.
R o s r ~ s .L. N., HEUER,J. E., CROUCHAN,
J.. & RATCLIFF. L. F. (1981) National Institute of
Mental Health Diagnostic Interview Schedule: its history, characteristics, and validity. Archives o/General Psychiafry, 38, 381-389.
SPIELBERGER,
C. D. (1973) Mantra1 for the State-Trait Anxiety Itrvenforyfor Children. Palo Alto,
CA: Consulting Psychologists Press.
SPIELBERCER,
C. D. (1983) State-Trait Anxiety Inventory: a comprehensive bibliography. Palo
Alto, CA: Consulting Psychologists Press.
D. J. (1989) Measuring health variables among Hispanic and nonSTEIN,R. E. K., & JESSOP,
Hispanic children. Public Heakh Reports, 104, 377-384.
Accepted Sepfember 17, 1776.