Download eudell-simmons2005

This article was downloaded by: [UNAM Ciudad Universitaria]
On: 24 December 2014, At: 22:43
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,
37-41 Mortimer Street, London W1T 3JH, UK
Journal of Personality Assessment
Publication details, including instructions for authors and subscription information:
http://www.tandfonline.com/loi/hjpa20
Reliability and Validity of the Social Cognition and
Object Relations Scale (SCORS) in the Assessment of
Dream Narratives
Erin M. Eudell-Simmons , Michelle B. Stein , Jared A. DeFife & Mark J. Hilsenroth
Published online: 10 Jun 2010.
To cite this article: Erin M. Eudell-Simmons , Michelle B. Stein , Jared A. DeFife & Mark J. Hilsenroth (2005) Reliability
and Validity of the Social Cognition and Object Relations Scale (SCORS) in the Assessment of Dream Narratives, Journal of
Personality Assessment, 85:3, 325-333, DOI: 10.1207/s15327752jpa8503_09
To link to this article: http://dx.doi.org/10.1207/s15327752jpa8503_09
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained
in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no
representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the
Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and
are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and
should be independently verified with primary sources of information. Taylor and Francis shall not be liable for
any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever
or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of
the Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematic
reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any
form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://
www.tandfonline.com/page/terms-and-conditions
EUDELL-SIMMONS,
SCORSSTEIN,
AND DREAMS
DEFIFE, HILSENROTH
JOURNAL OF PERSONALITY ASSESSMENT, 85(3), 325–333
Copyright © 2005, Lawrence Erlbaum Associates, Inc.
Reliability and Validity of the Social Cognition
and Object Relations Scale (SCORS)
in the Assessment of Dream Narratives
Erin M. Eudell-Simmons
Downloaded by [UNAM Ciudad Universitaria] at 22:43 24 December 2014
National Naval Medical Center
Bethesda, Maryland
Michelle B. Stein, Jared A. DeFife, and Mark J. Hilsenroth
Derner Institute of Advanced Psychological Studies
Adelphi University
In this study, we aimed to investigate the application of an externally rated measure of interpersonal representations (Social Cognition and Object Relations Scale [SCORS]; Westen, 1995)
to dream narratives. A total of 80 student participants enrolled at a private university in the New
York City metropolitan area completed a Dream Log and affect adjective checklist (Wellman,
2002) based on a recalled dream at both 1 month and 3 months following the September 11,
2001, terrorist attacks. Using the dreams provided in this study we examined the interrater reliability of SCORS ratings, the relationship of SCORS variables to an independently rated measure of dream distortion, and the relationship of SCORS variables to participants’ own ratings
of dream affect. Results indicated that dreams were reliably rated using the SCORS, 3 cognitive
SCORS variables were significantly related to dream distortion, and affective SCORS variables
were meaningfully related to participants’ own ratings of affect in their dreams. Findings from
this study provide support for the application of SCORS ratings to dream narratives. We discuss
implications for further research and clinical application.
Dreams are often remembered when they are seen by the
dreamer as disturbing or intrusive (e.g., nightmares) or when
they reflect an experience significant to the dreamer
(Hartmann, 1998; Hill, 1996). As narratives, dream reports
are subject to the same mechanisms of memory and selective
reporting that occur when patients tell stories about their
waking life. According to Mayman (1968), dreams should be
considered significant narratives, as they reflect a person’s
experience. Dream narratives should, therefore, be amenable
to analyses that aim to discover aspects of the individual’s
personality that can then be used to facilitate insight, understanding, and change. Eudell-Simmons & Hilsenroth (2005)
completed an empirical review on the conceptual uses of
dreams in psychotherapy and suggested that dreams can be
equivalent to other narrative and clinical data in their ability
to quickly and accurately portray significant aspects of the
dreamer’s personality and salient issues in the dreamer’s
waking life.
Two studies have empirically demonstrated the potential
of dreams to provide unique and relevant interpersonal infor-
mation. In one study, Krohn and Mayman (1974) found external judgess, using a quantitative Object Representation
Scale (ORS) for Dreams, could rate patients’ dreams with
“good” to “excellent” interrater reliability (58% of exact
agreement between judges and 83% agreement within 1
scale point). Krohn and Mayman found adequate construct
validity through comparisons of independent scale ratings of
patients’ dreams to therapists’ ratings of patient level of object relations (correlations with modal dream score, r = .53).
In addition, Krohn and Mayman determined concurrent validity through significant correlations between ORS ratings
of dreams and ORS ratings of other measures (e.g., Rorschach [Rorschach, 1942] and early memories responses;
correlations with modal dream scores were r = .46 and r =
.62, respectively). In other words, patients’ levels of object
representations, which Krohn and Mayman (1974) described
as a “central dimension” (p. 464) in personality, could be
concurrently identified in dream narratives as well as in Rorschach, Early Memory responses, and from treatment interactions with patients’ therapists.
Downloaded by [UNAM Ciudad Universitaria] at 22:43 24 December 2014
326
EUDELL-SIMMONS, STEIN, DEFIFE, HILSENROTH
A second study (Popp et al., 1996) was described by the
authors as the “first report of the application of a more
psychometrically developed measure of central relationship
pattern [i.e., the Core Conflictual Relationship Theme
(CCRT) method; Luborsky, 1984] to a large body of dreams
in psychotherapy” (p. 1074). The CCRT is a method of identifying and quantifying cyclical relationship patterns, consisting of wishes of the individual, typical response of others,
and subsequent responses of the individual as they appear in
narratives. Applying this measure to dream narratives, Popp
et al. found good to excellent interrater reliability in scoring
CCRTs from the dreams of 13 patients in psychoanalysis
(kappas of .58 to .83), similar to the results obtained when
scoring CCRTs from waking narratives of the same sample
(kappas of .67 to .75). Similar themes were found in dreams
and waking narratives of specific patients, even when narratives were taken from different sessions of therapy. In other
words, Popp et al.s’ study found that patients’ core relational
patterns could be reliably identified in their dreams and that
these patterns corresponded to patterns seen in other relational narratives told in therapy.
The findings of these two studies provide empirical support for the notion that quantitative content analysis of
dreams by external raters can provide reliable and valid judgments regarding dreamers’ functioning. This type of analysis, which can enhance the understanding of an individual’s
experience, might be especially useful following a type of
event that typically results in frequent, intense, and memorable dreams.
The presence of disturbing and intrusive dreams following
a traumatic event is well documented, particularly when the
dreamer is experiencing other symptoms of traumatic stress.
For example, a study by Mellman, David, Bustamante,
Torres, and Fins (2001) examined stress responses and
dream content reported soon after a variety of lifethreatening events including motor vehicle accidents, industrial accidents, and impersonal assaults (i.e., nonsexual, unfamiliar assailant) by participants injured in these events.
Mellman et al. reported that of the 30% of their sample to recall at least one dream, 46% of these were trauma dreams and
were reported by those participants who exhibited more severe posttraumatic stress disorder (PTSD) symptoms than
those who reported less disturbing dreams. Other studies
have documented similar findings of disturbing and intense
dreams even some time after a traumatic event. For example,
6 to 12 months following Hurricane Andrew, Mellman, David, Kulick-Bell, Hebding, and Nolan (1995) used sleep laboratory assessment to find increased dream recall, more
negative dream content, and more pervasive sleep disturbances among those individuals suffering from PTSD than
among those not reporting such symptoms.
Intrusive and disturbing dreams have also frequently been
associated with the traumatic effects of combat experience,
often in conjunction with other symptoms of distress. For instance, Esposito, Benitez, Barza, and Mellman (1999) found
spontaneously recalled dreams reported by combat veterans
suffering from chronic PTSD to contain threat, reality distortion, and replications of the trauma. Several studies specifically examining symptoms of combat-related PTSD have
indicated that frequent anxiety dreams as well as nightmares
that involve intense emotions and reliving the trauma seem to
accompany other PTSD symptoms (Inman, Silver, &
Doghramji, 1990; see Ross, Ball, Sullivan, & Caroff, 1989,
p. 698). Kramer (1993) found that 50% of the dreams of Vietnam veterans suffering from chronic PTSD contained combat related intrusive imagery. In addition, increases in
dreams and nightmares including aggressive, persecutory,
and frightening content were found in Palestinian adolescents and children living in violent, traumatic conditions
(Punamäki, 1998). In general, these studies of dream content
during stressful life experiences have seemed to indicate that
not only are frequent and disturbing dreams common in response to a traumatic experience but that these dreams also
serve to descriptively and accurately reflect individuals’ experiences of the trauma.
Studies of dream content as part of a stress response suggest that dreams can reflect individuals’ experiences of significant and emotionally intense current life events,
particularly traumatic incidents, interpersonal difficulties,
and ongoing stressful situations. Hartmann (1998) suggested
that dreams following a traumatic event can be useful illustrations of the individual’s affective experience of the event
given that “contextualizing images” of fear and helplessness
are common and clear reflections in trauma dreams of the
emotional activation caused by the trauma. A contextualizing
image is defined as a dominant emotion or emotional concern of the dreamer (Hartmann, 2001). Hartmann’s (1998,
2001) research is supported by other examinations on the effects of sexual assault, sexual abuse, and incest, which have
found excessive themes of fear, aggression, and hostility
within dream content (Belicki & Cuddy, 1996; King &
Sheehan, 1996). Wood and Bootzin (1992, p. 223), in their
study examining dreams immediately following a severe
earthquake, concluded that dream content can reflect “personal concerns” and recent threatening events. In addition,
those individuals who experienced more stress and discomfort as a result of a trauma (i.e., those with PTSD symptoms
or depression) appear to experience more frequent, intense,
and disturbing dreams regarding the trauma (Hartmann,
2001; Siegel, 1996; Wood & Bootzin, 1992).
To identify and describe dream content in a clinically
useful way, an externally rated measure of interpersonal
narrative content was used to score dream reports following
the September 11th World Trade Center (WTC) attacks.
The Social Cognition and Object Relations Scale (SCORS;
Hilsenroth, Stein, & Pinsker, 2004; Westen, 1991, 1995)
was designed to quantify patient narratives generated from
the Thematic Apperception Test (TAT; Murray, 1943),
Early Memories, and psychotherapy sessions in terms of internal representations and expectations of social and inter-
!!!
Downloaded by [UNAM Ciudad Universitaria] at 22:43 24 December 2014
SCORS AND DREAMS
personal functioning. The SCORS allows therapists to
identify thoughts and feelings that underlie patterns of in!!!!! terpersonal behavior (Westen, 1991) and has been shown to
be clinically useful in characterizing interpersonal functioning from psychotherapy narratives (Peters, Hilsenroth,
Eudell-Simmons, Blagys, & Handler, in press; Westen,
1995), TAT narratives (Ackerman, Clemence, Weatherill,
& Hilsenroth, 1999), and Early Memories (Fowler,
Hilsenroth, & Handler, 1995, 1996). The premise of this
study is that dreams are examples of qualitatively similar
narrative data and that the SCORS, which has been supported as clinically useful when applied to waking interpersonal narratives, could have a similar result when applied to
dreams. Despite the view that dream narratives often contain interpersonal and clinically relevant content that is similar to waking narratives, no study has yet examined the
application of SCORS ratings to dream narratives.
In this study, we examined the validity of using dream
content to assess internally represented patterns of relational
and affective functioning following trauma (the September
11th attacks on the WTC). First, we attempted to evaluate the
interrater reliability of the SCORS (Hilsenroth et al., 2004;
Westen, 1995) as applied to dream narratives. We hypothesized that dreams rated using the SCORS would achieve at
least good interrater reliability (intraclass correlation coefficients [ICC] > .6 as defined by Fleiss, 1981). Second, we
conducted correlational analyses of an independently rated
measure of dream distortion to extend validity for the use of
the SCORS in rating dreams. We hypothesized that dream
distortion would be significantly and inversely related to the
more cognitive variables of the SCORS (Complexity,
Morals, Causality, and Identity). Finally, we compared ratings of the SCORS variables to participant ratings of positive
and negative affect present in their dreams. These analyses
were an investigation of validity, the extent to which external
raters could perceive dreams with the same affective qualities as the dreamers themselves. We believed that adaptive
and maladaptive ratings on the SCORS, particularly on more
affective SCORS variables (Affect, Relationships, Aggression, and Self-Esteem), would correspond to positive and
negative affective ratings of dream content.
METHOD
Participants
!!! Die Informationen haben wir leider nicht —>
Adjective checklist
We collected data from 80 undergraduate student participants enrolled in introductory psychology classes at a private
university in the New York City metropolitan area approximately 30 miles from ground zero. In addition, many of these
participants had family members, friends, or acquaintances
affected by, and in some cases involved in, the attacks and
their aftermath. The proximity to the attacks and relational
involvement of the participants with those affected suggests
327
the sample would have seen the attacks as having happened
to individuals similar to themselves and consider themselves
more threatened, thereby experiencing the attacks as traumatic. Participant dreams were collected from the same sample of participants as part of a longitudinal study examining
the effects of the September 11th attacks reported elsewhere
(Callahan, Hilsenroth, Jonay, & Waehler, 2005). Although
the participant sample is the same, the measures we utilized
in this investigation are independent of the data analyzed for
the Callahan et al. study. The mean age of the students was
18.25 years (SD = 0.88). Of the participants, 20% were male,
and 80% were female. All participants were single. Participants were offered extra credit in their classes for completing
the study protocol. Although no student refused to participate
in the study, data used were only from those participants who
completed the Dream Log at both collection times, resulting
in 160 dreams. Participants completed measures at two
times: 1 month following the attacks (Month 1; week of October 8, 2001) and 3 months following the attacks (Month 3;
week of December 10, 2001). At each time, each participant
completed a Dream Log based on one recent recalled dream.
All participants signed an informed consent form regarding
the completion of measures in relation to the events of September 11th.
Measures
Dream log and adjective checklist. Participants completed a dream log for each of the two dreams (at Month 1
and Month 3) they were asked to recall in the study. Dreams
were completed at home, and we asked participants to fill out
the following information on waking after a dream
(Wellman, 2002). The dream log included a space for the
dream narrative as well as an adjective checklist consisting of
24 possible adjectives that participants could rate as being
present in their dream(s). The adjectives were then divided
into scales representing Positive Affect (Comfortable, Exciting, Happy, Loving, Sensual, and Sexy) and Negative Affect
(Confusing, Deadly, Depression, Dreadful, Hateful, NerveRacking, Painful, Sad, Scary, and Unpleasant). Measurement
of positive and negative affect constructs is an important area
of evaluation in clinical psychology, as these constructs have
exhibited significant relations to clinical conditions (Brown,
Chorpita, & Barlow, 1998; Clark, Watson, & Mineka, 1994;
Watson & Clark, 1984; Watson, Clark, & Harkness, 1994).
We computed coefficient alphas for the Positive Affect
scale at Month 1 (α = .61) and Month 3 (α = .74) and for the
Negative Affect scale at Month 1 (α = .77) and Month 3 (α =
.75). With the exception of the Positive Affect scale at Month
1, these alphas surpassed the generally accepted lower bound
of .70 (Nunnally & Bernstein, 1994). Analyses using the Positive Affect scale at Month 1 were still conducted, as its coefficient alpha of .61 surpassed the .50 to .60 range originally
recommended by Nunnally (1967; Streiner, 2003) for the
early stages of research, it had a modest mean item-total cor-
328
EUDELL-SIMMONS, STEIN, DEFIFE, HILSENROTH
relation of .35, and it had a mean interitem correlation of .22
that surpasses the range of .15 to .20 recommended by Clark
and Watson (1995; Streiner, 2003) for scales that measure
broad characteristics. Positive and Negative Affect scales
were significantly inversely correlated at both Month 1 (r =
–.53, p < .01) and Month 3 (r = –.47, p < .01).
SCORS. The SCORS (Hilsenroth et al., 2004; Westen,
1995) is a clinician-rated assessment tool used to characterize representations of interpersonal functioning seen in narratives. In this study, we utilized the most recent, global rating version of the SCORS (Hilsenroth et al., 2004; Westen,
1995), which is made up of eight variables: Complexity of
Representation (Complexity), Affective Quality of
Representation (Affect), Emotional Investment in Relationships (Relationships), Emotional Investment in Values and
Moral Standards (Morals), Understanding of Social Causality (Causality), Experience and Management of Aggressive
Impulses (Aggression), Self-Esteem, and Identity and Coherence of Self (Identity). Each of these eight variables is
now scored on a 7-point anchored rating scale ranging from 1
to 7. Lower scores (e.g., 1, 2, or 3) indicate the presence of
more pathological responses, whereas higher scores (e.g., 5,
6, or 7) indicate healthy responses.
Ackerman et al. (1999, p. 430) provided descriptions of
the eight global SCORS variables. The Complexity variable
assesses the richness of an individual’s representations of
!!! self and others including the individual’s abilities to integrate
both positive and negative attributes of the self and others
and to take others’ points of view. The Affect variable assesses an individual’s positive and/or negative expectations
from others in relationships and how the individual describes
relationships. The Relationships variable identifies the individual’s level of commitment and emotional sharing in relationships. The Morals variable distinguishes between
individuals who show no remorse for “inconsiderate or selfish actions” (p. 30) and those who “think about moral questions in a way that combines abstract thought, a willingness
to challenge or question convention, and genuine compassion and thoughtfulness in actions (i.e., not just intellectualMentalisierungs
ization)” (p. 30; Westen, 1995). The Causality variable
assesses how well an individual understands why people do
fähigkeit what they do. The Aggression variable assesses an individual’s ability to control and appropriately express his or her
aggression and anger. The Self-Esteem variable assesses the
affective quality of self-representation. The Identity variable
assesses an individual’s level of fragmentation and integration (Westen, 1995). These variables can be divided into
those with more affective and object relational content (i.e.,
Affect, Relationships, Aggression, and Self-Esteem) and
those with more cognitive content (i.e., Complexity, Morals,
Causality, and Identity; Cogan & Porcerelli, 1996; Fowler et
al., 1995; Hibbard, Hilsenroth, Hibbard, & Nash, 1995;
Porcerelli, Cogan, & Hibbard, 1998; and Porcerelli, Hill, &
Dauphin, 1995).
Ackerman et al. (1999) found good to excellent interrater
reliability on this version of the SCORS (Spearman–Brown
corrected ICCs ranging from .70 to .95) when used on TAT
narratives, and reported on a number of other studies with
similar results. Other studies have found good to excellent
interrater reliability of SCORS variables when rating psychotherapy narratives and Early Memories (Fowler et al.,
1995, 1996; see Peters et al., in press; Westen, 1991).
Downloaded by [UNAM Ciudad Universitaria] at 22:43 24 December 2014
!!!
Dream Distortion Scale. The Dream Distortion Scale
(Zepelin, 1979) was used to characterize the level of bizarreness evident in each dream narrative. This scale is a 6-point
ordinal scale ranging from 0 (The events closely resemble the
recent waking experience of subject or the common, everyday experience of people in general) to 5 (Important aspects
of the dream are impossible, or the dream contains a combination of illogical or improbable elements which make the
dream narrative as a whole seem impossible) designed to
measure the extent to which dream content differs from the
dreamer’s waking life. A score of 0 indicates the dream content reflects “common, everyday experience of people in
general.” A score of 5 indicates the dream contains elements
that seem “impossible.” Each point on the scale is extensively
described and includes examples for ease of scoring. Zepelin
(1979) found excellent interrater reliability when using this
scale to rate dreams reported in the laboratory or from the
home, with Pearson correlation coefficients of .84 for both
sets of dream narratives.
Dream length. We examined the length of the dream
narrative as a possible confounding variable in the rating process. As such, we counted the words in each dream narrative
by computer word processor to provide an accurate measure
of the length of the dream report.
Procedure for Raters
The three raters we used in the study were two graduate students enrolled in an American Psychological Association approved Clinical Psychology PhD program and one graduate
student enrolled in a psychology masters program at the
same university. Raters received extensive training on the
SCORS, using both TAT narratives and nonstudy dreams,
and were supervised by a licensed Clinical Psychologist with
experience in using the measure. Raters met weekly for 2-hr
meetings for 16 weeks. We evaluated raters for reliability on
both the SCORS and the Dream Distortion Scale prior to applying the measures to dream narratives in the study. Each
rater achieved good to excellent interrater reliability on all
SCORS variables, ICC (1, 3) on practice TAT narratives from
.71 to .90 and excellent interrater reliability on the Dream
Distortion Scale, ICC (1, 3) = .88 on 30 nonstudy dreams.
We transcribed and printed each dream narrative to avoid
inconsistencies in scoring due to handwriting quality. Each of
the three raters independently rated all dream narratives using
329
SCORS AND DREAMS
each measure (i.e., SCORS and Dream Distortion). Each measure was used in turn to rate all dream narratives to avoid carryover effects. More specifically, we did not want ratings of the
SCORS to affect the ratings of the Dream Distortion Scale. All
dreams were first scored on the Dream Distortion Scale followed by the SCORS. Each transcribed dream narrative was
rated in random order and independently by all raters. Raters
were blind to participant identity, month of the dream, and all
other subject material such as adjective ratings, demographics, and other assessment materials. Raters scored each dream
in isolation from each other.
Downloaded by [UNAM Ciudad Universitaria] at 22:43 24 December 2014
RESULTS
Interrater Reliability
Table 1 reports the means and standard deviations of all variables used in the analyses. The interrater reliability of dream
ratings on the eight SCORS variables and the Dream Distortion Scale were evaluated via one-way random effects model
ICC (1) (Shrout & Fleiss, 1979, see Table 2). This ICC model
was chosen, as it is seen as the most conservative and
generalizable calculation of ICCs (Meyer et al., 2002). Mean
SCORS and Dream Distortion ratings across the three raters
were used for the analyses, as all scored each dream. As such,
the Spearman–Brown correction for the one-way random effects model, ICC (1, 3), was calculated to examine the reliability of the mean score for each of the SCORS variables
and the Dream Distortion score for each dream (see Table 2).
ICCs are considered to be excellent if greater than .74, good
TABLE 1
Means and Standard Deviations
for Dream Measures
Month 1 Dreams
SCORS Measure
Complexity
Affect
Relationships
Morals
Causality
Aggression
Self-Esteem
Identity
Dream Distortion Scale
Dream length
Adjective Checklist
Negative Affect
Positive Affect
Month 3 Dreams
M
SD
M
SD
3.75
3.66
3.62
3.91
3.55
3.79
3.94
4.09
2.48
95.40
0.67
1.19
1.06
0.54
0.65
0.58
0.41
0.54
1.50
72.62
3.83
3.91
3.67
3.86
3.55
3.83
3.93
4.20
1.93
66.19
0.51
1.03
1.07
0.53
0.68
0.56
0.43
0.43
1.46
59.48
3.11
1.29
2.43
1.42
2.78
1.08
2.10
1.29
Note. N=80. SCORS = Social Cognition and Object Relations Scale;
Complexity = complexity of representation; Affect = affective quality of
representation; Relationships = emotional investment in relationships;
Morals = emotional investment in values and moral standards; Causality =
understanding of social causality; Aggression = experience and management
of aggressive impulses; Identity = identity and coherence of self.
TABLE 2
ICC for Ratings on SCORS Variables
and Dream Distortion Scale
Month 1 ICC
Month 3 ICC
SCORS Variable
(1)
(1, 3)
(1)
(1, 3)
Complexity
Affect
Relationships
Morals
Causality
Aggression
Self-Esteem
Identity
M SCORS variable
Dream Distortion Scale
.40
.72
.59
.30
.26
.32
.23
.39
.40
.84
.67
.89
.81
.56
.52
.59
.47
.66
.65
.94
.25
.71
.68
.46
.40
.55
.38
.19
.45
.77
.50
.88
.86
.72
.67
.79
.65
.42
.69
.91
Note. N=80. ICC = intraclass correlation coefficients; ICC(1) = one-way
random effects model ICC; ICC(1, 3) = Spearman–Brown correction for
three raters (Shrout & Fleiss, 1979); Complexity = complexity of
representation; Affect = affective quality of representation; Relationships =
emotional investment in relationships; Morals = emotional investment in
values and moral standards; Causality = understanding of social causality;
Aggression = experience and management of aggressive impulses; Identity =
identity and coherence of self.
if ranging from .60 to .74, fair if ranging from .40 to .59, and
poor if under .40 (Fleiss, 1981).
We hypothesized that one would be able to rate dreams
using the SCORS with good interrater reliability (ICC >
.6). Although uncorrected ICC values were generally in the
fair range of reliability, the Spearman–Brown corrected
ICCs for the eight SCORS variables and the Dream Distortion Scale were considered to be of good reliability. Table 2
illustrates Spearman–Brown ICCs for the Month 1 (.47 to
.89, M = .65) and Month 3 (.42 to .88, M = .69); SCORS
variables ranged from fair to excellent. The Dream Distortion Scale demonstrated excellent interrater reliability with
ICCs of .84 and .94 at Month 1 and .77 and .91 at Month 3.
In all subsequent analyses, therefore, the mean of the three
raters’ scores on the SCORS variables and Dream Distortion Scale was used.
SCORS Ratings and the Dream Distortion Scale
The second hypothesis stated that independent ratings of
dream distortion would be negatively correlated with
SCORS variables—particularly Complexity, Morals, Social
Causality, and Identity—as they intend to address more cognitive components of narratives. To test this hypothesis, we
examined independent ratings of the Dream Distortion Scale
and SCORS variables at Month 1 and Month 3 using Pearson
product–moment correlation coefficients (see Table 3). As
dream length was significantly correlated with Dream Distortion scores at both Month 1 (r = .37, p < .01) and at Month
3 (r = .45, p < .01), the effects of dream length were controlled for using partial correlations as can be seen in parentheses in Table 3.
330
EUDELL-SIMMONS, STEIN, DEFIFE, HILSENROTH
TABLE 3
Pearson Product Moment Correlation
Coefficients Comparing Mean Dream
Distortion Ratings and Mean SCORS
Ratings
SCORS
Variable
Downloaded by [UNAM Ciudad Universitaria] at 22:43 24 December 2014
Complexity
Affect
Relationships
Morals
Causality
Aggression
Self-Esteem
Identity
Dream Distortion Scale
Month 1
Month 3
–.33** (–.51**)
–.38** (–.37**)
–.26* (–.37**)
–.16 (–.23*)
–.33** (–.50**)
–.20 (–.29*)
–.17 (–.19)
–.46** (–.45**)
–.27* (–.33**)
–.20 (–.13)
.03 (–.08)
.12 (.11)
–.31** (–.34**)
–.20 (–.09)
–.16 (–.21)
–.24* (–.37**)
Note. N = 80. Correlations in parentheses represent partial correlations
between SCORS variables and Dream Distortion Scale controlling for dream
length. SCORS = Social Cognition and Object Relations Scale; Complexity
= complexity of representation; Affect = affective quality of representation;
Relationships = emotional investment in relationships; Morals = emotional
investment in values and moral standards; Causality = understanding of
social causality; Aggression = experience and management of aggressive
impulses; Identity = identity and coherence of self.
*p < .05. **p < .01.
After controlling for dream length, more distorted dreams
were related to less adaptive affective states as measured by
the SCORS at 1 month following the September 11th attacks
(correlations with Affect, Relationships, and Aggression
were negative and significant at the p < .05 level). These
findings did not hold up with more temporal distance from
the traumatic event, and more affective SCORS variables
were not found to be significantly related to dream distortion
at Month 3.
In support of our hypothesis, it is important to note that
dream distortion was negatively and significantly correlated
with three of the four more cognitive SCORS variables at
Month 1 and Month 3 both before and after controlling for
dream length. More distorted dreams exhibited significantly
lowered Complexity of Representations (r = –.51 and –.33),
less understanding of Social Causality (r = –.50 and –.34),
and less Identity and Coherence of Self (r = –.45 and –.37) at
Month 1 and Month 3, respectively.
SCORS Ratings and Participant-Rated Affect
Our final hypothesis stated that ratings on SCORS variables,
particularly on affective variables (i.e., Affect, Relationships,
Aggression, and Self-Esteem), would be positively correlated to participant ratings of positive affective dream content
and negatively correlated to participant ratings of negative affective dream content. To test this hypothesis, we examined
relationships between SCORS ratings and participant ratings
of positive affect and negative affect present in their dream at
each time using Pearson product–moment correlation coefficients (see Table 4). As dream distortion was significantly
correlated with affective SCORS variables at Month 1 and
cognitive SCORS variables at both Month 1 and Month 3,
correlations of dream distortion with the participant rated affect scales were also included in Table 4. Dream length was
significantly correlated to ratings of dream distortion (see
previously), SCORS ratings of Complexity (r = .32, p < .01)
and Causality (r = .31, p < .01) at Month 1, as well as SCORS
ratings of Relationships (r = .22, p < .05) and Aggression (r =
–.28, p = .01) at Month 3; thus, the analyses included partial
correlations controlling for the effects of dream length on
these variables. Partial correlations can be seen in parentheses on Table 4.
As Table 4 illustrates, ratings of dream narratives using
SCORS variables were related in the expected directions to
participant ratings of dream affect on the adjective checklist.
More adaptive SCORS ratings positively correlated with the
Positive Affect scale and negatively correlated with the Negative Affect scale of the checklist.
Examination of the affective component SCORS variables shows that the variables of Affect and Relationships
were positively and significantly correlated to the Positive
Affect scale at Months 1 (correlations of .59 and .33, respectively) and 3 (correlations of .54 and .31 and .38 [for partial
correlation], respectively), with Self-Esteem demonstrating
a trend toward significance at Month 3 (r = .21, p = .07). Correlations of Self-Esteem at Month 1 and Aggression at
Months 1 and 3 with Positive Affect were in the expected direction but not significant.
TABLE 4
Pearson Product Moment Correlation
Coefficients Comparing Mean SCORS
Ratings and Mean Dream Distortion Scale
Ratings With Participant Rated Adjective
Checklist Scales
Positive Affect
Negative Affect
Scale
Month 1
Month 3
Month 1
Month 3
Complexity
Affect
Relationships
Morals
Causality
Aggression
Self-Esteem
Identity
Dream
Distortion
Scale
.13 (.21)
.59**
.33**
.12
.13 (.21)
.12
.18
.17
.28*
.54**
.31** (.38**)
.07
.19
.16 (.11)
.21
.16
–.18 (–.24*)
–.66**
–.29**
–.19
–.22* (–.28*)
–.45**
–.23*
–.23*
–.37**
–.64**
–.16 (–.21)
–.01
–.27*
–.24* (–.19)
–.25*
–.25*
–.13 (–.06)
–.10 (–.01)
.30**(.27*)
.29** (.24*)
Note. N = 80. Correlations in parentheses represent partial correlations
between SCORS variables and Dream Distortion Scale controlling for dream
length. SCORS = Social Cognition Object Relations Scale; Complexity =
complexity of representation; Affect = affective quality of representation;
Relationships = emotional investment in relationships; Morals = emotional
investment in values and moral standards; Causality = understanding of
social causality; Aggression = experience and management of aggressive
impulses; Identity = identity and coherence of self.
*p < .05. **p < .01.
Downloaded by [UNAM Ciudad Universitaria] at 22:43 24 December 2014
SCORS AND DREAMS
Similarly, ratings of affective SCORS variables were negatively correlated with experiences of negative affect in the
dreams. Again, the Affect variable demonstrated the strongest relationship to participant rated affect and correlated
–.66 with the Negative Affect scale at Month 1 and –.64 at
Month 3. Self-Esteem was also significantly negatively correlated with the Negative Affect scale at Month 1 (r = –.23)
and at Month 3 (r = –.25). The SCORS variables of
Relationships and Aggression demonstrated significant correlations with Negative Affect at Month 1 (rs = –.29 and
–.45, respectively) yet only trended toward significance after
controlling for dream length at Month 3 (r = –.21, p = .07 for
Relationships and r = –.19, p = .09 for Aggression).
In a more exploratory analysis, cognitive-based ratings of
dream narratives were examined for their relationship to participant-rated dream affect. Ratings of more cognitive narrative variables were not as often or as strongly related to
participant experiences of positive affect in their dreams.
Only the SCORS variable of Complexity demonstrated a significant relation to the Positive Affect scale of the checklist at
Month 3 (r = .28), and Complexity and Causality trended toward significance after controlling for dream length (both rs
= .21, ps = .07). However, dream content that was less complex, less coherent, and more distorted was consistently related to participant experiences of negative affect in the
dreams. Significant correlations of the Negative Affect scale
were found with Complexity (r = –.24 after controlling for
dream length at Month 1, and r = –.37 at Month 3), Causality
(r = –.28 after controlling for dream length at Month 1, and
Month 3 r = –.27), Identity (Month 1 r = –.23, and Month 3 r
= –.25), and dream distortion (after controlling for dream
length at both points; Month 1 r = .27, and Month 3 r = .24).
DISCUSSION
Research has documented the presence of disturbing and intrusive dreams following many types of traumatic experiences (Belicki & Cuddy, 1996; Esposito et al., 1999;
Hartmann, Zborowski, Rosen, & Grace, 2001; Horowitz,
2001; Inman et al., 1990; King & Sheehan, 1996; Mellman et
al., 2001, 1995; Punamäki, 1998; Ross et al., 1989; Siegel,
1996; Wood & Bootzin, 1992). In fact, due to the easily accessible and frequently disturbing nature of dreams following a trauma, these dreams may provide an important source
of information regarding the affective and interpersonal experiences of individuals following such an event. In addition,
research examining the reliability and validity of interpersonal patterns identified from dreams suggests that the current status of an individual’s perceptions of his or her support
system and his or her role in the world may be portrayed not
only by stories they tell (Luborsky, Barber, & Diguer, 1992)
but also by dreams they report. In this study, we aimed to explore the potential use of dreams in the assessment of interpersonal representations and subjective affective experience.
331
It is also the first study to attempt to quantify dreams in terms
of their representation of interpersonal and social relationships using the SCORS.
Our first goal in the study was to determine if SCORS ratings could be reliably applied to dream narratives. Results indicate that the 160 dreams we collected in this study were
generally scored with fair to excellent interrater reliability.
At both Month 1 and Month 3, the mean Spearman–Brown
ICCs of SCORS ratings were in the good range, with ICCs
for two SCORS variables—Affect and Relationships—located in the excellent range across the two groups of dreams.
The Affect and Relationships variables are the most salient to
this study, as they examine an individual’s affective experience and ability to invest in relationships. It may be that these
two variables are the clearest variables to see in dreams following a traumatic experience. Findings of good to excellent
interrater reliability coefficients using the SCORS are similar
to other research that has used mean SCORS ratings from
multiple narrative items (e.g., multiple TAT stories and early
memories; Ackerman, et al., 1999; Fowler et al., 1995, 1996;
see Westen, 1991).
Our second goal in this study was to extend the construct
validity of the SCORS as applied to dream narratives. To
achieve this, the Dream Distortion Scale was rated independently from the SCORS by trained raters. In agreement with
prior research (Zepelin, 1979), interrater reliability for the
Dream Distortion Scale was found to be in the excellent
range, further supporting the reliable application of this scale
to narratives. The Dream Distortion Scale, designed as a
measure to score bizarreness in dream content versus typical
descriptions of everyday waking life, was hypothesized to be
related to cognitive variables of the SCORS that evaluate a
narrative’s complexity, causality, and coherence in representations of self and others. Although representations of morality were not significantly related to dream distortion, more
distorted dreams did exhibit less complexity of self/other
representations, more gaps in interpersonal narrative accounts, and less stability/coherence in self-representations.
Furthermore, dream distortion covaried with participantrated dream affect in the same way that the SCORS
cognitively based variables did; dreams rated as more distorted and less cogent were not systematically related to reports of positive dream affect but were experienced by
participants as containing more distressing affective states.
The relationship between the Dream Distortion Scale and
cognitively based SCORS variables not only extends the
construct validity for assessing structural aspects of dream
narratives but also, by their systematic relation to negative
affective states, supports the practical utility of the SCORS
measure for application to dream narratives.
Our final goal in this study was to examine the criterion validity of the SCORS, that is, the relationship between external
ratings of dream representational quality to the dreamers’ own
ratings of affect present in their dreams. Several significant
correlations and trends supported the clinically based premise
Downloaded by [UNAM Ciudad Universitaria] at 22:43 24 December 2014
332
EUDELL-SIMMONS, STEIN, DEFIFE, HILSENROTH
that external ratings of adaptive–maladaptive interpersonal
dream content corresponded to participant ratings of more
positive–negative affective content. The relationships between affectively based SCORS variables (i.e., Affect,
Relationships, Aggression, and Self-Esteem) and adjective
checklist ratings were as expected in that these SCORS variables are more targeted toward the various affective qualities
in representations of the self or others. The variables affect and
relationships were most globally related to the dreamers’ affective ratings, as they were significantly bidirectionally correlated with participant ratings of both positive and negative
affect. As hypothesized, dreams judged as portraying positive
expectations of, commitment to, and emotional sharing in relational interactions were experienced by the dreamer as having significantly more positive affects. Dreams judged as
portraying caustic, shallow, and distant relationships were experienced by the dreamer as manifesting more negative affects. The remaining affective SCORS variables did not
appear to be systematically related to positive affect yet did exhibit significant relationships to ratings of negative dream affect. Thus, dreams experienced by participants as having more
distressing affects were also rated as containing self/other representations with lower self-esteem and less adaptive management of hostile/aggressive impulses.
Some limitations of note include considerations of the
lack of premorbid data, absence of a control group, and homogeneity of the participant sample. Due to the unanticipated and widespread nature of the WTC attacks, there were
no comparable data regarding participants’ interpersonal
representations prior to September 11th; nor was a control
group of unaffected individuals pragmatically available.
However, the focus of this study was on reliable and valid application of SCORS ratings to a sample of collected dream
narratives and not on changes of representations through
time/events or quantitative comparisons of SCORS ratings
among groups; thus, premorbid assessment or a participant
control group would be of little utility to the main aims of the
study. Concerns about the college-age sample of limited diversity used in this study may raise questions regarding
generalizability of results. Finally, collecting a series of
dreams yielding one average rating score on each of the eight
SCORS variables as opposed to scoring one dream per participant at each collection time may provide some beneficial
effects in terms of reliability and validity. However, this
method is more appropriate for an investigation of the participants’ stability of representations between dream life and
waking life. Again, our aims in this study were to investigate
the reliable and valid application of the SCORS to single
dream narratives and not an investigation into the stability of
representational styles or personality characteristics. Future
research may move toward investigations of cognitive/affective representations from dreams versus waking life, over the
course of development, in relation to symptomatic/interpersonal/personality characteristics, and through the course of
psychotherapy.
In conclusion, these findings suggest that external raters
can fairly reliably apply a narrative-based measure of interpersonal representations to dream narratives. Furthermore,
raters may accurately identify participants’ affective valence
toward their dreams, particularly when directly rating the affect tone (i.e., affect) of those representations. Demonstration of validity for SCORS ratings of dream narratives may
prove especially relevant toward applications for dream use
in psychotherapy (see Eudell-Simmons & Hilsenroth, 2005).
The SCORS, as a measure of social cognition and object relations, is meant to reflect meaningful aspects of personality
functioning. As SCORS ratings can be reliably applied to
dream narratives and demonstrate meaningful relationships
to dream affects, future research should examine the practical application of SCORS ratings to dreams and narratives
told in psychotherapy. Popp et al. (1996) illustrated in their
application of the CCRT method to dreams and psychotherapy narratives that relational patterns found in dream narratives are thematically similar to relational patterns described
by patients in their psychotherapy sessions. Thus, dream narratives may provide an additional source of data for therapists and patients to examine in collaborative endeavors to
recognize, understand, and work through maladaptive patterns of cognitive/affective representations. Active and collaborative use of the SCORS through assessment and
treatment may guide clinicians working with their patients in
efforts to establish more complex, benevolent, and adaptive
representations of self and others.
REFERENCES
Ackerman, S., Clemence, A. J., Weatherill, R., & Hilsenroth, M. J. (1999).
Use of the TAT in the assessment of DSM–IV cluster B personality disorders. Journal of Personality Assessment, 73, 422–448.
Belicki, K., & Cuddy, M. (1996). Identifying sexual trauma histories from
patterns of sleep and dreams. In D. Barrett (Ed.), Trauma and dreams (pp.
46–57). Cambridge, MA: Harvard University Press.
Brown, T. A., Chorpita, B. F., & Barlow, D. H. (1998). Structural relationships among dimensions of the DSM–IV anxiety and mood disorders and
dimensions of negative affect, positive affect, and autonomic arousal.
Journal of Abnormal Psychology, 107, 179–192.
Callahan, K., Hilsenroth, M., Jonay, T., & Waehler, C. (2005). Longitudinal
stress responses to the 9/11 terrorist attacks in a New York metropolitan
college sample. Stress, Trauma, and Crisis: An International Journal, 8,
45–60.
Clark, L. A., & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309–319.
Clark, L. A., Watson, D., & Mineka, S. (1994). Temperament, personality,
and the mood and anxiety disorders. Journal of Abnormal Psychology,
103, 103–116.
Cogan, R., & Porcerelli, J. H. (1996). Object relations in abusive partner relationships: An empirical investigation. Journal of Personality Assessment, 66, 106–115.
Esposito, K., Benitez, A., Barza, L., & Mellman, T. (1999). Evaluation of
dream content in combat-related PTSD. Journal of Traumatic Stress, 12,
681–687.
Eudell-Simmons, E. M., & Hilsenroth, M. J. (2005). A review of empirical
research supporting four conceptual uses of dreams in psychotherapy.
Clinical Psychology and Psychotherapy, 12, 255–269.
Downloaded by [UNAM Ciudad Universitaria] at 22:43 24 December 2014
SCORS AND DREAMS
Fleiss, J. L. (1981). Statistical methods for rates and proportions (2nd ed.).
New York: Wiley.
Fowler, C., Hilsenroth, M. J., & Handler, L. (1995). Early memories: An exploration of theoretically derived queries and their clinical utility. Bulletin
of the Menninger Clinic, 52, 31–52.
Fowler, C., Hilsenroth, M. J., & Handler, L. (1996). Two methods of Early
Memories data collection: An empirical comparison of the projective
yield. Assessment, 3, 63–71.
Hartmann, E. (1998). Nightmare after trauma as paradigm for all dreams: A
new approach to the nature and functions of dreaming. Psychiatry, 61,
223–238.
Hartmann, E., Zborowski, M., Rosen, R., & Grace, N. (2001).
Contextualizing images in dreams: More intense after abuse and trauma.
Dreaming, 11, 115–126.
Hibbard, S., Hilsenroth, M. J., Hibbard, J. K., & Nash, M. R. (1995). A validity study of two projective object representations measures. Psychological Assessment, 7, 432–439.
Hill, C. E. (1996). Working with dreams in psychotherapy. New York:
Guilford.
Hilsenroth, M., Stein, M., & Pinsker, J. (2004). Social Cognition and Object
Relations Scale: Global Rating Method (SCORS–G). Unpublished manuscript, The Derner Institute of Advanced Psychological Studies, Adelphi
University, Garden City, NY.
Horowitz, M. J. (2001). Stress response syndromes: Personality styles and
interventions (4th ed.). Northvale, NJ: Aronson.
Inman, D. J., Silver, S. M., & Doghramji, D. (1990). Sleep disturbance in
post-traumatic stress disorder: A comparison with non-PTDS insomnia.
Journal of Traumatic Stress, 3, 429–437.
King, J., & Sheehan, J. R. (1996). The use of dreams with incest survivors. In
D. Barrett (Ed.), Trauma and dreams (pp. 56–67). Cambridge, MA: Harvard University Press.
Kramer, M. (1993). The selective mood regulatory function of dreaming: An
update and revision. In A. Moffitt, M. Kramer, & R. Hoffman (Eds.), The
functions of dreaming (pp. 139–195). Albany: State University of New
York Press.
Krohn, A., & Mayman, M. (1974). Object representations in dreams and
projective tests. Bulletin of the Menninger Clinic, 38, 445–466.
Luborsky, L. (1984). Principles of psychoanalytic psychotherapy: A manual
for supportive-expressive therapy. New York: Basic.
Luborsky, L., Barber, J. P., & Diguer, L. (1992). The meanings of narratives
told during psychotherapy: The fruits of a new observational unit. Psychotherapy Research, 2, 277–290.
Mayman, M. (1968). Early memories and character structure. Journal of
Projective Techniques and Personality Assessment, 32, 303–316.
Mellman, T. A., David, D., Bustamante, V., Torres, J., & Fins, A. (2001).
Dreams in the acute aftermath of trauma and their relationship to PTSD.
Journal of Traumatic Stress, 14, 241–247.
Mellman, T. A., David, D., Kulick-Bell, R., Hebding, J., & Nolan, B. (1995).
Sleep disturbance and its relationship to psychiatric morbidity after Hurricane Andrew. American Journal of Psychiatry, 152, 1659–1663.
Meyer, G. J., Hilsenroth, M. J., Baxter, D., Exner, J. E., Jr., Fowler, J. C.,
Piers, C. C., et al. (2002). An examination of interrater reliability for scoring the Rorschach Comprehensive System in eight data sets. Journal of
Personality Assessment, 78, 219–274.
Murray, H. E. (1943). Thematic Apperception Test manual. Cambridge, MA:
Harvard University Press.
Nunnally, J. C. (1967). Psychometric theory. New York: McGraw-Hill.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.).
New York: McGraw-Hill.
333
Peters, E., Hilsenroth, M., Eudell-Simmons, E., Blagys, M., & Handler, L.
(in press). Reliability and validity of the Social Cognition and Object Relations Scale (SCORS) in clinical use. Psychotherapy Research.
Porcerelli, J. H., Cogan, R., & Hibbard, S. (1998). Cognitive and affective
representations of people and MCMI–II personality psychopathology.
Journal of Personality Assessment, 70, 535–540.
Porcerelli, J. H., Hill, K. A., & Dauphin, V. B. (1995). Need-gratifying object relations and psychopathology. Bulletin of the Menninger Clinic, 59,
99–104.
Popp, C., Diguer, L., Luborsky, L., Faude, J., Johnson, S., Morris, M., et
al. (1996). Repetitive relationship themes in waking narratives and
dreams. Journal of Consulting and Clinical Psychology, 64,
1073–1078.
Punamäki, R. L. (1998). The role of dreams in protecting psychological
well-being in traumatic conditions. International Journal of Behavioral
Development, 22, 559–588.
Rorschach, H. (1942). Psychodiagnostics (5th ed.). Berne, Switzerland:
Hans-Huber.
Ross, R. J., Ball, W. A., Sullivan, K. A., & Caroff, S. N. (1989). Sleep disturbance as the hallmark of posttraumatic stress disorder. American Journal
of Psychiatry, 146, 697–707.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing
rater reliability. Psychological Bulletin, 86, 420–428.
Siegel, A. (1996). Dreams of firestorm survivors. In D. Barrett (Ed.),
Trauma and dreams (pp. 159–176). Cambridge, MA: Harvard University
Press.
Streiner, D. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment,
80, 99–103.
Watson, D., & Clark, L. A. (1984). Negative affectivity: The disposition to
experience aversive emotional states. Psychological Bulletin, 96,
465–490.
Watson, D., Clark, L. A., & Harkness, A. R. (1994). Structures of personality
and their relevance to psychopathology. Journal of Abnormal Psychology,
103, 18–31.
Wellman, P. (2002). In Instructor’s manual to accompany Psychology (3rd
ed., p. 131). New York: Wiley.
Westen, D. (1991). Social cognition and object relations. Psychological Bulletin, 109, 429–455.
Westen, D. (1995). Social Cognition and Object Relations Scale: Q-Sort for
projective stories (SCORS–Q). Unpublished manuscript, Cambridge
Hospital and Harvard Medical School, Cambridge, MA.
Wood, J., & Bootzin, R. R. (1992). Effects of the 1989 San Francisco earthquake on frequency and content of nightmares. Journal of Abnormal Psychology, 101, 219–224.
Zepelin, H. (1979). Dream distortion. In C. Winget & M. Kramer (Eds.), Dimensions of dreams (pp. 140–143). Gainesville: University Presses of
Florida.
Erin M. Eudell-Simmons
U.S. Naval Hospital, Okinawa
PSC 482, Box 2944
FPO AP 96362
Email: [email protected]
Received August 30, 2004
Revised February 10, 2005