Download Congruence in the therapists` and clients` ratings of the therapeutic

Congruence in the therapists’ and clients’ ratings of the therapeutic alliance does not predict
outcome in psychological treatment.
Mikael Sinclair
Department of Behavioral Sciences and Learning
Linköping University
Sweden
Abstract
It would seem natural and in line with theoretical assumptions that congruence in the
clients’ and therapists’ ratings of the therapeutic alliance might enhance treatment outcome. The
aim of this study was to investigate to what extent the agreement or congruence of alliance
ratings between the patient and the therapist was associated with outcome in psychotherapy. Data
from a naturalistic sample consisting of 700 psychological treatments in primary care were used
to try to study this question. Congruence was measured as the difference between clients’ and
therapists’ ratings of the working alliance. The results indicate that there is no apparent
association between congruence in alliance and treatment outcome. This result seems to be true
for different levels of analysis and does not depend on clients’ distress level pre-treatment,
number of treatment sessions or whether the clients rates the alliance higher than the therapists
or vice versa. The results of the study show that client ratings of the working alliance are more
strongly associated with treatment outcome and that clients typically rate the alliance higher than
the therapists. The results also show that treatments where therapists rate the alliance higher than
clients produce worse outcome. The results of the study were interpreted as showing that clients
and therapists perceive and understand the concept of the working alliance from different
viewpoints that do not necessarily match. It also seems that discrepancies in alliance ratings
typically are of no discernible consequence for psychological treatment to yield acceptable
outcome and that clients’ perception of the alliance is a key factor for a positive treatment
outcome which mirrors previous results from research on the association between the working
alliance and treatment impact.
Keywords: working alliance, congruence, treatment outcome, CORE-OM, WAI
Contents
Introduction ..................................................................................................................................... 5
The working alliance............................................................................................................... 5
The working alliance and treatment outcome ......................................................................... 6
Congruence in the working alliance ....................................................................................... 8
Congruence in the working alliance and treatment outcome. ................................................. 9
Method .......................................................................................................................................... 12
Participants ............................................................................................................................ 12
Treatments............................................................................................................................. 13
Clients ................................................................................................................................... 13
Therapists .............................................................................................................................. 14
Measures ............................................................................................................................... 14
Procedure .............................................................................................................................. 15
Statistical analyses ................................................................................................................ 16
Results ........................................................................................................................................... 17
Analysis of the whole sample ............................................................................................... 17
Analysis of a sample consisting of five treatment sessions or more..................................... 19
Analysis of the sample when clients or therapists rate the alliance higher ........................... 20
Analysis of treatments with clients showing clinically significant distress .......................... 21
Analysis of treatments with clients showing clinically significant distress and consisting of
five treatment sessions or more............................................................................................. 23
Analysis of treatments with clients showing clinically significant distress and clients or
therapists rate the alliance higher .......................................................................................... 24
Discussion ..................................................................................................................................... 25
Limitations ............................................................................................................................ 31
References ..................................................................................................................................... 34
4
Introduction
The working alliance
The concept of the therapeutic alliance holds a prominent place throughout the
contemporary history of psychotherapy research and theory. Traditionally the therapeutic
relationship has been thought of as an “arena” where the individual’s (i.e. the clients) thoughts,
feelings and experiences could be given an opportunity to unfold in a safe environment. Freud
(1913) argued that the therapeutic relationship and process not only consisted of “negative”
transference but also of a positive transference that enabled the patient to withstand increased
levels of anxiety and at the same time attach the patient to the therapist as a person which was
hypothesized to prevent patient dropout. However, theorists and researchers soon found the need
to further study this “arena” in an effort to better understand its contents and function and how it
relates to the psychotherapeutic endeavor. There have been multiple attempts to conceptualize
this relationship but one of the more familiar or well-known notions is the concept of the
therapeutic alliance which was first formulated by Sterba (1934). Zetzel (1956) further
developed the concept, suggesting both an affective and a cognitive component. Greenson (1965)
differentiated between the therapeutic alliance and other aspects of the therapeutic relationship.
Bordin (1979) expanded the model and took a more interpersonal approach where he included
both the patient’s and the therapist’s subjective perspectives in a process which included not only
the relational bond but also the participants’ ideas and perceptions of therapeutic tasks and goals.
Bordin’s ideas departed somewhat from the psychodynamic approach in understanding the
alliance concept. He emphasized a more collaborative stance, where the alliance was the result of
a developmental process that included three aspects, agreement on therapeutic goals, consensus
on therapeutic tasks and the relational bond between patient and therapist (Horvath, Del Re,
5
Flückiger & Symonds, 2011). This “new” standpoint emphasized conscious aspects of the
relationship, collaboration and consensus. A more recent model, with strong ties to a relational
approach regarding the therapeutic alliance was developed by Safran and Muran (2000). From
this perspective, the therapeutic alliance is regarded as a process involving not only agreement
but also a negotiation between the patient and the therapist. This model emphasizes that the
bond, tasks and goals develop and change during and within the therapeutic process. However
there is still a lack of consensus on what constitutes the therapeutic alliance and how to best
conceptualize and measure the construct (Horvath, Del Re, Flückiger & Symonds, 2011;
Horvath, Krause & Altimir, 2011). In an attempt to summarize diverse definitions of the
alliance, it has been concluded that bond and collaboration are the fundamental features of a
“general, overarching alliance definition” (Constantino, Castonguay & Schut, 2002).
The working alliance and treatment outcome
There are a relatively large number of studies that have established the importance of a
strong working alliance between the patient and the therapist for psychotherapy to yield
acceptable results with regard to clients improved functioning and psychological wellbeing
(Horvath, 2001). However several questions remain that are less well understood in this area.
Researchers have found mixed results regarding the association between the working alliance
and outcome in psychotherapy and how to best capture and measure the alliance. One important
question is whose perspective on the alliance best correlates with treatment outcome. Horvath
(2001) found in a meta-analyses that the clients’ ratings of early alliance was the best predictor
of outcome. In the same meta-analysis the relationship between alliance ratings and
psychotherapy outcome was found to yield effect sizes of r = .22 and r = .15 for client and
6
therapist ratings respectively. The authors proposed that because of considerable overlap
between the two distributions, client alliance-outcome and therapist alliance-outcome may have
similar effect sizes (Horvath & Bedi, 2002). Another study found that observer-rated instruments
were associated with treatment outcome but therapist-rated and client- rated measures of alliance
were not (Fenton et al., 2001). Horvath & Symmonds (1991) speculated that these results might
be explained by the fact that client and the therapist pay attention to different aspects of
collaboration and that the client and the therapist rate and perceive the alliance on the basis of
different knowledge and experience. This perspective is well in line with Elliot and James’s
(1989) assumptions who argue that therapists and clients usually hold different perspectives on
the therapeutic relationship at a general level. However Kivlighan and Shaughnessy (1995)
argued that low correlations between patient and therapist ratings do not always occur due to
their results where they found that correlation between therapist and client alliance ratings
increased over time during therapy.
In a recent meta-analysis the overall aggregated correlation between alliance and
outcome was found to be r = .28 (Horvath, Del Re, Flückiger & Symonds, 2011). This metaanalysis indicates that the correlation between alliance scores and outcome scores is moderate,
repeatedly found in studies linking alliance to psychotherapy outcome and that similar results
have been replicated comparatively consistently in the psychotherapy research literature by
different researchers (Norcross & Lambert, 2011). In a moderator analysis, Horvath, Del Re,
Flückiger and Symonds (2011) found that that the alliance is a robust predictor of therapy
outcome, regardless of how the alliance is measured, from whose perspective it is assessed, when
it is measured, what outcome measures are used and the form of therapy that is studied. A
comparable conclusion was reached by Martin, Garske and Davies (2000). It is however
7
important to bear in mind that what we so far know of the working alliance and its relationship to
outcome in psychotherapy is the result of a number of studies where different instruments have
been used to measure this connection.
Congruence in the working alliance
Kivlighan (2007) maintain that the working alliance is a theoretical concept that
describes the shared perception and the shared creation of the therapist and the client and that
this theoretical foundation has implications for how to measure and conduct research related to
questions involving the alliance. According to these researchers there is a risk that research
related to the working alliance misses the dyadic and interactional nature of the construct even if
the measures and ratings being used are constructed to and perceived to take into account the
collaboration between therapist and client. According to these authors (Kivlighan, 2007) and
through searching a number of different databases in preparation for this study one can conclude
that there are unexpectedly few authors and researchers that have previously tried to examine the
joint contribution of clients and therapists alliance ratings related to therapy outcome.
According to Lichtenberg et al., (1998) a vital aspect of psychotherapy process is the
negotiation of the working relationship until there is an agreement on the nature of this
relationship. This agreement, where the participants in a therapeutic dyad interpret each other’s
behaviors’ as satisfactory in terms of meeting their respective definitions of the relationship, is
sometimes referred to as congruence. In terms of congruence related to the alliance concept this
would mean that the patient and the therapist are congruent when their subjective ratings of the
alliance are identical or very similar and incongruent when their ratings and perceptions diverge.
Strong (1982) proposed that the therapy process starts as an incongruent relationship were the
8
therapist and the patient work towards finding a congruent and collaborative stance and that
successful therapies are therapies were an initially incongruent relationship is gradually
transformed into a congruent relationship through the therapeutic process. This concept has
many similarities to Safran and Muran’s (2000) definition of negotiation in the therapeutic
process.
Congruence in the working alliance and treatment outcome.
A study by Rozmarin et al., (2008) is an attempt at finding a measure of congruence and
at the same time linking this construct to therapy outcome. In this study, congruence of alliance
was conceptualized as a correlation index consisting of the relationship between the therapist’s
and the patient’s subjective ratings of the working alliance. They found that the correlation index
was a stronger predictor of therapy outcome than the patient’s and the therapist´s ratings
respectively. The correlation index was found to have a large predictive capacity (β = 0.61, t =
2.37, p < 0.05), whereas the patient’s and the therapist’s alliance ratings yielded moderate
results. The measures correlated to a certain degree but seemed to capture different aspect of the
alliance construct. The authors hypothesized that the correlation index is one way of measuring
alliance at an intersubjective and dyadic level and that this construct may be an important find
for future attempts to capture and understand the concept of the working alliance.
However, the authors argue that their results need further investigation due to several limitations
in their design. One of the limitations put forward by the authors is that their study is based on a
limited sample size (22 treatment dyads) and represents only one therapy orientation which is
specifically constructed and aimed at working directly through the therapeutic alliance (Brief
relational therapy, BRT). Another possible limitation that the authors mentioned and thought to
9
be in need of further inquiry is how to empirically capture congruence in alliance, a limitation
which is supported by other researchers (Kivlighan & Marmarosh, 2012).
Kivlighan (2010) and Kivlighan and Marmarosh (2012) use a different approach in order
to capture congruence in the working alliance. They argue that using difference scores or
correlations as measures of congruence have inherent conceptual and methodological limitations
that might result in erroneous results. They propose that the best way to conceptualize and
measure congruence and its relation to psychotherapy outcome is to use more advanced
statistical methods such as polynomial regression and response surface analysis. A discussion of
these methodological issues is to a certain extent beyond the scope of this study. However some
of their main concerns when research is conducted on alliance and outcome are that difference
scores and profile similarities fail to take into consideration the absolute level of alliance scores,
possible statistical dependency between alliance ratings and who rates the alliance the highest,
client or therapist. Statistical methods that take these factors into account might produce different
results when the relationship between congruence in the working alliance and psychotherapy
outcome is investigated. However, Kivlighan and Marmarosh (2012), using more advanced
statistical methods, also found mixed results with regard to associations between congruence in
alliance ratings and outcome. They found that congruence seems to be important in predicting
session smoothness if alliance levels are high, that clients rated session smoothness higher when
client’s ratings of the alliance were higher than the therapist’s but no significant relationship with
session depth was found. Results also revealed that early congruence in alliance ratings was
positively related to symptom change but at the same time, contrary to their hypothesis,
increased levels of disagreement in alliance yielded greater symptom improvements. The authors
argue that these findings may be due to clients and therapists holding different perspectives on
10
the quality of the alliance and/or that therapists may address and work through alliance ruptures
even though they are not aware of their occurrence, which would work in favor of a positive
outcome of the treatment.
The concept of convergence is a theoretical construct which shares a number of aspects
with congruence. Convergence has been defined as a two-way process where differences in
judgments, perspectives, beliefs and behaviors between clients and therapists are lessened. This
phenomenon has been observed in several types of relationships ranging from interaction related
to conformity to group norms to the interplay between parent and child. The concept of
convergence has been hypothesized to be an important aspect of relationship formation in
psychotherapy, a key component of the working alliance and a possible predictor of
psychotherapy outcome (Swift & Callahan, 2009, Gaston et al., 1995). The literature related to
studies of convergence between therapist and patient has to a great extent focused on the
therapeutic alliance where research examining the construct and its connection to the working
alliance has yielded mixed results. According to a study conducted by Fitzpatrick, Iwakabe and
Stalikas (2005) the results indicated that patient and therapist ratings of the alliance were
significantly divergent, that the divergence was greater for ratings of task- and goal-aspects of
the alliance construct and that the level of divergence/convergence did not change over the
course of treatment. They also concluded that there is no apparent relationship between absolute
and relative divergence in alliance scores on session impact.
In a meta-analysis performed by Tryon, Blackwell and Hammel (2007), the results
indicated that regardless of therapy orientation, client´s level of distress, length of therapy, type
of measure being used and the therapist´s experience only moderate correlations (r = .36) were
found between therapists and clients rating s of the working alliance where clients rated the
11
alliance significantly higher (d = 0.63). However, studies of convergence relative to working
alliance have also found that convergence increases over time during psychotherapy (Kivlighan
& Arthur, 2000).
The main purpose of this study was to investigate whether congruence in the therapists’
and the clients’ alliance ratings was positively associated with treatment outcome. There is an
apparent lack of studies on alliance as a dyadic construct and its association with treatment
outcome and the few previous studies have shown mixed results. Another purpose was to study
associations between alliance and outcome in a large, naturalistic sample with diverse forms of
treatments, clients and therapists. The hypothesis that was tested was that congruence in alliance
ratings is positively correlated to treatment outcome.
Method
Participants
The data used and analyzed in this study was collected from a naturalistic research
project primarily aimed at studying process and outcome in psychological treatments in primary
care. The participants in the study consisted of clients who were referred for counseling or
psychotherapy at primary care centers in two Swedish regions. All clients, who were referred to
primary care units, within a six months period (November 2009 through April 2010), were asked
to participate in the study. The majority of the clients were referred to psychological treatment by
their general practitioner (GP), however a small number of clients were self-referred. Almost
1200 clients, about two thirds of the clients informed and asked, accepted to take part in the
study. Approximately one third of the clients were subscribed psychotropic medication parallel
to psychological treatment. The main reasons given for not wanting to participate in the study at
12
all were either that the participants found the procedure too stressful or arduous, or that the
therapists concluded that the patient should not participate, due to age, severe distress or
language problems.
Treatments
The psychological treatments in this sample are best described as psychological treatment
given to those clients whom the GP evaluated as being in need of more or another treatment than
prescribed medication. The treatments were all delivered and measured at the primary care units.
They were mainly of fairly short length (number of sessions: mean = 5.68, median = 5.00, SD =
3.95) although the treatments ranged in length from 1 to 29 sessions. Due to the nature of the
study, no control was made for treatment integrity of the therapies that this study is based on.
The therapists marked a box in an End-of-therapy form informing which type of treatment was
being delivered. The therapists had a choice of 14 different therapy orientations:
psychodynamic, cognitive, behavioral, CBT, time-limited, existential, relational, patientcentered, systemic, supportive, expressive, interpersonal, crisis intervention, and counseling.
Therapists could mark several methods for the same treatment, indicating that they used more
than one therapy orientation.
Clients
The patient group consisted of 76% women and 24% men with diverse backgrounds (i.e.
occupation, socioeconomic background, marital status, cultural background, educational level)
and psychological complaints, dysfunction and severity of distress. The mean age of the patient
group was 37.3 years (SD = 14.3, median 35)
13
Therapists
The 79 therapists were social workers, psychologists, psychiatric nurses or occupational
therapists. Their average experience of working with psychotherapy was 10 years. Sixty-five of
the therapists had basic training in psychotherapy in addition to their basic education and 21 had
advanced training as psychotherapist. Thirty-five therapists had training in psychodynamic
therapy, 29 in cognitive therapy and 20 in CBT or behavior therapy. Several of the therapists
where trained in more than one treatment method.
Measures
The main outcome measure that was used was the Clinical Outcomes in Routine
Evaluation – Outcome Measure (CORE-OM, Evans et al., 2002). The CORE-OM consists of 34
items that measure psychological distress on a five-point scale ranging from “Not at all” to
“Most or all the time”. The 34 items cover four major problem areas (subjective well-being,
problems/symptoms, life functioning, and risk to self or others). The scoring is problem-oriented
where higher scores indicate more distress and more impaired functioning. The instrument has
shown good internal reliability in clinical and nonclinical samples (0.94) and test-retest
reliability (0.90), convergent and discriminant validity, and sensitivity to change (Barkham et al.,
2001, Barkham et al., 2006).
The Working Alliance Inventory – short version revised (WAI-S, Tracey & Kokotovic,
1989, Swedish translation Holmqvist & Skjulsvik) which is a self-report instrument was used to
measure the alliance, both from the client´s and the therapist’s subjective perspectives. The
original Working Alliance Inventory is a 36 item scale designed to assess the three dimensions of
Bordin's working alliance concept in adults across all types of therapy. The short version, called
14
WAI-S (Tracey & Kokotovic, 1989) consists of 12 items, four items for each of Bordin’s
dimension. All items are rated on a 7-point scale ranging from 1 (never) to 7 (always) where
higher ratings indicate perceived stronger working alliance. This shortened version (WAI–S) has
previously been used in several studies (Elvins & Green, 2008) and correlates with a variety of
outcome indices (Horvath, 1994) and other alliance measures. Verification for discriminant
validity has been found through its use in a large number of different populations with diverse
levels of alliance (Raue et al., 1993; Samstag et al., 1998).
Procedure
All clients were informed of the aim of the study, the voluntary nature of participation
and were given a written description of all different aspects of the study. When clients arrived at
the primary care center for a treatment session, a plain envelope with the patient’s code written
on the front, containing the CORE-OM and the Working Alliance Inventory – short version
(WAI-S), was handed to him or her. Before the session started, the patient completed the COREOM in the waiting room. The questionnaire was then put in the envelope and kept by the patient
during the treatment session. After the session, the patient completed the WAI-S, and then
returned the questionnaires in the primary care center reception. The therapist completed the
therapist version of the WAI-S after each session throughout the whole treatment. All possible
precautions were taken to insure that the patient and the therapist had no knowledge of the other
person’s ratings throughout the treatment. The questionnaires (CORE-OM and WAI-S) were
completed at each treatment session. In total, 5060 CORE-OM questionnaires, completed by
1105 unique clients, were returned for analysis. For purposes of this study CORE-OM and WAIS questionnaires collected from 259 clients were not used in the analysis due to the fact that they
15
represented treatments consisting of only one treatment session. Another 146 treatments were
omitted because of too many missing values in alliance ratings. The analyses in this paper are
thus based on information from 700 unique clients who participated in at least two therapy
sessions and were treated by 79 different therapists.
Statistical analyses
All statistical computations were made using SPSS (v. 20). The main method of statistical
analysis is based on a correlational approach using Pearson’s r. Group differences were analyzed
using paired and independent samples t-tests. Effect sizes were calculated using Cohen’s d
(Clark- Carter, 2004). To analyze treatments where all clients showed a clinically significant
level of distress a cut-off score was computed. The cut-off score was calculated using a criterion
used by Jacobson and Truax (1991) which has previously been used in studies using CORE-OM
as outcome measure (Connell et al., 2007).
Mean(clin) x SD(norm) + Mean(norm) x SD(clin)
SD(norm) + SD (clin)
Using clinical scores (mean (clin), SD (clin)) from this sample and the non clinical scores (mean
(norm), SD (norm)) (CORE-OM mean = 9.0, SD = 5.2) from Elfström et al., (in press) this cutoff score was computed to be 13.4. The cut-off limit was therefor set at 13. This score is
comparable to Barkham et al., (2006) who found a cut-off score of 11.9 for men and 12.9 for
women. Congruence in working alliance was measured as the mean difference between the
therapist’s and the patient’s overall alliance scores. It was computed by adding the difference
between the client’s and the therapist’s alliance ratings at session level. The sum of differences
16
was calculated and divided by the number of sessions creating a mean value of differences in
alliance ratings for each therapy.
Results
Analysis of the whole sample
The treatments in the whole sample were of varying length (mean = 6.1, median = 5, SD
= 4.1). Mean distress- and problem level at first session was CORE-OM = 17.3 (SD = 5.50) and
mean distress level at last session was CORE-OM = 12.4 (SD = 6.20). The effect of the
treatments was calculated using a paired samples t-test, t (699) = 22.03, p < 0.01, Cohen’s d =
0.83, showing a large effect size for the whole sample. Average working alliance level for clients
was for WAI-S = 5.58 (SD = 0.89) and average working alliance level for therapists was for
WAI-S = 5.23 (SD = 0.80). Clients rated the working alliance significantly higher than the
therapists (t (699) = 9.60, p < 0.01, Cohen’s d = 0.36). Congruence in working alliance (WAI
Con) in this study was measured as the absolute mean difference (difference score) in working
alliance, assessed by subtracting therapists’ scores from the clients’. For all treatments the mean
congruence score was WAI Con = 0.89 (SD = 0.59). Due to the relatively large standard
deviation for WAI Con the distribution was further analyzed. The analysis showed that the data
was somewhat skewed with a peak indicating a leptokurtic distribution with too many values in
the low range (S = 1.12 and K = 1.24). The mean scores and standard deviations are displayed in
table 1. Table 1 also includes statistics for two groups of treatments, one consisting of treatments
with five treatment sessions or more and one group consisting of four treatment sessions or less.
This is described later in this section.
17
Table 1
Means of CORE- OM pre and post treatment, WAI-S ratings by therapist (WAI T),
client (WAI C) and congruence (WAI Con).
mean
SD
max
min
All treatments (n =700):
Number of sessions
CORE-OM pre
CORE-OM post
WAI C
WAI T
WAI Con
6.09
17.3
12.4
5.58
5.23
0.89
4.08
5.50
6.20
0.89
0.80
0.59
29.0
31.8
35.6
7.00
7.00
3.58
2.00
2.10
0.00
2.83
2.54
0.00
Five sessions or more (n = 392):
Number of sessions
CORE-OM pre
CORE-OM post
WAI C
8.51
17.5
11.8
5.72
3.99
5.40
5.90
0.81
29.0
31.8
35.6
6.99
5.00
4.20
0.00
2.87
WAI T
5.37
0.74
6.92
3.25
WAI Con
0.89
0.53
2.92
0.04
Four sessions or less (n =308):
Number of sessions
CORE-OM pre
CORE-OM post
WAI C
WAI T
WAI Con
3.01
17.09
13.17
5.41
5.06
0.90
0.80
5.66
6.52
0.95
0.85
0.65
4.00
31.2
31.5
7.00
7.00
3.58
2.00
2.10
0.00
2.83
2.54
0.00
In order to analyze if congruence in working alliance was associated with treatment
outcome correlations between WAI Con and outcome as measured by CORE-OM were
computed. The results showed no significant correlation between congruence in working alliance
and outcome (r = .05, ns). The clients’ and therapists’ ratings of the working alliance were
associated with treatment outcome (r = .16, p < 0.01 and r = .08, p < 0.05 respectively) as shown
in table 2. Table 2 also displays correlations for treatment subgroups presented later in this
section.
18
Table 2
Correlations between outcome and alliance scores.
CORE-OM for:
All treatments (n = 700)
Number of sessions ≥ 5 (n = 392)
Number of sessions < 5 (n = 308)
Clients rate alliance higher (n = 458)
Therapists rate alliance higher (n = 242)
WAI Client
WAI Therapist
WAI Congruence
0.16**
0.19**
0.08
0.17**
0.09
0.08*
0.09
0.01
0.10*
0.13*
0.05
0.02
0.08
0.04
0.01
* p < 0.05, ** p < 0.01
Analysis of a sample consisting of five treatment sessions or more
The sample consisted of treatments of varying length, some of them relatively short. To
test whether treatments consisting of five treatment sessions (median score) or more differed in
outcome from shorter treatments an independent t-test was performed. The difference in outcome
between treatments with five sessions or more and treatments with four sessions or less was
statistically significant ( t (392) = 3.96, p < 0.01, Cohen’s d = 0.31) indicating that treatments
consisting of five treatment sessions or more had better outcome (for sample characteristics see
table 1). Alliance ratings also differed significantly between the two groups. The alliance ratings
both by the therapists and clients in the sample consisting of five treatment sessions or more
were higher than for the group with four treatment sessions or less although the effect sizes were
relatively small (t (698) = 5.06, p < 0.01, Cohen’s d = 0.38 and t (698) = 4.72, p < 0.01, Cohen’s
d = 0.36 respectively). In both groups clients tended to rate the alliance higher than the therapists
(t (303) = 5.76, p < 0.01, Cohen’s d = 0.33 for the group with longer treatments and t (231) =
4.68, p < 0.01, Cohen’s d = 0.31 for the group with shorter treatments).
19
The results for the sample with longer treatments were similar to the results for the whole
group indicating that congruence in alliance ratings was not associated with treatment outcome (r
= .02, ns). Clients’ ratings of the working alliance were still significantly correlated with
outcome (r = .19, p < 0.01) while therapists’ alliance scores were no longer significantly
associated with outcome (r = .09, ns, see table 2).
Analysis of the sample when clients or therapists rate the alliance higher
Since WAI Con does not take in to consideration whether the client or the therapist rates
the working alliance higher a separate analysis was conducted where the sample was split into
two groups. One group consisted of treatments where the client rated alliance higher than the
therapist and the other group consisted of treatments where the therapist rated the alliance higher
than the client. An independent samples t-test indicated that treatments where clients rated the
alliance higher were superior in terms of outcome, t (698) = 2.21, p < 0.05, Cohen’s d = 0.18.
Correlations between outcome and alliance were tested separately in these two groups. In
treatments where the therapist rated the alliance higher than the client the correlation between
WAI Con and outcome was r = .01 (ns.) while the correlation between therapists’ ratings and
outcome was r = .13 (p < 0.05). Clients’ ratings of the working alliance was not significantly
associated with outcome (r = .09, ns). In the group where the clients rated the alliance higher
than the therapists the correlation between WAI Con and outcome was r = .04 (ns.) The
correlation between clients’ ratings of the alliance and outcome was r = .17 (p < 0.01). The
correlation between therapists’ ratings of the alliance and outcome scores was r = .10 (p < 0.05).
All correlations are displayed in table 2.
20
Analysis of treatments with clients showing clinically significant distress
An additional analysis of the association between alliance congruence and outcome was
made in the group of patients that started therapy above the cut-off score for clinical significant
distress. This group consisted of 536 therapies with mean number of treatment sessions 6.3
(median = 5, SD = 4.3). Mean distress- and problem level, measured at first session was COREOM = 19.6 (SD = 4.10) and mean distress level after the last session was found to be CORE-OM
= 13.6 (SD = 6.20). The effect of the treatments was calculated using a paired samples t-test (t
(535) = 23.69, p < 0.01, Cohen’s d = 1.02) showing a significant and large effect size. Average
working alliance level for clients was WAI-S = 5.51 (SD = 0.91) and average working alliance
level for therapists was WAI-S = 5.20 (SD = 0.82). Clients again rated the working alliance
significantly higher than the therapists (t (535) = 7.35, p < 0.01, Cohen’s d = 0.32) although the
effect size was small. For all treatments in this sample, the mean congruence score was WAI Con
= 0.91 (SD = 0.59). The distribution for WAI Con was further analyzed due to the relatively
large standard deviation. The analysis once again showed that data was somewhat skewed with a
peak indicating a leptokurtic distribution with too many values in the low range (S = 1.09 and K
= 1.26). The mean scores and standard deviations are displayed in table 3.
21
Table 3
Means of CORE- OM pre and post treatment, WAI-S ratings by therapist (WAI T),
client (WAI C) and congruence (WAI Con) for CORE-OM pre ≥ 13.
mean
SD
max
min
All treatments (n =536):
Number of sessions
6.28
4.28
29.00
2.00
CORE-OM pre
19.56
4.10
31.80
13.00
CORE-OM post
WAI C
WAI T
WAI Con
13.58
5.51
5.20
0.91
6.20
0.91
0.82
0.59
35.60
7.00
7.00
3.58
0.00
2.83
2.54
0.00
Five sessions or more (n = 304):
Number of sessions
CORE-OM pre
CORE-OM post
WAI C
8.77
19.60
12.80
5.65
4.19
3.95
6.00
0.83
29.00
31.80
35.60
6.99
5.00
13.20
0.00
2.87
WAI T
5.35
0.75
6.92
3.25
WAI Con
0.88
0.51
2.92
0.04
Four sessions or less (n =232):
Number of sessions
CORE-OM pre
CORE-OM post
WAI C
WAI T
WAI Con
3.03
19.45
14.59
5.34
5.00
0.94
0.81
4.18
0.64
0.99
0.86
0.69
4.00
31.20
31.50
7.00
7.00
3.58
2.00
1.30
1.20
2.83
2.54
0.00
The results for this group showed no significant correlation between congruence in
working alliance and outcome (r = .04, ns). Both clients’ and therapists’ scores of alliance were
positively and significantly correlated with outcome scores (r = .24, p < 0.01 and r = .13, p <
0.05 respectively) as shown in table 4. Table 4 contains information analyzed later in the study.
22
Table 4
Correlations between outcome and alliance scores for CORE-OM pre ≥ 13
CORE-OM for:
All treatments (n = 536)
Number of sessions ≥ 5 (n = 304)
Number of sessions < 5 (n = 232)
Clients rate alliance higher (n = 342)
Therapists rate alliance higher (n = 194)
WAI Client
WAI Therapist
WAI Congruence
0.24**
0.28**
0.14*
0.27**
0.15*
0.13*
0.12*
0.06
0.16**
0.16*
0.04
0.07
0.04
0.07
0.05
* p < 0.05, ** p < 0.01
Analysis of treatments with clients showing clinically significant distress and consisting of five
treatment sessions or more
A separate analysis of the treatments where CORE- OM at first session was 13 or more
and lasted for five treatment sessions or more (median treatment length). To test whether this
sample yielded higher outcome scores than the sample consisting of four sessions or less an
independent t-test was performed. Treatment outcome for the sample consisting of treatments
with five sessions or more was found to be superior, t (534) = 3.96, p < 0.001, Cohen’s d = 0.35,
mirroring previous results in this study. Alliance ratings once again differed significantly
between the two groups where the ratings both by the therapists and clients in the sample
consisting of five treatment sessions or more were higher than for the group with four treatment
sessions or less (t (534) = 5.10, p < 0.01, Cohen’s d = 0.44 and t (534) = 3.99, p < 0.01, Cohen’s
d = 0.35 respectively). However, the effect sizes were relatively small. For both groups (five
treatment sessions or more and four treatment sessions and less) clients tended to rate the
alliance higher than the therapists (t (303) = 5.76, p < 0.01, Cohen’s d = 0.33 and t (231) = 4.68,
p < 0.01, Cohen’s d = 0.31). Sample characteristics are shown in table 3. The results for the
group consisting of five sessions or more were also found to be similar to previous analysis
23
regarding the association between alliance measures and treatment outcome. The result found
was that congruence in alliance ratings was not associated with treatment outcome (r = .07, ns)
and clients’ and therapists’ ratings of the working alliance were once again found to be
significantly correlated with outcome (r = .28, p < 0.01 and r = .12, p < 0.05 respectively, see
table 4).
Analysis of treatments with clients showing clinically significant distress and clients or therapists
rate the alliance higher
This sample was also divided into two groups, one group representing treatments where
the clients rated alliance higher than the therapists and the other group representing treatments
where the therapists rated the alliance higher than the clients. An independent samples t-test was
performed showing that treatments where clients rated the alliance higher were superior in terms
of outcome, t (534) = 2.24, p = 0.03, Cohen’s d = 0.21, again yielding a small effect size. The
results from the analysis, where alliance scores were correlated with outcome, also showed
similarities with previous results (see table 4). In treatments where the therapist rated the alliance
higher than the client there was no relationship between WAI Con and outcome (r = .05, ns)
while the correlation between therapists’ ratings and outcome was r = .16 (p < 0.05). Clients’
ratings of the working alliance in this sample was also significantly correlated with outcome (r =
.15, p < 0.05). In the group where clients rated the alliance higher than the therapists the results
yet again show no correlation between WAI Con and outcome (r = .07, ns) while the correlation
between clients’ ratings of the alliance and outcome was r = .27 (p < 0.01). Therapists’ ratings of
the alliance and outcome scores were also significant and positively correlated (r = .16, p <
0.01).
24
Discussion
The primary objective of the study was to investigate whether congruence in working
alliance as rated by client and therapist independently was associated with outcome. In an
attempt to capture this interactional level of the working alliance in this study a measure of
congruence had to be defined. The measure used to capture congruence in alliance scores was
constructed as the mean difference between the clients’ and therapists’ ratings of the alliance
(WAI Con). Several analyses’ were performed to try to analyze if this construct was associated
with treatment outcome. A secondary objective was to study how the clients’ and the therapist’
ratings of the working alliance in a large naturalistic sample was associated with treatment
outcome and whether the results were comparable to previous findings in the alliance literature.
The main reasons for this approach was that previous research related to the association between
the working alliance and treatment outcome has found both theoretical and methodological
limitations and yielded somewhat mixed results even though the main findings indicate that
alliance is a stable and moderately strong predictor of treatment outcome (Horvath, Del Re,
Flückiger & Symonds, 2011). The theoretical concerns that have been brought forward are
mainly that the working alliance as a construct is dyadic in theory but usually captured through
self-report measures rated by client and/or therapist independently. Authors representing a more
interpersonal position argue that the use of subjective ratings by two individual raters might miss
out on crucial information of a more dyadic and interpersonal nature which in the end might
produce erroneous or skewed results.
Previous studies that have tried to capture the working alliance at a dyadic level have
used different definitions of congruence and have found mixed results. Fitzpatrick, Iwakabe and
25
Stalikas (2005) argue that patient and therapist ratings of the alliance are significantly divergent,
with greater differences for ratings of the task- and goal-aspects of the alliance construct than for
the bond aspect. They also found that the level of divergence/convergence did not seem to
change over the course of treatment. They also concluded that there is no relationship between
differences in alliance scores and session impact. Tryon, Blackwell and Hammel (2007), argue
that regardless of therapy orientation, client´s level of distress, length of therapy, type of measure
being used and the therapist´s experience only moderate correlations could be found between
therapists’ and clients’ ratings of the working alliance and that clients rated the alliance
significantly higher. This result mirrors Horvath and Symmonds (1991) conclusions to some
extent. However, Kivlighan and Arthur, (2000), found that congruence increased over time
during treatment.
Rozmarin et al., (2008) conceptualized congruence of alliance as a correlation index and
found that this index was a stronger predictor of therapy outcome than the patient’s and the
therapist´s ratings respectively. This measure correlated with clients’ and therapists’ alliance
ratings to some degree but seemed to capture another aspect of the alliance construct. Kivlighan
and Marmarosh (2012), using polynomial regression and response surface analysis also found
mixed results and argue that congruence seems to be important in predicting some aspects of
treatment outcome or change. Their results also indicated that early congruence in alliance
ratings was associated with symptom change but at the same time increased levels of
disagreement in alliance seemed to be associated with greater symptom improvements.
The initial analysis was made using the whole sample consisting of 700 treatments of
varying lengths. Contrary to the initial hypothesis the correlations found were small and non
significant indicating that congruence in the working alliance was not a predictor of treatment
26
outcome. Further analyses using subgroups gave similar results: no association between
congruence and outcome. In one sub-grouping the sample was divided into treatments with five
sessions or more and treatments with four sessions or less. No correlations between WAI Con
and treatment outcome were found in the either group. However the group with five treatment
sessions or more had a slight advantage in terms of treatment outcome which could be said to
indicate that treatment length may be a more important factor for treatment outcome than
whether the clients’ and therapists’ were in agreement or congruent in their perception of the
quality of the working alliance. In an attempt to control for the problem, described in previous
studies (Kivlighan, 2010; Kivlighan & Marmarosh, 2012), that difference scores fail to take into
account whether the clients’ or the therapists’ score the alliance higher a separate analysis was
made by splitting the sample into two groups based on which rater scored the alliance higher.
However no correlations between WAI Con and treatment outcome were found in either group.
As a final step of analysis all treatments meeting a pre-established cut-off score for clinically
significant levels of distress were studied. Results from these analyses did not show any
association between congruence in working alliance ratings and treatment outcome regardless of
how the sample was grouped for analysis. These results are to some extent somewhat surprising
given previous theoretical assumptions and research findings where it has been argued that a
strong alliance, indicating agreement or congruence, between clients and therapists is one
important factor for positive treatment outcome. The results in this study show that the level of
congruence in alliance ratings is not associated with outcome while a closer analysis of the
distribution of this measure also indicates that it is quite common with low discrepancies in
therapists’ and clients’ alliance ratings for this sample. The result could indicate that there are
typically no clinically important differences between raters throughout successful treatments
27
which would explain the lack of association with outcome. The results could also be interpreted
as a consequence of therapists typically acknowledging and working through alliance ruptures
whether they are aware of their occurrence or not. This result could however also signify that an
analysis based on mean scores for whole treatments fail to capture nuances or fluctuations of
alliance congruence within or between treatment sessions. These possible patterns of fluctuation
might hold important information about how congruence in alliance varies and how this could be
relevant to treatment processes, potentially giving clinically relevant indications for how to
improve treatment outcome. If this premise is true then one might argue that studies of
congruence in alliance and the potential association with outcome could benefit from a level of
analyses set at either between or within session level.
To summarize these results it made no difference whether the analysis’ were conducted
using the whole sample, treatments consisting of five treatment sessions or more, treatments
where the clients’ rated alliance higher than the therapists’, treatments where the therapists’ rated
the alliance higher than the clients’ or treatments where the clients’ pre-treatment scores on
CORE-OM met the cut-off level for clinically significant levels of distress. Congruence in the
working alliance did not correlate with treatment outcome.
As a secondary purpose the relationship between the working alliance and treatment
outcome was assessed in order to compare results from this naturalistic study with previous
research on the alliance and treatment outcome. Horvath (2001) found that the clients’ ratings of
early alliance was the best predictor of outcome and that the relationship between alliance ratings
and treatment outcome was found to be r = .22 and r = .15 for client and therapist ratings
respectively. Horvath et al., (2011) found that the overall aggregated correlation between alliance
and outcome was r = .28. The consensus in alliance research seems to be that the correlation
28
between alliance scores and outcome scores is moderate and frequently found in studies linking
alliance to treatment outcome (Norcross & Lambert, 2011).
In the present sample the clients consistently rated the alliance higher than the therapists
regardless of how the sample was grouped. In all groups the clients’ alliance ratings were
significantly higher than the therapists. For the whole sample, correlations between clients’
ratings of the alliance and treatment outcome mainly showed low levels of association between
client’s alliance ratings and treatment outcome. This result is somewhat different from
correlations found in the alliance-outcome literature. Correlations between therapists’ ratings of
the working alliance and treatment outcome were found to be even lower and slightly more
varied. The main result for the whole sample is that the correlations between therapists’ alliance
ratings and treatment outcome were in the small range, but in a few instances no correlation was
found. The association between alliance ratings and treatment outcome was however affected by
the clients’ pre-treatment levels of distress. The correlations between alliance level and treatment
outcome showed a more stable pattern for treatments where clients’ showed pre-treatment levels
of distress above the cut-off level for clinically significance. The correlations between alliance
and treatment outcome found in this group were predominantly comparable to previous research.
A possible explanation for this result is that treatments where clients start treatment at a subclinical level of distress have less room for improvement. If clients entered treatment with lower
scores on CORE-OM then there would be less room for variation and improvement on outcome
measures. A restricted span for improvement on an outcome measure would affect the possibility
of finding correlations between outcome and alliance negatively, hence the lack of association
between outcome and alliance in this sample compared to previous research on the alliance and
outcome. An interesting and clinically relevant result can also be found in the correlation
29
patterns when the sample was grouped based on who rated the alliance higher, the clients or the
therapists. In terms of outcome the group where clients rated the alliance higher showed better
outcome, regardless of pre-treatment level of distress. The correlations between alliance and
outcome were for the group where therapists rated the alliance higher mainly lower for the
clients. This result indicates that it is potentially of vital importance that therapists take note of
ongoing processes regarding alliance development. Judging by this result it would seem
important that therapists continually make realistic assessments of whether there is an agreement
or not regarding therapeutic goals and tasks and the quality of the relational bond if one does not
want a potentially negative therapeutic process and sub standard treatment outcome. This result
can be said to strengthen the hypotheses postulated by previous research which claims that it is
essential to continuously monitor and negotiate the therapeutic alliance, specifically when there
is reason to believe that the clients perceive that there is lack of agreement on the quality of the
relationship and how the treatment is moving along in regards to therapeutic aims and methods.
It can thus be argued that a possible explanation for this result is that these therapies reflect a
treatment process were therapists fail to acknowledge a lack of agreement or a lack of a shared
perception of the quality of the working alliance.
To summarize: Congruence, measured as a difference score based on clients’ and
therapists’ working alliance ratings, was not associated with treatment outcome. The results also
indicate that the client’s rating of the alliance seems to be a stronger predictor of outcome than
the therapist’s. The results in this study show mainly small to moderate, significant and positive
correlations between client’s alliance scores and treatment outcome whereas the therapist’s
scores range from non significant to small correlations. The results mirror and replicate the
results from previous research to a large extent. This conclusion was particularly true for
30
treatments with patients who showed clinically significant distress at the beginning of treatment.
For this group the correlation between clients’ and therapists’ alliance ratings and treatment
outcome were similar to previous research results. The results also show that therapies where
therapists rate the alliance higher than the clients seem to predict a more limited treatment
outcome, seemingly regardless of pre-treatment levels of distress.
Limitations
Practice based studies have both advantages and disadvantages. No control for treatment
integrity was made. The only control for type of treatment administered was the end of treatment
form where treatment modality was specified by the therapists. Because of the lack of control for
treatment integrity it can be argued that there are problems with generalizations of the results
across different types of psychological treatments and that congruence in alliance might be of
varying importance depending on treatment mode. Some support for this hypothesis can be
found in previous research where one study using a small sample and a treatment modality
specifically targeted at working through the working alliance and alliance ruptures found that
congruence in alliance had a significant impact on treatment outcome (Rozmarin et al., 2008).
However other research aimed at the association between congruence and outcome have found
non-significant or mixed results with treatment methods that were not specifically targeted at the
alliance (Kivlighan & Marmarosh, 2012). It is reasonable to argue that this sample could hold
important information due to the large sample size with a large number of different clients and
therapists and the fact that it represents treatment as performed in primary care even though the
sample was drawn from only one region of Sweden. This sample consisted of a mix of different
types of psychological treatments that also varied in length and previous researchers have
31
typically used small samples with limited range of treatment modalities (Rozmarin et al., 2008;
Kivlighan & Marmarosh, 2012) and argued that there is a need for research on samples of greater
size and multiple treatment types.
A second limitation in this study concerns the method of analyzing congruence. Some
researchers have argued that the use of difference scores and profile similarity correlations is
flawed due to statistical issues (Kivlighan & Marmarosh, 2012). One concern is that combining
data from two different raters into one measure makes interpretation of results uncertain:
reliability of the measure is not as good as for the clients’ and the therapists’ independent ratings.
The use of difference scores also fails to take into account the absolute level of alliance as this
type of measure only captures the difference between raters regardless of level of alliance.
A third problem is that a difference score does not identify who in the treatment dyad
rates the alliance higher, the client or the therapist. In this study there is also a problem with
dependency in data due to the fact that mean difference scores for the whole treatments are
calculated from alliance ratings at session level. This problem is to some extent reduced by the
use of a large sample but there is still a concern that the results are somewhat skewed due to this
limitation. The use of a mean difference score is also problematic due to insensitivity to possible
variations in congruence. When using this type of statistics it is not possible to find alternative
explanations or nuances that might shed some light on the lack of results. However previous
research has found mixed results regarding this topic. Fitzpatrick, Iwakabe and Stalikas (2005)
argue that there is no change in divergence over the course of treatment while Kivlighan and
Arthur (2000) found that convergence increases over time during psychotherapy. For future
research it seems prudent to recommend more advanced statistical methods that take these
limitations into account. One such approach is the one proposed by Kivlighan (2010) where
32
polynomial regression and response surface analysis was used to capture the association between
congruence and treatment outcome. However, it seems important to bear in mind that the results
in this study do not indicate any association between similarities in alliance ratings and treatment
outcome while the use of more advanced statistical methods have so far at best yielded
ambiguous results. Maybe these results actually show that the hypothesis that congruence in
alliance ratings is important for good treatment outcome is not accurate.
These results may be a consequence of several different factors including sample
characteristics and methodological limitations. One might however also argue that the results
confirm previous findings in alliance research and that the results clarify aspects of the
therapeutic process. If this argument is valid it would mean that the hypothesis that was tested in
this study was somewhat flawed or in need of refinement. These results may also partially be
explained by earlier reasoning which claims that clients and therapists hold different perspectives
on the quality of the alliance and that therapists may address and work through alliance ruptures
even though they are not aware of their occurrence (Kivlighan & Marmarosh, 2012).
33
References
Barkham, M., Margison, F., Leach, C., Lucock, M., Mellor-Clark, J., Evans, C., et al. (2001).
Service profiling and outcomes benchmarking using the CORE–OM: Towards practicebased evidence in the psychological therapies. Journal of Consulting and Clinical
Psychology, 69,184–196.
Barkham M, Connell J, Stiles WB., Miles JNV, Margison F, Evans C, et al. (2006). Dose-effect
relations and responsive regulation of treatment duration: The good enough level. Journal
of Consulting and Clinical Psychology, 74, 160–167.
Bordin, E.S. (1979). The generalizability of the psychoanalytic concept of the working alliance.
Psychotherapy: Theory, Research and Practice, 16, 252-260.
Clark- Carter, D. (2004). Quantitative psychological research. Hove: Psychology Press
Connell, J., Barkham, M., Stiles,W. B., Twigg, E., Singleton, N., Evans, O., et al.(2007).
Distribution of CORE-OM scores in a general population, clinical cut-off points, and
comparison with the CIS-R. British Journal of Psychiatry, 190, 69–74.
Constantino, M.J., Castonguay, L.G. & Schut, A.J. (2002). The working alliance: A flagship for
the “scientist-practitioner” model in psychotherapy. In G.S. Tryon (Ed.), Counseling
based on process research: Applying what we know (pp. 81-131). Boston: Allyn &
Bacon.
Elliot, R., & James, E. (1989). Varieties of client experience in psychotherapy: An analysis of
the literature. Clinical Psychology Review, 9, 443-467.
34
Evans, C., Connell, J., Barkham, M., Margison, F., Mellor-Clark, J., McGrath, G., Audin, K.
(2002).Towards a standardized brief outcome measure: psychometric properties and
utility of the CORE-OM. British Journal of Psychiatry, 180, 51–60.
Elvins, R. & Green, J. (2008). The conceptualization and measurement of therapeutic alliance:
An empirical review. Clinical Psychology Review, 28, 1167-1187.
Elfström, M., Evans, C., & Carlsson, S. (in press).Validation of the Swedish version of the
Clinical Outcomes in Routine Evaluation Outcome Measure (CORE-OM). Clinical
Psychology and Psychotherapy
Freud, S. (1913). On the beginning of treatment: Further recommendations on the technique of
psychoanalysis. In J. Strachey (Ed.), Standard edition of the complete psychological
works of Sigmund Freud (Original work published 1913, Vol. 12, pp. 122-144). London:
Hogarth Press.
Fenton, L.R., Cecero, J.J., Nich, C., Frankforter, T.L., & Carroll, K.M. (2001). Perspective is
everything: The predictive validity of working alliance instruments. Journal of
Psychotherapy Practice and Research, 10, 262-268.
Fitzpatrick, M.R., Iwakabe, S., & Stalikas, A. (2005). Perspective divergence in the working
alliance. Psychotherapy Research, 15, 69-79.
Gaston, L., Goldfried, M.R., Greenberg, L.S., Horvath, A.O., Raue, P.J. & Watson, J. (1995).
The therapeutic alliance in psychodynamic, cognitive-behavioural and experiential
therapies. Journal of Psychotherapy Integration, 15, 1-26.
Greenson, R. R. (1965). The working alliance and the transference neuroses. Psychoanalysis
Quarterly, 35,155-181.
35
Horvath, A. O. (1994). Empirical validation of Bordin's pan theoretical model of alliance: the
working alliance inventory perspective. In A. O. Horvath, & L. S.Greenberg (Eds.),
The working alliance: theory, research and practice (pp. 109−128). New York: Wiley.
Horvath, A. O. (2005). The therapeutic relationship: Research and theory – An introduction to
the special issue. Psychotherapy Research, 15, 3-7.
Horvath, A.O., Del Re, A. C., Flückiger, C. & Symonds, D. (2011). Alliance in Individual
Psychotherapy. Psychotherapy, 48(1), 9-16.
Horvath, A., Krause, M., & Altimir, C. (2011). Deconstructing the therapeutic alliance:
Reflections on the underlying dimensions of the concept. Clinica y Salud, 22(3), 267-283.
Horvath, A., & Symonds, B.D. (1991). Relation between working alliance and outcome in
psychotherapy: A meta-analysis. Journal of Counseling Psychology, 38, 139-149.
Jacobson, N.S., & Truax, P. (1991). Clinical significance: a statistical approach to defining
meaningful change in psychotherapy research. Journal of Consulting and Clinical
Psychology, 59,12–19.
Kivlighan, D.M. & Shaugnessy, P. (1995). An analysis of the development of the working
alliance using hierarchical linear modeling. Journal of Counseling Psychology, 42, 338349.
Kivlighan, D. M. & Arthur, E. G. (2000). Convergence in client and counselor recall of
important session events. Journal of Counseling Psychology, 47, 79-84.
Kivlighan, D.M. (2007). Where is the relationship in research on the alliance? Two methods for
analyzing dyadic data. Journal of Counseling Psychology, 54(4), 423-433.
36
Lichtenberg, J.W., Wettersten, K.B., Mull, H., Moberly, R.L., Merkley, K.B. & Corey
Tiongson, A. (1998). Relationship formation and relational control as correlates of
psychotherapy quality and outcome. Journal of Counseling Psychology, 45(3), 322-337
Marmarosh, C.L. & Kivlighan, D.M. (2012). Relationships among client and counselor
agreement about the working alliance, session evaluations, and change in clients
symtoms using response surface analysis. Journal of Counseling Psychology, 59(3), 352367.
Martin, D. J., Garske, J.P. & Davis, M.K. (2000). Relation of the therapeutic alliance with
outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical
Psychology, 68(3), 438-450.
Raue, P. J., Castonguay, L. G., & Goldfried, M. R. (1993). The working alliance: a comparison
of two therapies. Psychotherapy Research, 3, 197−207.
Rozmarin, E., Muran, C. J., Safran, J., Gorman, B., Nagy, J. & Winston, A. (2008). Subjective
and intersubjective analysis of the therapeutic alliance in a brief relational therapy.
American Journal of Psychotherapy, 62(3), 313-328.
Safran, J.D., & Muran, J.C. (2000). Negotiating the therapeutic alliance: A relational treatment
manual. New York: Guilford Press.
Safran, J.D. (2002). Brief relational psychonalytic treatment. Psychoanalytic Dialogues, 12,
171–195.
Samstag, L., Batchelder, S. T., Muran, J. C., Safran, J. D., & Winston, A. (1998). Early
identification of treatment failures in short term psychotherapy. An assessment of
therapeutic alliance and interpersonal behavior. Journal of Psychotherapy Practice and
Research, 7, 126−143.
37
Sterba, R. (1934). The fate of the ego in analytic therapy. International Journal of
Psychoanalysis, 15, 117-126.
Strong, S. (1982). Emerging integration of clinical and social psychology: A clinicians
perspective. In G. Weary & H. Mirels (Eds.), Integration of clinical and social
psychology (pp 181-213). New York: Oxford University.
Swift, J. & Callahan, J. (2009). Early psychotherapy process: An examination of client and
trainee clinician perspective convergence. Clinical Psychology and Psychotherapy. 16,
228-236.
Tracey, T. J. & Kokotovic, A.M. (1989). Factor structure of the working alliance inventory.
Journal of Consulting and Clinical Psychology, 57, 207-210.
Tryon, G.S., Blackwell, S.C. & Hammel, E.F. (2007). A meta-analytic examination of client
therapist perspectives of the working alliance, Psychotherapy Research, 17, 629-642.
Zetzel, E. (1956). Current concepts of tranceference. International Journal of Psychoanalysis,
37, 369-375.
38
[First Authors Last Name] Page 39
[Insert Running title of <72 characters]