Download Psychotherapy Research

This article was downloaded by:[Society for Psychotherapy Research (SPR)]
On: 6 December 2007
Access Details: [subscription number 762317397]
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954
Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Psychotherapy Research
Publication details, including instructions for authors and subscription information:
http://www.informaworld.com/smpp/title~content=t713663589
Meta-Analysis of Therapist Effects in Psychotherapy
Outcome Studies
Paul Crits-Christoph a; Kathryn Baranackie a; Julie Kurcias a; Aaron Beck a;
Kathleen Carroll b; Kevin Perry c; Lester Luborsky a; A. McLellan a; George
Woody a; Larry Thompson d; Dolores Gallagher d; Charlotte Zitrin e
a
University of Pennsylvania.
b
Yale University.
c
Northwestern University.
d
Stanford University School of Medicine.
e
Long Island Jewish Hillside Medical Center.
Online Publication Date: 01 October 1991
To cite this Article: Crits-Christoph, Paul, Baranackie, Kathryn, Kurcias, Julie,
Beck, Aaron, Carroll, Kathleen, Perry, Kevin, Luborsky, Lester, McLellan, A., Woody, George, Thompson, Larry,
Gallagher, Dolores and Zitrin, Charlotte (1991) 'Meta-Analysis of Therapist Effects in Psychotherapy Outcome Studies',
Psychotherapy Research, 1:2, 81 - 91
To link to this article: DOI: 10.1080/10503309112331335511
URL: http://dx.doi.org/10.1080/10503309112331335511
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf
This article maybe used for research, teaching and private study purposes. Any substantial or systematic reproduction,
re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly
forbidden.
The publisher does not give any warranty express or implied or make any representation that the contents will be
complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be
independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or
arising out of the use of this material.
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
Psychotherapy Research l ( 2 ) 81-91, 1991
META-ANMYSIS OF THERAPIST EFFECTS
IN PSYCHOTHERAPY OUTCOME STUDIES
Paul Crits-Christoph, Kathryn Baranackie, Julie S. Kurcias,
Aaron T. Beck
University of Pennsylvania
Kathleen Carroll
Yale University
Kevin Perry
Northwestern University
Lester Luborsky, A. Thomas McLellan, George E. Woody
University of Pennsylvania
Larry Thompson, Dolores Gallagher
Stanford University School of Medicine
Charlotte Zitrin
Long Island Jewish Hillside Medical Center
In a meta-analysis, we examined factors that could account for the
differences in therapist efficacy evidenced in psychotherapy outcome
studies. The factors investigated were: ( 1) the use of a treatment manual,
(2) the average level of therapist experience, (3) the length of treatment,
and (4) the type of treatment (cognitivehehavioral versus psychodynamic). Data were obtained from frfteen psychotherapy outcome
studies that produced 27 separate treatment groups. For each treatment
group, the amount of outcome variance due to differences between
therapists was calculated and served as the dependent variable for the
meta-analysis. Each separate treatment group was coded on the above
four variables, and multiple regression analyses related the independent
variables to the size of therapist effects. Results indicated that the use of a
treatment manual and more experienced therapists were associated with
small differences between therapists, whereas more inexperienced therapists and no treatment manual were associated with larger therapist
effects. The findings are discussed in terms of the design and the analysis
of psychotherapy outcome research.
Preparation of this manuscript was supported in part by National Institute of Mental Health Grant
MH-40472 and NIMH Career Development Award MH-00756to Paul Crits-Christoph.The authors wish
to acknowledge the generosity of the investigators who lent us their data, including David Barlow, Tom
Borkovec, Tim Brown, Joseph Collins, Robert DeReubeis, Irene Elkin and Principal Investigators of the
NIMH Collaborative Treatment of Depression Program (John Watkins, Stuart Sotsky, Stan Imber), Steve
Hollon, Bernard Liberman, Paul Pilkonis, and William Piper.
Address correspondence to Paul Crits-Christoph,Ph.D. Hospital of the University of Pennsylvania
308 Piersol Building, 3400 Spruce Street Philadelphia, PA 19104-4283.
81
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
82
CRITS-CHRISTOPH ET AL.
The therapist’s contribution to psychotherapy outcome has long been a topic of
concern among clinicians and researchers (Beutler, Crago, & Arizmendi, 1986;
Gurman & Razin, 1977). Recently, reports of differences between therapists in their
patients’ outcomes have been appearing (Luborsky, McLellan, Woody, O’Brien, &
Auerbach, 1985, Luborsky et al., 1986;see review by Lambert, 1989). The statistical
implications of such “therapist effects” for conducting comparative studies of psychotherapies have been detailed by Crits-Christoph and Mintz (1991). These authors also observed a wide variation in the magnitude of therapist effects across a
number of studies. We report here the results of a meta-analysis attempting to
describe and to account for some of the variability in therapist effects across studies.
Although a variety of factors might be hypothesized to account for why some
studies evidence larger therapist effects than others, we chose to study four particular factors. Because of the need, given the sample size, to restrict the number of
independent variables, only four factors were studied. The four factors included
were treatment type, length of treatment, use of a treatment manual, and experience
level of the therapists. We describe the rationale for our choice of each of these
factors and our hypotheses about them, below.
Of primary interest for our meta-analysiswas the question of whether the use of
a treatment manual leads to smaller therapist effects. The original justification for the
implementation of treatment manuals in psychotherapy outcome research was
based upon the need to standardize the delivery of treatment techniques (Luborsky
& DeRubeis, 1984). To the extent that techniques produce therapeutic change, it
follows that making the delivery of such techniques uniform across therapists should
reduce differences between therapists in their outcomes. Studies that did not
employ treatment manuals, therefore, should be on the average more prone to yield
therapist effects.
Experience level of therapists is an obvious variable to explore in terms of its
relevance to therapist effects. In fact, Perry and Howard (1989) recently reported
data from three samples where the size of therapist effects may have been a function
of differential experience levels. It might be expected that a sample of inexperienced therapists (e.g., psychology interns or psychiatric residents) would
contain some ineffective therapists. These therapists might improve with further
training, or they might choose not to pursue careers as therapists after being
disappointed by their results. A sample of inexperienced therapists might also
contain some naturally gifted therapists, and thus variability within such a sample
could be expected. On the other hand, a group of experienced therapistsespecially those who participate in research studies-might be expected to be more
uniform in their delivery of treatment.
Type of treatment is an additional factor that could be related to the size of
therapist effects. In more structured therapies (e.g., cognitive or behavioral treatments), where the treatment approach is well defined, there may be less opportunity for the therapist to stray off course. Psychodynamic therapies, in contrast,
generally do not spec@ or mandate precise therapist behaviors; therefore, the
dynamic therapists may have more leeway to go in different directions. The complexity of the theory and clinical material in psychodynamic therapy may also
require more skill and/or produce more varied therapist interpretations and responses. Fewer therapists may be proficient at these complex skills, and therefore
larger differences between therapists would be expected for psychodynamic treatment.
Length of treatment is the final factor considered as a predictor of therapist
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
META-ANALYSIS O F THERAPIST EFFECTS
83
effects. We hypothesized that short-term,goal-directed therapies, regardless of the
treatment orientation, would be expected to show more uniform outcomes across
therapists. In a longer-term treatment it might be expected that the therapist’s skill
or personality would have more positive or negative effects on patients; increased
variability among therapists would result.
METHOD
SAMPLE
Data were obtained from the following treatment studies for inclusion in the
meta-analysis:
Woody, McLellan, Luborsky, and O’Brien (1990)
Beck, Hollon, Young, Bedrosian, and Budenz (1985), and Rush, Beck, Kovacs,
and Hollon ( 1977) (data from these two studies were combined)
Pilkonis, Imber, Lewis, and Rubinsky (1984)
Hollon et al. (1983)
Piper, Debbane, Bienvenu, and Garant (1984)
Thompson, Gallagher, and Breckenridge ( 1987)
Klein, Zitrin, Woerner, and Ross (1983)
Luborsky and Crits-Christoph ( 1988)
Nash et al. (1965)
Borkovec and Mathews (1988)
Perry and Howard ( 1989)
Carroll, Rounsaville, and Gawin ( 1990a)
Carroll, Rounsaville, and Gawin ( 1990b)
Elkin et al. (1989)
Barlow, Craske, Cerny, and Klosko (1989)
Many of the above studies included comparisons between different forms of
treatments or different lengths of treatments. Because we were interested in the
relationship of these factors to the size of therapist effects, we included each
treatment group as a separate unit for the meta-analysis. We were, however, concerned with potential nonindependence resulting from the same therapists treating
patients in different treatment conditions within a study. For studies where therapists were crossed with treatment condition, and the treatment conditions were not
distinguishable on our independent variables (i.e., all treatment conditions involved
the same form of therapy and same length of treatment), we either selected one of
the treatment conditions for our meta-analysis, or examined the therapist main
effect (plus therapist by treatment interaction) across treatment conditions. For
studies that involved a comparison of psychotherapy plus medication with psychotherapy alone, or psychotherapy plus placebo, we selected the psychotherapy
alone or psychotherapy plus placebo condition, since a combined medication plus
psychotherapy condition might obscure potential differences between psychotherapists.
The above 15 studies produced a total of 27 treatment groups and a total of 141
therapists. Each of the treatment groups is described briefly below. All studies
involved the treatment of outpatients.
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
84
CRITS-CHRISTOPH ET AL.
In data available to date from the Woody et al. (1990) study, 30 methadonemaintained, opiate-dependent patients were treated by three therapists using
dynamic psychotherapy. We combined the Beck et al. (1985) and Rush et al. (1977)
studies. Of the 37 depressed patients treated with cognitive therapy alone, we used
for our analysis only those therapists who treated at least two treatment-completing
patients. This left a total of eight therapists who treated 23 patients. The Pikonis
et al. (1984) investigation yielded three treatment groups: individual ( n = 22);
conjoint ( n = 21); and group ( n = 21). These were comprised of mixed diagnosis patients who were treated by nine therapists, three for each treatment
modality.
From the Hollon et al. (1983) study, we examined the data from the cognitive
therapy condition alone. This consisted of four therapists who provided cognitive
therapy to a total of 16 depressed patients. The Piper et al. (1984) study supplied
us with four groups: short-term individual therapy ( n = 21); long-term individual therapy ( n = 20); short-term group therapy ( n = 19); and long-term group
therapy ( n = 19). The patients in this study were all suffering from neurotic or
mild-to-moderate characterological problems. Three therapists, all psychoanalytically oriented, treated between five and eight patients with each therapy
modality.
We analyzed the data from the three treatment groups of the Thompson et al.
( 1987) study. The behavioral therapy group consisted of three therapists treating 2 5
patients. There were 27 patients in the cognitive therapy group who were treated
by four therapists. In the brief psychodynamic therapy group, three therapists
treated 24 patients. The patients were all elderly people with a diagnosis of major
depression. The Klein et al. (1983) study examined the treatment ofphobias (agoraphobia, simple phobia, and mixed phobia). From this study we examined the data
from the behavior therapy plus placebo (n = 67) condition. Five therapists participated.
The data from the psychodynamic treatment group of the Luborsky and CritsChristoph (1988) study were analyzed. A total of 23 depressed patients were treated
by four therapists. The Nash et al. ( 1965) project included two conditions (dynamic
therapy with and without a role induction interview), and a total of 40 patients.
Since the same four therapists treated patients in both conditions and the conditions
were the same in regard to our independent variables (i.e., treatment type, length,
and use or nonuse of a manual), we analyzed the data as a whole, calculating the
main effect for therapist and condition by therapist interaction. The Borkovec and
Mathews ( 1988) study compared three treatment groups (nondirective therapy,
coping desensitization, and cognitive therapy; all groups also included relaxation
therapy) in the treatment of generalized anxiety disorder and panic disorder. The
same four therapists were used in all three conditions. Thus, for this study we also
calculated the main effect for therapists across conditions and the therapist by
treatment interaction.
The Perry and Howard (1989) investigation provided data on five treatment
groups. Patients were mixed diagnosis outpatients treated with psychodynamic psychotherapy. One group of 80 patients was treated by 15 highly experienced therapists. A second group of 45 patients was treated by 8 moderately
experienced therapists. A third group of 30 patients was treated by 9 highly
experienced therapists. The fourth group of 40 patients was treated by 13 moderately experienced therapists; the fifth group consisted of 6 inexperienced therapists who treated 17 patients. The Carroll, Rounsaville, and Gawin ( 1990a) study
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
META-ANALYSIS OF THERAPIST EFFECTS
85
included four conditions in the treatment of cocaine abuse: a cognitive-behaviorallyoriented relapse prevention versus clinical management, crossed with medication
versus placebo. We selected the relapse prevention plus placebo condition ( n = 18
patients treated by 4 therapists) for inclusion in our meta-analysis.A second study by
Carroll, Rounsaville, and Gawin (1990b) on treatment of cocaine abuse included an
interpersonal psychotherapy condition. Two therapists treated a total of 2 1 patients.
For the Collaborative Treatment of Depression study (Elkin et al., l989), data were
available on 7 cognitive-behavioraltherapists who treated 36 completer patients and
9 interpersonal therapists who treated 46 completer patients. The Barlow et al.,
(1989) study included a total of 36 patients with panic disorder, and involved three
treatment conditions: cognitive therapy plus exposure, relaxation therapy, and a
combined cognitive plus exposure and relaxation condition. The same five therapists treated patients in all conditions, and therefore the data from this study were
analyzed as a whole.
MEASURES
The dependent variable of interest was the size of the therapist effect for each
treatment group. For each outcome measure within each separate group, we performed a one-way analysis of variance (or covariance when a pretreatment score
was appropriate to use as a covariate), specfying therapist as a random factor. From
the equations for expected mean squares we calculated the variance component for
the therapist factor, and divided it by total variance to obtain a percent of variance
explained by the therapist factor. For the Borkovec and Mathews (1988), Nash et al.
(1965), and Barlow et al. ( 1989) studies, which were not broken down into separate
conditions, we performed two-way ANOVAs with treatment condition and therapist
as factors. With these studies we combined the therapist main effect and therapist by
treatment interaction in order to obtain an overall index of therapist contribution to
outcome.
The number of separate, available outcome measures within each study ranged
from 1 to 18. The percent of variance due to therapist was averaged across all
outcome measures within each study. In addition, we retained the highest value for
each study as a measure of the largest therapist effect demonstrated, since the
average might tend to obscure therapist effects on certain measures.
The four independent variables were coded on each treatment group as follows.
For type of treatment, we created a dichotomous variable designating all cognitive
or behavioral treatments as one type, and all psychodynamic or interpersonal
treatments as another. Similarly, a dichotomous variable was coded for the use
versus the nonuse of a treatment manual in the study. For therapist experience,
we created a three-level variable: ( 1) limited experience (psychology intern, psychiatric resident, or less than one year of postdoctoraVpostresidency experience), ( 2 ) moderate experience (1 to 5 years of postdoctoral or postresidency
experience, and (3) extensive experience (greater than 5 years postdoctoral or
postresidency experience). The average level of experience of the therapists
for a given treatment group was coded into one of the above categories. Note that
these experience levels refer to total experience performing psychotherapy, not
experience with a given population under study (e.g., depressed patients). The
final independent variable was length of treatment. The score for this was the logarithm of the average number of treatment sessions completed by each group of
patients.
CRITS-CHRISTOPH ET AL.
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
86
RESULTS
Descriptive Characteristicsof Treatment Groups. Table 1 presents descriptive
information about the 27 treatment groups on the four independent variables, as
well as on the measures of size of therapist effects. Most notable is the variability
across treatment groups on the size of therapist effects. Some studies showed large
differences (up to 48.7% of the outcome variance) between therapists,while others
show no differences. This large variability allowed us to proceed with our attempt to
explain the variability from characteristics of the studies. The average treatment
group exhibited a therapist effect of 8.6% of the outcome variance (mean across
outcome measures). This is close to Cohen’s (1969) definition (ix., 9 % ) of a
medium-size effect for the behavioral sciences.
A treatment manual was employed for 48% of the treatment groups. Two-thirds
of the treatments were psychodynamic, and most (59% ) of the studies used highly
experienced therapists. There was, however, sufficient range on these variables to
allow testing their relationship to the size of therapist effects. The average treatment
lasted about 27 sessions.
Intercorrelations Among Independent Variables. The intercorrelations among
the four independent variables are presented in Table 2. As can be seen, a substantial
correlation ( r = -.75) between the use of a treatment manual and length of
treatment was evident (the manual guided therapies tended to have fewer sessions).
In addition, treatment type was moderately correlated ( r = .58) with use of a
manual (the psychodynamic therapies less often involved the use of a formal
treatment manual). Experience level of the therapists was relatively uncorrelated
with the other factors.
Table 1. Descriptive Data on 27 Treatment Groups
%
Percent of Variance due to therapist
Average across measures
Mean
SD
Range
Selecting largest effect
Mean
SD
Range
Use of Treatment Manual
yes
no
Length of Treatment (number of sessions)
Mean
SD
Experience of therapists
Inexperienced (less than 1 year)
Moderately experienced (1-5 years)
Highly experienced (greater than 5 years)
Type of treatment
Cognitive/Behavioral
Psychodynamic
8.6
10.1
0 - 48.7
22.3
19.8
0-72.9
48.1
51.9
27.6
24.2
14.8
25.9
59.3
33.3
66.7
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
META-ANALYSIS OF THERAPIST EFFECTS
87
Table 2. Intercorrelations Among Independent Variables
Length
Manual
Treatment type
( 1 = Dynamic, 2 = Cognitive/
Behavioral)
Use of manual
1 = no, 2 = yes)
Length of treatment
(log of sessions)
.58"
Therapist
Experience
-.42'
.oo
-.75**
-.08
.31
*p < .05.**p < .01. AU tests are two-tailed.
The high correlation between use of a manual and length of treatment makes it
difficult to unravel the unique effects of these factors. To avoid multicollinearity
problems in multiple regression analyses, we selected the treatment manual variable
for inclusion, rather than length of treatment, since it was of primary interest to us.
Prediction of Size of Therapist Effects. Simple correlations were calculated
between the four independent variables and the two measures of the size of the
therapist effect (average and largest therapist effect within each treatment group).
As can be seen in Table 3, both use of a treatment manual and therapist experience
evidenced statistically significant correlations with the average therapist effect. A
trend 0, < .lo) for type of treatment was also apparent. For the largest therapist
effect, type of treatment, use of a manual, and length of treatment showed significant
correlations. All relationships were in the predicted direction.
The results of a multiple regression analysis entering treatment type, use of a
manual, and therapist experience are presented in Table 4. Partial correlations of
each independent variable with the two criteria are given.
The partial correlations with average therapist effect revealed that both use of a
manual and therapist experience remain as statistically significant independent
predictors. Use of a treatment manual also produced a sizeable ( r = -.50) partial
correlation with the largest therapist effect criterion. The findings for type of
treatment, however, disappeared when we controlled for the overlap between it and
use of a manual. These three predictors jointly explained 4 1% of the variance in the
average therapist effects across studies (adjusted R squared = .33), and 42% of
the variance in the largest therapist effect (adjusted R squared = .34).
Table 3. Relationships of Treatment Type, Length, Use of Manual, and Therapist
Experience Level to Average Therapist Effect and Largest Therapist Effect in 27
Treatment Groups: Simple Correlations
Predictor
Average Effect
Treatment type
( 1 = Dynamic, 2 = CognitiveBehavioral)
Use of manual
( 1 = no, 2 = yes)
Therapist experience
Treatment length
*p < .lo. **p< .05.***p< .01. All tests are two-tailed
Largest Effect
-.33'
-.41"
-.45"
-. 58' ''
-.41"
.28
-.24
.39"
CRITS-CHRISTOPHET AL.
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
88
Table 4. Relationship of Treatment Type, Use of Manual,and Therapist Experience
Level to Average Therapist Effect and Largest Therapist Effect in 27 Treatment
Groups: Partial Correlations
Predictor
Average Effect
Largest Effect
Treatment type
( 1 = Dynamic, 2 = CognitiveBehavioral)
-.09
-.I0
Use of manual
(1 = no, 2 = yes)
-.42*'
-.50* *
Therapist experience
-. 50"
-.34'
*p < .10 **p< .05. ***p< .01. All tests are two-tailed
DISCUSSION
One important finding in our meta-analysis was that differences between therapists
in outcome were, on the average, at the level of a medium effect size. Wide
variability across studies, however, was apparent, rendering our analysis of the
factors contributing to the size of the therapist effects more important.
Summarizing our predictive findings, the factors most highly related to the size
of therapist effects were the use of a treatment manual and the experience level of
therapists. These variables gave the highest partial correlations with the largest
therapist effect and the average therapist effect, respectively. As mentioned, however, the overlap among the four predictor variables makes it difficult to separate the
independent effects of specific variables.
In addition to the confounding of independent variables, a variety of other
limitations of this meta-analysis must also be mentioned to put the findings in
context. For one, the relatively small sample size (N = 27 treatment groups) limited the statistical power of the analysis. Secondly, our collapsing the treatment
types into two broad categories (cognitive-behavioral versus psychodynamic)
may have obscured potential effects of a more fine-grained analysis of treatment
type. The cognitive-behavioral category, for example, ranged from a Beck cognitive therapy approach, through relapse prevention and exposure treatment.
In addition, our results say nothing about the factors related to therapist effects in forms of psychotherapy other than cognitive-behavioral and psychodynamic.
A third factor to consider in interpreting the results is the potential role played
by type of outcome measure. We analyzed the results across all available measures
without any attention to differences among types of measures. In addition, for some
studies, a large number of outcome measures were available; for other studies, only
one outcome measure was available.
Other aspects of the studies not examined here, such as characteristics of
the patient sample, may be important to the size of therapist effects. Certain patient diagnoses may respond to treatment more uniformly. Some patient diagnoses,
such as antisocial personality, may not respond at all to psychotherapy (Woody,
McLellan, Luborsky, & O'Brien, 19S5), and therefore differences between therapists will not emerge, since all therapists may achieve poor results. Moreover, therapist experience may interact with patient characteristics. For example,
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
META-ANALYSIS O F THERAPIST EFFECTS
89
with difficult patients, such as those with a borderline personality disorder, therapist
experience may be crucial. On the other hand, in the treatment of simple phobias with a behavioral approach, uniformly good outcomes may be attainable regardless of therapist experience. Number of years of therapist experience in
working with a particular population (e.g., elderly depressed patients) may be
more important than total amount of experience, as was measured in this metaanalysis.
Assuming the findings reported here are reliable, the results for use of a
treatment manual and therapist experience have implications for the design of
future research studies, as well as for the interpretation of published studies. These
data lend support to the movement in the psychotherapy research field towards the
standardization of the treatment variable through the use of treatment manuals and
the use of experienced therapists trained to performance criteria prior to the start of
the studies. These procedures appear to achieve their goal of reducing variability
due to differences among therapists.
In terms of the interpretation of published studies, the concerns raised by
Crits-Christoph and Mintz ( 199 1 ) regarding the statistical implications of therapist effects for conducting comparative outcome studies seem to be especially
relevant to the older treatment outcome studies that were less well controlled
(i.e., less experienced therapists were used and treatment manuals were not employed). In particular, Crits-Christoph and Mintz ( 1991) argue that by ignoring
the therapist factor, some investigators may have reported differences between
treatments that were actually a function of therapist differences. More recent comparative outcome studies that use experienced therapists and treatment manuals are
less likely to have large therapist effects, and therefore the results are less likely
to be in question because of unanalyzed therapist effects. Despite this tendency for the better-controlled, recent comparative studies to have smaller therapist effects, the suggestion of Crits-Christoph and Mintz ( 199 1 ) that investigators
routinely perform preliminary analyses for the presence of therapist effects is
still germane. The current meta-analysis is only preliminary, and our understanding
of the circumstances that produce differences between therapists is far from
complete.
The success of the treatment manuals and the use of experienced therapists in
reducing differences between therapists, while important to comparative outcome studies, does not necessarily suggest that all research studies should implement these types of controls. Rather, the use of these procedures to reduce
therapist variability would depend upon the particular research question at hand.
In fact, for many purposes, it may be advisable to maximize therapist differences.
If, for example, an investigator is interested in the relationship of the quality or
quantity of the therapist’s technique to the outcome of treatment, a less standardized sample of therapists may be necessary to test the hypothesis adequately.
The use of experienced, manual-guided therapists may restrict the range of the
technique variable of interest and prevent the uncovering of a relationship with
outcome. Further studies of the conditions that produce differences between therapists (e.g., Perry & Howard, 1989) also should not be restricted to the manualguided methodology that has otherwise become the standard in the field. A better
understanding of the psychotherapeutic process can come from both controlled
trials of standardized treatment packages, and more naturalistic investigations
of the differences between therapists in the process and the outcomes of their
treatments.
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
90
CFUTS-CHRISTOPH ET AL.
REFERENCES
Barlow, D. H., Craske, M. G., Cerny,J. A,, & Klosko,
J. S. (1989). Behavioral treatment of panic disorders. Behavior Therapy, 20, 261-282.
Beck, A. T., Hollon, S. D., Young, J. E., Bedrosian,
R C., & Budenz, D. (1985). Treatment of depression with cognitive therapy and amitriptyline. Archives of General Psychiatry, 42, 142148.
Beutler, L. E., Crago, M., & Arizmendi, T. G.
(1986). Therapist variables in psychotherapy
process and outcome. In S. L. Garfield & A. E.
Bergin (Eds.), Handbook of psychotherapy
and behavior change (pp. 257-310). New
York, NY:John Wiley and Sons.
Borkovec, T. D., & Mathews, A. M. (1988). Treatment of nonphobic anxiety disorders: A comparison of nondirective, cognitive, and
coping desensitizationtherapy.Journal of Consulting and Clinical Psychology, 56, 877-884.
Carroll, K, Rounsaville, B., & Gawin, F. (1990a).
Psychotherapy & pharmucotherapy for
cocaine abuse. Unpublished manuscript.
Carroll, K, Rounsaville, B., & Gawin, F. (1990b).
A comparative trial of psychotherapies for
ambulatory cocaine abusm: Relapse prevention and interpersonal psychotherapy. Submitted for publication.
Cohen, J. (1969). Statistical power analysis for
the behavioral sciences. New York: Academic
Press.
Crits-Christoph, P., & Mintz, J. (1991). Implications of therapist effects for the design and
analysis of comparative studies of psychotherapies.Journal of Consulting and Clinical Psychology, 59, 20-26.
Elkin, I., Shea, M. T., Watkins, J., Imber, S. D.,
Sotsky, S., Collins, J. F., Glass, D. R , Pilkonis, P.,
Leber, W. R., Docherty, J. D., Fiester, S. J., &
Parloff, M. B. (1989). National Institute of Mental Health Treatment of Depression Collaborative Research Program: General effectiveness of treatments. Archives of General Psychiat?y, 46, 971-982.
Gurman, A. S., & W i n , A. M. (Eds.). ( 1977).Effective psychotherapy: A handbook of research.
New York: Pergamon Press.
Hollon, S. D., Tuason, V. B., Wiemer, M. J., DeRubeis, R J., Evans, M. D., & Garvey, M. J.
( 1983). Cognitive therapy, pharmucotherapy,
and combined cognitive-pharmucotherapyin
the treatment of depression:I. Diffaential outcome in the CPT project. Unpublished manuscript.
Klein, D. F., Zitrin, C., Marker, Woerner, M. G., &
Ross, D. C. (1983). Treatment of p h o b i a s
11: Behavior therapy and supportive psychotherapy: Are there any specific ingre-
dients? Archives of General Psychiatry, 40,
139-145.
Lambert, M. J. (1989). The individual therapist’s
contribution to psychotherapy process and outcome. Clinical Psychology Review, 9, 46%
485.
Luborsky, L., & DeRubeis, R. (1984). The use of
psychotherapy treatment manuals: A small
revolution in psychotherapy research style.
Clinical Psychology Review, 4, 5-1 4.
Luborsky, L., McLellan, A. T., Woody, G. E.,
O’Brien,C. P., & Auerbach,A. (1985). Therapist
success and its determinants. Archives of
General Psychiaty, 42, 602-61 1.
Luborsky, L., Crits-Christoph,P., McLellan, A. T.,
Woody, G., Piper, W., Imber, S., & Liberman, B.
(1986). Do therapists vary much in their success? Findings from four outcome studies.
American Journal of Orthopsychiaty, 56,
501-512.
Luborsky, L., & Crits-Christoph,P. ( 1988, May). A
pilot study of psychodynamic psychotherapy
for major depressive disordff.Paper presented
at an NIMH workshop on planning of clinical
trials with psychodynamic psychotherapy,
Rockville, MD.
Nash, E., Hoehn-Sacric, R., Battle, C., Stone, A,,
Imber, S., & Frank, J. (1965). Systematic preparation of patients for short-term psychotherapy-11: Relation to characteristics of patient,
therapist, and the psychotherapeutic process.
Journal ofNeruous andMentaIDisordm, 140,
374-383.
Perry, K, & Howard, K I. (1989, June). Therapist
effects: In search of the therapists’ contribution to psychotherapy outcome. Paper
presented at the Society for Psychotherapy
Research, Toronto.
Pilkonis, P. A,, Imber, S. D., Lewis, P., & Rubinsky,
P. (1984). A comparative outcome study of
individual, group, and conjoint psychotherapy. Archives of General Psychiatry, 41,
431-437.
Piper, W. E., Debbane, E. G., Bienvenu, J. P., &
Garant, J. (1984). A comparative study of four
forms of psychotherapy.Journal of Consulting
and Clinical Psychology, 52, 268-279.
Rush, A. J., Beck, A. T., Kovacs, M., & Hollon, S. D.
(1977). Comparative efficacy of cognitive therapy and pharmacotherapy in the treatment of
depressed outpatients. Cognitive Therapy and
Research, I , 17-37.
Thompson, L. W., Gallagher,D., & Breckenridge,J.
S. (1987). Comparative effectiveness of psychotherapies for depressed elders. Journal of
Consulting and Clinical Psychology, 55, 385390.
Downloaded By: [Society for Psychotherapy Research (SPR)] At: 13:57 6 December 2007
META-ANALYSIS OF THERAPIST EFFECTS
Woody, G., McLellan, A. T., Luborsky, L., &
O’Brien, C. (1985). Sociopathy and psychotherapy outcome. Archives of General Psychiatv, 42, 1081-1086.
91
Woody, G., McLellan, A. T., Luborsky, L., &
O’Brien, C. ( 1990).Supportive expressivepsychotherapy in the treatment of opiate dependence. Unpublished manuscript.