Download “Process Variables”

6/11/14
A Generic Model of Psychotherapy
Before discussing Process Variables,
will skip to bottom of Generic Model
and focus on Outcome
“Process Variables”
A Generic Model of Psychotherapy
Outcome Variables
•  What are you interested in with regard to the
outcome of psychotherapy?
•  What is the purpose of psychotherapy?
•  What is it that you are trying to change in your
patients?
•  How will you measure it?
•  Do you rate it?
•  Can you trust the patient to rate it?
•  Should you get an observer to rate it?
1
6/11/14
Psychotherapy Outcomes
•  What is the outcome desired in psychotherapy?
•  May vary by input variables
•  Societal context: Family, Employer
•  Patients vary in terms of desired outcome
•  Psychotherapists vary in terms of desired outcome
•  May vary by interest in process variables
•  May want a measure that reflects the effect of the alliance
•  May want a measure that reflects particular treatment
•  E.g., cognitive change
•  E.g., ability to relate to others
Psychotherapy Outcomes
•  Examples of outcome research questions
•  Does psychotherapy lead to improvement?
•  Does it ever harm?
•  How quickly does change occur?
•  Does change endure?
2
6/11/14
Psychotherapy Outcomes (cont.)
•  Measuring outcome is fundamental issue
•  Need reliable and valid measurements
•  Reliability is a necessary condition for validity
•  (Reliability does not guarantee validity, but you cannot
measure something with validity unless your measurement is
reliable.)
Psychotherapy Outcomes (cont.)
Reliability (cont.)
•  The most common concern about reliability is inter-item
reliability or internal consistency
•  Is a requirement for test-retest and inter-rater reliability
•  Means the items are measuring the same concept
1.  Are you depressed lately?
2.  Are you feeling sad a lot?
3.  Has your sleep been bad?
4.  Is your self-esteem bad?
5.  Are you drinking too much?
3
6/11/14
Psychotherapy Outcomes (cont.)
Reliability (cont.)
•  Internal consistency usually measured using “Cronbach’s
alpha”
•  Developed by Lee Cronbach
•  Want an adequate value of alpha to be
sure that the items are measuring the
same underlying construct
Psychotherapy Outcomes (cont.)
Reliability
•  A reliable measure will yield the same results (assuming
the conditions have not changed) regardless of:
•  When administered
•  Called test-retest reliability
•  For example, a depressed patient gets same score on
measure from one week to the next
4
6/11/14
Psychotherapy Outcomes (cont.)
Reliability (cont.)
•  Test-retest reliability:
•  Imagine an IQ score changing from year to year
•  Intelligence is supposedly a fixed (permanent) characteristic
of a person
•  We would question the measure’s reliability
Psychotherapy Outcomes (cont.)
Reliability (cont.)
•  Test-retest reliability:
•  Imagine if a measure of depression changed a lot even though
the patient (or group) was not getting better (or worse)
5
6/11/14
Psychotherapy Outcomes (cont.)
Reliability (cont.)
•  A reliable measure will yield the same results (assuming
the conditions have not changed) regardless of:
•  Who administered it
•  Called inter-rater reliability
•  For example, a depressed patient gets same score on
measure regardless of person who gives and takes back
(e.g., the assistant)
•  Rater may need to be trained in administration (see
“standardization”)
•  Measures differ a great deal with regard to how much
training the administrator requires
•  For example, psychologists are trained in the IQ tests
Psychotherapy Outcomes (cont.)
Reliability (cont.)
•  Getting reliable results also depends on standardization
of administration of measures (including training)
•  Given/Taken the same way each time
•  For example:
•  always administered by the assistant
•  always taken before session
•  Need to ensure standardization
•  Can measure success of standardization
6
6/11/14
Psychotherapy Outcomes (cont.)
•  Summary: Issues in choosing a measure (what, who,
when, where, how)
•  Does it have internal consistency?
•  Does the measure have test-retest reliability?
•  Is there good inter-rater reliability? (Often means, “Is
administration standardized? Was there training? [If no
training] Is it relatively simple to administer?”)
•  Who administers the measure?
•  When is it completed?
•  Where is it completed?
•  How is it completed?
Psychotherapy Outcomes (cont.)
•  Most studies use paper-pencil, self-report measures
•  What: Tend to be specific to study (e.g., depression, PTSD,
drinking behavior)
•  Who: Often completed by patients
•  When: Completed at multiple time points; usually just prior
to or just after session
•  Where: Completed at clinic or research office
•  How: Given by research assistant; done anonymously
•  Why not given by clinician?
7
6/11/14
Psychotherapy Outcomes (cont.)
Psychotherapy Outcomes (cont.)
8
6/11/14
Psychotherapy Outcomes
Questions?
Outcome Measurement in Psychotherapy
Studies
•  Researchers have distinguished “statistical significance”
and “clinical significance”
•  Jacobson, N. S., & Truax, P. (1991). Clinical significance: A
statistical approach to defining meaningful change in
psychotherapy research. Journal of Consulting and Clinical
Psychology, 59(1), 12-19.
9
6/11/14
Outcome Measurement in Psychotherapy
Studies (cont.)
•  Statistical significance has long been the “norm”
•  The difference between the mean score of the treatment
group and the mean score of the control group is larger than
would be expected to occur by chance
•  Compares groups
•  Clinical Significance
•  The practical value of the effect of an intervention to the
individual clients
•  Does it make a “real” difference?
Outcome Measurement in Psychotherapy
Studies (cont.)
•  Statistical significance question: Is change score on a
certain measure different between the two groups?
•  For example, we do a study of treatment of depression
•  Family-focused therapy:
•  32 patients
•  Pre-treatment scores M = 32; Post-treatment scores M = 18
(difference = 14)
•  Pharmacotherapy:
•  34 patients
•  Pre-treatment scores M = 35; Post-treatment scores M = 26
(difference = 9)
10
6/11/14
Outcome Measurement in Psychotherapy
Studies (cont.)
•  Clinical Significance evaluates each patient individually
•  Tries to answer question: How many patients experienced
“meaningful change” due to the treatment?
•  Defines “meaningful change” as moving from the
dysfunctional population to the functional population
Outcome Measurement in Psychotherapy
Studies (cont.)
•  Clinical significance (study of treatment of depression)
•  Family-focused therapy:
•  32 patients: 18 (56%) “Improved” but did not cross into “normal”,
12 (38%) improved and moved to “normal” (“Recovered”)
•  Pharmacotherapy:
•  34 patients: 10 (29%) moved to “normal”, 15 improved (44%)
11
6/11/14
Clinical vs Statistical Significance
Questions?
Seminal Studies in Psychotherapy Outcome
•  Meta analytic studies
•  Dose-effect model
•  Phases of change model
12
6/11/14
Meta Analyses
•  Meta-analysis involves the use of statistical techniques
to combine results from multiple studies to establish
general trends and consistencies in results across
studies
•  Meta-analyses are
extremely important in
the provision of EBPP
Patient
Clinician
•  Are a summary of the
“best available evidence”
Evidence
Meta Analyses: Does Psychotherapy Work?
•  Eysenck (1952)
•  Concluded psychotherapy does not work
•  “Spontaneous improvement” of untreated persons not
significantly different than improvement due to therapy
•  Launched a great deal of psychotherapy research
•  Smith & Glass (1977)
•  Meta-analysis of 475 studies
•  Average effect size of 0.85
•  Average treated person better than 80% of untreated
13
6/11/14
Meta Analyses: Does Psychotherapy Work?
•  Lambert (2004) Bergin & Garfield’s Handbook of
Psychotherapy & Behavior Change. Conducted a meta analysis
of meta analyses.
Does Psychotherapy Work?
•  Psychotherapy is, without question, the most
frequently, intensively and widely studied health care
in the history of mankind.
•  There is a plethora of evidence that it does work.
•  Quickly
•  Better than medications for most problems (except
psychosis, bipolar disorder, and severe depression)
•  Change endures
14
6/11/14
Meta Analyses Overall
Questions?
Dose-Effect Model of Psychotherapy
Outcome
•  Howard, Kopta, Krause & Orlinsky. (1986). American
Psychologist.
•  Conducted meta analysis of multiple effectiveness studies
•  Dependent variable = percentage of patients “improved”
•  Independent variable = # sessions
15
6/11/14
Dose-Effect Curve
By session 8, half of patients had improved
Phase Model of Psychotherapy Outcome
•  Howard et al. (1993). Journal of Consulting and Clinical
Psychology.
•  Tracked three measures over time
•  Subjective Distress
•  Symptoms
•  (e.g., anxiety, depression, drinking, delusions)
•  Life Functioning
•  (e.g., self-care, relationships, occupation)
16
6/11/14
Phase Model of Psychotherapy Outcome (cont.)
Phase Model of Psychotherapy Outcome (cont.)
•  In first phase, “Subjective Distress” decreases
•  Patients grow more hopeful
•  Experience immediate relief
•  In next phase, Symptoms improve
•  Patients become less depressed, anxious, etc.
•  Experience immediate relief
•  In last phase, Life Functioning improves
•  Patients able to do better in life roles, as romantic partner, as
employee/student, as parent, etc.
Patients expect better, feel better, do better
17
6/11/14
Dose-Effect and Phase Models
Questions?
Outline of Presentation
•  Define psychotherapy
•  Inputs, Processes, Outcomes
•  Contextualize psychotherapy research within Evidence-Based
Medicine (EBM)
•  Review basics of scientific inquiry
•  Reliability, Validity, Measurement, Control
•  Review primary methods of studying psychotherapy
•  Naturalistic studies
•  RCTs (experiments)
•  (Single case studies)
•  How to apply in practice (“How to practice EBPP”)
18
6/11/14
A Generic Model of Psychotherapy
Process variables are among the most commonly
researched
•  The contract and the interventions can be
controlled (efficacy)
•  The alliance tends to need to be measured
(effectiveness)
“Process Variables”
}
A Generic Model of Psychotherapy
•  Starting with the therapeutic alliance
19
6/11/14
The Therapeutic Alliance
In all treatment models, a good therapeutic alliance is
considered a necessary condition for patients to engage
in the difficult work of therapy
•  The “alliance” concept in research = the participants are
“confident in and committed to the process”
• 
The Therapeutic Alliance in
Psychotherapy Research
•  Therapeutic Alliance has three parts (Bordin)
•  Emotional bond
•  Empathic understanding
•  Sense of genuineness and commitment
•  Unconditional warmth and acceptance
•  Agreement on goals
•  What do they want to achieve?
•  Agreement on tasks
•  How will it be achieved?
20
6/11/14
The Therapeutic Alliance in
Psychotherapy Research (cont.)
•  Most common measure developed by Horvath and
colleagues
•  The “Working Alliance Inventory”
21
6/11/14
Process Variables - - Alliance
•  Zuroff & Blatt (2006). Journal of Consulting and
Clinical Psychology
•  Research Question (Societal Context): Does therapeutic
• 
• 
• 
• 
alliance predict outcome?
Patients: 250 persons with severe depression; 11 never
started treatment; 191 finished treatment
Therapists: Multiple sites; multiple therapists
Contract: 16 sessions planned
Treatments: Four treatments compared (IPT, CBT, drug,
placebo)
Process Variables - - Alliance (cont.)
•  Design: Randomized clinical trial (RCT)
•  Process measure (Therapeutic Alliance)
•  The Barrett-Lennard Relationship Inventory (1962)
•  Patient’s perception of therapist accurate empathy,
unconditional positive regard, and genuineness
•  Not manipulated; only measured
•  Outcome
•  Measured (a) depression (Beck Depression Inventory) and
Social Adjustment Scale
•  Results showed that a more positive therapeutic
relationship early in treatment predicted more rapid
improvement
22
6/11/14
Therapeutic Alliance
•  Often studied within studies of an intervention
•  Alliance difficult to manipulate
•  Would potentially be unethical
•  Many, many, many studies of the alliance
•  More associated with outcome than diagnosis
•  More associated with outcome than technique
•  Poor alliance predicts client dropout
•  Lambert meta-analysis (2001)
Therapeutic Alliance
•  Process factor
•  Often studied within studies of an intervention
•  That is, studied “naturalistically” within an RCT
•  Alliance difficult to manipulate
•  Would potentially be unethical
23
6/11/14
Therapeutic Alliance – Research Summary
•  Many, many, many studies of the alliance
•  More associated with outcome than diagnosis
•  More associated with outcome than technique
•  Poor alliance predicts client dropout
•  Lambert meta-analysis (2001)
Therapeutic Alliance – Research Summary
•  Lambert meta-analysis (2001)
24
6/11/14
Therapeutic Alliance – A Common Factor
•  Effectiveness of all treatments due in part to factors
common to all treatments
•  Hope
•  Expectancy
•  Therapeutic alliance
A Generic Model of Psychotherapy
25
6/11/14
Psychotherapy Contract
•  The agreement between the patient and therapist
regarding the practical and legal aspects of the provision
of psychotherapy services
•  Who, what, when, how often, why, how, how paid
•  Is there a professional relationship at all? When is it
established?
•  What does the contract obligate the provider to “do”?
•  Who is involved? Individual/Couple/Family
•  Who gets to know? (Parents, Legal system, Employer)
•  When does therapy happen? How often?
•  How to cancel?
•  Who pays?
The Psychotherapy “Contract” in
Treatment Research
•  In psychotherapy research, the “contract” comprises the
specifics of the therapy offered
•  Model of treatment (CBT, IPT, psychodynamic)
•  How therapy is monitored
•  How long therapy will endure
26
6/11/14
The Psychotherapy “Contract” in
Treatment Research (cont.)
•  In efficacy studies (R.C.T.’s), experimental control is
paramount concern
•  Therefore, the contract is highly specified
•  Type of treatment, duration, frequency
•  In effectiveness studies, contract is allowed to occur
naturally
•  In either type of study, important to measure what
actually happened
•  For example, McRae et al. (insomnia study) was planned for
6 sessions, but it reported on number of patients that
finished at least 3
A Generic Model of Psychotherapy
27
6/11/14
Interventions
•  Psychoanalysis
•  Psychodynamic therapy
•  Cognitive therapy
•  Behavioral therapy
•  Cognitive-behavior therapy
•  Acceptance and Commitment Therapy
•  Interpersonal therapy (IPT)
•  Emotion-focused therapy
•  Behavioral marital therapy
•  And so forth ….
Intervention Research Questions
•  Does treatment X work better than nothing?
•  Does treatment X work better than treatment Y?
•  What components of the treatment lead to positive
therapeutic change?
•  What are the “active components” of treatment?
•  Insight? Cognitive change? Acceptance?
•  Can the effects of treatment be enhanced?
•  Can the effects of treatment be made to last?
28
6/11/14
Methods for Studying Interventions
•  Naturalistic studies
•  Study interventions as conducted in real world
•  High external validity
•  Effectiveness studies
•  Experimental studies (i.e., RCTs)
•  Study interventions in controlled settings (i.e., labs)
•  High internal validity
•  Efficacy studies
Psychotherapy Research Strategies
•  Naturalistic studies
•  Non-random assignment; patients and therapists (especially)
choose what treatment to conduct
•  May be several conditions, but no random assignment
•  “Naturally-occurring” psychotherapy
However, cannot be certain what was actually done
29
6/11/14
Experimental Studies (RCTs)
•  Investigator completely controls treatment
•  Type, amount, duration
•  Who receives what (randomization)
•  Regarded as the most scientifically vigorous study design
•  Can confidently attribute effect to the treatment
RCTs – Overview
1. Inclusion and Exclusion Criteria
2. Baseline, termination and follow-up measurements
3. Random assignment
4. Treatment condition (“standardized”)
5. Control or comparison condition (also “standardized”)
6. Statistical comparison
30
6/11/14
RCT Basics
Inclusion
Criteria
Many
persons
Sample
apply
to be in
study
Exclusion
Criteria
Baseline
Assessment
(Week 1)
Post-treatment
Assessment
(Week 16)
Follow-up
Assessment
(Week 40)
1. Inclusion and Exclusion Criteria
•  Inclusion criteria = Requirements of persons to be
study participants
•  e.g., depressed, literate (able to complete questionnaires)
•  Want persons that enter study to be somewhat similar
•  Exclusion criteria = Qualities of persons that would
disqualify them from study
•  e.g., suicidal, psychotic, low IQ, unstable living situation
•  Try to exclude persons at high risk of not completing study
31
6/11/14
2. Measurements
•  Measure characteristics that might be related to
treatment outcome
•  Demographic information (gender, age, etc.)
•  Measurements of problems related to disorder
•  e.g., severity of depression
•  other measures (e.g., life functioning)
•  Done at numerous time points to compare change
•  Baseline, mid-treatment, post-treatment
•  Follow-up measurements have proven very important
3. Random Assignment
•  Randomize to conditions
•  Treatment vs. No Treatment (or Wait List)
•  Treatment A vs. Treatment B vs. Treatment C
•  Usually “Treatment of Interest” vs. “Control Condition”
100 depressed
persons that meet
study criteria
Random Assignment
Treatment of Interest
50 receive CBT
(16 weeks)
Control Condition
50 placed on “wait
list” (will get treatment
after 6 week wait)
32
6/11/14
3. Random Assignment (cont.)
•  What if we allowed persons to choose?
•  What if we assigned all males to Treatment A and all
females to Treatment B?
•  What about “severely depressed to Meds Condition and
less depressed to Psychotherapy Condition”?
3. Random Assignment (cont.)
•  Randomization maximizes likelihood that the two
groups will be comparable
•  Random assignment controls for nuisance variables (e.g., choice,
severity of illness) that might otherwise affect outcome
•  Groups should be equally depressed, equal number of males
and females, etc.
•  Randomization ensures that any observed differences at
end of study can be attributed to differences in
conditions (i.e., to the treatment)
33
6/11/14
4. Treatment Condition and
5. Control Condition
•  In an experiment, the Independent Variable (IV) needs
to be highly controlled
•  Need to be sure that CBT Condition received CBT
•  Need to be sure that Control Condition did not receive
CBT but rather the alternative
How can we do that?
4. Treatment Condition and
5. Control Condition
•  Train therapists
•  Use “treatment manuals”
•  Evaluate “fidelity” or “adherence” to treatment
34
6/11/14
RCT of Depression Treatments
•  TDCRP (Treatment of Depression Collaborative
Research Project) (Elkin et al.)
•  Research Question (Societal Context): Is psychotherapy as
effective as medications in the treatment of depression?
•  Patients: 250 persons with severe depression; 11 never
started treatment; 191 finished treatment
•  Four conditions compared
•  Interpersonal Psychotherapy (IPT)
•  Cognitive Therapy (CT)
•  Imipramine (drug of choice in 1980s)
•  Placebo (sugar pill)
}
Treatment Conditions
Control Condition
“Manualized” Treatment
•  Manuals specify treatment
•  How disorder is conceptualized
•  What to do session by session (or phase by phase)
•  What topics to cover
•  What homework to give
•  Cognitive Therapy (CT) condition
•  One of original treatment manuals
•  Developed for this study (as was IPT
manual)
•  Therapists were trained to do manual
35
6/11/14
Drug and Control Condition
•  The drug and placebo conditions were also
manualized
•  Physicians were trained to follow manual
•  “Don’t talk about cognitions”
•  Ask how patient is feeling
•  Monitor worsening of symptoms
•  Monitor for suicide risk
6. Statistical Analyses
•  Classical analytic strategy = compare groups
36
6/11/14
6. Statistical Analyses (cont.)
•  Current strategy is to examine individual responses
Experiments vs. Naturalistic Studies of
Psychotherapy
•  Experiments have high internal validity
•  Naturalistic studies have high external validity
37
6/11/14
Internal Validity
Treatment via
Improvement of
manual (IV) Internal validity patient (DV)
External Validity
Treatment in
community (IV)
External validity
Improvement of
patient (DV)
Experiments vs. Naturalistic Studies of
Psychotherapy (cont.)
•  Therapy manuals standardize IV and increase internal
validity, but probably reduce external validity
•  Therapy as actually practiced less standardized
•  Manuals to control differences between therapists
(make them all “standardized”)
•  But differences between therapists major determinant of
outcome
•  Capacity for empathy
•  Expertise
38
6/11/14
Experiments vs. Naturalistic Studies of
Psychotherapy (cont.)
•  RCTs considered by some to be “best” or “gold
standard”
•  However, naturalistic studies are vital
Experiments vs. Naturalistic Studies of
Psychotherapy (cont.)
•  Properties of psychotherapy as done in the community
differ than RCTs
•  Patients may actively seek out certain techniques or
practitioners (vs. random assignment)
•  Psychotherapy not of fixed duration
•  Continues until patient is improved or until patient
terminates (sometimes against advice)
•  Psychotherapy is self-correcting
•  Therapist might add another technique, switch entirely to
another technique, suggest some time off
•  Patient might be referred to another therapist
39
6/11/14
Experiments vs. Naturalistic Studies of
Psychotherapy (cont.)
•  Patients usually have multiple problems (high
“comorbidity”)
•  Psychotherapy will focus on numerous difficulties/
problems/diagnoses
•  Psychotherapy attempts to provide general improvement
(efficacy studies usually focused on a single disorder,
meaning symptoms of a single disorder)
Summary of Treatment-Based Research
40
6/11/14
Meta Analyses of Different Treatments
•  Rosenzweig S. (1936) American Journal of Orthopsychiatry
•  Quoted “Alice in Wonderland”
•  As in the results of the race, comparing treatments
reveals that, “At last the Dodo said, ‘Everybody has won
and all must have prizes.’”
•  Has been found repeatedly
•  e.g., Wampold et al. (1997). Psychological Bulletin.
A meta-analysis of outcome studies comparing bona
fide psychotherapies: Empirically, “All must have
prizes.”
Meta Analyses of Different Treatments (cont.)
•  Different treatments account for a small amount of the
variance in psychotherapy outcomes
•  Virtually all psychotherapy models and techniques are
effective with some people, some of the time
•  Outcome differences between therapists using the same
model have been found to be 2-3 times greater than the
differences between models
41
6/11/14
What Diagnoses Have Been Shown to Be
Treatable?
•  So far, all of them
•  Depression, Bipolar Disorder
•  Panic disorder, OCD, Phobias
•  Bulimia Nervosa, Anorexia Nervosa
•  PTSD
•  Schizophrenia
•  Headache, Pain
•  Oppositional Children, ADHD
•  Marital distress, Sexual Dysfunction
•  Etc. ………
“Patient-Focused Research”
•  Different research questions about treatments
(a) Does the treatment work under experimental
conditions? (Treatment efficacy studies)
(b) Does the treatment work in practice? (Treatment
effectiveness studies)
•  Question of ultimate interest to this patient and this
therapist:
(c) Is the treatment working for a particular patient?
•  Referred to as “patient focused research”
42
6/11/14
Patient Profiling
Howard, K. I., Moras, K., Brill, P., Martinovich, Z., &
Lutz, W. (1996). The evaluation of psychotherapy.
American Psychologist, 52, 1059-1064.
Patient Profiling
43
6/11/14
Patient Profiling
Outline of Presentation
•  Define psychotherapy
•  Inputs, Processes, Outcomes
•  Contextualize psychotherapy research within Evidence-Based
Medicine (EBM)
•  Review basics of scientific inquiry
•  Reliability, Validity, Measurement, Control
•  Review primary methods of studying psychotherapy
•  Naturalistic studies
•  RCTs (experiments)
•  (Single case studies)
•  How to apply in practice (“How to practice EBPP”)
44
6/11/14
Single Case Psychotherapy Research
•  Single case designs can be either experimental or
naturalistic
Single Subject Experimental Design
•  Can be used to evaluate any type of psychotherapy
•  Frequently used to evaluate new therapy
•  Method
•  Repeated assessment of behavior of primary interest
•  Establishment of a baseline; Monitor progress
•  Treatment clearly specified; Treatment implemented
then withdrawn, then implemented
•  Called “ABAB design”
•  A = No treatment (baseline/mid-treatment )
•  B = Treatment of interest
45
6/11/14
Single Subject Experimental Design (cont.)
•  Changes in dependent variable (outcome) after
introduction of Treatment
•  Return to baseline after removal of treatment
Using Psychotherapy Research Methods
in Your Own Practice (Single Subject
Naturalistic Design)
•  Research supports outcome monitoring
•  Routine measurement (patient self report outcome
questionnaires) to monitor response
•  Early identification of poor response
•  Given large body of evidence related to alliance
•  Can also routinely monitor
•  Identify and “repair” problems
46
6/11/14
Thank you!
•  Questions?
47