Download Propensity Score Matching

Propensity Score Matching
Lava Timsina
Kristina Rabarison
CPH 786-001 Doctoral Seminar
Fall 2012
Introduction
• Program evaluation
• Counterfactual outcome
– what would have happened to the participants in
absence of treatment
• Statistical techniques
• Propensity score
Concept of PSM
• Identify neighborhoods that are as similar as
possible to each other with respect to the
probability of receiving the treatment
(Chiavegatto Filho, Kawachi, & Gotlieb, 2012).
• The average treatment effects is then
measured based on the mean difference in
outcomes across these comparison and
treatment groups.
Experimental Vs Non-experimental
• Experimental evaluation
– Random assignment to treatment and control 
control group can be regarded as counterfactual.
Motivation to Propensity Score
Matching
• Non-experimental evaluation
– Random assignment may not be possible in
nonexperimental evaluation methods (Heinrich et al.,
2010).
– Assignment to treatment is often nonrandom and
hence may bias participation and treatment
outcomes.
– Treatment units are matched with their “similar”
counterparts that differ only in the treatment under
study.
– Extent of matching is challenging
• Propensity score matching allows this
matching problem to be reduced to a single
dimension.
• Let me restate that propensity score is defined
as the probability that a unit in the combined
sample of treated and control units receive
the treatment, given a set of observed
covariates.
Assumptions for PSM
• PSM holds under two assumptions (Khandker,
2010; Rosenbaum & Rubin, 1983):
– Conditional Independence or Unconfounded
Assumption
– Common Support or Overlap Condition
• Conditional Independence or Unconfounded
Assumption:
– conditional on observable covariates, the
outcomes are independent of treatment
• In absence of randomization, the groups may differ not
only in the treatment status, but also in other
covariates. Thus it is necessary to control for these
covariates to avoid potential biases. There is a set of
covariates observable to the researcher, such that after
controlling for these covariates, the potential outcomes
(r1,r0) are independent of the treatment status:
(r1,r0) ⊥ Z|X
• Common Support Condition Assumption
– This condition ensures that treatment
observations have comparison observations
“nearby” in the propensity score distribution. For
each value of X, there is positive probability of
being both treated and untreated:
0<P(Z=1|X)<1
– Also called overlap condition
Steps of PSM (European Commission,
2009; Khandker, 2010):
1. Estimating a model of program participation
2. Defining the region of common support and
balancing tests
3. Matching participants to nonparticipants
4. Estimating the Average Effect and its
Standard Error
Steps of PSM (European Commission,
2009; Khandker, 2010):
1. Estimating a model of program participation
i.
Samples of participants and nonparticipants
should be pooled,
ii. Participation Z should be estimated on all the
observed covariates X in the data that are likely
to determine participation.
 Probit or logit model of program participation
 This predicted outcome represents the estimated
probability of participation or propensity score for
every sampled participants and non-participants.
Steps of PSM (European Commission,
2009; Khandker, 2010):
2. Defining the region of common support and
balancing tests
– The region of common support needs to be defined where
distributions of the propensity score for treatment and
comparison group overlap.
Steps of PSM (European Commission,
2009; Khandker, 2010):
• Some of the participant and nonparticipant
observation falling outside the region of
common support may have to be dropped.
Steps of PSM (European Commission,
2009; Khandker, 2010):
• Balancing tests can be conducted to check
whether, within each quantile of the
propensity score distribution, the treatment
and comparison groups have similar average
propensity scores and the mean of X. That is,
the distributions of the treated group and the
control group must be similar:
p̂(X|Z=1) = p̂(X|Z=0)
Steps of PSM (European Commission,
2009; Khandker, 2010):
3. Matching participants to nonparticipants
– Matching participants to nonparticipants on the
basis of propensity score can be done using
different matching technique.
Techniques of matching
1. Nearest-neighbor (NN) matching
– Each treatment unit is matched to the comparison
unit with the closest propensity score. Matching can
be done with or without replacement.
– In this method, both treatment and control groups are
first randomly sorted. Then the first treatment unit is
selected to find its closest control match based on the
absolute value of the difference between the
propensity score (or logit of the propensity score) of
the selected treatment and that of the control under
consideration. The closest control unit is selected as a
match.
Techniques of matching
2. Caliper or radius matching
– This method is similar to NN matching except it
adds restriction. Both treatment and control units
are randomly sorted and then the first treated unit
is selected to find its closest control match in terms
of the propensity score but only if the control’s
propensity score is within a certain radius (caliper).
– This avoids bad matching and ensures that the
matched pairs are within a certain range of
propensity scores.
Techniques of matching
3. Stratification or interval matching
– Partitions the common support into different strata
(or intervals) and calculates the program’s impact
within each interval.
– The treated and control groups are ranked on the
basis of their propensity scores, and then grouped
into K intervals (strata).
– Then the impact for each kth stratum is evaluated.
– The overall impact is the weighted average of the
strata effects, with weights proportional to the
number of treated units in each stratum.
Techniques of matching
4. Kernel and local linear matching
– Kernel matching uses weighted averages of all
individuals in the control group to construct the
counterfactual outcome.
– Weights depend on the distance between each
individual from the control group and the participant
observation for which the counterfactual is estimated.
– The kernel function assigns higher weight to
observations close in terms of propensity score to a
treated individual and lower weight on more distant
observations.
Steps of PSM (European Commission,
2009; Khandker, 2010):
4. Estimating the Average Effect and its Standard
Error
– compute the sample averages of the two groups and
calculate the difference
– Standard errors can be computed using bootstrapping
methods.
Example
• Research question: Does homelessness affect
physical health, as measured by pcs score
from the SF-36?
A glimpse at the dataset
proc print data=ref.help
(obs=10);
var id pcs mcs age female i1
homeless;
run;
Obs ID
PCS MCS AGE FEMALE I1 HOMELESS
37
0 13
0
1 1 58.4137 25.1120
37
0 56
1
2 2 36.0369 26.6703
3
74.8063
6.7629
26
0
0
0
3
39
1 5
0
4 4 61.9317 43.9679
32
0 10
1
5 5 37.3456 21.6758
47
1 4
0
6 6 46.4752 55.5090
49
1 13
0
7 7 24.5150 21.7930
8
65.1380
9.1605
28
0
12
1
8
50
1 71
1
9 9 38.2709 22.0297
39
0 20
1
10 10 22.6106 36.1438
Modeling as linear regression
proc reg data=ref.help;
model pcs=homeless;
run;
Number of Observations Read 453
Number of Observations Used 453
Variable
Intercept
HOMELESS
Parameter Estimates
Parameter Standard
Error t Value Pr > |t|
DF Estimate
1
49.00083
0.68802
71.22
<.0001
1
-2.06405
1.01292
-2.04
0.0422
proc reg data=ref.help;
model pcs = homeless age female i1
mcs;
run;
quit;
Variable
Intercept
HOMELESS
AGE
FEMALE
I1
MCS
Parameter Estimates
Parameter Standard
Error t Value Pr > |t|
DF Estimate
1
58.21224
2.56675
22.68
<.0001
1
-1.14707
0.99794
-1.15
0.2510
1
-0.26593
0.06410
-4.15
<.0001
1
-3.95519
1.15142
-3.44
0.0006
1
-0.08079
0.02538
-3.18
0.0016
1
0.07032
0.03807
1.85
0.0654
Creating propensity scores
proc logistic data=ref.help desc;
model homeless = age female i1 mcs;
output out=propen pred=propensity;
run;
Association of Predicted Probabilities and
Observed Responses
64.9 Somers' D 0.302
Percent Concordant
34.7 Gamma
0.304
Percent Discordant
0.4 Tau-a
0.151
Percent Tied
50996 c
0.651
Pairs
Looking for the assumption of
common support
proc means data=propen;
class homeless;
var propensity;
run;
Analysis Variable : propensity Estimated Probability
N
HOMELESS Obs N
Mean Std Dev Minimum Maximum
0
1
244 244 0.4296704 0.1166290
209 209 0.4983750 0.1382013
0.2136791
0.2635031
0.7876000
0.9642827
Looking for the assumption of
common support
proc univariate data=propen;
class homeless;
var propensity;
histogram propensity;
run;
Restricting analysis to common
support region
proc reg data=propen;
where propensity<0.8;
model pcs=homeless propensity;
run;
quit;
Parameter Estimates
Parameter Standard
Error t Value Pr > |t|
DF Estimate
Variable
Label
Intercept
Intercept
HOMELESS
Estimated Probability
propensity
1
54.19945
1.98265
27.34
<.0001
1
-1.19613
1.03893
-1.15
0.2502
1
-12.09909
4.33385
-2.79
0.0055
Macros
data prop2;
set propen;
if propensity<0.8;
run;
%include "C:\Documents and Settings\LRTI222\Desktop\vmatch.sas";
%include "C:\Documents and Settings\LRTI222\Desktop\dist.sas";
%include "C:\Documents and Settings\LRTI222\Desktop\nobs.sas";
%dist(data=prop2, group=homeless, id=id,
mvars=propensity,
wts=1, vmatch=Y, a=1, b=1, lilm=201, dmax=0.1,
outm=mp1_b, summatch=n, printm=N,
mergeout=mpropen);
Before & After Matching
proc means data=propen mean;
class homeless;
var age female i1 mcs;
run;
N
HOMELESS Obs Variable
0
1
proc means data=mpropen mean;
where matched;
class homeless;
var age female i1 mcs;
run;
244 AGE
FEMALE
I1
MCS
209 AGE
FEMALE
I1
MCS
N
HOMELESS Obs Variable
0
1
201 AGE
FEMALE
I1
MCS
201 AGE
FEMALE
I1
MCS
Mean
35.0409836
0.2745902
13.5122951
32.4868303
36.3684211
0.1913876
23.0382775
30.7308549
Mean
35.6218905
0.1791045
15.9154229
31.4815123
36.1492537
0.1990050
19.9452736
30.9176772
Propensity score matched analysis
proc reg data=mpropen;
where matched;
model pcs=homeless;
run;
quit;
Variable
Intercept
HOMELESS
Parameter Estimates
Parameter Standard
Error t Value Pr > |t|
DF Estimate
1
48.95273
0.76200
64.24
<.0001
1
-1.79386
1.07763
-1.66
0.0968