Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Propensity Score Matching Lava Timsina Kristina Rabarison CPH 786-001 Doctoral Seminar Fall 2012 Introduction • Program evaluation • Counterfactual outcome – what would have happened to the participants in absence of treatment • Statistical techniques • Propensity score Concept of PSM • Identify neighborhoods that are as similar as possible to each other with respect to the probability of receiving the treatment (Chiavegatto Filho, Kawachi, & Gotlieb, 2012). • The average treatment effects is then measured based on the mean difference in outcomes across these comparison and treatment groups. Experimental Vs Non-experimental • Experimental evaluation – Random assignment to treatment and control control group can be regarded as counterfactual. Motivation to Propensity Score Matching • Non-experimental evaluation – Random assignment may not be possible in nonexperimental evaluation methods (Heinrich et al., 2010). – Assignment to treatment is often nonrandom and hence may bias participation and treatment outcomes. – Treatment units are matched with their “similar” counterparts that differ only in the treatment under study. – Extent of matching is challenging • Propensity score matching allows this matching problem to be reduced to a single dimension. • Let me restate that propensity score is defined as the probability that a unit in the combined sample of treated and control units receive the treatment, given a set of observed covariates. Assumptions for PSM • PSM holds under two assumptions (Khandker, 2010; Rosenbaum & Rubin, 1983): – Conditional Independence or Unconfounded Assumption – Common Support or Overlap Condition • Conditional Independence or Unconfounded Assumption: – conditional on observable covariates, the outcomes are independent of treatment • In absence of randomization, the groups may differ not only in the treatment status, but also in other covariates. Thus it is necessary to control for these covariates to avoid potential biases. There is a set of covariates observable to the researcher, such that after controlling for these covariates, the potential outcomes (r1,r0) are independent of the treatment status: (r1,r0) ⊥ Z|X • Common Support Condition Assumption – This condition ensures that treatment observations have comparison observations “nearby” in the propensity score distribution. For each value of X, there is positive probability of being both treated and untreated: 0<P(Z=1|X)<1 – Also called overlap condition Steps of PSM (European Commission, 2009; Khandker, 2010): 1. Estimating a model of program participation 2. Defining the region of common support and balancing tests 3. Matching participants to nonparticipants 4. Estimating the Average Effect and its Standard Error Steps of PSM (European Commission, 2009; Khandker, 2010): 1. Estimating a model of program participation i. Samples of participants and nonparticipants should be pooled, ii. Participation Z should be estimated on all the observed covariates X in the data that are likely to determine participation. Probit or logit model of program participation This predicted outcome represents the estimated probability of participation or propensity score for every sampled participants and non-participants. Steps of PSM (European Commission, 2009; Khandker, 2010): 2. Defining the region of common support and balancing tests – The region of common support needs to be defined where distributions of the propensity score for treatment and comparison group overlap. Steps of PSM (European Commission, 2009; Khandker, 2010): • Some of the participant and nonparticipant observation falling outside the region of common support may have to be dropped. Steps of PSM (European Commission, 2009; Khandker, 2010): • Balancing tests can be conducted to check whether, within each quantile of the propensity score distribution, the treatment and comparison groups have similar average propensity scores and the mean of X. That is, the distributions of the treated group and the control group must be similar: p̂(X|Z=1) = p̂(X|Z=0) Steps of PSM (European Commission, 2009; Khandker, 2010): 3. Matching participants to nonparticipants – Matching participants to nonparticipants on the basis of propensity score can be done using different matching technique. Techniques of matching 1. Nearest-neighbor (NN) matching – Each treatment unit is matched to the comparison unit with the closest propensity score. Matching can be done with or without replacement. – In this method, both treatment and control groups are first randomly sorted. Then the first treatment unit is selected to find its closest control match based on the absolute value of the difference between the propensity score (or logit of the propensity score) of the selected treatment and that of the control under consideration. The closest control unit is selected as a match. Techniques of matching 2. Caliper or radius matching – This method is similar to NN matching except it adds restriction. Both treatment and control units are randomly sorted and then the first treated unit is selected to find its closest control match in terms of the propensity score but only if the control’s propensity score is within a certain radius (caliper). – This avoids bad matching and ensures that the matched pairs are within a certain range of propensity scores. Techniques of matching 3. Stratification or interval matching – Partitions the common support into different strata (or intervals) and calculates the program’s impact within each interval. – The treated and control groups are ranked on the basis of their propensity scores, and then grouped into K intervals (strata). – Then the impact for each kth stratum is evaluated. – The overall impact is the weighted average of the strata effects, with weights proportional to the number of treated units in each stratum. Techniques of matching 4. Kernel and local linear matching – Kernel matching uses weighted averages of all individuals in the control group to construct the counterfactual outcome. – Weights depend on the distance between each individual from the control group and the participant observation for which the counterfactual is estimated. – The kernel function assigns higher weight to observations close in terms of propensity score to a treated individual and lower weight on more distant observations. Steps of PSM (European Commission, 2009; Khandker, 2010): 4. Estimating the Average Effect and its Standard Error – compute the sample averages of the two groups and calculate the difference – Standard errors can be computed using bootstrapping methods. Example • Research question: Does homelessness affect physical health, as measured by pcs score from the SF-36? A glimpse at the dataset proc print data=ref.help (obs=10); var id pcs mcs age female i1 homeless; run; Obs ID PCS MCS AGE FEMALE I1 HOMELESS 37 0 13 0 1 1 58.4137 25.1120 37 0 56 1 2 2 36.0369 26.6703 3 74.8063 6.7629 26 0 0 0 3 39 1 5 0 4 4 61.9317 43.9679 32 0 10 1 5 5 37.3456 21.6758 47 1 4 0 6 6 46.4752 55.5090 49 1 13 0 7 7 24.5150 21.7930 8 65.1380 9.1605 28 0 12 1 8 50 1 71 1 9 9 38.2709 22.0297 39 0 20 1 10 10 22.6106 36.1438 Modeling as linear regression proc reg data=ref.help; model pcs=homeless; run; Number of Observations Read 453 Number of Observations Used 453 Variable Intercept HOMELESS Parameter Estimates Parameter Standard Error t Value Pr > |t| DF Estimate 1 49.00083 0.68802 71.22 <.0001 1 -2.06405 1.01292 -2.04 0.0422 proc reg data=ref.help; model pcs = homeless age female i1 mcs; run; quit; Variable Intercept HOMELESS AGE FEMALE I1 MCS Parameter Estimates Parameter Standard Error t Value Pr > |t| DF Estimate 1 58.21224 2.56675 22.68 <.0001 1 -1.14707 0.99794 -1.15 0.2510 1 -0.26593 0.06410 -4.15 <.0001 1 -3.95519 1.15142 -3.44 0.0006 1 -0.08079 0.02538 -3.18 0.0016 1 0.07032 0.03807 1.85 0.0654 Creating propensity scores proc logistic data=ref.help desc; model homeless = age female i1 mcs; output out=propen pred=propensity; run; Association of Predicted Probabilities and Observed Responses 64.9 Somers' D 0.302 Percent Concordant 34.7 Gamma 0.304 Percent Discordant 0.4 Tau-a 0.151 Percent Tied 50996 c 0.651 Pairs Looking for the assumption of common support proc means data=propen; class homeless; var propensity; run; Analysis Variable : propensity Estimated Probability N HOMELESS Obs N Mean Std Dev Minimum Maximum 0 1 244 244 0.4296704 0.1166290 209 209 0.4983750 0.1382013 0.2136791 0.2635031 0.7876000 0.9642827 Looking for the assumption of common support proc univariate data=propen; class homeless; var propensity; histogram propensity; run; Restricting analysis to common support region proc reg data=propen; where propensity<0.8; model pcs=homeless propensity; run; quit; Parameter Estimates Parameter Standard Error t Value Pr > |t| DF Estimate Variable Label Intercept Intercept HOMELESS Estimated Probability propensity 1 54.19945 1.98265 27.34 <.0001 1 -1.19613 1.03893 -1.15 0.2502 1 -12.09909 4.33385 -2.79 0.0055 Macros data prop2; set propen; if propensity<0.8; run; %include "C:\Documents and Settings\LRTI222\Desktop\vmatch.sas"; %include "C:\Documents and Settings\LRTI222\Desktop\dist.sas"; %include "C:\Documents and Settings\LRTI222\Desktop\nobs.sas"; %dist(data=prop2, group=homeless, id=id, mvars=propensity, wts=1, vmatch=Y, a=1, b=1, lilm=201, dmax=0.1, outm=mp1_b, summatch=n, printm=N, mergeout=mpropen); Before & After Matching proc means data=propen mean; class homeless; var age female i1 mcs; run; N HOMELESS Obs Variable 0 1 proc means data=mpropen mean; where matched; class homeless; var age female i1 mcs; run; 244 AGE FEMALE I1 MCS 209 AGE FEMALE I1 MCS N HOMELESS Obs Variable 0 1 201 AGE FEMALE I1 MCS 201 AGE FEMALE I1 MCS Mean 35.0409836 0.2745902 13.5122951 32.4868303 36.3684211 0.1913876 23.0382775 30.7308549 Mean 35.6218905 0.1791045 15.9154229 31.4815123 36.1492537 0.1990050 19.9452736 30.9176772 Propensity score matched analysis proc reg data=mpropen; where matched; model pcs=homeless; run; quit; Variable Intercept HOMELESS Parameter Estimates Parameter Standard Error t Value Pr > |t| DF Estimate 1 48.95273 0.76200 64.24 <.0001 1 -1.79386 1.07763 -1.66 0.0968