Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
National Institute for Health and Care Excellence wikipedia , lookup
Drug design wikipedia , lookup
Pharmaceutical industry wikipedia , lookup
Prescription costs wikipedia , lookup
Pharmacokinetics wikipedia , lookup
Polysubstance dependence wikipedia , lookup
Adherence (medicine) wikipedia , lookup
Using Propensity Scores To Adjust For Treatment Selection Bias R. Scott Leslie, MedImpact Healthcare Systems, Inc., San Diego, CA ABSTRACT A key strength of observational studies is the ability to estimate treatment effect on health outcomes in “real world” conditions. However, studies that lack randomization of subjects into treatment groups must address selection bias to properly estimate the effect of treatment as non randomized groups usually differ on observed and unobserved characteristics. That is, the observed treatment effect may be due to the treatment itself or due to the differential selection into treatment groups from non-randomization. Conventional regression adjustment, matching, and stratification using propensity scores are widely used techniques to adjust for selection bias. Included in this tutorial is a description of these propensity score methods, an explanation of the advantages and disadvantages of each method, and an application of the methods using various examples. This tutorial is intended for intermediate level statisticians, SAS® programmers, and/or data analysts. INTRODUCTION AND WHY USE PROPENSITY SCORES Randomized control trials (RCTs) measure drug efficacy in controlled environments: however, can often be restricted to subpopulations that limit generalizability of results. Observational studies, on the other hand, can evaluate treatment effectiveness in routine care settings or everyday use patterns. However, a limitation of observational studies is the lack of treatment assignment. Non randomized groups usually differ in observed and unobserved characteristics causing selection bias when evaluating the effect of treatment. For example, the below figure shows how 2 patient groups may differ in age, gender, health status, and previous medication use. Patients eligible for Drug A or Drug B Drug A Patient Characteristics Drug B Patient Characteristics Younger, More females, Currently taking less medications, Poorer medication use behavior Older, More male, Currently taking more medications, Better medication use behavior Statistical techniques such as matching, stratification, and regression adjustment are commonly used to account for differences in treatment groups but may be limited if using too few covariates in the adjustment process. The use of propensity score techniques avoids this limitation because it can summarize more or all the covariate information into a single score. So what is a propensity score? The propensity score is the conditional probability of being treated based on individual covariates. Rosenbaum and Rubin demonstrated that propensity scores can account for imbalances in treatment groups and reduce bias by resembling randomization of subjects into treatment groups. By using propensity scores to balance groups, traditional adjustment methods can better estimate treatment effect on outcomes while adjusting for covariates. One method professed by Ralph B. D’Agostino, Jr. to adjust for the non randomized treatment selection is to use a propensity score method in conjunction with traditional regression techniques. This process is performed using two steps, the first of which calculates propensity scores as the probability of patients being included in each treatment group based on pre-treatment observables. The aim of this step is to create balance treatment groups and simulate random treatment allocation. The second step utilizes the created propensity scores with ANCOVA to more accurately estimate outcomes and study the possible covariate predictors. HOW TO CREATE PROPENSITY SCORES The logistic model describes the relationship of several independent variables to a dichotomous dependent variable. Furthermore, logistic regression is used to predict the probability of event occurring as a function of independent variables (continuous and/or dichotomous) The logistic model: P( X ) = 1 1 + e −(α + ΣβiXi1) Propensity scores are easily created using PROC LOGISTIC. In the cases described in this paper, the dependent variable is treatment group (treatment for the patient) and the independent variables are patient and baseline characteristics. In other cases the dependent variable may be any dichotomous outcome (treated or untreated, disenrolled or not, visited specialist or not). The GENMOD procedure for generalized linear models may also create propensity scores by using the OUTPUT statement and keyword PREDICTED. Example of Creating Propensity Scores Using PROC LOGISTIC The following example illustrates the use of PROC LOGISTIC to create propensity scores. A large pharmacy claims database was used to identify 19,433 patients using oral antidiabetic therapy. Patients were categorized into two drug treatment groups, A and B, with the main objective of comparing compliance and adherence rates. Compliance was measured as the proportion of days a medication was supplied over a 180 day period. Adherent patients were identified as those reaching a threshold of 80% compliance. Selection bias is believed to be a factor as the two drug treatment groups differ in patient tolerance, adverse events, and side effects which possibly influence drug choice and compliance to each drug. Other variables controlled for in the analyses include demographic variables (age, gender) and previous medication use/patterns measured in a 6 month baseline period prior to treatment. Previous medication use was recorded by the use of specific cardiovascular, asthma, and antidepressant medications and previous medication pattern use was measured by refill patterns of maintenance type medications. PROC LOGISTIC calculates propensity scores as the conditional probability of each patient receiving a particular treatment based on pre-treatment variables and can output the propensity score to a data set. In this example, propensity scores were calculated based on the covariates listed in Table 1 below. The objective was to balance the treatment groups so to reduce bias of treatment selection and obtain better idea of treatment effect on the outcome of compliance. The logit function is specified in the LINK option to fit the binary logit model and the RSQUARE option assesses the amount of variation explained by the independent variables. The propensity score is output to data set named “psdataset”. The predicted probabilities are output to a variable named “ps”. proc logistic data = wuss; class naive0; model tx (event=’Drug A’) = age female b_hmo pre_drug_cnt_subset naive0 pre_refill_pct copay_idxdrug pre_sulf pre_htn pre_asthma pre_pain pre_lipo pre_depress /link=logit rsquare; output out=psdataset pred=ps; run; Where, tx = treatment selection indicator, age = age of patient at date of treatment initiation (index date), female = indicator variable, b_hmo= HMO insurance, copay_idxdrug = patient co-payment for initial prescription All variables with the prefix “pre_” describe utilization in the baseline period. pre_drug_cnt = number of medications (generic) utilized excluding medications for the categories below pre_sulf = sulfonylurea use pre_htn = hypertension use pre_asthma = asthma use pre_pain = pain medication use pre_lipo = lipotropic use pre_depress = antidepressant use pre_refill_pct = refill percentage for maintenance medications 2 After creating the propensity scores, an evaluation of the distributions can check comparability of the treatment groups. Sizeable overlaps among the groups illustrate satisfactory overlap in covariate distributions and indicate that the groups are comparable. 14 12 P D e r r u c g e n A t 10 8 6 4 2 0 14 12 P D e r r u c g e n B t 10 8 6 4 2 0 0. 33 0. 35 0. 37 0. 39 0. 41 0. 43 0. 45 0. 47 0. 49 0. 51 0. 53 0. 55 0. 57 0. 59 0. 61 0. 63 Est i m at ed Pr obabi l i t y Tables 1 and 2 demonstrate how propensity scoring can balance the groups. Table 1 contains the unadjusted values (before propensity scores) for the two treatment groups. Descriptors include demographic variables (age, gender), type of health plan insurance, and medication use in a 6 month period prior to treatment (baseline period). Age was the age of the patient at date of treatment initiation (index date). Type of insurance was categorized by health maintenance organization (HMO), Medicare, Medicaid, or self-insured. Prior medication use was measured for sulfonylureas, antihypertensives, lipid lowering agents as well as asthma and antidepressant medications. Refill patterns of medications for chronic diseases, or maintenance medications, were used to estimate patient’s behavior to following prescribed dosing. The TABULATE procedure was used to create the table below and PROC TTEST and PROC FREQ was used to test for differences between groups. Those observables that differed significantly in the two groups of patients include type of insurance (proportion of patients enrolled in an HMO), oral antidiabetic, sulfonylurea, and pain medication use. Table 1. Unadjusted Demographic and Baseline Measures UNADJUSTED VALUES Drug A Member Count 9,129 Age Mean 57.7 SD 10.5 Female % 43.1% HMO * % 73.3% # of Drugs Utilized Mean 4.9 3 Drug B 10,304 57.5 10.7 43.4% 75.3% 5.0 p-value 0.3786 0.7115 0.0018 0.0841 UNADJUSTED VALUES Maintenance Medication Refill * Prior Oral Antidiabetic Use * Sulfonylurea * Hypertension Lipid Irregularity Pain Management * Antidepressant Asthma * p < .05 SD % % % % % % % % Drug A 3.0 59.6% 77.5 55.3% 79.1% 68.3% 23.6% 19.5% 9.3% Drug B 3.1 60.5% 80.4 51.3% 78.4% 68.5% 24.9% 18.4% 9.9% p-value 0.0492 <.0001 <.0001 0.2381 0.7509 0.0357 0.0575 0.1967 Table 2 contains the adjusted values (after propensity scores) for the two treatment groups. PROC GLM was used to compare groups while adjusting for the propensity score. Differences between groups were minimized when using the propensity score method. Table 2. Propensity Score Adjusted Demographic and Baseline Measures WITH PROPENSTIY SCORING Drug A Drug B Member Count 9,129 10,304 Age Mean 57.4 57.4 SD 10.5 10.7 Female % 42.9% 42.8% HMO % 75.2% 75.1% # of Drugs Utilized Mean 4.9 4.9 SD 3.0 3.0 Maintenance Medication Refill % 60.3 60.3 Prior Oral Antidiabetic Use % 80.2% 80.7% Sulfonylurea % 53.2% 53.6% Hypertension % 78.6% 78.7% Lipid Irregularity % 68.7% 68.8% Pain Management % 24.0% 23.9% Antidepressant % 18.8% 18.8% Asthma % 9.5% 9.5% p-value 0.9329 0.9062 0.8922 0.9390 0.0687 0.3088 0.5204 0.8674 0.9303 0.8380 0.9112 0.9407 USE OF PROPENSITY SCORES Once the propensity score is calculated what to do you with them? As explained above, the 3 methods commonly used are matching on propensity score, stratification, and regression adjustment. Regression Adjustment Continuing with the study example described above, the created propensity scores were used in regression adjustment where a propensity score weight, also referred to as the inverse probability of treatment weight (IPTW), was calculated as the inverse of the propensity score (Hogan and Lancaster). The treatment selection model above modeled the propensity to receive drug a. For those patients receiving drug b, the propensity score would be 1- ps and the propensity score weight would be the inverse of 1-ps. data psdataset; set psdataset; if druga=1 then ps_weight=1/ps;else ps_weight=1/(1-ps); run; Next, a propensity score-weighted linear regression model, using the GLM procedure, was fitted to compare drug treatment on the outcome of compliance while controlling for other covariates. The LSMEANS statement computes the least-squares means for the treatment variable allowing for multiple comparisons. The ADJUST=TUKEY option uses the Tukey-Kramer method to adjust the least-squares means and the PDIFF and CL options give the p values and corresponding confidence intervals for the differences in the least-squares means. 4 proc glm data=psdataset; class tx naive0 female; model p_dayscovered = tx age female b_hmo pre_drug_cnt_subset naive0 pre_refill_pct copay_idxdrug pre_sulf pre_htn pre_asthma pre_pain pre_lipo pre_depress /solution; lsmeans tx/OM ADJUST=TUKEY PDIFF CL; weight ps_weight; quit; Results of the model above showed no difference between treatment groups (p = 0.2066). Comparing this result to unadjusted compliance means (a model with drug treatment as the only independent variable), where there is a significant difference (p=0.0071), shows the effect of controlling for selection bias and confounding. The increase in compliance from the unadjusted means (0.6673 and 0.6793) to the adjusted means (0.7032 and 0.7082) was probably due to the large number of patients with prior use of the drug. These patients had higher compliance values than those new to therapy and therefore when this variable is placed in the model as a covariate the mean compliance values rise. Table 3. Mean Compliance by Treatment: Unadjusted vs. PS Adjusted Models Model Treatment Compliance Outcome 95% Confidence Limits Unadjusted Model P = 0.0071 Drug A 0.6673 0.6601 0.6736 Drug B 0.6793 0.6733 0.6852 Propensity Score Adjusted Model P = 0.2066 Drug A 0.7032 0.6975 0.7089 Drug B 0.7082 0.7029 0.7135 Stratification Stratification, subclassification or binning using propensity scores involves grouping subjects into classes or strata based on the subject’s observed characteristics. Once the propensity scores are calculated, subjects are placed into strata (Cochran states that 5 strata can remove 90% of the bias) with the idea that subjects in the same stratum are similar in the characteristics used in the propensity score development process. The tutorial by D’Agostino details how to perform this technique. Briefly, quintiles are used to group subjects into five strata after making sure that there is adequate propensity score overlap between treatment groups. To prove that the propensity scores removed any bias due to differences in covariates between treatment groups, t-tests or chi-square tests are conducted before and after propensity score creation. Finally, outcomes and treatment effects can be assessed using models while adjusting for the propensity scores. Continuing with the example and code above, subjects are divided into 5 classes based on the common propensity score overlap using the RANK procedure. Checking for difference between treatment group before and after stratifying subjects by propensity scores can be done using PROC FREQ, PROC TTEST and PROC GLM. proc rank data= psdataset groups=5 out = r; ranks rnks + 1; var ps; run; data quintile; set r; quintile = rnks + 1; run; /*check for differences in groups before propensity score*/ proc freq data=quintile; tables tx*(female b_hmo naive0 pre_sulf pre_htn pre_asthma pre_pain pre_lipo pre_depress)/chisq;run; 5 proc ttest data=quintile; var age pre_drug_cnt_subset maintrefillratio; class tx;run; /*check for differences in groups while adjusting for propensity scores*/ proc glm data=quintile; class tx quintile; model age female b_hmo pre_drug_cnt_subset maintrefillratio naive0 pre_sulf pre_htn pre_asthma pre_pain pre_lipo pre_depress = tx quintile; lsmeans tx; quit; The result is similar to Tables 1 and 2 above showing minimal differences between groups when subclassifying subjects. Outcomes can be compared within the 5 subclasses or averaged to report for the overall treatment groups. Matching Matching groups by propensity scores is a common method to balance groups on covariates. Once the propensity score is calculated, subjects are matched by this single score as opposed to traditional direct matching by one or more covariates. A disadvantage of matching methods includes incomplete matching and inexact matching. That is, subjects may be excluded because of difficulty finding a match. Reducing this bias is well explained in Lori Parsons’ papers (see reference section). Her papers offer code for performing case-control match using a greedy matching algorithm. As she explains in the paper, cases were matched to controls based on propensity scores. Tables 1 and 2 below show how propensity scores were used to balance a treated (“Early Intervention” group) and untreated group of subjects (“Conservative” group). Table 1 is the original population and includes results of rank-sum tests and chisquare tests showing differences between groups in many characteristics (only a few a shown in the tables). Table 2 shows how differences are eliminated after matching. This matched subset of patients can now be used to model outcomes and assess effect of treatment. Table 1: Original Population Total Patients Age (Mean±sd) Male Gender White Race Hx Angina Hx MI Early Intervention N (%) 2,402 61.3 ±12.2 1,744 (72.6) 2,079 (91.8) 444 (18.5) 574 (23.9) Conservative N (%) 17,735 68.2±13.0 10,914 (61.5) 15,002 (88.4) 4,441 (25.0) 5,382 (30.3) p-value Conservative N (%) 2,036 61.7±13.3 1,445 (71.0) 1,858 (91.3) 381 (18.7) 491 (24.1) p-value <0.0001 <0.0001 <0.0001 <0.0001 <0.0001 Table 2: Greedy 5 to 1 Digit Matched Population Total Patients Age (Mean±sd) Male Gender White Race Hx Angina Hx MI Early Intervention N (%) 2,036 61.9 ±12.0 1,452 (71.3) 1,865 (91.6) 390 (19.2) 488 (24.0) 0.5405 0.8087 0.6952 0.7189 0.9124 SUMMARY OF AND STRENGTHS AND LIMITATIONS OF PROPENSITY SCORING Calculating the propensity score as the conditional probability of treatment summarizes observed values into a single score. The scores can then be used to control for selection bias by matching subjects, stratifying subjects, and/or as a regressor. All techniques have the purpose of balancing groups to remove bias when assessing treatment effect on outcomes. Traditional techniques to control for bias may be limited if accounting for only a few covariates. Compared to multiple regression, the propensity score methodology summarizes many observables and is less sensitive to model misspecification (Perkins et al). Propensity scores can also diagnose if groups are comparable before moving onto the modeling stage of analysis. If distributions of the propensity scores fail to show much overlap in covariate values, the comparison groups are too different making it difficult to balance groups. When creating propensity scores all covariates that affect both treatment and outcome must be included in the model and it is assumed that all patients have a non zero probability of receiving each treatment. The technique only looks at observed characteristics of a patient thus does not account for unobserved factors, such as patient attitudes, 6 socioeconomic status, and education level. This limitation is modified if unobserved covariates are correlated to observed factors. Analyses also need large samples sizes in order to establish adequate variance in covariate distributions. CONCLUSION This tutorial introduces potential selection bias in observational studies and describes how propensity score methodology can control for overt bias when estimating treatment effectiveness. Propensity scoring methodology attempts to balance groups before comparing outcomes between treatment groups. Commonly used techniques via propensity scores include matching, stratification, and regression adjustment using the inverse of the propensity score. Each method can be used in conjunction with traditional risk adjustment techniques to reduce bias and better describe the effect of treatment on outcomes. REFERENCES D’Agostino R.B. Sr, Kwan H. 1995. “Measuring Effectiveness: What to Expect Without a Randomized Control Group”. Medical Care. 195:33 (4 suppl): AS95-AS105. D’Agostino R.B., Jr, D’Agostino R.B., Sr. 2007. “Estimating Treatment Effects Using Observational Data”. JAMA. 297 (3). 314-316 Rosenbaum P.R. and Rubin D.B. 1983. “The Central Role of the Propensity Score in Observational Studies for Causal Effects”, Biometrika, 70, 41-55. D’Agostino, R.B. 1998. “Tutorial on Biostatistics: Propensity Score Methods for Bias Reduction in the comparison of a treatment to a non-randomized control group”. Statistics in Medicine 17, 2265-2281. Hogan, J.W., Lancaster, T. 2004. ”Instrumental variable and propensity weighting for causal inference from longitudinal observational studies”. Statistical Methods in Medical Research 13: 17-48. Obenchain, R.L., Melfi, C.A., “Propensity Score and Heckman Adjustments for Treatment Selection Bias in Database Studies”. Pasta, David J. 2000. “Using Propensity Scores to Adjust for Group Differences: Examples Comparing Alternative Surgical Methods”. Proceedings of the Twenty-Fifth Annual SAS Users Group International Conference, Indianapolis, IN, 261-25. Parsons, Lori. 2000. “Using SAS® Software to Perform a Case Control Match on Propensity Score in an Observational Study”. Proceedings of the Twenty-Fifth Annual SAS Users Group International Conference, Indianapolis, IN, 214-26. SAS Institute Inc. 2004. “SAS Procedures: The LOGISTIC Procedure”. SAS OnlineDoc® 9.1.3. Cary, NC: SAS Institute Inc. http://support.sas.com/documentation/onlinedoc/91pdf/sasdoc_91/stat_ug_7313.pdf CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the author at: R. Scott Leslie MedImpact Healthcare Systems, Inc. 10680 Treena Street San Diego, CA 92131 Work Phone: 858-790-6685 Fax: 858-689-1799 Email: [email protected] SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are trademarks of their respective companies. 7