Evaluating treatment effectiveness in patient subgroups: a comparison of propensity score methods with an automated matching approach

Rosalba Radice 1 , Roland Ramsahai 2 , Richard Grieve 3 , Noemi Kreif 4 , Zia Sadique 5  and Jasjeet S. Sekhon 6
  • 1 CLondon School of Hygiene & Tropical Medicine
  • 2 Centre for Statistical Methodology, LSHTM
  • 3 Centre for Statistical Methodology, LSHTM
  • 4 Centre for Statistical Methodology, LSHTM
  • 5 Centre for Statistical Methodology, LSHTM
  • 6 University of California, Berkeley


Propensity score (Pscore) matching and inverse probability of treatment weighting (IPTW) can remove bias due to observed confounders, if the Pscore is correctly specified. Genetic Matching (GenMatch) matches on the Pscore and individual covariates using an automated search algorithm to balance covariates. This paper compares common ways of implementing Pscore matching and IPTW, with Genmatch for balancing time-constant baseline covariates}. The methods are considered when estimates of treatment effectiveness are required for patient subgroups, and the treatment allocation process differs by subgroup. We apply these methods in a prospective cohort study that estimates the effectiveness of Drotrecogin alfa activated, for subgroups of patients with severe sepsis. In a simulation study we compare the methods when the Pscore is correctly specified, and then misspecified by ignoring the subgroup-specific treatment allocation. The simulations also consider poor overlap in baseline covariates, and different sample sizes. In the case study, GenMatch reports better covariate balance than IPTW or Pscore matching. In the simulations with correctly specified Pscores, good overlap and reasonable sample sizes, all methods report minimal bias. When the Pscore is misspecified, GenMatch reports the least imbalance and bias. With small sample sizes, IPTW is the most efficient approach, but all methods report relatively high bias of treatment effects. This study shows that overall GenMatch achieves the best covariate balance for each subgroup, and is more robust to Pscore misspecification than common alternative Pscore approaches.

If the inline PDF is not rendering correctly, you can download the PDF file here.


Journal + Issues

IJB publishes biostatistical models and methods, statistical theory, as well as original applications of statistical methods, for important practical problems arising from various sciences. It covers the entire range of biostatistics, from theoretical advances to relevant and sensible translations of a practical problem into a statistical framework, including advances in biostatistical computing.