Jump to ContentJump to Main Navigation
Show Summary Details

The International Journal of Biostatistics

Ed. by Chambaz, Antoine / Hubbard, Alan E. / van der Laan, Mark J.

2 Issues per year

IMPACT FACTOR 2015: 0.667
5-year IMPACT FACTOR: 1.188

SCImago Journal Rank (SJR) 2015: 0.495
Source Normalized Impact per Paper (SNIP) 2015: 0.180
Impact per Publication (IPP) 2015: 0.319

Mathematical Citation Quotient (MCQ) 2015: 0.04

See all formats and pricing


Type I Error Rates, Coverage of Confidence Intervals, and Variance Estimation in Propensity-Score Matched Analyses

Peter C Austin
  • Institute for Clinical Evaluative Sciences
Published Online: 2009-04-14 | DOI: https://doi.org/10.2202/1557-4679.1146

Propensity-score matching is frequently used in the medical literature to reduce or eliminate the effect of treatment selection bias when estimating the effect of treatments or exposures on outcomes using observational data. In propensity-score matching, pairs of treated and untreated subjects with similar propensity scores are formed. Recent systematic reviews of the use of propensity-score matching found that the large majority of researchers ignore the matched nature of the propensity-score matched sample when estimating the statistical significance of the treatment effect. We conducted a series of Monte Carlo simulations to examine the impact of ignoring the matched nature of the propensity-score matched sample on Type I error rates, coverage of confidence intervals, and variance estimation of the treatment effect. We examined estimating differences in means, relative risks, odds ratios, rate ratios from Poisson models, and hazard ratios from Cox regression models. We demonstrated that accounting for the matched nature of the propensity-score matched sample tended to result in type I error rates that were closer to the advertised level compared to when matching was not incorporated into the analyses. Similarly, accounting for the matched nature of the sample tended to result in confidence intervals with coverage rates that were closer to the nominal level, compared to when matching was not taken into account. Finally, accounting for the matched nature of the sample resulted in estimates of standard error that more closely reflected the sampling variability of the treatment effect compared to when matching was not taken into account.

Keywords: propensity score; matching; propensity-score matching; variance estimation; coverage; simulations; type I error; observational studies

Published Online: 2009-04-14

Citation Information: The International Journal of Biostatistics. Volume 5, Issue 1, ISSN (Online) 1557-4679, DOI: https://doi.org/10.2202/1557-4679.1146, April 2009

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Onkar V. Khullar, Yuan Liu, Theresa Gillespie, Kristin A. Higgins, Suresh Ramalingam, Joseph Lipscomb, and Felix G Fernandez
Journal of Thoracic Oncology, 2015, Volume 10, Number 11, Page 1625
Rajender R. Aparasu, Satabdi Chatterjee, Sandhya Mehta, and Hua Chen
Medical Care, 2012, Volume 50, Number 11, Page 961
Gyung-Min Park, Seon Ha Kim, Min-Woo Jo, Sung Ho Her, Seungbong Han, Jung-Min Ahn, Duk-Woo Park, Soo-Jin Kang, Seung-Whan Lee, Young-Hak Kim, Cheol Whan Lee, Beom-Jun Kim, Jung-Min Koh, Hong-Kyu Kim, Jaewon Choe, Seong-Wook Park, and Seung-Jung Park
Medicine, 2015, Volume 94, Number 21, Page e917
Masatsugu Hamaji, Fengshi Chen, Yukinori Matsuo, Atsushi Kawaguchi, Satoshi Morita, Nami Ueki, Makoto Sonobe, Yasushi Nagata, Masahiro Hiraoka, and Hiroshi Date
The Annals of Thoracic Surgery, 2015, Volume 99, Number 4, Page 1122
Victor A. Ferraris, Daniel L. Davenport, Sibu P. Saha, Alethea Bernard, Peter C. Austin, and Joseph B. Zwischenberger
The Annals of Thoracic Surgery, 2011, Volume 91, Number 6, Page 1674
Sandhya Mehta, Hua Chen, Michael Johnson, and Rajender R. Aparasu
The American Journal of Geriatric Pharmacotherapy, 2011, Volume 9, Number 2, Page 120
Sandhya Mehta, Hua Chen, Michael L. Johnson, and Rajender R. Aparasu
Drugs & Aging, 2010, Volume 27, Number 10, Page 815
Dallas P. Seitz, Sudeep S. Gill, Chaim M. Bell, Peter C. Austin, Andrea Gruneir, Geoff M. Anderson, and Paula A. Rochon
Journal of the American Geriatrics Society, 2014, Volume 62, Number 11, Page 2102
Peter C. Austin
Statistics in Medicine, 2013, Volume 32, Number 16, Page 2837
Peter C. Austin and Dylan S. Small
Statistics in Medicine, 2014, Volume 33, Number 24, Page 4306
Liang Li
Communications in Statistics - Simulation and Computation, 2014, Volume 43, Number 10, Page 2498
David J. Peters, Andy Hochstetler, Matt DeLisi, and Hui-Ju Kuo
Journal of Quantitative Criminology, 2015, Volume 31, Number 1, Page 149
Melissa M. Garrido, Amy S. Kelley, Julia Paris, Katherine Roza, Diane E. Meier, R. Sean Morrison, and Melissa D. Aldridge
Health Services Research, 2014, Volume 49, Number 5, Page 1701
Etienne Gayat, Matthieu Resche-Rigon, Jean-Yves Mary, and Raphaël Porcher
Pharmaceutical Statistics, 2012, Volume 11, Number 3, Page 222
Robin Mitra and Jerome P. Reiter
Statistics in Medicine, 2011, Volume 30, Number 6, Page 627

Comments (0)

Please log in or register to comment.