Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter March 11, 2015

Model Uncertainty and Model Averaging in Regression Discontinuity Designs

Patrick Button

Abstract

Parametric (polynomial) models are popular in research employing regression discontinuity designs and are required when data are discrete. However, researchers often choose a parametric model based on data inspection or pretesting. These approaches lead to standard errors and confidence intervals that are too small because they do not incorporate model uncertainty. I propose using Frequentist model averaging to incorporate model uncertainty into parametric models. My Monte Carlo experiments show that Frequentist model averaging leads to mean square error and coverage probability improvements over pretesting. An application to [Lee, D. S. 2008. “Randomized Experiments From Non-Random Selection in US House Elections.” Journal of Econometrics 142 (2): 675–697.] shows how this approach works in practice, and how conventionally selected models may not be ideal.

JEL classifications:: C21; C26; C52; C54

Corresponding author: Patrick Button, PhD Candidate, Department of Economics, University of California, Irvine, CA, USA, E-mail: ; Website: http://www.patrickbutton.com

Acknowledgement

I am grateful to Marianne Bitler, David Brownstone, Yingying Dong, Atsushi Inoue, Ivan Jeliazkov, David Neumark, and two anonymous referees for valuable feedback. I also thank Joseph A. Farran at UCI’s High Performance Computing Cluster for assistance.

References

Akaike, H. 1974. “A New Look at The Statistical Model Identification.” Automatic Control, IEEE Transactions on 19 (6): 716–723.10.1109/TAC.1974.1100705Search in Google Scholar

Breiman, L. 1992. “The Little Bootstrap and Other Methods for Dimensionality Selection in Regression: X-Fixed Prediction Error.” Journal of the American Statistical Association 87 (419): 738–754.10.1080/01621459.1992.10475276Search in Google Scholar

Buckland, S. T., K. P. Burnham, and N. H. Augustin. 1997. “Model Selection: An Integral Part of Inference.” Biometrics 53: 603–618.10.2307/2533961Search in Google Scholar

Burnham, K. P., and D. R. Anderson. 2002. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. New York, NY: Springer Verlag.Search in Google Scholar

Card, D., C. Dobkin, and N. Maestas. 2009. “Does Medicare Save Lives?” The Quarterly Journal of Economics 124 (2): 597–636.10.1162/qjec.2009.124.2.597Search in Google Scholar PubMed PubMed Central

Chatfield, C. 1995. “Model Uncertainty, Data Mining and Statistical Inference.” Journal of the Royal Statistical Society Series a-Statistics in Society 158: 419–466.10.2307/2983440Search in Google Scholar

Chib, S., and E. Greenberg. 2014. Nonparametric Bayes Analysis of the Sharp and Fuzzy Regression Discontinuity Designs. Mimeo 2014. http://economics.sas.upenn.edu/system/files/event_papers/cgbayesrddmarch2014.pdf (accessed February 17, 2015).Search in Google Scholar

Claeskens, G., and N. L. Hjort. 2008. Model Selection and Model Averaging. Cambridge, UK: Cambridge University Press.Search in Google Scholar

Cohen, A. 1965. “Estimates of Linear Combinations of the Parameters in The Mean Vector of a Multivariate Distribution.” The Annals of Mathematical Statistics 36 (1): 78–87.10.1214/aoms/1177700272Search in Google Scholar

Cook, T. D. 2008. “’Waiting for Life to Arrive’: A History of The Regression-Discontinuity Design in Psychology, Statistics and Economics.” Journal of Econometrics 142 (2): 636–654.10.1016/j.jeconom.2007.05.002Search in Google Scholar

Davidson, R., and J. G. MacKinnon. 1993. Estimation and Inference in Econometrics. USA: Oxford University Press.Search in Google Scholar

Draper, D., J. S. Hodges, E. E. Leamer, C. N. Morris, and D. B. Rubin. 1987. A Research Agenda for Assessment and Propagation of Model Uncertainty. Santa Monica, CA: RAND. http://www.rand.org/content/dam/rand/pubs/notes/2009/N2683.pdf (accessed February 17, 2015).Search in Google Scholar

Gelman, A., and G. Imbens. 2014. “Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs.” National Bureau of Economic Research Working Paper #20405.10.3386/w20405Search in Google Scholar

Giles, J. A., and D. E. A. Giles. 1993. “Pre-test Estimation and Testing in Econometrics: Recent Developments.” Journal of Economic Surveys 7 (2): 145–197.10.1111/j.1467-6419.1993.tb00163.xSearch in Google Scholar

Greene, W. H. 2008. Econometric Analysis, Vol. 6. Upper Saddle River, NJ: Pearson Prentice Hall.Search in Google Scholar

Hansen, B. E. 2007. “Least Squares Model Averaging.” Econometrica 75 (4): 1175–1189.10.1111/j.1468-0262.2007.00785.xSearch in Google Scholar

Hansen, B. E., and J. S. Racine. 2012. “Jackknife Model Averaging.” Journal of Econometrics 167 (1): 38–46.10.1016/j.jeconom.2011.06.019Search in Google Scholar

Hjort, N. L., and G. Claeskens. 2003. “Frequentist Model Average Estimators.” Journal of the American Statistical Association 98 (464): 879–899.10.1198/016214503000000828Search in Google Scholar

Hodges, J. S. 1987. “Uncertainty, Policy Analysis and Statistics.” Statistical Science 2 (3): 259–275.10.1214/ss/1177013224Search in Google Scholar

Hoeting, J. A., D. Madigan, A. E. Raftery, and C. T. Volinsky. 1999. “Bayesian Model Averaging: A Tutorial.” Statistical Science 14 (4): 382–401.Search in Google Scholar

Lee, D. S. 2008. “Randomized Experiments From Non-Random Selection in Us House Elections.” Journal of Econometrics 142 (2): 675–697.10.1016/j.jeconom.2007.05.004Search in Google Scholar

Lee, D. S., and D. Card. 2008. “Regression Discontinuity Inference With Specification Error.” Journal of Econometrics 142 (2): 655–674.10.1016/j.jeconom.2007.05.003Search in Google Scholar

Lee, D. S., and T. Lemieux. 2010. “Regression Discontinuity Designs in Economics.” Journal of Economic Literature 48: 281–355.10.1257/jel.48.2.281Search in Google Scholar

Lee, D. S., E. Moretti, and M. J. Butler. 2004. “Do Voters Affect or Elect Policies? Evidence From The Us House.” The Quarterly Journal of Economics 119 (3): 807–859.10.1162/0033553041502153Search in Google Scholar

Leeb, H., and B. M. Pötscher. 2005. “Model Selection and Inference: Facts and Fiction.” Econometric Theory 21 (1): 21–59.10.1017/S0266466605050036Search in Google Scholar

Liang, H., G. Zou, A. T. K. Wan, and X. Zhang. 2011. “Optimal Weight Choice for Frequentist Model Average Estimators.” Journal of the American Statistical Association 106 (495): 1053–1066.10.1198/jasa.2011.tm09478Search in Google Scholar

Moral-Benito, E. 2010.“Model Averaging In Economics.” Centro de Estudios Monetarios y Financieros (CEMFI) Working Paper, 8.Search in Google Scholar

Oreopoulos, P. 2006. “Estimating Average and Local Average Treatment Effects of Education When Compulsory Schooling Laws Really Matter.” The American Economic Review 96: 152–175.10.1257/000282806776157641Search in Google Scholar

Pötscher, B. M. 1991. “Effects of Model Selection on Inference.” Econometric Theory 7 (2): 163–185.10.1017/S0266466600004382Search in Google Scholar

Raftery, A. E., D. Madigan, and J. A. Hoeting. 1997. “Bayesian Model Averaging for Linear Regression Models.” Journal of the American Statistical Association 92 (437): 179–191.10.1080/01621459.1997.10473615Search in Google Scholar

Rau, T. 2011. “Bayesian Inference in the Regression Discontinuity Model.” Vigesimosextas Jornadas Anuales de Economa. http://eco3147.facea.puc.cl/economia_puc/images/stories/Instituto/Profesores/rd4.pdf (accessed February 17, 2015).Search in Google Scholar

Schwarz, G. 1978. “Estimating the Dimension of a Model.” The Annals of Statistics 6 (2): 461–464.10.1214/aos/1176344136Search in Google Scholar

Sun, Y. 2005. “Adaptive Estimation of the Regression Discontinuity Model.” Unpublished Manuscript, University of California, San Diego. http://econweb.ucsd.edu/~yisun/ard.pdf (accessed February 17, 2015).Search in Google Scholar

Wallace, T. D. 1977. “Pretest Estimation in Regression: A Survey.” American Journal of Agricultural Economics 59 (3): 431–443.10.2307/1239645Search in Google Scholar

Yuan, Z., and Y. Yang. 2005. “Combining Linear Regression Models.” Journal of the American Statistical Association 100 (472): 1202–1214.10.1198/016214505000000088Search in Google Scholar

Published Online: 2015-3-11
Published in Print: 2016-1-1

©2016 by De Gruyter