Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Journal of Econometric Methods

Ed. by Giacomini, Raffaella / Li, Tong

1 Issue per year

Online
ISSN
2156-6674
See all formats and pricing
More options …

Model Uncertainty and Model Averaging in Regression Discontinuity Designs

Patrick Button
Published Online: 2015-03-11 | DOI: https://doi.org/10.1515/jem-2014-0016

Abstract

Parametric (polynomial) models are popular in research employing regression discontinuity designs and are required when data are discrete. However, researchers often choose a parametric model based on data inspection or pretesting. These approaches lead to standard errors and confidence intervals that are too small because they do not incorporate model uncertainty. I propose using Frequentist model averaging to incorporate model uncertainty into parametric models. My Monte Carlo experiments show that Frequentist model averaging leads to mean square error and coverage probability improvements over pretesting. An application to [Lee, D. S. 2008. “Randomized Experiments From Non-Random Selection in US House Elections.” Journal of Econometrics 142 (2): 675–697.] shows how this approach works in practice, and how conventionally selected models may not be ideal.

Keywords: causal inference; discrete data; model averaging; model selection; quasi-experimental methods; regression discontinuity design

JEL classifications:: C21; C26; C52; C54

References

  • Akaike, H. 1974. “A New Look at The Statistical Model Identification.” Automatic Control, IEEE Transactions on 19 (6): 716–723.Google Scholar

  • Breiman, L. 1992. “The Little Bootstrap and Other Methods for Dimensionality Selection in Regression: X-Fixed Prediction Error.” Journal of the American Statistical Association 87 (419): 738–754.Google Scholar

  • Buckland, S. T., K. P. Burnham, and N. H. Augustin. 1997. “Model Selection: An Integral Part of Inference.” Biometrics 53: 603–618.Google Scholar

  • Burnham, K. P., and D. R. Anderson. 2002. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. New York, NY: Springer Verlag.Google Scholar

  • Card, D., C. Dobkin, and N. Maestas. 2009. “Does Medicare Save Lives?” The Quarterly Journal of Economics 124 (2): 597–636.Google Scholar

  • Chatfield, C. 1995. “Model Uncertainty, Data Mining and Statistical Inference.” Journal of the Royal Statistical Society Series a-Statistics in Society 158: 419–466.Google Scholar

  • Chib, S., and E. Greenberg. 2014. Nonparametric Bayes Analysis of the Sharp and Fuzzy Regression Discontinuity Designs. Mimeo 2014. http://economics.sas.upenn.edu/system/files/event_papers/cgbayesrddmarch2014.pdf (accessed February 17, 2015).

  • Claeskens, G., and N. L. Hjort. 2008. Model Selection and Model Averaging. Cambridge, UK: Cambridge University Press.Google Scholar

  • Cohen, A. 1965. “Estimates of Linear Combinations of the Parameters in The Mean Vector of a Multivariate Distribution.” The Annals of Mathematical Statistics 36 (1): 78–87.CrossrefGoogle Scholar

  • Cook, T. D. 2008. “’Waiting for Life to Arrive’: A History of The Regression-Discontinuity Design in Psychology, Statistics and Economics.” Journal of Econometrics 142 (2): 636–654.Google Scholar

  • Davidson, R., and J. G. MacKinnon. 1993. Estimation and Inference in Econometrics. USA: Oxford University Press.Google Scholar

  • Draper, D., J. S. Hodges, E. E. Leamer, C. N. Morris, and D. B. Rubin. 1987. A Research Agenda for Assessment and Propagation of Model Uncertainty. Santa Monica, CA: RAND. http://www.rand.org/content/dam/rand/pubs/notes/2009/N2683.pdf (accessed February 17, 2015).

  • Gelman, A., and G. Imbens. 2014. “Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs.” National Bureau of Economic Research Working Paper #20405.Google Scholar

  • Giles, J. A., and D. E. A. Giles. 1993. “Pre-test Estimation and Testing in Econometrics: Recent Developments.” Journal of Economic Surveys 7 (2): 145–197.CrossrefGoogle Scholar

  • Greene, W. H. 2008. Econometric Analysis, Vol. 6. Upper Saddle River, NJ: Pearson Prentice Hall.Google Scholar

  • Hansen, B. E. 2007. “Least Squares Model Averaging.” Econometrica 75 (4): 1175–1189.Web of ScienceGoogle Scholar

  • Hansen, B. E., and J. S. Racine. 2012. “Jackknife Model Averaging.” Journal of Econometrics 167 (1): 38–46.Web of ScienceGoogle Scholar

  • Hjort, N. L., and G. Claeskens. 2003. “Frequentist Model Average Estimators.” Journal of the American Statistical Association 98 (464): 879–899.Google Scholar

  • Hodges, J. S. 1987. “Uncertainty, Policy Analysis and Statistics.” Statistical Science 2 (3): 259–275.CrossrefGoogle Scholar

  • Hoeting, J. A., D. Madigan, A. E. Raftery, and C. T. Volinsky. 1999. “Bayesian Model Averaging: A Tutorial.” Statistical Science 14 (4): 382–401.Google Scholar

  • Lee, D. S. 2008. “Randomized Experiments From Non-Random Selection in Us House Elections.” Journal of Econometrics 142 (2): 675–697.Web of ScienceGoogle Scholar

  • Lee, D. S., and D. Card. 2008. “Regression Discontinuity Inference With Specification Error.” Journal of Econometrics 142 (2): 655–674.Web of ScienceGoogle Scholar

  • Lee, D. S., and T. Lemieux. 2010. “Regression Discontinuity Designs in Economics.” Journal of Economic Literature 48: 281–355.Web of ScienceGoogle Scholar

  • Lee, D. S., E. Moretti, and M. J. Butler. 2004. “Do Voters Affect or Elect Policies? Evidence From The Us House.” The Quarterly Journal of Economics 119 (3): 807–859.Google Scholar

  • Leeb, H., and B. M. Pötscher. 2005. “Model Selection and Inference: Facts and Fiction.” Econometric Theory 21 (1): 21–59.CrossrefGoogle Scholar

  • Liang, H., G. Zou, A. T. K. Wan, and X. Zhang. 2011. “Optimal Weight Choice for Frequentist Model Average Estimators.” Journal of the American Statistical Association 106 (495): 1053–1066.Google Scholar

  • Moral-Benito, E. 2010.“Model Averaging In Economics.” Centro de Estudios Monetarios y Financieros (CEMFI) Working Paper, 8.Google Scholar

  • Oreopoulos, P. 2006. “Estimating Average and Local Average Treatment Effects of Education When Compulsory Schooling Laws Really Matter.” The American Economic Review 96: 152–175.Google Scholar

  • Pötscher, B. M. 1991. “Effects of Model Selection on Inference.” Econometric Theory 7 (2): 163–185.CrossrefGoogle Scholar

  • Raftery, A. E., D. Madigan, and J. A. Hoeting. 1997. “Bayesian Model Averaging for Linear Regression Models.” Journal of the American Statistical Association 92 (437): 179–191.Google Scholar

  • Rau, T. 2011. “Bayesian Inference in the Regression Discontinuity Model.” Vigesimosextas Jornadas Anuales de Economa. http://eco3147.facea.puc.cl/economia_puc/images/stories/Instituto/Profesores/rd4.pdf (accessed February 17, 2015).

  • Schwarz, G. 1978. “Estimating the Dimension of a Model.” The Annals of Statistics 6 (2): 461–464.CrossrefGoogle Scholar

  • Sun, Y. 2005. “Adaptive Estimation of the Regression Discontinuity Model.” Unpublished Manuscript, University of California, San Diego. http://econweb.ucsd.edu/~yisun/ard.pdf (accessed February 17, 2015).

  • Wallace, T. D. 1977. “Pretest Estimation in Regression: A Survey.” American Journal of Agricultural Economics 59 (3): 431–443.Google Scholar

  • Yuan, Z., and Y. Yang. 2005. “Combining Linear Regression Models.” Journal of the American Statistical Association 100 (472): 1202–1214.Google Scholar

About the article

Corresponding author: Patrick Button, PhD Candidate, Department of Economics, University of California, Irvine, CA, USA, E-mail: ; Website: http://www.patrickbutton.com


Published Online: 2015-03-11

Published in Print: 2016-01-01


Citation Information: Journal of Econometric Methods, Volume 5, Issue 1, Pages 103–116, ISSN (Online) 2156-6674, ISSN (Print) 2194-6345, DOI: https://doi.org/10.1515/jem-2014-0016.

Export Citation

©2016 by De Gruyter.Get Permission

Citing Articles

Here you can find all Crossref-listed publications in which this article is cited. If you would like to receive automatic email messages as soon as this article is cited in other publications, simply activate the “Citation Alert” on the top of this page.

Comments (0)

Please log in or register to comment.
Log in