Model Uncertainty and Model Averaging in Regression Discontinuity Designs

Patrick Button 1
  • 1 Candidate, Department of Economics, University of California, Irvine, CA, USA
Patrick Button

Abstract

Parametric (polynomial) models are popular in research employing regression discontinuity designs and are required when data are discrete. However, researchers often choose a parametric model based on data inspection or pretesting. These approaches lead to standard errors and confidence intervals that are too small because they do not incorporate model uncertainty. I propose using Frequentist model averaging to incorporate model uncertainty into parametric models. My Monte Carlo experiments show that Frequentist model averaging leads to mean square error and coverage probability improvements over pretesting. An application to [Lee, D. S. 2008. “Randomized Experiments From Non-Random Selection in US House Elections.” Journal of Econometrics 142 (2): 675–697.] shows how this approach works in practice, and how conventionally selected models may not be ideal.

  • Akaike, H. 1974. “A New Look at The Statistical Model Identification.” Automatic Control, IEEE Transactions on 19 (6): 716–723.

    • Crossref
    • Export Citation
  • Breiman, L. 1992. “The Little Bootstrap and Other Methods for Dimensionality Selection in Regression: X-Fixed Prediction Error.” Journal of the American Statistical Association 87 (419): 738–754.

    • Crossref
    • Export Citation
  • Buckland, S. T., K. P. Burnham, and N. H. Augustin. 1997. “Model Selection: An Integral Part of Inference.” Biometrics 53: 603–618.

    • Crossref
    • Export Citation
  • Burnham, K. P., and D. R. Anderson. 2002. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach. New York, NY: Springer Verlag.

  • Card, D., C. Dobkin, and N. Maestas. 2009. “Does Medicare Save Lives?” The Quarterly Journal of Economics 124 (2): 597–636.

    • Crossref
    • Export Citation
  • Chatfield, C. 1995. “Model Uncertainty, Data Mining and Statistical Inference.” Journal of the Royal Statistical Society Series a-Statistics in Society 158: 419–466.

    • Crossref
    • Export Citation
  • Chib, S., and E. Greenberg. 2014. Nonparametric Bayes Analysis of the Sharp and Fuzzy Regression Discontinuity Designs. Mimeo 2014. http://economics.sas.upenn.edu/system/files/event_papers/cgbayesrddmarch2014.pdf (accessed February 17, 2015).

  • Claeskens, G., and N. L. Hjort. 2008. Model Selection and Model Averaging. Cambridge, UK: Cambridge University Press.

  • Cohen, A. 1965. “Estimates of Linear Combinations of the Parameters in The Mean Vector of a Multivariate Distribution.” The Annals of Mathematical Statistics 36 (1): 78–87.

    • Crossref
    • Export Citation
  • Cook, T. D. 2008. “’Waiting for Life to Arrive’: A History of The Regression-Discontinuity Design in Psychology, Statistics and Economics.” Journal of Econometrics 142 (2): 636–654.

    • Crossref
    • Export Citation
  • Davidson, R., and J. G. MacKinnon. 1993. Estimation and Inference in Econometrics. USA: Oxford University Press.

  • Draper, D., J. S. Hodges, E. E. Leamer, C. N. Morris, and D. B. Rubin. 1987. A Research Agenda for Assessment and Propagation of Model Uncertainty. Santa Monica, CA: RAND. http://www.rand.org/content/dam/rand/pubs/notes/2009/N2683.pdf (accessed February 17, 2015).

  • Gelman, A., and G. Imbens. 2014. “Why High-Order Polynomials Should Not Be Used in Regression Discontinuity Designs.” National Bureau of Economic Research Working Paper #20405.

    • Crossref
    • Export Citation
  • Giles, J. A., and D. E. A. Giles. 1993. “Pre-test Estimation and Testing in Econometrics: Recent Developments.” Journal of Economic Surveys 7 (2): 145–197.

    • Crossref
    • Export Citation
  • Greene, W. H. 2008. Econometric Analysis, Vol. 6. Upper Saddle River, NJ: Pearson Prentice Hall.

  • Hansen, B. E. 2007. “Least Squares Model Averaging.” Econometrica 75 (4): 1175–1189.

    • Crossref
    • Export Citation
  • Hansen, B. E., and J. S. Racine. 2012. “Jackknife Model Averaging.” Journal of Econometrics 167 (1): 38–46.

    • Crossref
    • Export Citation
  • Hjort, N. L., and G. Claeskens. 2003. “Frequentist Model Average Estimators.” Journal of the American Statistical Association 98 (464): 879–899.

    • Crossref
    • Export Citation
  • Hodges, J. S. 1987. “Uncertainty, Policy Analysis and Statistics.” Statistical Science 2 (3): 259–275.

    • Crossref
    • Export Citation
  • Hoeting, J. A., D. Madigan, A. E. Raftery, and C. T. Volinsky. 1999. “Bayesian Model Averaging: A Tutorial.” Statistical Science 14 (4): 382–401.

  • Lee, D. S. 2008. “Randomized Experiments From Non-Random Selection in Us House Elections.” Journal of Econometrics 142 (2): 675–697.

  • Lee, D. S., and D. Card. 2008. “Regression Discontinuity Inference With Specification Error.” Journal of Econometrics 142 (2): 655–674.

    • Crossref
    • Export Citation
  • Lee, D. S., and T. Lemieux. 2010. “Regression Discontinuity Designs in Economics.” Journal of Economic Literature 48: 281–355.

    • Crossref
    • Export Citation
  • Lee, D. S., E. Moretti, and M. J. Butler. 2004. “Do Voters Affect or Elect Policies? Evidence From The Us House.” The Quarterly Journal of Economics 119 (3): 807–859.

    • Crossref
    • Export Citation
  • Leeb, H., and B. M. Pötscher. 2005. “Model Selection and Inference: Facts and Fiction.” Econometric Theory 21 (1): 21–59.

    • Crossref
    • Export Citation
  • Liang, H., G. Zou, A. T. K. Wan, and X. Zhang. 2011. “Optimal Weight Choice for Frequentist Model Average Estimators.” Journal of the American Statistical Association 106 (495): 1053–1066.

    • Crossref
    • Export Citation
  • Moral-Benito, E. 2010.“Model Averaging In Economics.” Centro de Estudios Monetarios y Financieros (CEMFI) Working Paper, 8.

  • Oreopoulos, P. 2006. “Estimating Average and Local Average Treatment Effects of Education When Compulsory Schooling Laws Really Matter.” The American Economic Review 96: 152–175.

    • Crossref
    • Export Citation
  • Pötscher, B. M. 1991. “Effects of Model Selection on Inference.” Econometric Theory 7 (2): 163–185.

    • Crossref
    • Export Citation
  • Raftery, A. E., D. Madigan, and J. A. Hoeting. 1997. “Bayesian Model Averaging for Linear Regression Models.” Journal of the American Statistical Association 92 (437): 179–191.

    • Crossref
    • Export Citation
  • Rau, T. 2011. “Bayesian Inference in the Regression Discontinuity Model.” Vigesimosextas Jornadas Anuales de Economa. http://eco3147.facea.puc.cl/economia_puc/images/stories/Instituto/Profesores/rd4.pdf (accessed February 17, 2015).

  • Schwarz, G. 1978. “Estimating the Dimension of a Model.” The Annals of Statistics 6 (2): 461–464.

    • Crossref
    • Export Citation
  • Sun, Y. 2005. “Adaptive Estimation of the Regression Discontinuity Model.” Unpublished Manuscript, University of California, San Diego. http://econweb.ucsd.edu/~yisun/ard.pdf (accessed February 17, 2015).

  • Wallace, T. D. 1977. “Pretest Estimation in Regression: A Survey.” American Journal of Agricultural Economics 59 (3): 431–443.

    • Crossref
    • Export Citation
  • Yuan, Z., and Y. Yang. 2005. “Combining Linear Regression Models.” Journal of the American Statistical Association 100 (472): 1202–1214.

    • Crossref
    • Export Citation
Purchase article
Get instant unlimited access to the article.
$42.00
Log in
Already have access? Please log in.


or
Log in with your institution

Journal + Issues

The Journal of Econometric Methods aims to bridge the widening gap between econometric research and empirical practice. It publishes papers from top scholars in all areas of econometrics that consider a topic of broad interest to practitioners and are written in a style that is targeted at practitioners.

Search