Skip to content
Accessible Unlicensed Requires Authentication Published by De Gruyter November 6, 2015

A Simple Estimator for Dynamic Models with Serially Correlated Unobservables

Yingyao Hu, Matthew Shum, Wei Tan and Ruli Xiao


We present a method for estimating Markov dynamic models with unobserved state variables which can be serially correlated over time. We focus on the case where all the model variables have discrete support. Our estimator is simple to compute because it is noniterative, and involves only elementary matrix manipulations. Our estimation method is nonparametric, in that no parametric assumptions on the distributions of the unobserved state variables or the laws of motions of the state variables are required. Monte Carlo simulations show that the estimator performs well in practice, and we illustrate its use with a dataset of doctors’ prescription of pharmaceutical drugs.

Corresponding author: Yingyao Hu, Department of Economics, Johns Hopkins University, 440 Mergenthaler Hall, 3400 N. Charles Street, Baltimore, MD 21218, USA, E-mail:


We thank Wei Zhao for extraordinary research assistance.

Appendix Derivation of Auxiliary Results

A Derivation of Equation (5)

Consider the observed density f(Wt, Wt–1, Wt–2). Assumptions 1 and 2(i) imply


After integrating out Mt–2, assumption 2(ii) then implies


The expression in the parenthesis can be simplified as f(Yt|Mt,Mt1,Xt1). We then have


as claimed in equation (5).■

B Proof of Claim (*)



Identification of H is equivalent to identification of the h(…) function.

By integrating h(k, j; mt, mt–1) over mt, we can identify the f(xt1=k|mt1,yt2=j) function:


because f(mt|mt1,xt1) is a probability density function. Consequently, f(mt|mt1,xt1) is also identified as


Hence, from knowledge of H, we are able to identify the function corresponding to the matrices D2 and C also.■


Abbring, J., and J. Heckman. 2007. “Econometric Evaluation of Social Programs, Part III: Distributional Treatment Effects, Dynamic Treatment Effects, Dynamic Discrete Choice, and General Equilibrium Policy Evaluation.” In Handbook of Econometrics, Vol. 6B, edited by by J. Heckman and E. Leamer, chap. 72. Amsterdam: Elsevier.Search in Google Scholar

Aguirregabiria, V., and P. Mira. 2002. “Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models.” Econometrica 70: 1519–1543.Search in Google Scholar

Aguirregabiria, V., and P. Mira. 2007. “Sequential Estimation of Dynamic Discrete Games.” Econometrica 75: 1–53.Search in Google Scholar

Andrew, A., K.-W. Chu, and P. Lancaster. 1993. “Derivatives of Eigenvalues and Eigenvectors of Matrix Functions.” SIAM Journal on Matrix Analysis and Applications 14 (4): 903–926.Search in Google Scholar

Arcidiacono, P., and R. Miller. 2011. “Conditional Choice Probability Estimation of Dynamic Discrete Choice Models with Unobserved Heterogeneity.” Econometrica 79: 1823–1867.Search in Google Scholar

Bajari, P., L. Benkard, and J. Levin. 2007. “Estimating Dynamic Models of Imperfect Competition.” Econometrica 75: 1331–1370.Search in Google Scholar

Bajari, P., V. Chernozhukov, H. Hong, and D. Nekipelov. 2007. “Nonparametric and Semiparametric Analysis of a Dynamic Game Model.” Manuscript, University of Minnesota.Search in Google Scholar

Blevins, J. forthcoming. “Sequential Monte Carlo Methods for Estimating Dynamic Microeconomic Models.” Journal of Applied Econometrics.Search in Google Scholar

Connault, B. 2014. Hidden Rust Models. Priceton University, Working Paper.Search in Google Scholar

Cunha, F., J. Heckman, and S. Schennach. 2006. “Estimating the Technology of Cognitive and Noncognitive Skill Formation.” Econometrica 78: 883–931.Search in Google Scholar

Food and Drug Administration. 2003, 2005. “Labels for Crestor.” .Search in Google Scholar

Gallant, R., H. Hong, and A. Khwaja. 2009. “Estimating a Dynamic Oligopolistic Game with Serially Correlated Unobserved Production Costs.” manuscript, Duke University.Search in Google Scholar

Ghahramani, Z. 2001. “An Introduction to Hidden Markov Models and Bayesian Networks.” International Journal of Pattern Recognition and Artificial Intelligence 15: 9–42.Search in Google Scholar

Hansen, L. 2014: “Nobel Lecture: Uncertainty Outside and Inside Economic Models.” Journal of Political Economy 122: 945–987.Search in Google Scholar

Heckman, J., and S. Navarro. 2007. “Dynamic Discrete Choice and Dynamic Treatment Effects.” Journal of Econometrics 136: 341–396.Search in Google Scholar

Hong, H., and M. Shum. 2010. “Pairwise-Difference Estimation of a Dynamic Optimization Model.” Review of Economic Studies 77: 273–304.Search in Google Scholar

Hotz, J., and R. Miller. 1993. “Conditional Choice Probabilties and the Estimation of Dynamic Models.” Review of Economic Studies 60: 497–529.Search in Google Scholar

Hu, Y. 2008. “Identification and Estimation of Nonlinear Models with Misclassification Error Using Instrumental Variables: A General Solution.” Journal of Econometrics 144: 27–61.Search in Google Scholar

Hu, Y., and M. Shum. 2012. “Nonparametric Identification of Dynamic Models with Unobserved State Variables.” Journal of Econometrics 171: 32–44.Search in Google Scholar

Imai, S., N. Jain, and A. Ching. 2009. “Bayesian Estimation of Dynamic Discrete Choice Models.” Econometrica 77: 1865–1899.Search in Google Scholar

Kasahara, H., and K. Shimotsu. 2009. “Nonparametric Identification of Finite Mixture Models of Dynamic Discrete Choice.” Econometrica 77: 135–175.Search in Google Scholar

Keane, M., and K. Wolpin. 1994. “The Solution and Estimation of Discrete Choice Dynamic Programming Models by Simulation and Interpolation: Monte Carlo Evidence.” Review of Economics and Statistics 76: 648–672.Search in Google Scholar

Magnac, T., and D. Thesmar. 2002. “Identifying Dynamic Discrete Decision Processes.” Econometrica 70: 801–816.Search in Google Scholar

Manchanda, P., and S. Narayanan. 2009. “Heterogeneous Learning and the Targeting of Marketing Communication for New Products.” Marketing Science 28: 424–441.Search in Google Scholar

Miller, R. 1984. “Job Matching and Occupational Choice.” Journal of Political Economy 92: 1086–1120.Search in Google Scholar

Norets, A. 2009. “Inference in Dynamic Discrete Choice Models with Serially Correlated Unobserved State Variables.” Econometrica 77: 1665–1682.Search in Google Scholar

Pakes, A. 1986. “Patents as Options: Some Estimates of the Value of Holding European Patent Stocks.” Econometrica 54 (4): 755–784.Search in Google Scholar

Pakes, A., M. Ostrovsky, and S. Berry. 2007. “Simple Estimators for the Parameters of Discrete Dynamic Games (with Entry Exit Examples).” RAND Journal of Economics 38: 373–399.Search in Google Scholar

Pesendorfer, M., and P. Schmidt-Dengler. 2008. “Asymptotic Least Squares Estimators for Dynamic Games.” Review of Economic Studies 75: 901–928.Search in Google Scholar

Rust, J. 1987. “Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher.” Econometrica 55: 999–1033.Search in Google Scholar

Rust, J. 1994. “Structural Estimation of Markov Decision Processes.” In Handbook of Econometrics, Vol. 4, edited by R. Engle and D. McFadden, 3082–3146. Amsterdam: Elsevier.Search in Google Scholar

Shum, M., and W. Tan. 2007. “Is Advertising Informative? Evidence from Contraindicated Drug Prescriptions.” Working paper.Search in Google Scholar

Siebert, R., and C. Zulehner. 2008. “The Impact of Market Demand and Innovation on Market Structure.” Purdue University, Working Paper.Search in Google Scholar

Supplemental Material:

The online version of this article (DOI: 10.1515/jem-2015-0011) offers supplementary material, available to authorized users.

Published Online: 2015-11-6
Published in Print: 2017-1-1

©2017 Walter de Gruyter GmbH, Berlin/Boston