An Information Theoretic Approach for Estimating Nonlinear Dynamic Models

Amos Golan 1 , 1
  • 1 American University,

Given the objective of estimating the unknown parameters of a possibly nonlinear dynamic model using a finite (and relatively small) data set, it is common to use a Kalman filter Maximum Likelihood (ML) approach, ML-type estimators or more recently a GMM (Imbens, Spady and Johnson, 1998), BMOM (Zellner 1997), or other information theoretic estimators (e.g., Golan, Judge and Miller, 1996). Except for the BMOM, the above ML-type methods require some distributional assumptions while the moment-type estimators require some assumptions on the moments of the underlying distribution that generated the data. In the BMOM approach however, sampling assumptions underlying most ML and other approaches are not employed for the given data. The error terms are viewed as parameters with unknown values.Based on a generalization of the Maximum Entropy (ME), a semi-parametric, Information-Theoretic (IT) framework for estimating dynamic models with minimal distributional assumptions is formulated here. Like the BMOM approach, under this formulation, one views the errors as another set of unknown parameters to be estimated. Thus, for any data set, the estimation problem is ill-posed (under-determined) where the number of unknowns is always greater than the number of data points. The Information-Theoretic approach is one way to estimate the unknown parameters.After developing the basic IT (entropy) model, a computationally efficient concentrated model is developed where the optimization is done with respect to the Lagrange multipliers associated with each observation. The dual concentrated model is used to contrast this IT approach with the more traditional ML-type estimators. Statistics and inference procedures are developed as well. Monte Carlo results for estimating the parameters of noisy, chaotic systems are presented.

    • Program code
Purchase article
Get instant unlimited access to the article.
Log in
Already have access? Please log in.

Log in with your institution

Journal + Issues

SNDE recognizes that advances in statistics and dynamical systems theory can increase our understanding of economic and financial markets. The journal seeks both theoretical and applied papers that characterize and motivate nonlinear phenomena. Researchers are required to assist replication of empirical results by providing copies of data and programs online. Algorithms and rapid communications are also published.