Skip to content
Licensed Unlicensed Requires Authentication Published by De Gruyter July 21, 2018

Methods for Computing Numerical Standard Errors: Review and Application to Value-at-Risk Estimation

David Ardia EMAIL logo , Keven Bluteau and Lennart F. Hoogerheide


Numerical standard error (NSE) is an estimate of the standard deviation of a simulation result if the simulation experiment were to be repeated many times. We review standard methods for computing NSE and perform a Monte Carlo experiments to compare their performance in the case of high/extreme autocorrelation. In particular, we propose an application to risk management where we assess the precision of the value-at-risk measure when the underlying risk model is estimated by simulation-based methods. Overall, heteroscedasticity and autocorrelation estimators with prewhitening perform best in the presence of large/extreme autocorrelation.


All analyses have been performed in the R statistical language (Core Team 2015) with the package nse (Ardia and Bluteau 2017) available at We thank the Fonds de Recherche du Québec – Société et Culture and Industrielle–Alliance for their financial support. All errors and omissions are the sole responsibility of the authors.


Akaike, H. 1974. “A New Look at the Statistical Model Identification.” IEEE Transactions on Automatic Control 19: 716–23. doi:10.1007/978-1-4612-1694-0\_16.Search in Google Scholar

Andrews, D. W. K. 1991. “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation.” Econometrica 59: 817–58. doi:10.2307/2938229.Search in Google Scholar

Andrews, D. W. K., and J. C. Monahan. 1992. “An Improved Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimator.” Econometrica 60: 953–66. doi:10.2307/2951574.Search in Google Scholar

Ardia, D. 2008. Financial Risk Management with Bayesian Estimation of GARCH Models. Berlin/Heidelberg: Springer. doi:10.1007/978-3-540-78657-3.Search in Google Scholar

Ardia, D., and K. Bluteau. 2017. “nse: Computation of Numerical Standard Errors in R.” Journal of Open Source Software 10. doi:10.21105/joss.00172.Search in Google Scholar

Ardia, D., K. Bluteau, K. Boudt, and D.-A. Trottier. Markov-Switching GARCH Models in R: The MSGARCH package. Working paper 2016a Forthcoming in Journal of Statistical Software.10.2139/ssrn.2845809Search in Google Scholar

Ardia, D., K. Bluteau, K. Bout, B. Peterson, and D.-A. Trottier. 2016b. “MSGARCH: Markov Switching GARCH Models in R.” in Google Scholar

Flegal, J. M., and G. L. Jones. 2010. “Batch Means and Spectral Variance Estimators in Markov Chain Monte Carlo.” Annals of Statistics 38: 1034–70. doi:10.1214/09-aos735.Search in Google Scholar

Geyer, C. J. 1992. “Practical Markov Chain Monte Carlo.” Statistical Science 7: 473–83. doi:10.1214/ss/1177011137.Search in Google Scholar

Haas, M., S. Mittnik, and M. Paollela. 2004. “A New Approach to Markov-Switching GARCH Models.” Journal of Financial Econometrics 2: 493–530. doi:10.1093/jjfinec/nbh020.Search in Google Scholar

Heidelberger, P., and P. D. Welch. 1981. “A Spectral Method for Confidence Interval Generation and Run Length Control in Simulations.” Communications of the ACM 24: 233–45. doi:10.1145/358598.358630.Search in Google Scholar

Hirukawa, M. 2010. A Two-Stage Plug-in Bandwidth Selection and Its Implementation for Covariance Estimation.” Econometric Theory 26: 710–43. doi:10.1017/s0266466609990089.Search in Google Scholar

Hoogerheide, L., and H. K. van Dijk. 2010. “Bayesian Forecasting of Value at Risk and Expected Shortfall Using Adaptive Importance Sampling.” International Journal of Forecasting 26: 231–47. doi:10.1016/j.ijforecast.2010.01.007.Search in Google Scholar

Hurvich, C. M. 1985. “Data-Driven Choice of a Spectrum Estimate: Extending the Applicability of Cross-Validation Methods.” Journal of the American Statistical Association 80: 933–40. doi:10.1080/01621459.1985.10478207.Search in Google Scholar

Lahiri, S. N. 1999. “Theoretical Comparisons of Block Bootstrap Methods.” Annals of Statistics 27: 386–404. doi:10.1214/aos/1018031117.Search in Google Scholar

Newey, W. K., and K. D. West. 1987. “A Simple, Positive Semi-definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix.” Econometrica 55: 703–8. doi:10.2307/1913610.Search in Google Scholar

Newey, W. K., and K. D. West. 1994. “ Automatic Lag Selection in Covariance Matrix Estimation.” Review of Economic Studies 61: 631–53. doi:10.3386/t0144.Search in Google Scholar

Nordman, D. J. 2009. “A Note on the Stationary Bootstrap’s Variance.” Annals of Statistics 37: 359–70. doi:10.1214/07-aos567.Search in Google Scholar

Parzen, E. 1957. “On Consistent Estimates of the Spectrum of a Stationary Time Series.” The Annals of Mathematical Statistics 28: 329–48.10.1214/aoms/1177706962Search in Google Scholar

Patton, A., D. N. Politis, and H. White. 2009. “Correction to ‘Automatic Block-length Selection for the Dependent Bootstrap’ by D. Politis and H. White.” Econometric Reviews 28: 372–75. doi:10.1080/07474930802459016.Search in Google Scholar

Percival, D., and W. Constantine. 2006. “Exact Simulation of Gaussian Time Series from Nonparametric Spectral Estimates with Application to Bootstrapping.” Statistics and Computing 16: 25–35. doi:10.1007/s11222-006-5198-0.Search in Google Scholar

Politis, D. N., and J. P. Romano. 1992. “A Circular Block-Resampling Procedure for Stationary Data.” In Exploring the Limits of Bootstrap, 263–70. John Wiley & Sons.Search in Google Scholar

Politis, D. N., and J. P. Romano. 1994. “The Stationary Bootstrap.” Journal of the American Statistical Association 89: 1303–13. doi:10.2307/2290993.Search in Google Scholar

Politis, D. N., and H. White. 2004. “Automatic Block-Length Selection for the Dependent Bootstrap.” Econometric Reviews 23: 53–70. doi:10.1081/etc-120028836.Search in Google Scholar

Press, H., and J. Tukey. 1956. “Power Spectral Methods of Analysis and Application in Airplane Dynamics.” In Vol. IV of AGARD Flight Test Manual, 1–41. Paris, Ch. C.10.1016/B978-1-4831-9728-9.50036-9Search in Google Scholar

R Core Team. 2015. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. in Google Scholar

Vihola, M. 2012. “Robust Adaptive Metropolis Algorithm with Coerced Acceptance Rate.” Statistics and Computing 22: 997–1008. doi:10.1007/s11222-011-9269-5.Search in Google Scholar

Welch, P. D. 1967. “The Use of fast Fourier Transform for the Estimation of Power Spectra: A Method Based on Time Averaging Over Short, Modified Periodograms.” IEEE Transactions on Audio and Electroacoustics 15: 70–73. doi:10.1109/TAU.1967.1161901.Search in Google Scholar

Published Online: 2018-07-21

© 2018 Walter de Gruyter GmbH, Berlin/Boston

Downloaded on 2.12.2022 from
Scroll Up Arrow