Skip to content
BY-NC-ND 3.0 license Open Access Published by De Gruyter Open Access April 12, 2016

Regularization for high-dimensional covariance matrix

  • Xiangzhao Cui , Chun Li , Jine Zhao , Li Zeng , Defei Zhang and Jianxin Pan
From the journal Special Matrices


In many applications, high-dimensional problem may occur often for various reasons, for example, when the number of variables under consideration is much bigger than the sample size, i.e., p >> n. For highdimensional data, the underlying structures of certain covariance matrix estimates are usually blurred due to substantial random noises, which is an obstacle to draw statistical inferences. In this paper, we propose a method to identify the underlying covariance structure by regularizing a given/estimated covariance matrix so that the noises can be filtered. By choosing an optimal structure from a class of candidate structures for the covariance matrix, the regularization is made in terms of minimizing Frobenius-norm discrepancy. The candidate class considered here includes the structures of order-1 moving average, compound symmetry, order-1 autoregressive and order-1 autoregressive moving average. Very intensive simulation studies are conducted to assess the performance of the proposed regularization method for very high-dimensional covariance problem. The simulation studies also show that the sample covariance matrix, although performs very badly in covariance estimation for high-dimensional data, can be used to correctly identify the underlying structure of the covariance matrix. The approach is also applied to real data analysis, which shows that the proposed regularization method works well in practice.


[1] A. Belloni, V. Chernozhukov and L. Wang. Square-root lasso: Pivotal recovery of sparse signals via conic programming. Biometrika, 98, (2012), 791-806. 10.1093/biomet/asr043Search in Google Scholar

[2] P. Bickel and E. Levina. Covariance regularization by thresholding. Ann Stat, 36, (2008), 2577-2604. 10.1214/08-AOS600Search in Google Scholar

[3] M. Buscema. MetaNet: The Theory of Independent Judges, Substance Use & Misuse, 33, (1998), 439-461. Search in Google Scholar

[4] T. Cai and W. Liu. Adaptive thresholding for sparse covariance estimation. J Am Stat Assoc, 106, (2011), 672-684. 10.1198/jasa.2011.tm10560Search in Google Scholar

[5] X. Cui, C. Li, J. Zhao, L. Zeng, D. Zhang and J. Pan. Covariance structure regularization via Frobenius-norm discrepancy. Revised for Linear Algebra Appl. (2015). 10.1016/j.laa.2016.08.013Search in Google Scholar

[6] X. Deng and K. Tsui. Penalized covariance matrix estimation using a matrix-logarithm transformation. J Comput Stat Graph, 22, (2013), 494-512. 10.1080/10618600.2012.715556Search in Google Scholar

[7] D. Donoho. Aide-Memoire. High-dimensional data analysis: The curses and blessings of dimensionality. American Mathematical Society. Available at, (2000). Search in Google Scholar

[8] N. El Karoui. Operator norm consistent estimation of large dimensional sparse covariance matrices. Ann Stat, 36, (2008), 2712-2756. 10.1214/07-AOS559Search in Google Scholar

[9] J. Fan, Y. Liao and M. Mincheva. Large covariance estimation by thresholding principal orthogonal complements. J Roy Stat Soc B, 75, (2013), 656-658. 10.1111/rssb.12016Search in Google Scholar PubMed PubMed Central

[10] L. Lin, N. J. Higham and J. Pan. Covariance structure regularization via entropy loss function. Computational Statistics & Data Analysis, 72, (2014), 315-327. 10.1016/j.csda.2013.10.004Search in Google Scholar

[11] A. Rothman. Positive definite estimators of large covariance matrices. Biometrika, 99, (2012), 733-740. 10.1093/biomet/ass025Search in Google Scholar

[12] A. Rothman, E. Levina and J. Zhu. Generalized thresholding of large covariance matrices. J. Am. Stat. Assoc., 104, (2009), 177-186. 10.1198/jasa.2009.0101Search in Google Scholar

[13] L. Xue, S. Ma and H. Zou. Positive definite `1 penalized estimation of large covariance matrices. J. Am. Stat. Assoc., 107, (2012), 1480-1491. 10.1080/01621459.2012.725386Search in Google Scholar

Received: 2015-10-9
Accepted: 2016-3-28
Published Online: 2016-4-12

©2016 Xiangzhao Cui et al.

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Downloaded on 11.12.2023 from
Scroll to top button