Large Covariance Estimation by Thresholding Principal Orthogonal Complements

Abstract

This paper deals with the estimation of a high-dimensional covariance with a conditional sparsity structure and fast-diverging eigenvalues. By assuming sparse error covariance matrix in an approximate factor model, we allow for the presence of some cross-sectional correlation even after taking out common but unobservable factors. We introduce the Principal Orthogonal complEment Thresholding (POET) method to explore such an approximate factor structure with sparsity. The POET estimator includes the sample covariance matrix, the factor-based covariance matrix (Fan, Fan, and Lv, 2008), the thresholding estimator (Bickel and Levina, 2008) and the adaptive thresholding estimator (Cai and Liu, 2011) as specific examples. We provide mathematical insights when the factor analysis is approximately the same as the principal component analysis for high-dimensional data. The rates of convergence of the sparse residual covariance matrix and the conditional sparse covariance matrix are studied under various norms. It is shown that the impact of estimating the unknown factors vanishes as the dimensionality increases. The uniform rates of convergence for the unobserved factors and their factor loadings are derived. The asymptotic results are also verified by extensive simulation studies. Finally, a real data application on portfolio allocation is presented.

Keywords: High-dimensionality; approximate factor model; cross-sectional correlation; diverging eigenvalues; low-rank matrix; principal components; sparse matrix; thresholding; unknown factors.

Figure 1

Minimum eigenvalue of ${\widehat{\mathbf{\sum }}}_{u,\widehat{K}}^{\mathcal{T}}(C)$ as a function of C for three choices of thresholding rules. The plot is based on the simulated data set in Section 6.2.

Figure 2

Averages (left panel) and standard deviations (right panel) of the relative error p^−1/2||Σ^−1/2Σ̂Σ^−1/2 − I_p||_F with known factors (Σ̂ = Σ̂_obs solid red curve), POET (Σ̂ = Σ̂_K̂ solid blue curve), and sample covariance (Σ̂ = Σ̂_sam dashed curve) over 200 simulations, as a function of the dimensionality p. Top panel: p ranges in 20 to 600 with increment 20; bottom panel: p ranges in 1 to 100 with increment 1.

Figure 3

Averages (left panel) and standard deviations (right panel) of ||Σ̂⁻¹ − Σ⁻¹|| with known factors (Σ̂ = Σ̂_obs solid red curve), POET (Σ̂ = Σ̂_K̂ solid blue curve), and sample covariance (Σ̂ = Σ̂_sam dashed curve) over 200 simulations, as a function of the dimensionality p. Top panel: p ranges in 20 to 600 with increment 20; middle panel: p ranges in 1 to 100 with increment 1; Bottom panel: the same as the top panel with dashed curve excluded.

Figure 4

Averages (left panel) and standard deviations (right panel) of ||Σ̂ − Σ||_max with known factors (Σ̂ = Σ̂_obs solid red curve), POET (Σ̂ = Σ̂_K̂ solid blue curve), and sample covariance (Σ̂ = Σ̂_sam dashed curve) over 200 simulations, as a function of the dimensionality p. They are nearly indifferentiable.

Figure 5

Averages of ||Σ̂ − Σ|| (left panel) and ||Σ^−1/2Σ̂Σ^−1/2 − I_p|| with known factors (Σ̂ = Σ̂_obs solid red curve), POET (Σ̂= Σ̂_K̂ ω solid blue curve), and sample covariance (Σ̂ = Σ̂_sam dashed curve) over 200 simulations, as a function of the dimensionality p. The three curves are hardly distinguishable on the left panel.

Figure 6

Robustness of K as p increases for various choices of K (Design 1, T = 300). Top left: || ${\widehat{\mathbf{\sum }}}_{u,K}^{\mathcal{T}}-{\mathbf{\sum }}_u$||; top right: || ${({\widehat{\mathbf{\sum }}}_{u,K}^{\mathcal{T}})}^{-1}-{\mathbf{\sum }}_u^{-1}$||; bottom left: ||Σ̂_K − Σ||_Σ; bottom right: || ${\widehat{\mathbf{\sum }}}_K^{-1}-{\mathbf{\sum }}^{-1}$||.

Figure 7

Box plots of regrets R(ŵ) − R^* for p = 80 and 140. In each panel, the box plots from left to right correspond to ŵ obtained using Σ̂ based on approximate factor model, strict factor model, and sample covariance, respectively.

Figure 8

Estimation errors for risk assessments as a function of the portfolio size p. Left panel plots the average absolute error |R(ŵ) − R̂ (ŵ)| and right panel depicts the average relative error |R̂ (ŵ)/R(ŵ) − 1|. Here, ŵ and R̂ are obtained based on three estimators of Σ̂.

Figure 9

Heatmap of thresholded error correlation matrix for number of factors K = 0, K = 1, K = 2 and K = 3.

Figure 10

Risk of portfolios created with POET and SFM (strict factor model)