On the spectrum of noisy blown-up matrices

Abstract We study the eigenvalues of large perturbed matrices. We consider a pattern matrix P, we blow it up to get a large block-matrix Bn. We can observe only a noisy version of matrix Bn. So we add a random noise Wn to obtain the perturbed matrix An = Bn + Wn. Our aim is to find the structural eigenvalues of An. We prove asymptotic theorems on this problem and also suggest a graphical method to distinguish the structural and the non-structural eigenvalues of An.


Introduction
Spectral theory of random matrices has a long history (see e.g. [3], [7], [11], [12], [13] and the references therein). There are papers where the spectrum of a perturbed random matrix is studied (e.g. [11]). However, in this paper our point of view is the opposite, the random matrix is considered as a perturbation of a deterministic matrix.
Our aim is to extend the results of Bolla (see [4] and [5]). Therefore we consider a xed deterministic pattern matrix P, blow it up to obtain a 'large' block-matrix Bn and add a random noise Wn, say. We prove limit theorems for the eigenvalues of An = Bn + Wn as n → ∞. Our results are extensions of the results of Bolla [4] and [5] as we consider both real and complex matrices, we use several types of noise matrices, moreover we apply novel limit theorems for the random matrices. We also present numerical results and o er a graphical method to distinguish the structural and the non-structural eigenvalues.
In Section 2 and 3 we consider noise matrices with entries having zero mean values. Our results for symmetric matrices are presented in Section 2. Proposition 2.1 describes the eigenvalues of a Hermitian blown-up matrix. In Theorem 2.1 and Corollary 2.1 complex Hermitian blown-up matrices are perturbed by complex Wigner matrices. Using a known result on the eigenvalues of Wigner matrices, we prove that the structural eigenvalues of the perturbed blown-up matrix are of order n, while the eigenvalues 'generated' by the random noise are of order √ n almost surely. In Theorem 2.2 and Corollary 2.2 we obtain the above mentioned results for the case of perturbation with complex sample covariance matrices. In Theorem 2.3 and Corollary 2.3 real symmetric blown-up matrices are perturbed by real elliptic matrices. Applying the result of [11] on the eigenvalues of elliptic matrices, we prove that the structural eigenvalues of the perturbed blown-up matrix are of order n, while the other eigenvalues are of order √ n log n almost surely. In Section 3 non-symmetric complex matrices are considered. In Theorem 3.1 and Corollary 3.1 the asymptotic behaviour of the singular values of complex blown-up matrices perturbed by complex i.i.d. valued matrices are described.
In Section 4 we study random matrices with entries having positive mean values. Our Theorem 4.1 states that the maximal eigenvalue is 'large' when the entries of the matrix satisfy certain acceptability condition.
Our result is an extension of Theorem 4.1 in [4], where symmetric random matrices having independent entries above the diagonal were considered. To prove our Theorem 4.1, we use a new Bernstein type inequality for acceptable random variables (Proposition 4.1).
In Section 5 we present numerical results. In most cases the computer experiments support our theorems. The typical behaviour of the eigenvalues of the perturbed blown-up matrix is the following. Let |λ | ≥ |λ | ≥ . . . be the absolute values of the eigenvalues of the perturbed blown-up matrix in descending order. Then the structural eigenvalues |λ | ≥ |λ | ≥ · · · ≥ |λ k | are 'large' (and they rapidly decrease). The other eigenvalues |λ k+ | ≥ |λ k+ | ≥ . . . are relatively small and they decrease very slowly. So it is easy to nd the structural eigenvalues. To obtain the structural eigenvalues we suggest the following numerical (graphical) procedure. Calculate some eigenvalues of An starting with the largest ones in absolute value. Stop when the last 5-10 eigenvalues are close to zero and they are almost the same in absolute value. Then we obtain the increasing sequence |λ t | ≤ |λ t− | ≤ · · · ≤ |λ |. Plot their values in the above order, then nd the rst abrupt change. If, say, that is the abrupt change is at l, then λ l , λ l− , . . . , λ can be considered as the structural eigenvalues. However, there are more or less obvious exceptional cases. When the signal-noise ratio is too small or the magnitudes of the block sizes are diverse or the smallest non-zero eigenvalues of Bn are approximately zero, then our method does not give the precise border. In these cases usually a few of the structural eigenvalues behave like the non-structural ones. We shall denote by λ l (A) the l'th largest eigenvalue of the matrix A.z denotes the complex conjugate of the number z. For a random variable ξ , Eξ and Dξ stand for the expectation and the variance, respectively.

Eigenvalues of perturbed symmetric matrices
In this section we study the perturbations of Hermitian (resp. symmetric) blown-up matrices. We are interested in the eigenvalues of matrices perturbed by certain random matrices.
Let P be a symmetric pattern matrix, that is a xed complex Hermitian (in the real valued case symmetric) k × k matrix of rank r. Denote by p ij the (i, j)'th entry of P. Let n , . . . , n k be positive integers, n = n + · · · + n k . LetBn be an n × n matrix consisting of k blocks. Its block (i, j) is of size n i × n j and all elements in that block are equal to p ij . A matrix Bn is called blown-up matrix if it can be obtained fromBn by rearranging its rows and columns using the same permutation.
Following [4], we shall use the growth rate condition n → ∞ so that n i /n ≥ c for all i, Proof. First we remark that the eigenvalues of Bn are real and there is an orthonormal basis of the ndimensional space consisting of eigenvectors of Bn. Let β j denote the eigenvalues and u j (j = , . . . , n) the orthonormal eigenvectors of Bn. Suppose that β , . . . , βr are the non-zero eigenvalues. Using the ideas of [4], we can see the following. Let u and β denote a generic eigenvector-eigenvalue pair of Bn. Then for each l = , , . . . , k, n l coordinates of u are equal to u(l), say. Letũ denote the k-dimensional vector having coordinates u( ), . . . , u(k) and let N denote the diagonal matrix with diagonal elements n , . . . , n k . So the eigen-value equation Bnu = βu is equivalent to PNũ = βũ. Moreover, this equation is equivalent to where v = N / ũ. Finally, this equation is equivalent tõ We see that the vectors v , . . . , vr are orthonormal eigenvectors ofÑ / PÑ / and the eigenvalues are β /n, . . . , βr /n. Now we apply the Courant-Fischer-Weyl min-max principle. First assume that the l'th eigenvalue of P is positive, that is λ l (P) > . Then for any k − l + dimensional subspace H. AsÑ / is a non-degenerated matrix, we obtain On the other hand for any l dimensional subspace H. Again, we obtain As λ l (Ñ / PÑ / ) = β l /n, we obtain the result for positive λ l (P). If λ l (P) < , similar considerations lead to the result.
For simplicity, we assume that the rank of P is k. We shall consider the eigenvalues in descending order, so we have λ (Bn) ≥ · · · ≥ λn(Bn). We know that k out of the eigenvalues of Bn are non-zero and the remaining ones are equal to zero. The k non-zero eigenvalues are called structural eigenvalues of Bn. Similarly, we shall call structural eigenvalues those eigenvalues of the perturbed blown-up matrix which correspond to the structural eigenvalues of Bn. This correspondence will be described by our forthcoming theorems. We shall see that the magnitude of any structural eigenvalue is large and it is small for the other eigenvalues. First we consider perturbations by Wigner matrices. Next theorem is a generalization of Theorem 2.3 of [4] where the real valued case and uniformly bounded perturbations were considered. In our theorem both Bn and Wn are complex Hermitian matrices (in particular, symmetric real matrices). The perturbation Wn is a Wigner matrix with entries having nite 4th moment.
for j = , . . . , n as n → ∞ holds with probability tending to (where σ is an upper bound for the variances of the w ij random variables). To this end a result of [7] was applied. In the next remark we use Theorem 1.3 of [13], to improve the above remainder term.
Remark 2.1. Let Bn, n = , , . . . , be a sequence of real symmetric matrices. Let the Wigner matrices Wn, n = , , . . . , be real symmetric random matrices satisfying the following assumptions. Let w ij , ≤ i ≤ j ≤ n, be independent (but not necessarily identically distributed). Assume that for j = , . . . , n holds with probability tending to as n → ∞.
Now we turn to perturbations by sample covariance matrices. The limiting behaviour of the eigenvalues of sample covariance matrices was rst studied by Marchenko and Pastur [10]. Let x jl , j, l = , , . . . , be an in nite array of independent and identically distributed complex valued random variables with mean and variance σ . Let X = (x jl ) n, m j= , l= be the left upper block of size n×m. Sn = m XX * is called the sample covariance matrix.
Theorem 2.2. Let Bn, n = , , . . . , be a sequence of complex Hermitian matrices. Let Sn, n = , , . . . , be complex valued sample covariance matrices satisfying the above conditions. Moreover, assume that the entries of X have nite fourth moments. Assume that limn→∞ n/m = y ∈ ( , ∞). Then for all i almost surely.
Proof. By Theorem 2.16 of [3], the largest eigenvalue of Sn converges to σ ( + √ y) almost surely. Now, Weyl's perturbation theorem implies the result.  for the zero eigenvalues of Bn, almost surely as n → ∞. This result is a simple consequence of Proposition 2.1 and the result of [12].
Now we turn to elliptic perturbations. It can be considered as a common generalization of Wigner type and i.i.d. type perturbations. We shall use condition (C0) given in [11]. The sequence of real random matrices Yn, n = , , . . . satisfy condition (C0) if the following properties hold true. Let (ξ , ξ ) be real random variables (so called atom variables) both of them have mean zero, unit variance and nite fourth moment. Let y ij , i, j = , , . . . be a double array of real random variables such that for ≤ i < j the random vectors (y ij , y ji ) are independent copies of (ξ , ξ ),  [9] we have the following. If β ≥ β ≥ · · · ≥ βn are the eigenvalues of the complex Hermitian matrix B and (λ + iµ ), . . . , (λn + iµn) are the eigenvalues of the perturbed matrix B + Y so that λ ≥ λ ≥ · · · ≥ λn, then

Singular values of perturbed matrices
In this section we study the perturbations of arbitrary blown-up matrices. We are interested in the singular values of matrices perturbed by certain random matrices. Let P be a pattern matrix, that is a xed complex a × b matrix of rank r. Denote by p ij the (i, j)'th entry of P. Let m , . . . , ma and n , . . . , n b be positive integers, m = m + · · · + ma, n = n + · · · + n b . LetB be an m × n matrix consisting of mn blocks. Its block (i, j) is of size m i × n j and all elements in that block are equal to p ij . A matrix B is called blown-up matrix if it can be obtained fromB by rearranging its rows and columns.
Following [5], we shall use the following growth rate condition m, n → ∞ so that m i /m ≥ c and n i /n ≥ d for all i, (3.6) where c, d > are xed constants. The following proposition is an extension of Proposition 6 of [5] to the complex valued case. The proof is similar, so we omit it.
as m, n → ∞ almost surely.
Proof. Let S = Sn = n XX * be the sample covariance matrix. By Theorem 2.16 of [3], if limn→∞ m/n = y ∈ ( , ∞), then the largest eigenvalue of Sn converges to σ ( + √ y) almost surely. Therefore the largest singular value of √ n X is bounded from above. Now, by Weyl's perturbation theorem, |s i − z i | ≤ X , where X is the spectral norm, that is the largest singular value of X. This implies the result.

Eigenvalues of random matrices having entries with non-zero mean values
In the previous section we considered perturbations with random matrices having zero mean entries. Now we shall show that under general conditions the eigenvalues are 'large' if the mean values are positive. Our theorem is an extension of the result of [4] where the case of independent entries were studied.
First we shall obtain a general Bernstein type inequality. For the sequence of real r.v.'s η , η , . . . , ηn we shall consider the condition If condition (4.1) is satis ed for all t ∈ R, then we are at the notion of acceptable r.v.'s, see [1]. It is known that negatively orthant dependent, negatively associated and independent random variables are acceptable. If (4.1) is true for η , η , . . . , ηn, then it remains true for η − a , η − a , . . . , ηn − an for any real numbers a , . . . , an, in particular for η − Eη , η − Eη , . . . , ηn − Eηn. Several inequalities which are known for the independent case can be easily transferred to acceptable random variables (see e.g. [6]). Now we start with a well-known lemma. for any t.
Proof. We follow the usual method. By Markov's inequality, (4.1) and Lemma 4.1, we have for t ≥ Let t = C log + εC σ . Then the above expression is equal to Proof. According to the Perron-Frobenius theorem a non-negative square matrix has an eigenvalue which is not less than the smallest row sum. Therefore where S i = n k= x ik . Now, using Proposition 4.1 and the condition < ε < δ ≤ ∆ ≤ / , we obtain for the complementary event where γ > . So we can apply the Borel-Cantelli lemma to get that λmax(Xn) < c n ε+ / is satis ed only nitely often with probability 1.

Numerical results
The simulations were performed in The Julia Programming Language [8].
Example 5.1. Our rst example supports Theorem 2.1 and Corollary 2.1 in the real valued case. Let P be the following real symmetric pattern matrix Here the size k = , the rank r = , so An has structural eigenvalues. To obtain Bn we blow up the matrix P using block sizes n , . . . , n as follows , , , Then let Wn be a real symmetric Wigner matrix with elements w ij such that w ij are i.i.d. standard normal for i ≤ j. We generated times Wn and calculated the largest eigenvalues of An = Bn + Wn (here largest means largest in absolute value). The results are the following.  In the second column we listed the eigenvalues of Bn, in the third column a typical outcome (that is at a xed experiment) for the eigenvalues of An, in the fourth (resp. fth) column the averages (resp. variances) of the absolute values of the eigenvalues of An using the repetitions of our experiment. We see that the variances are small, that is the experiment is stable. One can distinguish the structural and the non-structural eigenvalues because the absolute values of the non-structural eigenvalues are small and they increase very slowly, but the structural eigenvalues increase rapidly, and there is an abrupt change of the increase at the non-structural-structural border (see Figure 1, where a typical realization is presented). On the left hand side of Figure 1 we showed |λ (An)| < · · · < |λ (An)|. Here one can see that the values of |λ (An)|, . . . , |λ (An)| are almost the same, and one can guess that there is a change at |λ (An)|. On the right hand side of Figure 1 we showed |λ (An)|, . . . , |λ (An)|. Here (because of the new scale on the vertical axis) the abrupt change is clearly seen at |λ (An)|. So the structural eigenvalues are |λ (An)|, . . . , |λ (An)|. indistinguishable from the non-structural ones. Nevertheless, the numerical results show, that there is an abrupt change at the non-structural-structural border.
Here P is a real, positive, symmetric × pattern matrix of rank , the magnitudes of its elements are between and . The eigenvalues of P are . . . . . . -. -.
-. . We blow up P using block sizes , , , . The non-zero eigenvalues of the obtained blown-up matrix Bn are the following ---We see that the last eigenvalue is relatively small for P and therefore for Bn, too. The perturbation is again a real symmetric Wigner matrix Wn with elements w ij such that w ij are i.i.d. standard normal for i ≤ j. We calculated the rst eigenvalues of the perturbed matrix An = Bn+Wn. We performed again repetitions of the experiment. The variances of the eigenvalues were again small, so the experiment was stable. To obtain Bn we blow up the matrix P using the same block sizes as in Example 5.1. The sample covariance matrix Sn was generated by using real valued i.i.d. standard normal sample. We generated times Sn and calculated the largest eigenvalues of An = Bn + Sn (here largest means largest in absolute value). The results were stable, that is the variances were very small. The means of the 20 largest absolute values among the eigenvalues of An were . .
. So we can distinguish the structural and the non-structural eigenvalues. On Figure 3 a typical realization is presented. On the left hand side of the gure we showed |λ (An)| < · · · < |λ (An)|. On the right hand side of the gure we showed |λ (An)|, . . . , |λ (An)|. Here the abrupt change is clearly seen at |λ (An)|. So we can decide that the structural eigenvalues are λ (An), . . . , λ (An). Example 5.4. This example supports Theorem 3.1 and Corollary 3.1. Let P be the following × real nonsymmetric pattern matrix  To obtain Bn we blow up the matrix P using block sizes , , , The  . Here the smallest singular values seem to be far from . However, the small values of the pattern matrix caused low signal-noise ratio, and so the two smallest out of the structural singular values of An became almost indistinguishable from the non-structural ones as the following data and gures show.  There is an abrupt change after the second smallest structural singular value.
Example 5.6. This example presents numerical results for Theorem 2.1 and Corollary 2.1 in the complex valued case. It shows the e ect of the di erent block sizes on the eigenvalues of the noisy blown up matrices. We choose P as a × complex Hermitian matrix with rank r = , having the following eigenvalues . To create Bn, we blow up the matrix P by using the method described in Section 2. Wn is a Wigner matrix satisfying the assumptions in Theorem 2.1 and having normally distributed elements, more precisely w ii is real standard normal, and w ij is complex standard normal if i ≠ j. Then An = Bn + Wn. We studied 3 di erent block sizes. Each case we made 1000 repetitions.  . .
One can see, that the 10 structural eigenvalues are well distinguishable from the non-structural ones.   Figure 8 show that the structural eigenvalue with the smallest absolute value is getting closer to the non-structural ones, but it is still well distinguishable from the non-structural ones. This example showed that changes in the values of n , . . . , n , can cause considerable changes in the eigenvalues of the noisy blown up matrices, but the structural eigenvalues remain well distinguishable. So our method is reliable for a wide range of block sizes.
Example 5.7. In this example we show some numerical results for Theorem 2.3 and Corollary 2.3. Let P be the following matrix: (a) In the rst case the block sizes were: n = n = · · · = n = . We repeated the experiment 1000 times. After calculating the eigenvalues of the perturbed matrix, we got the following results: . . Figure 9 shows the result of one xed experiment: (b) In this case the pattern matrix P and the method to generate Yn were not changed, but we used diverse block sizes, n = n = , n = · · · = n = , n = n = . Now the eigenvalues of An in a typical realization were the following: .
. . . Figure 10 shows the graphical visualization of the eigenvalues.  Figure 9: Left side: |λ (An)| < · · · < |λ (An)|; right side: |λ (An)| < · · · < |λ (An)| in a typical realization in case (a) of Example 5.7. (c) In this case the pattern matrix P and the method we used to generate Yn were not changed compared to the previous cases, the only change is in the sizes of the blocks. Now, the block sizes are the following: n = · · · = n = , n = witch means, that in the blown-up matrix the rst nine rows of blocks contain nine blocks of size × and one block of size × , the last row contains nine blocks of size × and one block of size × . The result of this case is presented in the following table and Figure 11. j λ j (Bn) λ j (An) mean(|λ j (An)|) var(|λ j (An)|   Figure 11: Left side: |λ (An)| < · · · < |λ (An)|; right side: |λ (An)| < · · · < |λ (An)| in a typical realization in case (c) of Example 5.7.
This example showed that changes in the values of n , . . . , n , can cause changes in the eigenvalues of the noisy blown up matrices, but the structural eigenvalues remain well distinguishable. So our method is reliable for a wide range of block sizes.

Conclusion
In the case when the perturbation matrix has zero mean entries, then our theoretical results show that the structural eigenvalues are 'large' and the non-structural ones are 'small'. Our numerical results give insight into behaviour of the sequence of eigenvalues. Let |λ | ≥ |λ | ≥ . . . be the absolute values of the eigenvalues of the perturbed blown-up matrix in descending order. Then the structural eigenvalues |λ | ≥ |λ | ≥ · · · ≥ |λ l | are 'large' and they rapidly decrease. The other eigenvalues |λ l+ | ≥ |λ l+ | ≥ . . . are relatively small and they decrease very slowly. So it is easy to nd the structural eigenvalues. To obtain the structural eigenvalues we suggest the following numerical (graphical) procedure. Calculate some eigenvalues of An starting with the largest ones in absolute value. Stop when the last 5-10 eigenvalues are close to zero and they are almost the same in absolute value. Then we obtain the increasing sequence |λ t | ≤ |λ t− | ≤ · · · ≤ |λ |. Plot their values in the above order, then nd the rst abrupt change. If, say, ≈ |λ t | ≈ |λ t− | ≈ · · · ≈ |λ l+ | |λ l | < · · · < |λ |, that is the abrupt change is at l, then λ l , λ l− , . . . , λ can be considered as the structural eigenvalues. The typical abrupt change after the non-structural eigenvalues can be seen on Figure 12. Similar method is valid for the singular values, too.
Our results are stable in the sense that variances are small and the behaviour of the eigenvalues is the same in wide range of conditions. Our method does not work only in extreme circumstances (the signal-noise ratio is too small, the smallest eigenvalue of the pattern matrix is approximately zero, or the block sizes are very diverse).