Non-unitary CMV-decomposition

Abstract An important decomposition for unitary matrices, the CMV-decomposition, is extended to general non-unitary matrices. This relates to short recurrence relations constructing biorthogonal bases for a particular pair of extended Krylov subspaces.


Introduction
A Krylov subspace [7] is a subspace constructed by repeatedly multiplying a given matrix A ∈ C m×m with some vector h ∈ C m , i.e., span{h, Ah, A h, . . . }. Extended Krylov subspaces [4] generalize this concept by allowing multiplication with the inverse of A as well in the construction of the subspace. In this manuscript short pairs of recurrence relations are derived which construct biorthogonal bases for a pair of particular extended Krylov subspaces. These subspaces are constructed by multiplication with A and its inverse alternately. Consider (possibly) distinct vectors v, w ∈ C m , then the pair of extended Krylov subspaces considered is We note that when K has a positive (negative) power of A, S has a negative (positive) power of A H . The subspace used for the CMV-decomposition [2,8] is formed in the same way. Watkins [10] showed that, for a unitary matrix and normalized v = w, an orthogonal basis can be constructed by a short recurrence relation. This orthogonal basis spans K(A, v) and S(A H , v) simultaneously. He also discussed the link to orthogonal Laurent polynomials, quadrature formulas, Szegő polynomials and, Toeplitz matrices. Here the restriction to unitary matrices is dropped. Short recurrence relations are given to construct biorthogonal bases for the involved Krylov subspaces. The connection between extended Krylov subspaces and Laurent polynomials implies short recurrence relations for biorthogonal Laurent polynomials [11]. The latter is shown by making use of the moment matrix related to the pair of extended Krylov subspaces. The link to biorthogonal Szegő polynomials [1] and quadrature rules is not discussed. Section 2 derives the main result of this manuscript, a pair of short recurrence relations to construct biorthogonal bases for the pair of extended Krylov subspaces. A sparse, factored matrix representation of these recurrence relations is also given, to reveal the oblique projection of A onto the subspaces. Using the moment matrix the equivalent result for Laurent polynomials is derived. The section concludes with providing the link to the paper of Watkins [10] and the CMV-decomposition. Section 3 discusses the numerical properties of the proposed recurrence relations. This discussion is limited to a proof of concept. It veri es that the recurrence relations are valid. An indication of the stability of the methods is provided. A detailed study of the numerical properties of the methods is the topic of future research.

Short recurrence relations
Consider a nonunitary, nonsingular matrix A ∈ C m×m , vectors v, w ∈ C m and the pair of subspaces K(A, v) (1) and S(A H , w) (2). The goal is to construct biorthogonal bases V l and W l for the nite extended Krylov subspaces K l (A, v) and S l (A H , w), respectively. These subspaces are, for a nonnegative integer k: For even l = k, For odd l = k + , The biorthogonal bases are formed by the columns of the matrices The columns of these matrices, i.e., the basis vectors v i and w i , must satisfy the conditions -W H l V l = I, called the biorthogonality conditions.
The core idea in the iterative construction of nested bases for Krylov subspaces is to start from already known bases V l− and W l− for subspaces K l− (A, v) and S l− (A H , w) and construct V l and W l for subspaces K l (A, v) and S l (A H , w) satisfying all aforementioned conditions. The bases V l , W l are obtained by computing the basis vectors v l and w l such that . The outset of this manuscript is to compute these basis vectors e ciently by orthogonalizing with respect to a small amount of basis vectors instead of all of them. For simplicity we assume that no breakdowns occur, neither lucky nor serious. This no-breakdown assumption implies that v l , w l ≠ and i.e., strict subsets. Note that this also implies that l < m. The no-breakdown assumption is not a real restriction, since the results presented here are valid up to the occurrence of a breakdown. Our analysis will use the Euclidean inner product x, y := y H x. Lemma 1 provides a classical identity on which the analysis relies heavily.
Lemma 1 (Inner product property). Consider a nonsingular matrix A ∈ C m×m , two vectors x, y ∈ C m and the Euclidean inner product ., . , then the following equation holds x, y = Ax, A −H y .
In numerical methods it is of interest to obtain the projection of the given matrix A onto the subspaces. Such projections should be of smaller size than A and capture as much of the relevant information present in A as possible, e.g., model order reduction [5] and approximation of matrix functions [4] rely on this concept. These projections are given by the matrix of recurrence coe cients and exhibit a particular structure. The discussion here is restricted to projection of A onto K(A, v) and orthogonal to S(A H , w). Its dual. projection onto S(A H , w) and orthogonal to K(A, v), will turn out to be similar and is therefore omitted.

. Four-term recurrence relation
A short (four-term) recurrence relation is derived for the biorthogonal bases V l (3) and W l (4). This derivation extends the results of Watkins [10], in the sense that it is not restricted to the case where A is a unitary matrix. Lemma 1 and the orthogonality properties of V l and W l are the key to obtain the short recurrence relations in Theorem 1.

Theorem 1.
Let A ∈ C m×m be a nonsingular matrix and v, w ∈ C m , then biorthogonal bases V l ∈ C m×(l+ ) (3) and W l ∈ C m×(l+ ) (4) can be constructed by four-term recurrence relations.
where α i, k = Av k− , w i and β i, where α i, k+ = A − v k , w i and β i, k+ = A H w k , v i . Normalization is done by choosing η l,l− and ν l,l− such that v l , w l = . This is assumed to always be possible, i.e., the assumption that breakdowns do not occur.
Proof. Recurrence relations (5) and (7) are proven. The proof of (6) and (8) is analogous. Assume, without loss of generality, that l = k. The next basis vector v k+ must be constructed such that it expands K k (A, v) to K k+ (A, v), i.e., introduces a component along the direction A k+ v. And it must be orthogonal to S k (A H , w).

Multiplication with A results in
which shows that Av k− has a component along the required direction A k+ v. And by the no-breakdown assumption, this vector will be linearly independent of K k (A, v). Using Lemma 1, i.e., x, y = Ax, A −H y , we obtain Hence, vector Av k− is orthogonal with respect to S k− (A H , w). It remains to orthogonalize it with respect to w k− , w k− and w k in order to satisfy the orthogonality condition v k+ ⊥ S k (A H , w). Thus, (5) is proven, since α i, k is chosen such that it eliminates the aforementioned directions from Av k− . Similar reasoning can be applied to construct v k+ . Consider v k , with properties Multiplication with A − and Lemma 1, i.e., x, y = A − x, A H y provides Variables α i, k+ are chosen such that they eliminate the directions such that v k+ ⊥ S k+ (A H , w). Thus proving (7).
Next, in Theorem 2, the matrix of recurrence coe cients is given, which has pentadiagonal structure. The projection of A onto K l (A, h) and orthogonal to S l (A H , w) is given by W H l AV l = Z l . The projection matrix Z l ∈ C (l+ )×(l+ ) is the matrix of recurrence coe cientsẐ l ∈ C (l+ )×(l+ ) with its last row removed.
has pentadiagonal structure. More precisely, for h i, k = Av k , w i and, α i, k = Av k− , w i and η k+ , k as a normalizing constant, i.e., as in Theorem 1, the matrix iŝ Proof. Consider the recurrence relation following immediately from Equation (5), This relation forms the even columns ofẐ l . To obtain a recurrence relation for Av k , i.e., the odd columns of Z l , look at the space in which this vector lives Thus the short recurrence relation, with h i, k = Av k , w i , is The four term recurrence relation contains some redundant information. This is suggested by the similarity of the coe cients α i,l and β i,l occurring in Theorem 1 and veri ed by the low rank structure appearing in the matrix of recurrence coe cientsẐ l . Example 1 illustrates the low rank structure ofẐ l . Example 1. Following the notation of Theorem 2, let l = . Then the matrix of recurrence coe cientŝ where × and * denote a generic nonzero element, exhibits some low rank structure. Namely, the pairs of nonzero elements represented as * equal their neighouring elements × multiplied with the same factor. Or in other words, every submatrix × * × * or * × * × has rank equal to 1. This implies that there is redundant information present inẐ l .

. Two-term recurrence relation
The low rank structure in the matrix of recurrence coe cientsẐ l from Theorem 2 implies that a shorter (2term) recurrence relation can be derived for the bases V l and W l , respectively spanning the (l+ )-dimensional subspaces of As before, they satisfy the conditions The shorter recurrence relation is achieved by simultaneously building biorthogonal bases for Thus we have now 4 subspaces: whose columns satisfy the conditions The recurrence relations are given in Theorem 3.

Theorem 3.
Let A ∈ C m×m be a nonsingular matrix and v, w ∈ C m , then biorthogonal bases V l (3), W l (4), V l (10) and W l (11) can be constructed by pairs of two-term recurrence relations.
where γ k+ = ṽ k+ , w k+ andγ k+ = v k+ ,w k+ . Normalization coe cients η i , ν i ,η i andν i are chosen such that v i , w i = and ṽ i ,w i = . This is assumed to be possible under the no-breakdown assumption.
Proof. The proof is given here for (12), (13), (16) and (17). For the remaining recurrence relations the proof is analogous. Assume, without loss of generality, l = k. The next basis vector v k+ must be constructed such

Multiplication with A and Lemma 1 provides us with
The required component A k+ v is present (20) and orthogonality is satis ed with respect to S k− (A H , w) (21). Note that Aṽ k is orthogonal to a larger subspace of S k (A H , w) than Av k− , this is the key observation to explain the shorter recurrence relation. Orthogonalization only remains to be done to eliminate components along w k , thus obtaining (12). The same derivation can be done forṽ k+ , which must expand S k (A, v) to S k+ (A, v) and must be orthogonal to K k (A H , w). Thereby proving (13). For v k+ , considerṽ k+ The component along A −k− v is present inṽ k+ (22). So it only remains to enforce the orthogonality conditions, orthogonalize along w k+ to obtain (16). Similarly forṽ k+ , to obtain (17).
From these recurrence relations, a matrix pencil representation of the matrix of recurrence coe cientsẐ l from Theorem 2 can be derived. This result is given in Theorem 4. This representation reveals thatẐ l can be represented by a product of essentially × matrices. This allows for an e cient way to store and manipulate this matrix on a computer.

Theorem 4.
Consider a nonsingular matrix A ∈ C m×m , v, w ∈ C m and basis V l ∈ C m×(l+ ) (3) spanning K l (A, v). The matrix pencil of recurrence coe cients (T l ,Ŝ l ), withT l ,Ŝ l ∈ C (l+ )×l+ satisfying can be represented by two tridiagonal matrices with a particular structure. More precisely, for the same coecients as used in Theorem 3, Proof. Rewrite the pairs of recurrence relations (12), (13), (16) and (17) in matrix notation to obtain The proof consists of substituting (25) into (24). Substitution is done as follows, consider . Then (24) can be used to obtain Repeating this procedure for v i ,ṽ i , i = , , . . . , l proves the statement.
Note that the structures of the matrices of recurrence coe cients appearing in Theorem 2 and Theorem 4 are known [9]. The contribution of this manuscript is the procedure to compute these matrices in an e cient manner.

. Levinson type derivation
The two-term recurrence relation can also be derived starting from the moment matrix arising from the subspaces K(A, v) and S (A H , w). The derivation here will follow a Levinson type procedure. Such procedures rely on the isomorphism between a vector space of (n + )-tuples and of polynomials of degree n. In the case studied here, the connection between vectors vn ∈ Kn(A, v) and Laurent polynomials an ∈ Fn(z) is used. Similar to the de nition of K l (A, v) and S l (A H , w), we de ne, for a nonnegative integer k: For even l = k, For odd l = k + , Note that M = S H K = M , where and the matrices representing multiplication with z and z − , respectively are An important observation is that, for Z k ∈ C ( k+ )×( k+ ) , the ( k + ) × ( k + ) principal submatrix of Z and e k ∈ C k+ the kth column of the unit matrix, So Z k represents multiplication with z in the basis { , z, z − , z , z − , . . . , z k , z −k } up to a rank one term. Before providing the two term recurrence relations in terms of Laurent polynomials in Theorem 5, an appropriate inner product must be de ned.

wherel(z) denotes the same Laurent polynomial asl(z) with its coe cients complex conjugated.
The inner product in De nition 1 relates closely to the Euclidean product from above.

Theorem 5. Consider the Laurent polynomials a i
. If they are constructed via η k a k+ (z) = zã k (z) − γ k a k (z), η k+ a k+ (z) =ã k+ (z) − γ k+ a k+ (z), without loss of generality, that n = k and an(z),ãn(z) are such that their coe cients an ,ãn ∈ C n+ with respect to bases { , z, z − , z , z − , . . . , z k , z −k } and { , z − , z, z − , z , . . . , z −k , z k }, respectively, satisfy These are matrix representations of orthogonality conditions [3, p.44]. Orthogonality is interpreted as follows, vector an in the basis formed by the columns of Kn, i.e., Kn an, is orthogonal to the space spanned by the columns of S n− , i.e., S H n− Kn an = . Then, for the next moment matrix M n+ and an , the embedding of an ∈ C n+ in the space C n+ , further denoted simply by an (same forãn), we obtain The goal is to nd a n+ ∈ C n+ such that it satis es the orthogonality condition, with τ n+ ≠ , i.e., no breakdown, It is easy to verify that ηn a n+ = Z n+ P n+ ã n − γn an, with γn =α n τn , will satisfy the orthogonality condition. Writing this in terms of Laurent polynomials provides ηn a n+ (z) = zãn(z)−γn an(z) and the recurrence relation for odd indices has been shown. Similarly forã n+ (z), consider Hence,ηnã n+ (z) = z − an(z) −γnãn(z), withγn = αñ τn , satis es the proposed recurrence relation. For even indices a small adjustment must be made to the derivation. Consider and η n+ a n+ (z) =ã n+ (z) − γ n+ a n+ (z), γ n+ =α n+ τn+ . Similarlyã n+ (z) can be shown to satisfyη n+ a n+ (z) = a n+ (z) −γ n+ a n+ (z),γ n+ = αn+ τ n+ . The proof of the recurrence relations for b i (z) can be done in a similar way.

. Connection with CMV
To retrieve the results reported by Watkins [10], it su ces to consider the same normalized starting vector for all spaces v = w = h and a unitary matrix U. The following corollaries summarize the results for the CMV which are given above in the more general case. For the CMV, the two pairs of four-term recurrence relations from Theorem 1 collapse into one pair of recurrence relations. The corresponding projected matrix is unitary. These results are given in Corollary 1 and 2, respectively.

Corollary 1. Consider a unitary matrix U ∈ C m×m and h ∈ C m . Then, with normalization of η i such that
Corollary 2. Consider a unitary matrix U ∈ C m×m , h ∈ C m and an orthogonal basis V l ∈ C m×(l+ ) for K l (U, h). Then the orthogonal projection of U onto K l (U, h), is a unitary matrix Z l ∈ C (l+ )×(l+ ) with pentadiagonal structure.
For the two-term recurrence relations, given in Theorem 3, the four pairs of recurrence relations collapse into two pairs. The coe cients also simplify, sinceγ =γ. The result is stated in Corollary 3. The related matrix pencil of the projected matrix consists of two unitary tridiagonal matrices, Corollary 4 states this result.

Corollary 3.
Consider a unitary matrix U ∈ C m×m and h ∈ C m . Then, with normalization of η i andη i such that v i , v i = and ṽ i ,ṽ i = , the recurrence relations (12) -(19) become The vectors v i ,ṽ i , i = , , . . . , l, form orthogonal bases for K l (U, h) and S l (U, h), respectively.

Corollary 4.
Consider a unitary matrix U ∈ C m×m , a vector h ∈ C m and an orthogonal basis V l ∈ C m×(l+ ) for K l (U, h). The matrix pencil of recurrence coe cients (T l ,Ŝ l ), withT l ,Ŝ l ∈ C (l+ )×l satisfying can be represented by two unitary tridiagonal matrices with the same sparsity structure as in Theorem 4.
The recurrence relations for the Laurent polynomials simplify in a similar manner, stated in Corollary 5.

Corollary 5. Consider Laurent polynomials a i
. If these are constructed as η k a k+ (z) = zã k (z) − γ k a k (z), Also note that the moment matrix is Hermitian and positive de nite. Hence, it allows for a Cholesky factorization which means Ln = R H n , i.e., coe cients are the same. This implies that, under suitable choice of normalization and starting vector, the biorthogonal bases collapse into one orthogonal basis.

Numerical behaviour
The numerical behaviour of the proposed four-and two-term recurrence relations from Theorem 1 and Theorem 3 is analysed. The matrices A ∈ C m×m used are restricted to well-conditioned normal matrices. Testing the recurrence relations with these matrices will shed some light on the stability of the methods. We are interested in the biorthogonality of the generated bases and how accurate the projection matrices represent the projection. Biorthogonality of the generated bases Vn ∈ C m×(n+ ) and Wn ∈ C m×(n+ ) for the Krylov  respectively. The projection matrix and projection matrix pencil are related to the matrices of recurrence coe cients from Theorem 2 and Theorem 4, respectively. Two other metrics are useful in order to interpret the results. These are the condition number of the matrix A, denoted by κ(A), and the growth factor, de ned, for the two-and four-term recurrence relation respectively, as ρ := max max(|T|, |S|) max (|A|) and ρ := max (|Z|) max (|A|) .
The growth factor provides a metric for stability. Throughout this section a modest size of matrices is chosen, a dimension of m = . This can be done without loss of generality, larger matrices (with the same properties) do inherently cause larger numerical errors, however the numerical behaviour remains similar.

. Unitary matrices
Consider the case corresponding to the CMV-decomposition, U ∈ C m×m is a unitary matrix and the starting vectors for the Krylov subspaces are equal, i.e., v = w = h ∈ C m . Let m = and h be a random vector. Both recurrence relations have a low biorthogonality and projection error, shown in Figure 1. When the dimension of the subspaces, n + , nears the dimension of the matrix, m, biorthogonality is lost more rapidly since the Ritz values (i.e., eigenvalues of the projection matrix) start to accurately approximate the eigenvalues of A. Furthermore, all projection matrices, Z, S and T are close to unitary. Hence, we conclude that for a unitary matrix and same starting vectors, we retrieve the CMV case. and same starting vectors. Two-term recurrence relation in blue and four-term recurrence relation in red.

. Scaled and shifted unitary matrices
Consider scaled and shifted unitary matrices, i.e., for a unitary matrix U ∈ C m×m and scalars µ, ω ∈ R, A = (µU + ωI) ∈ C m×m . Starting vectors are equal for all Krylov subspaces. Figure 2 shows the biorthogonality and projection errors for µ = . and ω = . Both errors are still small, however compared to the unitary case, shown in Figure 1, they are larger. The projection error increases steadily from n = and n = , for the two-and four-term recurrence relations, respectively. This can be (at least partially) explained by the increasing growth factor ρ, which is approximately ρ(n) = µ n . Figure 3 shows the errors for µ = and ω = . . The two-term recurrence relation performs better than the four-term recurrence relation, especially in terms of projection error.  Figure 3: Biorthogonality and projection error, respectively left and right, for a shifted unitary matrix (U + ωI) ∈ C × , with ω = . and same starting vectors. Two-term recurrence relation in blue and four-term recurrence relation in red.

. General normal matrices
Consider a normal matrix A ∈ C m×m with condition number κ(A) = and starting vectors v, w ∈ C m not necessarily equal. The errors are shown in Figure 4. The two-and four-term recurrence relations perform similar for both metrics. Biorthogonality is lost quickly and there is no Ritz value which approximates an eigenvalue accurately. Hence, this loss of biorthogonality is due to the recurrence relations themselves. Figure 5 shows the magnitude of the elements in the moment matrix W H V − I. There is clearly a pattern visible, however further research into the numerical properties of the recurrence relation must be done in order to interpret this.

. Conclusion
The numerical experiments verify the validity of the recurrence relations. The two-term recurrence relation performs better than the four-term recurrence relation. Biorthogonality can be lost quickly, further research must be done in order to understand and improve this. Reorthogonalization strategies can improve the biorthogonality of the generated bases.