In this paper we are interested in the eigenvalue problem associated with integral operators
defined from kernel functions
Our approach is general, but driven by practical applications we focus on kernels
Of our main interest is the numerical approximation of the eigenvalue problem associated with A, namely,
Eigenproblems of the above type arise frequently in various research areas such as geology [8, 7], uncertainty quantification [3, 10, 19], machine learning , etc. The analysis of the underlying eigenvalue problem is beneficial in the derivation of the error control, algorithm design, and overall numerical practice, etc.
Mathematically, the problem is usually formulated in either the space of continuous functions
or the space of
For the Nyström method, though various error estimates were derived, e.g., in [12, 25, 2, 18, 21, 22], it is not known if they are consistent with actual numerical results, especially when the kernel function is not smooth enough, for example, not continuously differentiable. Additionally, the proofs require the mesh size to be sufficiently small.
For the Galerkin method, which generally requires to use certain quadrature rule to evaluate the double integrals when assembling the matrix, the impact of the quadrature error to the computed eigenvalues is of practical interest and needs special investigation especially when the integrand is not sufficiently smooth (for example, functions with unbounded derivatives). This is the case of the kernels of the form (1.1) studied in the present paper.
Our aim is to present a comprehensive study
of the eigenvalue problems for integral operators associated with kernel functions of exponential type as defined in (1.1).
Those kernel functions are not necessarily smooth, i.e., they may not have continuous (partial) derivatives.
Theoretically, we focus on the analysis of two formulations of
the operator eigenvalue problem in terms of
The contributions are listed below (see Section 4 for details).
Firstly, we present a new framework to analyze the Nyström discretization. To obtain the Nyström discretization error, we show that it is numerically equivalent to the Galerkin discretization, and thus the error estimate for Galerkin discretization immediately carries over, which reads
where h is the mesh size,
Secondly, to the best of our knowledge, we prove for the first time that the Galerkin method and Nyström method are numerically equivalent in the sense that
Thirdly, we perform several numerical experiments to examine various theoretical estimates, including the convergence rate, dependence of the asymptotic constant on λ, approximation of eigenfunctions, etc. Our numerical results indicate that the eigenvalue convergence rate is quadratic with respect to the mesh size and for different eigenvalues, the approximation error is roughly independent of the eigenvalue magnitude. A detailed discussion relating our estimates and the ones from [12, 2, 18, 21] is presented.
The rest of the paper is organized as follows. In Section 2, we study the integral operator associated with kernel functions of exponential type and state the positive (semi-)definiteness of the operator as well as the related matrices. Section 3 presents abstract estimates for the Galerkin approximation to the underlying eigenvalue problem. The main results are presented in Section 4, including convergence rates of Galerkin and Nyström discretizations, the equivalence between the two discretizations, etc. Section 5 provides a numerical study of various theoretical results in Section 4 and in existing literature [12, 2, 18, 21]. The proof for the positive (semi-)definiteness of the operator and the related matrices is given in the appendix (Section 7).
2 Integral Operators with Kernel Functions of Exponential Type
For notational convenience, for any given bounded Lipschitz domain in
2.1 Some Auxiliary Estimates
The following result is immediate using straightforward calculation.
The kernel function
if A is the integral operator defined in (1.2),
then for each
Next we estimate the second derivatives of the kernels of our interest, which is needed in the error analysis that we provide later on. Let
A direct calculation shows that the second order partial derivatives of K are unbounded at
where α is a multi-index and C is a generic constant depending only on
2.2 Mapping Properties
The following well-known mapping properties of integral operators associated with continuous kernel functions are collected below (e.g., ).
Let A be defined in (1.2). Then:
The above proposition ensures that the theoretical results presented in Section 3 apply to our particular case of kernels of exponential type.
2.3 Positive Definiteness
A continuous function
For bounded continuous functions, the positive semi-definiteness is equivalent to that of the associated integral operator (cf. ).
A bounded continuous function
for all functions v in the Schwartz space
3 Abstract Results
In this section,
An operator A on a Hilbert space
Note that a positive operator is necessarily self-adjoint, i.e.,
A crucial tool we use in the estimate of eigenvalues is the Courant-Fischer min-max (or max-min) principle (cf. ).
Let A be a compact, self-adjoint operator on
In this paper, we are interested in positive eigenvalues of A, and
the eigenvalue problem is to find
Under the assumptions in Theorem 3.2,
The scaling of
Note that if the multiplicity of
4 Eigenvalue Problems
For the integral operator A defined in (1.2),
we present two formulations for its eigenvalue problem
4.1 Two Formulations:
Recall that A is compact on both
For the Banach space
In addition to being compact on
where the algebraic multiplicity is equal to geometric multiplicity for any
4.2 Galerkin Discretization for
In this subsection, we consider the Galerkin discretization of the eigenvalue problem in (4.1).
and let the projection
The result below is standard.
In addition to (4.4), an
4.2.1 The Matrix Eigenvalue Problem
Given a subdivision
In practice, for the ease of implementation,
we use certain quadrature rule to compute the double integral of the kernel function
4.3 Nyström Discretization for
The eigenfunction can then be recovered from nodal values
transforms the above matrix problem into a standard symmetric eigenvalue problem identical to (4.8):
Therefore, we see that, the Galerkin method coincides with the Nyström method up to quadrature errors from (4.6). We will show in Section 4.4 (see Theorem 4.3) that the quadrature error does not dominate the discretization error in the eigenvalue computation. Therefore, the convergence result for the Nyström method below can be obtained with the help of Theorem 4.1 for the Galerkin method.
where C is a constant that only depends on the shape parameter of the mesh and
4.4 Equivalence of Galerkin and Nyström Discretizations
We have shown in Proposition 4.1 that at the continuous level the two formulations in (4.1) and (4.2) are equivalent. In this subsection, we build the discrete counterpart of such an equivalence. Namely, we estimate the error in computed eigenvalues from two discretizations discussed in Section 4.2 and Section 4.3. The main result is stated below.
where the constant C is independent of any eigenvalue.
is Lipschitz continuous, then
Here α is a multi-index,
The Taylor expansion of
where the remainder satisfies
where the summands in the bracket are estimated using the Lipschitz condition. ∎
Under the assumptions in Theorem 4.3,
Without loss of generality, assume
In this case, we illustrate the proof for a uniform rectangular mesh and the same idea applies to the general case.
we estimate for each fixed i the quantity
The 0th layer is
Next we estimate
Note that for each
The number of elements in layer
where L denotes the maximal number of layers and obviously
which completes the proof of Case (1).
The inequality above can also be shown via the following argument:
In this case,
Hence Lemma 4.1 implies
Then the same argument as in Case (1) yields the desired estimate:
In this case,
where the last identity follows from the Taylor expansion:
The proof of the theorem is complete. ∎
4.4.1 Numerical Illustration
To show that the
5 Numerical Experiments
We perform various numerical tests for the integral operator
We first consider an example with known eigenpairs from 
The decay rate of eigenvalues is known to be
We consider in this example the kernel function associated with the
Since the exact eigenvalues are not known, we use the computed eigenvalues
over a finer mesh with mesh size
5.1 Rate of Convergence
From Figure 4 and Figure 5 (with fixed mesh size in each plot),
we see that (for leading eigenvalues) the error
We then examine the magnitude of the constant C.
For the four problems shown in Figure 4 and Figure 5,
the maximal approximation errors
5.2 Eigenfunction Approximation
Since all theoretical error bounds for eigenvalues
are expressed in terms of certain approximation errors of eigenfunctions,
we compute the actual approximation error of the eigenfunctions in this subsection
using Example 1 with
5.3 Comparison of Existing Theoretical Estimates
Using the exact eigenpairs in Example 1, we compare different error estimates, e.g., in [12, 2, 18, 21] and (4.10), to true errors in the eigenvalue computations. It will be seen that all theoretical error bounds overestimate the true error by a large margin of various degrees and the error bound in (4.10) is more accurate.
Estimates in (4.10).
where the scaling
which is inconsistent with the
For smooth kernel functions like
5.4 A Conjecture of a Sharp Bound
Following the investigation in Section 5.2
on the actual approximation error of eigenfunctions,
we derive a similar estimate in two dimensions
and then propose a conjecture concerning the actual convergence rate.
Using the one-dimensional result in (5.3), we deduce that
in accordance with the one-dimensional counterpart in (5.3).
The numerical results lead us to the following conjecture:
Let λ and
where the constants
We obtain eigenvalue error estimates of second order for the lowest order Galerkin and Nyström discretizations. The equivalence between the two discretizations is established, which makes the analysis of the Nyström method a consequence of the Galerkin one. The resulting estimates appear more accurate than the previously available ones. Numerical experiments illustrate and complement the new and previously existing theoretical results.
A function f is called completely monotone on
The Bernstein–Widder Theorem
shows that the Laplace transform of a nonnegative
Proposition 7.1 lists two completely monotone functions that are needed in the proof.
The following two functions are completely monotone on
Now we are in a position to carry out the proof of Theorem 2.1.
If ρ is the weighted
Thus the matrix
For the rest two forms of ρ, the result follows from the Schoenberg Interpolation Theorem and Proposition 7.1. ∎
K. E. Atkinson, The numerical solutions of the eigenvalue problem for compact integral operators, Trans. Amer. Math. Soc. 129 (1967), 458–465.
K. E. Atkinson, Convergence rates for approximate eigenvalues of compact integral operators, SIAM J. Numer. Anal. 12 (1975), 213–222.
I. Babuška, F. Nobile and R. Tempone, A stochastic collocation method for elliptic partial differential equations with random input data, SIAM Rev. 52 (2010), no. 2, 317–355.
R. Bhatia, Perturbation Bounds for Matrix Eigenvalues, Class. Appl. Math. 53, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 2007.
F. Chatelin, Spectral Approximation of Linear Operators, Class. Appl. Math. 65, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 2011.
W. Cheney and W. Light, A Course in Approximation Theory, Grad. Stud. Math. 101, American Mathematical Society, Providence, 2009.
G. Christakos, Modern Spatiotemporal Geostatistics. Vol. 6, Oxford University Press, Oxford, 2000.
L. W. Gelhar, Stochastic subsurface hydrology from theory to applications, Water Resources Res. 22 (1986), 1–9.
R. G. Ghanem and P. D. Spanos, Stochastic Finite Elements: A Spectral Approach, Springer, New York, 1991.
M. D. Gunzburger, C. G. Webster and G. Zhang, Stochastic finite element methods for partial differential equations with random input data, Acta Numer. 23 (2014), 521–650.
R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University Press, Cambridge, 1990.
H. B. Keller, On the accuracy of finite difference approximations to the eigenvalues of differential and integral operators, Numer. Math. 7 (1965), 412–419.
M. A. Krasnosel’skii, G. M. Vainikko, R. P. Zabreyko, Y. B. Ruticki and V. Y. Stetsenko, Approximate Solution of Operator Equations, Springer, Cham, 2012.
R. Kress, Linear Integral Equations, 3rd ed., Appl. Math. Sci. 82, Springer, New York, 2014.
P. D. Lax, Functional Analysis, Pure Appl. Math. (New York), Wiley-Interscience, New York, 2002.
N. M. Nasrabadi, Pattern recognition and machine learning, J. Electron. Imag. 16 (2007), no. 4, Article ID 049901.
E. J. Nyström, Über Die Praktische Auflösung von Integralgleichungen mit Anwendungen auf Randwertaufgaben, Acta Math. 54 (1930), no. 1, 185–204.
S. Osborn, P. S. Vassilevski and U. Villa, A multilevel, hierarchical sampling technique for spatially correlated random fields, SIAM J. Sci. Comput. 39 (2017), no. 5, S543–S562.
W. Rudin, Functional Analysis, 2nd ed., Int. Ser. Pure Appl. Math., McGraw-Hill, New York, 1991.
A. Spence, Error bounds and estimates for eigenvalues of integral equations, Numer. Math. 29 (1977/78), no. 2, 133–147.
G. W. Stewart, Matrix Algorithms. Vol. II. Eigensystems, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, 2001.
G. M. Vainikko, On the speed of convergence of approximate methods in the eigenvalue problem, Zh. Vychisl. Mat. Mat. Fiz. 7 (1967), 977–987.
H. Wendland, Scattered Data Approximation, Cambridge Monogr. Appl. Comput. Math. 17, Cambridge University Press, Cambridge, 2005.
H. Weyl, Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung), Math. Ann. 71 (1912), no. 4, 441–479.