Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Computational Methods in Applied Mathematics

Editor-in-Chief: Carstensen, Carsten

Managing Editor: Matus, Piotr

IMPACT FACTOR 2018: 1.218
5-year IMPACT FACTOR: 1.411

CiteScore 2018: 1.42

SCImago Journal Rank (SJR) 2018: 0.947
Source Normalized Impact per Paper (SNIP) 2018: 0.939

Mathematical Citation Quotient (MCQ) 2017: 0.76

See all formats and pricing
More options …
Volume 19, Issue 1


A Tensor Decomposition Algorithm for Large ODEs with Conservation Laws

Sergey V. Dolgov
Published Online: 2018-09-11 | DOI: https://doi.org/10.1515/cmam-2018-0023


We propose an algorithm for solution of high-dimensional evolutionary equations (ODEs and discretized time-dependent PDEs) in the Tensor Train (TT) decomposition, assuming that the solution and the right-hand side of the ODE admit such a decomposition with a low storage. A linear ODE, discretized via one-step or Chebyshev differentiation schemes, turns into a large linear system. The tensor decomposition allows to solve this system for several time points simultaneously using an extension of the Alternating Least Squares algorithm. This method computes a reduced TT model of the solution, but in contrast to traditional offline-online reduction schemes, solving the original large problem is never required. Instead, the method solves a sequence of reduced Galerkin problems, which can be set up efficiently due to the TT decomposition of the right-hand side. The reduced system allows a fast estimation of the time discretization error, and hence adaptation of the time steps. Besides, conservation laws can be preserved exactly in the reduced model by expanding the approximation subspace with the generating vectors of the linear invariants and correction of the Euclidean norm. In numerical experiments with the transport and the chemical master equations, we demonstrate that the new method is faster than traditional time stepping and stochastic simulation algorithms, whereas the invariants are preserved up to the machine precision irrespectively of the TT approximation accuracy.

Keywords: High-Dimensional Problems; Tensor Train Format; DMRG; Alternating Iteration; Differential Equations; Conservation Laws

MSC 2010: 65F10; 65L05; 65L04; 65M22; 65M70; 65D15; 15A23; 15A69


  • [1]

    A. C. Antoulas, D. C. Sorensen and S. Gugercin, A survey of model reduction methods for large-scale systems, Structured Matrices in Mathematics, Computer Science, and Engineering. I (Boulder 1999), Contemp. Math. 280, American Mathematical Society, Providence (2001), 193–219. Google Scholar

  • [2]

    P. Benner, S. Gugercin and K. Willcox, A survey of projection-based model reduction methods for parametric dynamical systems, SIAM Rev. 57 (2015), no. 4, 483–531. Web of ScienceCrossrefGoogle Scholar

  • [3]

    H.-J. Bungartz and M. Griebel, Sparse grids, Acta Numer. 13 (2004), 147–269. CrossrefGoogle Scholar

  • [4]

    G. D. Byrne and A. C. Hindmarsh, A polyalgorithm for the numerical solution of ordinary differential equations, ACM Trans. Math. Software 1 (1975), no. 1, 71–96. CrossrefGoogle Scholar

  • [5]

    V. de Silva and L.-H. Lim, Tensor rank and the ill-posedness of the best low-rank approximation problem, SIAM J. Matrix Anal. Appl. 30 (2008), no. 3, 1084–1127. Web of ScienceCrossrefGoogle Scholar

  • [6]

    S. Dolgov and B. Khoromskij, Two-level QTT-Tucker format for optimized tensor calculus, SIAM J. Matrix Anal. Appl. 34 (2013), no. 2, 593–623. Web of ScienceCrossrefGoogle Scholar

  • [7]

    S. Dolgov and B. Khoromskij, Simultaneous state-time approximation of the chemical master equation using tensor product formats, Numer. Linear Algebra Appl. 22 (2015), no. 2, 197–219. Web of ScienceCrossrefGoogle Scholar

  • [8]

    S. V. Dolgov, B. N. Khoromskij and I. V. Oseledets, Fast solution of parabolic problems in the tensor train/quantized tensor train format with initial application to the Fokker–Planck equation, SIAM J. Sci. Comput. 34 (2012), no. 6, A3016–A3038. Web of ScienceCrossrefGoogle Scholar

  • [9]

    S. V. Dolgov and D. V. Savostyanov, Alternating minimal energy methods for linear systems in higher dimensions, SIAM J. Sci. Comput. 36 (2014), no. 5, A2248–A2271. CrossrefWeb of ScienceGoogle Scholar

  • [10]

    M. Fannes, B. Nachtergaele and R. F. Werner, Finitely correlated states on quantum spin chains, Comm. Math. Phys. 144 (1992), no. 3, 443–490. CrossrefGoogle Scholar

  • [11]

    D. T. Gillespie, A general method for numerically simulating the stochastic time evolution of coupled chemical reactions, J. Comput. Phys. 22 (1976), no. 4, 403–434. CrossrefGoogle Scholar

  • [12]

    I. G. Graham, F. Y. Kuo, D. Nuyens, R. Scheichl and I. H. Sloan, Quasi-Monte Carlo methods for elliptic PDEs with random coefficients and applications, J. Comput. Phys. 230 (2011), no. 10, 3668–3694. Web of ScienceCrossrefGoogle Scholar

  • [13]

    L. Grasedyck, Hierarchical singular value decomposition of tensors, SIAM J. Matrix Anal. Appl. 31 (2009/10), no. 4, 2029–2054. Web of ScienceGoogle Scholar

  • [14]

    A. Gupta and M. Khammash, Determining the long-term behavior of cell populations: A new procedure for detecting ergodicity in large stochastic reaction networks, IFAC Proc. 47 (2014), no. 3, 1711–1716. CrossrefGoogle Scholar

  • [15]

    W. Hackbusch, Tensor Spaces and Numerical Tensor Calculus, Springer Ser. Comput. Math. 42, Springer, Heidelberg, 2012. Google Scholar

  • [16]

    M. Hegland, C. Burden, L. Santoso, S. MacNamara and H. Booth, A solver for the stochastic master equation applied to gene regulatory networks, J. Comput. Appl. Math. 205 (2007), no. 2, 708–724. CrossrefWeb of ScienceGoogle Scholar

  • [17]

    F. L. Hitchcock, Multiple invariants and generalized rank of a p-way matrix or tensor, J. Math. Phys. 7 (1927), no. 1, 39–79. Google Scholar

  • [18]

    S. Holtz, T. Rohwedder and R. Schneider, The alternating linear scheme for tensor optimization in the tensor train format, SIAM J. Sci. Comput. 34 (2012), no. 2, A683–A713. CrossrefWeb of ScienceGoogle Scholar

  • [19]

    T. Jahnke, On reduced models for the chemical master equation, Multiscale Model. Simul. 9 (2011), no. 4, 1646–1676. Web of ScienceCrossrefGoogle Scholar

  • [20]

    T. Jahnke and W. Huisinga, A dynamical low-rank approach to the chemical master equation, Bull. Math. Biol. 70 (2008), no. 8, 2283–2302. Web of ScienceCrossrefGoogle Scholar

  • [21]

    E. Jeckelmann, Dynamical density–matrix renormalization–group method, Phys. Rev. B 66 (2002), Article ID 045114. Google Scholar

  • [22]

    V. A. Kazeev, B. N. Khoromskij and E. E. Tyrtyshnikov, Multilevel Toeplitz matrices generated by tensor-structured vectors and convolution with logarithmic complexity, SIAM J. Sci. Comput. 35 (2013), no. 3, A1511–A1536. CrossrefWeb of ScienceGoogle Scholar

  • [23]

    V. A. Kazeev, O. Reichmann and C. Schwab, hp-DG-QTT solution of high-dimensional degenerate diffusion equations, Technical Report 2012-11, ETH SAM, Zürich, 2012. Google Scholar

  • [24]

    G. Kerschen, J.-C. Golinval, A. F. Vakakis and L. A. Bergman, The method of proper orthogonal decomposition for dynamical characterization and order reduction of mechanical systems: an overview, Nonlinear Dynam. 41 (2005), no. 1–3, 147–169. CrossrefGoogle Scholar

  • [25]

    B. N. Khoromskij, O(dlogN)-quantics approximation of N-d tensors in high-dimensional numerical modeling, Constr. Approx. 34 (2011), no. 2, 257–280. Google Scholar

  • [26]

    B. N. Khoromskij, Tensor numerical methods for multidimensional PDEs: Theoretical analysis and initial applications, CEMRACS 2013—Modelling and Simulation of Complex Systems: Stochastic and Deterministic Approaches, ESAIM Proc. Surveys 48, EDP Science, Les Ulis (2015), 1–28. Google Scholar

  • [27]

    O. Koch and C. Lubich, Dynamical tensor approximation, SIAM J. Matrix Anal. Appl. 31 (2010), no. 5, 2360–2375. Web of ScienceCrossrefGoogle Scholar

  • [28]

    T. G. Kolda and B. W. Bader, Tensor decompositions and applications, SIAM Rev. 51 (2009), no. 3, 455–500. Web of ScienceCrossrefGoogle Scholar

  • [29]

    C. Lubich, I. V. Oseledets and B. Vandereycken, Time integration of tensor trains, SIAM J. Numer. Anal. 53 (2015), no. 2, 917–941. CrossrefWeb of ScienceGoogle Scholar

  • [30]

    J. L. Lumley, The structure of inhomogeneous turbulent flows, Atmospheric Turbulence and Radio Wave Propagation, Nauka, Moscow (1967), 166–178. Google Scholar

  • [31]

    C. Moler and C. Van Loan, Nineteen dubious ways to compute the exponential of a matrix, twenty-five years later, SIAM Rev. 45 (2003), no. 1, 3–49. CrossrefGoogle Scholar

  • [32]

    B. Munsky and M. Khammash, A multiple time interval finite state projection algorithm for the solution to the chemical master equation, J. Comput. Phys. 226 (2007), no. 1, 818–835. Web of ScienceCrossrefGoogle Scholar

  • [33]

    H. Niederreiter, Quasi-Monte Carlo methods and pseudo-random numbers, Bull. Amer. Math. Soc. 84 (1978), no. 6, 957–1041. CrossrefGoogle Scholar

  • [34]

    A. Nouy, A priori model reduction through proper generalized decomposition for solving time-dependent partial differential equations, Comput. Methods Appl. Mech. Engrg. 199 (2010), no. 23–24, 1603–1626. CrossrefGoogle Scholar

  • [35]

    I. V. Oseledets, Tensor-train decomposition, SIAM J. Sci. Comput. 33 (2011), no. 5, 2295–2317. Web of ScienceCrossrefGoogle Scholar

  • [36]

    I. V. Oseledets and S. V. Dolgov, Solution of linear systems and matrix inversion in the TT-format, SIAM J. Sci. Comput. 34 (2012), no. 5, A2718–A2739. Web of ScienceCrossrefGoogle Scholar

  • [37]

    T. Rohwedder and A. Uschmajew, On local convergence of alternating schemes for optimization of convex problems in the tensor train format, SIAM J. Numer. Anal. 51 (2013), no. 2, 1134–1162. CrossrefWeb of ScienceGoogle Scholar

  • [38]

    D. V. Savostyanov, S. V. Dolgov, J. M. Werner and I. Kuprov, Exact NMR simulation of protein-size spin systems using tensor train formalism, Phys. Rev. B 90 (2014), Article ID 085139. Web of ScienceGoogle Scholar

  • [39]

    U. Schollwöck, The density-matrix renormalization group in the age of matrix product states, Ann. Physics 326 (2011), no. 1, 96–192. Web of ScienceCrossrefGoogle Scholar

  • [40]

    D. Schötzau, hp-DGFEM for parabolic evolution problems. Applications to diffusion and viscous incompressible fluid flow, PhD thesis, ETH, Zürich, 1999. Google Scholar

  • [41]

    L. Sirovich, Turbulence and the dynamics of coherent structures. I. Coherent structures, Quart. Appl. Math. 45 (1987), no. 3, 561–571. CrossrefGoogle Scholar

  • [42]

    C. G. Small and J. Wang, Numerical Methods for Nonlinear Estimating Equations, Oxford Statist. Sci. Ser. 29, The Clarendon Press, Oxford, 2003. Google Scholar

  • [43]

    S. A. Smoljak, Quadrature and interpolation formulae on tensor products of certain function classes, Dokl. Akad. Nauk SSSR 148 (1963), 1042–1045. Google Scholar

  • [44]

    E. Tadmor, The exponential accuracy of Fourier and Chebyshev differencing methods, SIAM J. Numer. Anal. 23 (1986), no. 1, 1–10. CrossrefGoogle Scholar

  • [45]

    L. N. Trefethen, Spectral Methods in MATLAB, Software Environ. Tools 10, Society for Industrial and Applied Mathematics, Philadelphia, 2000. Google Scholar

  • [46]

    G. Vidal, Efficient simulation of one-dimensional quantum many-body systems, Phys. Rev. Lett. 93 (2004), Article ID 040502. Google Scholar

  • [47]

    T. von Petersdorff and C. Schwab, Numerical solution of parabolic equations in high dimensions, M2AN Math. Model. Numer. Anal. 38 (2004), no. 1, 93–127. CrossrefGoogle Scholar

  • [48]

    S. R. White, Density-matrix algorithms for quantum renormalization groups, Phys. Rev. B 48 (1993), no. 14, 10345–10356. CrossrefGoogle Scholar

  • [49]

    S. R. White and A. E. Feiguin, Real-time evolution using the density matrix renormalization group, Phys. Rev. Lett. 93 (2004), Article ID 076401. Google Scholar

About the article

Received: 2017-10-27

Revised: 2018-01-10

Accepted: 2018-05-02

Published Online: 2018-09-11

Published in Print: 2019-01-01

Funding Source: Engineering and Physical Sciences Research Council

Award identifier / Grant number: EP/M019004/1

The author acknowledges funding from the EPSRC fellowship EP/M019004/1.

Citation Information: Computational Methods in Applied Mathematics, Volume 19, Issue 1, Pages 23–38, ISSN (Online) 1609-9389, ISSN (Print) 1609-4840, DOI: https://doi.org/10.1515/cmam-2018-0023.

Export Citation

© 2018 Walter de Gruyter GmbH, Berlin/Boston.Get Permission

Comments (0)

Please log in or register to comment.
Log in