Approximation of Information Divergences for Statistical Learning with Applications

Milan Stehlík 1 , Ján Somorčík 2 , Luboš Střelec 3  and Jaromír Antoch 4
  • 1 Institute of Applied Statistics and Linz Institute of Technology, Altenberger Strasse 69 A–4040 Linz, Austria, Austria
  • 2 Department of Applied Mathematics and Statistics Comenius University, SK–842 48 Bratislava, Mlynská Dolina, Slovakia
  • 3 Department of Statistics and Operation Analysis Mendel University in Brno, Zemědělská 1 CZ–613 00, Brno, Czech Republic
  • 4 Department of Probability and Mathematical Statistics Charles University, 83 CZ–186 75, Praha, Sokolovská
Milan Stehlík
  • Corresponding author
  • Institute of Applied Statistics and Linz Institute of Technology, Johannes Kepler University, Altenberger Strasse 69 A–4040 Linz, Austria, Institute of Statistics University of Valparaíso, Gran Bretana 1111 Valparaíso Chile, Austria
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Ján Somorčík
  • Department of Applied Mathematics and Statistics Comenius University, Mlynská Dolina, SK–842 48 Bratislava, Slovakia
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
, Luboš Střelec
  • Department of Statistics and Operation Analysis Mendel University in Brno, Zemědělská 1 CZ–613 00, Brno, Czech Republic
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar
and Jaromír Antoch
  • Department of Probability and Mathematical Statistics Charles University, Sokolovská, 83 CZ–186 75, Praha, Czech Republic
  • Email
  • Search for other articles:
  • degruyter.comGoogle Scholar

Abstract

In this paper we give a partial response to one of the most important statistical questions, namely, what optimal statistical decisions are and how they are related to (statistical) information theory. We exemplify the necessity of understanding the structure of information divergences and their approximations, which may in particular be understood through deconvolution. Deconvolution of information divergences is illustrated in the exponential family of distributions, leading to the optimal tests in the Bahadur sense. We provide a new approximation of I-divergences using the Fourier transformation, saddle point approximation, and uniform convergence of the Euler polygons. Uniform approximation of deconvoluted parts of I-divergences is also discussed. Our approach is illustrated on a real data example.

  • [1]

    ANTOCH, J.—JARUšKOVá, D.: Testing a homogeneity of stochastic processes, Kybernetika 41 (2007),415–430.

  • [2]

    CORLESS, R. M.—GONNET, G. H.—HARE, D. E. G.—JEFFREY, D. J: On Lambert’s W Function. Technical Report Cs-93-03, Department of Computer Science, University of Waterloo.

  • [3]

    JOHNSON, N. L.—KOTZ, S.—BALAKRISHNAN, N.: Continuous Univariate Distributions, Volume 1, 2nd ed., J. Wiley, New York, 1994.

  • [4]

    MILLER, G. K.: Maximum likelihood estimation for the Erlang integer parameter, Statist. Probab. Lett. 43 (1999), 335–341.

  • [5]

    PáZMAN, A.: Nonlinear Statistical Models, Dordrecht, Kluwer Academic Publishers, 1993.

  • [6]

    PáZMAN, A.: The density of the parameter estimators when the observations are distributed exponentially, Metrika 44 (1996), 9–26.

  • [7]

    RÉNYI, A.: Wahrscheinlichkeitsrechnung mit einem Anhang über Informationstheorie. VEB Deutcher Verlag der Wissenschaften, Berlin, 1962.

  • [8]

    RUBLíK, F.: On optimality of the likelihood ratio tests in the sense of exact slopes. Part 1, General case, Kybernetika 25 (1989), 13–25.

  • [9]

    RUBLíK, F.: On optimality of the likelihood ratio tests in the sense of exact slopes. Part 2, Application to individual distributions, Kybernetika 25 (1989), 117–135.

  • [10]

    STEHLíK, M.: Exact likelihood ratio tests of the scale in the Gamma family, Tatra Mt. Math. Publ. 26 (2003), 381–390.

  • [11]

    STEHLíK, M.: Distributions of exact tests in the exponential family, Metrika 57 (2003), 145–164.

  • [12]

    STEHLíK, M.: Decompositions of information divergences: Recent development, open problems and applications. In: AIP Conf. Proc. 1493, 2012, pp. 972–976.

  • [13]

    STEHLíK, M.—ECONOMOU, P.—KISEL’áK, J.—RICHTER, W. D.: Kullback-Leibler life time testing, Appl. Math. Comput. 240 (2014), 122–139.

  • [14]

    STEHLíK, M.—OSOSKOV, G. A.: Efficient testing of the homogeneity, scale parameters and number of components in the Rayleigh mixture. JINR Rapid Communications E-11-2003-116, 2003, 1493.

  • [15]

    STEHLíK, M.—WAGNER, H.: Exact likelihood ratio testing for homogeneity of the exponential distribution, Comm. Statist. Simulation Comput. 40 (2011), 663–684.

  • [16]

    WILKS, S. S.: Mathematical Statistics, New York – London, J. Wiley and Sons, 1962.

Purchase article
Get instant unlimited access to the article.
$42.00
Log in
Already have access? Please log in.


Journal + Issues

Mathematica Slovaca, the oldest and best mathematical journal in Slovakia, was founded in 1951 at the Mathematical Institute of the Slovak Academy of Science, Bratislava. It covers practically all mathematical areas. As a respectful international mathematical journal, it publishes only highly nontrivial original articles with complete proofs by assuring a high quality reviewing process.

Search