Jump to ContentJump to Main Navigation
Show Summary Details

Formalized Mathematics

(a computer assisted approach)

Editor-in-Chief: Matuszewski, Roman

4 Issues per year

SCImago Journal Rank (SJR) 2015: 0.134
Source Normalized Impact per Paper (SNIP) 2015: 0.686
Impact per Publication (IPP) 2015: 0.296

Open Access
See all formats and pricing
Volume 15, Issue 3 (Jan 2007)


Definition and some Properties of Information Entropy

Bo Zhang
  • Shinshu University, Nagano, Japan
/ Yatsuka Nakamura
  • Shinshu University, Nagano, Japan
Published Online: 2008-06-09 | DOI: https://doi.org/10.2478/v10037-007-0012-9

Definition and some Properties of Information Entropy

In this article we mainly define the information entropy [3], [11] and prove some its basic properties. First, we discuss some properties on four kinds of transformation functions between vector and matrix. The transformation functions are LineVec2Mx, ColVec2Mx, Vec2DiagMx and Mx2FinS. Mx2FinS is a horizontal concatenation operator for a given matrix, treating rows of the given matrix as finite sequences, yielding a new finite sequence by horizontally joining each row of the given matrix in order to index. Then we define each concept of information entropy for a probability sequence and two kinds of probability matrices, joint and conditional, that are defined in article [25]. Further, we discuss some properties of information entropy including Shannon's lemma, maximum property, additivity and super-additivity properties.

  • [1] Grzegorz Bancerek. The ordinal numbers. Formalized Mathematics, 1(1):91-96, 1990.

  • [2] Grzegorz Bancerek and Krzysztof Hryniewiecki. Segments of natural numbers and finite sequences. Formalized Mathematics, 1(1):107-114, 1990.

  • [3] P. Billingsley. Ergodic Theory and Information. John Wiley & Sons, 1964.

  • [4] Czesław Byliński. Binary operations. Formalized Mathematics, 1(1):175-180, 1990.

  • [5] Czesław Byliński. Finite sequences and tuples of elements of a non-empty sets. Formalized Mathematics, 1(3):529-536, 1990.

  • [6] Czesław Byliński. Functions and their basic properties. Formalized Mathematics, 1(1):55-65, 1990.

  • [7] Czesław Byliński. Functions from a set to a set. Formalized Mathematics, 1(1):153-164, 1990.

  • [8] Czesław Byliński. Some basic properties of sets. Formalized Mathematics, 1(1):47-53, 1990.

  • [9] Czesław Byliński. The sum and product of finite sequences of real numbers. Formalized Mathematics, 1(4):661-668, 1990.

  • [10] Agata Darmochwał. The Euclidean space. Formalized Mathematics, 2(4):599-603, 1991.

  • [11] Shigeichi Hirasawa. Information Theory. Baifukan CO., 1996.

  • [12] Katarzyna Jankowska. Matrices. Abelian group of matrices. Formalized Mathematics, 2(4):475-480, 1991.

  • [13] Artur Korniłowicz. On the real valued functions. Formalized Mathematics, 13(1):181-187, 2005.

  • [14] Jarosław Kotowicz. Real sequences and basic operations on them. Formalized Mathematics, 1(2):269-272, 1990.

  • [15] Rafał Kwiatek. Factorial and Newton coefficients. Formalized Mathematics, 1(5):887-890, 1990.

  • [16] Yatsuka Nakamura, Nobuyuki Tamaura, and Wenpai Chang. A theory of matrices of real elements. Formalized Mathematics, 14(1):21-28, 2006.

  • [17] Library Committee of the Association of Mizar Users. Binary operations on numbers. To appear in Formalized Mathematics.

  • [18] Konrad Raczkowski and Andrzej Nedzusiak. Real exponents and logarithms. Formalized Mathematics, 2(2):213-216, 1991.

  • [19] Yasunari Shidama. The Taylor expansions. Formalized Mathematics, 12(2):195-200, 2004.

  • [20] Andrzej Trybulec. Subsets of complex numbers. To appear in Formalized Mathematics.

  • [21] Andrzej Trybulec. Tarski Grothendieck set theory. Formalized Mathematics, 1(1):9-11, 1990.

  • [22] Wojciech A. Trybulec. Vectors in real linear space. Formalized Mathematics, 1(2):291-296, 1990.

  • [23] Zinaida Trybulec. Properties of subsets. Formalized Mathematics, 1(1):67-71, 1990.

  • [24] Edmund Woronowicz. Relations and their basic properties. Formalized Mathematics, 1(1):73-83, 1990.

  • [25] Bo Zhang and Yatsuka Nakamura. The definition of finite sequences and matrices of probability, and addition of matrices of real elements. Formalized Mathematics, 14(3):101-108, 2006.

About the article

Published Online: 2008-06-09

Published in Print: 2007-01-01

Citation Information: Formalized Mathematics, ISSN (Online) 1898-9934, ISSN (Print) 1426-2630, DOI: https://doi.org/10.2478/v10037-007-0012-9. Export Citation

This content is open access.

Comments (0)

Please log in or register to comment.
Log in