Jump to ContentJump to Main Navigation
Show Summary Details
More options …

Formalized Mathematics

(a computer assisted approach)

Editor-in-Chief: Matuszewski, Roman

4 Issues per year


SCImago Journal Rank (SJR) 2016: 0.207
Source Normalized Impact per Paper (SNIP) 2016: 0.315

Open Access
Online
ISSN
1898-9934
See all formats and pricing
More options …
Volume 15, Issue 3 (Jan 2007)

Issues

Definition and some Properties of Information Entropy

Bo Zhang / Yatsuka Nakamura
Published Online: 2008-06-09 | DOI: https://doi.org/10.2478/v10037-007-0012-9

Definition and some Properties of Information Entropy

In this article we mainly define the information entropy [3], [11] and prove some its basic properties. First, we discuss some properties on four kinds of transformation functions between vector and matrix. The transformation functions are LineVec2Mx, ColVec2Mx, Vec2DiagMx and Mx2FinS. Mx2FinS is a horizontal concatenation operator for a given matrix, treating rows of the given matrix as finite sequences, yielding a new finite sequence by horizontally joining each row of the given matrix in order to index. Then we define each concept of information entropy for a probability sequence and two kinds of probability matrices, joint and conditional, that are defined in article [25]. Further, we discuss some properties of information entropy including Shannon's lemma, maximum property, additivity and super-additivity properties.

  • [1] Grzegorz Bancerek. The ordinal numbers. Formalized Mathematics, 1(1):91-96, 1990.Google Scholar

  • [2] Grzegorz Bancerek and Krzysztof Hryniewiecki. Segments of natural numbers and finite sequences. Formalized Mathematics, 1(1):107-114, 1990.Google Scholar

  • [3] P. Billingsley. Ergodic Theory and Information. John Wiley & Sons, 1964.Google Scholar

  • [4] Czesław Byliński. Binary operations. Formalized Mathematics, 1(1):175-180, 1990.Google Scholar

  • [5] Czesław Byliński. Finite sequences and tuples of elements of a non-empty sets. Formalized Mathematics, 1(3):529-536, 1990.Google Scholar

  • [6] Czesław Byliński. Functions and their basic properties. Formalized Mathematics, 1(1):55-65, 1990.Google Scholar

  • [7] Czesław Byliński. Functions from a set to a set. Formalized Mathematics, 1(1):153-164, 1990.Google Scholar

  • [8] Czesław Byliński. Some basic properties of sets. Formalized Mathematics, 1(1):47-53, 1990.Google Scholar

  • [9] Czesław Byliński. The sum and product of finite sequences of real numbers. Formalized Mathematics, 1(4):661-668, 1990.Google Scholar

  • [10] Agata Darmochwał. The Euclidean space. Formalized Mathematics, 2(4):599-603, 1991.Google Scholar

  • [11] Shigeichi Hirasawa. Information Theory. Baifukan CO., 1996.Google Scholar

  • [12] Katarzyna Jankowska. Matrices. Abelian group of matrices. Formalized Mathematics, 2(4):475-480, 1991.Google Scholar

  • [13] Artur Korniłowicz. On the real valued functions. Formalized Mathematics, 13(1):181-187, 2005.Google Scholar

  • [14] Jarosław Kotowicz. Real sequences and basic operations on them. Formalized Mathematics, 1(2):269-272, 1990.Google Scholar

  • [15] Rafał Kwiatek. Factorial and Newton coefficients. Formalized Mathematics, 1(5):887-890, 1990.Google Scholar

  • [16] Yatsuka Nakamura, Nobuyuki Tamaura, and Wenpai Chang. A theory of matrices of real elements. Formalized Mathematics, 14(1):21-28, 2006.Google Scholar

  • [17] Library Committee of the Association of Mizar Users. Binary operations on numbers. To appear in Formalized Mathematics.Google Scholar

  • [18] Konrad Raczkowski and Andrzej Nedzusiak. Real exponents and logarithms. Formalized Mathematics, 2(2):213-216, 1991.Google Scholar

  • [19] Yasunari Shidama. The Taylor expansions. Formalized Mathematics, 12(2):195-200, 2004.Google Scholar

  • [20] Andrzej Trybulec. Subsets of complex numbers. To appear in Formalized Mathematics.Google Scholar

  • [21] Andrzej Trybulec. Tarski Grothendieck set theory. Formalized Mathematics, 1(1):9-11, 1990.Google Scholar

  • [22] Wojciech A. Trybulec. Vectors in real linear space. Formalized Mathematics, 1(2):291-296, 1990.Google Scholar

  • [23] Zinaida Trybulec. Properties of subsets. Formalized Mathematics, 1(1):67-71, 1990.Google Scholar

  • [24] Edmund Woronowicz. Relations and their basic properties. Formalized Mathematics, 1(1):73-83, 1990.Google Scholar

  • [25] Bo Zhang and Yatsuka Nakamura. The definition of finite sequences and matrices of probability, and addition of matrices of real elements. Formalized Mathematics, 14(3):101-108, 2006.Google Scholar

About the article


Published Online: 2008-06-09

Published in Print: 2007-01-01


Citation Information: Formalized Mathematics, ISSN (Online) 1898-9934, ISSN (Print) 1426-2630, DOI: https://doi.org/10.2478/v10037-007-0012-9.

Export Citation

This content is open access.

Comments (0)

Please log in or register to comment.
Log in