Paper Info
Title
The Channel Multivariate Entropy Triangle and Balance Equation.
Abstract
In this paper we use information-theoretic measures to provide a theory and tools to analyze the flow of information from a discrete, multivariate source of information $overline X$ to a discrete, multivariate sink of information $overline Y$ joined by a distribution $P_{overline X overline Y}$. The first contribution is a decomposition of the maximal potential entropy of $(overline X, overline Y)$ that we call a balance equation, that can also be split into decompositions for the entropies of $overline X$ and $overline Y$ respectively. Such balance equations accept normalizations that allow them to be represented in de Finetti entropy diagrams, our second contribution. The most important of these, the aggregate Channel Multivariate Entropy Triangle CMET is an exploratory tool to assess the efficiency of multivariate channels. We also present a practical contribution in the application of these balance equations and diagrams to the assessment of information transfer efficiency for PCA and ICA as feature transformation and selection procedures in machine learning applications.
Year
Venue
Field
2017
arXiv: Information Theory
Applied mathematics,Information flow (information theory),Feature transformation,Discrete mathematics,Information transfer,Multivariate statistics,Communication channel,Balance equation,Overline,Mathematics
DocType
Volume
Citations 
Journal
abs/1711.11510
0
PageRank 
References 
Authors
0.34
0
2
Name
Order
Citations
PageRank
Francisco J. Valverde-Albacete111620.84
Carmen Peláez-moreno213022.07