Name
Abstract | ||
|---|---|---|
The measure-theoretic definition of Kullback-Leibler relative-entropy (or simply KL-entropy) plays a basic role in defining various classical information measures on general spaces. Entropy, mutual information and conditional forms of entropy can be expressed in terms of KL-entropy and hence properties of their measure-theoretic analogs will follow from those of measure-theoretic KL-entropy. These measure-theoretic definitions are key to extending the ergodic theorems of information theory to non-discrete cases. A fundamental theorem in this respect is the Gelfand-Yaglom-Perez (GYP) Theorem [M.S. Pinsker, Information and Information Stability of Random Variables and Process, 1960, Holden-Day, San Francisco, CA (English ed., 1964, translated and edited by Amiel Feinstein), Theorem. 2.4.2] which states that measure-theoretic relative-entropy equals the supremum of relative-entropies over all measurable partitions. This paper states and proves the GYP-theorem for Renyi relative-entropy of order greater than one. Consequently, the result can be easily extended to Tsallis relative-entropy. |
Year | DOI | Venue |
|---|---|---|
2007 | 10.1016/j.ins.2007.07.017 | Inf. Sci. |
Keywords | Field | DocType |
various classical information measure,kullback-leibler relative-entropy,measure-theoretic definition,tsallis relative-entropy,renyi relative-entropy,measure-theoretic analog,mutual information,generalized relative entropy functionals,measure-theoretic relative-entropy,gelfand-yaglom-perez theorem,information theory,measure-theoretic kl-entropy,random variable,ergodic theorem,general relativity,relative entropy,kullback leibler | Discrete mathematics,No-go theorem,Generalized relative entropy,Joint quantum entropy,Information theory and measure theory,Information diagram,Fundamental theorem,Joint entropy,Conditional entropy,Mathematics | Journal |
Volume | Issue | ISSN |
177 | 24 | 0020-0255 |
Citations | PageRank | References |
5 | 0.92 | 5 |
Authors | ||
3 | ||
Name | Order | Citations | PageRank |
|---|---|---|---|
| Ambedkar Dukkipati | 1 | 86 | 29.68 |
| Shalabh Bhatnagar | 2 | 802 | 87.78 |
| M. Narasimha Murty | 3 | 824 | 86.07 |