Towards a generalization of information theory for hierarchical partitions
- Autores
- Perotti, Juan Ignacio; Almeira, Nahuel; Saracco, Fabio
- Año de publicación
- 2020
- Idioma
- inglés
- Tipo de recurso
- artículo
- Estado
- versión publicada
- Descripción
- Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this phenomenon, we introduce a generalization of information theory that works with hierarchical partitions. We begin revisiting the recently introduced hierarchical mutual information (HMI), and show that it can be written as a level by level summation of classical conditional mutual information terms. Then, we prove that the HMI is bounded from above by the corresponding hierarchical joint entropy. In this way, in analogy to the classical case, we derive hierarchical generalizations of many other classical information-theoretic quantities. In particular, we prove that, as opposed to its classical counterpart, the hierarchical generalization of the variation of information is not a metric distance, but it admits a transformation into one. Moreover, focusing on potential applications of the existing developments of the theory, we show how to adjust by chance the HMI. We also corroborate and analyze all the presented theoretical results with exhaustive numerical computations, and include an illustrative application example of the introduced formalism. Finally, we mention some open problems that should be eventually addressed for the proposed generalization of information theory to reach maturity.
Fil: Perotti, Juan Ignacio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Almeira, Nahuel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Saracco, Fabio. IMT School for Advanced Studies Lucca; Italia - Materia
-
INFORMATION THEORY
HIERARCHIES
COMPLEX SYSTEMS - Nivel de accesibilidad
- acceso abierto
- Condiciones de uso
- https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
- Repositorio
- Institución
- Consejo Nacional de Investigaciones Científicas y Técnicas
- OAI Identificador
- oai:ri.conicet.gov.ar:11336/143545
Ver los metadatos del registro completo
id |
CONICETDig_59f977071d2291a72aa4dcaa7ada2575 |
---|---|
oai_identifier_str |
oai:ri.conicet.gov.ar:11336/143545 |
network_acronym_str |
CONICETDig |
repository_id_str |
3498 |
network_name_str |
CONICET Digital (CONICET) |
spelling |
Towards a generalization of information theory for hierarchical partitionsPerotti, Juan IgnacioAlmeira, NahuelSaracco, FabioINFORMATION THEORYHIERARCHIESCOMPLEX SYSTEMShttps://purl.org/becyt/ford/1.3https://purl.org/becyt/ford/1Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this phenomenon, we introduce a generalization of information theory that works with hierarchical partitions. We begin revisiting the recently introduced hierarchical mutual information (HMI), and show that it can be written as a level by level summation of classical conditional mutual information terms. Then, we prove that the HMI is bounded from above by the corresponding hierarchical joint entropy. In this way, in analogy to the classical case, we derive hierarchical generalizations of many other classical information-theoretic quantities. In particular, we prove that, as opposed to its classical counterpart, the hierarchical generalization of the variation of information is not a metric distance, but it admits a transformation into one. Moreover, focusing on potential applications of the existing developments of the theory, we show how to adjust by chance the HMI. We also corroborate and analyze all the presented theoretical results with exhaustive numerical computations, and include an illustrative application example of the introduced formalism. Finally, we mention some open problems that should be eventually addressed for the proposed generalization of information theory to reach maturity.Fil: Perotti, Juan Ignacio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Almeira, Nahuel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Saracco, Fabio. IMT School for Advanced Studies Lucca; ItaliaAmerican Physical Society2020-06-02info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/143545Perotti, Juan Ignacio; Almeira, Nahuel; Saracco, Fabio; Towards a generalization of information theory for hierarchical partitions; American Physical Society; Physical Review E; 101; 6; 2-6-2020; 1-132470-00452470-0053CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://link.aps.org/doi/10.1103/PhysRevE.101.062148info:eu-repo/semantics/altIdentifier/doi/10.1103/PhysRevE.101.062148info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-29T10:47:52Zoai:ri.conicet.gov.ar:11336/143545instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-29 10:47:52.754CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse |
dc.title.none.fl_str_mv |
Towards a generalization of information theory for hierarchical partitions |
title |
Towards a generalization of information theory for hierarchical partitions |
spellingShingle |
Towards a generalization of information theory for hierarchical partitions Perotti, Juan Ignacio INFORMATION THEORY HIERARCHIES COMPLEX SYSTEMS |
title_short |
Towards a generalization of information theory for hierarchical partitions |
title_full |
Towards a generalization of information theory for hierarchical partitions |
title_fullStr |
Towards a generalization of information theory for hierarchical partitions |
title_full_unstemmed |
Towards a generalization of information theory for hierarchical partitions |
title_sort |
Towards a generalization of information theory for hierarchical partitions |
dc.creator.none.fl_str_mv |
Perotti, Juan Ignacio Almeira, Nahuel Saracco, Fabio |
author |
Perotti, Juan Ignacio |
author_facet |
Perotti, Juan Ignacio Almeira, Nahuel Saracco, Fabio |
author_role |
author |
author2 |
Almeira, Nahuel Saracco, Fabio |
author2_role |
author author |
dc.subject.none.fl_str_mv |
INFORMATION THEORY HIERARCHIES COMPLEX SYSTEMS |
topic |
INFORMATION THEORY HIERARCHIES COMPLEX SYSTEMS |
purl_subject.fl_str_mv |
https://purl.org/becyt/ford/1.3 https://purl.org/becyt/ford/1 |
dc.description.none.fl_txt_mv |
Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this phenomenon, we introduce a generalization of information theory that works with hierarchical partitions. We begin revisiting the recently introduced hierarchical mutual information (HMI), and show that it can be written as a level by level summation of classical conditional mutual information terms. Then, we prove that the HMI is bounded from above by the corresponding hierarchical joint entropy. In this way, in analogy to the classical case, we derive hierarchical generalizations of many other classical information-theoretic quantities. In particular, we prove that, as opposed to its classical counterpart, the hierarchical generalization of the variation of information is not a metric distance, but it admits a transformation into one. Moreover, focusing on potential applications of the existing developments of the theory, we show how to adjust by chance the HMI. We also corroborate and analyze all the presented theoretical results with exhaustive numerical computations, and include an illustrative application example of the introduced formalism. Finally, we mention some open problems that should be eventually addressed for the proposed generalization of information theory to reach maturity. Fil: Perotti, Juan Ignacio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Almeira, Nahuel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina Fil: Saracco, Fabio. IMT School for Advanced Studies Lucca; Italia |
description |
Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this phenomenon, we introduce a generalization of information theory that works with hierarchical partitions. We begin revisiting the recently introduced hierarchical mutual information (HMI), and show that it can be written as a level by level summation of classical conditional mutual information terms. Then, we prove that the HMI is bounded from above by the corresponding hierarchical joint entropy. In this way, in analogy to the classical case, we derive hierarchical generalizations of many other classical information-theoretic quantities. In particular, we prove that, as opposed to its classical counterpart, the hierarchical generalization of the variation of information is not a metric distance, but it admits a transformation into one. Moreover, focusing on potential applications of the existing developments of the theory, we show how to adjust by chance the HMI. We also corroborate and analyze all the presented theoretical results with exhaustive numerical computations, and include an illustrative application example of the introduced formalism. Finally, we mention some open problems that should be eventually addressed for the proposed generalization of information theory to reach maturity. |
publishDate |
2020 |
dc.date.none.fl_str_mv |
2020-06-02 |
dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion http://purl.org/coar/resource_type/c_6501 info:ar-repo/semantics/articulo |
format |
article |
status_str |
publishedVersion |
dc.identifier.none.fl_str_mv |
http://hdl.handle.net/11336/143545 Perotti, Juan Ignacio; Almeira, Nahuel; Saracco, Fabio; Towards a generalization of information theory for hierarchical partitions; American Physical Society; Physical Review E; 101; 6; 2-6-2020; 1-13 2470-0045 2470-0053 CONICET Digital CONICET |
url |
http://hdl.handle.net/11336/143545 |
identifier_str_mv |
Perotti, Juan Ignacio; Almeira, Nahuel; Saracco, Fabio; Towards a generalization of information theory for hierarchical partitions; American Physical Society; Physical Review E; 101; 6; 2-6-2020; 1-13 2470-0045 2470-0053 CONICET Digital CONICET |
dc.language.none.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
info:eu-repo/semantics/altIdentifier/url/https://link.aps.org/doi/10.1103/PhysRevE.101.062148 info:eu-repo/semantics/altIdentifier/doi/10.1103/PhysRevE.101.062148 |
dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
eu_rights_str_mv |
openAccess |
rights_invalid_str_mv |
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
dc.format.none.fl_str_mv |
application/pdf application/pdf |
dc.publisher.none.fl_str_mv |
American Physical Society |
publisher.none.fl_str_mv |
American Physical Society |
dc.source.none.fl_str_mv |
reponame:CONICET Digital (CONICET) instname:Consejo Nacional de Investigaciones Científicas y Técnicas |
reponame_str |
CONICET Digital (CONICET) |
collection |
CONICET Digital (CONICET) |
instname_str |
Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.name.fl_str_mv |
CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.mail.fl_str_mv |
dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar |
_version_ |
1844614523597619200 |
score |
13.070432 |