Towards a generalization of information theory for hierarchical partitions

Autores
Perotti, Juan Ignacio; Almeira, Nahuel; Saracco, Fabio
Año de publicación
2020
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this phenomenon, we introduce a generalization of information theory that works with hierarchical partitions. We begin revisiting the recently introduced hierarchical mutual information (HMI), and show that it can be written as a level by level summation of classical conditional mutual information terms. Then, we prove that the HMI is bounded from above by the corresponding hierarchical joint entropy. In this way, in analogy to the classical case, we derive hierarchical generalizations of many other classical information-theoretic quantities. In particular, we prove that, as opposed to its classical counterpart, the hierarchical generalization of the variation of information is not a metric distance, but it admits a transformation into one. Moreover, focusing on potential applications of the existing developments of the theory, we show how to adjust by chance the HMI. We also corroborate and analyze all the presented theoretical results with exhaustive numerical computations, and include an illustrative application example of the introduced formalism. Finally, we mention some open problems that should be eventually addressed for the proposed generalization of information theory to reach maturity.
Fil: Perotti, Juan Ignacio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Almeira, Nahuel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Saracco, Fabio. IMT School for Advanced Studies Lucca; Italia
Materia
INFORMATION THEORY
HIERARCHIES
COMPLEX SYSTEMS
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/143545
Acceder
id CONICETDig_59f977071d2291a72aa4dcaa7ada2575
oai_identifier_str oai:ri.conicet.gov.ar:11336/143545
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling Towards a generalization of information theory for hierarchical partitionsPerotti, Juan IgnacioAlmeira, NahuelSaracco, FabioINFORMATION THEORYHIERARCHIESCOMPLEX SYSTEMShttps://purl.org/becyt/ford/1.3https://purl.org/becyt/ford/1Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this phenomenon, we introduce a generalization of information theory that works with hierarchical partitions. We begin revisiting the recently introduced hierarchical mutual information (HMI), and show that it can be written as a level by level summation of classical conditional mutual information terms. Then, we prove that the HMI is bounded from above by the corresponding hierarchical joint entropy. In this way, in analogy to the classical case, we derive hierarchical generalizations of many other classical information-theoretic quantities. In particular, we prove that, as opposed to its classical counterpart, the hierarchical generalization of the variation of information is not a metric distance, but it admits a transformation into one. Moreover, focusing on potential applications of the existing developments of the theory, we show how to adjust by chance the HMI. We also corroborate and analyze all the presented theoretical results with exhaustive numerical computations, and include an illustrative application example of the introduced formalism. Finally, we mention some open problems that should be eventually addressed for the proposed generalization of information theory to reach maturity.Fil: Perotti, Juan Ignacio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Almeira, Nahuel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; ArgentinaFil: Saracco, Fabio. IMT School for Advanced Studies Lucca; ItaliaAmerican Physical Society2020-06-02info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/143545Perotti, Juan Ignacio; Almeira, Nahuel; Saracco, Fabio; Towards a generalization of information theory for hierarchical partitions; American Physical Society; Physical Review E; 101; 6; 2-6-2020; 1-132470-00452470-0053CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://link.aps.org/doi/10.1103/PhysRevE.101.062148info:eu-repo/semantics/altIdentifier/doi/10.1103/PhysRevE.101.062148info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-10-15T15:46:36Zoai:ri.conicet.gov.ar:11336/143545instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-10-15 15:46:37.237CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv Towards a generalization of information theory for hierarchical partitions
title Towards a generalization of information theory for hierarchical partitions
spellingShingle Towards a generalization of information theory for hierarchical partitions
Perotti, Juan Ignacio
INFORMATION THEORY
HIERARCHIES
COMPLEX SYSTEMS
title_short Towards a generalization of information theory for hierarchical partitions
title_full Towards a generalization of information theory for hierarchical partitions
title_fullStr Towards a generalization of information theory for hierarchical partitions
title_full_unstemmed Towards a generalization of information theory for hierarchical partitions
title_sort Towards a generalization of information theory for hierarchical partitions
dc.creator.none.fl_str_mv Perotti, Juan Ignacio
Almeira, Nahuel
Saracco, Fabio
author Perotti, Juan Ignacio
author_facet Perotti, Juan Ignacio
Almeira, Nahuel
Saracco, Fabio
author_role author
author2 Almeira, Nahuel
Saracco, Fabio
author2_role author
author
dc.subject.none.fl_str_mv INFORMATION THEORY
HIERARCHIES
COMPLEX SYSTEMS
topic INFORMATION THEORY
HIERARCHIES
COMPLEX SYSTEMS
purl_subject.fl_str_mv https://purl.org/becyt/ford/1.3
https://purl.org/becyt/ford/1
dc.description.none.fl_txt_mv Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this phenomenon, we introduce a generalization of information theory that works with hierarchical partitions. We begin revisiting the recently introduced hierarchical mutual information (HMI), and show that it can be written as a level by level summation of classical conditional mutual information terms. Then, we prove that the HMI is bounded from above by the corresponding hierarchical joint entropy. In this way, in analogy to the classical case, we derive hierarchical generalizations of many other classical information-theoretic quantities. In particular, we prove that, as opposed to its classical counterpart, the hierarchical generalization of the variation of information is not a metric distance, but it admits a transformation into one. Moreover, focusing on potential applications of the existing developments of the theory, we show how to adjust by chance the HMI. We also corroborate and analyze all the presented theoretical results with exhaustive numerical computations, and include an illustrative application example of the introduced formalism. Finally, we mention some open problems that should be eventually addressed for the proposed generalization of information theory to reach maturity.
Fil: Perotti, Juan Ignacio. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Almeira, Nahuel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina
Fil: Saracco, Fabio. IMT School for Advanced Studies Lucca; Italia
description Complex systems often exhibit multiple levels of organization covering a wide range of physical scales, so the study of the hierarchical decomposition of their structure and function is frequently convenient. To better understand this phenomenon, we introduce a generalization of information theory that works with hierarchical partitions. We begin revisiting the recently introduced hierarchical mutual information (HMI), and show that it can be written as a level by level summation of classical conditional mutual information terms. Then, we prove that the HMI is bounded from above by the corresponding hierarchical joint entropy. In this way, in analogy to the classical case, we derive hierarchical generalizations of many other classical information-theoretic quantities. In particular, we prove that, as opposed to its classical counterpart, the hierarchical generalization of the variation of information is not a metric distance, but it admits a transformation into one. Moreover, focusing on potential applications of the existing developments of the theory, we show how to adjust by chance the HMI. We also corroborate and analyze all the presented theoretical results with exhaustive numerical computations, and include an illustrative application example of the introduced formalism. Finally, we mention some open problems that should be eventually addressed for the proposed generalization of information theory to reach maturity.
publishDate 2020
dc.date.none.fl_str_mv 2020-06-02
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/143545
Perotti, Juan Ignacio; Almeira, Nahuel; Saracco, Fabio; Towards a generalization of information theory for hierarchical partitions; American Physical Society; Physical Review E; 101; 6; 2-6-2020; 1-13
2470-0045
2470-0053
CONICET Digital
CONICET
url http://hdl.handle.net/11336/143545
identifier_str_mv Perotti, Juan Ignacio; Almeira, Nahuel; Saracco, Fabio; Towards a generalization of information theory for hierarchical partitions; American Physical Society; Physical Review E; 101; 6; 2-6-2020; 1-13
2470-0045
2470-0053
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/https://link.aps.org/doi/10.1103/PhysRevE.101.062148
info:eu-repo/semantics/altIdentifier/doi/10.1103/PhysRevE.101.062148
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv American Physical Society
publisher.none.fl_str_mv American Physical Society
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1846083562710761472
score 13.22299

Publicaciones similares