Post-training discriminative pruning for RBMs

Autores
Sánchez Gutiérrez, Máximo; Albornoz, Enrique Marcelo; Rufiner, Hugo Leonardo; Close, John Goddard
Año de publicación
2017
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
One of the major challenges in the area of artificial neural networks is the identification of a suitable architecture for a specific problem. Choosing an unsuitable topology can exponentially increase the training cost, and even hinder network convergence. On the other hand, recent research indicates that larger or deeper nets can map the problem features into a more appropriate space, and thereby improve the classification process, thus leading to an apparent dichotomy. In this regard, it is interesting to inquire whether independent measures, such as mutual information, could provide a clue to finding the most discriminative neurons in a network. In the present work we explore this question in the context of Restricted Boltzmann Machines, by employing different measures to realize post-training pruning. The neurons which are determined by each measure to be the most discriminative, are combined and a classifier is applied to the ensuing network to determine its usefulness. We find that two measures in particular seem to be good indicators of the most discriminative neurons, producing savings of generally more than 50% of the neurons, while maintaining an acceptable error rate. Further, it is borne out that starting with a larger network architecture and then pruning is more advantageous than using a smaller network to begin with. Finally, a quantitative index is introduced which can provide information on choosing a suitable pruned network.
Fil: Sánchez Gutiérrez, Máximo. Universidad Autónoma Metropolitana; México
Fil: Albornoz, Enrique Marcelo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina
Fil: Rufiner, Hugo Leonardo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina. Universidad Nacional de Entre Ríos; Argentina
Fil: Close, John Goddard. Universidad Autónoma Metropolitana; México
Materia
RESTRICTED BOLTZMANN MACHINES
DISCRIMINATIVE INFORMATION
PRUNING
EMOTION CLASSIFICATION
PHONEME CLASSIFICATION
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/47032
Acceder
id CONICETDig_e42b0114406be73357a0d2eb425d8362
oai_identifier_str oai:ri.conicet.gov.ar:11336/47032
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling Post-training discriminative pruning for RBMsSánchez Gutiérrez, MáximoAlbornoz, Enrique MarceloRufiner, Hugo LeonardoClose, John GoddardRESTRICTED BOLTZMANN MACHINESDISCRIMINATIVE INFORMATIONPRUNINGEMOTION CLASSIFICATIONPHONEME CLASSIFICATIONhttps://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1One of the major challenges in the area of artificial neural networks is the identification of a suitable architecture for a specific problem. Choosing an unsuitable topology can exponentially increase the training cost, and even hinder network convergence. On the other hand, recent research indicates that larger or deeper nets can map the problem features into a more appropriate space, and thereby improve the classification process, thus leading to an apparent dichotomy. In this regard, it is interesting to inquire whether independent measures, such as mutual information, could provide a clue to finding the most discriminative neurons in a network. In the present work we explore this question in the context of Restricted Boltzmann Machines, by employing different measures to realize post-training pruning. The neurons which are determined by each measure to be the most discriminative, are combined and a classifier is applied to the ensuing network to determine its usefulness. We find that two measures in particular seem to be good indicators of the most discriminative neurons, producing savings of generally more than 50% of the neurons, while maintaining an acceptable error rate. Further, it is borne out that starting with a larger network architecture and then pruning is more advantageous than using a smaller network to begin with. Finally, a quantitative index is introduced which can provide information on choosing a suitable pruned network.Fil: Sánchez Gutiérrez, Máximo. Universidad Autónoma Metropolitana; MéxicoFil: Albornoz, Enrique Marcelo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; ArgentinaFil: Rufiner, Hugo Leonardo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina. Universidad Nacional de Entre Ríos; ArgentinaFil: Close, John Goddard. Universidad Autónoma Metropolitana; MéxicoSpringer2017-08info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/47032Sánchez Gutiérrez, Máximo; Albornoz, Enrique Marcelo; Rufiner, Hugo Leonardo; Close, John Goddard; Post-training discriminative pruning for RBMs; Springer; Soft Computing; 8-20171432-76431433-7479CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/http://link.springer.com/10.1007/s00500-017-2784-3info:eu-repo/semantics/altIdentifier/doi/10.1007/s00500-017-2784-3info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-29T09:40:09Zoai:ri.conicet.gov.ar:11336/47032instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-29 09:40:09.649CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv Post-training discriminative pruning for RBMs
title Post-training discriminative pruning for RBMs
spellingShingle Post-training discriminative pruning for RBMs
Sánchez Gutiérrez, Máximo
RESTRICTED BOLTZMANN MACHINES
DISCRIMINATIVE INFORMATION
PRUNING
EMOTION CLASSIFICATION
PHONEME CLASSIFICATION
title_short Post-training discriminative pruning for RBMs
title_full Post-training discriminative pruning for RBMs
title_fullStr Post-training discriminative pruning for RBMs
title_full_unstemmed Post-training discriminative pruning for RBMs
title_sort Post-training discriminative pruning for RBMs
dc.creator.none.fl_str_mv Sánchez Gutiérrez, Máximo
Albornoz, Enrique Marcelo
Rufiner, Hugo Leonardo
Close, John Goddard
author Sánchez Gutiérrez, Máximo
author_facet Sánchez Gutiérrez, Máximo
Albornoz, Enrique Marcelo
Rufiner, Hugo Leonardo
Close, John Goddard
author_role author
author2 Albornoz, Enrique Marcelo
Rufiner, Hugo Leonardo
Close, John Goddard
author2_role author
author
author
dc.subject.none.fl_str_mv RESTRICTED BOLTZMANN MACHINES
DISCRIMINATIVE INFORMATION
PRUNING
EMOTION CLASSIFICATION
PHONEME CLASSIFICATION
topic RESTRICTED BOLTZMANN MACHINES
DISCRIMINATIVE INFORMATION
PRUNING
EMOTION CLASSIFICATION
PHONEME CLASSIFICATION
purl_subject.fl_str_mv https://purl.org/becyt/ford/1.2
https://purl.org/becyt/ford/1
dc.description.none.fl_txt_mv One of the major challenges in the area of artificial neural networks is the identification of a suitable architecture for a specific problem. Choosing an unsuitable topology can exponentially increase the training cost, and even hinder network convergence. On the other hand, recent research indicates that larger or deeper nets can map the problem features into a more appropriate space, and thereby improve the classification process, thus leading to an apparent dichotomy. In this regard, it is interesting to inquire whether independent measures, such as mutual information, could provide a clue to finding the most discriminative neurons in a network. In the present work we explore this question in the context of Restricted Boltzmann Machines, by employing different measures to realize post-training pruning. The neurons which are determined by each measure to be the most discriminative, are combined and a classifier is applied to the ensuing network to determine its usefulness. We find that two measures in particular seem to be good indicators of the most discriminative neurons, producing savings of generally more than 50% of the neurons, while maintaining an acceptable error rate. Further, it is borne out that starting with a larger network architecture and then pruning is more advantageous than using a smaller network to begin with. Finally, a quantitative index is introduced which can provide information on choosing a suitable pruned network.
Fil: Sánchez Gutiérrez, Máximo. Universidad Autónoma Metropolitana; México
Fil: Albornoz, Enrique Marcelo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina
Fil: Rufiner, Hugo Leonardo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional. Universidad Nacional del Litoral. Facultad de Ingeniería y Ciencias Hídricas. Instituto de Investigación en Señales, Sistemas e Inteligencia Computacional; Argentina. Universidad Nacional de Entre Ríos; Argentina
Fil: Close, John Goddard. Universidad Autónoma Metropolitana; México
description One of the major challenges in the area of artificial neural networks is the identification of a suitable architecture for a specific problem. Choosing an unsuitable topology can exponentially increase the training cost, and even hinder network convergence. On the other hand, recent research indicates that larger or deeper nets can map the problem features into a more appropriate space, and thereby improve the classification process, thus leading to an apparent dichotomy. In this regard, it is interesting to inquire whether independent measures, such as mutual information, could provide a clue to finding the most discriminative neurons in a network. In the present work we explore this question in the context of Restricted Boltzmann Machines, by employing different measures to realize post-training pruning. The neurons which are determined by each measure to be the most discriminative, are combined and a classifier is applied to the ensuing network to determine its usefulness. We find that two measures in particular seem to be good indicators of the most discriminative neurons, producing savings of generally more than 50% of the neurons, while maintaining an acceptable error rate. Further, it is borne out that starting with a larger network architecture and then pruning is more advantageous than using a smaller network to begin with. Finally, a quantitative index is introduced which can provide information on choosing a suitable pruned network.
publishDate 2017
dc.date.none.fl_str_mv 2017-08
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/47032
Sánchez Gutiérrez, Máximo; Albornoz, Enrique Marcelo; Rufiner, Hugo Leonardo; Close, John Goddard; Post-training discriminative pruning for RBMs; Springer; Soft Computing; 8-2017
1432-7643
1433-7479
CONICET Digital
CONICET
url http://hdl.handle.net/11336/47032
identifier_str_mv Sánchez Gutiérrez, Máximo; Albornoz, Enrique Marcelo; Rufiner, Hugo Leonardo; Close, John Goddard; Post-training discriminative pruning for RBMs; Springer; Soft Computing; 8-2017
1432-7643
1433-7479
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/http://link.springer.com/10.1007/s00500-017-2784-3
info:eu-repo/semantics/altIdentifier/doi/10.1007/s00500-017-2784-3
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Springer
publisher.none.fl_str_mv Springer
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1844613270623748096
score 13.070432

Publicaciones similares