Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization

Autores
Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos
Año de publicación
2018
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for obtaining good performances regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the extit{Q}-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions.
Fil: Barsce, Juan Cruz. Universidad Tecnológica Nacional; Argentina
Fil: Palombarini, Jorge Andrés. Universidad Tecnológica Nacional; Argentina
Fil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; Argentina
Materia
REINFORCEMENT LEARNING
AUTONOMOUS SYSTEMS
BAYESIAN OPTIMIZATION
HYPER-PARAMETERS SETTING
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/86940

id CONICETDig_095f027cf5c32459e244ad921695984a
oai_identifier_str oai:ri.conicet.gov.ar:11336/86940
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian OptimizationBarsce, Juan CruzPalombarini, Jorge AndrésMartínez, Ernesto CarlosREINFORCEMENT LEARNINGAUTONOMOUS SYSTEMSBAYESIAN OPTIMIZATIONHYPER-PARAMETERS SETTINGhttps://purl.org/becyt/ford/2.2https://purl.org/becyt/ford/2With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for obtaining good performances regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the extit{Q}-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions.Fil: Barsce, Juan Cruz. Universidad Tecnológica Nacional; ArgentinaFil: Palombarini, Jorge Andrés. Universidad Tecnológica Nacional; ArgentinaFil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; ArgentinaCLEI (Latin-american Center for Informatics Studies)2018info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/86940Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos; Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization; CLEI (Latin-american Center for Informatics Studies); CLEI Electronic Journal; 21; 2; 2018; 1-220717-5000CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/http://www.clei.org/cleiej-beta/index.php/cleiej/article/view/33info:eu-repo/semantics/altIdentifier/doi/10.19153/cleiej.21.2.1info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-10-15T15:11:44Zoai:ri.conicet.gov.ar:11336/86940instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-10-15 15:11:44.688CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
title Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
spellingShingle Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
Barsce, Juan Cruz
REINFORCEMENT LEARNING
AUTONOMOUS SYSTEMS
BAYESIAN OPTIMIZATION
HYPER-PARAMETERS SETTING
title_short Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
title_full Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
title_fullStr Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
title_full_unstemmed Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
title_sort Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
dc.creator.none.fl_str_mv Barsce, Juan Cruz
Palombarini, Jorge Andrés
Martínez, Ernesto Carlos
author Barsce, Juan Cruz
author_facet Barsce, Juan Cruz
Palombarini, Jorge Andrés
Martínez, Ernesto Carlos
author_role author
author2 Palombarini, Jorge Andrés
Martínez, Ernesto Carlos
author2_role author
author
dc.subject.none.fl_str_mv REINFORCEMENT LEARNING
AUTONOMOUS SYSTEMS
BAYESIAN OPTIMIZATION
HYPER-PARAMETERS SETTING
topic REINFORCEMENT LEARNING
AUTONOMOUS SYSTEMS
BAYESIAN OPTIMIZATION
HYPER-PARAMETERS SETTING
purl_subject.fl_str_mv https://purl.org/becyt/ford/2.2
https://purl.org/becyt/ford/2
dc.description.none.fl_txt_mv With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for obtaining good performances regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the extit{Q}-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions.
Fil: Barsce, Juan Cruz. Universidad Tecnológica Nacional; Argentina
Fil: Palombarini, Jorge Andrés. Universidad Tecnológica Nacional; Argentina
Fil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; Argentina
description With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for obtaining good performances regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the extit{Q}-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions.
publishDate 2018
dc.date.none.fl_str_mv 2018
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/86940
Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos; Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization; CLEI (Latin-american Center for Informatics Studies); CLEI Electronic Journal; 21; 2; 2018; 1-22
0717-5000
CONICET Digital
CONICET
url http://hdl.handle.net/11336/86940
identifier_str_mv Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos; Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization; CLEI (Latin-american Center for Informatics Studies); CLEI Electronic Journal; 21; 2; 2018; 1-22
0717-5000
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/http://www.clei.org/cleiej-beta/index.php/cleiej/article/view/33
info:eu-repo/semantics/altIdentifier/doi/10.19153/cleiej.21.2.1
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv CLEI (Latin-american Center for Informatics Studies)
publisher.none.fl_str_mv CLEI (Latin-american Center for Informatics Studies)
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1846083265620869120
score 12.891075