Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
- Autores
- Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos
- Año de publicación
- 2018
- Idioma
- inglés
- Tipo de recurso
- artículo
- Estado
- versión publicada
- Descripción
- With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for obtaining good performances regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the extit{Q}-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions.
Fil: Barsce, Juan Cruz. Universidad Tecnológica Nacional; Argentina
Fil: Palombarini, Jorge Andrés. Universidad Tecnológica Nacional; Argentina
Fil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; Argentina - Materia
-
REINFORCEMENT LEARNING
AUTONOMOUS SYSTEMS
BAYESIAN OPTIMIZATION
HYPER-PARAMETERS SETTING - Nivel de accesibilidad
- acceso abierto
- Condiciones de uso
- https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
- Repositorio
- Institución
- Consejo Nacional de Investigaciones Científicas y Técnicas
- OAI Identificador
- oai:ri.conicet.gov.ar:11336/86940
Ver los metadatos del registro completo
id |
CONICETDig_095f027cf5c32459e244ad921695984a |
---|---|
oai_identifier_str |
oai:ri.conicet.gov.ar:11336/86940 |
network_acronym_str |
CONICETDig |
repository_id_str |
3498 |
network_name_str |
CONICET Digital (CONICET) |
spelling |
Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian OptimizationBarsce, Juan CruzPalombarini, Jorge AndrésMartínez, Ernesto CarlosREINFORCEMENT LEARNINGAUTONOMOUS SYSTEMSBAYESIAN OPTIMIZATIONHYPER-PARAMETERS SETTINGhttps://purl.org/becyt/ford/2.2https://purl.org/becyt/ford/2With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for obtaining good performances regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the extit{Q}-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions.Fil: Barsce, Juan Cruz. Universidad Tecnológica Nacional; ArgentinaFil: Palombarini, Jorge Andrés. Universidad Tecnológica Nacional; ArgentinaFil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; ArgentinaCLEI (Latin-american Center for Informatics Studies)2018info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/86940Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos; Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization; CLEI (Latin-american Center for Informatics Studies); CLEI Electronic Journal; 21; 2; 2018; 1-220717-5000CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/http://www.clei.org/cleiej-beta/index.php/cleiej/article/view/33info:eu-repo/semantics/altIdentifier/doi/10.19153/cleiej.21.2.1info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-10-15T15:11:44Zoai:ri.conicet.gov.ar:11336/86940instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-10-15 15:11:44.688CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse |
dc.title.none.fl_str_mv |
Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization |
title |
Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization |
spellingShingle |
Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization Barsce, Juan Cruz REINFORCEMENT LEARNING AUTONOMOUS SYSTEMS BAYESIAN OPTIMIZATION HYPER-PARAMETERS SETTING |
title_short |
Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization |
title_full |
Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization |
title_fullStr |
Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization |
title_full_unstemmed |
Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization |
title_sort |
Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization |
dc.creator.none.fl_str_mv |
Barsce, Juan Cruz Palombarini, Jorge Andrés Martínez, Ernesto Carlos |
author |
Barsce, Juan Cruz |
author_facet |
Barsce, Juan Cruz Palombarini, Jorge Andrés Martínez, Ernesto Carlos |
author_role |
author |
author2 |
Palombarini, Jorge Andrés Martínez, Ernesto Carlos |
author2_role |
author author |
dc.subject.none.fl_str_mv |
REINFORCEMENT LEARNING AUTONOMOUS SYSTEMS BAYESIAN OPTIMIZATION HYPER-PARAMETERS SETTING |
topic |
REINFORCEMENT LEARNING AUTONOMOUS SYSTEMS BAYESIAN OPTIMIZATION HYPER-PARAMETERS SETTING |
purl_subject.fl_str_mv |
https://purl.org/becyt/ford/2.2 https://purl.org/becyt/ford/2 |
dc.description.none.fl_txt_mv |
With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for obtaining good performances regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the extit{Q}-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions. Fil: Barsce, Juan Cruz. Universidad Tecnológica Nacional; Argentina Fil: Palombarini, Jorge Andrés. Universidad Tecnológica Nacional; Argentina Fil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; Argentina |
description |
With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for obtaining good performances regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the extit{Q}-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions. |
publishDate |
2018 |
dc.date.none.fl_str_mv |
2018 |
dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion http://purl.org/coar/resource_type/c_6501 info:ar-repo/semantics/articulo |
format |
article |
status_str |
publishedVersion |
dc.identifier.none.fl_str_mv |
http://hdl.handle.net/11336/86940 Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos; Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization; CLEI (Latin-american Center for Informatics Studies); CLEI Electronic Journal; 21; 2; 2018; 1-22 0717-5000 CONICET Digital CONICET |
url |
http://hdl.handle.net/11336/86940 |
identifier_str_mv |
Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos; Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization; CLEI (Latin-american Center for Informatics Studies); CLEI Electronic Journal; 21; 2; 2018; 1-22 0717-5000 CONICET Digital CONICET |
dc.language.none.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
info:eu-repo/semantics/altIdentifier/url/http://www.clei.org/cleiej-beta/index.php/cleiej/article/view/33 info:eu-repo/semantics/altIdentifier/doi/10.19153/cleiej.21.2.1 |
dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
eu_rights_str_mv |
openAccess |
rights_invalid_str_mv |
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
dc.format.none.fl_str_mv |
application/pdf application/pdf |
dc.publisher.none.fl_str_mv |
CLEI (Latin-american Center for Informatics Studies) |
publisher.none.fl_str_mv |
CLEI (Latin-american Center for Informatics Studies) |
dc.source.none.fl_str_mv |
reponame:CONICET Digital (CONICET) instname:Consejo Nacional de Investigaciones Científicas y Técnicas |
reponame_str |
CONICET Digital (CONICET) |
collection |
CONICET Digital (CONICET) |
instname_str |
Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.name.fl_str_mv |
CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.mail.fl_str_mv |
dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar |
_version_ |
1846083265620869120 |
score |
12.891075 |