A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning
- Autores
- Barsce, Juan Cruz; Palombarini, Jorge Andrés; Martínez, Ernesto Carlos
- Año de publicación
- 2019
- Idioma
- inglés
- Tipo de recurso
- documento de conferencia
- Estado
- versión publicada
- Descripción
- Optimization of hyper-parameters in reinforcement learning (RL) algorithms is a key task, because they determine how the agent will learn its policy by interacting with its environment, and thus what data is gathered. In this work, an approach that uses Bayesian optimization to perform a two-step optimization is proposed: first, categorical RL structure hyper-parameters are taken as binary variables and optimized with an acquisition function tailored for such variables. Then, at a lower level of abstraction, solution-level hyper-parameters are optimized by resorting to the expected improvement acquisition function, while using the best categorical hyper-parameters found in the optimization at theupper-level of abstraction. This two-tier approach is validated in a simulated control task. Results obtained are promising and open the way formore user-independent applications of reinforcement learning.
Fil: Barsce, Juan Cruz. Universidad Tecnologica Nacional. Facultad Regional Villa Maria; Argentina
Fil: Palombarini, Jorge Andrés. Universidad Tecnologica Nacional. Facultad Regional Villa Maria; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Centro de Investigaciones y Transferencia de Villa María. Universidad Nacional de Villa María. Centro de Investigaciones y Transferencia de Villa María; Argentina
Fil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; Argentina
XX Simposio Argentino de Inteligencia Artificial
Salta
Argentina
Sociedad Argentina de Informática - Materia
-
AUTONOMOUS LEARNING
BAYESIAN OPTIMIZATION
DEEP LEARNING
REINFORCEMENT LEARNING - Nivel de accesibilidad
- acceso abierto
- Condiciones de uso
- https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
- Repositorio
- Institución
- Consejo Nacional de Investigaciones Científicas y Técnicas
- OAI Identificador
- oai:ri.conicet.gov.ar:11336/182932
Ver los metadatos del registro completo
id |
CONICETDig_880332495c829a447afe95e9b8f0e2d7 |
---|---|
oai_identifier_str |
oai:ri.conicet.gov.ar:11336/182932 |
network_acronym_str |
CONICETDig |
repository_id_str |
3498 |
network_name_str |
CONICET Digital (CONICET) |
spelling |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learningBarsce, Juan CruzPalombarini, Jorge AndrésMartínez, Ernesto CarlosAUTONOMOUS LEARNINGBAYESIAN OPTIMIZATIONDEEP LEARNINGREINFORCEMENT LEARNINGhttps://purl.org/becyt/ford/2.2https://purl.org/becyt/ford/2Optimization of hyper-parameters in reinforcement learning (RL) algorithms is a key task, because they determine how the agent will learn its policy by interacting with its environment, and thus what data is gathered. In this work, an approach that uses Bayesian optimization to perform a two-step optimization is proposed: first, categorical RL structure hyper-parameters are taken as binary variables and optimized with an acquisition function tailored for such variables. Then, at a lower level of abstraction, solution-level hyper-parameters are optimized by resorting to the expected improvement acquisition function, while using the best categorical hyper-parameters found in the optimization at theupper-level of abstraction. This two-tier approach is validated in a simulated control task. Results obtained are promising and open the way formore user-independent applications of reinforcement learning.Fil: Barsce, Juan Cruz. Universidad Tecnologica Nacional. Facultad Regional Villa Maria; ArgentinaFil: Palombarini, Jorge Andrés. Universidad Tecnologica Nacional. Facultad Regional Villa Maria; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Centro de Investigaciones y Transferencia de Villa María. Universidad Nacional de Villa María. Centro de Investigaciones y Transferencia de Villa María; ArgentinaFil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; ArgentinaXX Simposio Argentino de Inteligencia ArtificialSaltaArgentinaSociedad Argentina de InformáticaSociedad Argentina de Informática e Investigación Operativa2019info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/conferenceObjectSimposioJournalhttp://purl.org/coar/resource_type/c_5794info:ar-repo/semantics/documentoDeConferenciaapplication/pdfapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/182932A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning; XX Simposio Argentino de Inteligencia Artificial; Salta; Argentina; 2019; 32-382451-7585CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/http://sedici.unlp.edu.ar/handle/10915/87851Nacionalinfo:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-10-15T15:12:18Zoai:ri.conicet.gov.ar:11336/182932instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-10-15 15:12:19.18CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse |
dc.title.none.fl_str_mv |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning |
title |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning |
spellingShingle |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning Barsce, Juan Cruz AUTONOMOUS LEARNING BAYESIAN OPTIMIZATION DEEP LEARNING REINFORCEMENT LEARNING |
title_short |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning |
title_full |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning |
title_fullStr |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning |
title_full_unstemmed |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning |
title_sort |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning |
dc.creator.none.fl_str_mv |
Barsce, Juan Cruz Palombarini, Jorge Andrés Martínez, Ernesto Carlos |
author |
Barsce, Juan Cruz |
author_facet |
Barsce, Juan Cruz Palombarini, Jorge Andrés Martínez, Ernesto Carlos |
author_role |
author |
author2 |
Palombarini, Jorge Andrés Martínez, Ernesto Carlos |
author2_role |
author author |
dc.subject.none.fl_str_mv |
AUTONOMOUS LEARNING BAYESIAN OPTIMIZATION DEEP LEARNING REINFORCEMENT LEARNING |
topic |
AUTONOMOUS LEARNING BAYESIAN OPTIMIZATION DEEP LEARNING REINFORCEMENT LEARNING |
purl_subject.fl_str_mv |
https://purl.org/becyt/ford/2.2 https://purl.org/becyt/ford/2 |
dc.description.none.fl_txt_mv |
Optimization of hyper-parameters in reinforcement learning (RL) algorithms is a key task, because they determine how the agent will learn its policy by interacting with its environment, and thus what data is gathered. In this work, an approach that uses Bayesian optimization to perform a two-step optimization is proposed: first, categorical RL structure hyper-parameters are taken as binary variables and optimized with an acquisition function tailored for such variables. Then, at a lower level of abstraction, solution-level hyper-parameters are optimized by resorting to the expected improvement acquisition function, while using the best categorical hyper-parameters found in the optimization at theupper-level of abstraction. This two-tier approach is validated in a simulated control task. Results obtained are promising and open the way formore user-independent applications of reinforcement learning. Fil: Barsce, Juan Cruz. Universidad Tecnologica Nacional. Facultad Regional Villa Maria; Argentina Fil: Palombarini, Jorge Andrés. Universidad Tecnologica Nacional. Facultad Regional Villa Maria; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Centro de Investigaciones y Transferencia de Villa María. Universidad Nacional de Villa María. Centro de Investigaciones y Transferencia de Villa María; Argentina Fil: Martínez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Santa Fe. Instituto de Desarrollo y Diseño. Universidad Tecnológica Nacional. Facultad Regional Santa Fe. Instituto de Desarrollo y Diseño; Argentina XX Simposio Argentino de Inteligencia Artificial Salta Argentina Sociedad Argentina de Informática |
description |
Optimization of hyper-parameters in reinforcement learning (RL) algorithms is a key task, because they determine how the agent will learn its policy by interacting with its environment, and thus what data is gathered. In this work, an approach that uses Bayesian optimization to perform a two-step optimization is proposed: first, categorical RL structure hyper-parameters are taken as binary variables and optimized with an acquisition function tailored for such variables. Then, at a lower level of abstraction, solution-level hyper-parameters are optimized by resorting to the expected improvement acquisition function, while using the best categorical hyper-parameters found in the optimization at theupper-level of abstraction. This two-tier approach is validated in a simulated control task. Results obtained are promising and open the way formore user-independent applications of reinforcement learning. |
publishDate |
2019 |
dc.date.none.fl_str_mv |
2019 |
dc.type.none.fl_str_mv |
info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/conferenceObject Simposio Journal http://purl.org/coar/resource_type/c_5794 info:ar-repo/semantics/documentoDeConferencia |
status_str |
publishedVersion |
format |
conferenceObject |
dc.identifier.none.fl_str_mv |
http://hdl.handle.net/11336/182932 A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning; XX Simposio Argentino de Inteligencia Artificial; Salta; Argentina; 2019; 32-38 2451-7585 CONICET Digital CONICET |
url |
http://hdl.handle.net/11336/182932 |
identifier_str_mv |
A hierarchical two-tier approach to hyper-parameter optimization in reinforcement learning; XX Simposio Argentino de Inteligencia Artificial; Salta; Argentina; 2019; 32-38 2451-7585 CONICET Digital CONICET |
dc.language.none.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
info:eu-repo/semantics/altIdentifier/url/http://sedici.unlp.edu.ar/handle/10915/87851 |
dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
eu_rights_str_mv |
openAccess |
rights_invalid_str_mv |
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
dc.format.none.fl_str_mv |
application/pdf application/pdf application/pdf |
dc.coverage.none.fl_str_mv |
Nacional |
dc.publisher.none.fl_str_mv |
Sociedad Argentina de Informática e Investigación Operativa |
publisher.none.fl_str_mv |
Sociedad Argentina de Informática e Investigación Operativa |
dc.source.none.fl_str_mv |
reponame:CONICET Digital (CONICET) instname:Consejo Nacional de Investigaciones Científicas y Técnicas |
reponame_str |
CONICET Digital (CONICET) |
collection |
CONICET Digital (CONICET) |
instname_str |
Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.name.fl_str_mv |
CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.mail.fl_str_mv |
dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar |
_version_ |
1846083271307296768 |
score |
12.891075 |