Learning effective state-feedback controllers through efficient multilevel importance samplers

Autores
Menchón, Silvia Adriana; Kappen, Hilbert Johan
Año de publicación
2019
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
Monte Carlo sampling can be used to estimate the solution of path integral control problems, which are a restricted class of nonlinear control problems with arbitrary dynamics and state cost, but with a linear dependence of the control on the dynamics and quadratic control cost. Although importance sampling is used to improve numerical computations, the effective sample size may still be low or many samples could be required. In this work, we propose a method to learn effective state-feedback controllers for nonlinear stochastic control problems based on multilevel importance samplers. In particular, we focus on the question of how to compute effective importance samplers considering a multigrid scenario. We test our algorithm in finite horizon control problems based on Lorenz-96 model with chaotic and non-chaotic behaviour, showing, in all cases, that our multigrid implementation reduces the computational time and improves the effective sample size.
Fil: Menchón, Silvia Adriana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Radboud Universiteit Nijmegen. Donders Instituto Brain Cognition and Behavior. SNN Machine Learning Group; Países Bajos
Fil: Kappen, Hilbert Johan. Radboud Universiteit Nijmegen. Donders Instituto Brain Cognition and Behavior. SNN Machine Learning Group; Países Bajos
Materia
IMPORTANCE SAMPLING
MULTILEVEL MONTE CARLO METHOD
PATH INTEGRAL CONTROL PROBLEMS
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/91650
Acceder
id CONICETDig_be612f58b05b4d4f07b603454f383d55
oai_identifier_str oai:ri.conicet.gov.ar:11336/91650
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling Learning effective state-feedback controllers through efficient multilevel importance samplersMenchón, Silvia AdrianaKappen, Hilbert JohanIMPORTANCE SAMPLINGMULTILEVEL MONTE CARLO METHODPATH INTEGRAL CONTROL PROBLEMShttps://purl.org/becyt/ford/1.3https://purl.org/becyt/ford/1Monte Carlo sampling can be used to estimate the solution of path integral control problems, which are a restricted class of nonlinear control problems with arbitrary dynamics and state cost, but with a linear dependence of the control on the dynamics and quadratic control cost. Although importance sampling is used to improve numerical computations, the effective sample size may still be low or many samples could be required. In this work, we propose a method to learn effective state-feedback controllers for nonlinear stochastic control problems based on multilevel importance samplers. In particular, we focus on the question of how to compute effective importance samplers considering a multigrid scenario. We test our algorithm in finite horizon control problems based on Lorenz-96 model with chaotic and non-chaotic behaviour, showing, in all cases, that our multigrid implementation reduces the computational time and improves the effective sample size.Fil: Menchón, Silvia Adriana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Radboud Universiteit Nijmegen. Donders Instituto Brain Cognition and Behavior. SNN Machine Learning Group; Países BajosFil: Kappen, Hilbert Johan. Radboud Universiteit Nijmegen. Donders Instituto Brain Cognition and Behavior. SNN Machine Learning Group; Países BajosTaylor & Francis Ltd2019-12-21info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/91650Menchón, Silvia Adriana; Kappen, Hilbert Johan; Learning effective state-feedback controllers through efficient multilevel importance samplers; Taylor & Francis Ltd; International Journal Of Control; 92; 12; 21-12-2019; 2776-27830020-71791366-5820CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/doi/10.1080/00207179.2018.1459857info:eu-repo/semantics/altIdentifier/url/https://www.tandfonline.com/doi/full/10.1080/00207179.2018.1459857info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-nd/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-29T09:55:33Zoai:ri.conicet.gov.ar:11336/91650instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-29 09:55:33.543CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv Learning effective state-feedback controllers through efficient multilevel importance samplers
title Learning effective state-feedback controllers through efficient multilevel importance samplers
spellingShingle Learning effective state-feedback controllers through efficient multilevel importance samplers
Menchón, Silvia Adriana
IMPORTANCE SAMPLING
MULTILEVEL MONTE CARLO METHOD
PATH INTEGRAL CONTROL PROBLEMS
title_short Learning effective state-feedback controllers through efficient multilevel importance samplers
title_full Learning effective state-feedback controllers through efficient multilevel importance samplers
title_fullStr Learning effective state-feedback controllers through efficient multilevel importance samplers
title_full_unstemmed Learning effective state-feedback controllers through efficient multilevel importance samplers
title_sort Learning effective state-feedback controllers through efficient multilevel importance samplers
dc.creator.none.fl_str_mv Menchón, Silvia Adriana
Kappen, Hilbert Johan
author Menchón, Silvia Adriana
author_facet Menchón, Silvia Adriana
Kappen, Hilbert Johan
author_role author
author2 Kappen, Hilbert Johan
author2_role author
dc.subject.none.fl_str_mv IMPORTANCE SAMPLING
MULTILEVEL MONTE CARLO METHOD
PATH INTEGRAL CONTROL PROBLEMS
topic IMPORTANCE SAMPLING
MULTILEVEL MONTE CARLO METHOD
PATH INTEGRAL CONTROL PROBLEMS
purl_subject.fl_str_mv https://purl.org/becyt/ford/1.3
https://purl.org/becyt/ford/1
dc.description.none.fl_txt_mv Monte Carlo sampling can be used to estimate the solution of path integral control problems, which are a restricted class of nonlinear control problems with arbitrary dynamics and state cost, but with a linear dependence of the control on the dynamics and quadratic control cost. Although importance sampling is used to improve numerical computations, the effective sample size may still be low or many samples could be required. In this work, we propose a method to learn effective state-feedback controllers for nonlinear stochastic control problems based on multilevel importance samplers. In particular, we focus on the question of how to compute effective importance samplers considering a multigrid scenario. We test our algorithm in finite horizon control problems based on Lorenz-96 model with chaotic and non-chaotic behaviour, showing, in all cases, that our multigrid implementation reduces the computational time and improves the effective sample size.
Fil: Menchón, Silvia Adriana. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba. Instituto de Física Enrique Gaviola. Universidad Nacional de Córdoba. Instituto de Física Enrique Gaviola; Argentina. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Radboud Universiteit Nijmegen. Donders Instituto Brain Cognition and Behavior. SNN Machine Learning Group; Países Bajos
Fil: Kappen, Hilbert Johan. Radboud Universiteit Nijmegen. Donders Instituto Brain Cognition and Behavior. SNN Machine Learning Group; Países Bajos
description Monte Carlo sampling can be used to estimate the solution of path integral control problems, which are a restricted class of nonlinear control problems with arbitrary dynamics and state cost, but with a linear dependence of the control on the dynamics and quadratic control cost. Although importance sampling is used to improve numerical computations, the effective sample size may still be low or many samples could be required. In this work, we propose a method to learn effective state-feedback controllers for nonlinear stochastic control problems based on multilevel importance samplers. In particular, we focus on the question of how to compute effective importance samplers considering a multigrid scenario. We test our algorithm in finite horizon control problems based on Lorenz-96 model with chaotic and non-chaotic behaviour, showing, in all cases, that our multigrid implementation reduces the computational time and improves the effective sample size.
publishDate 2019
dc.date.none.fl_str_mv 2019-12-21
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/91650
Menchón, Silvia Adriana; Kappen, Hilbert Johan; Learning effective state-feedback controllers through efficient multilevel importance samplers; Taylor & Francis Ltd; International Journal Of Control; 92; 12; 21-12-2019; 2776-2783
0020-7179
1366-5820
CONICET Digital
CONICET
url http://hdl.handle.net/11336/91650
identifier_str_mv Menchón, Silvia Adriana; Kappen, Hilbert Johan; Learning effective state-feedback controllers through efficient multilevel importance samplers; Taylor & Francis Ltd; International Journal Of Control; 92; 12; 21-12-2019; 2776-2783
0020-7179
1366-5820
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/doi/10.1080/00207179.2018.1459857
info:eu-repo/semantics/altIdentifier/url/https://www.tandfonline.com/doi/full/10.1080/00207179.2018.1459857
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Taylor & Francis Ltd
publisher.none.fl_str_mv Taylor & Francis Ltd
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1844613674145153024
score 13.070432

Publicaciones similares