Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes

Autores
de Paula, Mariano; Avila, Luis Omar; Martinez, Ernesto Carlos
Año de publicación
2015
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
Automated control of blood glucose (BG) concentration with a fully automated artificial pancreas will certainly improve the quality of life for insulin-dependent patients. Closed-loop insulin delivery is challenging due to inter- and intra-patient variability, errors in glucose sensors and delays in insulin absorption. Responding to the varying activity levels seen in outpatients, with unpredictable and unreported food intake, and providing the necessary personalized control for individuals is a challenging task for existing control algorithms. A novel approach for controlling glycemic variability using simulation-based learning is presented. A policy iteration algorithm that combines reinforcement learning with Gaussian process approximation is proposed. To account for multiple sources of uncertainty, a control policy is learned off-line using an Ito´s stochastic model of the glucose-insulin dynamics. For safety and performance, only relevant data are sampled through Bayesian active learning. Results obtained demonstrate that a generic policy is both safe and efficient for controlling subject-specific variability due to a patient´s lifestyle and its distinctive metabolic response.
Fil: de Paula, Mariano. Universidad Nacional del Centro de la Provincia de Buenos Aires. Facultad de Ingeniería Olavarria. Departamento de Electromecánica. Grupo INTELYMEC; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Tandil. Centro de Investigaciones en Física e Ingeniería del Centro de la Provincia de Buenos Aires; Argentina
Fil: Avila, Luis Omar. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Santa Fe. Instituto de Desarrollo y Diseño (i); Argentina
Fil: Martinez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Santa Fe. Instituto de Desarrollo y Diseño (i); Argentina
Materia
Artificial Pancreas
Diabetes
Gaussian Processes
Policy Iteration
Reinforcement Learning
Stochastic Optimal Control
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/6897
Acceder
id CONICETDig_3f2789cf425f72165f28f87bc66e3fab
oai_identifier_str oai:ri.conicet.gov.ar:11336/6897
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processesde Paula, MarianoAvila, Luis OmarMartinez, Ernesto CarlosArtificial PancreasDiabetesGaussian ProcessesPolicy IterationReinforcement LearningStochastic Optimal Controlhttps://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1https://purl.org/becyt/ford/3.4https://purl.org/becyt/ford/3Automated control of blood glucose (BG) concentration with a fully automated artificial pancreas will certainly improve the quality of life for insulin-dependent patients. Closed-loop insulin delivery is challenging due to inter- and intra-patient variability, errors in glucose sensors and delays in insulin absorption. Responding to the varying activity levels seen in outpatients, with unpredictable and unreported food intake, and providing the necessary personalized control for individuals is a challenging task for existing control algorithms. A novel approach for controlling glycemic variability using simulation-based learning is presented. A policy iteration algorithm that combines reinforcement learning with Gaussian process approximation is proposed. To account for multiple sources of uncertainty, a control policy is learned off-line using an Ito´s stochastic model of the glucose-insulin dynamics. For safety and performance, only relevant data are sampled through Bayesian active learning. Results obtained demonstrate that a generic policy is both safe and efficient for controlling subject-specific variability due to a patient´s lifestyle and its distinctive metabolic response.Fil: de Paula, Mariano. Universidad Nacional del Centro de la Provincia de Buenos Aires. Facultad de Ingeniería Olavarria. Departamento de Electromecánica. Grupo INTELYMEC; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Tandil. Centro de Investigaciones en Física e Ingeniería del Centro de la Provincia de Buenos Aires; ArgentinaFil: Avila, Luis Omar. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Santa Fe. Instituto de Desarrollo y Diseño (i); ArgentinaFil: Martinez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Santa Fe. Instituto de Desarrollo y Diseño (i); ArgentinaElsevier2015-06info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/6897de Paula, Mariano; Avila, Luis Omar; Martinez, Ernesto Carlos; Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes; Elsevier; Applied Soft Computing; 35; 6-2015; 310-3321568-4946enginfo:eu-repo/semantics/altIdentifier/url/http://www.sciencedirect.com/science/article/pii/S1568494615003932info:eu-repo/semantics/altIdentifier/doi/10.1016/j.asoc.2015.06.041info:eu-repo/semantics/altIdentifier/doi/info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-nd/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-29T10:40:06Zoai:ri.conicet.gov.ar:11336/6897instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-29 10:40:06.35CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes
title Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes
spellingShingle Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes
de Paula, Mariano
Artificial Pancreas
Diabetes
Gaussian Processes
Policy Iteration
Reinforcement Learning
Stochastic Optimal Control
title_short Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes
title_full Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes
title_fullStr Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes
title_full_unstemmed Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes
title_sort Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes
dc.creator.none.fl_str_mv de Paula, Mariano
Avila, Luis Omar
Martinez, Ernesto Carlos
author de Paula, Mariano
author_facet de Paula, Mariano
Avila, Luis Omar
Martinez, Ernesto Carlos
author_role author
author2 Avila, Luis Omar
Martinez, Ernesto Carlos
author2_role author
author
dc.subject.none.fl_str_mv Artificial Pancreas
Diabetes
Gaussian Processes
Policy Iteration
Reinforcement Learning
Stochastic Optimal Control
topic Artificial Pancreas
Diabetes
Gaussian Processes
Policy Iteration
Reinforcement Learning
Stochastic Optimal Control
purl_subject.fl_str_mv https://purl.org/becyt/ford/1.2
https://purl.org/becyt/ford/1
https://purl.org/becyt/ford/3.4
https://purl.org/becyt/ford/3
dc.description.none.fl_txt_mv Automated control of blood glucose (BG) concentration with a fully automated artificial pancreas will certainly improve the quality of life for insulin-dependent patients. Closed-loop insulin delivery is challenging due to inter- and intra-patient variability, errors in glucose sensors and delays in insulin absorption. Responding to the varying activity levels seen in outpatients, with unpredictable and unreported food intake, and providing the necessary personalized control for individuals is a challenging task for existing control algorithms. A novel approach for controlling glycemic variability using simulation-based learning is presented. A policy iteration algorithm that combines reinforcement learning with Gaussian process approximation is proposed. To account for multiple sources of uncertainty, a control policy is learned off-line using an Ito´s stochastic model of the glucose-insulin dynamics. For safety and performance, only relevant data are sampled through Bayesian active learning. Results obtained demonstrate that a generic policy is both safe and efficient for controlling subject-specific variability due to a patient´s lifestyle and its distinctive metabolic response.
Fil: de Paula, Mariano. Universidad Nacional del Centro de la Provincia de Buenos Aires. Facultad de Ingeniería Olavarria. Departamento de Electromecánica. Grupo INTELYMEC; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Tandil. Centro de Investigaciones en Física e Ingeniería del Centro de la Provincia de Buenos Aires; Argentina
Fil: Avila, Luis Omar. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Santa Fe. Instituto de Desarrollo y Diseño (i); Argentina
Fil: Martinez, Ernesto Carlos. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Santa Fe. Instituto de Desarrollo y Diseño (i); Argentina
description Automated control of blood glucose (BG) concentration with a fully automated artificial pancreas will certainly improve the quality of life for insulin-dependent patients. Closed-loop insulin delivery is challenging due to inter- and intra-patient variability, errors in glucose sensors and delays in insulin absorption. Responding to the varying activity levels seen in outpatients, with unpredictable and unreported food intake, and providing the necessary personalized control for individuals is a challenging task for existing control algorithms. A novel approach for controlling glycemic variability using simulation-based learning is presented. A policy iteration algorithm that combines reinforcement learning with Gaussian process approximation is proposed. To account for multiple sources of uncertainty, a control policy is learned off-line using an Ito´s stochastic model of the glucose-insulin dynamics. For safety and performance, only relevant data are sampled through Bayesian active learning. Results obtained demonstrate that a generic policy is both safe and efficient for controlling subject-specific variability due to a patient´s lifestyle and its distinctive metabolic response.
publishDate 2015
dc.date.none.fl_str_mv 2015-06
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/6897
de Paula, Mariano; Avila, Luis Omar; Martinez, Ernesto Carlos; Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes; Elsevier; Applied Soft Computing; 35; 6-2015; 310-332
1568-4946
url http://hdl.handle.net/11336/6897
identifier_str_mv de Paula, Mariano; Avila, Luis Omar; Martinez, Ernesto Carlos; Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes; Elsevier; Applied Soft Computing; 35; 6-2015; 310-332
1568-4946
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/http://www.sciencedirect.com/science/article/pii/S1568494615003932
info:eu-repo/semantics/altIdentifier/doi/10.1016/j.asoc.2015.06.041
info:eu-repo/semantics/altIdentifier/doi/
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1844614428306178048
score 13.070432

Publicaciones similares