Model error estimation using the expectation maximization algorithm and a particle flow filter
- Autores
- Lucini, María Magdalena; Leeuwen, Peter Jan van; Pulido, Manuel Arturo
- Año de publicación
- 2021
- Idioma
- inglés
- Tipo de recurso
- artículo
- Estado
- versión publicada
- Descripción
- Model error covariances play a central role in the performance of data assimilation methods applied to nonlinear state-space models. However, these covariances are largely unknown in most of the applications. A misspecification of the model error covariance has a strong impact on the computation of the posterior probability density function, leading to unreliable estimations and even to a total failure of the assimilation procedure. In this work, we propose the combination of the expectation maximization (EM) algorithm with an efficient particle filter to estimate the model error covariance using a batch of observations. Based on the EM algorithm principles, the proposed method encompasses two stages: the expectation stage, in which a particle filter is used with the present updated value of the model error covariance as given to find the probability density function that maximizes the likelihood, followed by a maximization stage, in which the expectation under the probability density function found in the expectation step is maximized as a function of the elements of the model error covariance. This novel algorithm here presented combines the EM algorithm with a fixed point algorithm and does not require a particle smoother to approximate the posterior densities. We demonstrate that the new method accurately and efficiently solves the linear model problem. Furthermore, for the chaotic nonlinear Lorenz-96 model the method is stable even for observation error covariance 10 times larger than the estimated model error covariance matrix and also is successful in moderately large dimensional situations where the dimension of the estimated matrix is 40 x 40.
Fil: Lucini, María Magdalena. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina
Fil: Leeuwen, Peter Jan van. State University of Colorado - Fort Collins; Estados Unidos
Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina - Materia
-
EM ALGORITHM
MODEL ERROR COVARIANCE
PARTICLE FILTERS
STATE-SPACE MODELS - Nivel de accesibilidad
- acceso abierto
- Condiciones de uso
- https://creativecommons.org/licenses/by/2.5/ar/
- Repositorio
- Institución
- Consejo Nacional de Investigaciones Científicas y Técnicas
- OAI Identificador
- oai:ri.conicet.gov.ar:11336/149232
Ver los metadatos del registro completo
id |
CONICETDig_90369941f7797e4f7545d9ccc4111934 |
---|---|
oai_identifier_str |
oai:ri.conicet.gov.ar:11336/149232 |
network_acronym_str |
CONICETDig |
repository_id_str |
3498 |
network_name_str |
CONICET Digital (CONICET) |
spelling |
Model error estimation using the expectation maximization algorithm and a particle flow filterLucini, María MagdalenaLeeuwen, Peter Jan vanPulido, Manuel ArturoEM ALGORITHMMODEL ERROR COVARIANCEPARTICLE FILTERSSTATE-SPACE MODELShttps://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1Model error covariances play a central role in the performance of data assimilation methods applied to nonlinear state-space models. However, these covariances are largely unknown in most of the applications. A misspecification of the model error covariance has a strong impact on the computation of the posterior probability density function, leading to unreliable estimations and even to a total failure of the assimilation procedure. In this work, we propose the combination of the expectation maximization (EM) algorithm with an efficient particle filter to estimate the model error covariance using a batch of observations. Based on the EM algorithm principles, the proposed method encompasses two stages: the expectation stage, in which a particle filter is used with the present updated value of the model error covariance as given to find the probability density function that maximizes the likelihood, followed by a maximization stage, in which the expectation under the probability density function found in the expectation step is maximized as a function of the elements of the model error covariance. This novel algorithm here presented combines the EM algorithm with a fixed point algorithm and does not require a particle smoother to approximate the posterior densities. We demonstrate that the new method accurately and efficiently solves the linear model problem. Furthermore, for the chaotic nonlinear Lorenz-96 model the method is stable even for observation error covariance 10 times larger than the estimated model error covariance matrix and also is successful in moderately large dimensional situations where the dimension of the estimated matrix is 40 x 40.Fil: Lucini, María Magdalena. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; ArgentinaFil: Leeuwen, Peter Jan van. State University of Colorado - Fort Collins; Estados UnidosFil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; ArgentinaSociety of Industrial and Applied Mathematics2021-03info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/149232Lucini, María Magdalena; Leeuwen, Peter Jan van; Pulido, Manuel Arturo; Model error estimation using the expectation maximization algorithm and a particle flow filter; Society of Industrial and Applied Mathematics; Journal on Uncertainty Quantification; 9; 2; 3-2021; 681-7072166-2525CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/doi/10.1137/19M1297300info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-29T10:02:42Zoai:ri.conicet.gov.ar:11336/149232instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-29 10:02:43.125CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse |
dc.title.none.fl_str_mv |
Model error estimation using the expectation maximization algorithm and a particle flow filter |
title |
Model error estimation using the expectation maximization algorithm and a particle flow filter |
spellingShingle |
Model error estimation using the expectation maximization algorithm and a particle flow filter Lucini, María Magdalena EM ALGORITHM MODEL ERROR COVARIANCE PARTICLE FILTERS STATE-SPACE MODELS |
title_short |
Model error estimation using the expectation maximization algorithm and a particle flow filter |
title_full |
Model error estimation using the expectation maximization algorithm and a particle flow filter |
title_fullStr |
Model error estimation using the expectation maximization algorithm and a particle flow filter |
title_full_unstemmed |
Model error estimation using the expectation maximization algorithm and a particle flow filter |
title_sort |
Model error estimation using the expectation maximization algorithm and a particle flow filter |
dc.creator.none.fl_str_mv |
Lucini, María Magdalena Leeuwen, Peter Jan van Pulido, Manuel Arturo |
author |
Lucini, María Magdalena |
author_facet |
Lucini, María Magdalena Leeuwen, Peter Jan van Pulido, Manuel Arturo |
author_role |
author |
author2 |
Leeuwen, Peter Jan van Pulido, Manuel Arturo |
author2_role |
author author |
dc.subject.none.fl_str_mv |
EM ALGORITHM MODEL ERROR COVARIANCE PARTICLE FILTERS STATE-SPACE MODELS |
topic |
EM ALGORITHM MODEL ERROR COVARIANCE PARTICLE FILTERS STATE-SPACE MODELS |
purl_subject.fl_str_mv |
https://purl.org/becyt/ford/1.2 https://purl.org/becyt/ford/1 |
dc.description.none.fl_txt_mv |
Model error covariances play a central role in the performance of data assimilation methods applied to nonlinear state-space models. However, these covariances are largely unknown in most of the applications. A misspecification of the model error covariance has a strong impact on the computation of the posterior probability density function, leading to unreliable estimations and even to a total failure of the assimilation procedure. In this work, we propose the combination of the expectation maximization (EM) algorithm with an efficient particle filter to estimate the model error covariance using a batch of observations. Based on the EM algorithm principles, the proposed method encompasses two stages: the expectation stage, in which a particle filter is used with the present updated value of the model error covariance as given to find the probability density function that maximizes the likelihood, followed by a maximization stage, in which the expectation under the probability density function found in the expectation step is maximized as a function of the elements of the model error covariance. This novel algorithm here presented combines the EM algorithm with a fixed point algorithm and does not require a particle smoother to approximate the posterior densities. We demonstrate that the new method accurately and efficiently solves the linear model problem. Furthermore, for the chaotic nonlinear Lorenz-96 model the method is stable even for observation error covariance 10 times larger than the estimated model error covariance matrix and also is successful in moderately large dimensional situations where the dimension of the estimated matrix is 40 x 40. Fil: Lucini, María Magdalena. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina Fil: Leeuwen, Peter Jan van. State University of Colorado - Fort Collins; Estados Unidos Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina |
description |
Model error covariances play a central role in the performance of data assimilation methods applied to nonlinear state-space models. However, these covariances are largely unknown in most of the applications. A misspecification of the model error covariance has a strong impact on the computation of the posterior probability density function, leading to unreliable estimations and even to a total failure of the assimilation procedure. In this work, we propose the combination of the expectation maximization (EM) algorithm with an efficient particle filter to estimate the model error covariance using a batch of observations. Based on the EM algorithm principles, the proposed method encompasses two stages: the expectation stage, in which a particle filter is used with the present updated value of the model error covariance as given to find the probability density function that maximizes the likelihood, followed by a maximization stage, in which the expectation under the probability density function found in the expectation step is maximized as a function of the elements of the model error covariance. This novel algorithm here presented combines the EM algorithm with a fixed point algorithm and does not require a particle smoother to approximate the posterior densities. We demonstrate that the new method accurately and efficiently solves the linear model problem. Furthermore, for the chaotic nonlinear Lorenz-96 model the method is stable even for observation error covariance 10 times larger than the estimated model error covariance matrix and also is successful in moderately large dimensional situations where the dimension of the estimated matrix is 40 x 40. |
publishDate |
2021 |
dc.date.none.fl_str_mv |
2021-03 |
dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion http://purl.org/coar/resource_type/c_6501 info:ar-repo/semantics/articulo |
format |
article |
status_str |
publishedVersion |
dc.identifier.none.fl_str_mv |
http://hdl.handle.net/11336/149232 Lucini, María Magdalena; Leeuwen, Peter Jan van; Pulido, Manuel Arturo; Model error estimation using the expectation maximization algorithm and a particle flow filter; Society of Industrial and Applied Mathematics; Journal on Uncertainty Quantification; 9; 2; 3-2021; 681-707 2166-2525 CONICET Digital CONICET |
url |
http://hdl.handle.net/11336/149232 |
identifier_str_mv |
Lucini, María Magdalena; Leeuwen, Peter Jan van; Pulido, Manuel Arturo; Model error estimation using the expectation maximization algorithm and a particle flow filter; Society of Industrial and Applied Mathematics; Journal on Uncertainty Quantification; 9; 2; 3-2021; 681-707 2166-2525 CONICET Digital CONICET |
dc.language.none.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
info:eu-repo/semantics/altIdentifier/doi/10.1137/19M1297300 |
dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess https://creativecommons.org/licenses/by/2.5/ar/ |
eu_rights_str_mv |
openAccess |
rights_invalid_str_mv |
https://creativecommons.org/licenses/by/2.5/ar/ |
dc.format.none.fl_str_mv |
application/pdf application/pdf |
dc.publisher.none.fl_str_mv |
Society of Industrial and Applied Mathematics |
publisher.none.fl_str_mv |
Society of Industrial and Applied Mathematics |
dc.source.none.fl_str_mv |
reponame:CONICET Digital (CONICET) instname:Consejo Nacional de Investigaciones Científicas y Técnicas |
reponame_str |
CONICET Digital (CONICET) |
collection |
CONICET Digital (CONICET) |
instname_str |
Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.name.fl_str_mv |
CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.mail.fl_str_mv |
dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar |
_version_ |
1844613834456694784 |
score |
13.070432 |