Estimating model error covariances using particle filters

Autores
Lucini, María Magdalena; van Leeuwen, Peter Jan; Cocucci, Tadeo Javier; Pulido, Manuel Arturo
Año de publicación
2019
Idioma
inglés
Tipo de recurso
documento de conferencia
Estado
versión publicada
Descripción
State-space models are the framework in data assimilation to mathematically describe the hidden state of a system by combining observations with constraints from a physical model. The formulation of these models usually involves statistical parameters that do not rely on physical constants and therefore must be estimated, since they play a central role in the performance of the data assimilation method. In particular, model error and observation error covariance matrices describe the second-order statistical properties of the system and observation stochastic equations, respectively. The model error covariance matrix Q is the least constrained statistical parameter since it depends on the model physics imperfections. Moreover, a misspecification of Q has a strong impact on the computation of the probability density functions involved in a particle filter algorithm, leading to an unreliable and inaccurate inference. In this work, we propose the combination of the Expectation-Maximization algorithm (EM) with an efficient particle filter to estimate the model error covariance matrix Q, using a batch of observations over a time window. The proposed method encompasses two stages: the expectation step, in which a particle filter is used with the present estimate of the model error covariance to find the probability density function that maximises the likelihood, followed by a maximization step in which this expectation is maximised as function of the model error covariance. The model evidence is written in terms of the sequential marginal likelihoods and therefore the likelihood maximization requires a particle filter and a particle smoother is not needed. Since the problem is highly nonlinear an analytical solution for this maximum is not available so that we use a fixed point iteration for the maximization step. We show that this methodology converges to the true model error covariance in stochastic twin experiments using a linear model and the Lorenz-96 system, but at different rates and with different accuracies depending on the system parameters. The extension to online estimation using the Expectation-Maximization algorithm is also discussed and evaluated.
Fil: Lucini, María Magdalena. University of Reading; Reino Unido. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina
Fil: van Leeuwen, Peter Jan. University of Reading; Reino Unido
Fil: Cocucci, Tadeo Javier. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina
Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina
EGU General Assembly 2019
Viena
Austria
European Geosciences Union
Materia
DATA ASSIMILATION
PARTICLE FILTERS
STATE SPACE MODEL
EXPECTATION-MAXIMIZATION
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/151363

id CONICETDig_89783de610b5808b2c7ba42f3e17dde0
oai_identifier_str oai:ri.conicet.gov.ar:11336/151363
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling Estimating model error covariances using particle filtersLucini, María Magdalenavan Leeuwen, Peter JanCocucci, Tadeo JavierPulido, Manuel ArturoDATA ASSIMILATIONPARTICLE FILTERSSTATE SPACE MODELEXPECTATION-MAXIMIZATIONhttps://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1State-space models are the framework in data assimilation to mathematically describe the hidden state of a system by combining observations with constraints from a physical model. The formulation of these models usually involves statistical parameters that do not rely on physical constants and therefore must be estimated, since they play a central role in the performance of the data assimilation method. In particular, model error and observation error covariance matrices describe the second-order statistical properties of the system and observation stochastic equations, respectively. The model error covariance matrix Q is the least constrained statistical parameter since it depends on the model physics imperfections. Moreover, a misspecification of Q has a strong impact on the computation of the probability density functions involved in a particle filter algorithm, leading to an unreliable and inaccurate inference. In this work, we propose the combination of the Expectation-Maximization algorithm (EM) with an efficient particle filter to estimate the model error covariance matrix Q, using a batch of observations over a time window. The proposed method encompasses two stages: the expectation step, in which a particle filter is used with the present estimate of the model error covariance to find the probability density function that maximises the likelihood, followed by a maximization step in which this expectation is maximised as function of the model error covariance. The model evidence is written in terms of the sequential marginal likelihoods and therefore the likelihood maximization requires a particle filter and a particle smoother is not needed. Since the problem is highly nonlinear an analytical solution for this maximum is not available so that we use a fixed point iteration for the maximization step. We show that this methodology converges to the true model error covariance in stochastic twin experiments using a linear model and the Lorenz-96 system, but at different rates and with different accuracies depending on the system parameters. The extension to online estimation using the Expectation-Maximization algorithm is also discussed and evaluated.Fil: Lucini, María Magdalena. University of Reading; Reino Unido. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; ArgentinaFil: van Leeuwen, Peter Jan. University of Reading; Reino UnidoFil: Cocucci, Tadeo Javier. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; ArgentinaFil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; ArgentinaEGU General Assembly 2019VienaAustriaEuropean Geosciences UnionCopernicus Publications2019info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/conferenceObjectCongresoJournalhttp://purl.org/coar/resource_type/c_5794info:ar-repo/semantics/documentoDeConferenciaapplication/pdfapplication/pdfapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/151363Estimating model error covariances using particle filters; EGU General Assembly 2019; Viena; Austria; 2019; 1-11029-70061607-7962CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://meetingorganizer.copernicus.org/EGU2019/EGU2019-5230-1.pdfInternacionalinfo:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-10T13:20:55Zoai:ri.conicet.gov.ar:11336/151363instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-10 13:20:55.666CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv Estimating model error covariances using particle filters
title Estimating model error covariances using particle filters
spellingShingle Estimating model error covariances using particle filters
Lucini, María Magdalena
DATA ASSIMILATION
PARTICLE FILTERS
STATE SPACE MODEL
EXPECTATION-MAXIMIZATION
title_short Estimating model error covariances using particle filters
title_full Estimating model error covariances using particle filters
title_fullStr Estimating model error covariances using particle filters
title_full_unstemmed Estimating model error covariances using particle filters
title_sort Estimating model error covariances using particle filters
dc.creator.none.fl_str_mv Lucini, María Magdalena
van Leeuwen, Peter Jan
Cocucci, Tadeo Javier
Pulido, Manuel Arturo
author Lucini, María Magdalena
author_facet Lucini, María Magdalena
van Leeuwen, Peter Jan
Cocucci, Tadeo Javier
Pulido, Manuel Arturo
author_role author
author2 van Leeuwen, Peter Jan
Cocucci, Tadeo Javier
Pulido, Manuel Arturo
author2_role author
author
author
dc.subject.none.fl_str_mv DATA ASSIMILATION
PARTICLE FILTERS
STATE SPACE MODEL
EXPECTATION-MAXIMIZATION
topic DATA ASSIMILATION
PARTICLE FILTERS
STATE SPACE MODEL
EXPECTATION-MAXIMIZATION
purl_subject.fl_str_mv https://purl.org/becyt/ford/1.2
https://purl.org/becyt/ford/1
dc.description.none.fl_txt_mv State-space models are the framework in data assimilation to mathematically describe the hidden state of a system by combining observations with constraints from a physical model. The formulation of these models usually involves statistical parameters that do not rely on physical constants and therefore must be estimated, since they play a central role in the performance of the data assimilation method. In particular, model error and observation error covariance matrices describe the second-order statistical properties of the system and observation stochastic equations, respectively. The model error covariance matrix Q is the least constrained statistical parameter since it depends on the model physics imperfections. Moreover, a misspecification of Q has a strong impact on the computation of the probability density functions involved in a particle filter algorithm, leading to an unreliable and inaccurate inference. In this work, we propose the combination of the Expectation-Maximization algorithm (EM) with an efficient particle filter to estimate the model error covariance matrix Q, using a batch of observations over a time window. The proposed method encompasses two stages: the expectation step, in which a particle filter is used with the present estimate of the model error covariance to find the probability density function that maximises the likelihood, followed by a maximization step in which this expectation is maximised as function of the model error covariance. The model evidence is written in terms of the sequential marginal likelihoods and therefore the likelihood maximization requires a particle filter and a particle smoother is not needed. Since the problem is highly nonlinear an analytical solution for this maximum is not available so that we use a fixed point iteration for the maximization step. We show that this methodology converges to the true model error covariance in stochastic twin experiments using a linear model and the Lorenz-96 system, but at different rates and with different accuracies depending on the system parameters. The extension to online estimation using the Expectation-Maximization algorithm is also discussed and evaluated.
Fil: Lucini, María Magdalena. University of Reading; Reino Unido. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina
Fil: van Leeuwen, Peter Jan. University of Reading; Reino Unido
Fil: Cocucci, Tadeo Javier. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina
Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina
EGU General Assembly 2019
Viena
Austria
European Geosciences Union
description State-space models are the framework in data assimilation to mathematically describe the hidden state of a system by combining observations with constraints from a physical model. The formulation of these models usually involves statistical parameters that do not rely on physical constants and therefore must be estimated, since they play a central role in the performance of the data assimilation method. In particular, model error and observation error covariance matrices describe the second-order statistical properties of the system and observation stochastic equations, respectively. The model error covariance matrix Q is the least constrained statistical parameter since it depends on the model physics imperfections. Moreover, a misspecification of Q has a strong impact on the computation of the probability density functions involved in a particle filter algorithm, leading to an unreliable and inaccurate inference. In this work, we propose the combination of the Expectation-Maximization algorithm (EM) with an efficient particle filter to estimate the model error covariance matrix Q, using a batch of observations over a time window. The proposed method encompasses two stages: the expectation step, in which a particle filter is used with the present estimate of the model error covariance to find the probability density function that maximises the likelihood, followed by a maximization step in which this expectation is maximised as function of the model error covariance. The model evidence is written in terms of the sequential marginal likelihoods and therefore the likelihood maximization requires a particle filter and a particle smoother is not needed. Since the problem is highly nonlinear an analytical solution for this maximum is not available so that we use a fixed point iteration for the maximization step. We show that this methodology converges to the true model error covariance in stochastic twin experiments using a linear model and the Lorenz-96 system, but at different rates and with different accuracies depending on the system parameters. The extension to online estimation using the Expectation-Maximization algorithm is also discussed and evaluated.
publishDate 2019
dc.date.none.fl_str_mv 2019
dc.type.none.fl_str_mv info:eu-repo/semantics/publishedVersion
info:eu-repo/semantics/conferenceObject
Congreso
Journal
http://purl.org/coar/resource_type/c_5794
info:ar-repo/semantics/documentoDeConferencia
status_str publishedVersion
format conferenceObject
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/151363
Estimating model error covariances using particle filters; EGU General Assembly 2019; Viena; Austria; 2019; 1-1
1029-7006
1607-7962
CONICET Digital
CONICET
url http://hdl.handle.net/11336/151363
identifier_str_mv Estimating model error covariances using particle filters; EGU General Assembly 2019; Viena; Austria; 2019; 1-1
1029-7006
1607-7962
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/https://meetingorganizer.copernicus.org/EGU2019/EGU2019-5230-1.pdf
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
application/pdf
application/pdf
dc.coverage.none.fl_str_mv Internacional
dc.publisher.none.fl_str_mv Copernicus Publications
publisher.none.fl_str_mv Copernicus Publications
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1842981145943212032
score 12.48226