Estimating model error covariances using particle filters
- Autores
- Lucini, María Magdalena; van Leeuwen, Peter Jan; Cocucci, Tadeo Javier; Pulido, Manuel Arturo
- Año de publicación
- 2019
- Idioma
- inglés
- Tipo de recurso
- documento de conferencia
- Estado
- versión publicada
- Descripción
- State-space models are the framework in data assimilation to mathematically describe the hidden state of a system by combining observations with constraints from a physical model. The formulation of these models usually involves statistical parameters that do not rely on physical constants and therefore must be estimated, since they play a central role in the performance of the data assimilation method. In particular, model error and observation error covariance matrices describe the second-order statistical properties of the system and observation stochastic equations, respectively. The model error covariance matrix Q is the least constrained statistical parameter since it depends on the model physics imperfections. Moreover, a misspecification of Q has a strong impact on the computation of the probability density functions involved in a particle filter algorithm, leading to an unreliable and inaccurate inference. In this work, we propose the combination of the Expectation-Maximization algorithm (EM) with an efficient particle filter to estimate the model error covariance matrix Q, using a batch of observations over a time window. The proposed method encompasses two stages: the expectation step, in which a particle filter is used with the present estimate of the model error covariance to find the probability density function that maximises the likelihood, followed by a maximization step in which this expectation is maximised as function of the model error covariance. The model evidence is written in terms of the sequential marginal likelihoods and therefore the likelihood maximization requires a particle filter and a particle smoother is not needed. Since the problem is highly nonlinear an analytical solution for this maximum is not available so that we use a fixed point iteration for the maximization step. We show that this methodology converges to the true model error covariance in stochastic twin experiments using a linear model and the Lorenz-96 system, but at different rates and with different accuracies depending on the system parameters. The extension to online estimation using the Expectation-Maximization algorithm is also discussed and evaluated.
Fil: Lucini, María Magdalena. University of Reading; Reino Unido. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina
Fil: van Leeuwen, Peter Jan. University of Reading; Reino Unido
Fil: Cocucci, Tadeo Javier. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina
Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina
EGU General Assembly 2019
Viena
Austria
European Geosciences Union - Materia
-
DATA ASSIMILATION
PARTICLE FILTERS
STATE SPACE MODEL
EXPECTATION-MAXIMIZATION - Nivel de accesibilidad
- acceso abierto
- Condiciones de uso
- https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
- Repositorio
- Institución
- Consejo Nacional de Investigaciones Científicas y Técnicas
- OAI Identificador
- oai:ri.conicet.gov.ar:11336/151363
Ver los metadatos del registro completo
id |
CONICETDig_89783de610b5808b2c7ba42f3e17dde0 |
---|---|
oai_identifier_str |
oai:ri.conicet.gov.ar:11336/151363 |
network_acronym_str |
CONICETDig |
repository_id_str |
3498 |
network_name_str |
CONICET Digital (CONICET) |
spelling |
Estimating model error covariances using particle filtersLucini, María Magdalenavan Leeuwen, Peter JanCocucci, Tadeo JavierPulido, Manuel ArturoDATA ASSIMILATIONPARTICLE FILTERSSTATE SPACE MODELEXPECTATION-MAXIMIZATIONhttps://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1State-space models are the framework in data assimilation to mathematically describe the hidden state of a system by combining observations with constraints from a physical model. The formulation of these models usually involves statistical parameters that do not rely on physical constants and therefore must be estimated, since they play a central role in the performance of the data assimilation method. In particular, model error and observation error covariance matrices describe the second-order statistical properties of the system and observation stochastic equations, respectively. The model error covariance matrix Q is the least constrained statistical parameter since it depends on the model physics imperfections. Moreover, a misspecification of Q has a strong impact on the computation of the probability density functions involved in a particle filter algorithm, leading to an unreliable and inaccurate inference. In this work, we propose the combination of the Expectation-Maximization algorithm (EM) with an efficient particle filter to estimate the model error covariance matrix Q, using a batch of observations over a time window. The proposed method encompasses two stages: the expectation step, in which a particle filter is used with the present estimate of the model error covariance to find the probability density function that maximises the likelihood, followed by a maximization step in which this expectation is maximised as function of the model error covariance. The model evidence is written in terms of the sequential marginal likelihoods and therefore the likelihood maximization requires a particle filter and a particle smoother is not needed. Since the problem is highly nonlinear an analytical solution for this maximum is not available so that we use a fixed point iteration for the maximization step. We show that this methodology converges to the true model error covariance in stochastic twin experiments using a linear model and the Lorenz-96 system, but at different rates and with different accuracies depending on the system parameters. The extension to online estimation using the Expectation-Maximization algorithm is also discussed and evaluated.Fil: Lucini, María Magdalena. University of Reading; Reino Unido. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; ArgentinaFil: van Leeuwen, Peter Jan. University of Reading; Reino UnidoFil: Cocucci, Tadeo Javier. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; ArgentinaFil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; ArgentinaEGU General Assembly 2019VienaAustriaEuropean Geosciences UnionCopernicus Publications2019info:eu-repo/semantics/publishedVersioninfo:eu-repo/semantics/conferenceObjectCongresoJournalhttp://purl.org/coar/resource_type/c_5794info:ar-repo/semantics/documentoDeConferenciaapplication/pdfapplication/pdfapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/151363Estimating model error covariances using particle filters; EGU General Assembly 2019; Viena; Austria; 2019; 1-11029-70061607-7962CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://meetingorganizer.copernicus.org/EGU2019/EGU2019-5230-1.pdfInternacionalinfo:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-10T13:20:55Zoai:ri.conicet.gov.ar:11336/151363instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-10 13:20:55.666CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse |
dc.title.none.fl_str_mv |
Estimating model error covariances using particle filters |
title |
Estimating model error covariances using particle filters |
spellingShingle |
Estimating model error covariances using particle filters Lucini, María Magdalena DATA ASSIMILATION PARTICLE FILTERS STATE SPACE MODEL EXPECTATION-MAXIMIZATION |
title_short |
Estimating model error covariances using particle filters |
title_full |
Estimating model error covariances using particle filters |
title_fullStr |
Estimating model error covariances using particle filters |
title_full_unstemmed |
Estimating model error covariances using particle filters |
title_sort |
Estimating model error covariances using particle filters |
dc.creator.none.fl_str_mv |
Lucini, María Magdalena van Leeuwen, Peter Jan Cocucci, Tadeo Javier Pulido, Manuel Arturo |
author |
Lucini, María Magdalena |
author_facet |
Lucini, María Magdalena van Leeuwen, Peter Jan Cocucci, Tadeo Javier Pulido, Manuel Arturo |
author_role |
author |
author2 |
van Leeuwen, Peter Jan Cocucci, Tadeo Javier Pulido, Manuel Arturo |
author2_role |
author author author |
dc.subject.none.fl_str_mv |
DATA ASSIMILATION PARTICLE FILTERS STATE SPACE MODEL EXPECTATION-MAXIMIZATION |
topic |
DATA ASSIMILATION PARTICLE FILTERS STATE SPACE MODEL EXPECTATION-MAXIMIZATION |
purl_subject.fl_str_mv |
https://purl.org/becyt/ford/1.2 https://purl.org/becyt/ford/1 |
dc.description.none.fl_txt_mv |
State-space models are the framework in data assimilation to mathematically describe the hidden state of a system by combining observations with constraints from a physical model. The formulation of these models usually involves statistical parameters that do not rely on physical constants and therefore must be estimated, since they play a central role in the performance of the data assimilation method. In particular, model error and observation error covariance matrices describe the second-order statistical properties of the system and observation stochastic equations, respectively. The model error covariance matrix Q is the least constrained statistical parameter since it depends on the model physics imperfections. Moreover, a misspecification of Q has a strong impact on the computation of the probability density functions involved in a particle filter algorithm, leading to an unreliable and inaccurate inference. In this work, we propose the combination of the Expectation-Maximization algorithm (EM) with an efficient particle filter to estimate the model error covariance matrix Q, using a batch of observations over a time window. The proposed method encompasses two stages: the expectation step, in which a particle filter is used with the present estimate of the model error covariance to find the probability density function that maximises the likelihood, followed by a maximization step in which this expectation is maximised as function of the model error covariance. The model evidence is written in terms of the sequential marginal likelihoods and therefore the likelihood maximization requires a particle filter and a particle smoother is not needed. Since the problem is highly nonlinear an analytical solution for this maximum is not available so that we use a fixed point iteration for the maximization step. We show that this methodology converges to the true model error covariance in stochastic twin experiments using a linear model and the Lorenz-96 system, but at different rates and with different accuracies depending on the system parameters. The extension to online estimation using the Expectation-Maximization algorithm is also discussed and evaluated. Fil: Lucini, María Magdalena. University of Reading; Reino Unido. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina Fil: van Leeuwen, Peter Jan. University of Reading; Reino Unido Fil: Cocucci, Tadeo Javier. Universidad Nacional de Córdoba. Facultad de Matemática, Astronomía y Física; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Córdoba; Argentina Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina EGU General Assembly 2019 Viena Austria European Geosciences Union |
description |
State-space models are the framework in data assimilation to mathematically describe the hidden state of a system by combining observations with constraints from a physical model. The formulation of these models usually involves statistical parameters that do not rely on physical constants and therefore must be estimated, since they play a central role in the performance of the data assimilation method. In particular, model error and observation error covariance matrices describe the second-order statistical properties of the system and observation stochastic equations, respectively. The model error covariance matrix Q is the least constrained statistical parameter since it depends on the model physics imperfections. Moreover, a misspecification of Q has a strong impact on the computation of the probability density functions involved in a particle filter algorithm, leading to an unreliable and inaccurate inference. In this work, we propose the combination of the Expectation-Maximization algorithm (EM) with an efficient particle filter to estimate the model error covariance matrix Q, using a batch of observations over a time window. The proposed method encompasses two stages: the expectation step, in which a particle filter is used with the present estimate of the model error covariance to find the probability density function that maximises the likelihood, followed by a maximization step in which this expectation is maximised as function of the model error covariance. The model evidence is written in terms of the sequential marginal likelihoods and therefore the likelihood maximization requires a particle filter and a particle smoother is not needed. Since the problem is highly nonlinear an analytical solution for this maximum is not available so that we use a fixed point iteration for the maximization step. We show that this methodology converges to the true model error covariance in stochastic twin experiments using a linear model and the Lorenz-96 system, but at different rates and with different accuracies depending on the system parameters. The extension to online estimation using the Expectation-Maximization algorithm is also discussed and evaluated. |
publishDate |
2019 |
dc.date.none.fl_str_mv |
2019 |
dc.type.none.fl_str_mv |
info:eu-repo/semantics/publishedVersion info:eu-repo/semantics/conferenceObject Congreso Journal http://purl.org/coar/resource_type/c_5794 info:ar-repo/semantics/documentoDeConferencia |
status_str |
publishedVersion |
format |
conferenceObject |
dc.identifier.none.fl_str_mv |
http://hdl.handle.net/11336/151363 Estimating model error covariances using particle filters; EGU General Assembly 2019; Viena; Austria; 2019; 1-1 1029-7006 1607-7962 CONICET Digital CONICET |
url |
http://hdl.handle.net/11336/151363 |
identifier_str_mv |
Estimating model error covariances using particle filters; EGU General Assembly 2019; Viena; Austria; 2019; 1-1 1029-7006 1607-7962 CONICET Digital CONICET |
dc.language.none.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
info:eu-repo/semantics/altIdentifier/url/https://meetingorganizer.copernicus.org/EGU2019/EGU2019-5230-1.pdf |
dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
eu_rights_str_mv |
openAccess |
rights_invalid_str_mv |
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
dc.format.none.fl_str_mv |
application/pdf application/pdf application/pdf application/pdf |
dc.coverage.none.fl_str_mv |
Internacional |
dc.publisher.none.fl_str_mv |
Copernicus Publications |
publisher.none.fl_str_mv |
Copernicus Publications |
dc.source.none.fl_str_mv |
reponame:CONICET Digital (CONICET) instname:Consejo Nacional de Investigaciones Científicas y Técnicas |
reponame_str |
CONICET Digital (CONICET) |
collection |
CONICET Digital (CONICET) |
instname_str |
Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.name.fl_str_mv |
CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.mail.fl_str_mv |
dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar |
_version_ |
1842981145943212032 |
score |
12.48226 |