Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods

Autores
Pulido, Manuel Arturo; Tandeo, Pierre; Bocquet, Marc; Carrasi, Alberto; Lucini, María Magdalena
Año de publicación
2018
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-of-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal successfully because both suffer from the collapse of the posterior distribution of the parameters. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of the ensemble Kalman filter (EnKF) with either the expectation–maximization (EM) or the Newton–Raphson (NR) used to maximize a likelihood associated to the parameters to be estimated. The EM and NR are applied primarily in the statistics and machine learning communities and are brought here in the context of data assimilation for the geosciences. The methods are derived using a Bayesian approach for a hidden Markov model and they are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof of concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorporated to represent the small-scale dynamics. The algorithms are able to identify the optimal stochastic parameterization with good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising efficient statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models.
Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina
Fil: Tandeo, Pierre. Centre National de la Recherche Scientifique; Francia. Universite de Bretagne Occidentale; Francia
Fil: Bocquet, Marc. Université Paris-Est; Francia
Fil: Carrasi, Alberto. Nansen Environmental and Remote Sensing Center; Noruega
Fil: Lucini, María Magdalena. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina
Materia
EXPECTATION–MAXIMIZATION ALGORITHM
MODEL ERROR ESTIMATION
PARAMETER ESTIMATION
STOCHASTIC PARAMETERIZATION
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/86787

id CONICETDig_4bb45e7c50f23da5e175d621327cc0e1
oai_identifier_str oai:ri.conicet.gov.ar:11336/86787
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methodsPulido, Manuel ArturoTandeo, PierreBocquet, MarcCarrasi, AlbertoLucini, María MagdalenaEXPECTATION–MAXIMIZATION ALGORITHMMODEL ERROR ESTIMATIONPARAMETER ESTIMATIONSTOCHASTIC PARAMETERIZATIONhttps://purl.org/becyt/ford/1.5https://purl.org/becyt/ford/1For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-of-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal successfully because both suffer from the collapse of the posterior distribution of the parameters. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of the ensemble Kalman filter (EnKF) with either the expectation–maximization (EM) or the Newton–Raphson (NR) used to maximize a likelihood associated to the parameters to be estimated. The EM and NR are applied primarily in the statistics and machine learning communities and are brought here in the context of data assimilation for the geosciences. The methods are derived using a Bayesian approach for a hidden Markov model and they are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof of concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorporated to represent the small-scale dynamics. The algorithms are able to identify the optimal stochastic parameterization with good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising efficient statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models.Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; ArgentinaFil: Tandeo, Pierre. Centre National de la Recherche Scientifique; Francia. Universite de Bretagne Occidentale; FranciaFil: Bocquet, Marc. Université Paris-Est; FranciaFil: Carrasi, Alberto. Nansen Environmental and Remote Sensing Center; NoruegaFil: Lucini, María Magdalena. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; ArgentinaTaylor & Francis2018-01info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/86787Pulido, Manuel Arturo; Tandeo, Pierre; Bocquet, Marc; Carrasi, Alberto; Lucini, María Magdalena; Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods; Taylor & Francis; Tellus A; 70; 1; 1-2018; 1-151600-08700280-6495CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://www.tandfonline.com/doi/full/10.1080/16000870.2018.1442099info:eu-repo/semantics/altIdentifier/doi/10.1080/16000870.2018.1442099info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-nd/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-29T10:09:45Zoai:ri.conicet.gov.ar:11336/86787instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-29 10:09:46.231CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods
title Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods
spellingShingle Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods
Pulido, Manuel Arturo
EXPECTATION–MAXIMIZATION ALGORITHM
MODEL ERROR ESTIMATION
PARAMETER ESTIMATION
STOCHASTIC PARAMETERIZATION
title_short Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods
title_full Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods
title_fullStr Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods
title_full_unstemmed Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods
title_sort Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods
dc.creator.none.fl_str_mv Pulido, Manuel Arturo
Tandeo, Pierre
Bocquet, Marc
Carrasi, Alberto
Lucini, María Magdalena
author Pulido, Manuel Arturo
author_facet Pulido, Manuel Arturo
Tandeo, Pierre
Bocquet, Marc
Carrasi, Alberto
Lucini, María Magdalena
author_role author
author2 Tandeo, Pierre
Bocquet, Marc
Carrasi, Alberto
Lucini, María Magdalena
author2_role author
author
author
author
dc.subject.none.fl_str_mv EXPECTATION–MAXIMIZATION ALGORITHM
MODEL ERROR ESTIMATION
PARAMETER ESTIMATION
STOCHASTIC PARAMETERIZATION
topic EXPECTATION–MAXIMIZATION ALGORITHM
MODEL ERROR ESTIMATION
PARAMETER ESTIMATION
STOCHASTIC PARAMETERIZATION
purl_subject.fl_str_mv https://purl.org/becyt/ford/1.5
https://purl.org/becyt/ford/1
dc.description.none.fl_txt_mv For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-of-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal successfully because both suffer from the collapse of the posterior distribution of the parameters. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of the ensemble Kalman filter (EnKF) with either the expectation–maximization (EM) or the Newton–Raphson (NR) used to maximize a likelihood associated to the parameters to be estimated. The EM and NR are applied primarily in the statistics and machine learning communities and are brought here in the context of data assimilation for the geosciences. The methods are derived using a Bayesian approach for a hidden Markov model and they are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof of concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorporated to represent the small-scale dynamics. The algorithms are able to identify the optimal stochastic parameterization with good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising efficient statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models.
Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina
Fil: Tandeo, Pierre. Centre National de la Recherche Scientifique; Francia. Universite de Bretagne Occidentale; Francia
Fil: Bocquet, Marc. Université Paris-Est; Francia
Fil: Carrasi, Alberto. Nansen Environmental and Remote Sensing Center; Noruega
Fil: Lucini, María Magdalena. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina
description For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-of-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal successfully because both suffer from the collapse of the posterior distribution of the parameters. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of the ensemble Kalman filter (EnKF) with either the expectation–maximization (EM) or the Newton–Raphson (NR) used to maximize a likelihood associated to the parameters to be estimated. The EM and NR are applied primarily in the statistics and machine learning communities and are brought here in the context of data assimilation for the geosciences. The methods are derived using a Bayesian approach for a hidden Markov model and they are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof of concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorporated to represent the small-scale dynamics. The algorithms are able to identify the optimal stochastic parameterization with good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising efficient statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models.
publishDate 2018
dc.date.none.fl_str_mv 2018-01
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/86787
Pulido, Manuel Arturo; Tandeo, Pierre; Bocquet, Marc; Carrasi, Alberto; Lucini, María Magdalena; Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods; Taylor & Francis; Tellus A; 70; 1; 1-2018; 1-15
1600-0870
0280-6495
CONICET Digital
CONICET
url http://hdl.handle.net/11336/86787
identifier_str_mv Pulido, Manuel Arturo; Tandeo, Pierre; Bocquet, Marc; Carrasi, Alberto; Lucini, María Magdalena; Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods; Taylor & Francis; Tellus A; 70; 1; 1-2018; 1-15
1600-0870
0280-6495
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/https://www.tandfonline.com/doi/full/10.1080/16000870.2018.1442099
info:eu-repo/semantics/altIdentifier/doi/10.1080/16000870.2018.1442099
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
application/pdf
application/pdf
dc.publisher.none.fl_str_mv Taylor & Francis
publisher.none.fl_str_mv Taylor & Francis
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1844613979915157504
score 13.070432