Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods
- Autores
- Pulido, Manuel Arturo; Tandeo, Pierre; Bocquet, Marc; Carrasi, Alberto; Lucini, María Magdalena
- Año de publicación
- 2018
- Idioma
- inglés
- Tipo de recurso
- artículo
- Estado
- versión publicada
- Descripción
- For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-of-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal successfully because both suffer from the collapse of the posterior distribution of the parameters. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of the ensemble Kalman filter (EnKF) with either the expectation–maximization (EM) or the Newton–Raphson (NR) used to maximize a likelihood associated to the parameters to be estimated. The EM and NR are applied primarily in the statistics and machine learning communities and are brought here in the context of data assimilation for the geosciences. The methods are derived using a Bayesian approach for a hidden Markov model and they are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof of concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorporated to represent the small-scale dynamics. The algorithms are able to identify the optimal stochastic parameterization with good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising efficient statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models.
Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina
Fil: Tandeo, Pierre. Centre National de la Recherche Scientifique; Francia. Universite de Bretagne Occidentale; Francia
Fil: Bocquet, Marc. Université Paris-Est; Francia
Fil: Carrasi, Alberto. Nansen Environmental and Remote Sensing Center; Noruega
Fil: Lucini, María Magdalena. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina - Materia
-
EXPECTATION–MAXIMIZATION ALGORITHM
MODEL ERROR ESTIMATION
PARAMETER ESTIMATION
STOCHASTIC PARAMETERIZATION - Nivel de accesibilidad
- acceso abierto
- Condiciones de uso
- https://creativecommons.org/licenses/by-nc-nd/2.5/ar/
- Repositorio
- Institución
- Consejo Nacional de Investigaciones Científicas y Técnicas
- OAI Identificador
- oai:ri.conicet.gov.ar:11336/86787
Ver los metadatos del registro completo
id |
CONICETDig_4bb45e7c50f23da5e175d621327cc0e1 |
---|---|
oai_identifier_str |
oai:ri.conicet.gov.ar:11336/86787 |
network_acronym_str |
CONICETDig |
repository_id_str |
3498 |
network_name_str |
CONICET Digital (CONICET) |
spelling |
Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methodsPulido, Manuel ArturoTandeo, PierreBocquet, MarcCarrasi, AlbertoLucini, María MagdalenaEXPECTATION–MAXIMIZATION ALGORITHMMODEL ERROR ESTIMATIONPARAMETER ESTIMATIONSTOCHASTIC PARAMETERIZATIONhttps://purl.org/becyt/ford/1.5https://purl.org/becyt/ford/1For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-of-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal successfully because both suffer from the collapse of the posterior distribution of the parameters. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of the ensemble Kalman filter (EnKF) with either the expectation–maximization (EM) or the Newton–Raphson (NR) used to maximize a likelihood associated to the parameters to be estimated. The EM and NR are applied primarily in the statistics and machine learning communities and are brought here in the context of data assimilation for the geosciences. The methods are derived using a Bayesian approach for a hidden Markov model and they are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof of concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorporated to represent the small-scale dynamics. The algorithms are able to identify the optimal stochastic parameterization with good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising efficient statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models.Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; ArgentinaFil: Tandeo, Pierre. Centre National de la Recherche Scientifique; Francia. Universite de Bretagne Occidentale; FranciaFil: Bocquet, Marc. Université Paris-Est; FranciaFil: Carrasi, Alberto. Nansen Environmental and Remote Sensing Center; NoruegaFil: Lucini, María Magdalena. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; ArgentinaTaylor & Francis2018-01info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/86787Pulido, Manuel Arturo; Tandeo, Pierre; Bocquet, Marc; Carrasi, Alberto; Lucini, María Magdalena; Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods; Taylor & Francis; Tellus A; 70; 1; 1-2018; 1-151600-08700280-6495CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://www.tandfonline.com/doi/full/10.1080/16000870.2018.1442099info:eu-repo/semantics/altIdentifier/doi/10.1080/16000870.2018.1442099info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-nd/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-29T10:09:45Zoai:ri.conicet.gov.ar:11336/86787instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-29 10:09:46.231CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse |
dc.title.none.fl_str_mv |
Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods |
title |
Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods |
spellingShingle |
Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods Pulido, Manuel Arturo EXPECTATION–MAXIMIZATION ALGORITHM MODEL ERROR ESTIMATION PARAMETER ESTIMATION STOCHASTIC PARAMETERIZATION |
title_short |
Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods |
title_full |
Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods |
title_fullStr |
Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods |
title_full_unstemmed |
Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods |
title_sort |
Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods |
dc.creator.none.fl_str_mv |
Pulido, Manuel Arturo Tandeo, Pierre Bocquet, Marc Carrasi, Alberto Lucini, María Magdalena |
author |
Pulido, Manuel Arturo |
author_facet |
Pulido, Manuel Arturo Tandeo, Pierre Bocquet, Marc Carrasi, Alberto Lucini, María Magdalena |
author_role |
author |
author2 |
Tandeo, Pierre Bocquet, Marc Carrasi, Alberto Lucini, María Magdalena |
author2_role |
author author author author |
dc.subject.none.fl_str_mv |
EXPECTATION–MAXIMIZATION ALGORITHM MODEL ERROR ESTIMATION PARAMETER ESTIMATION STOCHASTIC PARAMETERIZATION |
topic |
EXPECTATION–MAXIMIZATION ALGORITHM MODEL ERROR ESTIMATION PARAMETER ESTIMATION STOCHASTIC PARAMETERIZATION |
purl_subject.fl_str_mv |
https://purl.org/becyt/ford/1.5 https://purl.org/becyt/ford/1 |
dc.description.none.fl_txt_mv |
For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-of-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal successfully because both suffer from the collapse of the posterior distribution of the parameters. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of the ensemble Kalman filter (EnKF) with either the expectation–maximization (EM) or the Newton–Raphson (NR) used to maximize a likelihood associated to the parameters to be estimated. The EM and NR are applied primarily in the statistics and machine learning communities and are brought here in the context of data assimilation for the geosciences. The methods are derived using a Bayesian approach for a hidden Markov model and they are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof of concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorporated to represent the small-scale dynamics. The algorithms are able to identify the optimal stochastic parameterization with good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising efficient statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models. Fil: Pulido, Manuel Arturo. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste. Instituto de Modelado e Innovación Tecnológica. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas Naturales y Agrimensura. Instituto de Modelado e Innovación Tecnológica; Argentina Fil: Tandeo, Pierre. Centre National de la Recherche Scientifique; Francia. Universite de Bretagne Occidentale; Francia Fil: Bocquet, Marc. Université Paris-Est; Francia Fil: Carrasi, Alberto. Nansen Environmental and Remote Sensing Center; Noruega Fil: Lucini, María Magdalena. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Nordeste; Argentina. Universidad Nacional del Nordeste. Facultad de Ciencias Exactas y Naturales y Agrimensura; Argentina |
description |
For modelling geophysical systems, large-scale processes are described through a set of coarse-grained dynamical equations while small-scale processes are represented via parameterizations. This work proposes a method for identifying the best possible stochastic parameterization from noisy data. State-of-the-art sequential estimation methods such as Kalman and particle filters do not achieve this goal successfully because both suffer from the collapse of the posterior distribution of the parameters. To overcome this intrinsic limitation, we propose two statistical learning methods. They are based on the combination of the ensemble Kalman filter (EnKF) with either the expectation–maximization (EM) or the Newton–Raphson (NR) used to maximize a likelihood associated to the parameters to be estimated. The EM and NR are applied primarily in the statistics and machine learning communities and are brought here in the context of data assimilation for the geosciences. The methods are derived using a Bayesian approach for a hidden Markov model and they are applied to infer deterministic and stochastic physical parameters from noisy observations in coarse-grained dynamical models. Numerical experiments are conducted using the Lorenz-96 dynamical system with one and two scales as a proof of concept. The imperfect coarse-grained model is modelled through a one-scale Lorenz-96 system in which a stochastic parameterization is incorporated to represent the small-scale dynamics. The algorithms are able to identify the optimal stochastic parameterization with good accuracy under moderate observational noise. The proposed EnKF-EM and EnKF-NR are promising efficient statistical learning methods for developing stochastic parameterizations in high-dimensional geophysical models. |
publishDate |
2018 |
dc.date.none.fl_str_mv |
2018-01 |
dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion http://purl.org/coar/resource_type/c_6501 info:ar-repo/semantics/articulo |
format |
article |
status_str |
publishedVersion |
dc.identifier.none.fl_str_mv |
http://hdl.handle.net/11336/86787 Pulido, Manuel Arturo; Tandeo, Pierre; Bocquet, Marc; Carrasi, Alberto; Lucini, María Magdalena; Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods; Taylor & Francis; Tellus A; 70; 1; 1-2018; 1-15 1600-0870 0280-6495 CONICET Digital CONICET |
url |
http://hdl.handle.net/11336/86787 |
identifier_str_mv |
Pulido, Manuel Arturo; Tandeo, Pierre; Bocquet, Marc; Carrasi, Alberto; Lucini, María Magdalena; Stochastic parameterization identification using ensemble Kalman filtering combined with maximum likelihood methods; Taylor & Francis; Tellus A; 70; 1; 1-2018; 1-15 1600-0870 0280-6495 CONICET Digital CONICET |
dc.language.none.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
info:eu-repo/semantics/altIdentifier/url/https://www.tandfonline.com/doi/full/10.1080/16000870.2018.1442099 info:eu-repo/semantics/altIdentifier/doi/10.1080/16000870.2018.1442099 |
dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess https://creativecommons.org/licenses/by-nc-nd/2.5/ar/ |
eu_rights_str_mv |
openAccess |
rights_invalid_str_mv |
https://creativecommons.org/licenses/by-nc-nd/2.5/ar/ |
dc.format.none.fl_str_mv |
application/pdf application/pdf application/pdf application/pdf |
dc.publisher.none.fl_str_mv |
Taylor & Francis |
publisher.none.fl_str_mv |
Taylor & Francis |
dc.source.none.fl_str_mv |
reponame:CONICET Digital (CONICET) instname:Consejo Nacional de Investigaciones Científicas y Técnicas |
reponame_str |
CONICET Digital (CONICET) |
collection |
CONICET Digital (CONICET) |
instname_str |
Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.name.fl_str_mv |
CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.mail.fl_str_mv |
dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar |
_version_ |
1844613979915157504 |
score |
13.070432 |