GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA

Autores
Rosenberg, Duane; Mininni, Pablo Daniel; Reddy, Raghu; Pouquet, Annick
Año de publicación
2020
Idioma
inglés
Tipo de recurso
artículo
Estado
versión publicada
Descripción
An existing hybrid MPI-OpenMP scheme is augmented with a CUDA-based fine grain parallelization approach for multidimensional distributed Fourier transforms, in a well-characterized pseudospectral fluid turbulence code. Basics of the hybrid scheme are reviewed, and heuristics provided to show a potential benefit of the CUDA implementation. The method draws heavily on the CUDA runtime library to handle memory management and on the cuFFT library for computing local FFTs. The manner in which the interfaces to these libraries are constructed, and ISO bindings utilized to facilitate platform portability, are discussed. CUDA streams are implemented to overlap data transfer with cuFFT computation. Testing with a baseline solver demonstrated significant aggregate speed-up over the hybrid MPI-OpenMP solver by offloading to GPUs on an NVLink-based test system. While the batch streamed approach provided little benefit with NVLink, we saw a performance gain of 30% when tuned for the optimal number of streams on a PCIe-based system. It was found that strong GPU scaling is nearly ideal, in all cases. Profiling of the CUDA kernels shows that the transform computation achieves 15% of the attainable peak FlOp-rate based on a roofline model for the system. In addition to speed-up measurements for the fiducial solver, we also considered several other solvers with different numbers of transform operations and found that aggregate speed-ups are nearly constant for all solvers.
Fil: Rosenberg, Duane. State University of Colorado - Fort Collins; Estados Unidos
Fil: Mininni, Pablo Daniel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física de Buenos Aires. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física de Buenos Aires; Argentina
Fil: Reddy, Raghu. Environmental Modeling Center; Estados Unidos
Fil: Pouquet, Annick. State University of Colorado at Boulder; Estados Unidos. National Center for Atmospheric Research; Estados Unidos
Materia
COMPUTATIONAL FLUIDS
CUDA
GPU
MPI
NUMERICAL SIMULATION
OPENMP
PARALLEL COMPUTING
Nivel de accesibilidad
acceso abierto
Condiciones de uso
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
Repositorio
CONICET Digital (CONICET)
Institución
Consejo Nacional de Investigaciones Científicas y Técnicas
OAI Identificador
oai:ri.conicet.gov.ar:11336/146032

id CONICETDig_ee106ee5faebb6588992594f563bbc83
oai_identifier_str oai:ri.conicet.gov.ar:11336/146032
network_acronym_str CONICETDig
repository_id_str 3498
network_name_str CONICET Digital (CONICET)
spelling GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDARosenberg, DuaneMininni, Pablo DanielReddy, RaghuPouquet, AnnickCOMPUTATIONAL FLUIDSCUDAGPUMPINUMERICAL SIMULATIONOPENMPPARALLEL COMPUTINGhttps://purl.org/becyt/ford/1.5https://purl.org/becyt/ford/1https://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1An existing hybrid MPI-OpenMP scheme is augmented with a CUDA-based fine grain parallelization approach for multidimensional distributed Fourier transforms, in a well-characterized pseudospectral fluid turbulence code. Basics of the hybrid scheme are reviewed, and heuristics provided to show a potential benefit of the CUDA implementation. The method draws heavily on the CUDA runtime library to handle memory management and on the cuFFT library for computing local FFTs. The manner in which the interfaces to these libraries are constructed, and ISO bindings utilized to facilitate platform portability, are discussed. CUDA streams are implemented to overlap data transfer with cuFFT computation. Testing with a baseline solver demonstrated significant aggregate speed-up over the hybrid MPI-OpenMP solver by offloading to GPUs on an NVLink-based test system. While the batch streamed approach provided little benefit with NVLink, we saw a performance gain of 30% when tuned for the optimal number of streams on a PCIe-based system. It was found that strong GPU scaling is nearly ideal, in all cases. Profiling of the CUDA kernels shows that the transform computation achieves 15% of the attainable peak FlOp-rate based on a roofline model for the system. In addition to speed-up measurements for the fiducial solver, we also considered several other solvers with different numbers of transform operations and found that aggregate speed-ups are nearly constant for all solvers.Fil: Rosenberg, Duane. State University of Colorado - Fort Collins; Estados UnidosFil: Mininni, Pablo Daniel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física de Buenos Aires. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física de Buenos Aires; ArgentinaFil: Reddy, Raghu. Environmental Modeling Center; Estados UnidosFil: Pouquet, Annick. State University of Colorado at Boulder; Estados Unidos. National Center for Atmospheric Research; Estados UnidosMolecular Diversity Preservation International2020-02info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/146032Rosenberg, Duane; Mininni, Pablo Daniel; Reddy, Raghu; Pouquet, Annick; GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA; Molecular Diversity Preservation International; Atmosphere; 11; 2; 2-2020; 1-222073-4433CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://www.mdpi.com/2073-4433/11/2/178info:eu-repo/semantics/altIdentifier/doi/10.3390/atmos11020178info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-03T09:43:44Zoai:ri.conicet.gov.ar:11336/146032instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-03 09:43:44.462CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse
dc.title.none.fl_str_mv GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA
title GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA
spellingShingle GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA
Rosenberg, Duane
COMPUTATIONAL FLUIDS
CUDA
GPU
MPI
NUMERICAL SIMULATION
OPENMP
PARALLEL COMPUTING
title_short GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA
title_full GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA
title_fullStr GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA
title_full_unstemmed GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA
title_sort GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA
dc.creator.none.fl_str_mv Rosenberg, Duane
Mininni, Pablo Daniel
Reddy, Raghu
Pouquet, Annick
author Rosenberg, Duane
author_facet Rosenberg, Duane
Mininni, Pablo Daniel
Reddy, Raghu
Pouquet, Annick
author_role author
author2 Mininni, Pablo Daniel
Reddy, Raghu
Pouquet, Annick
author2_role author
author
author
dc.subject.none.fl_str_mv COMPUTATIONAL FLUIDS
CUDA
GPU
MPI
NUMERICAL SIMULATION
OPENMP
PARALLEL COMPUTING
topic COMPUTATIONAL FLUIDS
CUDA
GPU
MPI
NUMERICAL SIMULATION
OPENMP
PARALLEL COMPUTING
purl_subject.fl_str_mv https://purl.org/becyt/ford/1.5
https://purl.org/becyt/ford/1
https://purl.org/becyt/ford/1.2
https://purl.org/becyt/ford/1
dc.description.none.fl_txt_mv An existing hybrid MPI-OpenMP scheme is augmented with a CUDA-based fine grain parallelization approach for multidimensional distributed Fourier transforms, in a well-characterized pseudospectral fluid turbulence code. Basics of the hybrid scheme are reviewed, and heuristics provided to show a potential benefit of the CUDA implementation. The method draws heavily on the CUDA runtime library to handle memory management and on the cuFFT library for computing local FFTs. The manner in which the interfaces to these libraries are constructed, and ISO bindings utilized to facilitate platform portability, are discussed. CUDA streams are implemented to overlap data transfer with cuFFT computation. Testing with a baseline solver demonstrated significant aggregate speed-up over the hybrid MPI-OpenMP solver by offloading to GPUs on an NVLink-based test system. While the batch streamed approach provided little benefit with NVLink, we saw a performance gain of 30% when tuned for the optimal number of streams on a PCIe-based system. It was found that strong GPU scaling is nearly ideal, in all cases. Profiling of the CUDA kernels shows that the transform computation achieves 15% of the attainable peak FlOp-rate based on a roofline model for the system. In addition to speed-up measurements for the fiducial solver, we also considered several other solvers with different numbers of transform operations and found that aggregate speed-ups are nearly constant for all solvers.
Fil: Rosenberg, Duane. State University of Colorado - Fort Collins; Estados Unidos
Fil: Mininni, Pablo Daniel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física de Buenos Aires. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física de Buenos Aires; Argentina
Fil: Reddy, Raghu. Environmental Modeling Center; Estados Unidos
Fil: Pouquet, Annick. State University of Colorado at Boulder; Estados Unidos. National Center for Atmospheric Research; Estados Unidos
description An existing hybrid MPI-OpenMP scheme is augmented with a CUDA-based fine grain parallelization approach for multidimensional distributed Fourier transforms, in a well-characterized pseudospectral fluid turbulence code. Basics of the hybrid scheme are reviewed, and heuristics provided to show a potential benefit of the CUDA implementation. The method draws heavily on the CUDA runtime library to handle memory management and on the cuFFT library for computing local FFTs. The manner in which the interfaces to these libraries are constructed, and ISO bindings utilized to facilitate platform portability, are discussed. CUDA streams are implemented to overlap data transfer with cuFFT computation. Testing with a baseline solver demonstrated significant aggregate speed-up over the hybrid MPI-OpenMP solver by offloading to GPUs on an NVLink-based test system. While the batch streamed approach provided little benefit with NVLink, we saw a performance gain of 30% when tuned for the optimal number of streams on a PCIe-based system. It was found that strong GPU scaling is nearly ideal, in all cases. Profiling of the CUDA kernels shows that the transform computation achieves 15% of the attainable peak FlOp-rate based on a roofline model for the system. In addition to speed-up measurements for the fiducial solver, we also considered several other solvers with different numbers of transform operations and found that aggregate speed-ups are nearly constant for all solvers.
publishDate 2020
dc.date.none.fl_str_mv 2020-02
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
http://purl.org/coar/resource_type/c_6501
info:ar-repo/semantics/articulo
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/11336/146032
Rosenberg, Duane; Mininni, Pablo Daniel; Reddy, Raghu; Pouquet, Annick; GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA; Molecular Diversity Preservation International; Atmosphere; 11; 2; 2-2020; 1-22
2073-4433
CONICET Digital
CONICET
url http://hdl.handle.net/11336/146032
identifier_str_mv Rosenberg, Duane; Mininni, Pablo Daniel; Reddy, Raghu; Pouquet, Annick; GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA; Molecular Diversity Preservation International; Atmosphere; 11; 2; 2-2020; 1-22
2073-4433
CONICET Digital
CONICET
dc.language.none.fl_str_mv eng
language eng
dc.relation.none.fl_str_mv info:eu-repo/semantics/altIdentifier/url/https://www.mdpi.com/2073-4433/11/2/178
info:eu-repo/semantics/altIdentifier/doi/10.3390/atmos11020178
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
eu_rights_str_mv openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Molecular Diversity Preservation International
publisher.none.fl_str_mv Molecular Diversity Preservation International
dc.source.none.fl_str_mv reponame:CONICET Digital (CONICET)
instname:Consejo Nacional de Investigaciones Científicas y Técnicas
reponame_str CONICET Digital (CONICET)
collection CONICET Digital (CONICET)
instname_str Consejo Nacional de Investigaciones Científicas y Técnicas
repository.name.fl_str_mv CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas
repository.mail.fl_str_mv dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar
_version_ 1842268619056087040
score 13.13397