GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA
- Autores
- Rosenberg, Duane; Mininni, Pablo Daniel; Reddy, Raghu; Pouquet, Annick
- Año de publicación
- 2020
- Idioma
- inglés
- Tipo de recurso
- artículo
- Estado
- versión publicada
- Descripción
- An existing hybrid MPI-OpenMP scheme is augmented with a CUDA-based fine grain parallelization approach for multidimensional distributed Fourier transforms, in a well-characterized pseudospectral fluid turbulence code. Basics of the hybrid scheme are reviewed, and heuristics provided to show a potential benefit of the CUDA implementation. The method draws heavily on the CUDA runtime library to handle memory management and on the cuFFT library for computing local FFTs. The manner in which the interfaces to these libraries are constructed, and ISO bindings utilized to facilitate platform portability, are discussed. CUDA streams are implemented to overlap data transfer with cuFFT computation. Testing with a baseline solver demonstrated significant aggregate speed-up over the hybrid MPI-OpenMP solver by offloading to GPUs on an NVLink-based test system. While the batch streamed approach provided little benefit with NVLink, we saw a performance gain of 30% when tuned for the optimal number of streams on a PCIe-based system. It was found that strong GPU scaling is nearly ideal, in all cases. Profiling of the CUDA kernels shows that the transform computation achieves 15% of the attainable peak FlOp-rate based on a roofline model for the system. In addition to speed-up measurements for the fiducial solver, we also considered several other solvers with different numbers of transform operations and found that aggregate speed-ups are nearly constant for all solvers.
Fil: Rosenberg, Duane. State University of Colorado - Fort Collins; Estados Unidos
Fil: Mininni, Pablo Daniel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física de Buenos Aires. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física de Buenos Aires; Argentina
Fil: Reddy, Raghu. Environmental Modeling Center; Estados Unidos
Fil: Pouquet, Annick. State University of Colorado at Boulder; Estados Unidos. National Center for Atmospheric Research; Estados Unidos - Materia
-
COMPUTATIONAL FLUIDS
CUDA
GPU
MPI
NUMERICAL SIMULATION
OPENMP
PARALLEL COMPUTING - Nivel de accesibilidad
- acceso abierto
- Condiciones de uso
- https://creativecommons.org/licenses/by-nc-sa/2.5/ar/
- Repositorio
- Institución
- Consejo Nacional de Investigaciones Científicas y Técnicas
- OAI Identificador
- oai:ri.conicet.gov.ar:11336/146032
Ver los metadatos del registro completo
id |
CONICETDig_ee106ee5faebb6588992594f563bbc83 |
---|---|
oai_identifier_str |
oai:ri.conicet.gov.ar:11336/146032 |
network_acronym_str |
CONICETDig |
repository_id_str |
3498 |
network_name_str |
CONICET Digital (CONICET) |
spelling |
GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDARosenberg, DuaneMininni, Pablo DanielReddy, RaghuPouquet, AnnickCOMPUTATIONAL FLUIDSCUDAGPUMPINUMERICAL SIMULATIONOPENMPPARALLEL COMPUTINGhttps://purl.org/becyt/ford/1.5https://purl.org/becyt/ford/1https://purl.org/becyt/ford/1.2https://purl.org/becyt/ford/1An existing hybrid MPI-OpenMP scheme is augmented with a CUDA-based fine grain parallelization approach for multidimensional distributed Fourier transforms, in a well-characterized pseudospectral fluid turbulence code. Basics of the hybrid scheme are reviewed, and heuristics provided to show a potential benefit of the CUDA implementation. The method draws heavily on the CUDA runtime library to handle memory management and on the cuFFT library for computing local FFTs. The manner in which the interfaces to these libraries are constructed, and ISO bindings utilized to facilitate platform portability, are discussed. CUDA streams are implemented to overlap data transfer with cuFFT computation. Testing with a baseline solver demonstrated significant aggregate speed-up over the hybrid MPI-OpenMP solver by offloading to GPUs on an NVLink-based test system. While the batch streamed approach provided little benefit with NVLink, we saw a performance gain of 30% when tuned for the optimal number of streams on a PCIe-based system. It was found that strong GPU scaling is nearly ideal, in all cases. Profiling of the CUDA kernels shows that the transform computation achieves 15% of the attainable peak FlOp-rate based on a roofline model for the system. In addition to speed-up measurements for the fiducial solver, we also considered several other solvers with different numbers of transform operations and found that aggregate speed-ups are nearly constant for all solvers.Fil: Rosenberg, Duane. State University of Colorado - Fort Collins; Estados UnidosFil: Mininni, Pablo Daniel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física de Buenos Aires. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física de Buenos Aires; ArgentinaFil: Reddy, Raghu. Environmental Modeling Center; Estados UnidosFil: Pouquet, Annick. State University of Colorado at Boulder; Estados Unidos. National Center for Atmospheric Research; Estados UnidosMolecular Diversity Preservation International2020-02info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionhttp://purl.org/coar/resource_type/c_6501info:ar-repo/semantics/articuloapplication/pdfapplication/pdfhttp://hdl.handle.net/11336/146032Rosenberg, Duane; Mininni, Pablo Daniel; Reddy, Raghu; Pouquet, Annick; GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA; Molecular Diversity Preservation International; Atmosphere; 11; 2; 2-2020; 1-222073-4433CONICET DigitalCONICETenginfo:eu-repo/semantics/altIdentifier/url/https://www.mdpi.com/2073-4433/11/2/178info:eu-repo/semantics/altIdentifier/doi/10.3390/atmos11020178info:eu-repo/semantics/openAccesshttps://creativecommons.org/licenses/by-nc-sa/2.5/ar/reponame:CONICET Digital (CONICET)instname:Consejo Nacional de Investigaciones Científicas y Técnicas2025-09-03T09:43:44Zoai:ri.conicet.gov.ar:11336/146032instacron:CONICETInstitucionalhttp://ri.conicet.gov.ar/Organismo científico-tecnológicoNo correspondehttp://ri.conicet.gov.ar/oai/requestdasensio@conicet.gov.ar; lcarlino@conicet.gov.arArgentinaNo correspondeNo correspondeNo correspondeopendoar:34982025-09-03 09:43:44.462CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicasfalse |
dc.title.none.fl_str_mv |
GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA |
title |
GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA |
spellingShingle |
GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA Rosenberg, Duane COMPUTATIONAL FLUIDS CUDA GPU MPI NUMERICAL SIMULATION OPENMP PARALLEL COMPUTING |
title_short |
GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA |
title_full |
GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA |
title_fullStr |
GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA |
title_full_unstemmed |
GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA |
title_sort |
GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA |
dc.creator.none.fl_str_mv |
Rosenberg, Duane Mininni, Pablo Daniel Reddy, Raghu Pouquet, Annick |
author |
Rosenberg, Duane |
author_facet |
Rosenberg, Duane Mininni, Pablo Daniel Reddy, Raghu Pouquet, Annick |
author_role |
author |
author2 |
Mininni, Pablo Daniel Reddy, Raghu Pouquet, Annick |
author2_role |
author author author |
dc.subject.none.fl_str_mv |
COMPUTATIONAL FLUIDS CUDA GPU MPI NUMERICAL SIMULATION OPENMP PARALLEL COMPUTING |
topic |
COMPUTATIONAL FLUIDS CUDA GPU MPI NUMERICAL SIMULATION OPENMP PARALLEL COMPUTING |
purl_subject.fl_str_mv |
https://purl.org/becyt/ford/1.5 https://purl.org/becyt/ford/1 https://purl.org/becyt/ford/1.2 https://purl.org/becyt/ford/1 |
dc.description.none.fl_txt_mv |
An existing hybrid MPI-OpenMP scheme is augmented with a CUDA-based fine grain parallelization approach for multidimensional distributed Fourier transforms, in a well-characterized pseudospectral fluid turbulence code. Basics of the hybrid scheme are reviewed, and heuristics provided to show a potential benefit of the CUDA implementation. The method draws heavily on the CUDA runtime library to handle memory management and on the cuFFT library for computing local FFTs. The manner in which the interfaces to these libraries are constructed, and ISO bindings utilized to facilitate platform portability, are discussed. CUDA streams are implemented to overlap data transfer with cuFFT computation. Testing with a baseline solver demonstrated significant aggregate speed-up over the hybrid MPI-OpenMP solver by offloading to GPUs on an NVLink-based test system. While the batch streamed approach provided little benefit with NVLink, we saw a performance gain of 30% when tuned for the optimal number of streams on a PCIe-based system. It was found that strong GPU scaling is nearly ideal, in all cases. Profiling of the CUDA kernels shows that the transform computation achieves 15% of the attainable peak FlOp-rate based on a roofline model for the system. In addition to speed-up measurements for the fiducial solver, we also considered several other solvers with different numbers of transform operations and found that aggregate speed-ups are nearly constant for all solvers. Fil: Rosenberg, Duane. State University of Colorado - Fort Collins; Estados Unidos Fil: Mininni, Pablo Daniel. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Física de Buenos Aires. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Instituto de Física de Buenos Aires; Argentina Fil: Reddy, Raghu. Environmental Modeling Center; Estados Unidos Fil: Pouquet, Annick. State University of Colorado at Boulder; Estados Unidos. National Center for Atmospheric Research; Estados Unidos |
description |
An existing hybrid MPI-OpenMP scheme is augmented with a CUDA-based fine grain parallelization approach for multidimensional distributed Fourier transforms, in a well-characterized pseudospectral fluid turbulence code. Basics of the hybrid scheme are reviewed, and heuristics provided to show a potential benefit of the CUDA implementation. The method draws heavily on the CUDA runtime library to handle memory management and on the cuFFT library for computing local FFTs. The manner in which the interfaces to these libraries are constructed, and ISO bindings utilized to facilitate platform portability, are discussed. CUDA streams are implemented to overlap data transfer with cuFFT computation. Testing with a baseline solver demonstrated significant aggregate speed-up over the hybrid MPI-OpenMP solver by offloading to GPUs on an NVLink-based test system. While the batch streamed approach provided little benefit with NVLink, we saw a performance gain of 30% when tuned for the optimal number of streams on a PCIe-based system. It was found that strong GPU scaling is nearly ideal, in all cases. Profiling of the CUDA kernels shows that the transform computation achieves 15% of the attainable peak FlOp-rate based on a roofline model for the system. In addition to speed-up measurements for the fiducial solver, we also considered several other solvers with different numbers of transform operations and found that aggregate speed-ups are nearly constant for all solvers. |
publishDate |
2020 |
dc.date.none.fl_str_mv |
2020-02 |
dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion http://purl.org/coar/resource_type/c_6501 info:ar-repo/semantics/articulo |
format |
article |
status_str |
publishedVersion |
dc.identifier.none.fl_str_mv |
http://hdl.handle.net/11336/146032 Rosenberg, Duane; Mininni, Pablo Daniel; Reddy, Raghu; Pouquet, Annick; GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA; Molecular Diversity Preservation International; Atmosphere; 11; 2; 2-2020; 1-22 2073-4433 CONICET Digital CONICET |
url |
http://hdl.handle.net/11336/146032 |
identifier_str_mv |
Rosenberg, Duane; Mininni, Pablo Daniel; Reddy, Raghu; Pouquet, Annick; GPU parallelization of a hybrid pseudospectral geophysical turbulence framework using CUDA; Molecular Diversity Preservation International; Atmosphere; 11; 2; 2-2020; 1-22 2073-4433 CONICET Digital CONICET |
dc.language.none.fl_str_mv |
eng |
language |
eng |
dc.relation.none.fl_str_mv |
info:eu-repo/semantics/altIdentifier/url/https://www.mdpi.com/2073-4433/11/2/178 info:eu-repo/semantics/altIdentifier/doi/10.3390/atmos11020178 |
dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
eu_rights_str_mv |
openAccess |
rights_invalid_str_mv |
https://creativecommons.org/licenses/by-nc-sa/2.5/ar/ |
dc.format.none.fl_str_mv |
application/pdf application/pdf |
dc.publisher.none.fl_str_mv |
Molecular Diversity Preservation International |
publisher.none.fl_str_mv |
Molecular Diversity Preservation International |
dc.source.none.fl_str_mv |
reponame:CONICET Digital (CONICET) instname:Consejo Nacional de Investigaciones Científicas y Técnicas |
reponame_str |
CONICET Digital (CONICET) |
collection |
CONICET Digital (CONICET) |
instname_str |
Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.name.fl_str_mv |
CONICET Digital (CONICET) - Consejo Nacional de Investigaciones Científicas y Técnicas |
repository.mail.fl_str_mv |
dasensio@conicet.gov.ar; lcarlino@conicet.gov.ar |
_version_ |
1842268619056087040 |
score |
13.13397 |