Towards high-end scalability on biologically-inspired computational models

• Autores: Dematties, Dario Jesus; Thiruvathukal, George K.; Rizzi, Silvio; Wainselboim, Alejandro Javier; Zanutto, Bonifacio Silvano
• Año de publicación: 2020
• Idioma: inglés
• Tipo de recurso: parte de libro
• Materia: MPI; OPENMP; CENTRAL PROCESSING UNITS; BIOLOGICAL MODELS; NEUROSCIENCE; IRREGULAR COMPUTATION
• Institución: Consejo Nacional de Investigaciones Científicas y Técnicas

The interdisciplinary field of neuroscience has made significant progress in recent decades, providing the scientific community in general with a new level of understanding on how the brain works beyond the store-and-fire model found in traditional neural networks. Meanwhile, Machine Learning (ML) based on established models has seen a surge of interest in the High Performance Computing (HPC) community, especially through the use of high-end accelerators, such as Graphical Processing Units(GPUs), including HPC clusters of same. In our work, we are motivated to exploit these high-performance computing developments and understand the scaling challenges for new–biologically inspired–learning models on leadership-class HPC resources. These emerging models feature sparse and random connectivity profiles that map to more loosely-coupled parallel architectures with a large number of CPU cores per node. Contrasted with traditional ML codes, these methods exploit loosely-coupled sparse data structures as opposed to tightly-coupled dense matrix computations, which benefit from SIMD-style parallelism found on GPUs. In this paper we introduce a hybrid Message Passing Interface (MPI) and Open Multi-Processing (OpenMP) parallelization scheme to accelerate and scale our computational model based on the dynamics of cortical tissue. We ran computational tests on a leadership class visualization and analysis cluster at Argonne National Laboratory. We include a study of strong and weak scaling, where we obtained parallel efficiency measures with a minimum above 87% and a maximum above 97% for simulations of our biologically inspired neural network on up to 64 computing nodes running 8 threads each. This study shows promise of the MPI+OpenMP hybrid approach to support flexible and biologically-inspired computational experimental scenarios. In addition, we present the viability in the application of these strategies in high-end leadership computers in the future.
Fil: Dematties, Dario Jesus. Universidad de Buenos Aires. Facultad de Ingeniería. Instituto de Ingeniería Biomédica; Argentina
Fil: Thiruvathukal, George K.. University of Chicago; Estados Unidos
Fil: Rizzi, Silvio. Argonne National Laboratory; Estados Unidos
Fil: Wainselboim, Alejandro Javier. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Mendoza. Instituto de Ciencias Humanas, Sociales y Ambientales; Argentina
Fil: Zanutto, Bonifacio Silvano. Universidad de Buenos Aires. Facultad de Ingeniería. Instituto de Ingeniería Biomédica; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Instituto de Biología y Medicina Experimental. Fundación de Instituto de Biología y Medicina Experimental. Instituto de Biología y Medicina Experimental; Argentina