tutorial-multi-gpu

KO | EN

gitlite — search

#javascript #llm #rust #deep-learning #docker #java #nodejs #awesome #kubernetes #awesome-list #security #openai #ios #open-source

tutorial-multi-gpu

(★ 382)

Efficient Distributed GPU Programming for Exascale, an SC/ISC Tutorial

File Explorer

Download Latest Version (.zip)

- - .set-facl-permissions.sh
  - deploy.sh
  - jsccourse-bashrc.sh
  - modules.sh
  - README.md
- .gitignore
- .update-scripts.sh
- deploy-material.sh
- env.sh
- instructions-header.md
- instructions-update-header.sh
- printout.tex
- - ci.yml
- slides.pdf
- slides.pdf
- slides.pdf
- - copy.mk
  - Instructions.ipynb
  - Instructions.md
  - jacobi.cu
  - Makefile.in
- - Instructions.ipynb
  - Instructions.md
  - jacobi.cu
  - Makefile
- - Instructions.ipynb
  - Instructions.md
  - jacobi.cu
  - Makefile
- slides.pdf
- slides.pdf
- - copy.mk
  - Instructions.ipynb
  - Instructions.md
  - jacobi.cpp
  - jacobi_kernels.cu
  - Makefile.in
- - Instructions.ipynb
  - Instructions.md
  - jacobi.cpp
  - jacobi_kernels.cu
  - Makefile
- - Instructions.ipynb
  - Instructions.md
  - jacobi.cpp
  - jacobi_kernels.cu
  - Makefile
- slides.pdf
- - copy.mk
- slides.pdf
- - copy.mk
- slides.pdf
.archive.mk
.gitignore
.template.json
.zenodo.json
CITATION.cff
LICENSE
README.md

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# ISC26 Tutorial: Efficient Distributed GPU Programming for Exascale

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.5745504.svg)](https://doi.org/10.5281/zenodo.5745504)

Repository with talks and exercises of our Efficient GPU Programming for Exascale tutorial, to be held at [ISC26](https://app.swapcard.com/event/isc-high-performance-2026/planning/UGxhbm5pbmdfNDM5MDIyMg==).

## Coordinates

* Date: 22 July 2026
* Occasion: ISC26 Tutorial (Full Day)
* Tutors: Andreas Herten (JSC), Laura Morgenstern (NVIDIA), Lena Oden (Uni Hagen)
* Support: David Appelhans (NVIDIA), Simon Garcia de Gonzalo (SNL), Markus Hrywniak (NVIDIA) and Jiri Kraus (NVIDIA)

## Setup

The tutorial is an interactive tutorial with introducing lectures and practical exercises to apply knowledge. The exercises have been derived from the Jacobi solver implementations available in [NVIDIA/multi-gpu-programming-models](https://github.com/NVIDIA/multi-gpu-programming-models).

Walk-through (only possible on-site at ISC26!):

* Sign up at JuDoor
* Open Jupyter JSC: https://jupyter.jsc.fz-juelich.de
* Create new Jupyter instance on [JUPITER, using training26XX account, on **LoginNode**](https://jupyter.jsc.fz-juelich.de/workshops/isc26mg) [^1]
* Source course environment: `source $PROJECT_training26XX/env.sh`
* Sync material: `jsc-material-sync`
* Locally install NVIDIA Nsight Systems: https://developer.nvidia.com/nsight-systems

1. Lecture: Tutorial Overview, Introduction to System + Onboarding *Andreas*
2. Lecture: MPI-Distributed Computing with GPUs *Lena*
3. Hands-on: Multi-GPU Parallelization
4. Lecture: Performance / Debugging Tools *Laura*
5. Lecture: Optimization Techniques for Multi-GPU Applications *Lena*
6. Hands-on: Overlap Communication and Computation with MPI
7. Lecture: Overview of NCCL and NVSHMEN in MPI *Lena*
8. Hands-on: Using NCCL and NVSHMEM
9. Lecture: Device-initiated Communication with NVSHMEM *Laura*
10. Hands-on: Using Device-Initiated Communication with NVSHMEM
11. Lecture: Conclusion and Outline of Advanced Topics *Andreas*

[^1]: JUPITER is the default. In the case the tutors tell to move to the backup (JUWELS Booster), launch a [new Jupyter instance here](https://jupyter.jsc.fz-juelich.de/workshops/isc26mg-juwels).

tutorial-multi-gpu

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

tutorial-multi-gpu

File Explorer

Today's picks

Was this content helpful?

Recent Feedback

Download README