GPU programming with HIP

Course material for the CSC course "GPU programming with HIP". The course is part of the EuroCC training activities at CSC.

Agenda

Day 1

Time	Topic
09:00–09:30	Welcome, LUMI access, slurm, git, web interface (JL)
09:30–09:45	Break/debugging access
09:45–10:30	Introduction to GPU programming (JL)
10:30–10:45	Break & Snacks
10:45-11:30	HIP and GPU kernels (JL)
11:30-11:45	Break
11:45-12:15	Exercises
12:15-13:00	Lunch
13:00-13:45	Streams, events, and synchronization (JK)
13:45-14:00	Break
14:00-14:30	Exercises
14:30-15:15	Memory allocations, access and unified memory (JK)
15:15-15:30	Break
15:30-16:00	Exercises
16:00-16:15	Day summary

Day 2

Time	Topic
09:00–10:00	Kernel optimizations (JK)
10:00–10:15	Break & Snacks
10:15–10:45	Exercises
10:45-11:30	Multi-GPU programming, HIP+MPI (CA)
11:30-11:45	Break
11:45-12:15	Exercises
12:15-13:00	Lunch
13:00-13:30	Fortran and HIP (CA)
13:30-13:45	Break
13:45-14:15	Exercises
13:30-14:15	Porting Applications to HIP (CA)
14:15-14:30	Break
14:30-15:45	Exercises
15:45-16:00	Close-up

Slides

Link to slides

First steps

• Which technologies have you used?
• First steps

Exercises

General instructions

Introduction and GPU kernels

• Mental model quiz
• Hello world
• Error checking
• Kernel saxpy
• Kernel copy2d

Streams, events, and synchronization

• Understanding asynchronity using events
• Investigating streams and events

Memory allocations, access, and unified memory

• Memory management strategies
• The stream-ordered memory allocator and memory pools
• Unified memory and structs

Fortran and HIP

• SAXPY
• HIPRAND

Optimization

• Coalescing
• Matrix Transpose
• Tracing

Multi-GPU programming and HIP+MPI

• Peer to peer device access
• Vector sum on two GPUs without MPI
• Ping-pong with multiple GPUs and MPI

Porting to HIP

• HIPIFY Tools

Bonus

• Heat equation with HIP