Skip to content

Experiment with IB/OFED libs from distro repo #756

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Experiment with IB/OFED libs from distro repo #756

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8391274
bump CUDA to 12.4.0
yhtang Apr 9, 2024
537ce5f
Merge branch 'main' into bump-cuda-12.4
yhtang Apr 10, 2024
3711dbe
Merge branch 'main' into bump-cuda-12.4
yhtang Apr 13, 2024
0dc4032
12.4.1
yhtang Apr 23, 2024
7b356c9
Experiment with IB/OFED libs from distro repo
yhtang Apr 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 11 additions & 8 deletions .github/container/Dockerfile.base
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# syntax=docker/dockerfile:1-labs
ARG BASE_IMAGE=nvidia/cuda:12.3.0-devel-ubuntu22.04
ARG BASE_IMAGE=nvidia/cuda:12.4.1-devel-ubuntu22.04
ARG GIT_USER_NAME="JAX Toolbox"
ARG [email protected]
ARG SRC_MANIFEST_FILE=manifest.yaml
Expand Down Expand Up @@ -59,6 +59,16 @@ apt_packages=(
wget
# llvm.sh
lsb-release software-properties-common
# OFED for RoCE and InfiniteBand support
rdma-core
libibverbs1
libibverbs-dev
librdmacm1
librdmacm-dev
libibumad3
libibumad-dev
ibverbs-utils
ibverbs-providers
)
if [[ $(dpkg --print-architecture) == arm64 ]]; then
# h5py: The newest release of of h5py (3.11.0) does not include ARM wheels and causes pip to build h5py.
Expand Down Expand Up @@ -155,13 +165,6 @@ RUN install-cudnn.sh
ADD install-nccl.sh /usr/local/bin
RUN install-nccl.sh

###############################################################################
## RoCE and InfiniteBand support
###############################################################################

ADD install-ofed.sh /usr/local/bin
RUN install-ofed.sh

##############################################################################
## Amazon EFA support (need to run it inside container separately)
##############################################################################
Expand Down