Linux

🐧 Installing Docker on Linux (Ubuntu)

This guide walks you through installing Docker on a Linux system (tested on Ubuntu) and optionally enabling NVIDIA GPU support using the NVIDIA Container Toolkit.

⚠️ Note: This is a general guide. Your system may differ slightly.
For help, check the official Docker docs, NVIDIA docs, or ask in our Discord.

---

📦 Step 1: Install Required Packages

sudo apt update
sudo apt install -y \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

---

🔑 Step 2: Add Docker’s GPG Key

sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
    sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

---

📚 Step 3: Set Up Docker Repository

echo \
  "deb [arch=$(dpkg --print-architecture) \
  signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

---

🐳 Step 4: Install Docker Engine

sudo apt update
sudo apt install -y \
    docker-ce \
    docker-ce-cli \
    containerd.io \
    docker-buildx-plugin \
    docker-compose-plugin

---

🔧 Step 5: Enable Docker Without sudo

sudo usermod -aG docker $USER
newgrp docker

Test Docker:

docker run hello-world

---

⚡ Step 6: Install NVIDIA Container Toolkit for GPU Support

Only needed if you have an NVIDIA GPU and want hardware acceleration.

🛠️ Prerequisites

Make sure your system has:

• A supported NVIDIA GPU
• NVIDIA drivers installed (run nvidia-smi to check)

📦 Install the NVIDIA Container Toolkit

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
    sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit.gpg

curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
    sed 's#https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit.gpg] https://#' | \
    sudo tee /etc/apt/sources.list.d/nvidia-docker.list > /dev/null

sudo apt update
sudo apt install -y nvidia-container-toolkit
sudo systemctl restart docker

---

✅ Step 7: Test GPU Support

docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi

You should see your GPU info printed from inside the container.

If not, check:

• Drivers are installed (nvidia-smi works on host)
• You installed nvidia-container-toolkit
• Docker was restarted

---

🎯 Next Step: Set Up the JoeyLLM Container

Now that Docker is ready, continue here:
🚀 Launching the JoeyLLM Development Container