Skip to content

Commit

Permalink
feat(shaka-lab-github-runner): Add support for nested containers (#63)
Browse files Browse the repository at this point in the history
If you want to run workflows with jobs that inside containers, you need support for nested containers.  This is now possible with a little bit of configuration.

To make this work, we need to synchronize a few important folders between the host and the main container so that they can be forwarded on correctly to nested containers.

Part of the solution to shaka-project/static-ffmpeg-binaries#28
  • Loading branch information
joeyparrish authored Oct 10, 2024
1 parent 04a7d3c commit 695cc80
Show file tree
Hide file tree
Showing 2 changed files with 44 additions and 3 deletions.
7 changes: 7 additions & 0 deletions shaka-lab-github-runner/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,13 @@ them in text files inside `/etc/shaka-lab-github-runner.args.d/`.
To add Docker command line arguments that apply to specific runner instances,
add them in text files inside `/etc/shaka-lab-github-runner@$INSTANCE.args.d/`.

To support nested containers, put this in
`/etc/shaka-lab-github-runner.args.d/docker-nested`:

```
-v /var/run/docker.sock:/var/run/docker.sock
```

## Updates

```sh
Expand Down
40 changes: 37 additions & 3 deletions shaka-lab-github-runner/linux/start-runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -85,11 +85,46 @@ if dpkg -s shaka-lab-cert-generator &>/dev/null || \
extra_docker_args+=(--mount type=bind,src=/etc/letsencrypt,dst=/etc/letsencrypt,ro)
fi

# To support nested containers in self-hosted workflows, certain folders
# expected by GitHub Actions must be consistently mapped from the outer host to
# the first container. To keep workflows ephemeral, we also wipe these before
# every run.
RUNNER_WORKDIR=/home/runner/work
MAPPED_FOLDERS=(
$RUNNER_WORKDIR
/opt/hostedtoolcache
)
for i in "${MAPPED_FOLDERS[@]}"; do
rm -rf "$i"
mkdir -p "$i"
extra_docker_args+=(--mount type=bind,src="$i",dst="$i")
done

# This folder already exists inside the container image, but we want to keep our
# own copy of it at the host level. This will allow it to be correctly mapped
# to nested containers, and modified if necessary.
EXTERNALS=/actions-runner/externals
rm -rf "$EXTERNALS"
mkdir -p "$EXTERNALS"

# Create a temporary docker container to extract these files.
docker pull "$DOCKER_IMAGE"
docker container create --name "$CONTAINER_NAME" "$DOCKER_IMAGE"

# Copy "$EXTERNALS" itself from the container into the local parent of the same.
# This is because "docker cp" doesn't do wildcards, so you can't copy "e/* e/".
docker cp "$CONTAINER_NAME":"$EXTERNALS" "$EXTERNALS"/..

# Clean up the temporary container.
docker container rm "$CONTAINER_NAME"

# Create a special mount for this folder.
extra_docker_args+=(--mount type=bind,src="$EXTERNALS",dst="$EXTERNALS",ro)

# Start a docker container.
# --rm: Remove the container when it shuts down.
# --name: The name of the container.
# --network host: Use the host directly for networking, rather than NAT.
# --pull always: Always use the most up-to-date docker image.
# -e ALLOCATED_PORT=...: A port number allocated to this instance. Not every
# workflow needs this, but Shaka Player does.
# -e RUNNER_NAME=...: The runner name, which shows up on GitHub Actions.
Expand All @@ -100,10 +135,9 @@ docker run \
--rm \
--name "$CONTAINER_NAME" \
--network host \
--pull always \
-e ALLOCATED_PORT=$(( 61700 + $INSTANCE )) \
-e RUNNER_NAME="$RUNNER_NAME" \
-e RUNNER_WORKDIR=/tmp/runner/work \
-e RUNNER_WORKDIR="$RUNNER_WORKDIR" \
-e DISABLE_AUTO_UPDATE=1 \
-e EPHEMERAL=1 \
"${extra_docker_args[@]}" \
Expand Down

0 comments on commit 695cc80

Please sign in to comment.