Skip to content

Commit

Permalink
Rename runner.sh -> entrypoint.sh
Browse files Browse the repository at this point in the history
Rename the entrypoint script to `entrypoint.sh` to better reflect its
purpose and to avoid confusion with the `runner.sh` script in the root
directory. Rewrite the various READMEs and documentation files to
reflect the change.

Describe the additional level of directory isolation in the
documentation.
  • Loading branch information
efrecon committed Mar 27, 2024
1 parent 3e52de1 commit 459eece
Show file tree
Hide file tree
Showing 4 changed files with 57 additions and 54 deletions.
52 changes: 28 additions & 24 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,40 +11,43 @@ This document contains notes about the internals of the implementation.

When environment isolation is turned on, i.e. when the variable
`ORCHESTRATOR_ISOLATION` is turned on, the processes will communicate through a
common (temporary) directory created in the orchestrator and stored in the
variable `ORCHESTRATOR_ENVIRONMENT`. That directory is mounted into the microVM
at `/_environment`.
common (temporary) directory created by the orchestrator and stored in the
variable `ORCHESTRATOR_ENVIRONMENT`. Each runner loop will be associated to a
separate sub-directory (the `RUNNER_ENVIRONMENT` variable) and that directory is
mounted into the microVM at `/_environment`. This provides isolation between the
different running loops.

Runners are identified using a loop iteration, e.g. `1`, `2`, etc. followed by a
random string (and separated by a `-` (dash sign))

The orchestrator will wait for a file with the `.tkn` extension and named after
the loop iteration, i.e. independently of the random string. That token file is
set by the `runner.sh` script running inside the microVM. This file is created
by the microVM once the runner has been registered, but not started, at GitHub.
It contains the result of the `token.sh` script, i.e. the runner registration
token.
set by the `entrypoint.sh` script running inside the microVM. This file is
created by the microVM once the runner has been registered, but not started, at
GitHub. It contains the result of the `token.sh` script, i.e. the runner
registration token.

Each runner loop implemented in the `runner.sh` script is allocated a "secret"
(a random string). When a termination signal is caught inside the `runner.sh`
script inside the microVM, a file with the same name (and location) as the token
file, but the extension `.brk` (break) is created with the content of the
secret. Once a microVM has ended, the `runner.sh` loop script will detect if the
`.brk` file exists and contains the secret. If it does, it will abort the loop
-- instead of creating yet another runner. Using a random secret is for security
and to avoid that workflows are able to actually force end the runner loop.
Since the value of the secret is passed through the `.env` file that is
automatically removed as soon as the microVM has booted is running the
`runner.sh` script, workflows are not able to break the external loop: they are
able to create files in the `/_environment` directory, but they cannot know the
value of the secret to put into the file to force the exiting handshake.
(a random string). When a termination signal is caught inside the
`entrypoint.sh` script inside the microVM, a file with the same name (and
location) as the token file, but the extension `.brk` -- for "break" -- is
created with the content of the secret. Once a microVM has ended, the
`runner.sh` loop script will detect if the `.brk` file exists and contains the
secret. If it does, it will abort the loop -- instead of creating yet another
runner. Using a random secret is for security and to avoid that workflows are
able to actually force end the runner loop. Since the value of the secret is
passed through the `.env` file that is automatically removed as soon as the
microVM has booted and is running the `entrypoint.sh` script, workflows are not
able to break the external loop: they are able to create files in the
`/_environment` directory, but they cannot know the value of the secret to put
into the file to force the exiting handshake.

The same type of handshaking happens when the main runner loop is terminating,
for example after the life-time period provided with the command-line option
`-k`. In that case, a file containing the secret and ending with the `.trm`
extension is created in what the VM sees as the `/_environment` directory. When
such a file is present, the main `runner.sh` script inside the VM will kill the
GitHub runner process and unregister it.
`-k`. In that case, a file containing the secret and ending with the `.trm` --
for "terminate" -- extension is created in what the VM sees as the
`/_environment` directory. When such a file is present, the main `entrypoint.sh`
script inside the VM will kill the GitHub runner process and unregister it.

## Changes to the Installation Scripts

Expand All @@ -62,4 +65,5 @@ Note that when changing the logic of the "entrypoints", i.e. the scripts run at
microVM initialisation, you do not need to wait for the image to be created.
Instead, pass `-D /local` to the [`runner.sh`](./runner.sh) script. This will
mount the [`runner`](./runner/) directory into the microVM at `/local` and run
the scripts that it contains from there instead.
the scripts that it contains from there instead. Which "entrypoint" to use is
driven by the `RUNNER_ENTRYPOINT` variable in [`runner.sh`](./runner.sh).
53 changes: 26 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,15 +39,14 @@ the base repository, e.g. `ubuntu` and `krunvm`. The GitHub runner
implementation will automatically add other labels in addition to those.

In the example above, the double-dash `--` separates options given to the
user-facing [orchestrator] from options to the loop implementation
[runner](./runner.sh) script. All options appearing after the `--` will be
blindly passed to the [runner] loop and script. All scripts within the project
accepts short options only and can either be controlled through options or
environment variables -- but CLI options have precedence. Running scripts with
the `-h` option will provide help and a list of those variables. Variables
starting with `ORCHESTRATOR_` will affect the behaviour of the [orchestrator],
while variables starting with `RUNNER_` will affect the behaviour of each
[runner] (loop).
user-facing [orchestrator] from options to the loop implementation [runner]
script. All options appearing after the `--` will be blindly passed to the
[runner] loop and script. All scripts within the project accepts short options
only and can either be controlled through options or environment variables --
but CLI options have precedence. Running scripts with the `-h` option will
provide help and a list of those variables. Variables starting with
`ORCHESTRATOR_` will affect the behaviour of the [orchestrator], while variables
starting with `RUNNER_` will affect the behaviour of each [runner] (loop).

[orchestrator]: ./orchestrator.sh
[runner]: ./runner.sh
Expand All @@ -69,8 +68,9 @@ while variables starting with `RUNNER_` will affect the behaviour of each
critical software tools.
+ Good compatibility with the regular GitHub [runners]: same user ID, member of
the `docker` group, etc.
+ In theory, the main [image] should be able to be used in more traditional
container-based solutions -- perhaps [sysbox]? Reports/changes are welcome.
+ In theory, the main [ubuntu] and [fedora] images should be able to be used in
more traditional container-based solutions -- perhaps [sysbox]?
Reports/changes are welcome.

[sysbox]: https://github.com/nestybox/sysbox

Expand Down Expand Up @@ -122,13 +122,12 @@ permissions.

## Architecture and Design

The [orchestrator](./orchestrator.sh) creates as many loops of ephemeral runners
as requested. These loops are implemented as part of the
[runner.sh](./runner.sh) script: the script will create a microVM based on the
default image (see below), memory and vCPU requirement. It will then start that
microVM using `krunvm` and that will start an (ephemeral) [runner][self]. As
soon as a job has been executed on that runner, the microVM will end and a new
will be created.
The [orchestrator] creates as many loops of ephemeral runners as requested.
These loops are implemented as part of the [runner.sh][runner] script: the
script will create a microVM based on the default image (see below), memory and
vCPU requirement. It will then start that microVM using `krunvm` and that will
start an (ephemeral) GitHub [runner][self]. As soon as a job has been executed
on that runner, the microVM will end and a new will be created.

The OCI image is built in two parts:

Expand All @@ -150,15 +149,15 @@ containers with the `--network host` option. This is made transparent through a
docker CLI [wrapper](./base/docker.sh) that will automatically add this option
to all (relevant) commands.

When the microVM starts, the [runner.sh](./runner/runner.sh) script will be
started. This script will pick its options using an `.env` file, shared from the
host. The file will be sourced and removed at once. This ensures that secrets
are not leaked to the workflows through the process table or a file. Upon start,
the script will [request](./runner/token.sh) a runner token, configure the
runner and then start the actions runner .NET implementation, under the `runner`
user. The `runner` user shares the same id as the one at GitHub and is also a
member of the `docker` group. Similarily to GitHub runners, the user is capable
of `sudo` without a password.
When the microVM starts, the [entrypoint.sh](./runner/entrypoint.sh) script will
be started. This script will pick its options using an `.env` file, shared from
the host. The file will be sourced and removed at once. This ensures that
secrets are not leaked to the workflows through the process table or a file.
Upon start, the script will [request](./runner/token.sh) a runner token,
configure the runner and then start the actions runner .NET implementation,
under the `runner` user. The `runner` user shares the same id as the one at
GitHub and is also a member of the `docker` group. Similarily to GitHub runners,
the user is capable of `sudo` without a password.

Runner tokens are written to the directory that is shared with the host. This is
used during initial synchronisation, to avoid starting up several runners at the
Expand Down
6 changes: 3 additions & 3 deletions runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -195,10 +195,10 @@ check_positive_number "$RUNNER_MEMORY" "Memory (in MB)"
# Decide which runner.sh implementation (this is the "entrypoint" of the
# microVM) to use: the one from the mount point, or the built-in one.
if [ -z "$RUNNER_DIR" ]; then
RUNNER_ENTRYPOINT=/opt/gh-runner-krunvm/bin/runner.sh
RUNNER_ENTRYPOINT=/opt/gh-runner-krunvm/bin/entrypoint.sh
else
check_command "${RUNNER_ROOTDIR}/runner/runner.sh"
RUNNER_ENTRYPOINT=${RUNNER_DIR%/}/runner/runner.sh
check_command "${RUNNER_ROOTDIR}/runner/entrypoint.sh"
RUNNER_ENTRYPOINT=${RUNNER_DIR%/}/runner/entrypoint.sh
fi

# Create the VM used for orchestration. Add --volume options for all necessary
Expand Down
File renamed without changes.

0 comments on commit 459eece

Please sign in to comment.