Add documentation

README.md

@@ -0,0 +1,148 @@
+# krunvm-based GitHub Runner(s)
+
+This project creates [self-hosted][self] (ephemeral) GitHub [runners] based on
+[krunvm]. [krunvm] creates [microVM]s, so the project enables fully isolated
+[runners] inside your infrastruture, as opposed to [solutions] based on
+Kubernetes or Docker containers. MicroVMs boot fast, providing an experience
+close to running containers. [krunvm] creates and starts VM based on the
+multi-platform OCI [images][image] created for this project.
+
+  [self]: https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/about-self-hosted-runners
+  [runners]: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners
+  [krunvm]: https://github.com/containers/krunvm
+  [microVM]: https://github.com/infracloudio/awesome-microvm
+  [solutions]: https://github.com/jonico/awesome-runners
+  [image]: https://github.com/efrecon/gh-runner-krunvm/pkgs/container/runner-krunvm
+
+## Example
+
+Provided you are at the root directory of this project, the following would
+create two runners that are bound to *this* repository (the
+`efrecon/gh-runner-krunvm` principal). Runners can also be registered at the
+`organization` or `enterprise` scope using the `-s` option. In the example
+below, the value of the `-T` option should be a [PAT].
+
+```bash
+./orchestrator.sh -v -T ghp_XXXX -p efrecon/gh-runner-krunvm -- 2
+```
+
+The project tries to have good default options and behaviour -- run with `-h`
+for a list of options. For example, nor the value of the token, nor the value of
+the runner registration token will be visible to the workflows using your
+runners. The default is however to create far-less capable runners than the
+GitHub [runners], i.e. 1G or memory and 2 vCPUs. By default, runners have random
+names and carry labels with the name of the base repository, e.g. `fedora` and
+`krunvm`. The GitHub runner implementation will automatically add other labels
+in addition to those.
+
+All scripts within the project accepts short options only and can either be
+controlled through options or environment variables. Running with the `-h`
+option will provide help and a list of those variables. Variables starting with
+`ORCHESTRATOR_` will affect the behaviour of the [orchestrator], while variables
+starting with `RUNNER_` will affect the behaviour of each runner. Usually, the
+only script that you will be using is the [orchestrator](./orchestrator.sh). But
+it is possible to create the microVM with the orchestrator and manually run
+loops using the [runner](./runner.sh) script.
+
+  [PAT]: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens
+
+## Features
+
++ Fully isolated GitHub [runners] on your [infrastructure][self], through
+  microVM.
++ container-like experience: microVMs boot quickly.
++ No special network configuration
++ Ephemeral runners, i.e. will start from a pristine "empty" state at each run.
++ Secrets isolation to avoid leaking to workflows.
++ Run on amd64 and arm64 platforms, probably able to run on MacOS.
++ Standard "medium-sized" base OS installations (node, python, dev tools, etc.)
++ Run on top of any OCI image -- base "OS" separated from runner installation.
++ Support for registration at the repository, organisation and enterprise level.
++ Support for github.com, but also local installations of the forge.
++ Ability to mount local directories to cache local runner-based requirements or
+  critical software tools.
++ Good compatibility with the regular GitHub [runners]: same user ID, member of
+  the `docker` group, etc.
++ In theory, the main [image] should be able to be used in more traditional
+  container-based solutions -- perhaps [sysbox]? Reports/changes are welcome.
+
+  [sysbox]: https://github.com/nestybox/sysbox
+
+## Requirements
+
+This project is coded in pure POSIX shell and has only been tested on Linux. The
+images are automatically [built] both for x86_64 and AArch64. However, [krunvm]
+also runs on MacOS. No "esoteric" options have been used when using the standard
+UNIX binary utilities. PRs are welcome to make the project work on MacOS, if it
+does not already.
+
+Apart from the standard UNIX binary utilities, you will need the following
+installed on the host. Installation is easiest on Fedora
+
++ `curl`
++ `jq`
++ `buildah`
++ `krunvm` (and its [requirements])
+
+  [built]: ./.github/workflows/ci.yml
+  [requirements]: https://github.com/containers/krunvm#installation
+
+## Limitations
+
++ Linux host installation easiest on Fedora
++ Runners are (also) based on Fedora. While standard images are based on Fedora,
+  running on top of ubuntu should also be possible.
++ Inside the runners: Docker not supported. Replaced by `podman` in [emulation]
+  mode.
++ Inside the runners: No support for docker network, containers run in "host"
+  (but: inside the microVM) networking mode only. This is alleviated by a docker
+  [shim](./base/docker.sh)
+
+  [emulation]: https://docs.podman.io/en/latest/markdown/podman-system-service.1.html
+
+## Architecture and Design
+
+The [orchestrator](./orchestrator.sh) focuses on creating (but not starting) a
+microVM based on the default OCI image (see below). It then creates as many
+loops of ephemeral runners as requested. These loops are implemented as part of
+the [runner.sh](./runner.sh) script: the script will start a microVM that will
+start an (ephemeral) [runner][self]. As soon as a job has been executed on that
+runner, the microVM will end and a new will be created.
+
+The OCI image is built in two parts:
+
++ The [base](./Dockerfile.base) image installs a minimal set of binaries and
+  packages, both the ones necessary to execute the runner, but also a sane
+  minimal default for workflows. Regular GitHub [runners] have a wide number of
+  installed packages. The base image has much less. Also note that it is based
+  on Fedora, rather than Ubuntu.
++ The [main](./Dockerfile) installs the runner binaries and scripts and creates
+  a directory structure that is used by the rest of the project.
+
+As Docker-in-Docker does not work in krunvm microVMs, the base image installs
+podman and associated binaries. This should be transparent to the workflows as
+podman will be run in the background, in compatibility mode, and listening to
+the Docker socket at its standard location. The Docker client (and compose and
+buildx plugins) are however installed on the base image. This is to ensure that
+most workflows should work without changes. The microVM also limits to running
+containers with the `--network host` option. This is made transparent through a
+docker CLI [wrapper](./base/docker.sh) that will automatically add this option
+to all (relevant) commands.
+
+When the microVM starts, the [runner.sh](./runner/runner.sh) script will be
+started. This script will pick its options using an `.env` file, shared from the
+host. The file will be sourced and removed at once. This ensures that secrets
+are not leaked to the workflows through the process table or a file. Upon start,
+the script will [request](./runner/token.sh) a runner token, configure the
+runner and then start the actions runner .NET implementation, under the `runner`
+user. The `runner` user shares the same id as the one at GitHub and is also a
+member of the `docker` group. Similarily to GitHub runners, the user is capable
+of `sudo` without a password.
+
+## History
+
+This project was written to combat my anxeity combatting my daughter's newly
+discovered eating disorder. It started as a rewrite of [this] project after
+having failed to run those images inside the microVMs generated by [krunvm].
+
+  [this]: https://github.com/myoung34/docker-github-actions-runner

base/base.sh

@@ -65,7 +65,7 @@ while getopts "dl:vh-" opt; do
     v) # Increase verbosity, will otherwise log on errors/warnings only
       BASE_VERBOSE=$((BASE_VERBOSE+1));;
     h) # Print help and exit
-      usage;;
+      usage 0 "BASE";;
     ?)
       usage 1;;
   esac

demo/demo-magic/demo-magic.sh

@@ -0,0 +1,228 @@
+#!/usr/bin/env bash
+
+###############################################################################
+#
+# demo-magic.sh
+#
+# Copyright (c) 2015-2022 Paxton Hare
+#
+# This script lets you script demos in bash. It runs through your demo script
+# when you press ENTER. It simulates typing and runs commands.
+#
+###############################################################################
+
+# the speed to simulate typing the text
+TYPE_SPEED=20
+
+# no wait after "p" or "pe"
+NO_WAIT=false
+
+# if > 0, will pause for this amount of seconds before automatically proceeding with any p or pe
+PROMPT_TIMEOUT=0
+
+# don't show command number unless user specifies it
+SHOW_CMD_NUMS=false
+
+
+# handy color vars for pretty prompts
+BLACK="\033[0;30m"
+BLUE="\033[0;34m"
+GREEN="\033[0;32m"
+GREY="\033[0;90m"
+CYAN="\033[0;36m"
+RED="\033[0;31m"
+PURPLE="\033[0;35m"
+BROWN="\033[0;33m"
+WHITE="\033[0;37m"
+BOLD="\033[1m"
+COLOR_RESET="\033[0m"
+
+C_NUM=0
+
+# prompt and command color which can be overriden
+DEMO_PROMPT="$ "
+DEMO_CMD_COLOR=$BOLD
+DEMO_COMMENT_COLOR=$GREY
+
+##
+# prints the script usage
+##
+function usage() {
+  echo -e ""
+  echo -e "Usage: $0 [options]"
+  echo -e ""
+  echo -e "  Where options is one or more of:"
+  echo -e "  -h  Prints Help text"
+  echo -e "  -d  Debug mode. Disables simulated typing"
+  echo -e "  -n  No wait"
+  echo -e "  -w  Waits max the given amount of seconds before "
+  echo -e "      proceeding with demo (e.g. '-w5')"
+  echo -e ""
+}
+
+##
+# wait for user to press ENTER
+# if $PROMPT_TIMEOUT > 0 this will be used as the max time for proceeding automatically
+##
+function wait() {
+  if [[ "$PROMPT_TIMEOUT" == "0" ]]; then
+    read -rs
+  else
+    read -rst "$PROMPT_TIMEOUT"
+  fi
+}
+
+##
+# print command only. Useful for when you want to pretend to run a command
+#
+# takes 1 param - the string command to print
+#
+# usage: p "ls -l"
+#
+##
+function p() {
+  if [[ ${1:0:1} == "#" ]]; then
+    cmd=$DEMO_COMMENT_COLOR$1$COLOR_RESET
+  else
+    cmd=$DEMO_CMD_COLOR$1$COLOR_RESET
+  fi
+
+  # render the prompt
+  x=$(PS1="$DEMO_PROMPT" "$BASH" --norc -i </dev/null 2>&1 | sed -n '${s/^\(.*\)exit$/\1/p;}')
+
+  # show command number is selected
+  if $SHOW_CMD_NUMS; then
+   printf "[$((++C_NUM))] $x"
+  else
+   printf "$x"
+  fi
+
+  # wait for the user to press a key before typing the command
+  if [ $NO_WAIT = false ]; then
+    wait
+  fi
+
+  if [[ -z $TYPE_SPEED ]]; then
+    echo -en "$cmd"
+  else
+    echo -en "$cmd" | pv -qL $[$TYPE_SPEED+(-2 + RANDOM%5)];
+  fi
+
+  # wait for the user to press a key before moving on
+  if [ $NO_WAIT = false ]; then
+    wait
+  fi
+  echo ""
+}
+
+##
+# Prints and executes a command
+#
+# takes 1 parameter - the string command to run
+#
+# usage: pe "ls -l"
+#
+##
+function pe() {
+  # print the command
+  p "$@"
+  run_cmd "$@"
+}
+
+##
+# print and executes a command immediately
+#
+# takes 1 parameter - the string command to run
+#
+# usage: pei "ls -l"
+#
+##
+function pei {
+  NO_WAIT=true pe "$@"
+}
+
+##
+# Enters script into interactive mode
+#
+# and allows newly typed commands to be executed within the script
+#
+# usage : cmd
+#
+##
+function cmd() {
+  # render the prompt
+  x=$(PS1="$DEMO_PROMPT" "$BASH" --norc -i </dev/null 2>&1 | sed -n '${s/^\(.*\)exit$/\1/p;}')
+  printf "$x\033[0m"
+  read command
+  run_cmd "${command}"
+}
+
+function run_cmd() {
+  function handle_cancel() {
+    printf ""
+  }
+
+  trap handle_cancel SIGINT
+  stty -echoctl
+  eval $@
+  stty echoctl
+  trap - SIGINT
+}
+
+
+function check_pv() {
+  command -v pv >/dev/null 2>&1 || {
+
+    echo ""
+    echo -e "${RED}##############################################################"
+    echo "# HOLD IT!! I require pv for simulated typing but it's " >&2
+    echo "# not installed. Aborting." >&2;
+    echo -e "${RED}##############################################################"
+    echo ""
+    echo -e "${COLOR_RESET}Disable simulated typing: "
+    echo ""
+    echo -e "   unset TYPE_SPEED"
+    echo ""
+    echo "Installing pv:"
+    echo ""
+    echo  "   Mac: $ brew install pv"
+    echo ""
+    echo  "   Other: https://www.ivarch.com/programs/pv.shtml"
+    echo  ""
+    exit 1;
+  }
+}
+
+#
+# handle some default params
+# -h for help
+# -d for disabling simulated typing
+#
+while getopts ":dhncw:" opt; do
+  case $opt in
+    h)
+      usage
+      exit 1
+      ;;
+    d)
+      unset TYPE_SPEED
+      ;;
+    n)
+      NO_WAIT=true
+      ;;
+    c)
+      SHOW_CMD_NUMS=true
+      ;;
+    w)
+      PROMPT_TIMEOUT=$OPTARG
+      ;;
+  esac
+done
+
+##
+# Do not check for pv. This trusts the user to not set TYPE_SPEED later in the
+# demo in which case an error will occur if pv is not installed.
+##
+if [[ -n "$TYPE_SPEED" ]]; then
+  check_pv
+fi