Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: GitHub runner as systemd service #3

Open
wants to merge 156 commits into
base: feature/CI
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
156 commits
Select commit Hold shift + click to select a range
b54a65b
PR inicial (#1)
rodrigondec Dec 12, 2023
5ce485e
WIP: systemd
PedroRegisPOAR Jan 3, 2024
38de610
WIP: systemd
PedroRegisPOAR Jan 3, 2024
81fc885
WIP: podman testes
PedroRegisPOAR Jan 3, 2024
e9cb171
WIP: podman testes
PedroRegisPOAR Jan 3, 2024
6a5c036
WIP: podman testes
PedroRegisPOAR Jan 3, 2024
bf7eb1f
WIP: podman testes
PedroRegisPOAR Jan 3, 2024
807c92d
WIP: desabilita podman, testa sudo id
PedroRegisPOAR Jan 4, 2024
4baa200
WIP: teste DinD
PedroRegisPOAR Jan 4, 2024
1f872cf
WIP: teste DinD
PedroRegisPOAR Jan 4, 2024
44956fe
WIP: teste DinD
PedroRegisPOAR Jan 4, 2024
2ab83f9
WIP: teste DinD
PedroRegisPOAR Jan 4, 2024
4e81761
WIP: refactoring
PedroRegisPOAR Jan 4, 2024
e122da6
Merge branch 'feature/CI' into feature/github-runner-as-systemd-service
PedroRegisPOAR Jan 4, 2024
a44e1d3
WIP: PinD
PedroRegisPOAR Jan 4, 2024
78106c1
WIP: PAT removido e invalidado no github
PedroRegisPOAR Jan 4, 2024
5165147
WIP: PinD
PedroRegisPOAR Jan 4, 2024
2760369
WIP: stress-ng
PedroRegisPOAR Jan 4, 2024
9b9fa81
WIP: stress-ng
PedroRegisPOAR Jan 4, 2024
92fe22b
WIP: testa nix build
PedroRegisPOAR Jan 4, 2024
6f969a1
Organiza steps
PedroRegisPOAR Jan 4, 2024
3e1e688
Documenta como testar
PedroRegisPOAR Jan 4, 2024
27986ee
Testa spice + virt-viewer
PedroRegisPOAR Jan 8, 2024
d6870af
Testa spice + virt-viewer refactor usando bash -lc read -s
PedroRegisPOAR Jan 8, 2024
b7c408a
typos
PedroRegisPOAR Jan 8, 2024
baf7d99
Ajustes em textos dos passos
PedroRegisPOAR Jan 9, 2024
27f4a12
Ajustes em textos dos passos
PedroRegisPOAR Jan 9, 2024
3cb8e02
Resolve warnings
PedroRegisPOAR Jan 9, 2024
05e92fb
Atualiza nixpkgs em WIP, atualiza actions/checkout@v4
PedroRegisPOAR Jan 9, 2024
e46a97b
Super nix run
PedroRegisPOAR Jan 9, 2024
03c9f6b
Documenta novos passos
PedroRegisPOAR Jan 9, 2024
7a2d175
Testa boot.kernelPackages = pkgs.linuxKernel.packages.linux_rt_5_15;
PedroRegisPOAR Jan 9, 2024
d5c3612
Comumenta como atualizar os inputs e system.stateVersion = "23.11";
PedroRegisPOAR Jan 15, 2024
3789303
Mais ajustes em scripts para inhjetar o PAT
PedroRegisPOAR Jan 16, 2024
f527a94
Reduz specs do hardware da VM
PedroRegisPOAR Jan 17, 2024
3b838fe
Adiciona TODO
PedroRegisPOAR Jan 17, 2024
7f396e8
WIP: race condition
PedroRegisPOAR Jan 17, 2024
256f354
Resolve race condition, hopefully
PedroRegisPOAR Jan 17, 2024
a647bd0
Resolve race condition, hopefully
PedroRegisPOAR Jan 17, 2024
6e5f00a
Remove app.vm e fmt
PedroRegisPOAR Jan 18, 2024
14b07bf
refatora algumas configurações
rodrigondec Jan 18, 2024
824de35
altera CI
rodrigondec Jan 18, 2024
2a4e054
adiciona label "nixos"
rodrigondec Jan 18, 2024
cd8d00e
fmt
rodrigondec Jan 18, 2024
f6fc04e
altera env.example
rodrigondec Jan 18, 2024
8c8af0d
adiciona o hostname no .env
rodrigondec Jan 18, 2024
f6341b0
Adiciona github-runner.extraEnvironment = { RUNNER_ROOT = "/tmp/.gith…
PedroRegisPOAR Jan 18, 2024
48c12f2
adiciona pwd para o ci
rodrigondec Jan 18, 2024
eef870f
adiciona profile install
rodrigondec Jan 19, 2024
22e9b48
corrige CI
rodrigondec Jan 19, 2024
a0cb1cd
install hello
rodrigondec Jan 19, 2024
fe17d05
adiciona mais comandos de teste
rodrigondec Jan 19, 2024
f3adfe0
debug info
rodrigondec Jan 19, 2024
5a41e8b
debug info
rodrigondec Jan 19, 2024
704215c
debug info
rodrigondec Jan 19, 2024
996467b
debug info
rodrigondec Jan 19, 2024
06203b7
alterações feitas
rodrigondec Jan 19, 2024
959b25d
altera CI
rodrigondec Jan 19, 2024
230a39d
ajusta ci
rodrigondec Jan 19, 2024
8f29871
remove configurações desnecessárias
rodrigondec Jan 19, 2024
2e07fba
testes
rodrigondec Jan 19, 2024
726bbe7
teste node
rodrigondec Jan 19, 2024
ab1c5af
commita mais info
rodrigondec Jan 19, 2024
3406e62
ajusta testes
rodrigondec Jan 19, 2024
f20785d
echo path
rodrigondec Jan 19, 2024
d215cfd
Refactor, ainda quebrado para mim
PedroRegisPOAR Jan 19, 2024
0256716
WIP: remove runs-on: group: nixgroup
PedroRegisPOAR Jan 19, 2024
6e2b1d1
Mais debug
PedroRegisPOAR Jan 19, 2024
fd54033
ReadWritePaths /nix ou /tmp?
PedroRegisPOAR Jan 19, 2024
e83ed3b
Debugando o nodejs
PedroRegisPOAR Jan 19, 2024
2204216
Debugando o nodejs
PedroRegisPOAR Jan 19, 2024
d91221b
Debugando o ping
PedroRegisPOAR Jan 19, 2024
2765eac
Debugando o ping
PedroRegisPOAR Jan 19, 2024
1df55d9
Debugando o ping
PedroRegisPOAR Jan 19, 2024
f5dd1cf
Debugando o ping
PedroRegisPOAR Jan 19, 2024
7afd353
Debugando o ping, ainda quebrado
PedroRegisPOAR Jan 19, 2024
b8a9871
Mais debug
PedroRegisPOAR Jan 22, 2024
8d7dce3
Mais debug
PedroRegisPOAR Jan 22, 2024
d453dc0
Mais debug, /run/wrappers/bin/sudo e outros
PedroRegisPOAR Jan 22, 2024
b82fc93
Mais debug, sudo e /run/wrappers/bin/sudo
PedroRegisPOAR Jan 22, 2024
3159968
Mais debug, o runner estava quebrado
PedroRegisPOAR Jan 22, 2024
48bfc50
Mais debug,
PedroRegisPOAR Jan 22, 2024
c60e4cb
Mais debug, muda ordem do ping
PedroRegisPOAR Jan 22, 2024
2e4b6df
Mais debug, ping
PedroRegisPOAR Jan 22, 2024
55e50d3
Corrige ping
PedroRegisPOAR Jan 22, 2024
2887744
Debugando nix profile install nixpkgs#blah
PedroRegisPOAR Jan 22, 2024
cf13f16
Debugando nix profile install nixpkgs#nodejs
PedroRegisPOAR Jan 22, 2024
2434423
Debugando nix profile install nixpkgs#hello
PedroRegisPOAR Jan 22, 2024
931b539
Debugando nix profile install nixpkgs#hello
PedroRegisPOAR Jan 22, 2024
405bacb
Debugando nix profile install nixpkgs#hello
PedroRegisPOAR Jan 22, 2024
1ebc384
Debugando nix profile install nixpkgs#hello
PedroRegisPOAR Jan 22, 2024
40f8c96
Debugando nix profile install nixpkgs#hello
PedroRegisPOAR Jan 22, 2024
63ba4a5
Debugando nix profile install nixpkgs#hello
PedroRegisPOAR Jan 22, 2024
b1fbf74
Debugando nix profile install nixpkgs#hello
PedroRegisPOAR Jan 22, 2024
0e898bc
Debugando nix profile install nixpkgs#nodejs
PedroRegisPOAR Jan 22, 2024
3de4e48
Debugando sudo
PedroRegisPOAR Jan 22, 2024
89f7768
Debugando podman
PedroRegisPOAR Jan 22, 2024
da54790
Debugando podman
PedroRegisPOAR Jan 22, 2024
ca31cfd
Debugando podman
PedroRegisPOAR Jan 22, 2024
2a9f620
Debugando podman e hostname
PedroRegisPOAR Jan 22, 2024
be9f830
Debugando podman, quase lá, I hope
PedroRegisPOAR Jan 22, 2024
2cfa343
Debugando podman
PedroRegisPOAR Jan 22, 2024
c75c58f
Debugando podman e sudo
PedroRegisPOAR Jan 22, 2024
197e9f6
Debugando podman
PedroRegisPOAR Jan 22, 2024
1b1c98f
Debugando podman
PedroRegisPOAR Jan 23, 2024
170b47f
Debugando podman e docker
PedroRegisPOAR Jan 23, 2024
292da8b
Debugando podman
PedroRegisPOAR Jan 23, 2024
092c8c6
Debugando podman
PedroRegisPOAR Jan 23, 2024
51560b3
Debugando podman
PedroRegisPOAR Jan 23, 2024
2da16b3
Debugando podman
PedroRegisPOAR Jan 23, 2024
ed602df
Debugando podman
PedroRegisPOAR Jan 23, 2024
b673130
Debugando podman
PedroRegisPOAR Jan 23, 2024
1c3b629
Debugando podman e salva links e TODOs
PedroRegisPOAR Jan 23, 2024
4e39a3e
Debugando podman
PedroRegisPOAR Jan 23, 2024
3f8c402
Debugando podman
PedroRegisPOAR Jan 23, 2024
143bf3e
Debugando podman, a saga continua
PedroRegisPOAR Jan 23, 2024
ad22945
Debugando podman, 2
PedroRegisPOAR Jan 23, 2024
3b03e31
Debugando podman, 3
PedroRegisPOAR Jan 23, 2024
ddf18f3
Debugando podman, 4
PedroRegisPOAR Jan 23, 2024
02b55ac
Debugando podman, 5
PedroRegisPOAR Jan 23, 2024
5e0d20e
Debugando podman, 6
PedroRegisPOAR Jan 23, 2024
7ff1d0c
Debugando podman, 6
PedroRegisPOAR Jan 23, 2024
83b123a
Debugando podman, 7
PedroRegisPOAR Jan 23, 2024
1a8d32e
Debugando podman, 8
PedroRegisPOAR Jan 23, 2024
bed78f3
Debugando podman, 9
PedroRegisPOAR Jan 23, 2024
bea8c8c
Debugando podman, 10
PedroRegisPOAR Jan 23, 2024
76e0bd8
Debugando podman, 11
PedroRegisPOAR Jan 23, 2024
697796a
Muitos TODOs, podman e sudo ainda não funcionam
PedroRegisPOAR Jan 23, 2024
dfbdd41
Faz github-runner iniciar ao ligar VM
PedroRegisPOAR Jan 23, 2024
37636d3
Faz testes passarem no CI
PedroRegisPOAR Jan 23, 2024
a8dd1cf
Debugando podman, mais um dia dessa saga
PedroRegisPOAR Jan 24, 2024
c959cf1
Debugando podman, sudo podman
PedroRegisPOAR Jan 24, 2024
2a37ec8
Debugando podman, podman
PedroRegisPOAR Jan 24, 2024
04e72f2
Debugando podman, podman
PedroRegisPOAR Jan 24, 2024
0e7ef74
Debugando podman, podman
PedroRegisPOAR Jan 24, 2024
dcac301
Debugando podman, podman
PedroRegisPOAR Jan 24, 2024
e34ebd1
Debugando podman, /proc saga
PedroRegisPOAR Jan 25, 2024
5948a2e
Debugando podman, /proc saga
PedroRegisPOAR Jan 25, 2024
ad39f86
Debugando podman, /proc saga
PedroRegisPOAR Jan 25, 2024
bb71c8d
Debugando podman, /proc saga, copy/pasta
PedroRegisPOAR Jan 25, 2024
cf9e61b
Debugando podman, /proc saga, copy/pasta
PedroRegisPOAR Jan 25, 2024
89b8a25
Debugando podman, /proc saga
PedroRegisPOAR Jan 25, 2024
16f2444
Debugando podman, podman
PedroRegisPOAR Jan 25, 2024
cb4bb72
Debugando podman in docker
PedroRegisPOAR Jan 25, 2024
24f7b0a
Debugando podman in docker
PedroRegisPOAR Jan 25, 2024
c7d7879
Debugando podman
PedroRegisPOAR Jan 25, 2024
ef8143a
Debugando podman
PedroRegisPOAR Jan 25, 2024
da76ea5
Debugando podman
PedroRegisPOAR Jan 25, 2024
735bb86
Debugando podman
PedroRegisPOAR Jan 25, 2024
f2cc8cb
Debugando podman
PedroRegisPOAR Jan 25, 2024
25dfc21
Debugando podman
PedroRegisPOAR Jan 25, 2024
bd258bd
Debugando podman
PedroRegisPOAR Jan 25, 2024
114af57
Debugando podman
PedroRegisPOAR Jan 25, 2024
e1217a2
Testa sudo
PedroRegisPOAR Jan 26, 2024
ac027e3
Testa podman e sudo
PedroRegisPOAR Jan 26, 2024
7467a4f
Corrige podman? systemd.services.github-runner.path
PedroRegisPOAR Feb 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
PAT_TOKEN=ghp_LADGFtReRTASJDIORTJueawiuoh1231afTFSED
export GH_TOKEN=ghp_LADGFtReRTASJDIORTJueawiuoh1231afTFSED
export HOSTNAME=$(hostname)
118 changes: 118 additions & 0 deletions .github/workflows/heavy_tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
name: heavy tests


on: workflow_dispatch


jobs:
test:
runs-on:
group: nixgroup
labels: nixos
name: Testes
steps:
- name: checkout PR merge commit
uses: actions/checkout@v4
with:
# Nix Flakes doesn't work on shallow clones
fetch-depth: 0

- name: Testes de esforço
run: |
docker \
run \
-t \
--rm \
polinux/stress-ng \
--cpu 4 \
--io 4 \
--vm 2 \
--vm-bytes 128M \
--fork 4 \
--timeout 10s

- name: Executa GNU hello e python3
run: |
hello
python --version

- name: Metadatas
run: |
cat /etc/os*release
echo
# cat /etc/group
uname -a
echo
id
echo
pwd
echo
ls -alh
echo
nproc
echo
free -h
echo
# sudo id
# A ideia aqui é testar persistência de estado
cat $HOME/logs.txt || true
echo FOO-BAR > $HOME/logs.txt

- name: Testes sobre docker
run: |
# env | sort
# touch /dev/kvm
which docker
readlink -f $(which docker)
docker --version
docker info
docker images
docker run --rm alpine cat /etc/os-release
docker images
docker build --tag custom-python .
docker images

- name: Testes sobre /dev/kvm
run: |
id
docker run --privileged=true --rm \
alpine sh -c 'id && touch /dev/kvm && stat /dev/kvm'

- name: Testes sobre DinD
run: |
docker rm --force dind-container || true
docker \
run \
--device=/dev/kvm \
-d \
-t \
--name=dind-container --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
docker:24.0.7-dind-alpine3.18 \
tail -f /dev/null
echo
docker \
exec \
-t \
dind-container \
/bin/sh \
-c \
'docker run -t --rm alpine cat /etc/os-release'

- name: Testes sobre PinD
run: |
docker \
run \
--privileged=true \
--rm \
quay.io/podman/stable \
podman run quay.io/podman/hello

- name: Mais metadados
run: |
nix flake --version
nix flake metadata nixpkgs
echo
nix build --no-link --print-out-paths nixpkgs#dockerTools.examples.redis
echo
nix build --no-link --print-out-paths --rebuild nixpkgs#dockerTools.examples.redis
124 changes: 100 additions & 24 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -1,37 +1,113 @@
name: tests

on:
push:
branches:
- main
pull_request:
branchs:
- 'feature/**'
- 'fix/**'
- 'refactor/**'

on: workflow_dispatch



jobs:
test:
runs-on: self-hosted
name: Testes
runs-on:
# group: nixgroup
labels: nixos
name: NixOS Testes
steps:
- name: checkout PR merge commit
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
# Nix Flakes doesn't work on shallow clones
fetch-depth: 0

- name: Recolhe info
run: |
id
pwd
ls -la /nix/store | grep hello
echo "${PATH//:/$'\n'}"
env | sort

- name: Recolhe info show-config
run: |
nix show-config

- name: Testa ping
run: |
python --version
stat -c '%a %n' $(which ping)
stat -c '%a %n' $(readlink -f $(which ping))
# /run/current-system/sw/bin/ping -c3 8.8.8.8 # Funciona tb, mas comentado para poupar um pouco de tempo.
ping -c3 8.8.8.8

- name: Rodar um echo
run: echo 'teste'
- name: profile install hello test
run: |
nix profile install nixpkgs#hello
echo "${PATH//:/$'\n'}"
nix profile list
# ls -alh "$HOME"/.nix-profile
# ls -alh "$HOME"/.nix-profile/
ls -alh /nix/var/nix/profiles/per-user/nixuser/profile
ls -alh /nix/var/nix/profiles/per-user/nixuser/profile/bin
hello

- name: profile install node test
run: |
nix profile install nixpkgs#nodejs
echo "${PATH//:/$'\n'}"
nix profile list
node --version

- name: Testa hostname
run: |
hostname
hostname a1b2c3
hostname
hostname nixos
hostname

- name: Testa /proc
run: |
mount | grep /proc
echo
findmnt -R /proc
echo
unshare -Ur -m bash -c 'id && pwd && ls -alh'
unshare -Ur -m bash -c 'cat /proc/$$/mounts'
cat /proc/$$/mountinfo
# unshare -pfr --mount-proc=/proc echo hi

- name: Executa docker do host
- name: Testa podman
run: |
# sudo apt-get install -y podman
docker pull alpine
docker images
docker info
cat /proc/sys/user/max_user_namespaces
cat /proc/self/uid_map
grep /run /proc/self/mountinfo
cat /etc/subuid
cat /etc/subgid
ls -l /run/wrappers/bin/newuidmap
ls -l /run/wrappers/bin/newuidmap
stat -c '%a %n' /run/wrappers/bin/newuidmap
stat -c '%a %n' /run/wrappers/bin/newgidmap
getcap /run/wrappers/bin/newuidmap
getcap /run/wrappers/bin/newgidmap
# podman images
# docker --version
# docker run --rm alpine cat /etc/os-release
# podman --log-level debug pull alpine
# podman images
podman run --privileged --rm -v /proc:/proc:rw -v /dev:/dev:rw alpine sh -c 'apk add python3 && python --version'
podman network ls
podman info
# podman unshare cat /proc/self/uid_map /proc/self/gid_map
# podman network ls
# podman pull alpine
# podman images

- name: Run alpine OCI image with docker and metadatas
- name: Testa sudo
run: |
docker run alpine cat /etc/os*release
cat /etc/os*release
uname -a
nix flake metadata nixpkgs
stat -c '%a %n' $(which sudo)
stat -c '%a %n' $(readlink -f $(which sudo))
stat -c '%a %n' /run/wrappers/bin/sudo
stat -c '%a %n' /run/current-system/sw/bin/sudo
echo 12345
# sudo id
/run/wrappers/bin/sudo id
# /run/current-system/sw/bin/sudo id
32 changes: 32 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
FROM python:3.9-slim-buster


# Set python environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV PIP_NO_CACHE_DIR 0
ENV PIP_DISABLE_PIP_VERSION_CHECK 1

ENV USER app_user

WORKDIR /home/app_user

RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends --no-install-suggests -y \
ca-certificates \
&& apt-get -y autoremove \
&& apt-get -y clean \
&& rm -rf /var/lib/apt/lists/*

RUN addgroup app_group \
&& adduser \
--quiet \
--disabled-password \
--shell /bin/bash \
--home /home/app_user \
--gecos "User" app_user \
--ingroup app_group \
&& chmod 0700 /home/app_user \
&& chown --recursive app_user:app_group /home/app_user

CMD ["/bin/bash"]
60 changes: 59 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -42,4 +42,62 @@ source .env
```bash
source .env
RUNNER_SCOPE="org" SCOPE_TARGET="imobanco" bash ./ops/bash/entrypoint.sh
```
```


# github self-hosted runner em uma máquina virtual NixOS usando systemd


Gerar o PAT:
- Onde gerar? https://github.com/settings/tokens/new
- Com os seguintes checks: https://github.com/myoung34/docker-github-actions-runner/wiki/Usage#token-scope


Passo 0: Clonar o repositório:
```bash
nix flake clone 'git+ssh://[email protected]/imobanco/github-ci-runner.git' --dest github-ci-runner \
&& cd github-ci-runner 1>/dev/null 2>/dev/null \
&& git checkout feature/github-runner-as-systemd-service \
&& (direnv --version 1>/dev/null 2>/dev/null && direnv allow) \
|| nix develop --command $SHELL
```


Passo 1: Iniciar a VM e o VNC:
```bash
rm -fv nixos.qcow2;

nix run --impure --refresh --verbose .#run-github-runner
```


Passo 2: Injetando manualmente o PAT. No terminal da VM use
"seta para cima" (para acessar o histórico):
```bash
run-github-runner && sudo systemctl restart github-runner-nixos.service
```


Passo 3: Verifique que o runner aparece no link:
https://github.com/imobanco/github-ci-runner/actions/runners?tab=self-hosted


Passo 4: No terminal do clone local (apenas para testes manuais) do repositório:
```bash
export GH_TOKEN=ghp_yyyyyyyyyyyyyyy
```


Passo 5: Iniciando manualmente o workflow
Note: o remoto tenta iniciar a execução com o código que está no REMOTO, ou seja,
modificações apenas locais não são executadas.
```bash
gh workflow run tests.yml --ref feature/github-runner-as-systemd-service
```
Refs.:
- https://docs.github.com/en/[email protected]/actions/using-workflows/manually-running-a-workflow?tool=cli#running-a-workflow


Pelo navegador:
https://github.com/imobanco/github-ci-runner/actions

36 changes: 27 additions & 9 deletions flake.lock
720 changes: 542 additions & 178 deletions flake.nix

Large diffs are not rendered by default.