diff --git a/README.md b/README.md index 8b1b840..baca80b 100644 --- a/README.md +++ b/README.md @@ -14,7 +14,7 @@ This repo drives the "build all the things" side of Armbian releases. It contain - **Reusable pipeline workflows** that clone [`armbian/build`](https://github.com/armbian/build), run `compile.sh` across a large per-board matrix, and publish the resulting artifacts to OCI (`ghcr.io/armbian/os/*`) and images to GitHub Releases. - **Thin "track" wrappers** (nightly, stable, community, apps, standard-support) that call the reusable pipeline on schedules or on manual dispatch. - **Userpatches** (`userpatches/`) — configs, image customization, and optional extensions that the build framework picks up during a run. -- **Release header generators** (`release-headers/`) — small scripts that emit the HTML body used when the release is created. +- **Release header generators** (`release-headers/`) — small scripts that emit the HTML body used when a release is created. - **Operational workflows** — a watchdog that auto-retries jobs killed by self-hosted runner stalls, and a housekeeping job that prunes old releases. ## Repository layout @@ -55,6 +55,15 @@ This repo drives the "build all the things" side of Armbian releases. It contain └── README.md ``` +## Built with + +- **GitHub Actions** (YAML workflows under `.github/workflows/`), including reusable workflows called via `workflow_call`. +- **Bash** — release-header generators (`release-headers/*.sh`), image customization (`userpatches/customize-image.sh`), extensions (`userpatches/extensions/*.sh`), and all `run:` step bodies inside the workflows. +- **`gh` CLI + `jq`** for GitHub API calls, release management and runner-info parsing. +- **`rsync`, `curl`, `mktorrent`, Docker** — used by the per-chunk build jobs. +- **[`armbian/actions`](https://github.com/armbian/actions)** — `runner-clean` and `team-check` composite actions. +- **[actionlint](https://github.com/rhysd/actionlint)** config in `.github/actionlint.yaml`; **Dependabot** config in `.github/dependabot.yml`. + ## How the pipeline is organised The pipeline is written **once** in `complete-artifact-matrix.yml`. Each per-track workflow is a thin wrapper that calls it via `uses:` with track-specific inputs (target repository, target path, team, targets file, defaults for the manual-dispatch menu). @@ -89,6 +98,7 @@ Common manual-dispatch inputs on the tracks that expose them: - `checkOci` — reuse existing artifacts already in OCI, or rebuild everything. - `extraParamsAllBuilds` — extra `KEY=value` passed to every `compile.sh` invocation (e.g. `DEBUG=yes`). - `branch` — framework build branch to check out from `armbian/build` (default `main`). +- `forceDockerPull` — fresh Docker image pull vs. cached. - `targetsFilterInclude` — matrix filter, e.g. `BOARD:odroidhc4,BOARD:odroidn2`. - `nightlybuild` — nightly vs. stable semantics. - `versionOverride` — force a specific version string. @@ -99,7 +109,7 @@ Common manual-dispatch inputs on the tracks that expose them: Versioning is driven entirely by GitHub releases on the target repository — there is no version file in this repo: -- **Stable** builds require `versionOverride` (e.g. `26.8.0`). +- **Stable** tracks reuse the latest existing `X.Y.Z` release as-is; pass `versionOverride` to cut a new version on purpose. If no `X.Y.Z` release exists yet, `versionOverride` is required to seed the series. - **Nightly** builds pick the newest `-trunk.N` release in the target repo and bump `N`; `versionOverride` can seed a new base series. The release is created empty up front by `version_prep`, and the image jobs attach assets to that tag. @@ -110,7 +120,7 @@ The `userpatches/` directory is checked out inside every build and copied into t - `config-armbian-images.conf`, `config-armbian-community.conf`, `config-armbian-apps.conf`, `config-armbian-cloud.conf` — per-config build settings (selected via each track's `prepare_config`). - `customize-image.sh` — image customization hook. -- `targets-all-not-eos.yaml` — targets list used by the "all" tracks (release-specific target lists are fetched at build time from [`armbian/armbian.github.io`](https://github.com/armbian/armbian.github.io) `data` branch). +- `targets-all-not-eos.yaml` — targets list used by the "all" tracks. For release-specific target lists, `build-images-chunk.yml` additionally checks out the `data` branch of [`armbian/armbian.github.io`](https://github.com/armbian/armbian.github.io) and rsyncs `release-targets/` into `userpatches/`. - `extensions/` — optional build extensions: `docker-ce.sh`, `ha.sh`, `kali.sh`, `omv.sh`, `openhab.sh`. ## Release headers @@ -135,34 +145,37 @@ Note: the build chunks are `fail-fast: false`, so good jobs finish and upload ar Runs daily at 03:00 UTC (and on manual dispatch). Only runs when the repo owner is `armbian`. For both full releases and pre-releases, it keeps the newest 3 (sorted by `created_at`) and deletes the rest via the GitHub API. -## Build execution details +## Runner selection and proxying + +The per-chunk workflows pick their runner from the matrix (`matrix.runs_on`), which is generated by `compile.sh gha-matrix`. On self-hosted runners the build steps fetch a per-runner map from `https://github.armbian.com/servers/github-runners.jq` and export: -Per-chunk jobs (`build-artifacts-chunk.yml`, `build-images-chunk.yml`): +- `APT_PROXY_ADDR` — apt-cacher-ng address (probed before use; cleared if unreachable). +- `GHCR_MIRROR_ADDRESS` — OCI cache mirror. +- `CCACHE_REMOTE_STORAGE` — remote ccache (enables the `ccache-remote` build extension). +- `GITPROXY_ADDRESS` — git proxy (artifact chunks only). -- Run with `fail-fast: false` so a single unstable board does not sink its neighbours. -- Timeouts: 65 min for artifact chunks (60 min build + OCI push), 75 min for image chunks (45 min build + sign/torrent/upload). -- Retry the actual `compile.sh` invocation up to `BUILD_ATTEMPTS` (default `3`) times to ride out transient network/runner flakes, replacing an older external watchdog. -- Look up per-runner metadata from `https://github.armbian.com/servers/github-runners.jq` and export `APT_PROXY_ADDR`, `GHCR_MIRROR_ADDRESS`, `CCACHE_REMOTE_STORAGE`, `GITPROXY_ADDRESS` when the runner advertises them. Any ambient `http_proxy`/`https_proxy` from the runner host is neutralised first; a configured apt proxy is probed before use and dropped if unreachable. -- Log in to `ghcr.io` and push artifacts to `ghcr.io/armbian/os/*`. This requires a PAT with `write:packages` on `armbian/os`; the workflow uses `ACCESS_TOKEN` and falls back to `GITHUB_TOKEN` when unset. -- Image chunks also pull release-target YAML files from the `data` branch of `armbian/armbian.github.io`. +Ambient `http_proxy` / `https_proxy` / `ftp_proxy` env is explicitly cleared so the framework doesn't derive a stale `APT_PROXY_ADDR` from the host. + +Each build step retries up to `BUILD_ATTEMPTS` (default 3) to ride out transient runner/network failures. ## Secrets -Used by the reusable pipeline / chunk workflows (declared as `secrets: inherit` on the callers): +The reusable image-chunk workflow (`build-images-chunk.yml`) declares: + +- `KEY_UPLOAD` and `KNOWN_HOSTS_ARMBIAN_UPLOAD` — **required** (SSH upload to Armbian storage). +- `ACCESS_TOKEN` — optional PAT with write access to the target release repo (`os` / `community` / `distribution`) and `write:packages` on `armbian/os` for OCI push; falls back to `GITHUB_TOKEN` where possible. +- `GPG_KEY1`, `GPG_PASSPHRASE1` — optional image signing. -- `ACCESS_TOKEN` — cross-repo PAT (write to `armbian/os` etc., `write:packages` on GHCR). Optional; falls back to `GITHUB_TOKEN` for read/limited paths. -- `ORG_MEMBERS` — used by `armbian/actions/team-check` to gate who can trigger a run. -- `KEY_UPLOAD`, `KNOWN_HOSTS_ARMBIAN_UPLOAD` — required by image chunks to publish images. -- `GPG_KEY1`, `GPG_PASSPHRASE1` — optional, for signing. +The orchestrator also uses `ORG_MEMBERS` for the `armbian/actions/team-check@main` gate at the top of every run. ## Related repositories -- [`armbian/build`](https://github.com/armbian/build) — the build framework (`compile.sh`) this pipeline drives. -- [`armbian/os`](https://github.com/armbian/os), [`armbian/community`](https://github.com/armbian/community), [`armbian/distribution`](https://github.com/armbian/distribution) — release repositories where images and artifact metadata land. -- [`armbian/actions`](https://github.com/armbian/actions) — shared composite actions (`runner-clean`, `team-check`, …) used by the workflows here. -- [`armbian/armbian.github.io`](https://github.com/armbian/armbian.github.io) — hosts the release-target YAML files (`data` branch). +- [`armbian/build`](https://github.com/armbian/build) — the build framework this pipeline drives (`compile.sh`). +- [`armbian/actions`](https://github.com/armbian/actions) — shared composite actions (`runner-clean`, `team-check`). +- [`armbian/armbian.github.io`](https://github.com/armbian/armbian.github.io) (`data` branch) — release target YAML files consumed by image builds. +- Release repositories: `armbian/os`, `armbian/community`, `armbian/distribution`. -## Further reading +## Documentation - Armbian documentation: -- Project home: +- Project site: