Skip to content

PMM + SEP feature-build harness - #1313

Draft
yyyyyyyan wants to merge 34 commits into
mainfrom
pmm
Draft

PMM + SEP feature-build harness#1313
yyyyyyyan wants to merge 34 commits into
mainfrom
pmm

Conversation

@yyyyyyyan

Copy link
Copy Markdown
Contributor

No description provided.

yyyyyyyan and others added 21 commits July 30, 2026 14:13
Ship only the inventory, mysql_backups, atw and snippets apps in the
side-car image: the other app packages are removed from the bundle at
image build so they cannot be activated, with two import-only stubs
(alerts/config.py, dipper/constants.py) kept for app.sep.main's
unconditional imports. The stripped alerts migration branch is dropped
from alembic.ini's version_locations inside the image.

Claude-Session: https://claude.ai/code/session_01Ws8ZLUEwNnLn1CNQDk1VbJ
Pair the PMM feature build (perconalab/pmm-server-fb:PR-4500-133001e —
SEP frontend + PMM_ENABLE_SEP PostgreSQL exposure) with the
app-restricted SEP side-car from this branch. All three SEP services and
the Celery beat store share the single `sep` database the PMM entrypoint
provisions; an nginx overlay adds the five SEP path prefixes the PMM UI
proxies, injecting the internal token server-side (interim auth Option
D); mint-grafana-token.sh mints a Grafana service-account token and
wires it into settings.yaml for SEP-side Grafana auth and the PMM
syncer. env/ is gitignored wholesale (virtualenv template), so the
harness directory is explicitly un-ignored.

Verified locally: side-car healthy (all programs running, three Alembic
tracks migrated into the shared db), /api/apps/ through PMM's nginx
returns the restricted app registry, and ambient session SSO signs a
pmm_session holder into SEP.

Claude-Session: https://claude.ai/code/session_01Ws8ZLUEwNnLn1CNQDk1VbJ
…is branch

The live Jenkins Build job predates the side-car stages in
build/jenkins-build.pipeline and only runs `make image`, so the
feature-build tag was publishing the standalone image. Point `image` at
the side-car Containerfile so the plain tag carries the feature-build
artifact, and reference it from the compose harness.

Claude-Session: https://claude.ai/code/session_01Ws8ZLUEwNnLn1CNQDk1VbJ
SEP task execution dispatches through PMM's Nomad; without
PMM_ENABLE_NOMAD=1 the pmm-server entrypoint never starts it.

Claude-Session: https://claude.ai/code/session_01Ws8ZLUEwNnLn1CNQDk1VbJ
…n for atw

The image strips the snippets app to the non-UI modules atw's
builtin-snippet execution imports (script_source + deps chain) plus
celery.py, whose sync task is the only path that ingests and
auto-approves the builtin library. Because the Celery include list is
activation-derived, the worker would never register that task with the
app deactivated, so snippets.celery joins STATIC_CELERY_INCLUDE — boot
ingestion keeps working while the periodic re-sync stays
activation-gated in seed.py. atw's requires_apps=("snippets",) is
removed: it depends on the execution seam and ingested rows, not the
management app, and the registry rejects a requires_apps key that is
not registered. The three nav-surface tests pinning that dependency are
skipped on this branch; the include-list expectations gain the new
static entry.

Claude-Session: https://claude.ai/code/session_01Ws8ZLUEwNnLn1CNQDk1VbJ
Replace the committed settings.yaml/pmm.conf (which carried test
credentials) with secret-free templates: bootstrap.sh generates a
gitignored .env (PG password, secret key, internal token) and renders
the two files from the templates, preserving an already-minted Grafana
token across re-renders and writing inode-preserving so running bind
mounts see the change. compose interpolates the PG password from .env.
The mint script's settings rewrite becomes inode-preserving too — a
docker restart does not re-resolve a bind-mounted file swapped by
sed -i.

Claude-Session: https://claude.ai/code/session_01Ws8ZLUEwNnLn1CNQDk1VbJ
A customImageTag parameter overrides the git-tag-derived image name
(shape-validated, never pushes :latest); a workspace-local `git tag -f`
keeps `git archive "${GIT_TAG}"` resolving without pushing any tag to
origin. Applied to the live job config in the same shape. The harness
pins the feature-build tag this branch publishes (pmm-<main-base-sha>).

Claude-Session: https://claude.ai/code/session_01Ws8ZLUEwNnLn1CNQDk1VbJ
env/ stays out of the repo; the harness lives beside the side-car build
inputs it pairs with, and the env/-un-ignore carve-out is reverted.

Claude-Session: https://claude.ai/code/session_01Ws8ZLUEwNnLn1CNQDk1VbJ
The entry existed only to satisfy atw's requires_apps=("snippets",)
declaration, which this branch drops by taking main's atw/app.py. With
the app package emptied by the image's strip block, leaving it activated
makes build_app_registry reach app.sep.apps.snippets.router and fail the
container at startup.

The baked-profile test asserts the activation list, so its expected set
narrows to the three apps this build actually ships.
…ld harness

entrypoint.sh now sources settings-env.sh, which exits before supervisord
unless SECRET_KEY is set in the container environment. The harness supplies
its secret inside the bind-mounted settings.yaml and passed no environment,
so any image built from this branch failed to start.

settings-env.sh also exports CELERY__BEAT_DBURI unconditionally, deriving a
passwordless DSN when SEP_DB_PASSWORD is absent. That overrides the mounted
profile's own DSN and leaves celery-beat restarting on "fe_sendauth: no
password supplied", so the password is forwarded too.

Both values come from the per-deployment secrets file bootstrap.sh generates.
The activation assertion is a subset check, so it stays green if the entry
is ever reintroduced. Nothing else would catch that: the app package is
emptied at image build, not in the source tree the suite runs against.
… image (#1257)

Drops the `alerts` and `dipper` import-only stubs from the PMM
feature-build side-car image, and inverts the app strip in
`sidecar/Containerfile.sidecar` from a deny-list to an allow-list.
Neither package is in the retained set, so dropping them is what removes
them — and several steps go with them:

- the `find alerts …` and `find dipper …` retention steps
- the `: > alerts/__init__.py` and `: > dipper/__init__.py` truncations
- the `sed -i` that stripped the `alerts` entry from `alembic.ini`'s
`version_locations`, together with the `chown 0:1001
$APP_HOME/alembic.ini` chained after it

The stubs existed only because `app/sep/main.py` and
`app/sep/api/routes/settings.py` reached into those packages at module
scope. Those unconditional cross-app imports were replaced with
activation-gated seams, so nothing forces the stubs any more and both
packages can be dropped whole.

**Why the strip is now an allow-list.** `rm -rf <everything we don't
ship>` fails open: a package added under `app/sep/apps/` later would
ship in the image unless someone remembered to extend the list, silently
breaking the invariant the comment states — that every app other than
`inventory`, `mysql_backups` and `atw` is stripped so it cannot be
activated. `SEP.APPS` in the baked profile does not cover that, since a
bind-mounted `settings.yaml` replaces the profile wholesale; the strip
is what holds the boundary. The list had already drifted —
`mysql_summary` was still named but no longer exists in the tree, and
`rm -rf` on a missing path is a silent no-op, so nothing surfaced the
gap. The retained set is behaviour-identical at HEAD (the same eleven
packages are stripped, plus any stray `__pycache__`); only the failure
direction changes. `framework` and `shared` are retained because
`inventory`, `mysql_backups`, `atw` and `app/sep/main.py` import them,
and `-type d` confines the strip to packages so the loose modules at the
apps root (`__init__.py`, `labels.py`, `nav_icons.py`) survive.

**Why the `sed` and the `chown` go together.** `alembic.ini` enters the
image exactly once, at `sidecar/Containerfile.sidecar:53` — `COPY
--chown=0:1001 --chmod=440 ./alembic.ini .` — already `root:sep` /
`440`. It does not arrive a second time via the bundle (`make pack`
archives `app snippets static templates`, none of which contains it),
and the deleted `sed -i` was the only command that rewrote it. The
`chown` existed to repair the ownership `sed -i`'s write-and-rename
produced; with the `sed` gone it is a no-op on a file already owned
`0:1001`, so both lines go.

**Why dropping the `sed` is safe.** The image's `alembic.ini` is now
byte-identical to the repo copy, including its still-present `alerts`
entry in `version_locations`. Alembic filters version locations by
`.exists()` before loading any script (`alembic/script/base.py:140-141`,
pinned 1.18.4), so an entry naming a stripped directory is skipped
silently — the `CommandError` existence check guards `script_location`,
a different setting. App migrations are independent branches and the
one-shots run `alembic --name=<service> upgrade heads` (plural), so
removing one branch orphans nothing and creates no head ambiguity.
Editing `alembic.ini` in the repo is not the alternative:
`scripts/sync_alembic_version_locations.py` regenerates
`version_locations` from a filesystem walk and pre-commit keeps it in
sync, and `alerts` still exists in the repo tree.

The comment block above the `RUN` is rewritten to describe the
allow-list and the surviving `snippets` thinning, and to describe the
latter accurately: the seven modules its retention list names now live
under `app/sep/snippets/`, outside the stripped tree, so the `find`
preserves only the package `__init__` — which the truncation then
empties.

`snippets` itself is untouched — its retention `find`, its truncation,
and every module the list names are byte-for-byte unchanged.
)

Drops the `snippets` app package whole from the PMM feature-build
side-car image, retiring the retain-and-thin stub.

`sidecar/Containerfile.sidecar` strips every app package under
`app/sep/apps/` that the image cannot activate. Since #1257 the strip is
an allow-list, so dropping a package means *removing* its `! -name`
token rather than adding it to a list. `snippets` was the one package
retained and thinned in place instead of dropped. Three edits, all
subtractive:

- `! -name snippets` removed from the allow-list — this is what drops
the package.
- The `find snippets ... -exec rm -rf {} +` retention step and the `: >
snippets/__init__.py` truncation deleted, along with the `&& \` that
chained them. The `RUN` collapses to a single `find`.
- The comment above the `RUN` loses the paragraph describing the stub.
What remains still describes the allow-list rationale, why `-type d` is
load-bearing, and why `framework` and `shared` are retained.

Nothing outside the stripped tree still resolves the package:
`STATIC_CELERY_INCLUDE`, `app/sep/db/seed.py`'s beat row,
`app/sep/main.py`'s boot ingestion and `atw`'s script resolution all
reach `app/sep/snippets/` — the library, which sits outside
`app/sep/apps/` and is never stripped. There is no snippets
`version_locations` entry in `alembic.ini`, so unlike #1257's `alerts`
this drop needs no Alembic handling.

**New drift guard.** Before this change the Containerfile retained
`snippets` while the baked profile did not activate it, so no test could
assert the two agree — `snippets` was the last such package. The
allow-list already fails closed for a package *added* under
`app/sep/apps/` and never listed; nothing caught the two lists
**drifting apart**. An app activated in the profile but not retained by
the strip fails only at container start, and a package retained but
never activated ships dead code indefinitely.

`tests/sidecar/test_app_strip.py` parses the strip block out of the
Containerfile and asserts its retained set equals
`settings.embedded.yaml`'s `SEP.APPS[].MODULE_NAME` plus `framework` and
`shared`. Equality binds in both directions. A second test pins the
single-`find` shape, covering the "no per-package retention step" half.
Both go red against the pre-change Containerfile (verified by stashing
the edit and re-running).
…1277)

## Summary

Removes the rendered-and-bind-mounted settings file from the PMM
feature-build harness. The side-car image now bakes its settings profile
at `$APP_HOME/settings.yaml` and expands a documented set of
per-deployment inputs, so the harness supplies only the values that vary
and lets environment outrank the baked file.

- **`compose.yaml`** — drops the `volumes:` key on `sep-sidecar` (it
held only the settings mount) and extends the existing `environment:`
block from two variables to six, adding `SEP_INTERNAL_TOKEN`,
`SEP_GRAFANA_TOKEN`, `SEP_NOMAD_ENDPOINT` and `BASE_URL`. The generated
secrets use the `${VAR:?run ./bootstrap.sh first}` guard already on
`pmm-server`'s password; `SEP_GRAFANA_TOKEN` deliberately uses `:-`
instead, because empty is the correct pre-mint state and
`settings-env.sh` gates on `-n`. The pinned image tag moves to one built
from the current branch, which carries the baked profile.
- **`bootstrap.sh`** — generates a `SEP_GRAFANA_TOKEN` slot in the
secrets file, and appends it to an existing secrets file that lacks the
key so re-bootstrapping an older harness stays idempotent. That append
emits a separator first when the file does not already end in a newline,
and the keep-existing branch re-asserts mode `600`, since the file is
now the sole carrier of every deployment secret. Stops rendering the
settings file and stops scraping a minted token back out of it;
`render()` keeps only the one substitution `pmm.conf.template` actually
uses (`__SEP_BEARER_TOKEN__`). The `.render.tmp` truncate-write stays,
since `pmm.conf` is still bind-mounted and the inode must survive.
- **`mint-grafana-token.sh`** — writes the token into the generated
secrets file instead of the settings file, and applies it with `docker
compose up -d --force-recreate --no-deps sep-sidecar` rather than
`restart`. This is the load-bearing change, not a cosmetic one:
`restart` reuses the existing container's environment, so a freshly
minted token would never reach the process. `--no-deps` keeps
`pmm-server` and its provisioning time untouched despite `depends_on`.
The `--no-restart` flag is renamed `--no-recreate`, with usage text and
progress messages following. A charset guard (`^glsa_[A-Za-z0-9_]+$`)
now rejects the token before it is written, because the secrets file is
sourced by `bootstrap.sh` and parsed by Compose — a token carrying `$`,
backticks, spaces or quotes would corrupt the line or expand at source
time, and the side-car would come up healthy on the wrong token. Both
write paths are guarded: the in-place rewrite refuses to truncate on a
failed or empty read, and the append emits a separator newline when the
file does not already end in one.
- **`settings.template.yaml`** — deleted. Its ignore entry stays: the
file is no longer generated, but a harness bootstrapped before this
change still has one on disk holding four secrets in cleartext, and
nothing else in the repository ignores that path.
- **`README.md`** — documents the environment bring-up, the
recreate-not-restart rotation steps for both containers, that PMM
annotations become active once a token is minted, which local filenames
the directory ignores, and a one-time migration note for harnesses
bootstrapped before this change.

Two keys change behavior rather than being no-ops:
`SEP.SESSION.COOKIE_NAME` falls back from the template's `casdoorToken`
to the code default `authToken`, and `PMM.ANNOTATIONS_ENABLED` moves
from the code default `False` — the deleted template never set it — to
`true` from the baked profile (inert until a Grafana token exists). The
cookie rename is self-consistent — one setting names both the setter and
every reader — and nothing outside SEP's own configuration hardcodes the
string.

## Blocking before merge

- **The pinned image tag `pmm-e11e9fd` is not published yet.** The image
was built from this branch's merge-base and verified locally (baked
profile present, only `inventory`/`atw`/`mysql_backups` activated,
`snippets` package absent), but the push to
`docker.io/percona/percona-sep` failed with `insufficient scopes` on the
available Docker Hub credential. Until it is pushed, `docker compose up
-d` cannot pull the pinned tag on any other machine.
- **The acceptance criterion "the harness's ignore file no longer lists
the rendered settings file" needs amending.** The entry was kept
deliberately — see the `settings.template.yaml` note above.

## Tested

- [x] From a clean state (`docker compose down -v`, remove the generated
secrets file and `pmm.conf`), run `./bootstrap.sh`: it creates the
secrets file with four keys including an empty `SEP_GRAFANA_TOKEN`,
renders `pmm.conf`, and produces no settings file.
- [x] `docker compose up -d` brings both containers up and `sep-sidecar`
reaches healthy within the 150 s grace period.
- [x] All three SEP APIs answer on `/health` (ports 9000, 9001, 9002).
This is the check that catches a profile/image mismatch — the `sep`
program crash-loops on a bad activation list while the others still
boot.
- [x] The SEP API is reachable through PMM's nginx overlay at
`https://127.0.0.1:8443/api/apps/`, proving the injected bearer still
matches the side-car's `SEP_INTERNAL_TOKEN`. The response lists exactly
`inventory`, `atw` and `mysql_backups`.
- [x] Settings genuinely arrive from the environment: the container's
baked profile is in place at `$APP_HOME/settings.yaml` (its header reads
`# Baked PMM-embedded settings profile`), and the six deployment inputs
are present in the container environment.
- [x] `./mint-grafana-token.sh` writes the token into the generated
secrets file and the side-car's container ID changes (a recreate, not a
restart). Afterwards the token authenticates against Grafana, and
`SEP_GRAFANA_TOKEN` has expanded into
`AUTH__PROVIDER__GRAFANA__SERVICE_ACCOUNT_TOKEN` and `PMM__API_KEY` in
the running process.
- [x] Triggering an inventory sync populates sync instances and
discovers nodes and services from PMM, confirming the PMM syncer is live
with the minted token.
- [x] Logging in to the SEP UI on port 9000 sets a session cookie named
`authToken` (not `casdoorToken`), the CSRF-protected login POST is
accepted, and the authenticated session reaches the Home Dashboard.
- [x] The SEP app pages render inside the PMM UI (`Inventory`, `Collect
Diagnostic Data`, `MySQL Backups` in the PMM sidebar).
- [x] Re-running `./bootstrap.sh` keeps the existing secrets file
including an already-minted token, adds no duplicate key, and still
produces no settings file.
- [x] Re-running `./bootstrap.sh` against a secrets file whose last line
has no trailing newline adds `SEP_GRAFANA_TOKEN=` on its own line,
leaving the preceding value intact.
- [x] Rotating `SEP_INTERNAL_TOKEN` in the secrets file, re-running
`./bootstrap.sh`, then `docker compose up -d --force-recreate` leaves
the nginx-proxied SEP API working with the new value.
- [x] Guards fire: with the secrets file absent, `docker compose config`
fails with `run ./bootstrap.sh first`.
- [x] `./mint-grafana-token.sh --dry-run` describes a recreate and names
the secrets file; the retired `--no-restart` flag is rejected.

## Checklist

- [x] ~~New/modified functions have type hints and rST docstrings~~
*(N/A — shell and compose configuration only)*
- [x] ~~New tests added for new features or bug fixes~~ *(N/A —
harness-only change; nothing in `app/` or `tests/` is touched)*
- [x] ~~Database migrations generated if models changed (`make
makemigrations`)~~ *(N/A for this change)*
- [x] User-facing changes documented (README, inline help, UI text)
- [x] Configuration changes documented with examples
…sh against a failed sed (#1310)

`render()` can truncate `pmm.conf` to zero bytes and still print `✓
Rendered pmm.conf` — a failing `sed` leaves the temp file empty, `cat`
overwrites the target with nothing, and the script exits 0. An empty
`pmm.conf` removes all SEP location blocks from the nginx overlay.

Adopts the same capture-then-guard pattern `mint-grafana-token.sh`
already uses for its structurally identical read-then-truncate-write:

- Capture `sed` output into a shell variable; check exit status
- Reject empty result with a distinct error message
- Truncate-write via `printf '%s\n' … > "${target}"` only on success,
preserving the existing inode (required for bind-mount visibility)
- Caller exits non-zero on failure (`render … || exit 1`)
- `.render.tmp` intermediate eliminated entirely

```bash
render() {
    local template="$1" target="$2"
    local rendered
    rendered="$(sed -e "s|__SEP_BEARER_TOKEN__|${SEP_INTERNAL_TOKEN}|g" \
        "${template}")" || {
        error "Render failed for ${target}; leaving it unchanged"
        return 1
    }
    [[ -n ${rendered} ]] || {
        error "Render produced empty output for ${target}; leaving it unchanged"
        return 1
    }
    printf '%s\n' "${rendered}" > "${target}"
    success "Rendered ${target}"
}
```

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yyyyyyyan <24644216+yyyyyyyan@users.noreply.github.com>
Co-authored-by: yyyyyyy <yan.orestes@percona.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@yyyyyyyan yyyyyyyan self-assigned this Aug 7, 2026
@yyyyyyyan yyyyyyyan added the do not merge There are reasons to not merge the code label Aug 7, 2026
…#1340)

Adds a third, opt-in compose service to the PMM feature-build harness so
MySQL Backups task execution can be exercised end to end. Until now the
harness ran `pmm-server` and `sep-sidecar` and no database, and the only
Nomad client lived inside `pmm-server` — which has no datadir, no backup
binaries and no PMM-registered MySQL service — so no backup could
execute anywhere.

`sep-mysql` is one container that is simultaneously the test MySQL
instance, a registered PMM Client (hence a Nomad client, hence a
selectable executor host) and the carrier of a seeded dataset.

**Why one container rather than a shared datadir volume.** SEP does not
schedule, it pins: the job is constrained to the node name the operator
picked and `raw_exec` runs it as a plain process in that node's
namespace. `app/sep/apps/mysql_backups/spec.py` pins the XtraBackup
server config to `host: localhost`, and the payload reads the datadir
directly. Splitting MySQL and the PMM Client into two containers
registers the service and populates the host picker — everything
upstream of exec looks correctly wired — and the run then fails on
*connect*, because the executor namespace has the files but no MySQL on
loopback. Mydumper is the counterpart: `spec.py` sends it to
`service.node.address`, so it exercises the network path and the remote
grant. A single combined node satisfies both, since it can reach itself
by its own address.

Seven files, all under `sidecar/pmm-fb/`:

- **`Containerfile.mysql`** (new) — `oraclelinux:9` with
`percona-server-server`, `percona-toolkit` and `python3-PyMySQL` (the
package set from `build/ansible/ansible-mysql-el-test-instance.yaml`),
plus `percona-xtrabackup-84`, `mydumper`, and `python3-pip`/`gzip` for
the dispatched `run-python` task. The PMM Client tree is copied from
`perconalab/pmm-client-fb` at the same feature-build tag as
`pmm-server`, not from the release repo: the client ships its own
`nomad` binary that has to speak RPC to the server's. The
`datacharmer/test_db` employees dataset is baked in at build time.
`HEALTHCHECK` covers MySQL readiness only — folding PMM registration in
would make a registration hiccup read as "the database is down".
- **`mysql-entrypoint.sh`** (new) — initialises the datadir on first
boot, creates `sep_backup@%` and `pmm@127.0.0.1` with the playbook's
grants, writes `/root/.my.cnf` at `0600` — `[client]` for the payloads'
own connections plus `[mysqld]` and `[xtrabackup]`, because the form
hands that file to the binary as `--defaults-file`, which suppresses
`/etc/my.cnf` and takes its groups rather than `[client]` — imports the
seed, then registers the node and its MySQL service and keeps
`pmm-agent` running beside `mysqld`. Both PMM steps are bounded retries,
and a registration failure is fatal rather than swallowed as "already
registered" — that fallback would also report success for a bad flag or
an unreachable PMM, and the symptom would surface much later as a
service missing from the backup-source picker.
- **`compose.yaml`** — the service behind a `mysql` profile, three named
volumes, `privileged: true` + `cgroup: host` + a read-write
`/sys/fs/cgroup` mount (a containerised Nomad client's fingerprinter
reads cgroups and `raw_exec` places tasks into cgroups the client
creates), and a fixed IP. `pmm-server`'s image tag and the client build
arg now read one `${PMM_FB_TAG}`, with the pinned default spelled out on
both lines, so exporting the variable moves both and a repin of the
default edits both; the default is the tag already pinned, so an unset
variable resolves to today's image. The node's address is defined once
as a YAML anchor and referenced by both `PMM_AGENT_SETUP_NODE_ADDRESS`
and `ipv4_address`: the first is what pmm-agent registers into PMM
inventory and therefore what SEP hands Mydumper, the second is what the
container answers on, and drift between them would surface only at
connect time.
- **`bootstrap.sh`** — generates three MySQL passwords into the
harness's gitignored, per-deployment secrets file. The single-slot
append branch is generalised into `ensure_slot`, keeping the existing
missing-trailing-newline guard and `chmod 600`; `SEP_GRAFANA_TOKEN`
still appends as an empty slot exactly as before.
- **`mysql-target.md`** (new) — the target node's own guide:
credentials, the co-location constraint, running a backup through the
form, and repinning.
- **`README.md`** — links that guide from `Bring-up` and documents the
profile-less fast path; the harness README stays the spin-up narrative.
- **`mint-grafana-token.sh`** — no relation to the MySQL node: it was
the last of the three harness scripts still printing a secret under
`DEBUG=1` (the minted token, the read-back secrets file, and
`PMM_ADMIN_PASSWORD`), so it gets the same `without_xtrace` treatment
the other two now have.

The default `docker compose up -d` is unchanged: without `--profile
mysql` the same two services come up.
@github-actions github-actions Bot added the large-diff Over 1500 changed lines, generated files discounted label Aug 16, 2026
@github-actions github-actions Bot removed the large-diff Over 1500 changed lines, generated files discounted label Aug 17, 2026
…ofile

The key sat at the top level of `default:`, where YamlPrefixConfigSettingsSource
never reads a SEPSettings field, so it was inert. SEPSettings already defaults it
to False, making the line redundant even where it would have been read. Removing
it leaves sidecar/ identical to main outside pmm-fb/.
pmm-server now ships the three capabilities this harness was standing in for, so
the stand-ins tested a topology no deployment uses and made every operator run a
manual mint step production does not have.

- Secrets arrive as files. PMM writes six files into the pmm-sep-secrets volume,
  mounted read-only at SECRETS_DIR; SECRET_KEY, the database password and the
  internal token leave the compose environment entirely, so no secret is visible
  in the process environment or in docker inspect. SEP_INTERNAL_TOKEN is dropped
  rather than relocated: it is derived from SECRET_KEY by HMAC when unset, and it
  existed here only because the deleted nginx overlay pinned a shared bearer.
- The side-car joins group 0. PMM writes the files 0640 owned by group root under
  a setgid directory, so group membership, not a matching uid, is what makes them
  readable.
- The side-car waits on pmm-server's health. PMM publishes the secrets in two
  stages and SEP builds its settings once at process start, so a side-car
  released early comes up with Grafana auth inert and never recovers. The gate is
  correct only against a pin carrying PMM-15331.
- PMM proxies SEP natively. The nginx overlay and its bind mount are gone;
  PMM_SEP_ADDRESS points PMM's own /sep location at the side-car, and because
  that location resolves a variable at request time, the fixed IPs for both
  containers go too. sep-mysql keeps its pin, which is load-bearing for the PMM
  inventory address handed to Mydumper.
- bootstrap.sh generates only what PMM cannot: the PostgreSQL password and the
  three sep-mysql passwords.
Covers which files PMM writes and when, why the side-car joins group 0, why it
waits on pmm-server's health and what breaks if the pin is rolled back, and how
to exchange a PMM browser session for a SEP bearer now that nothing injects one
server-side. Records that BASE_URL deliberately names a compose service here,
which the top-level README warns against, because the harness's only Nomad
executor shares pmm-server's network namespace — and that the harness therefore
does not exercise the production artifact-download path.

The image-pinning section drops the three-tag table: the build emits one
artifact, so there is a single suffix-less main-line tag to pin, and the two
properties that matter are worth checking on the artifact rather than the commit.

mysql-target.md loses its instruction to run the deleted mint script.
…uter

The previous pin predated the artifact-download fix, so the image mounted no
/artifacts router and an executor's download URL could not resolve regardless of
BASE_URL. This tag is cut from a main that carries it, and also ships the
pre-execution connectivity check defaulting to disabled.

Pre-flight against the published artifact: Healthcheck names healthcheck.sh,
runs as sep, and settings-env.sh resolves SECRETS_DIR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge There are reasons to not merge the code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants