Skip to content

feat(kargo): add a progressive delivery pipeline with a verification gate - #2

Merged
koorikla merged 1 commit into
mainfrom
feat/kargo-progressive-delivery
Aug 26, 2026
Merged

koorikla merged 1 commit into
mainfrom
feat/kargo-progressive-delivery

Conversation

@koorikla

Copy link
Copy Markdown
Owner

Adds Kargo 1.11.2 and a four-stage pipeline promoting a podinfo image through dev1-0 → dev1-1 → tst → prd, gated on a real health check at each step.

Promotion is a Git commit, not a cluster write

Each Stage's promotionTemplate clones the repo, rewrites spec.image in that stage's XMicroservice under crossplane/xrs/stages/<stage>/, commits and pushes. Argo CD then syncs it — so cluster state still comes only from Git, and every promotion is visible in history.

Freight flows one way: dev1-0 takes it directly from the Warehouse, and each later stage sources only from its predecessor. Nothing reaches prd without passing every gate before it.

The gate is an Argo Rollouts kind, not a Kargo one

AnalysisTemplate lives in argoproj.io — Kargo delegates verification to Rollouts' analysis engine. Its chart documents that it self-disables when those CRDs are missing:

When enabled, the API server will perform a sanity check at startup. If Argo Rollouts CRDs are not found, the API server will proceed as if this integration had been explicitly disabled.

So the gate would silently never run and nothing would say why. The bootstrap installs Argo Rollouts first and waits for analysistemplates.argoproj.io to be established before installing Kargo.

The check calls podinfo's /healthz through the Service the XMicroservice composition created — three consecutive successes, failureLimit: 0. That exercises the whole chain: Kargo committed → Argo CD synced → Crossplane composed a Deployment and Service → pods are serving. A bad image fails here and the Freight never becomes eligible for the next stage.

Choices worth reviewing

  • podinfo, not the nginx image the team-a demo uses — it publishes clean semver tags and serves a health endpoint, which is what makes the gate a real check rather than a formality.
  • Stage namespaces are separate from tenants.yamlteam-a/team-b are tenants, dev1-0/dev1-1/tst/prd are environments. Different axes, and a tenant's queues shouldn't be disturbed by a promotion.
  • No separate Argo CD Application for the stage XRs. The existing xrs app sets directory.recurse, so it already covers crossplane/xrs/stages. A second Application over the same path would leave two owners fighting over the same resources.
  • Git credentials follow the Backstage pattern — rendered from .env by the bootstrap into a Secret kept outside any Argo CD source path, since it holds a real token. Kargo matches credentials by the kargo.akuity.io/cred-type=git label rather than by name.

Note on the Warehouse

The Warehouse CRD sets x-kubernetes-preserve-unknown-fields on subscriptions, so a misspelled optional field is silently ignored rather than rejected. The subscription therefore uses only the minimal documented form and relies on the default SemVer selection strategy — verified against a running Kargo rather than guessed from the schema.

🤖 Generated with Claude Code

…gate

Adds Kargo 1.11.2 and a four-stage pipeline promoting a podinfo image through
dev1-0 -> dev1-1 -> tst -> prd, gated on a real health check at each step.

A promotion is a Git commit, not a cluster write. Each Stage's promotionTemplate
clones the repo, rewrites spec.image in that stage's XMicroservice under
crossplane/xrs/stages/<stage>/, commits and pushes; Argo CD then syncs it. The
cluster state still comes only from Git and every promotion is visible in
history. Freight flows one way -- dev1-0 takes it directly from the Warehouse,
each later stage sources only from its predecessor -- so nothing reaches prd
without passing every gate before it.

The gate is an Argo Rollouts AnalysisTemplate, not a Kargo kind: Kargo delegates
verification to Rollouts' analysis engine. Its chart checks for those CRDs at
startup and silently disables the integration when they are absent, so the
bootstrap installs Argo Rollouts first and waits for the CRDs to be established
-- otherwise the gate would never run and nothing would report why. The check
calls podinfo's /healthz through the Service the XMicroservice composition
created, three times with no retries, which exercises the whole chain: Kargo
committed, Argo CD synced, Crossplane composed a Deployment and Service, and
pods are serving.

podinfo rather than the nginx image the team-a demo uses, because it publishes
clean semver tags and serves a health endpoint, which is what makes the gate a
real check rather than a formality.

Stage namespaces are separate from tenants.yaml on purpose: team-a/team-b are
tenants and dev1-0/dev1-1/tst/prd are environments. They are different axes, and
a tenant's queues should not be disturbed by a promotion.

The per-stage XRs deliberately do not get their own Argo CD Application -- the
existing `xrs` app sets directory.recurse, so it already covers
crossplane/xrs/stages, and a second Application over the same path would leave
two owners fighting over the same resources.

Git credentials follow the Backstage pattern: rendered from .env by the
bootstrap into a Secret outside any Argo CD source path, since it holds a real
token. Kargo matches credentials by the kargo.akuity.io/cred-type=git label
rather than by name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@koorikla
koorikla merged commit 718a1f5 into main Aug 26, 2026
@koorikla
koorikla deleted the feat/kargo-progressive-delivery branch August 26, 2026 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant