Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/workflows/release-bundle.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Release bundle

on:
workflow_dispatch:
inputs:
bundle_version:
description: "Bundle version to publish, e.g. 4.4.2"
required: true

permissions:
contents: write

jobs:
release:
runs-on: ubuntu-latest
env:
BUNDLE_VERSION: ${{ inputs.bundle_version }}
HUGGING_FACE_TOKEN: ${{ secrets.HUGGING_FACE_TOKEN }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- uses: astral-sh/setup-uv@v8.1.0
with:
version: "0.11.14"
- name: Install validation tooling
run: python -m pip install --upgrade -e ".[dev]"
- name: Validate schemas and models
run: |
python scripts/validate_schemas.py
python scripts/validate_models.py
- name: Run tests
run: pytest
- name: Confirm private data access is configured
run: test -n "$HUGGING_FACE_TOKEN"
- name: Generate candidate bundle
run: |
python scripts/generate_bundle.py \
--input "candidates/${BUNDLE_VERSION}-all.json" \
--output ".tmp/${BUNDLE_VERSION}"
- name: Solve lockfiles
run: python scripts/solve_lockfiles.py ".tmp/${BUNDLE_VERSION}"
- name: Validate certified bundle
run: python scripts/validate_bundle.py ".tmp/${BUNDLE_VERSION}"
- name: Verify generated bundle matches committed bundle
run: |
python scripts/compare_bundle_directories.py \
"bundles/${BUNDLE_VERSION}" \
".tmp/${BUNDLE_VERSION}"
- name: Package certified bundle
run: |
python scripts/package_bundle_release.py \
"bundles/${BUNDLE_VERSION}" \
--output-dir dist
cp "bundles/${BUNDLE_VERSION}/bundle.json" \
"dist/bundle-${BUNDLE_VERSION}.json"
cp "bundles/${BUNDLE_VERSION}/validation-report.json" \
"dist/validation-report-${BUNDLE_VERSION}.json"
- name: Publish GitHub release assets
env:
GH_TOKEN: ${{ github.token }}
run: |
python scripts/publish_bundle_release_assets.py \
"${BUNDLE_VERSION}" \
--dist-dir dist \
--target-sha "${GITHUB_SHA}" \
--repo "${GITHUB_REPOSITORY}"
50 changes: 14 additions & 36 deletions .github/workflows/validate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,24 @@ jobs:
with:
python-version: "3.13"
- uses: astral-sh/setup-uv@v8.1.0
with:
version: "0.11.14"
- name: Install validation tooling
run: python -m pip install --upgrade -e ".[dev]"
- name: Validate schemas and example manifests
run: python scripts/validate_schemas.py
- name: Validate typed models
run: python scripts/validate_models.py
- name: Verify committed bundle digests
run: python scripts/verify_bundle_digests.py
- name: Run tests
run: pytest
- name: Check formatting
run: ruff format --check .
- name: Lint
run: ruff check .

certify-latest-bundle:
certify-live-4-4-5:
runs-on: ubuntu-latest
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == github.repository
env:
Expand All @@ -39,52 +43,26 @@ jobs:
with:
python-version: "3.13"
- uses: astral-sh/setup-uv@v8.1.0
with:
version: "0.11.14"
- name: Install validation tooling
run: python -m pip install --upgrade -e ".[dev]"
- name: Confirm private data access is configured
run: test -n "$HUGGING_FACE_TOKEN"
- name: Create work directory
run: mkdir -p .tmp
- name: Select latest candidate
id: latest
run: |
python - <<'PY' >> "$GITHUB_OUTPUT"
import json
from pathlib import Path

def version_key(version: str) -> tuple[int, ...]:
return tuple(int(part) for part in version.split("."))

candidates = []
for path in Path("candidates").glob("*-all.json"):
payload = json.loads(path.read_text())
version = payload["bundle_version"]
candidates.append((version_key(version), version, path))

if not candidates:
raise SystemExit("No bundle candidates found.")

_, version, path = max(candidates)
print(f"bundle_version={version}")
print(f"candidate={path.as_posix()}")
print(f"output=.tmp/bundle-{version}")
print(f"committed=bundles/{version}")
PY
- name: Generate bundle
run: |
python scripts/generate_bundle.py \
--input "${{ steps.latest.outputs.candidate }}" \
--output "${{ steps.latest.outputs.output }}"
--input candidates/4.4.5-all.json \
--output .tmp/bundle-4.4.5
- name: Solve lockfiles
run: python scripts/solve_lockfiles.py "${{ steps.latest.outputs.output }}"
run: python scripts/solve_lockfiles.py .tmp/bundle-4.4.5
- name: Validate bundle
run: python scripts/validate_bundle.py "${{ steps.latest.outputs.output }}"
run: python scripts/validate_bundle.py .tmp/bundle-4.4.5
- name: Compare regenerated bundle to committed bundle
run: |
python scripts/compare_bundle_directories.py \
"${{ steps.latest.outputs.committed }}" \
"${{ steps.latest.outputs.output }}"
run: python scripts/compare_bundle_directories.py bundles/4.4.5 .tmp/bundle-4.4.5
- uses: actions/upload-artifact@v4
with:
name: policyengine-bundle-${{ steps.latest.outputs.bundle_version }}-all
path: ${{ steps.latest.outputs.output }}
name: policyengine-bundle-4.4.5-all
path: .tmp/bundle-4.4.5
57 changes: 56 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,30 @@ their relative paths back into `bundle.json` as profile `install_targets`.
The supported Python versions come exclusively from
`bundle.json` `metadata.python_versions`; there is no per-run Python-version
override.
Candidates may also declare a resolver policy:

```json
{
"resolver": {
"name": "uv",
"version": "0.11.14",
"resolution": "highest",
"exclude_newer": "2026-05-09T10:39:06Z",
"universal": true
}
}
```

`generate_bundle.py` copies that policy into `bundle.json` metadata, and
`solve_lockfiles.py` passes it to `uv pip compile`. CI pins the same `uv`
version recorded in the resolver policy because `pylock.toml` output can change
across resolver versions. For published bundles, `exclude_newer` should be set
to a timestamp after all intended direct package releases exist and before
later unrelated transitive releases can change the solved graph. Published
bundles should keep `universal` enabled so `pylock.toml` captures a
platform-independent resolution instead of the runner host's wheel subset. This
keeps regeneration deterministic without storing a lockfile for every possible
platform.
Runtime validation requires every profile to contain exactly one install target
for each declared Python version, with no missing or undeclared targets.
Here, "lockfile" means an installation-resolution artifact, not a concurrency
Expand Down Expand Up @@ -295,6 +319,10 @@ constraints, verifies direct package versions, imports the profile packages, and
runs country household smoke checks where supported for every profile and every
declared install target. The resulting
`validation-report.json` is part of the bundle contract.
US analytics-only database artifacts are currently recorded as `unverified`
rather than certified, so they are discoverable in the country bundle but are
not included in SHA-based bundle certification until their publication contract
is stable.
Runtime validation records the current runner platform in check details, but
platform-specific lockfiles are intentionally out of scope for this contract.
For embedded release manifests, validation hashes the embedded file from the
Expand All @@ -315,6 +343,30 @@ Partial reports mark the skipped checks and set
`metadata.validation_scope` to `partial`. They are useful for schema fixtures,
but they are not evidence that a bundle is reproducible.

4. Package release artifacts:

```bash
python scripts/package_bundle_release.py bundles/4.4.0 --output-dir dist
```

This verifies the bundle has a passing full-validation report, computes the
stable `bundle_digest`, writes a missing digest into `bundle.json`, and creates a
reproducible `policyengine-bundle-4.4.0.tar.gz` archive plus a SHA256 checksum.
If `bundle.json` already records a digest, packaging fails unless that digest
matches the current bundle contents. The digest is computed from the same
bundle-relative files that go into the release archive. It intentionally ignores
run-local timestamps, temporary command paths, resolver comments, and the digest
field itself.

The manual `Release bundle` GitHub Actions workflow publishes those files as
GitHub release assets under tag `v<version>`. It regenerates the bundle from
`candidates/<version>-all.json`, solves lockfiles, runs full validation with the
repository Hugging Face token, and packages that freshly certified output. These
release assets are the stable distribution surface for consumers such as
`policyengine.py`; they do not replace `bundle.json` as the official manifest.
Existing release assets are not overwritten; the publisher refuses to create a
release if `v<version>` already exists.

CI regenerates the current certified bundle from the committed candidate spec
and compares it with the checked-in bundle using normalized output. The
comparison ignores run-local evidence such as timestamps, temporary paths, and
Expand Down Expand Up @@ -360,4 +412,7 @@ A bundle release should not be published unless:
- certified data artifacts include SHA256 hashes;
- country data release manifests are reachable;
- profile install targets solve for supported Python versions;
- integrated validation passes for each profile.
- integrated validation passes for each profile;
- the validation report has `metadata.validation_scope` set to `full`;
- no validation checks are skipped;
- the published release archive digest matches the checked bundle manifest.
Loading
Loading