feat(builder-api): OpenMeter session exchange with provisioning by eliteprox · Pull Request #57 · livepeer/clearinghouse

eliteprox · 2026-07-01T17:09:50Z

Summary

Adds builder-api (Auth0 user provision, RFC 8693 token exchange, signer-session mint) and supporting auth0-provisioner / identity-webhook work from the OpenMeter integration branch.
Implements minimal no-DB session provisioning on token exchange: deterministic customer key {clientId}:{externalUserId}, idempotent OpenMeter customer upsert, default starter-plan subscription, trial allowance grant, and balance read before signer JWT mint.
Returns real allowance/balance fields in the exchange response and blocks mint with HTTP 402 when trial credits are exhausted.

Key flow

resolve subject → ProvisionSession (customer + subscription + grant + balance)
  → allowance gate (402 if exhausted) → Auth0 signer JWT mint → response with balance fields

Test plan

go test ./... in openmeter-collector/builder-api (all pass)
Run builder-api locally with OpenMeter credentials; exchange a user token and confirm customer/subscription/grant created
Repeat exchange for same user — idempotent, no duplicate resources
Exhaust trial allowance — exchange returns 402 insufficient_allowance
End-to-end MCP/signer smoke test with LIVEPEER_SIGNER after exchange succeeds

Adds the Node.js identity webhook for remote signer authentication using builder-sdk's API-key provider (@pymthouse/builder-sdk@0.5.0), wires the remote-signer to depend on it via REMOTE_SIGNER_WEBHOOK_URL, and adds the Docker Compose stack for runtime services. Squashed from 15 commits on feat/identity-webhook (PR #36). Closes #2, #4. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…uth0 client provisioning - Introduced a new `bootstrap.sh` script for idempotent provisioning of Auth0 resources, including a resource server and client pairs. - Added `apps.json` to define the resource server and client configurations. - Updated `.gitignore` to exclude the generated `.env.livepeer` file. - Created a README.md to document usage and prerequisites for the new provisioning setup. This update streamlines the process of setting up Auth0 clients and resource servers, ensuring a smooth integration with the Livepeer clearinghouse API.

…co-location Add the Go Builder API beside Benthos in openmeter-collector: provision Auth0 end-users, upsert OpenMeter customers, and mint signer JWTs via client_credentials. Extend bootstrap.sh with a Management API M2M client and wire compose port 8095.

…otstrap Replace builder-sdk with in-repo protocol and jose verifiers so minted Auth0 signer JWTs resolve to auth_id alongside sk_* API keys. Add bootstrap script for the Credentials Exchange Action and document OIDC env for the local stack.

- Updated the OpenMeter collector to include a new endpoint for exchanging OIDC access tokens for signer sessions, allowing for better integration with Auth0. - Added a `DISCOVERY_URL` to the environment configuration for service discovery. - Enhanced the README documentation to reflect the new OIDC signer session functionality and updated environment variable requirements. - Improved the identity verification process by implementing a fallback mechanism for subject claims in the OIDC verifier. - Introduced tests for the new OIDC verification logic to ensure robust functionality.

…nce API key resolution - Updated the API to consolidate the signer-session endpoint, allowing both API keys and OIDC tokens to be exchanged for signer JWTs. - Enhanced the resolver logic to prioritize JWT verification over API key resolution, improving security and flexibility. - Updated the OpenAPI documentation to reflect the new endpoint structure and authentication methods. - Added tests for the new API key resolution and OIDC verification logic to ensure robust functionality. - Revised README files to clarify usage and configuration for the updated signer-session functionality.

…er JWTs - Introduced a new endpoint for exchanging OIDC access tokens and API keys for signer JWTs, enhancing integration with Auth0. - Updated the OpenAPI documentation to reflect the new token exchange functionality and its requirements. - Enhanced error handling for invalid requests and improved client credential validation. - Added comprehensive tests for the token exchange logic to ensure robust functionality and compliance with RFC 8693. - Revised README files to clarify usage and configuration for the new token exchange feature.

- Replaced the previous buildEndUserVerifier function with createEndUserVerifierFromEnv to streamline the configuration of verifiers based on the IDENTITY_AUTH_MODE environment variable. - Removed the first-match verifier logic, enforcing a strict selection between OIDC and API key verifiers. - Updated server initialization to log the selected authentication mode. - Enhanced error handling for missing required environment variables in both API key and OIDC modes. - Added tests for the new verifier setup, ensuring proper functionality and error management.

…lowance on session exchange Add no-database session provisioning keyed by clientId:externalUserId so user-scoped usage and trial credits are enforced before signer JWT mint.

…-webhook - Updated the configuration to support identity-webhook for verifying end-user JWTs, enhancing security and flexibility. - Removed deprecated trial feature keys and related configurations from the environment setup. - Simplified the provisioning logic by eliminating unnecessary trial grant handling. - Revised OpenAPI documentation to reflect changes in the token exchange process and updated environment variable requirements. - Enhanced README documentation to clarify the new identity verification process and configuration options.

…se configuration - Added support for loading multiple environment files in the entrypoint script, improving flexibility for configuration management. - Updated docker-compose.yml to expose port 8095 and include a new environment file for the auth0-provisioner, facilitating better integration and service communication. - Implemented cleanup logic in the entrypoint script to manage child processes effectively, ensuring graceful shutdowns.

…tomer ID - Modified the subscription listing function to use the correct bracketed filter format for the customer ID, ensuring accurate subscription retrieval from Konnect. - Added comments to clarify the necessity of this change for idempotency and correct filtering.

…binding and SPAT issuance - Introduced a new service for managing per-tenant Konnect organizations, allowing tenants to bind their organizations and issue Provisioner, Usage, and Ingest SPATs. - Added routes for binding organizations, issuing credentials, and managing catalog provisioning. - Implemented encryption for sensitive data at rest and ensured proper role assignments for system accounts. - Enhanced README documentation to provide clear instructions on usage and configuration for the new service. - Included tests to validate the functionality and isolation of tenant credentials.

… provisioning and trial management - Implemented a new session service in the builder-api to provision OpenMeter customers and subscriptions, including trial credits management. - Updated environment configuration to support trial feature keys and allowance enforcement. - Enhanced the token exchange process to check for trial credits before minting signer JWTs, returning appropriate error responses when allowances are exhausted. - Added new endpoints and updated existing ones to facilitate OpenMeter lookups and session provisioning. - Improved documentation to reflect changes in the provisioning logic and environment variable requirements. - Included comprehensive tests to validate the new session provisioning and trial grant functionalities.

Closes #9. Lands auth0-provisioner/ (split from #57) and a root bootstrap.sh that runs both provisioners and emits the two artifacts the SDK and platform consume. The orchestrator deliberately does not reimplement either provisioner. The Auth0 definition stays in auth0-provisioner/provision/apps.json and the catalog definition stays in openmeter-collector/provision/catalog.json, each the single source of truth for its half. This is why PR #33's Go CLI is being closed rather than rebased: it reimplemented both, with one Auth0 scope where apps.json defines seven, and a catalog predating #70's fee_wei, billable_secs and owner/demo plan split. The one thing #33 had that nothing else does is the sdk-config.json emitter, ported here to jq. Its output is byte-identical to #33's own golden file, which is committed as testdata/sdk-config.golden.json and asserted on in CI so the port cannot drift. .env.livepeer is parsed with sed rather than sourced — it holds secrets and arbitrary values, and sourcing would execute them. There is a test for that. CI gains a bootstrap job: shellcheck over every provisioner and the orchestrator, plus the emitter regression test. All existing shell in the repo already passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

The Go builder-api is not exercised by CI today: the openmeter-collector job only runs the Benthos suite and never builds it. PR #57 shows green on identity-webhook (22)/(24) and nothing else. Its own workflow rather than a job in ci.yml, because GitHub path filters are workflow-level. A job added to ci.yml would be part of every run that workflow triggers and would surface as a check on unrelated PRs, including before the service exists. A separate workflow with its own paths: filter does not run at all, so no check appears until the code is there and a PR touches it. ci.yml is left untouched — no existing job changes behaviour. The workflow does not trigger on its own file path, which is what keeps the check off PRs with no service code. The trade-off is that a later edit to it is not exercised until service files change in the same PR. It still guards on the service directory after checkout: a PR that deletes the service matches the path filter, and the run should report that rather than fail on a missing directory. konnect-credentials gets no workflow here. That service is parked pending the shared-tenant vs per-tenant-org decision and nothing in this stack adds it, so its CI should land with the service rather than sit as dead config. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Closes #9. Lands auth0-provisioner/ (split from #57) and a root bootstrap.sh that runs both provisioners and emits the two artifacts the SDK and platform consume. The orchestrator deliberately does not reimplement either provisioner. The Auth0 definition stays in auth0-provisioner/provision/apps.json and the catalog definition stays in openmeter-collector/provision/catalog.json, each the single source of truth for its half. This is why PR #33's Go CLI is being closed rather than rebased: it reimplemented both, with one Auth0 scope where apps.json defines seven, and a catalog predating #70's fee_wei, billable_secs and owner/demo plan split. The one thing #33 had that nothing else does is the sdk-config.json emitter, ported here to jq. Its output is byte-identical to #33's own golden file, which is committed as testdata/sdk-config.golden.json and asserted on in CI so the port cannot drift. .env.livepeer is parsed with sed rather than sourced — it holds secrets and arbitrary values, and sourcing would execute them. There is a test for that. CI gains a bootstrap job: shellcheck over every provisioner and the orchestrator, plus the emitter regression test. All existing shell in the repo already passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* ci(builder-api): add a path-filtered workflow instead of a ci.yml job Add a dedicated GitHub Actions workflow for builder-api so CI actually builds that Go service. The openmeter-collector job only runs the Benthos suite, so builder-api is never exercised today (PR #57 is green on identity-webhook only). The workflow is separate from ci.yml because GitHub path filters are workflow-level: a new job in ci.yml would run on every trigger of that workflow and show as a check on unrelated PRs, including before the service exists. A standalone workflow with its own paths filter does not run at all until a PR touches service files, so no check appears until then. ci.yml is unchanged. The workflow does not trigger on its own file path, which keeps the check off PRs with no service code; a later edit to the workflow is not exercised until service files change in the same PR. After checkout it still guards on the service directory so a PR that deletes the service reports that instead of failing on a missing path. konnect-credentials is not given a workflow here. That service is parked pending the shared-tenant vs per-tenant-org decision, so its CI should land with the service rather than sit as dead config.

Closes #9. Lands auth0-provisioner/ (split from #57) and a root bootstrap.sh that runs both provisioners and emits the two artifacts the SDK and platform consume. The orchestrator deliberately does not reimplement either provisioner. The Auth0 definition stays in auth0-provisioner/provision/apps.json and the catalog definition stays in openmeter-collector/provision/catalog.json, each the single source of truth for its half. This is why PR #33's Go CLI is being closed rather than rebased: it reimplemented both, with one Auth0 scope where apps.json defines seven, and a catalog predating #70's fee_wei, billable_secs and owner/demo plan split. The one thing #33 had that nothing else does is the sdk-config.json emitter, ported here to jq. Its output is byte-identical to #33's own golden file, which is committed as testdata/sdk-config.golden.json and asserted on in CI so the port cannot drift. .env.livepeer is parsed with sed rather than sourced — it holds secrets and arbitrary values, and sourcing would execute them. There is a test for that. CI gains a bootstrap job: shellcheck over every provisioner and the orchestrator, plus the emitter regression test. All existing shell in the repo already passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

eliteprox · 2026-08-13T02:28:26Z

Superseded — this has been split into four reviewable PRs, all green.

This PR's content	Landed as
`openmeter-collector/builder-api/`	#79
`entrypoint.sh`, `collector.yaml`, `Dockerfile`, compose	#80
admin surface + per-tenant boundary (new, closes #10 / #12)	#81
`auth0-provisioner/` + one-command bootstrap (closes #9)	#82
`konnect-credentials/`	not landing — see below

Plus #78, a one-file CI workflow, because none of this was exercised by CI: the openmeter-collector job only ran the Benthos suite and never built the Go service, so the only green checks here were identity-webhook (22)/(24).

Two things the split caught

collector.yaml needed a real three-way merge. This branch predates #70, and both rewrote that file. git merge-tree reports no conflict only because they touch different regions — taking this version verbatim would have silently reverted #70's app-owner vs M2M identity split and exact-fractional fee mapping. #80 carries the merged content and passes main's full Benthos suite, which had never once run against these changes.

The collector could not start. collector.yaml referenced KONNECT_CREDENTIALS_URL and PLATFORM_API_SECRET in dollar-brace interpolations. Benthos substitutes those in the raw config text before parsing, which makes them lint-required — so with the per-tenant credentials service absent, the container failed at startup. Found by running the built image, and fixed in #80 by removing the per-tenant ingest branch.

konnect-credentials

Not landing. livepeer/clearinghouse uses one shared Kong OpenMeter organization; customers are separated by the {clientId}:{externalUserId} customer-key contract already expressed in this branch and in pymthouse.

The premise behind per-tenant orgs is correct and is worth keeping on the record: Konnect metering roles are org-wide, so a SPAT in a shared org can read every customer. That is exactly why no tenant gets one — the platform credential stays inside the clearinghouse and the boundary is enforced above it, in the admin API. The four enforcement layers, the manual operator prerequisites, and the rejected alternative are documented in openmeter-collector/builder-api/docs/TENANT-ISOLATION.md in #81.

Consequently internal/openmeter/resolver.go is gone: it made the per-tenant lookup the primary path with the shared org as fallback, which is backwards under this model and had no service to call.

#58

Its two auth0-provisioner fixes are in #82 — the client-grant URL encoding in particular, since idempotency is the main argument for keeping this provisioner over the Go CLI in #33. Its remote-signer/entrypoint.sh fix is not included; that bug is on main, nothing in the stack touches remote-signer, and it should land on its own rather than disappear with this PR.

Leaving this open rather than closing it, since the branch is the only place konnect-credentials/ exists. Close it whenever you are satisfied the split is complete.

Third split out of #57: the collector-side wiring that actually runs the builder-api binary. Multi-stage Dockerfile builds the Go service and drops it beside the Benthos collector; entrypoint supervises both. collector.yaml here is the real three-way merge with main, not #57's copy. #57 branched before #70 and both rewrote this file, so taking #57's version verbatim would have silently reverted the app-owner vs M2M identity split and the exact-fractional fee mapping. The merged pipeline passes main's full Benthos suite, including #70's own identity and fee cases. docker-compose drops the konnect-credentials service, its depends_on health gate, KONNECT_CREDENTIALS_URL, and the auth0-provisioner .env.livepeer bind mount. Neither directory is in the tree; the bind mount in particular would have had Docker create a directory at that path and break the container. entrypoint.sh already falls back to OPENMETER_URL when KONNECT_CREDENTIALS_URL is unset, so the single-org path works as-is. Verified: benthos test suite passes, shellcheck clean on entrypoint.sh, docker compose config validates, and the image builds and runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Closes #9. Lands auth0-provisioner/ (split from #57) and a root bootstrap.sh that runs both provisioners and emits the two artifacts the SDK and platform consume. The orchestrator deliberately does not reimplement either provisioner. The Auth0 definition stays in auth0-provisioner/provision/apps.json and the catalog definition stays in openmeter-collector/provision/catalog.json, each the single source of truth for its half. This is why PR #33's Go CLI is being closed rather than rebased: it reimplemented both, with one Auth0 scope where apps.json defines seven, and a catalog predating #70's fee_wei, billable_secs and owner/demo plan split. The one thing #33 had that nothing else does is the sdk-config.json emitter, ported here to jq. Its output is byte-identical to #33's own golden file, which is committed as testdata/sdk-config.golden.json and asserted on in CI so the port cannot drift. .env.livepeer is parsed with sed rather than sourced — it holds secrets and arbitrary values, and sourcing would execute them. There is a test for that. CI gains a bootstrap job: shellcheck over every provisioner and the orchestrator, plus the emitter regression test. All existing shell in the repo already passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Third split out of #57: the collector-side wiring that actually runs the builder-api binary. Multi-stage Dockerfile builds the Go service and drops it beside the Benthos collector; entrypoint supervises both. collector.yaml here is the real three-way merge with main, not #57's copy. #57 branched before #70 and both rewrote this file, so taking #57's version verbatim would have silently reverted the app-owner vs M2M identity split and the exact-fractional fee mapping. The merged pipeline passes main's full Benthos suite, including #70's own identity and fee cases. docker-compose drops the konnect-credentials service, its depends_on health gate, KONNECT_CREDENTIALS_URL, and the auth0-provisioner .env.livepeer bind mount. Neither directory is in the tree; the bind mount in particular would have had Docker create a directory at that path and break the container. entrypoint.sh already falls back to OPENMETER_URL when KONNECT_CREDENTIALS_URL is unset, so the single-org path works as-is. Verified: benthos test suite passes, shellcheck clean on entrypoint.sh, docker compose config validates, and the image builds and runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Closes #9. Lands auth0-provisioner/ (split from #57) and a root bootstrap.sh that runs both provisioners and emits the two artifacts the SDK and platform consume. The orchestrator deliberately does not reimplement either provisioner. The Auth0 definition stays in auth0-provisioner/provision/apps.json and the catalog definition stays in openmeter-collector/provision/catalog.json, each the single source of truth for its half. This is why PR #33's Go CLI is being closed rather than rebased: it reimplemented both, with one Auth0 scope where apps.json defines seven, and a catalog predating #70's fee_wei, billable_secs and owner/demo plan split. The one thing #33 had that nothing else does is the sdk-config.json emitter, ported here to jq. Its output is byte-identical to #33's own golden file, which is committed as testdata/sdk-config.golden.json and asserted on in CI so the port cannot drift. .env.livepeer is parsed with sed rather than sourced — it holds secrets and arbitrary values, and sourcing would execute them. There is a test for that. CI gains a bootstrap job: shellcheck over every provisioner and the orchestrator, plus the emitter regression test. All existing shell in the repo already passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(collector): run builder-api alongside Benthos Build the Go builder-api into the openmeter-collector image and supervise it next to Benthos from the entrypoint. collector.yaml is the three-way merge with main, not #57's copy. #57 branched before #70 and both rewrote the file, so taking #57 verbatim would have silently reverted the app-owner vs M2M identity split and the exact-fractional fee mapping. JWT subject-token verification reuses the signer webhook (`REMOTE_SIGNER_WEBHOOK_URL` + `WEBHOOK_SECRET`) rather than a separate `IDENTITY_WEBHOOK_URL`; session exchange and go-livepeer already share that `/authorize` contract.

Closes #9. Lands auth0-provisioner/ (split from #57) and a root bootstrap.sh that runs both provisioners and emits the two artifacts the SDK and platform consume. The orchestrator deliberately does not reimplement either provisioner. The Auth0 definition stays in auth0-provisioner/provision/apps.json and the catalog definition stays in openmeter-collector/provision/catalog.json, each the single source of truth for its half. This is why PR #33's Go CLI is being closed rather than rebased: it reimplemented both, with one Auth0 scope where apps.json defines seven, and a catalog predating #70's fee_wei, billable_secs and owner/demo plan split. The one thing #33 had that nothing else does is the sdk-config.json emitter, ported here to jq. Its output is byte-identical to #33's own golden file, which is committed as testdata/sdk-config.golden.json and asserted on in CI so the port cannot drift. .env.livepeer is parsed with sed rather than sourced — it holds secrets and arbitrary values, and sourcing would execute them. There is a test for that. CI gains a bootstrap job: shellcheck over every provisioner and the orchestrator, plus the emitter regression test. All existing shell in the repo already passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

eliteprox and others added 11 commits June 30, 2026 18:27

feat(builder-api): provision OpenMeter customer, subscription, and al…

7f9c425

…lowance on session exchange Add no-database session provisioning keyed by clientId:externalUserId so user-scoped usage and trial credits are enforced before signer JWT mint.

rickstaa mentioned this pull request Jul 3, 2026

fix: provisioner + signer repro-blockers for local bring-up #58

Open

eliteprox added 3 commits July 7, 2026 12:37

eliteprox linked an issue Jul 17, 2026 that may be closed by this pull request

Administrative API wrapper over KongHQ metering & billing #10

Closed

Merge branch 'main' into feat/session-exchange-openmeter-provisioning

a4e62ee

This was referenced Aug 12, 2026

ci: add a builder-api workflow #78

Merged

feat(builder-api): Go session-exchange service #79

Merged

feat(collector): run builder-api alongside Benthos #80

Merged

feat(bootstrap): one-command provisioning + sdk-config emitter #82

Open

eliteprox closed this Aug 17, 2026

Navigation Menu

Uh oh!

feat(builder-api): OpenMeter session exchange with provisioning - #57

feat(builder-api): OpenMeter session exchange with provisioning#57
eliteprox wants to merge 15 commits into
mainfrom
feat/session-exchange-openmeter-provisioning

eliteprox commented Jul 1, 2026

Uh oh!

eliteprox commented Aug 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

eliteprox commented Jul 1, 2026

Summary

Key flow

Test plan

Uh oh!

eliteprox commented Aug 13, 2026

Two things the split caught

konnect-credentials

#58

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant