feat(agents): add `agents` content type, built and served alongside skills by gewenyu99 · Pull Request #181 · PostHog/context-mill

gewenyu99 · 2026-06-10T13:52:33Z

Adds the agents content type and the orchestrator integration flow, built and served alongside skills.

New agents content type: per-task agent prompts + mini step-skills for the integration flow, with per-flow registry markers (flow + seed frontmatter).
Framework commandments emitted as references/COMMANDMENTS.md; per-framework variants for the SDK-divergent step-skills (install, init, capture, error-tracking), each packaging that framework's docs page.
Parallel error-tracking, full-graph seeding, cached paths; dashboard treated as a non-CI task.

…kills New content type for orchestrator agent prompts (the WHAT), parallel to skills. Source under transformation-config/agents/, built to dist/agents/ as raw markdown plus agent-menu.json, served by the dev server at /agents/<type>.md and /agent-menu.json. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Agent prompts (agents/) and paired single-purpose mini-skills (skills/) for the task-queue orchestrator: install, init, identify, error-tracking, plan-capture, capture, build, dashboard, report, plus the integrate-posthog seed. Mini-skills are grounded in PostHog docs via docs_urls + {references}. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Merges plan-capture into capture (one pass, writes .posthog-events.json for the event-plan view and the report); makes error-tracking a single global boundary; orders dashboard and report after build, which now also lints and tests; install stays manifest-only and build flags out-of-scope conflicts and moves on; report draws on the queue log and the events file; the docs-only skills drop framework specifics in favor of the bundled docs and examples. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

gewenyu99 · 2026-06-10T13:52:47Z

feat(agents): add agents content type, built and served alongside skills #181 👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Every agent names its flow (the program id) and each flow marks exactly one prompt seed: true, the planner. The wizard's registry is scoped per flow, so audit and migration flows can ship their own agent sets alongside these. The canonical example moves to a README served to authors, not the menu — the build skips it, so 'example' is no longer an installable agent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

The orchestrator installs the framework's integration skill as the run reference and points task agents at individual files, so the tag-matched rules need to exist outside the SKILL.md body. The file also joins the references listing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

ci: false in the dashboard frontmatter — CI runs must not create dashboards, so the wizard's registry drops the type there. The seed plans around missing types, rewiring dependents to the nearest upstream step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…licy Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

…aths Seed wires error-tracking after install+init (parallel with identify/ capture) instead of after capture; error-tracking agent is instrument-only (no install/build/test, stay in project). Seed prompt drops the "plan without the missing ones" hedge so the full chain (through report) is always queued. Report reads .posthog-wizard-cache/ (renamed cache dir). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

error-tracking: use the framework's PostHog middleware / capture_exception(), never a hand-rolled middleware that builds exception events via capture(). capture: use the authed user's id; for unauthenticated actions emit a personless event instead of fabricating an 'anonymous' distinct id. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Framework specifics belong in docs/variants, not a step's prose. The fixed reference (integration-<framework>) now carries the framework's EXAMPLE.md + COMMANDMENTS (incl. PosthogContextMiddleware), so the step stays generic. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Stop the build agent flailing when install is slow/erroring: no offline-flag retries, no poking the package manager's global store/cache, nothing outside the project dir. A build/install it can't cleanly finish is fine as long as it's reported in the handoff conflict — reporting and moving on is the right outcome. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Add https://posthog.com/docs/product-analytics/identity-resolution.md alongside identify-users.md in every skill/doc that bundles the identity doc (identify, integration, the omnibus instrument skills, the audit family, revenue-analytics, best-practices, and the docs.yaml inline doc) so every identity-touching agent gets the resolution guidance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The build agent was re-running typecheck/lint in a loop chasing a green it could never reach, because the test app had pre-existing type errors in files the integration never touched. Make ownership explicit: only fix errors in files this integration changed; a failure in an untouched file is pre-existing, note it and move on. Don't re-run hoping it clears. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Adapt the agents content type to the post-reorg layout (transformation-config became context/, basics became example-apps): - agents move to context/agents/ as a new category alongside skills; the orchestrator mini-skills move to context/skills/ - skill-generator: keep emitting references/COMMANDMENTS.md, adopt the workflowSteps + {workflow} placeholder from main - dev-server: keep agents serving + watch, use example-apps (drop basics) - audit-3000 config superseded by audit-session-replay from main Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The branch carried an identity-resolution shared_doc that main removed; the merge kept the old version, adding the doc to ~43 built skills. Align the 8 configs with main so the build output drifts only where the agents content type is added. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…p stray file - move the 8 per-task step skills (build, capture, dashboard, error-tracking-step, identify, init, install, report) under skills/basic-integration/ so they stop polluting the top-level skills namespace; IDs become basic-integration-<step> (path-based, like omnibus-*) - update the agent prompts skills: refs to the new IDs - delete the stray orchestrator-ci-plan.md committed by accident Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…rd-orchestrator flag) (#606) Experimental task-queue orchestrator, collapsed from an 11-PR stack into this single branch. Behind the `wizard-orchestrator` feature flag, off by default — with the flag off it is a no-op and the existing linear flow runs unchanged. ## Wizard PRs folded in (squash-merged into this branch) - #607 — queue persistence - #608 — orchestrator MCP tools: enqueue_task / complete_task / read_handoffs, with guards - #609 — executor - #619 — agent loader - #620 — email targeting - #635 — CI flag overrides - #637 — ephemeral skills - #638 — telemetry (responsiveness A/B spine) - #639 — CI task filter - #677 — run cache + responsiveness analytics - #678 — CI health-check advisory ## Related PRs, other repos - PostHog/context-mill#181 — `agents` content type: agent prompts plus the `basic-integration` step-skills, built and served alongside skills - PostHog/wizard-workbench#1879 — CI flag-override and task-exclude dispatch inputs (dev/CI stack) ## Flag gating — why this is a no-op with the flag off - `runProgram` forks on `isOrchestratorEnabled(flags)` = `flags['wizard-orchestrator'] === 'true'` (default false). Off, it runs `runLinearProgram`, the existing path, untouched. - The shared `runAgent` threads the same flag through, so its orchestrator-specific message handling is gated too. - The CI flag-override (`WIZARD_CI_FLAG_OVERRIDES`) is guarded by `NODE_ENV === 'production'` and stripped from published builds — a smoke test asserts the env-var names are physically absent — so it cannot flip the flag for real users.

The step-skills baked JS/Node specifics into every framework's run. Strip the JS-pinned docs_urls from the install, init, capture, and error-tracking configs, and de-hardcode the identify and capture prose, so SDK specifics come from the per-framework reference, not the step. lower_snake_case and the correlation header names stay — stable conventions, not framework specifics. Recover the universal monolith guidance the steps were missing: the analytics contract and scan-first in capture, error capture at natural boundaries, and a before-you-merge checklist in the report. Move the event plan to .posthog-wizard-cache/.posthog-events.json so it is run scaffolding, cleaned up with the rest, not left in the project. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…-skills Install, init, capture, and error-tracking each carry framework-specific work, so mirror the integration skill's 36 variants onto them: each variant packages that framework's docs page (duplicated across steps for now). The step prose stays generic; the variant supplies the framework HOW. Generic steps (identify, report, dashboard, build) stay single-variant. Reverses the docs_urls strip from the previous commit — the per-framework pages return as variants rather than inline JS-only docs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…or-reshape

…skills (role internal), variants_from Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

…ype}.md Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

gewenyu99 · 2026-07-10T01:28:46Z

Updated for the current shape of main (clean merge, the staleness was semantic).

Agents move to a folder per flow: context/agents/posthog-integration/<type>.md. The folder is the registry scope, menu entries carry flow, the build rejects a frontmatter mismatch.
Step-skills renamed basic-integration → posthog-integration and marked cli: role: internal, so the ~148 orchestrator-internal skills never surface in wizard skill list.
New variants_from: integration: the four SDK-divergent steps borrow the canonical framework matrix instead of duplicating it (−1,568 lines). Runtime-category tags and future frameworks are inherited by construction.
Dev server serves /agents/{flow}/{type}.md.

Verified live: wizard orchestrator ran the full flow against this branch on pi (7/8 done, 1 reasoned skip, 0 failed), run posted on the wizard draft PR.

…or-reshape

luna for the mechanical short-context tasks (install, init), terra for the seed and the judgment tasks that read across the codebase. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Give capture Write so it can create the events cache, drop the fenced test suite from build verification, and make the report reconstruct a missing event manifest and skip the pre-create read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The init step now writes the required key names (placeholder values) to .env.example so other developers know what to set — the env-documentation gap the rubric flags. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…not failed A failed build blocks the dashboard and report that depend on it, so a build that fails only on pre-existing errors the integration never touched now completes as done with a conflict note; failed is reserved for integration-caused breakage. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The build/review step now reviews every change before the report: it builds (and reports a failure clearly in the handoff), checks the added code matches the project's conventions, and confirms no unrelated file was edited or mangled — the things users complain about most. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

… miniskills Extract the dashboard + insight examples, the notebook mirror (absent from the orchestrator until now), and the fuller pre-merge checklist out of the linear conclude step into miniskills the dashboard and report agents use; error-tracking and report run on luna (validated: same quality, ~half the cost/time). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…aude) Each orchestrator agent carries model_pi/effort_pi and model_sdk/effort_sdk so the flow benchmarks either provider; the pi/sdk mapping is intentionally not 1:1 (error-tracking and report are luna on pi but sonnet on sdk). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The orchestrator adds agents, step-skills, and dashboard/notebook examples that push the bundle past 5MB; raise the gate to fit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

sarahxsanders

superrrrr well designed and easy to reason about. left a few thoughts on prompting the agent vs. enforcing things a little deterministically if they aren't already (if they are, ignore)

sarahxsanders · 2026-07-15T14:36:16Z

+installed and initialized (see the context from previous steps); build on that,
+do not re-check it.
+
+This is an instrument-only task. Do not install dependencies, run the build, run


might want to enforce this in an allowlist if not already

Good call. We can do this with pi agents (finally cries) I want to do another pass on this though as a separate PR. Noted

sarahxsanders · 2026-07-15T14:37:04Z

+
+## Build and verify
+
+Run the project's build (or typecheck) and lint scripts if they exist


same here with enforcing allowed tools in frontmatter, both would be best

Ditto. I will cover this in another round of PRs to this and the Wizard. It'll actually be super noisy to do.

sarahxsanders · 2026-07-15T14:37:39Z

+## Hand off the dashboard
+
+Emit the dashboard URL on its own line in your final message with this exact
+marker so the wizard surfaces it: `[DASHBOARD_URL] <full https url>`. A URL only


is there a tool call for this instead of string markers? could be brittle

Yeah this makes a ton of sense. We've always done it this way but tool call iis better. Gonna update this before this goes in

…key, pure variants_from Follow-ups from the review of #181: - skill-menu.json entries now carry `group` (hyphenated skill-id prefix), `framework` (the detection id a variant serves), and `default` (the variant a bare framework id resolves to when a family has several). Declared in the integration group's variant matrix and copied across by `variants_from`, so consumers resolve variants by exact match instead of reverse-engineering ids with alias tables and prefix guessing. Pairs with the wizard-side resolver change (PostHog/wizard PR on experiment/orchestrator-pi-runtask). - agent prompts must declare `flow:` — the build now rejects a missing key instead of only a contradicting one, since consumers filter prompts by it and a missing key built green here but silently dropped the prompt from the runtime registry. - `resolveVariantsFrom` is pure: it returns a resolved copy instead of mutating the config in place, which removes the `_variantsResolved` marker that existed to dodge the "variants or variants_from, not both" validation on re-expansion. Generated-By: PostHog Code Task-Id: fafc230d-6f14-4e4d-9462-0e7f18a1eec1

Generated-By: PostHog Code Task-Id: fafc230d-6f14-4e4d-9462-0e7f18a1eec1

…plify) Generated-By: PostHog Code Task-Id: fafc230d-6f14-4e4d-9462-0e7f18a1eec1

fix(agents): review fixes — declared framework identity, strict flow key, pure variants_from

…mmandment Renames the orchestrator agents/ and skills/ flow to integration-v2, and adds a javascript_web commandment to pass the project key to init() straight from env rather than defaulting to an empty string. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Mirrors the iOS guidance for Android — the project token is a public client key safe to embed, so source it but never fall back to an empty string, which silently disables analytics (caught on a real Tusky run: getenv(...) ?: ""). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The rename commit staged the directory move but not the frontmatter edits, so the committed agents still declared flow: posthog-integration in agents/integration-v2/, failing assertFlowMatches in CI. This commits the corrected flow + skill ids. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…n wizard) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…lls, reshape notebook Splits the monolithic dashboard skill into a `dashboard` microskill (create the container + URL handoff) and a reusable `insight` microskill (the verified trends/funnel query shapes); the dashboard agent now pulls both. Reshapes the notebook microskill into a single clean step with one positive escaping rule. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Read-then-replace when posthog-setup-report.md already exists (harnesses refuse to overwrite an unread file) — the salvaged half of the closed #241. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Surfaces the canonical shared/mcp-tool-calling grammar as an integration-v2 microskill via the {{> }} partial (shared file stays the single source), and wires it into the two agents that call the MCP — dashboard and report. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

gewenyu99 and others added 3 commits June 9, 2026 16:17

gewenyu99 and others added 12 commits June 10, 2026 15:24

revert ci awareness from agent content — the harness owns run-mode po…

464c6fd

…licy Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

gewenyu99 marked this pull request as ready for review June 18, 2026 13:17

gewenyu99 mentioned this pull request Jun 18, 2026

feat(orchestrator): experimental task-queue orchestrator (behind wizard-orchestrator flag) PostHog/wizard#606

Merged

gewenyu99 and others added 6 commits June 18, 2026 11:30

Merge remote-tracking branch 'origin/main' into experiment/orchestrat…

94dc534

…or-reshape

refactor(orchestrator): flow-scoped agents, posthog-integration step-…

2f51b16

…skills (role internal), variants_from Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

docs(agents): flow folders, gateway model ids, flat frontmatter rule

9d0cccd

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

fix(dev-server): serve flow-scoped agent prompts at /agents/{flow}/{t…

fb1fed0

…ype}.md Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

gewenyu99 requested a review from a team as a code owner July 10, 2026 01:28

gewenyu99 mentioned this pull request Jul 10, 2026

feat(pi): orchestrator runTask — per-task pi sessions with in-process queue tools PostHog/wizard#853

Merged

gewenyu99 and others added 9 commits July 14, 2026 13:32

Merge remote-tracking branch 'origin/main' into experiment/orchestrat…

b797afe

…or-reshape

feat(agents): run orchestrator integration agents on gpt-5.6

f9e686d

luna for the mechanical short-context tasks (install, init), terra for the seed and the judgment tasks that read across the codebase. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

ci: raise the mcp-resources bundle limit to 8MB

b0d4fd0

The orchestrator adds agents, step-skills, and dashboard/notebook examples that push the bundle past 5MB; raise the gate to fit. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

sarahxsanders reviewed Jul 15, 2026

View reviewed changes

gewenyu99 mentioned this pull request Jul 15, 2026

fix(agents): review fixes — declared framework identity, strict flow key, pure variants_from #239

Merged

gewenyu99 and others added 3 commits July 15, 2026 11:34

chore(agents): trim review-fix comments to one-liners

1d6c63b

Generated-By: PostHog Code Task-Id: fafc230d-6f14-4e4d-9462-0e7f18a1eec1

refactor: drop the stale variants_from key from resolved groups (/sim…

6eeba8e

…plify) Generated-By: PostHog Code Task-Id: fafc230d-6f14-4e4d-9462-0e7f18a1eec1

Merge pull request #239 from PostHog/posthog-code/agents-review-fixes

ccb54bd

fix(agents): review fixes — declared framework identity, strict flow key, pure variants_from

sarahxsanders approved these changes Jul 15, 2026

View reviewed changes

gewenyu99 and others added 3 commits July 15, 2026 18:03

Merge remote-tracking branch 'origin/main' into experiment/orchestrator

b85d8a9

gewenyu99 added mcp-publish minor labels Jul 15, 2026

gewenyu99 and others added 5 commits July 15, 2026 18:44

refactor(commandments): drop per-SDK empty-key rules (now universal i…

012623d

…n wizard) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

fix(integration-v2): report step handles a stale report from a prior run

afa69f3

Read-then-replace when posthog-setup-report.md already exists (harnesses refuse to overwrite an unread file) — the salvaged half of the closed #241. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

gewenyu99 merged commit eba9474 into main Jul 15, 2026
14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(agents): add `agents` content type, built and served alongside skills#181

feat(agents): add `agents` content type, built and served alongside skills#181
gewenyu99 merged 43 commits into
mainfrom
experiment/orchestrator

gewenyu99 commented Jun 10, 2026 •

edited

Loading

Uh oh!

gewenyu99 commented Jun 10, 2026

Uh oh!

gewenyu99 commented Jul 10, 2026

Uh oh!

sarahxsanders left a comment

Uh oh!

sarahxsanders Jul 15, 2026

Uh oh!

gewenyu99 Jul 15, 2026

Uh oh!

sarahxsanders Jul 15, 2026

Uh oh!

gewenyu99 Jul 15, 2026

Uh oh!

sarahxsanders Jul 15, 2026

Uh oh!

gewenyu99 Jul 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		## Build and verify

		Run the project's build (or typecheck) and lint scripts if they exist

Uh oh!

Conversation

gewenyu99 commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gewenyu99 commented Jun 10, 2026

Uh oh!

gewenyu99 commented Jul 10, 2026

Uh oh!

sarahxsanders left a comment

Choose a reason for hiding this comment

Uh oh!

sarahxsanders Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

gewenyu99 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

sarahxsanders Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

gewenyu99 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

sarahxsanders Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

gewenyu99 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gewenyu99 commented Jun 10, 2026 •

edited

Loading