-
Notifications
You must be signed in to change notification settings - Fork 8
feat(agents): add agents content type, built and served alongside skills
#181
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
43 commits
Select commit
Hold shift + click to select a range
caf360a
feat(agents): add `agents` content type, built and served alongside s…
gewenyu99 e70da8b
feat(orchestrator): agent prompts + mini-skills for the integration flow
gewenyu99 ba51465
refine the orchestrator integration flow
gewenyu99 b8e66ec
feat(agents): per-flow registry markers — flow + seed frontmatter
gewenyu99 5ddce45
feat(skills): emit framework commandments as references/COMMANDMENTS.md
gewenyu99 0a62554
feat(agents): dashboard is not a CI task
gewenyu99 464c6fd
revert ci awareness from agent content — the harness owns run-mode po…
gewenyu99 5a3d5e4
feat(orchestrator): parallel error-tracking, seed full graph, cache p…
gewenyu99 8092ddb
fix(skills): SDK error integration + personless anon events
gewenyu99 3f7325e
revert: drop hardcoded Django middleware prose from error-tracking-step
gewenyu99 3a9a57d
fix(build skill): don't fight install, report and move on
gewenyu99 08a91ca
feat(docs): reference identity-resolution.md everywhere identity is
gewenyu99 5d937f5
build skill: leave pre-existing errors in untouched files alone
gewenyu99 0132217
Merge origin/main into experiment/orchestrator
gewenyu99 46e04e7
fix: drop stale identity-resolution shared_doc to match main
gewenyu99 1744e83
refactor: nest orchestrator step-skills under basic-integration/, dro…
gewenyu99 a16ed9b
fix(orchestrator): keep basic-integration step-skills goal-oriented
gewenyu99 4d4a682
feat(orchestrator): per-framework variants for the SDK-divergent step…
gewenyu99 94dc534
Merge remote-tracking branch 'origin/main' into experiment/orchestrat…
gewenyu99 2f51b16
refactor(orchestrator): flow-scoped agents, posthog-integration step-…
gewenyu99 9d0cccd
docs(agents): flow folders, gateway model ids, flat frontmatter rule
gewenyu99 fb1fed0
fix(dev-server): serve flow-scoped agent prompts at /agents/{flow}/{t…
gewenyu99 b797afe
Merge remote-tracking branch 'origin/main' into experiment/orchestrat…
gewenyu99 f9e686d
feat(agents): run orchestrator integration agents on gpt-5.6
gewenyu99 1ee8552
fix(agents): address orchestrator benchmark remarks
gewenyu99 08661e5
feat(agents): document PostHog env keys in .env.example during init
gewenyu99 4dafc79
fix(agents): build completes-with-conflict on pre-existing breakage, …
gewenyu99 de88ab5
feat(agents): make the build step review the integration
gewenyu99 662d92d
feat(agents): pull dashboard/notebook/report detail into orchestrator…
gewenyu99 dfc4f07
feat(agents): name model + effort per harness profile (pi/gpt, sdk/cl…
gewenyu99 b0d4fd0
ci: raise the mcp-resources bundle limit to 8MB
gewenyu99 d72f23c
fix(agents): review fixes — declared framework identity, strict flow …
gewenyu99 1d6c63b
chore(agents): trim review-fix comments to one-liners
gewenyu99 6eeba8e
refactor: drop the stale variants_from key from resolved groups (/sim…
gewenyu99 ccb54bd
Merge pull request #239 from PostHog/posthog-code/agents-review-fixes
gewenyu99 23276d8
feat: rename posthog-integration flow to integration-v2 + init-key co…
gewenyu99 b85d8a9
Merge remote-tracking branch 'origin/main' into experiment/orchestrator
gewenyu99 b528c4f
fix(commandments): android must not default the project key to empty
gewenyu99 deece51
fix(agents): commit integration-v2 flow + skill refs (CI build fix)
gewenyu99 012623d
refactor(commandments): drop per-SDK empty-key rules (now universal i…
gewenyu99 5741df3
feat(integration-v2): split dashboard into dashboard+insight microski…
gewenyu99 afa69f3
fix(integration-v2): report step handles a stale report from a prior run
gewenyu99 60bf5f0
feat(integration-v2): mcp microskill for the steps that call PostHog
gewenyu99 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| # Agent prompts | ||
|
|
||
| One `<flow>/<type>.md` per orchestrator task — the WHAT of the task. The folder | ||
| is the flow: the wizard's registry is scoped per flow, so audit or migration | ||
| agent sets get sibling folders. The frontmatter carries the artifacts the | ||
| executor configures the run with: the model, the mini-skills to load (the HOW), | ||
| the tools the task may and may not use, and the tasks it depends on. `flow` | ||
| repeats the folder name so the file stays self-describing on disk (the build | ||
| rejects a mismatch), and one prompt per flow is marked `seed: true`: the | ||
| planner that seeds the queue, not an enqueueable task type. | ||
|
|
||
| `model` is a gateway model id, not a runner binding — the wizard's switchboard | ||
| routes it, and its CLI/flag rungs may override it per run. `allowedTools` / | ||
| `disallowedTools` use the wizard's tool vocabulary; the executing harness maps | ||
| them to its native tool names. | ||
|
|
||
| The body is intent only — what to do and what done looks like. The client | ||
| injects the basics (project context, how to report, how to surface progress), | ||
| so a prompt never restates them. Frontmatter stays flat: scalars and inline | ||
| `[a, b]` arrays only, the wizard's parser is deliberately not a YAML engine. | ||
|
|
||
| This README is documentation, not data: the build serves every other `.md` | ||
| under a flow folder as an agent prompt. | ||
|
|
||
| Each agent names its model per harness profile — `model_pi`/`effort_pi` for the | ||
| gpt (pi) run and `model_sdk`/`effort_sdk` for the anthropic (sdk) run — so the | ||
| same flow benchmarks either provider. The mapping is not 1:1: a light task can be | ||
| luna on pi but sonnet on sdk. Effort is optional and overrides the model default. | ||
|
|
||
| ```markdown | ||
| --- | ||
| type: example | ||
| flow: my-flow | ||
| model_pi: openai/gpt-5.6-luna | ||
| effort_pi: low | ||
| model_sdk: claude-haiku-4-5-20251001 | ||
| skills: [] | ||
| allowedTools: [Read, Glob, Grep] | ||
| disallowedTools: [enqueue_task] | ||
| dependsOn: [] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| What this task does, in plain prose. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| Plain-text success criteria live here. State what done looks like, and what to | ||
| do when the task cannot be completed. | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| --- | ||
| type: build | ||
| flow: integration-v2 | ||
| label: Build and review the integration | ||
| model_pi: openai/gpt-5.6-terra | ||
| effort_pi: medium | ||
| model_sdk: claude-sonnet-4-6 | ||
| effort_sdk: high | ||
| skills: [integration-v2-build] | ||
| allowedTools: [Read, Edit, Glob, Grep, Bash] | ||
| disallowedTools: [enqueue_task] | ||
| dependsOn: [install, init, identify, error-tracking, capture] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| Bring the integration together and review it: install the dependencies the earlier | ||
| steps declared, verify the project builds and lints, then review every change for | ||
| convention fit and unintended edits. Until now the steps only edited code and the | ||
| manifest — this is where it installs, is checked, and is reviewed as a whole. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| The install completes, the integration is in place, and the changes read like the | ||
| rest of the codebase with nothing unrelated touched or mangled. If the build or | ||
| lint fails only on pre-existing errors you did not introduce, that still counts as | ||
| done — note the conflict and finish. Reserve a failed status for when your own | ||
| changes break the build. Put a one-line summary of any conflict in your handoff's | ||
| `conflict` field and the full detail in what you did; the user sees the one-liner | ||
| in the outro and the detail in the report. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| --- | ||
| type: capture | ||
| flow: integration-v2 | ||
| label: Capture events | ||
| model_pi: openai/gpt-5.6-terra | ||
| effort_pi: medium | ||
| model_sdk: claude-sonnet-4-6 | ||
| effort_sdk: high | ||
| skills: [integration-v2-capture] | ||
| allowedTools: [Read, Write, Edit, Glob, Grep] | ||
| disallowedTools: [enqueue_task] | ||
| dependsOn: [install, init] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| Decide which events are worth capturing in this app, then instrument them in the | ||
| same pass — read each file once, choose the events, and add the capture calls | ||
| while the file is already open. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| The meaningful user actions across the app have capture calls that fire on the | ||
| real action, not on page load, and `.posthog-wizard-cache/.posthog-events.json` | ||
| lists the events you instrumented. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| --- | ||
| type: dashboard | ||
| flow: integration-v2 | ||
| label: Create a starter dashboard | ||
| model_pi: openai/gpt-5.6-terra | ||
| effort_pi: medium | ||
| model_sdk: claude-sonnet-4-6 | ||
| effort_sdk: high | ||
| skills: [integration-v2-dashboard, integration-v2-insight, integration-v2-mcp] | ||
| allowedTools: [Read, Glob, Grep] | ||
| disallowedTools: [Write, Edit, Bash, enqueue_task] | ||
| dependsOn: [build] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| Create a starter PostHog dashboard with a few insights built on the events this | ||
| integration instruments, using the PostHog MCP. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| A dashboard exists with a handful of insights on the captured events, and you hand | ||
| off its URL for the report to link. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| --- | ||
| type: error-tracking | ||
| flow: integration-v2 | ||
| label: Add error tracking | ||
| model_pi: openai/gpt-5.6-luna | ||
| effort_pi: low | ||
| model_sdk: claude-sonnet-4-6 | ||
| effort_sdk: high | ||
| skills: [integration-v2-error-tracking-step] | ||
| allowedTools: [Read, Write, Edit, Glob, Grep] | ||
| disallowedTools: [enqueue_task] | ||
| dependsOn: [install, init] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| Set up the framework's single global error boundary so uncaught errors reach | ||
| PostHog. One place — the init or app entry — following the docs and the reference | ||
| example, not manual capture calls sprinkled across files. The SDK is already | ||
| installed and initialized (see the context from previous steps); build on that, | ||
| do not re-check it. | ||
|
|
||
| This is an instrument-only task. Do not install dependencies, run the build, run | ||
| tests, or start the app — a later `build` step does all verification. Stay inside | ||
| this project's directory and edit the one global handler; that is the whole job. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| A global error handler forwards exceptions to PostHog. You did not install | ||
| anything, run a build or tests, search outside the project, or read through the | ||
| whole app or hand-wrap individual components or routes. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| --- | ||
| type: identify | ||
| flow: integration-v2 | ||
| label: Wire user identification | ||
| model_pi: openai/gpt-5.6-terra | ||
| effort_pi: medium | ||
| model_sdk: claude-sonnet-4-6 | ||
| effort_sdk: high | ||
| skills: [integration-v2-identify] | ||
| allowedTools: [Read, Edit, Glob, Grep] | ||
| disallowedTools: [enqueue_task] | ||
| dependsOn: [install, init] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| Wire user identification: call PostHog identify wherever the app establishes who | ||
| the user is, typically at login and signup. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| An identify call fires at the point the user becomes known, with a stable | ||
| distinct id. If the app has no auth or user concept, say so and stop. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| --- | ||
| type: init | ||
| flow: integration-v2 | ||
| label: Set up PostHog initialization | ||
| model_pi: openai/gpt-5.6-luna | ||
| effort_pi: low | ||
| model_sdk: claude-haiku-4-5-20251001 | ||
| skills: [integration-v2-init] | ||
| allowedTools: [Read, Write, Edit, Glob, Grep] | ||
| disallowedTools: [enqueue_task] | ||
| dependsOn: [] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| Initialize PostHog: create the framework's init point so the SDK is configured | ||
| once and available across the app, set the PostHog environment variables through | ||
| the wizard tools, and document those keys in `.env.example` for other developers. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| The init file exists and the PostHog env keys are present. Keys live in the env | ||
| file, never hardcoded in source, and `.env.example` lists the key names (with | ||
| placeholder values) so the next developer knows what to set. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| --- | ||
| type: install | ||
| flow: integration-v2 | ||
| label: Add the PostHog SDK to the manifest | ||
| model_pi: openai/gpt-5.6-luna | ||
| effort_pi: low | ||
| model_sdk: claude-haiku-4-5-20251001 | ||
| skills: [integration-v2-install] | ||
| allowedTools: [Read, Edit, Glob, Grep] | ||
| disallowedTools: [enqueue_task] | ||
| dependsOn: [] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| Declare the PostHog SDK in the project's package manifest. Do not run the package | ||
| manager and do not build — the build task installs and verifies everything at the | ||
| end. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| The SDK is listed in the manifest's dependencies at a sensible version. If it is | ||
| already declared, leave it and say so. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| --- | ||
| type: integrate-posthog | ||
| flow: integration-v2 | ||
| seed: true | ||
| model_pi: openai/gpt-5.6-terra | ||
| effort_pi: medium | ||
| model_sdk: claude-sonnet-4-6 | ||
| effort_sdk: high | ||
| skills: [] | ||
| allowedTools: [Read, Glob, Grep] | ||
| disallowedTools: [Write, Edit, Bash, complete_task] | ||
| dependsOn: [] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| Plan a PostHog integration and seed the task queue with this graph: | ||
|
|
||
| - `install` and `init`, independent of each other. | ||
| - `identify`, `capture`, and `error-tracking`, each after `install` and `init` | ||
| and independent of one another, so they run in parallel. `capture` decides the | ||
| events and instruments them; `error-tracking` wires the single global error | ||
| boundary — it needs the SDK installed and initialized, not the events. | ||
| - `build`, after `install`, `init`, `identify`, `capture`, and `error-tracking` — | ||
| it installs the dependencies and verifies the project builds, lints, and passes | ||
| its tests. | ||
| - `dashboard`, after `build` — only once the integration is confirmed building, | ||
| linting, and testing cleanly. | ||
| - `report`, after `dashboard` — it writes the setup report last. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| Every task in the graph is queued with that dependency shape, the report last, | ||
| and the first task runnable. Keep labels short — the action in a few words. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,27 @@ | ||
| --- | ||
| type: report | ||
| flow: integration-v2 | ||
| label: Write the setup report | ||
| model_pi: openai/gpt-5.6-luna | ||
| effort_pi: low | ||
| model_sdk: claude-sonnet-4-6 | ||
| effort_sdk: high | ||
| skills: [integration-v2-report, integration-v2-notebook, integration-v2-mcp] | ||
| allowedTools: [Read, Write, Glob, Grep] | ||
| disallowedTools: [enqueue_task] | ||
| dependsOn: [dashboard] | ||
| --- | ||
|
|
||
| ## Goal | ||
|
|
||
| Write the setup report summarizing what this integration did, drawing only on the | ||
| run's queue log and event plan in `.posthog-wizard-cache/` (`queue.json` and | ||
| `.posthog-events.json`), then mirror it into a shareable PostHog notebook. | ||
|
|
||
| ## How you know you succeeded | ||
|
|
||
| `posthog-setup-report.md` exists at the project root: what was installed and | ||
| initialized, the events captured, whether identify was wired or skipped, error | ||
| tracking added, the dashboard link, any build conflict in full, and the next | ||
| steps for the user. The report is also mirrored into a PostHog notebook whose URL | ||
| is emitted with the `[NOTEBOOK_URL]` marker. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| type: docs-only | ||
| template: description.md | ||
| description: Install dependencies and verify the project builds | ||
| tags: [orchestrator, build] | ||
| variants: | ||
| - id: all | ||
| display_name: PostHog build step | ||
| tags: [orchestrator, build] | ||
| docs_urls: [] | ||
| cli: | ||
| role: internal |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
might want to enforce this in an allowlist if not already
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good call. We can do this with pi agents (finally cries) I want to do another pass on this though as a separate PR. Noted