Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
070a3e1
feat: add Email (AgentMail) as a communication channel
mrubens Aug 31, 2026
377a8a3
feat: one-click answer buttons for request_user_input over email
mrubens Aug 31, 2026
1e1b047
docs: describe one-click answer buttons for email questions
mrubens Aug 31, 2026
2c515f8
fix: add agentmail delivery branch to buildFastAgentSurfaceReplyDelivery
mrubens Aug 31, 2026
58aa947
fix: address code-review findings for the AgentMail channel
mrubens Aug 31, 2026
16e35f9
docs: describe the answer-button confirmation step
mrubens Aug 31, 2026
299a65a
fix: CI test failure and PR review feedback
mrubens Aug 31, 2026
4fa3e57
fix: pass REDIS_URL through turbo so CI tests reach the Redis service
mrubens Aug 31, 2026
22fe93d
fix: address follow-up review nits
mrubens Aug 31, 2026
7aa921d
fix: retry-stable email send identity and script/style tag boundaries
mrubens Aug 31, 2026
80c7a48
fix: scan for the block close tag only after the opening tag ends
mrubens Aug 31, 2026
1c62b52
fix: adopt the org's existing inbox and name the failing setup step
mrubens Aug 31, 2026
ab7662b
feat: name the AgentMail key permissions everywhere they matter
mrubens Aug 31, 2026
20fa062
fix: real-API Idempotency-Key charset and refusal-claim release
mrubens Aug 31, 2026
0940f2a
fix: include the email subject in agent-visible turn text
mrubens Aug 31, 2026
e5452cd
feat: link an email address from the refusal email
mrubens Aug 31, 2026
54fc4c8
feat: inbox chooser in AgentMail comms settings
mrubens Aug 31, 2026
fbdb9c5
fix: content digest in Fast email reply idempotency keys
mrubens Aug 31, 2026
bae3c05
feat: converge the inbox display name to Roomote
mrubens Aug 31, 2026
2c8b591
fix: drop the logo step for email setup and link the api-keys page
mrubens Aug 31, 2026
4ea6115
merge: origin/develop into feat/agentmail-channel
mrubens Sep 1, 2026
45c3c77
Add outbound (transactional) email with consent enforcement and suppr…
mrubens Sep 1, 2026
a4f9488
Address review: probe gaps, secret-in-URL, pagination, idempotency, c…
mrubens Sep 1, 2026
b2fc54a
Fail the save when the message_read probe cannot complete
mrubens Sep 1, 2026
061210b
Re-trigger checks after review replies
mrubens Sep 1, 2026
4ba03de
Route by inbox_id, not the email field
mrubens Sep 1, 2026
7d3ed13
Surface the deliverable email separately from the routed inbox id
mrubens Sep 1, 2026
103e403
Carry the deliverable email through inbox creation
mrubens Sep 1, 2026
448e3ad
Load the inbox chooser on demand, not on page load
mrubens Sep 1, 2026
0f1b41d
Use the standard tRPC mutation for the inbox chooser
mrubens Sep 1, 2026
7b0ccd3
Merge remote-tracking branch 'origin/develop' into feat/agentmail-cha…
mrubens Sep 1, 2026
ee035dd
Gate the email channel behind R_EMAIL_CHANNEL_ENABLED and wire accoun…
mrubens Sep 1, 2026
8974e3a
Make R_EMAIL_CHANNEL_ENABLED a complete kill switch
mrubens Sep 1, 2026
fe9eaa7
Treat empty managed-email variables as unset
mrubens Sep 1, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
182 changes: 182 additions & 0 deletions .agents/skills/mock-agentmail-testing/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,182 @@
---
name: mock-agentmail-testing
description: Run Roomote email (AgentMail) integration flows through the checked-in mock AgentMail API harness instead of the real AgentMail service. Use when testing email task entry, inbox provisioning, webhook registration, Svix-signed `message.received` deliveries, duplicate/oversize/auto-submitted email handling, outbound email replies, `AGENTMAIL_API_BASE_URL` routing, `/mock/state`, or `/mock/events`.
---

# Mock AgentMail Testing

Use this skill to exercise Roomote's email integration against the checked-in mock AgentMail harness. Do not invent another fake AgentMail stack and do not use the real AgentMail service unless the user explicitly asks for that parity test.

Email continuity is inferred from **thread id → conversation**. The harness mints reply headers (`in_reply_to`, `references`) the way a real mail chain would, and signs every webhook delivery exactly like Svix does, so the production verifier accepts mock deliveries unchanged. Webhook signature verification, thread continuity, and duplicate-delivery dedup are the highest-value things to test here.

## Quick Reference

| What | Value |
| ------------------------------ | ------------------------------------------------------------ |
| Harness port | `3015` |
| Harness base URL | `http://127.0.0.1:3015` |
| AgentMail API base for Roomote | `AGENTMAIL_API_BASE_URL=http://127.0.0.1:3015` |
| API webhook endpoint | `http://localhost:3001/api/webhooks/agentmail` |
| Mock state endpoint | `http://127.0.0.1:3015/mock/state` |
| Mock event replay endpoint | `http://127.0.0.1:3015/mock/events` |
| Example scenario | `packages/communication/scripts/mock-agentmail.example.json` |
| Mock inbox identity | `roomote@agentmail.to` |
| Seeded webhook secret | `whsec_...` from the scenario file (or minted at register) |

## Step 1: Wire the API server env

Set these in the API server's environment (or `.env.local`) before it starts:

```bash
R_AGENTMAIL_API_KEY=mock-agentmail-api-key # any value; the harness accepts all bearer tokens unless acceptedApiKeys is set
AGENTMAIL_API_BASE_URL=http://127.0.0.1:3015 # reroutes ALL outbound AgentMail API calls to the harness
```

Webhook secrets need no manual wiring: when the app registers its webhook through `POST /v0/webhooks`, the harness mints the `whsec_...` secret and returns it, exactly like real AgentMail. If the app relies on a pre-provisioned secret (`R_AGENTMAIL_WEBHOOK_SECRET`), seed a webhook with that secret in the scenario file instead — deliveries are signed with whatever secret the registration holds.

## Step 2: Create a scenario file

Copy the example and fix the webhook target to point at the sandbox API (port 3001, not 4000):

```bash
cp packages/communication/scripts/mock-agentmail.example.json /tmp/mock-agentmail-test.json
sed -i '' 's|localhost:4000|localhost:3001|g' /tmp/mock-agentmail-test.json # macOS; drop '' on Linux
grep url /tmp/mock-agentmail-test.json
# Should show: "url": "http://localhost:3001/api/webhooks/agentmail"
```

For custom scenarios, edit `/tmp/mock-agentmail-test.json` directly. Never mutate the committed example.

## Step 3: Start the harness

```bash
pnpm --filter @roomote/communication mock:agentmail --state /tmp/mock-agentmail-test.json
```

The harness starts on port 3015, replays any events in the `replay` array (delivering signed webhooks to every matching registration), and keeps listening. For one-shot replay that exits after: add `--exit-after-replay`.

## Step 4: Inject inbound emails manually (optional)

Ids (`msg_*`, `thread_*`, `evt_*`, svix delivery ids) are minted automatically and unique per run. Pass `threadId` to continue an existing thread; omit it to start a fresh one.

```bash
# New email → new thread, signed message.received delivery
curl -s -X POST http://127.0.0.1:3015/mock/events \
-H 'Content-Type: application/json' \
-d '{
"inboxId": "roomote@agentmail.to",
"from": "grace@example.com",
"subject": "Flaky login test",
"text": "Hi Roomote — can you look into the flaky login test?"
}'

# Follow-up in the same thread (use threadId from the previous dispatchResult)
curl -s -X POST http://127.0.0.1:3015/mock/events \
-H 'Content-Type: application/json' \
-d '{
"inboxId": "roomote@agentmail.to",
"from": "grace@example.com",
"text": "also check the retry logic please",
"threadId": "<threadId>"
}'

# Auto-generated sender (adds the Auto-Submitted header — loop-guard scenarios)
curl -s -X POST http://127.0.0.1:3015/mock/events \
-H 'Content-Type: application/json' \
-d '{
"inboxId": "roomote@agentmail.to",
"from": "noreply@example.com",
"text": "Your build failed.",
"autoSubmitted": true
}'

# Oversize payload: webhook arrives WITHOUT text/html (1MB cap); the app must
# re-fetch the full message via GET /v0/inboxes/{id}/messages/{message_id}
curl -s -X POST http://127.0.0.1:3015/mock/events \
-H 'Content-Type: application/json' \
-d '{
"inboxId": "roomote@agentmail.to",
"from": "grace@example.com",
"subject": "Huge recap",
"text": "pretend this is 2MB of text",
"oversize": true
}'

# Duplicate delivery: resends the PREVIOUS event verbatim with the SAME
# svix-id → exactly-once handling
curl -s -X POST http://127.0.0.1:3015/mock/events \
-H 'Content-Type: application/json' \
-d '{ "inboxId": "roomote@agentmail.to", "from": "grace@example.com", "duplicate": true }'

# Redeliver any past event by id (same svix-id, fresh timestamp + signature)
curl -s -X POST http://127.0.0.1:3015/mock/events \
-H 'Content-Type: application/json' \
-d '{ "kind": "redeliver", "eventId": "<eventId>" }'

# Permanent bounce → the app must suppress the recipient (message.bounced).
# bounceType defaults to "Permanent"; pass "Transient" to assert NO suppression.
curl -s -X POST http://127.0.0.1:3015/mock/events \
-H 'Content-Type: application/json' \
-d '{ "kind": "bounce", "inboxId": "roomote@agentmail.to", "recipients": ["gone@example.com"] }'

# Spam complaint → the app must suppress the recipient (message.complained)
curl -s -X POST http://127.0.0.1:3015/mock/events \
-H 'Content-Type: application/json' \
-d '{ "kind": "complaint", "inboxId": "roomote@agentmail.to", "recipients": ["angry@example.com"] }'
```

Every response carries `dispatchResult` with `eventId`, `svixId`, `messageId`, `threadId`, and per-webhook `deliveries` (status + body from the Roomote endpoint).

## Step 5: Inspect results

Always check the mock state after replay — do not declare success just because the harness returned 200:

```bash
# Full state dump
curl -s http://127.0.0.1:3015/mock/state | jq .

# Outbound emails the system under test sent (replies + fresh sends)
curl -s http://127.0.0.1:3015/mock/state | jq '.messages[] | select(.direction == "outbound")'

# Replies threaded onto the inbound email (email continuity)
curl -s http://127.0.0.1:3015/mock/state | jq '.messages[] | select(.direction == "outbound" and .in_reply_to != null)'

# Webhook registrations the app created (secret, inbox filter, event filter)
curl -s http://127.0.0.1:3015/mock/state | jq '.webhooks'

# Delivery log per event (status of every webhook POST, including retries)
curl -s http://127.0.0.1:3015/mock/state | jq '.events[] | {event_id, svix_id, deliveries}'
```

To reset between scenarios, `POST /mock/state` with a fresh state object (it replaces inboxes, webhooks, messages, and events wholesale).

## Scenario Selection

- **`email-task-entry`** — new inbound email creates a task; assert an outbound reply lands in the same thread
- **`followup-to-active-thread`** — second email with the same `threadId` queues to the running job instead of launching a new task
- **`duplicate-delivery`** — `duplicate: true` → same svix-id twice → exactly-once handling
- **`oversize-payload`** — `oversize: true` → app must re-fetch the message body by id before acting
- **`auto-submitted-loop-guard`** — `autoSubmitted: true` → automated senders must not trigger reply loops
- **`webhook-registration`** — app boots, registers its webhook via `POST /v0/webhooks` (idempotent per `client_id`), and the secret round-trips into signature verification
- **`reply-idempotency`** — app retries a reply with the same `Idempotency-Key` → exactly one outbound message in `/mock/state`
- **`bounce-suppression`** — `kind: 'bounce'` (Permanent) / `kind: 'complaint'` → the recipient lands in `agentmail_suppressions` and outbound-initiated email to them is refused; `bounceType: 'Transient'` must NOT suppress

## Guardrails

- Do not create a second mock AgentMail server. Use the harness in `packages/communication/`.
- Do not use the real AgentMail service unless the user explicitly asks for that.
- Do not declare success because the harness started. Always inspect `/mock/state`.
- Do not mutate the committed example scenario — copy it to `/tmp/` first.
- Do not assume the example webhook target port is correct. The sandbox API runs on 3001.
- Do not hand-roll webhook signatures in test drivers — deliver through `/mock/events` so the svix-id bookkeeping (and duplicate semantics) stays correct.
- Do not claim duplicate handling is covered unless you observed the second delivery being dropped (exactly one task/reply) in `/mock/state` and the app's own state.

## Output Standard

End each use of this skill with:

- the scenario used and the webhook target
- the inbound emails injected (if any), including flags (`duplicate`, `oversize`, `autoSubmitted`)
- the key outbound messages, webhook registrations, or delivery statuses observed in `/mock/state`
- a pass or fail judgment
- the next debugging lead if the behavior failed
1 change: 1 addition & 0 deletions apps/api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@
"jsdom": "26.1.0",
"p-map": "^7.0.4",
"snowflake-sdk": "^2.4.3",
"svix": "^1.99.1",
"undici": "^7.29.0",
"zod": "^3.25.76"
},
Expand Down
175 changes: 175 additions & 0 deletions apps/api/src/handlers/agentmail/__tests__/rui-answer.test.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading