feat: add Email (AgentMail) as a communication channel - #1949
Open
mrubens wants to merge 34 commits into
Open
Conversation
Fifth communication channel alongside Slack, Teams, Telegram, and Discord. Per-deployment configuration (operator's own AgentMail API key), one deployment-owned inbox, inbound-initiated only in v1. - Provider adapter with DB-resolved reply routing (durable conversation rows carry split inbound/outbound anchors, tuple-ordered advancement) - Svix-verified webhook -> agentmail_webhook_events ingestion outbox -> BullMQ -> ordered per-conversation Fast-turn drain (agentmail_inbound_turns) - Deterministic sender/conversation resolution with forwarded-thread forks and a participant join table enforcing the one-conversation-per-user- per-thread invariant - Settings -> Communications reconcile flow: capability-checked key, proposed inbox address (deployment app name + host hash), idempotent inbox/webhook reconciliation with drift repair - Low-frequency cadence: ~2 emails per task, no heartbeats or reactions; email-specific harness instructions - Mock AgentMail harness (Svix-signed deliveries, duplicate/oversize/ auto-submitted simulation) + testing skill + docs
Question emails now render each option as a button-styled signed answer link (magic-link trust model; HMAC domain-separated from the deployment signing key, 7-day expiry). Clicking records the answer through the same atomic pending -> submitted transition the chat callbacks use, so double clicks and stale links resolve as 'already answered'. Free-text reply stays the fallback and multi-question prompts remain reply-per-line. - AgentMail adapter renders URL buttons as HTML anchors + plain-text list - publishCommunicationRequestUserInput gains an agentmail branch - GET /api/webhooks/agentmail/answer serves the signed-link claim page - Worker RUI rendering + answer polling enabled for email
Found in the mock smoke test: the drain runner admitted turns but the Fast surface-reply delivery builder had no agentmail branch, so turns consumed with 'could not resolve a delivery route' and no email was sent. Replies post through the durable conversation route; replaceReply keeps the original message since sent email is immutable.
Contributor
|
No new code issues found. See task
Reviewed fe9eaa7 |
Correctness: - Widen the task-runs router request_user_input guards to agentmail; the email RUI publish (question emails + answer buttons) was unreachable - Capture sender identity and body text on agentmail_inbound_turns at admission (incl. re-fetched oversize bodies); the drain no longer re-parses payloads or re-resolves senders, fixing silent drops on oversize mail and on identity drift between admission and drain - Sweep now also re-dispatches 'failed' webhook events up to an attempt cap, uses DB-side now() for staleness (timezone-safe), bounds batches, and uses a stable per-conversation sweep job id - One-click answer links: GET is now read-only and renders a confirm form; only the POST claims, so mail link scanners can no longer auto-answer questions - Authorize cc'd participants via the participant table in the Fast delivery gate; their in-thread emails were silently dropped - Only retry AgentMail 5xx responses on idempotent calls; give the RUI prompt and stranger refusal idempotency keys; thread-reply keys are per-call so identical texts both deliver - Webhook reconcile converges inbox_ids and uses a deployment-hashed client id (legacy id adopted); inbox changes re-scope the webhook - Email Fast sessions get an email surface name + cadence framing in the fast-agent prompt instead of automation-conversation wording - Provider errors now notify the email thread; invocation identities include the deployment inbox; provider-message bindings record email Cleanup: - Remove committed debug script q.tmp.mjs, unused agentmail_pending_inputs table and dead conversation columns, dead label branches, and the duplicated HTML escapers (single shared escapeAgentMailHtml)
- Import vitest globals in agentmail-buttons.test.ts (CI ran with globals disabled; the suite failed to load) - Widen fast_agent_provider_messages_provider_v3_check to accept 'agentmail' — the TS union was widened but the CHECK made every email provider-message insert fail (review comment) - Stable idempotency keys on the durable email retry paths: parent-event posts key on the event's client-message seed, Fast surface replies key on the inbound message id + post index, and MCP thread replies key on (run, text digest, inbound anchor) so a lost-response tool retry dedupes while identical texts in later turns still deliver (review comments) - CodeQL: replace the backtracking script/style strip with a linear scanner and the trailing-slash regexes with a loop
turbo's strict env mode stripped REDIS_URL from test task processes, so CI tests fell back to .env.test's local port and ioredis hung until the per-test timeout. DATABASE_URL was already passed through; REDIS_URL now matches. Surfaced by the agentmail one-click answer tests, the first api tests to exercise the real Redis-backed pending-input store in CI.
- Append the reply anchor to the Idempotency-Key inside the adapter, where the route was just resolved, instead of a separate caller-side lookup that could race a concurrent inbound email - Keep a delimiter where script/style blocks are removed from HTML-only bodies so neighboring text does not merge
- The worker mints a clientSendId per send_chat_reply invocation and every HTTP retry of that call carries it; the agentmail thread-reply Idempotency-Key uses it, so the logical send's identity no longer depends on mutable route state (adapter anchor-append reverted, text digest kept only as a legacy fallback) - The script/style block stripper requires a tag-name boundary, so <scripture> is no longer swallowed as <script>
A literal </script> inside the opening tag's attribute values terminated the block early and leaked script content into the fallback body text.
Saving a valid key failed with 'AgentMail rejected this API key' because the reconcile ignored the inbox the org already had and tried to create a second one; the 403 from that create was then misclassified as a key rejection. Now: exactly one existing inbox is adopted, several prompt the operator to choose (listing the addresses), and permission errors name the operation that was refused instead of blaming the key.
Setup note, docs, and every permission-refusal error now list the exact permission set the channel needs (inbox_read/create, webhook_read/ create/update/delete, message_read/send), and validation probes webhook access up front since default console keys most often lack it.
Live testing against real AgentMail surfaced that Idempotency-Key only allows A-Za-z0-9-._~ while our logical keys are colon-delimited (and can embed RFC822 message ids): every keyed send 400'd. The header is now the hex digest of the logical key at the single request choke point, and the mock enforces the real charset so this class of parity bug fails tests. Also release the once-per-thread stranger-refusal claim when the send fails, so the sender still gets a refusal on their next email.
The subject often carries the actual request ('what time is it' over a
one-word body); Fast questions and follow-up queue text now prepend
'Subject: ...' from the conversation. The raw body stays separate for
request_user_input answer parsing, where a prepended subject would
corrupt option matching. Also allow subject-only emails with empty
bodies to start a turn.
Stranger refusals now include a signed 'Link this address to my Roomote account' button. The link opens an authenticated confirm page; linking writes the address mapping for the signed-in user and re-dispatches the sender's recent refused emails so the original request is processed without a resend. Possession of the mailbox (the token was delivered there) plus the signed-in session are the two factors; the atomic mapping insert makes conflicts and double clicks safe.
Once an API key is available, the settings section lists the org's existing inboxes in a select alongside 'Create new: <proposed address>' and an enter-manually escape hatch for custom-domain inboxes; choosing the proposed address creates it during the reconcile. Saved keys auto-load the list; typed keys load on demand so AgentMail is never queried per keystroke.
Web-initiated turns into an email conversation carry no unique inbound message id, so the key degenerated to a constant and AgentMail rejected every later reply with an idempotency conflict (same key, different body). The key now mixes in a digest of the composed message: distinct replies can never collide while true retries of one send still dedupe. Same treatment for the parent-event key, whose composed text can also vary across retries of one event.
Recipients see the inbox display name as the sender; AgentMail's default made replies read as 'AgentMail <address>'. The reconcile now sets the display name on adopted inboxes (best effort, never blocking the save) and new inboxes are created with it.
The shared provider-setup step offers the Roomote logo download for bot avatars; email has no avatar to set, so the line is hidden there. The AgentMail console links now point at the api-keys page directly, and the docs paragraph the permissions note had split is rejoined.
…ession Roomote can now initiate email through a single consent-checked entry point (startAgentMailConversation): only the recipient's verified account email or an explicitly linked address, never a suppressed one. Sends carry RFC 8058 one-click List-Unsubscribe headers, and every outbound-initiated email records a conversation so replies thread back into the normal pipeline. - agentmail_suppressions table; permanent bounces and spam complaints auto-suppress via new message.bounced / message.complained webhook events (reconcile converges event types on existing webhooks) - signed unsubscribe tokens + GET confirm / POST claim endpoint at /api/webhooks/agentmail/unsubscribe (one-click POST and footer link) - direct messages over email: sendUserDirectMessage agentmail arm; best-effort broadcasts use email as fallback reach only when no chat provider delivered - mock harness: bounce/complaint injection kinds with real payload shapes
…laim release - Quote-aware opening-tag scan in the HTML stripper: a '>' inside a quoted attribute no longer terminates the tag, so an embedded '</script>' in a later attribute cannot leak script content - AgentMailApiClient.listInboxes follows next_page_token pagination so a many-inbox account is never mistaken for a one-inbox account at setup - Typed AgentMailApiError (status) from the API client; the stranger-refusal once-per-thread claim is now released only on definite 4xx rejections and kept on ambiguous 5xx/network failures (no duplicate refusals) - Reconcile probes message_read via a sentinel-message fetch (403 vs 404) and prefers the inbox 'email' field over 'inbox_id' when adopting; inbox_update added to the documented required permissions; docs note that message_send has no side-effect-free probe - listAgentMailInboxes is a mutation so a freshly typed API key travels in the POST body instead of a GET URL (browser history, proxy logs) - startAgentMailConversation sends with an idempotency key so the client's internal retries replay the accepted send instead of emailing twice
Only the expected 404 proves the permission; a network error or timeout previously slipped through the typed-error check and persisted an unvalidated key.
inbox_id is the key the AgentMail API contract requires in request paths and webhook inbox_ids filters, so it must win over the display email if the fields ever diverge; preferring email traded a display quirk for broken routing.
The persisted value stays inbox_id (API paths, webhook scoping); every operator-facing surface now shows the deliverable email resolved live from AgentMail — settings status, the save toast, the inbox chooser (id as the submitted value, email as the label), and the multi-inbox error. Derived at read time rather than persisted, so it can never drift from AgentMail's record.
createProposedAgentMailInbox now returns the created inbox's email alongside the routing address, so the save result and success toast show the deliverable address for newly provisioned inboxes too.
A saved-and-connected config no longer fetches the AgentMail inbox list every time settings render — the status block already names the inbox, so the list loads only when the operator opens 'Choose from account inboxes' (or during fresh setup). The chooser's mutationFn also calls the tRPC HTTP endpoint directly: mutation promises through the client's link stack were observed hanging without settling in the browser while the request completed server-side, and a direct call is sturdier until that is root-caused.
With loading now on demand (click-triggered), the mutation resolves normally; the hand-rolled fetch bypass is no longer needed and the chooser matches every other mutation in the app. The mount-time hang that prompted the bypass is tracked separately.
…nnel # Conflicts: # packages/sdk/src/server/automations/custom-automations.ts
…t email verification The channel is off unless R_EMAIL_CHANNEL_ENABLED=true: the provider is absent from settings and refuses saves, inbound webhook deliveries are acknowledged and dropped, and no outbound email is sent. Self-hosted operators set the env var; Roomote Cloud sets it per deployment. Enabling the channel gives the deployment an email sender for the first time, so it also turns on Better Auth email verification: sign-ups and unverified sign-ins get a verification email, password resets are emailed as well as captured for the admin UI, and requireEmailVerification is on for password accounts. Account emails go through a narrower system-email sender: no unsubscribe header or link, bounce/complaint suppressions only (an unsubscribed address can still verify or reset), and no conversation record. The SDK exposes it via the server/agentmail-outbound subpath so auth does not import the whole server barrel.
The shared reply-path provider factory now returns null while the channel is disabled, so Fast surface replies, parent events, finish-run result emails, and MCP thread replies for already-admitted conversations stop too, not just new Roomote-initiated conversations. In-flight webhook processing and turn drains are left untouched while disabled (rows stay queued and the recovery sweep re-dispatches them on re-enable), so nothing is refused or replied to while off and no email is lost.
Cloud clears R_EMAIL_CHANNEL_ENABLED and the R_AGENTMAIL_* variables with empty strings when a tenant leaves the managed-email allowlist (Railway upserts never delete). An empty enum flag must fall back to its default instead of failing boot validation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds email as a fifth communication channel alongside Slack, Teams, Telegram, and Discord, built on AgentMail. Users email a deployment-owned inbox; replies arrive in the same thread. Email is treated as its own addressing and security model, not a chat channel with HTML formatting.
Design
Per-deployment configuration: each deployment brings its own AgentMail API key (Settings → Communications). On save, Roomote proposes an inbox address derived from the deployment hostname, creates/adopts it idempotently, and registers a webhook for
message.received,message.bounced, andmessage.complainedevents — the save flow is a reconcile that converges the webhook URL, inbox scoping, and event types, repairs drift, and heals partial failures on re-save. Webhook client ids are deployment-hashed so deployments sharing one AgentMail account cannot interfere with each other.Durable end to end: Svix-verified webhook →
agentmail_webhook_eventsingestion outbox (received → queued → processing → processed | failed; duplicates re-dispatch stranded rows, failed events are re-swept up to an attempt cap before becoming dead letters) → BullMQ →agentmail_inbound_turnsadmission (sender identity and body text — including re-fetched oversize bodies — are captured at admission) → per-conversation drain ordered by(provider_timestamp, message_id)into a Fast conversation. A crash at any point leaves a durable row, never a lost email.Email-specific security model: senders resolve against verified account emails only; conversations carry a participant table whose unique
(inbox, thread, user)index makes resolution deterministic and settles first-contact races; cc'd verified users join as participants and are authorized through the participant table; forwarded threads fork into isolated conversations (forwarding never grants access to an existing task); strangers get one refusal per thread with auto-responder loop protection.Reply routing state: conversations store split inbound/outbound anchors (tuple-ordered, version-guarded) plus the latest sender; the adapter resolves the reply anchor and recipient from the row at send time and never replies-all. 5xx retries are gated on idempotency keys so a failure after a successful send can never duplicate an email.
Low-frequency cadence: ~2 emails per task (ack + result), no heartbeats/reactions/play-by-play, enforced by email-specific instructions for both Fast conversations and delegated tasks.
request_user_input over email: question emails render options as signed answer links (magic-link trust). Clicking opens a one-tap confirmation page and only the confirming POST records the answer — mail link scanners that prefetch every URL cannot answer on the user's behalf. The claim is atomic, so double submits resolve as "already answered." Free-text reply remains the fallback; re-publishes are idempotent so a worker restart cannot send duplicate question emails.
Roomote-initiated (transactional) email: a single consent-checked entry point (
startAgentMailConversation) is the only way Roomote initiates email. It sends only to the recipient's own verified account address or an address they explicitly linked (mailbox-possession proven), never to a suppressed address, and records a conversation so replies thread straight back into the normal inbound pipeline — every transactional email is answerable. Direct messages use it (sendUserDirectMessage), and best-effort broadcasts treat email as fallback reach only when no chat provider delivered, preserving the low-frequency cadence.Suppression + unsubscribe: permanent bounces and spam complaints auto-suppress recipients via the new webhook events (transient bounces do not); every Roomote-initiated email carries RFC 8058 one-click
List-Unsubscribeheaders plus a footer link backed by signed tokens and a read-only-GET/confirming-POST endpoint (same scanner defense as answer links). Suppression only gates initiating email — replying inside an existing conversation is never suppressed.Still out of scope: channel discovery and automation destinations (typed out via
AutomationCapableCommunicationProvider), channel-style posting by agents.Account email verification: enabling the channel gives a deployment an email sender for the first time (password resets previously only surfaced a link in the admin UI, and email-and-password sign-ups were never verifiable), so it also turns on Better Auth email verification — sign-ups and unverified sign-ins get a verification email, password resets are emailed as well as captured, and
requireEmailVerificationis on for password accounts. This is what backs the consent rule above: Roomote only initiates email to an address it has verified. Account emails use a narrower system-email sender (sendAgentMailSystemEmail): no unsubscribe header or link, bounce/complaint suppressions only (an unsubscribed address can still verify or reset), no conversation record.Rollout
The channel is gated behind
R_EMAIL_CHANNEL_ENABLED=true. Without it the provider is absent from settings and refuses saves, inbound webhook deliveries are acknowledged and dropped, and no outbound email is sent — so a still-registered webhook is harmless and disabling the flag is a complete kill switch. Self-hosted operators set the env var; Roomote Cloud sets it per deployment. Schema is additive plus surface CHECK widening only (N-1 rollback safe; no drops).Testing
svixlib), answer confirm-then-claim flow (including the read-only-GET scanner defense and atomic double-submit), formatter escaping, webhook reconcile (inbox re-scoping, legacy-id adoption, cross-deployment isolation), mock-server contract, outbound consent resolution (account-email preference, suppression fallback and refusal), bounce/complaint suppression processing, unsubscribe token domain-separation and endpoint semantics — plus full suites green across sdk/api/worker/communication/web.pnpm --filter @roomote/communication mock:agentmailwith Svix-signed deliveries and duplicate/oversize/auto-submitted/bounce/complaint simulation, documented in themock-agentmail-testingskill.Docs: new
providers/communications/agentmailpage plus channel-table/env-var updates.