Skip to content

Commit 27e4632

Browse files
BillLeoutsakosvl346Bill LeoutsakosBill Leoutsakosicecrasher321
authored
improvement(selectors): unify server-side selector execution (#7185)
* feat(selectors): execute dynamic selectors server-side * fix(selectors): expose route verb to contract audit * test(selectors): align migration expectations * fix(selectors): address review findings * fix(selectors): harden exact reference handling * fix(selectors): address Cubic review findings * fix(selectors): address Cubic rerun findings * fix(selectors): close final Cubic review gaps * fix(selectors): close final review follow-ups * fix(imap): restore deployment binding on rollback * fix(imap): scope webhook updates to active deployment * test(selectors): consolidate migration regression coverage * refactor(selectors): isolate unified selector contract * fix(selectors): restore migration parity and egress guarantees * fix(selectors): revalidate resolved inputs and preserve safe errors * fix: bound server selector collection * feat(selectors): load provider options on demand * fix(selectors): recover expired pagination cursors * fix(selectors): address review findings * fix(selectors): preserve detail hydration and refresh state * fix(selectors): hydrate capped provider details directly --------- Co-authored-by: Bill Leoutsakos <billleoutsakos@Bills-MacBook-Pro.local> Co-authored-by: Bill Leoutsakos <billleoutsakos@Mac.localdomain> Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
1 parent afccb72 commit 27e4632

362 files changed

Lines changed: 18609 additions & 20864 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.agents/skills/add-block/SKILL.md

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,7 @@ silently available.
207207
id: 'channel',
208208
title: 'Channel',
209209
type: 'channel-selector',
210+
selectorKey: '{service}.channels',
210211
serviceId: '{service}',
211212
placeholder: 'Select channel',
212213
dependsOn: ['credential'],
@@ -217,6 +218,7 @@ silently available.
217218
id: 'project',
218219
title: 'Project',
219220
type: 'project-selector',
221+
selectorKey: '{service}.projects',
220222
serviceId: '{service}',
221223
dependsOn: ['credential'],
222224
}
@@ -226,6 +228,7 @@ silently available.
226228
id: 'file',
227229
title: 'File',
228230
type: 'file-selector',
231+
selectorKey: '{service}.files',
229232
serviceId: '{service}',
230233
mimeType: 'application/pdf',
231234
dependsOn: ['credential'],
@@ -236,6 +239,7 @@ silently available.
236239
id: 'user',
237240
title: 'User',
238241
type: 'user-selector',
242+
selectorKey: '{service}.users',
239243
serviceId: '{service}',
240244
dependsOn: ['credential'],
241245
}
@@ -1065,7 +1069,11 @@ After creating the block, you MUST validate it against every tool it references:
10651069

10661070
A sub-block gets its choices from exactly one of two places. There is no third.
10671071

1068-
**`selectorKey` — every remote list.** Register the list in `hooks/selectors/providers/<service>/selectors.ts`, add its key to `SelectorKey`, and point the sub-block at it. A selector is parameterized by an explicit `SelectorContext`, so the same definition serves the canvas, the workspace-fork sync modal, and anything added later.
1072+
**`selectorKey` — every remote list.** Use the `add-selector` skill to add browser-safe metadata in
1073+
`apps/sim/lib/selectors/manifest.ts`. Attach `provider-server` selectors under
1074+
`apps/sim/lib/selectors/server/providers/` and `internal-server` selectors in
1075+
`apps/sim/lib/selectors/server/internal.ts`. Point the sub-block at that key. All remote selectors
1076+
execute through `selectors.execute`; never add a client provider module or selector-only fetch route.
10691077

10701078
```ts
10711079
{ id: 'triggerCredentials', type: 'oauth-input', canonicalParamId: 'oauthCredential', mode: 'trigger' },
@@ -1074,7 +1082,13 @@ A sub-block gets its choices from exactly one of two places. There is no third.
10741082
{ id: 'manualLabelIds', type: 'short-input', mode: 'trigger-advanced' },
10751083
```
10761084

1077-
`canonicalParamId: 'oauthCredential'` on the credential sub-block is the line people forget. `buildSelectorContextFromBlock` keys the context on a sub-block's CANONICAL id, so without it `context.oauthCredential` is never set and the picker looks unfixable without reading the store. (A credential field is also recognised by its `oauth-input` TYPE as a fallback, so a block whose shipped param is already named something else does not have to rename it.)
1085+
`canonicalParamId: 'oauthCredential'` on the credential sub-block is the line people forget. The
1086+
shared context builder projects only active `dependsOn` values and keys canonical pairs by their
1087+
canonical id. Exact environment references such as `{{GMAIL_CREDENTIAL_ID}}` stay unresolved in the
1088+
browser and are resolved only by the authorized server executor. The builder does not infer a
1089+
nonstandard credential id from `type: 'oauth-input'`; give it
1090+
`canonicalParamId: 'oauthCredential'`, or declare an explicit manifest `sourceFields` alias when a
1091+
legacy source id must be retained.
10781092

10791093
**`options` — everything else.** A static array, or a pure function of the block's own values for a list that narrows to a sibling's selection. No I/O.
10801094

@@ -1089,5 +1103,6 @@ options: (params) => {
10891103

10901104
Two rules the checks enforce:
10911105

1092-
- **A secret never enters a selector's `getQueryKey`.** A query key identifies a resource; a credential authorizes access to it. A credential *id* is fine; a typed password is not (see `imap.mailboxes`).
1106+
- **Selector query keys contain no context values.** This includes credential IDs, raw secrets,
1107+
unresolved references, and hashes of those values; the shared facade uses an opaque local revision.
10931108
- **A sub-block that `dependsOn` a credential / knowledge-base / table selector must be reconfigurable at fork-sync time** — a `selectorKey`, a canonical pair whose basic member is a selector, or a `short-input`/`long-input`. `bun run check:fork-dependent-coverage` fails otherwise, because a fork sync clears those fields on every push and an unofferable one can never be set anywhere that sticks.

.agents/skills/add-connector/SKILL.md

Lines changed: 25 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -197,7 +197,14 @@ Three field types are supported: `short-input`, `dropdown`, and `selector`.
197197

198198
## Dynamic Selectors (Canonical Pairs)
199199

200-
Use `type: 'selector'` to fetch options dynamically from the existing selector registry (`hooks/selectors/registry.ts`). Selectors are always paired with a manual fallback input using the **canonical pair** pattern — a `selector` field (basic mode) and a `short-input` field (advanced mode) linked by `canonicalParamId`.
200+
Use `type: 'selector'` for a key declared in the browser-safe selector manifest at
201+
`apps/sim/lib/selectors/manifest.ts`. Remote selectors execute through the authorized
202+
`selectors.execute` server operation and a server attachment; connectors never call providers or
203+
resolve credentials in the browser. Apply the `add-selector` skill when the key does not exist.
204+
205+
Selectors are paired with a manual fallback input using the **canonical pair** pattern — a
206+
`selector` field (basic mode) and a `short-input` field (advanced mode) linked by
207+
`canonicalParamId`.
201208

202209
The user sees a toggle button (ArrowLeftRight) to switch between the selector dropdown and manual text input. On submit, the modal resolves each canonical pair to the active mode's value, keyed by `canonicalParamId`.
203210

@@ -217,7 +224,7 @@ configFields: [
217224
id: 'baseSelector',
218225
title: 'Base',
219226
type: 'selector',
220-
selectorKey: 'airtable.bases', // Must exist in hooks/selectors/registry.ts
227+
selectorKey: 'airtable.bases', // Must exist in lib/selectors/manifest.ts
221228
canonicalParamId: 'baseId',
222229
mode: 'basic',
223230
placeholder: 'Select a base',
@@ -260,7 +267,9 @@ configFields: [
260267

261268
### Selector with domain dependency (Jira/Confluence pattern)
262269

263-
When a selector depends on a plain `short-input` field (no canonical pair), `dependsOn` references that field's `id` directly. The `domain` field's value maps to `SelectorContext.domain` automatically via `SELECTOR_CONTEXT_FIELDS`.
270+
When a selector depends on a plain `short-input` field (no canonical pair), `dependsOn` references
271+
that field's `id` directly. Exact references such as `{{JIRA_DOMAIN}}` remain unresolved in the
272+
browser and are resolved only after workspace authorization on the server.
264273

265274
```typescript
266275
configFields: [
@@ -296,16 +305,16 @@ configFields: [
296305

297306
### How `dependsOn` maps to `SelectorContext`
298307

299-
The connector selector field builds a `SelectorContext` from dependency values. For the mapping to work, each dependency's `canonicalParamId` (or field `id` for non-canonical fields) must exist in `SELECTOR_CONTEXT_FIELDS` (`lib/workflows/subblocks/context.ts`):
300-
301-
```
302-
oauthCredential, domain, teamId, projectId, knowledgeBaseId, planId,
303-
siteId, collectionId, spreadsheetId, fileId, baseId, datasetId, serviceDeskId
304-
```
308+
The shared connector context builder projects only active dependencies. A canonical dependency uses
309+
its active basic or advanced value under `canonicalParamId`; a non-canonical dependency uses its
310+
field `id`. The resulting key must be a `SelectorContextKey` in
311+
`apps/sim/lib/selectors/types.ts` and must be explicitly allowed by that selector's manifest entry.
312+
The browser sends the connector's workspace scope, not the complete connector configuration.
305313

306314
### Available selector keys
307315

308-
Check `hooks/selectors/types.ts` for the full `SelectorKey` union. Common ones for connectors:
316+
Check `apps/sim/lib/selectors/manifest.ts` for the exhaustive selector keys. Common ones for
317+
connectors:
309318

310319
| SelectorKey | Context Deps | Returns |
311320
|-------------|-------------|---------|
@@ -607,9 +616,13 @@ export const CONNECTOR_META_REGISTRY: ConnectorMetaRegistry = {
607616
- [ ] **Selector fields configured correctly (if applicable):**
608617
- Every `type: 'selector'` field has a canonical pair (`short-input` or `dropdown` with same `canonicalParamId` and `mode: 'advanced'`)
609618
- `required` is identical on both fields in each canonical pair
610-
- `selectorKey` exists in `hooks/selectors/registry.ts`
619+
- `selectorKey` exists in `apps/sim/lib/selectors/manifest.ts`
611620
- `dependsOn` references selector field IDs (not `canonicalParamId`)
612-
- Dependency `canonicalParamId` values exist in `SELECTOR_CONTEXT_FIELDS`
621+
- Each projected dependency key is a `SelectorContextKey` allowed by the selector manifest
622+
- Every remote key has one server attachment with credential provider binding and a reviewed
623+
`fixed`, `credential-bound`, or `user-controlled` destination policy
624+
- No connector selector adds a client provider module, browser token request, or selector-only
625+
API route
613626
- [ ] `listDocuments` handles pagination with metadata-based content hashes
614627
- [ ] `syncContext.listingCapped = true` set whenever the listing is truncated (max-items cap or transient per-item error) — required to prevent the engine's deletion reconciliation from removing unseen documents
615628
- [ ] `contentDeferred: true` used if content requires per-doc API calls (file download, export, blocks fetch)

.agents/skills/add-integration/SKILL.md

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -270,15 +270,24 @@ export const {Service}Block: BlockConfig = {
270270
{
271271
id: 'project',
272272
type: 'project-selector',
273+
selectorKey: '{service}.projects',
273274
dependsOn: ['credential'],
274275
},
275276
{
276277
id: 'issue',
277278
type: 'file-selector',
279+
selectorKey: '{service}.issues',
278280
dependsOn: ['credential', 'project'],
279281
}
280282
```
281283

284+
Every remote `selectorKey` must use the unified server selector path. Apply the `add-selector` skill:
285+
add browser-safe metadata to `apps/sim/lib/selectors/manifest.ts`, reuse or extract a server-only
286+
provider listing primitive, and add a credential- and destination-bound server attachment. Do not
287+
add code under `hooks/selectors/providers`, a provider-specific query key, browser token acquisition,
288+
or a selector-only API route. The shared context builder sends only active `dependsOn` values and
289+
preserves exact `{{KEY}}` environment references for server-side resolution.
290+
282291
**Basic/Advanced mode for dual UX:**
283292
```typescript
284293
// Basic: Visual selector
@@ -630,6 +639,10 @@ If creating V2 versions (API-aligned outputs):
630639
- [ ] Added credential field with `requiredScopes: getScopesForService('{service}')`
631640
- [ ] Added conditional fields per operation
632641
- [ ] Set up dependsOn for cascading selectors
642+
- [ ] Every remote `selectorKey` exists in the shared manifest and has one server attachment with
643+
trusted credential provider binding and a fixed, credential-bound, or explicitly reviewed
644+
user-controlled destination policy
645+
- [ ] No selector provider logic, credential resolution, or provider route call runs in the browser
633646
- [ ] Configured tools.access with all tool IDs
634647
- [ ] Configured tools.config.tool selector
635648
- [ ] Defined outputs matching tool outputs
@@ -922,7 +935,8 @@ requiredScopes: getScopesForService('{service}'),
922935
3. **Block type is snake_case** - `type: 'stripe'`, not `type: 'Stripe'`
923936
4. **Alphabetical ordering** - Keep imports and registry entries alphabetically sorted
924937
5. **Required can be conditional** - Use `required: { field: 'op', value: 'create' }` instead of always true
925-
6. **DependsOn clears options** - When a dependency changes, selector options are refetched
938+
6. **DependsOn clears options** - When an active dependency changes, the shared selector facade
939+
refetches with an opaque query revision; dependency values and references never enter query keys
926940
7. **Never pass Buffer directly to fetch** - Convert to `new Uint8Array(buffer)` for TypeScript compatibility
927941
8. **Always handle legacy file params** - Keep hidden `fileContent` params for backwards compatibility
928942
9. **Optional fields use advanced mode** - Set `mode: 'advanced'` on rarely-used optional fields
Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
---
2+
name: add-selector
3+
description: Add or update a Sim dynamic selector using the shared manifest, server attachment, and selectors.execute path. Use for provider-backed, internal, or local option lists referenced by block, trigger, or connector selectorKey fields.
4+
argument-hint: <selector-key>
5+
---
6+
7+
# Add Selector
8+
9+
Dynamic selectors expose option metadata while a workflow or connector is being configured. Every
10+
remote selector executes through the authorized `selectors.execute` application operation; the
11+
browser never resolves credentials or calls a provider directly.
12+
13+
## Read the shared boundary
14+
15+
Before editing, read:
16+
17+
- `apps/sim/lib/selectors/types.ts`
18+
- `apps/sim/lib/selectors/manifest.ts`
19+
- `apps/sim/lib/selectors/context.ts`
20+
- `apps/sim/lib/selectors/server/types.ts`
21+
- `apps/sim/lib/selectors/server/registry.ts`
22+
- `apps/sim/hooks/queries/selectors.ts`
23+
24+
Then read the nearest existing selector attachment and the block, trigger, or connector declaration
25+
that will consume the key.
26+
27+
## Classify the selector
28+
29+
- `provider-server`: contacts an external provider or uses provider credentials.
30+
- `internal-server`: reads protected Sim data through an existing authorized application use case.
31+
- `local`: pure browser-safe data with no protected data, credentials, references, or network I/O.
32+
33+
Add every key to the browser-safe manifest in `lib/selectors/manifest.ts`. `SelectorKey` derives from
34+
that manifest; do not maintain a second union. Manifest entries contain data only: allowed context,
35+
readiness, scope kinds, list/search/detail capabilities, and stale time. Do not import provider SDKs,
36+
credentials, server helpers, or attachment functions into the manifest.
37+
38+
## Build context from active values
39+
40+
Declare `dependsOn` on the consuming sub-block or connector field. The shared context builder sends
41+
only declared, active dependencies:
42+
43+
- Canonical basic/advanced pairs contribute the active value under their canonical key.
44+
- Action and trigger modes contribute only fields active on that surface.
45+
- Exact environment references such as `{{GMAIL_CREDENTIAL_ID}}` remain unresolved in the browser.
46+
- Runtime block-output references are not selector context.
47+
- Embedded environment interpolation such as `https://{{HOST}}/path` is unsupported.
48+
- `impersonateUserEmail` is the one explicit compatibility hint projected when the manifest allows
49+
it, even when it is absent from `dependsOn`.
50+
51+
Add a new `SelectorContextKey` only when the value is a real, reusable selector dependency. Allow it
52+
explicitly on each relevant manifest entry. Never send a full block or connector configuration.
53+
54+
## Add the server attachment
55+
56+
For `provider-server`, add the service's attachment map under
57+
`apps/sim/lib/selectors/server/providers/` and include it in the exhaustive server registry. For
58+
`internal-server`, add the attachment in `apps/sim/lib/selectors/server/internal.ts`. Local keys use
59+
the exhaustive browser-safe registry in `apps/sim/lib/selectors/client/local.ts` and never enter the
60+
server registry. A provider attachment declares:
61+
62+
- For stored credentials, a credential policy with the exact context field and trusted
63+
`serviceIds`. Raw-connection selectors instead validate the connection material projected from
64+
their allowed context and bind it to a deliberate destination policy.
65+
- Destination policy: `fixed`, `credential-bound`, or `user-controlled`.
66+
- A list/detail adapter that explicitly projects `id`, `label`, and allowlisted scalar `meta`.
67+
68+
Stored credentials must pass actor-use, workspace, and provider/service binding checks. Do not trust
69+
a provider, service, operation kind, origin, or module name supplied by the browser.
70+
71+
Choose the destination policy deliberately:
72+
73+
- `fixed`: provider origin is code-defined.
74+
- `credential-bound`: origin/account/site comes from, or is verified against, the authorized
75+
credential.
76+
- `user-controlled`: the user selects the destination. Hidden use-only authentication requires an
77+
explicit security policy; do not combine it with an arbitrary destination by default.
78+
79+
Reuse or extract a server-only provider listing primitive. If an existing provider route has
80+
non-selector callers, keep the route as a thin caller of that primitive. If it is selector-only,
81+
move the logic and remove the obsolete route and contract. Never import a route handler or make an
82+
internal HTTP request from an attachment.
83+
84+
The attachment must return normalized selector results only. It must never deliberately or
85+
wholesale echo selector context, and hidden/server-only resolved material, credential IDs, tokens,
86+
and authentication secrets must never cross the response boundary. Browser-known literals and
87+
viewable personal/shared values are not automatically server-only secrets, but they may appear in
88+
an option only when the adapter intentionally projects them as provider resource metadata. Let the
89+
shared executor own scope authorization, exact-reference resolution, credential authorization,
90+
error projection, and output sanitization; adapters must pass and preserve the executor's abort
91+
signal during provider work.
92+
93+
## Wire the UI declaration
94+
95+
Point the block, trigger, or connector field at `selectorKey` and declare its `dependsOn` fields.
96+
Keep connector selector/manual canonical pairs and fork reconfiguration behavior intact. Static
97+
`options` stay local and need no selector.
98+
99+
Do not add:
100+
101+
- A module under `hooks/selectors/providers` or any client provider fetcher.
102+
- A provider-specific React Query key.
103+
- A selector-specific OAuth-token request.
104+
- A selector-only API route when the provider primitive can be called directly.
105+
106+
All server selectors use the shared POST contract and React Query facade. Query identities must stay
107+
opaque and must not include context values, references, credential IDs, secrets, or their hashes.
108+
Selector code must not add context, token, or result caches. The sole existing cache exception is
109+
authorized client-credential resolution after authorization and provider binding: it may reuse the
110+
credential service's TTL-governed, lazily pruned process-local token cache. This exception requires
111+
explicit security-owner acceptance; do not broaden it or describe it as hard-bounded.
112+
113+
## Focused validation
114+
115+
Follow nearby Vitest and route-test style. Do not add an authorization matrix for every ordinary
116+
provider attachment; the shared executor tests own shared security behavior.
117+
118+
Add a focused adapter test when behavior is special, such as pagination, nontrivial destination
119+
binding, provider-specific projection, or a raw-connection policy. For an ordinary fixed-origin OAuth
120+
list, manifest/registry exhaustiveness plus an existing provider primitive test is usually enough.
121+
122+
Run the smallest relevant set, then:
123+
124+
```bash
125+
bunx vitest run <focused selector tests>
126+
bun run --cwd apps/sim type-check
127+
bun run check:fork-dependent-coverage
128+
bun run check:client-boundary
129+
git diff --check
130+
```
131+
132+
Confirm there is no browser-side provider call, every server key has one attachment, and every
133+
returned option is explicitly projected.
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
interface:
2+
display_name: "Add Selector"
3+
short_description: "Build a secure dynamic selector"
4+
brand_color: "#2563EB"
5+
default_prompt: "Use $add-selector to add or update a Sim dynamic selector through the unified server execution path."

0 commit comments

Comments
 (0)