feat(eap): gate unpopulated normalized columns behind a per-org allowlist

[pbhandari]

What

Gate the not-yet-populated EAP-items normalized columns (sentry.session_id → session_id, gen_ai.conversation.id → ai_conversation_id) behind a per-organization allowlist instead of resolving them unconditionally.

A new sentry-option, eap_items_unpopulated_normalized_columns_org_allowlist (comma-separated org ids), controls it:

• Org in the allowlist → these attributes resolve to their dedicated, bloom-filter-indexed columns (fast granule pruning).
• Org not in it (default: empty ⇒ nobody) → they fall through to the attributes_* map path, which covers historical rows the dedicated columns were never populated with.

Unlike org_ids_delete_allowlist, an empty allowlist means no orgs are opted in — the feature stays off until an org is explicitly added. A malformed allowlist (stray comma, non-numeric entry) is logged and treated as opted out rather than breaking every EAP query.

How

get_normalized_columns_eap_items(organization_id) now takes the org and consults the allowlist. Rather than a global/ambient flag, organization_id is threaded explicitly as an argument through the query-build path:

• attribute_key_to_expression(attr_key, organization_id) and trace_item_filters_to_expression(..., *, organization_id) (keyword-only) carry it.
• The aggregation.py helpers that build filter conditions (aggregation_to_expression, get_confidence_interval_column, get_average_sample_rate_column, get_count_column, and their internals) take a keyword-only organization_id.
• At each resolver/endpoint entry point the org comes from request.meta.organization_id. The resolver callables are typed Callable[[AttributeKey, int], Expression] and receive the org id as a plain argument — no functools.partial binding.

Non-RPC delete path (delete_query.py): the org is taken from the delete's column_conditions (deletes are per-org), falling back to None (no org in context ⇒ feature off).

Test plan

• New tests/web/rpc/v1/test_indexed_columns.py coverage:
  • allowlisted org → skip index (bf_session_id / bf_ai_conversation_id) is applied and filters match the expected rows.
  • non-allowlisted org → skip index is not applied, and the map lookup matches nothing (the consumer writes these only to the dedicated columns).
• New TestGetNormalizedColumnsEapItems in tests/protos/test_protos_common.py exercises the gate directly (opted in, absent org, None org, empty allowlist, and malformed allowlists → opted out).
• mypy . clean (1074 files); ruff check/format clean.
• Updated the existing direct-call unit tests (test_common.py, test_aggregation.py, test_protos_common.py, etc.) for the new signatures.

Notes

• Stacked on top of feat(eap): Resolve session_id and gen_ai.conversation.id to their indexed columns #8185 (base branch ref/use-indexed-columns); review/merge that one first.

🤖 Generated with Claude Code

[onewland]

Is the idea here that we're going to evaluate this for a few orgs, and then delete this whole [logic] branch?

I'm asking because we have 10s of thousands of active orgs, and so if you want to gradually roll out 0 -> 100 then a hardcoded allow-list won't work very well and you'll have to do something like if org_id % rollout_percentage:. But if the idea is to smoke test and then roll out for everybody then this approach is fine

[pbhandari]

@onewland Yeah, the idea just to allow for smoke tests to happen with the data and we roll out to a small subset of orgs initially. And when we are ready to release it for everyone, we just move the fields into the _NORMALIZED... array.