feat(search_issues): rewrite the search issues consumer in Rust

[phacops]

Reimplements SearchIssuesMessageProcessor as a native Rust processor so the search issues / issue-occurrence consumer (snuba rust-consumer --storage=search_issues) runs in Rust instead of falling back to the Python processor over the multiprocessing bridge.

What changed

• New Rust processor rust_snuba/src/processors/search_issues.rs, registered in processors/mod.rs against the generic-events topic. Once registered, factory_v2.rs routes the consumer to the Rust processor natively (previously it fell through to PythonTransformStep).
• Python SearchIssuesMessageProcessor becomes a thin RustCompatProcessor shim, matching the pattern used by ErrorsProcessor, ReplaysProcessor, OutcomesProcessor, etc. The SearchIssueEvent / IssueEventData / IssueOccurrenceData TypedDicts are retained since other modules import them.
• Tests in tests/datasets/test_search_issues_processor.py now exercise the Rust implementation through the compat shim (timestamp columns are integers, invalid messages raise a generic exception).

Parity notes

The Rust processor mirrors the Python one exactly:

• Parses the [version, "insert", event, group_state?] payload, ignoring the trailing post-process group state.
• Promotes environment / sentry:release / sentry:dist / sentry:user tags; accepts tags and request headers as either an object or a list of pairs (_as_dict_safe); sorts tags by key.
• Preserves payload key order for contexts.key / contexts.value (the Python processor does not sort contexts) via order-preserving deserialization — necessary because serde_json's Map is a BTreeMap without the preserve_order feature.
• Promotes trace_id / profile_id / replay_id from contexts to UUID columns (erroring on invalid UUIDs); extracts user id/name/email + IPv4/IPv6, sdk name/version, http method/referer, and transaction duration.
• Derives client_timestamp (second precision) and timestamp_ms (millisecond precision) from data.client_timestamp when present, otherwise from the datetime field.

Testing

• 25 Rust unit tests ported from the Python processor tests (cargo test --lib search_issues).
• Snapshot tests over the two generic-events schema examples (test_schemas), reviewed for field-level parity with the Python output.
• cargo clippy and rustfmt clean.

Legal Boilerplate

Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WaxqQJj1RHRonojiRC3puS

---

Generated by Claude Code

[sentry]

Bug: The calculation for transaction_duration can overflow when casting to u32. A large difference between start_timestamp and timestamp will cause the value to wrap, resulting in an incorrect duration.
_{Severity: MEDIUM}

Suggested Fix

Instead of casting directly with as u32, which truncates on overflow, the value should be clamped to the maximum value of u32. Use a saturating cast or clamp(0, u32::MAX as i64) as u32 to prevent the overflow and ensure the duration is capped at u32::MAX if it's too large.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: rust_snuba/src/processors/search_issues.rs#L549

Potential issue: The calculation `((finish_secs - start_secs) * 1000).max(0) as u32` is
used to determine the transaction duration in milliseconds. If the time difference
between `finish_secs` and `start_secs` exceeds approximately 49.7 days, the result of
the multiplication will be larger than `u32::MAX`. The subsequent cast `as u32` will
then silently truncate the value, causing it to wrap around and produce an incorrect
duration. This is a realistic scenario for events with a very old `start_timestamp` and
a recent `timestamp`.

_{Did we get this right? 👍 / 👎 to inform future reviews.}

[cursor]

Old events no longer discarded

Medium Severity

The Python processor called enforce_retention with detection_timestamp and treated EventTooOld as a silent skip (return None). With DISCARD_OLD_EVENTS enabled by default, occurrences older than their retention window were never written. The Rust path only normalizes retention_days and always inserts, so delayed or replayed old issue occurrences that used to be dropped will now land in ClickHouse.

 

^{Reviewed by Cursor Bugbot for commit 9eefded. Configure here.}

[MeredithAnya]

@phacops this seems valid but not sure what we actually do for errors, if they are old enough i guess they will just get immediately TTL'd

[phacops]

The observation is accurate, and it's consistent with the other Rust-migrated processors. None of them port the Python EventTooOld / DISCARD_OLD_EVENTS skip — the shared enforce_retention in rust_snuba only normalizes retention_days and always inserts. So errors (and the rest) behave the same way today: an old occurrence is written and then aged out by ClickHouse TTL via retention_days + the partition TTL, rather than dropped at ingest — which matches your intuition about it getting immediately TTL'd.

I left the behavior as-is to stay consistent with the fleet, but happy to add an explicit age check here if we'd prefer search_issues to drop at ingest instead.

---

Generated by Claude Code

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 96cfce4. Configure here.}

[sentry]

Bug: Messages with a malformed group_first_seen datetime string are rejected, which is a stricter behavior than the previous Python implementation that likely ignored such values.
_{Severity: MEDIUM}

Suggested Fix

To align with the likely previous behavior and prevent message rejection, wrap the date parsing logic in a match or use .ok() to convert parsing errors into None. This would gracefully handle malformed date strings instead of failing the entire message. Additionally, add a test case with an invalid group_first_seen value to verify the intended error-handling behavior.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: rust_snuba/src/processors/search_issues.rs#L412-L413

Potential issue: The Rust implementation for processing `group_first_seen` uses strict
datetime parsing. If the `group_first_seen` field is present but contains a string that
does not match the expected format (`%Y-%m-%dT%H:%M:%S%.fZ`), the `?` operator will
propagate the parsing error, causing the entire message to be rejected and potentially
sent to the dead-letter queue. The previous Python implementation likely handled such
malformed dates more gracefully, possibly by treating them as null. This discrepancy
introduces a risk of message processing failures for data that was previously accepted.

[phacops]

This is actually parity-preserving, not stricter. The Python processor parsed group_first_seen with datetime.strptime(..., PAYLOAD_DATETIME_FORMAT), which raises ValueError on a malformed string. process_message re-raises ValueError (it's only caught to bump a metric), so a malformed group_first_seen was rejected/DLQ'd there too — it was never treated as null. Only an out-of-range-but-well-formed date became null (via _ensure_valid_date), and the Rust code mirrors that: a parse error bails, while an out-of-uint32-range timestamp yields None. So the reject-vs-null split matches the previous behavior.

(In practice group_first_seen is producer-generated via strftime, so a malformed value shouldn't occur in either implementation.)

---

Generated by Claude Code