Skip to content

Conversation

@ofir-frd
Copy link

Benchmark PR getsentry#103875

Type: Clean (correct implementation)

Original PR Title: feat(explorer): rpcs for getting log/metric attrs for a trace id + substring
Original PR Description: Adds specialized rpcs for getting a list of log/metric attribute dicts filtered by trace_id and optional substring. This is done with a single snuba rpc (not counting short id lookup) and is a lot more performant than the table queries we were doing earlier.

Return format is list of attr key-value dicts. This is different from the current tool output format which includes type, tbd if we still need to include that


GetTrace experiment:

Using https://snuba-admin.getsentry.net/#rpc-endpoints, I made requests to EndpointGetTrace and EndpointTraceItemTable for the same purpose - filter logs by a trace (2h old) in Seer project. Time range is last 30d, and I selected the same columns. The table query has an additional LIKE filter on sentry.body (message) - this isn't supported for GetTrace so we'd have to post-process in sentry or seer

EndpointGetTrace - 1318904 progressBytes (bytes processed by clickhouse)
EndpointTraceItemTable - 52792914551 progressBytes. 40k times more, prob scanning from the start of the time range. This is evidence the GetTrace endpoint is way better optimized for trace_id filters.

The requests:

GetTrace:

{
  "meta": {
    "organizationId": "1",
    "referrer": "seer.rpc",
    "projectIds": [
      "6178942"
    ],
    "startTimestamp": "2025-10-24T04:06:08.199596Z",
    "endTimestamp": "2025-11-23T04:06:08.199596Z",
    "traceItemType": "TRACE_ITEM_TYPE_LOG",
    "downsampledStorageConfig": {
      "mode": "MODE_NORMAL"
    }
  },
  "traceId": "983c9c1267364ce9b3ef9fc14c4b25a5",
  "items": [
    {
      "attributes": [
        {
          "type": "TYPE_STRING",
          "name": "sentry.item_id"
        },
        {
          "type": "TYPE_DOUBLE",
          "name": "sentry.timestamp_precise"
        },
        {
          "type": "TYPE_STRING",
          "name": "sentry.timestamp"
        },
        {
          "type": "TYPE_STRING",
          "name": "sentry.severity_text"
        },
        {
          "type": "TYPE_STRING",
          "name": "sentry.trace.parent_span_id"
        },
        {
          "type": "TYPE_STRING",
          "name": "sentry.body"
        }
      ],
      "itemType": "TRACE_ITEM_TYPE_LOG"
    }
  ]
}

TraceItemTable

{
  "meta": {
    "organizationId": "1",
    "referrer": "api.auth-token.events",
    "projectIds": [
      "6178942"
    ],
    "startTimestamp": "2025-10-24T04:06:08.199596Z",
    "endTimestamp": "2025-11-23T04:06:08.199596Z",
    "traceItemType": "TRACE_ITEM_TYPE_LOG",
    "downsampledStorageConfig": {
      "mode": "MODE_NORMAL"
    }
  },
  "columns": [
    {
      "key": {
        "type": "TYPE_STRING",
        "name": "sentry.item_id"
      },
      "label": "id"
    },
    {
      "key": {
      "type": "TYPE_DOUBLE",
          "name": "sentry.timestamp_precise"
        },
      "label": "timestamp_precise"
    },
    {
      "key": {
      "type": "TYPE_DOUBLE",
          "name": "sentry.timestamp"
        },
      "label": "timestamp"
    },
    {
      "key": {
          "type": "TYPE_STRING",
          "name": "sentry.severity_text"
        },
      "label": "severity"
    },
    {
      "key": {
          "type": "TYPE_STRING",
          "name": "sentry.trace.parent_span_id"
        },
      "label": "parent_span_id"
    },
    {
      "key":{
          "type": "TYPE_STRING",
          "name": "sentry.body"
        },
      "label": "message"
    }
  ],
  "orderBy": [
    {
      "column": {
        "key": {
          "type": "TYPE_DOUBLE",
          "name": "sentry.timestamp"
        },
        "label": "timestamp"
      },
      "descending": true
    }
  ],
  "pageToken": {
    "offset": "0"
  },
  "filter": {
    "andFilter": {
      "filters": [
        {
          "comparisonFilter": {
            "key": {
              "type": "TYPE_STRING",
              "name": "sentry.body"
            },
            "op": "OP_LIKE",
            "value": {
              "valStr": "%POST%"
            }
          }
        },
        {
          "comparisonFilter": {
            "key": {
              "type": "TYPE_STRING",
              "name": "sentry.trace_id"
            },
            "op": "OP_EQUALS",
            "value": {
              "valStr": "983c9c1267364ce9b3ef9fc14c4b25a5"
            }
          }
        }
      ]
    }
  }
}

Original PR URL: getsentry#103875

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants