fix(aci): Report Snuba exceptions as warnings and retry #94582
Conversation
🔍 Existing Issues For ReviewYour pull request is modifying functions with the following pre-existing issues: 📄 File: src/sentry/workflow_engine/processors/delayed_workflow.py
Did you find this useful? React with a 👍 or 👎 |
github-actions
bot
added
the
Scope: Backend
| @@ -675,7 +677,7 @@ def repr_keys[T, V](d: dict[T, V]) -> dict[str, V]: | |||
| ), | |||
| ), | |||
| ) | |||
| @retry | |||
| @retry(exclude=(MaxAttemptsExceededError,)) | |||
There was a problem hiding this comment.
should this also exclude NoRetriesRemainingError as well since we are invoking retry_task?
sentry/src/sentry/taskworker/retry.py
Line 62 in b147712
There was a problem hiding this comment.
Yes, but I decided maybe we shouldn't have to wire this up and sent #94580 but neglected to put this pr on ice in the meantime.
| @@ -9,6 +9,7 @@ | |||
|
|
|||
| import sentry_sdk | |||
| from celery import Task | |||
| from celery.exceptions import MaxAttemptsExceededError | |||
There was a problem hiding this comment.
🤔 ci seems to not like this.
There was a problem hiding this comment.
Oop, fixed that, then accidentally allowed cursor to slop it and didn't notice. Fixed.
Codecov ReportAll modified and coverable lines are covered by tests ✅ ✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## master #94582 +/- ##
==========================================
- Coverage 85.60% 80.72% -4.89%
==========================================
Files 10384 10384
Lines 601676 601673 -3
Branches 23408 23408
==========================================
- Hits 515091 485692 -29399
- Misses 86129 115525 +29396
Partials 456 456 |
Snuba errors are expected and should not be treated as errors; when they occur, we'd like to retry.
We log them as warnings before retry so we still have record of them; if we used
@retryfor this, it'd report them as high priority and we'd like to avoid that in this context.