Skip to content

Conversation

@yuvmen
Copy link
Member

@yuvmen yuvmen commented Oct 23, 2025

The File cleanup job was timing out trying to fetch by ordering the timestamp field which has no index in US. Switched to ordering by ID which is performant. This will cause us to always go through entire table in chunks but we want to do that right now. Once we cleaned a lot of rows we can add an index and revert.

Also corrected the child_relation to use ModelRelation type

@yuvmen yuvmen requested a review from wedamija October 23, 2025 23:04
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 23, 2025
(RuleFireHistory, "date_added", "date_added"),
(Release, "date_added", "date_added"),
(File, "timestamp", "timestamp"),
(File, "timestamp", "id"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The remove_cross_project_models function tries to remove a hardcoded tuple that was changed in this PR, causing a ValueError when cleanup --project is run.
Severity: HIGH | Confidence: 0.98

🔍 Detailed Analysis

The `cleanup` command will crash when run with the `--project` flag. A recent change updated the cleanup configuration for the `File` model from `(File, "timestamp", "timestamp")` to `(File, "timestamp", "id")`. However, the `remove_cross_project_models` function, which is called during project-specific cleanup, was not updated. It still attempts to remove the old, non-existent tuple `(File, "timestamp", "timestamp")` from a list. This operation will raise an unhandled `ValueError`, causing the entire cleanup command to fail and preventing any project-specific cleanup from completing.

💡 Suggested Fix

In remove_cross_project_models, update the line deletes.remove((File, "timestamp", "timestamp")) to deletes.remove((File, "timestamp", "id")) to match the new tuple structure defined elsewhere.

🤖 Prompt for AI Agent
Fix this bug. In src/sentry/runner/commands/cleanup.py at line 538: The `cleanup`
command will crash when run with the `--project` flag. A recent change updated the
cleanup configuration for the `File` model from `(File, "timestamp", "timestamp")` to
`(File, "timestamp", "id")`. However, the `remove_cross_project_models` function, which
is called during project-specific cleanup, was not updated. It still attempts to remove
the old, non-existent tuple `(File, "timestamp", "timestamp")` from a list. This
operation will raise an unhandled `ValueError`, causing the entire cleanup command to
fail and preventing any project-specific cleanup from completing.

Did we get this right? 👍 / 👎 to inform future reviews.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's actually right about this, although I don't know how much it actually matters

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh yea, no reason not to correct this, I will

The File cleanup job was timing out trying to fetch by ordering the timestamp field which has no index in US.
Switched to ordering by ID which is performant. This will cause us to always go through entire table in chunks
but we want to do that right now. Once we cleaned a lot of rows we can add an index and revert.

Also corrected the child_relation to use ModelRelation type
@codecov
Copy link

codecov bot commented Oct 24, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff            @@
##           master   #102055    +/-   ##
=========================================
  Coverage   80.97%    80.97%            
=========================================
  Files        8733      8733            
  Lines      388590    388701   +111     
  Branches    24654     24654            
=========================================
+ Hits       314657    314749    +92     
- Misses      73573     73592    +19     
  Partials      360       360            

@yuvmen yuvmen merged commit c11f09e into master Oct 24, 2025
69 checks passed
@yuvmen yuvmen deleted the yuvmen/fix-file-cleanup branch October 24, 2025 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants