Skip to content

ref(ci): Run more of sentry's tests when we change the API in snuba #8237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
pbhandari wants to merge 4 commits into
base: master
Choose a base branch
from
+13 −50
Draft

ref(ci): Run more of sentry's tests when we change the API in snuba #8237

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bb56d76
ref(ci): Run more of sentry's tests when we change the API in snuba
pbhandari Jul 29, 2026
5150e13
Run full test suite on CI to see times
pbhandari Jul 29, 2026
7ecbb74
Ignore acceptance tests
pbhandari Jul 30, 2026
299ff7c
Use Makefile instead of hardcoded pytest to run tests
pbhandari Jul 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 13 additions & 50 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,17 +225,16 @@ jobs:
.

- name: Publish snuba-ci image to artifacts
# we publish to github artifacts separately so that third-party
# contributions also work, as all the test jobs need this image.
# otherwise third-party contributors would have to provide a working,
# authenticated GHCR, which seems impossible to ensure in the general
# case.
# we publish to github artifacts separately so that third-party
# contributions also work, as all the test jobs need this image.
# otherwise third-party contributors would have to provide a working,
# authenticated GHCR, which seems impossible to ensure in the general
# case.
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: snuba-ci
path: /tmp/snuba-ci.tar


docker-deps:
name: Warm Docker dependency image cache
runs-on: ubuntu-latest
Expand Down Expand Up @@ -466,54 +465,18 @@ jobs:
- name: Bootstrap per-worker Snuba instances
run: python3 sentry/.github/workflows/scripts/bootstrap-snuba.py

- name: Run snuba tests
if: needs.files-changed.outputs.api_changes == 'false'
- name: Run snuba CI tests
working-directory: sentry
run: |
# Only these dirs carry the `snuba_ci` marker; collecting all of tests/ just to find them is wasteful.
Comment thread
cursor[bot] marked this conversation as resolved.

@pbhandari pbhandari Jul 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is valid; but given that pytest collection is pretty fast, and reverting a change can take up to 2+ hours if we miss something on the CI ... better to just collect everything

pytest -k 'not __In' \
tests/sentry/snuba \
tests/sentry/workflow_engine/endpoints \
tests/snuba/rules/conditions \
-m snuba_ci \
-n "$XDIST_WORKERS" --dist=loadfile --reuse-db \
-vv
# monolith acceptance tests contain some js
env:
PYTEST_ADDOPTS: "-k 'not __In' -m snuba_ci -n ${{ env.XDIST_WORKERS }} --dist=loadfile -vv"
run: make test-python-ci

- name: Run full tests
if: needs.files-changed.outputs.api_changes == 'true'

@cursor cursor Bot Jul 29, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full tests missing API-change gate

Medium Severity

The Run full tests step dropped its if: needs.files-changed.outputs.api_changes == 'true' guard, so the large -m snuba suite now runs on every PR. The PR description still treats that suite as API-change-only, and api_changes is computed but unused. Non-API PRs therefore pay for ~5,555 tests instead of the intended ~173 snuba_ci set.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 5150e13. Configure here.

working-directory: sentry
run: |
pytest -k 'not __In' \
tests/snuba \
tests/sentry/snuba \
tests/sentry/eventstream/kafka \
tests/sentry/post_process_forwarder \
tests/sentry/services/eventstore/snuba \
tests/sentry/search/events \
tests/sentry/event_manager \
tests/sentry/api/endpoints/test_organization_profiling_functions.py \
tests/sentry/integrations/slack/test_unfurl.py \
tests/sentry/uptime/endpoints/test_project_uptime_alert_check_index.py \
tests/sentry/uptime/endpoints/test_organization_uptime_stats.py \
tests/sentry/api/endpoints/test_organization_traces.py \
tests/sentry/api/endpoints/test_organization_spans_fields.py \
tests/sentry/api/endpoints/test_organization_ai_conversations.py \
tests/sentry/api/endpoints/test_organization_ai_conversation_details.py \
tests/sentry/sentry_metrics/querying \
-n "$XDIST_WORKERS" --dist=loadfile --reuse-db \
-vv

- name: Run CI module tests
if: needs.files-changed.outputs.api_changes == 'true'
working-directory: sentry
run: |
pytest -k 'not __In' \
tests/sentry/snuba \
tests/sentry/workflow_engine/endpoints \
tests/snuba/rules/conditions \
-m snuba_ci \
-n "$XDIST_WORKERS" --dist=loadfile --reuse-db \
-vv
env:
PYTEST_ADDOPTS: "-k 'not __In' -m snuba -n ${{ env.XDIST_WORKERS }} --dist=loadfile -vv"
run: make test-python-ci

clickhouse-versions:
needs: [linting, snuba-image, docker-deps]
Expand Down
Loading