Skip to content

Conversation

@MichaReiser
Copy link
Member

@MichaReiser MichaReiser commented Nov 7, 2025

Enable the walltime performance profile upload for codspeed walltime benchmarks.

This does increase the walltime benchmark runtime by ~2min (from ~8 to ~10 and ~9 to ~11min). I do find the profile's useful when investigating a regression (to get a first impression of where it might be coming from).

@MichaReiser MichaReiser added the ci Related to internal CI tooling label Nov 7, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Nov 7, 2025

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

Formatter (stable)

✅ ecosystem check detected no format changes.

Formatter (preview)

✅ ecosystem check detected no format changes.

@MichaReiser MichaReiser marked this pull request as ready for review November 7, 2025 20:41
@MichaReiser MichaReiser changed the title Experiment with enabling walltime perf profile upload Enable perf profile upload for walltime benchmarks Nov 9, 2025
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we must have had very different experiences when investigating ty performance regressions 😆

To me, the benefits still don't seem worth the cost here. I've found the codspeed flamegraphs very useful when investigating Ruff performance regressions, but I can't recall a time when I've found they've given me any useful information regarding a ty performance regression -- and all our walltime benchmarks are ty benchmarks. A 2-minute increase in CI times is a pretty significant increase, meanwhile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Related to internal CI tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants