Skip to content

feat(bench): post PR benchmark reports#15647

Draft
stevencartavia wants to merge 13 commits into
masterfrom
steven/pr-bench-reports
Draft

feat(bench): post PR benchmark reports#15647
stevencartavia wants to merge 13 commits into
masterfrom
steven/pr-bench-reports

Conversation

@stevencartavia

Copy link
Copy Markdown
Member

Benchmarks a PR against a source-built master baseline (merge-base) and posts a sticky comment with regressions and improvements. Triggered via the Benchmarks workflow with versions=master,local.

Closes OSS-478

Comment thread .github/workflows/benchmarks.yml Fixed
stevencartavia and others added 3 commits July 8, 2026 13:53
…execution of untrusted code'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

📊 Benchmark Results

cc @stevencartavia

✅ Benchmark complete! View job

Config: versions=master,local; baseline=e7ae2ad00376fac0827ff739fce35c155d241588; repos=per-benchmark defaults

⚪ no significant changes (0 regression, 0 improvement, 23 neutral)

Benchmark master this PR Change
forge_build_no_cache/ithacaxyz-account 34.99 s 33.28 s -4.9% ⚪ (±9.7%, floor 2.0%)
forge_build_no_cache/sparkdotfi-spark-psm 12.67 s 12.65 s -0.1% ⚪ (±0.5%, floor 2.0%)
forge_build_no_cache/uniswap-v4-core 2m 17.6s 2m 17.2s -0.3% ⚪ (±0.5%, floor 2.0%)
forge_build_no_cache/vectorized-solady 14.48 s 14.44 s -0.3% ⚪ (±0.4%, floor 2.0%)
forge_build_with_cache/ithacaxyz-account 0.092 s 0.092 s +0.0% ⚪ (±3.3%, floor 2.0%)
forge_build_with_cache/sparkdotfi-spark-psm 0.071 s 0.071 s -1.1% ⚪ (±3.8%, floor 2.0%)
forge_build_with_cache/uniswap-v4-core 0.074 s 0.075 s +0.7% ⚪ (±2.8%, floor 2.0%)
forge_build_with_cache/vectorized-solady 0.065 s 0.065 s +1.0% ⚪ (±3.1%, floor 2.0%)
forge_coverage/ithacaxyz-account 27.27 s 18.78 s -31.1% ⚪ (single run, not judged)
forge_coverage/sparkdotfi-spark-psm 1m 48.3s 1m 47.7s -0.5% ⚪ (single run, not judged)
forge_coverage/uniswap-v4-core 1m 6.3s 1m 6.8s +0.8% ⚪ (single run, not judged)
forge_fuzz_test/ithacaxyz-account 1.11 s 1.12 s +0.7% ⚪ (±6.3%, floor 2.0%)
forge_fuzz_test/sparkdotfi-spark-psm 2.48 s 2.50 s +0.7% ⚪ (±1.6%, floor 2.0%)
forge_fuzz_test/uniswap-v4-core 1.53 s 1.56 s +1.9% ⚪ (±2.9%, floor 2.0%)
forge_fuzz_test/vectorized-solady 0.649 s 0.667 s +2.8% ⚪ (±5.4%, floor 2.0%)
forge_isolate_test/ithacaxyz-account 1.16 s 1.14 s -1.8% ⚪ (±6.5%, floor 2.0%)
forge_isolate_test/sparkdotfi-spark-psm 18.70 s 18.80 s +0.5% ⚪ (±2.8%, floor 2.0%)
forge_isolate_test/uniswap-v4-core 1.73 s 1.69 s -2.3% ⚪ (±8.3%, floor 2.0%)
forge_isolate_test/vectorized-solady 0.715 s 0.758 s +6.0% ⚪ (±6.9%, floor 2.0%)
forge_test/ithacaxyz-account 1.14 s 1.09 s -4.2% ⚪ (±9.5%, floor 2.0%)
forge_test/sparkdotfi-spark-psm 18.13 s 18.49 s +2.0% ⚪ (±2.5%, floor 2.0%)
forge_test/uniswap-v4-core 1.64 s 1.64 s +0.1% ⚪ (±5.7%, floor 2.0%)
forge_test/vectorized-solady 0.705 s 0.681 s -3.4% ⚪ (±10.6%, floor 2.0%)

🔴 regression · 🟢 improvement · ⚪ within noise. A change is flagged only when the whole noise band (±) clears the 2.0% floor.

📈 Full benchmark results

📊 Foundry Benchmark Results

Generated at: 2026-07-09 07:56:21 UTC

Forge Test

Repositories Tested

  1. ithacaxyz/account
  2. vectorized/solady
  3. uniswap/v4-core
  4. sparkdotfi/spark-psm

Foundry Versions

  • master: forge Version: 1.7.2-dev (e7ae2ad 2026-07-09)
  • local: forge Version: 1.7.2-dev (59caad8 2026-07-09)
Repository master local
ithacaxyz-account 1.14 s 1.09 s
vectorized-solady 0.705 s 0.681 s
uniswap-v4-core 1.64 s 1.64 s
sparkdotfi-spark-psm 18.13 s 18.49 s

Forge Fuzz Test

Repository master local
ithacaxyz-account 1.11 s 1.12 s
vectorized-solady 0.649 s 0.667 s
uniswap-v4-core 1.53 s 1.56 s
sparkdotfi-spark-psm 2.48 s 2.50 s

Forge Test (Isolated)

Repositories Tested

  1. ithacaxyz/account
  2. vectorized/solady
  3. uniswap/v4-core
  4. sparkdotfi/spark-psm

Foundry Versions

  • master: forge Version: 1.7.2-dev (e7ae2ad 2026-07-09)
  • local: forge Version: 1.7.2-dev (59caad8 2026-07-09)
Repository master local
ithacaxyz-account 1.16 s 1.14 s
vectorized-solady 0.715 s 0.758 s
uniswap-v4-core 1.73 s 1.69 s
sparkdotfi-spark-psm 18.70 s 18.80 s

Forge Build

Repositories Tested

  1. ithacaxyz/account
  2. vectorized/solady
  3. uniswap/v4-core
  4. sparkdotfi/spark-psm

Foundry Versions

  • master: forge Version: 1.7.2-dev (e7ae2ad 2026-07-09)
  • local: forge Version: 1.7.2-dev (59caad8 2026-07-09)

No Cache

Repository master local
ithacaxyz-account 34.99 s 33.28 s
vectorized-solady 14.48 s 14.44 s
uniswap-v4-core 2m 17.6s 2m 17.2s
sparkdotfi-spark-psm 12.67 s 12.65 s

With Cache

Repository master local
ithacaxyz-account 0.092 s 0.092 s
vectorized-solady 0.065 s 0.065 s
uniswap-v4-core 0.074 s 0.075 s
sparkdotfi-spark-psm 0.071 s 0.071 s

Forge Coverage

Repositories Tested

  1. ithacaxyz/account
  2. uniswap/v4-core
  3. sparkdotfi/spark-psm

Foundry Versions

  • master: forge Version: 1.7.2-dev (e7ae2ad 2026-07-09)
  • local: forge Version: 1.7.2-dev (59caad8 2026-07-09)
Repository master local
ithacaxyz-account 27.27 s 18.78 s
uniswap-v4-core 1m 6.3s 1m 6.8s
sparkdotfi-spark-psm 1m 48.3s 1m 47.7s

System Information

  • OS: linux
  • CPU: 32
  • Rustc: rustc 1.96.1 (31fca3adb 2026-06-26)

🤖 Generated by the Foundry Benchmarks workflow.

…orts

# Conflicts:
#	.github/workflows/benchmarks-dispatch.yml
Reuse master's benchmarks.yml comment path instead of a parallel
workflow. Build origin/master from source as the baseline (not the
stable release) so master-only regressions surface, and inject
regression/improvement verdicts into the PR comment via a generalized
compare-nightly.sh. Drops the abandoned derek-bench dispatch job and the
duplicate compare/post scripts.
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results (master vs this branch)

View benchmark run · baseline 8db0443ce044

Verdicts

Benchmark master this branch Δ Status
forge_build_no_cache/ithacaxyz-account 48.92274749304s 46.24203410132s -5.5% ➡️ OK
forge_build_no_cache/sparkdotfi-spark-psm 17.2356160231s 17.11396952386s -0.7% ➡️ OK
forge_build_no_cache/uniswap-v4-core 187.58076379646s 187.70370389474002s +0.1% ➡️ OK
forge_build_no_cache/vectorized-solady 20.23258868366s 20.168393578019998s -0.3% ➡️ OK
forge_build_with_cache/ithacaxyz-account 0.11208298137999999s 0.11280990048000002s +0.6% ➡️ OK
forge_build_with_cache/sparkdotfi-spark-psm 0.08808396265999999s 0.08756451904000001s -0.6% ➡️ OK
forge_build_with_cache/uniswap-v4-core 0.09135119528s 0.09261097334000001s +1.4% ➡️ OK
forge_build_with_cache/vectorized-solady 0.07688931439999999s 0.07790389982s +1.3% ➡️ OK
forge_coverage/ithacaxyz-account 36.45250130366s 25.93642796024s -28.8% ⚪ Not judged
forge_coverage/sparkdotfi-spark-psm 157.30654309404s 157.0542728574s -0.2% ⚪ Not judged
forge_coverage/uniswap-v4-core 88.38071783124s 88.86174199706s +0.5% ⚪ Not judged
forge_fuzz_test/ithacaxyz-account 1.5744518219199999s 1.64751455418s +4.6% ➡️ OK
forge_fuzz_test/sparkdotfi-spark-psm 3.9333926249800006s 3.94306308364s +0.2% ➡️ OK
forge_fuzz_test/uniswap-v4-core 1.99647259876s 2.03341761716s +1.9% ➡️ OK
forge_fuzz_test/vectorized-solady 0.86597815614s 0.8695997116s +0.4% ➡️ OK
forge_isolate_test/ithacaxyz-account 1.6015629469400001s 1.6062219075400002s +0.3% ➡️ OK
forge_isolate_test/sparkdotfi-spark-psm 29.1876065762s 29.52251853436s +1.1% ➡️ OK
forge_isolate_test/uniswap-v4-core 2.2336540660999997s 2.1056122029999997s -5.7% ➡️ OK
forge_isolate_test/vectorized-solady 1.07503453044s 1.00325627766s -6.7% ➡️ OK
forge_test/ithacaxyz-account 1.7023597787s 1.71699783742s +0.9% ➡️ OK
forge_test/sparkdotfi-spark-psm 28.5987775517s 28.570898670360002s -0.1% ➡️ OK
forge_test/uniswap-v4-core 2.1567231896s 2.26646845382s +5.1% ➡️ OK
forge_test/vectorized-solady 0.9629535720400002s 0.9058880311599999s -5.9% ➡️ OK

📊 Foundry Benchmark Results

Generated at: 2026-07-09 18:20:29 UTC

Forge Test

Repositories Tested

  1. ithacaxyz/account
  2. vectorized/solady
  3. uniswap/v4-core
  4. sparkdotfi/spark-psm

Foundry Versions

  • master: forge Version: 1.7.2-dev (8db0443 2026-07-09)
  • local: forge Version: 1.7.2-dev (fc1186a 2026-07-09)
Repository master local
ithacaxyz-account 1.70 s 1.72 s
vectorized-solady 0.963 s 0.906 s
uniswap-v4-core 2.16 s 2.27 s
sparkdotfi-spark-psm 28.60 s 28.57 s

Forge Fuzz Test

Repository master local
ithacaxyz-account 1.57 s 1.65 s
vectorized-solady 0.866 s 0.870 s
uniswap-v4-core 2.00 s 2.03 s
sparkdotfi-spark-psm 3.93 s 3.94 s

Forge Test (Isolated)

Repositories Tested

  1. ithacaxyz/account
  2. vectorized/solady
  3. uniswap/v4-core
  4. sparkdotfi/spark-psm

Foundry Versions

  • master: forge Version: 1.7.2-dev (8db0443 2026-07-09)
  • local: forge Version: 1.7.2-dev (fc1186a 2026-07-09)
Repository master local
ithacaxyz-account 1.60 s 1.61 s
vectorized-solady 1.08 s 1.00 s
uniswap-v4-core 2.23 s 2.11 s
sparkdotfi-spark-psm 29.19 s 29.52 s
📈 View all benchmark results

Forge Build

Repositories Tested

  1. ithacaxyz/account
  2. vectorized/solady
  3. uniswap/v4-core
  4. sparkdotfi/spark-psm

Foundry Versions

  • master: forge Version: 1.7.2-dev (8db0443 2026-07-09)
  • local: forge Version: 1.7.2-dev (fc1186a 2026-07-09)

No Cache

Repository master local
ithacaxyz-account 48.92 s 46.24 s
vectorized-solady 20.23 s 20.17 s
uniswap-v4-core 3m 7.6s 3m 7.7s
sparkdotfi-spark-psm 17.24 s 17.11 s

With Cache

Repository master local
ithacaxyz-account 0.112 s 0.113 s
vectorized-solady 0.077 s 0.078 s
uniswap-v4-core 0.091 s 0.093 s
sparkdotfi-spark-psm 0.088 s 0.088 s

Forge Coverage

Repositories Tested

  1. ithacaxyz/account
  2. uniswap/v4-core
  3. sparkdotfi/spark-psm

Foundry Versions

  • master: forge Version: 1.7.2-dev (8db0443 2026-07-09)
  • local: forge Version: 1.7.2-dev (fc1186a 2026-07-09)
Repository master local
ithacaxyz-account 36.45 s 25.94 s
uniswap-v4-core 1m 28.4s 1m 28.9s
sparkdotfi-spark-psm 2m 37.3s 2m 37.1s

System Information

  • OS: linux
  • CPU: 32
  • Rustc: rustc 1.97.0 (2d8144b78 2026-07-07)

🤖 This comment was automatically generated by the Foundry Benchmarks workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants