Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Small smoke test fixes #1346

Merged
merged 2 commits into from
Mar 12, 2024
Merged

Small smoke test fixes #1346

merged 2 commits into from
Mar 12, 2024

Conversation

ch1bo
Copy link
Collaborator

@ch1bo ch1bo commented Mar 11, 2024

  • Added traces to give us more insights into when the smoke test fails (e.g. due to permission errors on the node.socket file)

  • Fix permission of node.socket if it exists in smoke test workflow.

Test run with these changes: https://github.com/input-output-hk/hydra/actions/runs/8230455844


  • CHANGELOG updated or not needed
  • Documentation updated or not needed
  • Haddocks updated or not needed
  • No new TODOs introduced or explained herafter

Copy link

github-actions bot commented Mar 11, 2024

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2024-03-11 09:26:29.834329727 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Script summary

Name Hash Size (Bytes)
νInitial bccf2a430c016bc960fbf31b02694011cd399d20da8882aac9d33611 4110
νCommit 56b0f0b597150e619c76bed60683f3b1e42d7bc0685ed951b882bfc5 1975
νHead 86bff95ba20e9d1d1b34899a56d86bbacc9fed999260b27dcc92d128 9351
μHead 88f533cf67cd0fc93d7d9ccf0a8b1d69ffd1208a825efbebbc1d36ba* 4213
  • The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 4792 8.97 3.44 0.46
2 4995 10.95 4.20 0.49
3 5194 12.66 4.84 0.52
5 5597 16.62 6.37 0.58
10 6604 26.21 10.04 0.72
48 14249 99.85 38.32 1.86

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 594 10.24 4.04 0.29
2 784 13.88 5.64 0.34
3 972 17.66 7.29 0.39
5 1344 25.66 10.74 0.49
10 2284 48.19 20.30 0.79
19 3964 97.83 40.79 1.41

Cost of CollectCom Transaction

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 57 544 16.52 6.53 0.35
2 113 654 26.90 10.72 0.47
3 170 764 39.81 15.96 0.62
4 227 874 52.75 21.34 0.77
5 283 984 69.85 28.31 0.96
6 338 1095 86.73 35.34 1.15

Cost of Close Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 554 8.05 3.84 0.27
2 817 9.49 6.05 0.31
3 816 9.17 6.33 0.31
5 1202 11.44 9.16 0.36
10 2058 17.01 16.04 0.49
50 4632 38.23 23.08 0.83

Cost of Contest Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 626 8.60 4.71 0.28
2 797 9.83 6.15 0.31
3 898 10.06 6.86 0.32
5 1267 12.33 9.68 0.38
10 1983 16.46 15.41 0.48
50 8101 50.64 63.10 1.37

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 4669 17.05 7.37 0.55
2 4788 27.78 12.11 0.67
3 4948 42.14 18.50 0.84
4 5062 57.33 25.23 1.02
5 5207 76.25 33.52 1.24
6 5399 94.60 41.63 1.45

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
5 0 0 4627 7.85 3.28 0.44
5 1 57 4662 9.00 3.99 0.46
5 5 285 4797 13.61 6.84 0.52
5 10 568 4965 19.25 10.35 0.60
5 20 1138 5306 30.90 17.53 0.76
5 30 1706 5645 42.22 24.58 0.92
5 40 2275 5984 53.96 31.81 1.08
5 50 2847 6326 65.80 39.07 1.24
5 79 4496 7309 98.79 59.60 1.70

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2024-03-11 09:28:51.034245383 UTC

Baseline Scenario

Number of nodes 3
Number of txs 9000
Avg. Confirmation Time (ms) 22.591252441
P99 54.895617670000036ms
P95 31.550057199999998ms
P50 20.6368825ms
Number of Invalid txs 0

Baseline Scenario

Number of nodes 1
Number of txs 3000
Avg. Confirmation Time (ms) 5.093810399
P99 8.839801049999995ms
P95 6.726090099999999ms
P50 4.912278499999999ms
Number of Invalid txs 0

Copy link

github-actions bot commented Mar 11, 2024

Test Results

417 tests  ±0   409 ✅ ±0   13m 51s ⏱️ -18s
138 suites ±0     8 💤 ±0 
  5 files   ±0     0 ❌ ±0 

Results for commit 6a6c1ca. ± Comparison against base commit cf5223e.

♻️ This comment has been updated with latest results.

@ch1bo ch1bo changed the title Add traces to hydra-cluster binary Small smoke test fixes Mar 11, 2024
ch1bo added 2 commits March 11, 2024 10:23
This should give us more insights into when the smoke test fails (e.g.
due to permission errors on the node.socket file)
This is needed in case the docker-based cardano-node is already running
and we use an existing host-mounted node.socket file.
@ch1bo ch1bo force-pushed the smoke-test-traces branch from 9a83975 to 6a6c1ca Compare March 11, 2024 09:23
@ch1bo ch1bo requested a review from a team March 12, 2024 10:36
@v0d1ch v0d1ch merged commit 34d2890 into master Mar 12, 2024
26 checks passed
@v0d1ch v0d1ch deleted the smoke-test-traces branch March 12, 2024 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants