Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace hlint shell job with check derivation #1308

Merged
merged 1 commit into from
Feb 21, 2024

Conversation

locallycompact
Copy link
Contributor

@locallycompact locallycompact commented Feb 14, 2024

This PR introduces lint-utils to demonstrate two principles.

  1. to ensure consistency between local and remote builds
  2. to not require no-op jobs to rerun

It does this by way of introducing a check derivation that produces no functional output, but will fail if the check is not met.

lint-utils is well supported by multiple people and in use commercially by third-parties


  • CHANGELOG updated or not needed
  • Documentation updated or not needed
  • Haddocks updated or not needed
  • No new TODOs introduced or explained herafter

Copy link

github-actions bot commented Feb 14, 2024

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2024-02-21 20:35:30.146508674 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Script summary

Name Hash Size (Bytes)
νInitial bccf2a430c016bc960fbf31b02694011cd399d20da8882aac9d33611 4110
νCommit 56b0f0b597150e619c76bed60683f3b1e42d7bc0685ed951b882bfc5 1975
νHead 86bff95ba20e9d1d1b34899a56d86bbacc9fed999260b27dcc92d128 9351
μHead 88f533cf67cd0fc93d7d9ccf0a8b1d69ffd1208a825efbebbc1d36ba* 4213
  • The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per Head.

Cost of Init Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 4794 9.06 3.47 0.46
2 4996 10.85 4.15 0.49
3 5197 12.60 4.82 0.52
5 5599 16.53 6.33 0.58
10 6604 26.27 10.07 0.73
48 14244 99.87 38.32 1.86

Cost of Commit Transaction

This is using ada-only outputs for better comparability.

UTxO Tx size % max Mem % max CPU Min fee ₳
1 592 10.24 4.04 0.29
2 777 13.88 5.64 0.34
3 968 17.66 7.29 0.39
5 1345 25.66 10.74 0.49
10 2279 48.19 20.30 0.79
19 3969 97.83 40.79 1.41

Cost of CollectCom Transaction

Parties UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
1 57 544 16.56 6.55 0.36
2 112 653 26.81 10.68 0.47
3 171 764 37.70 15.19 0.60
4 226 874 50.43 20.45 0.74
5 282 984 66.39 27.05 0.92
6 339 1099 87.80 35.74 1.17

Cost of Close Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 595 8.34 4.61 0.28
2 735 9.04 5.67 0.30
3 861 9.75 6.73 0.32
5 1176 11.52 9.17 0.36
10 1990 16.35 15.53 0.48
50 8049 48.15 61.85 1.34

Cost of Contest Transaction

Parties Tx size % max Mem % max CPU Min fee ₳
1 570 8.24 4.39 0.27
2 762 9.30 5.77 0.30
3 1004 10.88 7.54 0.34
5 1169 11.35 8.94 0.36
10 1980 16.42 15.38 0.48
50 8014 51.78 63.46 1.38

Cost of Abort Transaction

Some variation because of random mixture of still initial and already committed outputs.

Parties Tx size % max Mem % max CPU Min fee ₳
1 4688 17.00 7.36 0.55
2 4783 27.78 12.11 0.67
3 4905 40.96 17.92 0.83
4 5125 58.84 25.95 1.04
5 5133 71.70 31.37 1.18
6 5400 98.31 43.31 1.49

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties UTxO UTxO (bytes) Tx size % max Mem % max CPU Min fee ₳
5 0 0 4628 8.27 3.45 0.45
5 1 57 4661 9.00 3.99 0.46
5 5 284 4796 13.82 6.93 0.52
5 10 570 4967 19.58 10.49 0.60
5 20 1139 5306 30.48 17.36 0.76
5 30 1709 5648 42.64 24.76 0.92
5 40 2275 5984 53.96 31.81 1.08
5 50 2844 6323 65.50 38.95 1.24
5 79 4492 7304 98.58 59.51 1.70

End-To-End Benchmark Results

This page is intended to collect the latest end-to-end benchmarks results produced by Hydra's Continuous Integration system from the latest master code.

Please take those results with a grain of salt as they are currently produced from very limited cloud VMs and not controlled hardware. Instead of focusing on the absolute results, the emphasis should be on relative results, eg. how the timings for a scenario evolve as the code changes.

Generated at 2024-02-21 20:38:06.038979975 UTC

Baseline Scenario

Number of nodes 3
Number of txs 9000
Avg. Confirmation Time (ms) 23.097655885
P99 89.08291844000249ms
P95 33.096822049999986ms
P50 20.640029ms
Number of Invalid txs 0

Baseline Scenario

Number of nodes 1
Number of txs 3000
Avg. Confirmation Time (ms) 4.348328514
P99 7.771025639999994ms
P95 5.610061349999999ms
P50 4.1616859999999996ms
Number of Invalid txs 0

Copy link

github-actions bot commented Feb 14, 2024

Test Results

418 tests  ±0   409 ✅ ±0   14m 16s ⏱️ +13s
139 suites ±0     9 💤 ±0 
  5 files   ±0     0 ❌ ±0 

Results for commit 82633f7. ± Comparison against base commit f496652.

♻️ This comment has been updated with latest results.

@ch1bo ch1bo changed the title Repce hlint shell job with check derivation Replace hlint shell job with check derivation Feb 15, 2024
@ch1bo ch1bo requested review from v0d1ch, ffakenz and a user February 15, 2024 07:50
@ch1bo ch1bo force-pushed the lc/lint-utils-hlint branch from a58a3bb to e51af42 Compare February 15, 2024 07:50
flake.nix Outdated Show resolved Hide resolved
@@ -33,6 +37,9 @@
(system:
let
pkgs = import inputs.nixpkgs { inherit system; };
tools = {
hlint = hydraProject.pkgs.haskell-nix.tool hydraProject.compiler "hlint" "3.8";
};
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer it the other way around. i.e. put tools into the hydraProject expression (if this is possible?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would first bring the pkgs expression up to the flake.nix before attempting that. I don't think there is a need to match on more than one pkgs.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you mean the haskell.nix enhanced packages (here hydraProject.pkgs) merged into the nixpkgs pkgs?

That would make sense to me as well. It just seemed odd that the hydraProject.compiler is passed into this expression again, while it would be already available in there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, resolve the pkgs expression first with the iohk overlays included as the first binding and then use that in the whole program.

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems very nix specific, what's the impact for anyone that would want to use our stuff without using nix?

@locallycompact
Copy link
Contributor Author

This seems very nix specific, what's the impact for anyone that would want to use our stuff without using nix?

They can use hlint in the shell normally, but we could take steps to display what shell command are being run by the check in the CI.

@ch1bo
Copy link
Collaborator

ch1bo commented Feb 19, 2024

This seems very nix specific, what's the impact for anyone that would want to use our stuff without using nix?

I think there is no direct impact, other than the CI checking the code (already enabled) and users needing to follow suit before it's accepted. So maybe we should add instructions how to lint to CONTRIBUTING.md like we also do with formatting. This could include a link to (ideally a single) place where tool versions are visible, e.g. here

@locallycompact locallycompact merged commit 26b93f8 into master Feb 21, 2024
21 checks passed
@locallycompact locallycompact deleted the lc/lint-utils-hlint branch February 21, 2024 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants