Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark list utilities and alter if needed #79

Closed
TotallyNotChase opened this issue Jan 7, 2022 · 2 comments · Fixed by #87 or #99
Closed

Benchmark list utilities and alter if needed #79

TotallyNotChase opened this issue Jan 7, 2022 · 2 comments · Fixed by #87 or #99
Assignees

Comments

@TotallyNotChase
Copy link
Collaborator

Originally raised from this comment.

We want to make sure the list utilities are sufficiently efficient, as they will be the basis for many ledger API functions. In particular, I'm concerned about-

  • pelemList was benefitted by taking in Haskell level functions rather than Plutarch level functions. Are there other list utilities that might benefit from this?

  • Are there any work duplication hidden in plain sight, within the abstractions?

    Example of fixing work duplication.

Some essentials about Haskell level functions vs Plutarch level functions: #78

@srid
Copy link
Member

srid commented Jan 7, 2022

Note that Plutus provides some benchmark utilities, including integrating with criterion for reports. I started working on it here (not fully done). See Plutonomicon/pluton#26

Eventually I would like us to be able to produce reports like this for every PR to this repo, so we can assess the performance impact of changes. @MatthewCroughan and I talked about this a bit before

The Pluton repo generates a .csv report in CI:

Wrote to bench.csv:
   0.787 examples:contract:gift:haskell   ExCPU 177160813  ExMemory 411600  ScriptSizeBytes 2752
   0.787 examples:contract:gift:pluto     ExCPU 30880012   ExMemory 76830   ScriptSizeBytes 806 
   0.787 examples:contract:gift:plutarch  ExCPU 34074788   ExMemory 80789   ScriptSizeBytes 810

(This ^ is not using Plutus criterion yet)

https://hercules-ci.com/accounts/github/Plutonomicon/derivations/%2Fnix%2Fstore%2Fdc1z739d9x964bdi3ndl0xscvzjpnxny-benchmark.drv/log?via-job=fcf8c542-5726-4e95-bf91-8732466e7c7c

Which .csv can then be integrated with GitHub PR interface.


I'm curious how you plan to work on this, @TotallyNotChase - and if you need any assistance or guidance from me.

@TotallyNotChase
Copy link
Collaborator Author

The plan is to do pretty basic benchmarks for singular list utilites one by one. The evaluation machine seems pretty consistent and predictable so it should be sufficient in this simple case. We also have a production level plutarch script that uses some of the utilities - so I sometimes cross reference there.

I really really really want that benchmark suite you proposed, for every big Plutarch change - it'd be really useful given Plutarch's goals. Definitely something to think about. I don't know if we should have it here of in pluton - but whatever works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants