opam-ci-check lint is surprisingly slow when checking dozens of packages

[shonfeder]

Using the linter on dra27/opam-repository#20, which has 47 packages, took about 1 minute. This surprised me, as I don't think the linting is doing anything computationally expensive.

We likely have some room for low-hanging perf improvements. Perhaps this would start with making the executable run tasks concurrently, but some benchmarking should guide us.

This is low-priority, and not in scope for this quarter, IMO, since it is not a problem for 99% of package publication use cases, and never going to be bottleneck on the CI for publication of large packages. I'm just opening this to record the finding :)

[punchagan]

Using the linter on dra27/opam-repository#20, which has 47 packages, took about 1 minute. This surprised me, as I don't think the linting is doing anything computationally expensive.

Is a bulk of the time being spent in downloading the sources of the packages? (for the dune checks?) Doing that concurrently could help...

[punchagan]

I was testing for #405 and ran into this myself too.

I didn't do any detailed benchmarking but from crude printf benchmarking, it looks like we spend a bunch of time downloading the package sources and in running opam-lint with --check-upstream which checks if the upstream archive is available and the checksum matches. Like you said, running these tasks concurrently should easily give us some improvements.