Cache some of the build using Docker

[mjp41]

Based on my experiments for snmalloc's CI. I have quickly knocked together a similar approach to enable sharing across build steps, and increase parallelism.

@derSteFfi I in no way mean this to replace your work on #243. I think these could be used together. Though, with what you have ccache would probably work better than docker here.

[derSteFfi]

yeah, the two ideas seem somewhat orthogonal. Currently, your draft only builds mi, right? And also only runs mi? Because that goes into the same direction I proposed in #138: do we really need to test all allocs against all tests?

I'm curious (and don't know a lot about docker): how does docker cache artifacts? If we build all allocs and benches in a container, do we catch compile problems on specific distros? Still trying to make up my mental model about this... :)

[mjp41]

yeah, the two ideas seem somewhat orthogonal. Currently, your draft only builds mi, right? And also only runs mi? Because that goes into the same direction I proposed in #138: do we really need to test all allocs against all tests?

I'm curious (and don't know a lot about docker): how does docker cache artifacts? If we build all allocs and benches in a container, do we catch compile problems on specific distros? Still trying to make up my mental model about this... :)

So each layer of the dockerfile gets cached. The aim is to do something like

For each platform

• build all benchmarks
• For each Allocator
  • build the allocator
  • run all benchmarks with that allocator

The current version does one platform ubuntu:24.04 and two allocators, and runs one benchmark cfrac. I will expand this once it works. This would give a lot of parallelism to run the benchmarks for all the allocators, but in parallel across a bunch of machines.

[mjp41]

Though none of the caching is doing what I expected.

[mjp41]

With a bit of fiddling it gets to something like

[jvoisin]

Suggested change

	uses: actions/checkout@v3
	uses: actions/checkout@v4

[mjp41]

@jvoisin are you happy for this to replace the existing CI? I think it covers everything that was working in the previous CI. There are more things that should be fixed, but I think getting back to a Green CI is pretty useful.

It would be good to factor the scripts such that changes to allocators doesn't require the benchmark building layer to be rebuilt. Maybe the Makefile in #243 can help with that. I am wondering if there should be a top level structure like

  /common
    ... common scripts for both benchmarks and allocators
  /benchmarks
    ... scripts and patches for building benchmarks
  /allocators
    ... scripts and patches for building allocators

Then build-bench-env would call scripts in these bits for compatibility, but the Docker file could do them directly, so its dependencies didn't rebuild everything if we just change the version number of snmalloc.