CI costs too much time when building ocjs

[voldikss]

I inspected the history of the GitHub Action of the project, surprisedly found that building ocjs job would cost so much time. Are there some ways we could take to decrease that?

[donalffons]

Yeah, CI costs are extremely high at the moment. The ~5 hour builds actually happened on a compute instance with 56 cores, if I remember correctly. So, that's not great.

I've been doing lots of prototyping over the last couple of months to address this and other problems with the library (will write some sort of blog post on that, soon). For improving build times, here's roughly what I have in mind:

1. Don't rely on Docker layers for caching. Instead, make a proper monorepo, split the code up into several packages and use a caching system like turbo.
2. For generation of the binding code, stop using libclang for python and instead use libclangjs. I hope is that this will have better performance, but haven't measured anything yet.
3. Reduce the number of #include statement in each binding code chunk or use a single chunk for all bindings.

Step 1 is probably going to have the biggest impact and should be doable without major rewrites.

[voldikss]

@donalffons may i ask, would you plan to use cmake & emcmake buildkit?

[donalffons]

Yes, I'm definitely planning to move back to using OCCT's CMake build system, soon. I have been doing some prototyping locally with CMake and it's much better than how it's currently handled in this project.
I'm also planning to publish .a libraries and headers of OCJS (along with an example CMake project) that will make it much more convenient and faster to customize the library (adding / removing interfaces) than the current "custom build" workflow.

I'm currently working on collecting and prioritizing all these improvements. I'm planning to have everything ready to start making another round of iterations / improvements in a few weeks, I hope.

In case you have more suggestions - keep posting them!

[voldikss]

Hi @donalffons I noticed the CI runner of this project is self-hosted, could you tell me about the detail configuration of that machine(cpu, mem, OS, .etc)?

I made some changes to occt source code, tried compiling my customzied opencascade.wasm on my laptop which has 32Gi memory but failed. Then I feel I should look for a higher performance machine.

Any thing about self-hosted runner machine would help a lot, thanks!

[donalffons]

You need at least 40-50 Gigs of memory to run an optimized -O3 build. If your system doesn't have that, consider using swap space or a similar workaround.