Upstream OpenSSL wrap

[nazar-pc]

Conversation at openssl/openssl#16812 finally converged on considering to add official (optional for now) Meson support upstream.

There are two major steps seeing there:

• Consider adding generated sources into OpenSSL release or source tree openssl/openssl#16188 that will serve as a single source of build sources for multiple build systems without relying on Perl when compiling from official OpenSSL release
• Consider adding Meson build system support openssl/openssl#16812 to use ^ for Meson build system

Essentially right now OpenSSL build system uses a non-trivial generator to figure out what sources and other things should be used depending on target platform, which causes challenges for end users trying to compile it, especialy when they don't want to rely on Perl.

By abstracting it in reusable way and generating during release process upstream, it would be much easier to pick pre-generated files by secondary build system like Meson.

Creating this issue downstream as this is where most of knowledgable engineers are who already maintain OpenSSL wrap.

I do not have the time to do this myself in the near term, but happy to collaborate with someone/help with review to make sure things are moving forward.

[nirbheek]

CC: @amyspark @benoit-pierre @glimchb @hrxi

[amyspark]

Hi @nazar-pc , I maintained this OpenSSL 1.1.x wrap while the branch was supported: https://github.com/amyspark/wrapdb/tree/amyspark/openssl

Is there any way I can help? What do you need?

[nazar-pc]

Check the first item listed to be done above, that is where we need to start.

The biggest challenge is that you can't simply compile OpenSSL. The first step is to run Perl scripts that generate assembly and other stuff specific for configured platform. We need to make OpenSSL generate those as part of CI such that it can be included in the official OpenSSL release. That way no one else will have to use Perl when they compile OpenSSL downstream.

Once that is done, we'll need to create a source of files to compile that both current OpenSSL build system could use to decide what to compile and Meson build system as well.

I think those are two immediate next steps, but they are very non-trivial and will require upstream buy-in.

[amyspark]

We need to make OpenSSL generate those as part of CI such that it can be included in the official OpenSSL release

This should be covered by porting https://github.com/nodejs/node/commits/v20.x/deps/openssl 's bindings generator to OpenSSL itself. My wrap was able to drive this generator from within Meson, so I think we could use that as a start.

[nazar-pc]

Upstream wrap also is able to do that. However, note that for it to be upstreamable we need to ensure to cover different scenarios. There are various OpenSSL options that can be used to tweak its behavior and they need to be somehow exposed. Ideally we'd generate sources for the platform once and control the rest with defines.

Also I would recommend to focus on x86-64 Linux/macOS/Windows first and ignore other platforms for now to make diff easier to review (maybe add aarch64 if you feel adventurous). Once approach is agreed with upstream and initial code lands, code generation can be expanded to other platforms. Upstream maintainers may have a different opinion on this, but this is how I would approach it to reduce friction.

[amyspark]

So far I've found the following. There are broadly three classes of generated files:

• includes
• Assembly files
• configuration headers

Neither the includes nor the assembly headers depend on the configuration, they seem to be generated from Perl for convenience. What does depend on the architecture is the output format of the assembly files, the rest is controlled with defines.

The configuration headers do depend on the result of Perl's execution, so I guess it'd be useful to make it as cmakedefine ish as possible.

[amyspark]

maybe add aarch64 if you feel adventurous

So far as I can see, architectures and operating systems do not seem to be of concern here. Given the structure of the Configure script, pregenerating one or all OSes is just a matter of iterating over format flags.
I believe the real obstacle is the tight coupling of the generation process to the Configure system; were these files tied to a Makefile script, this would probably be much easier.

[nazar-pc]

I believe the real obstacle is the tight coupling of the generation process to the Configure system; were these files tied to a Makefile script, this would probably be much easier.

Iterating and configuring them one at a time is probably good enough initially like Node.js does it: https://github.com/nodejs/node/blob/b4e8f1b6bb3616ba222c4513218aa1fa9d543d8e/deps/openssl/config/Makefile