WASM feature selection at parse/validation time

[jmillikin]

(originally dylibso/chicory#768)

When WebAssembly is used as a plugin system it's desirable to be able to control which extensions to the base WebAssembly v1.0 spec are allowed. Otherwise the different levels of WebAssembly feature support between versions can cause plugins to accidentally depend on a feature that's only supported on the most recent version of the host program.

Imagine a plugin developer is writing WebAssembly plugins for their IDE, which uses Endive. One day they publish a new version of their plugin and some users complain that it's broken. After some testing they realize their updated C toolchain is auto-vectorizing some loops, which means it requires an Endive version with SIMD, which after some investigation is found to only ship in IDE release v2025.01 and later. If Endive allowed the WebAssembly feature set to be explicitly selected then the IDE could have configured it to be "just v1.0", so the toolchain issue would have been caught early. Updating WebAssembly feature support (e.g. v1.0 -> v2.0) would be more legible to the host application.

Similar issues also exist for ecosystems with multiple WebAssembly implementations in use, for example Endive for JVM host applications and one written in C++ for native applications. If Endive supports the GC proposal and the C++ impl doesn't, then plugins that require GC will work in some host applications but not others (with no easy way for a plugin user to realize what's going wrong).

Would it be possible to add some sort of "WebAssembly feature" enum, and let the parser builder have a set of enabled features? By default it would support everything, but calling .feature(Feature.V_1_0) would cause validation errors for non-MVP features, .feature(Feature.SIMD) would enable SIMD support, etc. Obviously adjustments to the API are fine, I'm more interested in the functionality than the exact shape.

Proposed subsets to target: V_1_0, V_2_0, V_3_0, LIME_1. Then add whatever additional per-feature selections the Endive devs feel would be reasonable to target by themselves.

[andreaTP]

Hi @jmillikin , thanks for bringing this up and good to see you around! 🙂

base WebAssembly v1.0 spec

we currently have support for a large subset of v3.0

with no easy way for a plugin user to realize what's going wrong

the error will point out to the specific op-code, I see room for improvements here.

To spell it out, the philosophy behind Endive has been: "reach full spec conformance so that feature flags are un-necessary".
I'm a bit afraid of what the maintenance of this feature looks like(especially given the different backends interpreter/compiler/redline).

Since this proposal can be implemented as an external tool I'd like to propose an incubation period, experimenting with it in a separate repository(I can help with the setup/publishing/etc. and host it in the https://github.com/roastedroot org when requested).
The functionality can be mentioned in the official docs ( like we do for Gradle ).
wdyt?

[jmillikin]

I'd prefer to avoid an external tool because then I'd have to rewrite that tool in Java and turn it into a library, and if I hand-wrote a big switch statement then I'd have to read enough of the WebAssembly spec to figure out exactly which opcodes + parameter arity correspond to the different WebAssembly versions. This isn't impossible and maybe an LLM could offload the work, but it sure would be nice if the Java library for WebAssembly stuff could handle this.

edit: I'm hopeful that the migration to the Bytecode Alliance organization might enable a broader scope for Endive than being a pure runtime, i.e. including some analysis functionality such as this that is useful for some runtime use cases.

To spell it out, the philosophy behind Endive has been: "reach full spec conformance so that feature flags are un-necessary".

The challenge is that the spec is evolving; Endive v1.0 might have 100% spec conformance, and a few years later v1.20 is also 100% spec conformant, but the definition of that spec has shifted.

This doesn't matter for users that want to use Endive purely to run your own WebAssembly modules, or power an evergreen SaaS-style platform. But if you want to embed it into a tool with its own compatibility story then being able to pin the specific WebAssembly feature set is useful.

The API doesn't have to be super fancy, even a simple boolean "does this loaded module require features not available in WebAssembly version X" would work fine.

[andreaTP]

I hand-wrote a big switch statement

That's what I'm afraid of too, embarking in something that requires a lot of time/maintenance to sustain while the current contributor base is limited.

Anyway, thinking about it out of the box, I put together #47 as a proposal(easier to get it in code than explaining).
I know is slightly different than your request, but this is very very low maintenance(for us) and I'd be comfortable in moving it forward.
Let me know what you think!