Using WASM instead of custom virtual machine?

[vlovich]

I was actually toying around with building a Rust database not too long ago & the interesting thought I had was about using WASM w/ a custom API instead of a VM to execute queries. The mental model I had was one of LLVM IR - that would make this suitable as a general database layer that's potentially accessible from any language. The JIT part of WASM is a nice "free" feature you get although I don't think it would meaningfully add performance but I think having a language-agnostic target API to build against could be interesting, letting any language target their queries against the runtime (i.e. write the database code in the language of your choice with domain-specific optimizations rather than using plain text SQL).

The open design question I didn't explore is what does the API look like & do you put the plan optimizer as part of the SQL layer or if you have the WASM API be the plan optimizer. I think the latter is probably the easiest to bring up while the former offers the largest opportunity for exploring greenfield ideas in this space.

I know there's exploration of integrating WASM for stored procedures, but you would get that implicitly for free if it was WASM at the lowest layers.

On the other hand, this could be a terrible idea. Wonder what your thought on this architecture is.

[joaoh82]

I like this idea. Let's explore a bit more. About the open design question I would put the plan optimizer as part of the SQL layer instead in the WASM API. It seems to be right place for it. Also like you said, it would offer a largest opportunity to explore greenfield ideas, which I am always for it.

I always wanted to explore WASM a bit more. So if you are up for it, let's do it.

[vlovich]

I'm down to explore. I'm a decent engineer but have absolutely no background designing databases for real, so I think the best results might come from pairing with someone with some experience who can handle the design aspect & I can just be the code monkey to start with.

[joshbenaron]

Hey, it would be great if you could somehow compile sqlrite down to a wasm file. I work on a distributed compute blockchain which only runs wasm, and frankly C is just painful to work with. I'm happy to help out with this

[joaoh82]

That is actually a really good idea.

[joaoh82]

Btw, I created a discord server for the project so we can start have a more active conversation and build a community. Here is the link: https://discord.gg/zHXUG66X .