RFC: Added libdeno package that compiles deno to a static C library

[alshdavid]

Note: This is a rough POC that aims to demonstrate a statically linkable "batteries included" Deno library. The intention is to get the team's thoughts and if it's a direction you like, I'm happy to take on the development

What is this?

This PR adds two crates to the Deno project

• libdeno
  • Provides Deno functionality with a C FFI via a static C library
• libdeno_bindings
  • Provides high level Rust bindings for the C FFI
  • Maybe worth splitting into libdeno_sys and libdeno_embeded for C FFI and high level bindings respectively
• Updates cli to re-export initialization circuits
  • This currently duplicates a bunch, but would be refactored if the team likes the direction

Why?

Deno is a phenomenal JavaScript runtime and many projects would like to embed Deno functionality into their applications. Use cases for this are;

• FaaS platforms
• Game engines that want to use JavaScript capabilities
• Bundlers
• Databases that offer a JavaScript API for consumers

Admittedly these are mostly my own personal and professional use cases, haha - but the point still stands

Current State

Deno currently exposes several crates, such asdeno_runtime deno_core etc. These crates can be composed by the consumer to build a custom runtime for their use case.

The challenge is that these crates do not contain the standard library, resolution logic, or Nodejs compatibility.

An approximation of Deno can be built by manually stitching together crates like deno_node deno_web and oxc_resolver, however the maintenance burden is high and there are often versioning clashes (in the case of bundlers, swc in deno_ast).

What this aims to solve

The idea with this PR is to offer a "batteries included" static C library that can be embedded within a consumer's binary and called to execute JavaScript as needed.

This is language independent, but with a focus on Rust consumers.

A static C library eliminates clashes of crate dependency versions.

This can also be distributed as a dynamic C library (.so .dylib .dll)

What about customization / ops / snapshots?

Snapshots can be loaded using v8 args.

Deno's ops API might be portable to a C FFI.

Deno supports Node.js's napi - which allows the reuse of the work done on the napi-rs project by a useland library (not self promotion, but I achieved a similar outcome by combining libnode and napi-rs - which I'd love to replace with an embedded Deno library)

Demo

Example consumer:

// main.rs

fn main() {
    let exit_code = libdeno_bindings::deno_embedded_eval("console.log(42)");
    println!("Exited with {}", exit_code);
}

The idea is to provide the eval and run commands via an FFI. The API would be expanded to include permissions and flags. This, combined with native extensions via n-api, solves most use cases.

This can later be expanded to allow the direct creation of "Workers" and ideally v8 contexts on a Worker.

Currently the POC only implements eval without flags or permissions, but you get the idea.

[CLAassistant]

All committers have signed the CLA.

[alshdavid]

cli contains all of the core Deno logic for the executable. Because the main target for Deno is a binary and I need access to the internals - I created a library (not to be published) that re-exports the parts needed to instantiate Deno.

[alshdavid]

There were some exported items that were public but their internals private

[bartlomieju]

Thanks for the PR, we discussed this internally and are not keen on doing this.

Once such integration becomes official it makes it very hard for us to change internals of Deno because of API stability concerns to end users.

We're already publish a lot of crates that provide libraries that make up Deno, so we feel this is enough for users to assemble their own library if needed.

[alshdavid]

Thanks for the PR, we discussed this internally and are not keen on doing this.

Once such integration becomes official it makes it very hard for us to change internals of Deno because of API stability concerns to end users.

We're already publish a lot of crates that provide libraries that make up Deno, so we feel this is enough for users to assemble their own library if needed.

Thanks, appreciate your consideration.