Resuming the progress on native HTTP/2 support

[ellie-idb]

Description

HTTP/2-compliant libraries are everywhere (several libraries already exist for Rust, Haskell, Ruby, hell, even Lisp)... however, D does not have a single production-ready & user friendly library that can be used.

Indeed, the Hunt framework does have a full implementation (and hats off to those guys), but.. from my personal experience, there's several gaping flaws with it:

• Using their library feels as if you're writing everything as if it was Java. The library is not nearly as user-friendly as vibe.d, and feels as if you're pulling teeth every time you need to debug a fatal error. (IIRC, when I was using it to try and build a gRPC server, I had to sift through a bunch of undocumented code)
• Using it also requires that a developer pull in their entire standard library in order to get anywhere with it. hunt-http depends on hunt-net, hunt-net depends on hunt... you get the gist. It's nowhere near ideal, and combining vibe.d + Hunt looks like it's a recipe for disaster and a bloated binary.

vibe.d hasn't had much progress on that front either. The pull request for merging HTTP/2 into mainline vibe.d (vibe-d/vibe-d#1160) has all but stagnated in progress and has been open for 5 years now with the last activity occurring nearly a year ago.

What are rough milestones of this project?

• Connecting libhttp2 with vibe.d
• Exposing a lot of the lower-level intrinsics to developers interested on expanding on top of HTTP/2
• Ensuring full compatibility with the HTTP/2 spec

How does this project help the D community?

HTTP/2 is vastly more efficient then the textual-based HTTP/1. This would allow for more widespread adoption of vibe.d (as vibe.d is already incredibly user-friendly and has tons of features to offer) while reducing the overhead associated with such, and providing incredible performance gains (real-world applications have seen reductions of page load times ranging from 20-80%), and would pull the web-developer segment of the community into the present.
As well, with a native HTTP/2 implementation, a full native implementation of gRPC would finally be possible (as it stands, the only two are hunt-grpc and mine), allowing for better interop between languages and web services.

Recommended skills

Familiarity with the vibe.d code-base, familiarity with HTTP, grit :P

References

HTTP/2 spec
libhttp2

[GallaFrancesco]

tl;dr: The vibe-http module supports HTTP/2 server side and can be used by pulling its dub package. The implementation lacks some features and introduces some (possibly breaking) changes with respect to the vibe-d:http codebase. This is why development has been excessively slow in the last year, yet the plan to include vibe-http as a complete replacement of the current module still holds.

---

During SAoC 2019 we (@s-ludwig and I) worked on a new implementation of the whole vibe-http module, which includes support for HTTP/2 server-side:

https://github.com/vibe-d/vibe-http

The codebase is compliant with the HTTP/2 RFC you are referring to. Unfortunately I currently lack the time to implement the missing features:

• HTTP/2 support client-side
• Stream Prioritization algorithm(s)
• A configurable HTTP/2 settings interface (currently the server follows the default settings of the RFC, it would be nice to expose the struct containing those settings to the end-users of the library.

Moreover, the reason behind vibe-http was to improve some aspects of the current vibe-d:http codebase. For this reason, the underlying data structures implementing the HTTP server / client have been modified. This means that, for the current implementation to be included in vibe.d, a significant amount of changes must be ported from the vibe-d:http module to vibe-http.

Anyway, if you are interested in using / improving / experimenting with the library, there's a separate dub package and a fair amount of documentation (see resources).

Resources:

• source code: https://github.com/vibe-d/vibe-http
• dub package: https://code.dlang.org/packages/vibe-http
• rejectedsofware.org forum thread detailing some of the new features of vibe-http
• video of me documenting the vibe-http library at DConf 2019 (featuring maccheroni english): https://youtu.be/nk9gQWRvoXM
• slides of the above presentation: http://dconf.org/2019/talks/galla.pdf
• (minimal) example of a webserver supporting HTTP/2 with TLS: https://github.com/GallaFrancesco/break-my-vibe/blob/master/source/app.d
• last update on project status: What is the status of this project vibe-d/vibe-http#21

[zoujiaqing]

HTTP/2 project already exists.

hunt-http already complete support HTTP 1.1 and HTTP/2 protocols.
grpc-dlang is a gRPC framework implemented entirely in D.

Resource:

dub package: https://code.dlang.org/packages/hunt-http
dub package: https://code.dlang.org/packages/grpc