wasi-http(p3): Consider exposing a knob to control outgoing body buffering

[saulecabrera]

Currently the http Preview 3 implementation does not provide a way of controlling the outgoing body buffering, which results in keeping the host/guest in lock-step, potentially affecting components that perform multiple writes to the body.

Compare the p3 implementation at https://github.com/bytecodealliance/wasmtime/blob/main/crates/wasi-http/src/p3/body.rs#L333, with the p2 implementation at https://github.com/bytecodealliance/wasmtime/blob/main/crates/wasi-http/src/p2/body.rs#L417

Consider the p3_http_echo.rs test program currently in-tree:

use {
    test_programs::p3::{
        service::exports::wasi::http::handler::Guest as Handler,
        wasi::http::types::{ErrorCode, Request, Response},
        wit_future, wit_stream,
    },
    wit_bindgen::StreamResult,
};

struct Component;

test_programs::p3::service::export!(Component);

impl Handler for Component {
    /// Return a response which echoes the request headers, body, and trailers.
    async fn handle(request: Request) -> Result<Response, ErrorCode> {
        let headers = request.get_headers();
        let (_, result_rx) = wit_future::new(|| Ok(()));
        let (body, trailers) = Request::consume_body(request, result_rx);

        let (response, _result) = if headers
            .get("x-host-to-host")
            .into_iter()
            .any(|v| v == b"true")
        {
            // This is the easy and efficient way to do it...
            Response::new(headers, Some(body), trailers)
        } else {
            // ...but we do it the more difficult, less efficient way here to exercise various component model
            // features (e.g. `future`s, `stream`s, and post-return asynchronous execution):
            let (trailers_tx, trailers_rx) = wit_future::new(|| todo!());
            let (mut pipe_tx, pipe_rx) = wit_stream::new();

            wit_bindgen::spawn_local(async move {
                let mut body_rx = body;
                let mut chunk = Vec::with_capacity(1024);
                loop {
                    let (status, buf) = body_rx.read(chunk).await;
                    chunk = buf;
                    match status {
                        StreamResult::Complete(_) => {
                            chunk = pipe_tx.write_all(chunk).await;
                            assert!(chunk.is_empty());
                        }
                        StreamResult::Dropped => break,
                        StreamResult::Cancelled => unreachable!(),
                    }
                }

                drop(pipe_tx);

                trailers_tx.write(trailers.await).await.unwrap();
            });

            Response::new(headers, Some(pipe_rx), trailers_rx)
        };

        Ok(response)
    }
}

// Unused function; required since this file is built as a `bin`:
fn main() {}

Running the component through wasmtime serve:

./target/release/wasmtime serve -Wcomponent-model-async -Sp3,cli -O pooling-allocator  --addr=127.0.0.1:8093 <component.wasm>

and benchmarking with oha for RPS:

oha --no-tui -z 20s -c 1 -m POST -D /tmp/body.bin http://127.0.0.1:8093/

Summary:
  Success rate:100.00%
  Total:20000.5816 ms
  Slowest:17.1687 ms
  Fastest:6.6237 ms
  Average:8.0752 ms
  Requests/sec:123.7464 <--

Re-running the component, assuming a knob to configure the outgoing body buffering, similar to preview 2's http-outgoing-body-buffer-chunks option:

 ./target/release/wasmtime serve -Wcomponent-model-async -Sp3,cli -O pooling-allocator -Shttp-p3-outgoing-body-buffer-chunks=8 --addr=127.0.0.1:8093 <component.wasm>

Re-running the RPS benchmark:

oha --no-tui -z 20s -c 1 -m POST -D /tmp/body.bin http://127.0.0.1:8093/
  Success rate:100.00%
  Total:20001.4951 ms
  Slowest:4.4801 ms
  Fastest:1.5558 ms
  Average:2.1880 ms
  Requests/sec:456.0159 <--

This represents ~3.7x improvement locally; the improvement is obviously variable depending on the number of chunks chosen.

As far as I can tell there is no existing way to influence this, however, it is possible that I am missing some context on why this was not added in the initial implementation and if that is the case, I can close this issue.