What is the best way to handle 1xx informational requests and responses?

[ioquatix]

In general, I (personally) see the complexity introduced by 1xx informational responses as significantly out weighting the benefits, especially given that it feels to me most of the advantages can be had by:

• "100 Continue" - stream cancellation.
• "103 Early Hints" - simply streaming responses (stream cancellation can be used for error handling).

The Complexity of Informational Responses

I would like to state clearly, that I think informational responses break the clean request/response semantic of HTTP, but specifically, the complexity shows up in a few places, and while we can talk about those specific cases, I also don't think that "HTTP the abstract semantic" should care about the specifics too much - the idea being that client and server behaviour tied to specific status codes, I think, should not be encouraged - HTTP is an abstract interface, and we shouldn't need to have specific code paths for specific status fields. In practice, this is a lot worse with HTTP/1.1 and significantly better in HTTP/2+ which focuses more on the abstract protocol.

100 Continue

Internally, if the client makes a request with expect: 100-continue, the server can transparently handle that and write an informational response when the server starts trying to read the body. This hides most of the practical value within the protocol itself, but this design won't work across proxies that aren't aware of it, i.e. if you imagine:

[Client]                [Falcon Virtual]   [Ruby Web Application]
POST /upload
expect: 100-continue
                        -> POST /upload    -> POST /upload
                                           ... request read
                        ... request read
                        <- 100 continue
... request write        
                        ... request write
                                           200 Okay

In the above picture, if 100-continue is considered purely at the protocol level, it won't be forwarded to the ruby web application. Probably this could be okay, but semantics of the original request are somewhat lost and essentially converted to normal stream semantics.

[Client]                [Falcon Virtual]   [Ruby Web Application]
POST /upload
expect: 100-continue
                        -> POST /upload    -> POST /upload
                        (Buffering request for proxy)
                        ... request read
                        <- 100 continue
                                            ... input close
... request write        
                        ... request write
                                            401 Unauthorized

I could imagine because the proxy may choose to buffer the request before sending it to the web application, we end up with sub-optimal request/response handling - entirely the problem we were trying to avoid.

103 Early Hints

A client may make a request to an application, and that application may choose to return an informational response with details like link headers to preload certain resources or perhaps other information. The client in theory should receive that information as early as possible so as to start preloading those resources if needed.

This design works quite okay for browsers, but presents challenges to existing interfaces, e.g. Async::HTTP::Client and Faraday.

response = client.get("/index")
response.status # 103 ??? 200???

# Extra hook?
response = client.get("/index", early_hints: proc{|headers| ...})
response.status # 200

It's not clear to me how to best expose this. In addition, it's not clear to me how those headers should be exposed to the user in the final response. This can impact the complexity of caching responses, as now we potentially have multiple sets of headers, should they be merged, or should we ignore them?

The only ideas I have about how to expose this are to change the response to potentially contain multiple responses.

response = client.get("/index")
while response.informational?
  response.headers # ... links
  response = response.next
end
# response now contains a final response.

However, I feel like this potentially introduces a huge amount of complexity to existing usage. It might be possible to do the following:

class Response
  def read
    response = self
    while response.informational?
      yield response
    end
    return response.body.join
  end
end

At least the semantics in some cases are preserved:

response = client.get("/index")
response.read # Skips over all informational responses.

The reason why such a feature is desirable, is it seems like the only way we preserve the general request/response symmetry as required by any kind of middleware and/or proxy. In other words, every single middleware would need to correctly deal with informational responses and I don't believe there is any way to hide that (although I'd like to be proven wrong or be inspired by a different approach.

Ignorance is Bliss

Perhaps one way we can solve the problem, is to simply hide all the informational responses from users, but process them where possible. In other words, 100 continue is handled at the protocol layer, and 103 early hints is probably ignored and/or merged into the final headers for the response.

@zarqman do you have any thoughts on the above?

[ioquatix]

I've been doing a little more digging and found this: https://github.com/bashi/early-hints-explainer/blob/main/explainer.md#fetch-and-html-integration - so it seems like even fetch overlooked this initially.

Then I found this: https://bugs.chromium.org/p/chromium/issues/detail?id=1361262 apparently browsers ignore all but the first set of early hints. The complexity is pretty astounding:

• Handle the first early hints and ignore subsequent early hints.
• When cross-origin redirect happens, discard the early hints
  initiated requests and treat the next early hints is the first
  early hints.

I started to think about what a middleware would look like that could deal with early hints:

class MyMiddleware
  def call(request)
    return Response.new(103, headers, next: proc{Response.new(200, final_headers, actual_body})
  end
end

This would change the entire model for response handling and in addition, make things like caching much more complex to implement correctly. Maybe it's viable? Or maybe there is an alternative way to do this I'm just not considering?

[byroot]

Then I found this: https://bugs.chromium.org/p/chromium/issues/detail?id=1361262 apparently browsers ignore all but the first set of early hints. The complexity is pretty astounding:

This shouldn't matter to you. This is a detail of how Chrome chose to implement it, and may change in the future. It's in no way related to the spec.

It's a concern for applications to not send streams of hints if they want to play well with Chrome. Libraries should allow multiple provisional responses regardless.

[zarqman]

I did some riffing around the Middleware and Client interfaces. I think separating the informational response(s) from the final response may result in something cleaner and mostly-backwards compatible (apart from Middleware stack initialization).

class Protocol::HTTP::Middleware
  def initialize(delegate, downstream)
    # need both the next and previous members of the middleware stack
    ...
  end
  def call(request)
    # I tried to decide if an extra Async{} call is needed in here somehow. I don't think so, but could be wrong.
    @delegate.call(request)
  end
  def push(response)
    # This is a side-channel for non-final responses. It allows the ability to see, mutate, or even drop info responses.
    # Probably should have a different name.
    @downstream.push(response)
      # Server Connection or Stream would have to provide its own push() to async send the provisional response
  end
end

Interestingly, this might facilitate handling of adding or discarding 1xx responses via middleware itself:

# use new API for generating 1xx by middleware itself
class Process100 < Middleware
  CONTINUE = Response[100]
  TOO_BIG  = Response[413]
  def call(request)
    # if request.body&.length > 1.gigabyte
    #   return TOO_BIG
    # end
    if request.headers['expect']&.include?('100-continue')
      push(CONTINUE)
      request.headers.delete 'expect'
    end
    super(request)
  end
end

# use new API to discard 1xx
class Discard1xx < Middleware
  def push(response)
    if response.status == 100
      # no-op
    else
      super
    end
  end
end  

For Client, the above fits the callback style fairly well:

# a proxy that's both a Server middleware and a Client:
class Proxy < Middleware
  def call(request)
    response = client.call(request, info_responses: ->(resp){ push(resp) })
      # or breakout into: client.request_method(..., info_responses: method(&:push))
    # do things with response
    return response
  end
end

A similar side-channel might be usable to trigger h2 server push requests/responses as well (although I admit to being even less favorable towards h2 server push than 1xx informational responses).

[byroot]

we potentially have multiple sets of headers, should they be merged, or should we ignore them?

The spec is quite clear, headers returned via early hints are an indication of what the final headers MAY contain. This allow clients to act on it early (e.g. preload assets), but only the headers in the final response are ... well final.

So if you have a response such as:

• 103 early hints with a header Foo: bar
• 200 OK without that headers

Then the final response doesn't have a Foo: bar, the server merely hinted that this header may appear in the final response, it can it's helpful to the client.

So the final response shouldn't include anything from the previous 103 early hint responses.

[zarqman]

And related, 1xx shouldn't be cached at all. Only the final response (and its headers) should be stored and replayed.

[byroot]

"100 Continue" - stream cancellation.

I don't understand what you mean by stream cancellation.

The main use of 100 Continue is for slow uploads or processing that can often timeout. It's a way for the server to notify the client that everything is still OK and that it can continue to send data or wait. It's also a way for the server to make sure the client is still reading.

In short it's useful for slow synchronous endpoints, e.g. holding a lock, resizing an image / video, or whatever. Rather than to make the client poll to wait for the operation to be done and download the response, you can do it all in a single query.

[ioquatix]

I was under the impression that 100 continue was a useful way to deal with large uploads where the upload might not be accepted, for example, one may be uploading and not have the appropriate access, thus receiving a 401 unauthorized instead of a 100 continue.

The main use of 100 Continue is for slow uploads or processing that can often timeout.

Are you saying the server would periodically send a 100 continue response until the final 200 okay? e.g.

Client: POST /upload; expect: 100-continue
Server: 100 Continue
Client: Send some data...
Server: 100 Continue
Client: Send some data...
Server: 100 Continue
Client: Send some data...
Server: 100 Continue
Client: Send some data...
Server: 100 Continue
Client: Send some data...
Server: 100 Continue
Client: Send final chunk...
Server: 200 Okay

?

[byroot]

Are you saying the server would periodically send a 100 continue response until the final 200 okay? e.g.

That's one use case yes.

https://www.rfc-editor.org/rfc/rfc7231#section-6.2.1

6.2.1. 100 Continue

The 100 (Continue) status code indicates that the initial part of a
request has been received and has not yet been rejected by the
server. The server intends to send a final response after the
request has been fully received and acted upon.

When the request contains an Expect header field that includes a
100-continue expectation, the 100 response indicates that the server
wishes to receive the request payload body, as described in
Section 5.1.1. The client ought to continue sending the request and
discard the 100 response.

If the request did not contain an Expect header field containing the
100-continue expectation, the client can simply discard this interim
response.

[byroot]

BTW:

A client MUST be able to parse one or more 1xx responses received
prior to a final response, even if the client does not expect one. A
user agent MAY ignore unexpected 1xx responses.

So regardless of whether 1xx are returned or exposes to the caller, a client MUST at least ignore them and keep reading.

[byroot]

For the interface, I think you listed the two possible designs. Either a callback, or returning an iterator.

For the iterator solution however, I'd suggest making it opt-in as to avoid calling code that doesn't expect it.

If the caller didn't opt-in, it's best to just drop the 1xx and keep reading, possibly resetting the read timeout. Similarly with the callback solution, if it isn't passed, the client should just drop them.

As for naming specifically:

client.get("/index", early_hints: proc{|headers| ...})

IMO it is a bad idea to add something for a specific error code, the naming and API should handle the entire 1xx range so that future additions are also automatically handled. So more something like provisional_responses: proc, etc.

[zarqman]

For a proxy to stream 1xx responses in real-time and not just end up buffering them (at which point they should just be discarded), an iterator would need to function much like a promise.

I think the callback is cleaner and provides better backwards compatibility, If not present, the 1xx responses can just be discarded, which is likely for most use-cases. It would be more useful it it was response for the arg, not headers, such as proc{|response| ...}. That way the callback can apply for all 1xx types.

[zarqman]

On the whole, I'm also of the mind that there's only minimal benefit and potentially high complexity to handling 1xx responses. My lean is to just discard them. Even for my use case in a reverse proxy, I don't presently think it's that important to implement. Others may have a different perspective though.

On the client side, sending Expect: 100-continue is a mess, as the payload should be sent anyway after a delay, even without receiving the desired 100. The server must also be prepared to handle the client sending the request body without waiting for the 100, so fallback handling (such as closing the connection/stream) for situations like refusing a too-big upload is required anyway. Upon receiving a 417, clients also should retry without the Expect.

The idea of having a proxy itself respond to Expect: 100-continue does have precedent: nginx appears to auto-respond to 100-continue when in proxy or xCGI mode. Heroku's vegur proxy also auto-responds to such. However, the RFCs forbid it except when the origin (or upstream proxy) is known to support only HTTP/1.0 (which doesn't support 1xx). I am aware of both curl and some .NET library (I forget which) sometimes self-choosing to add Expect to requests. curl is easy enough to disable. I can't speak to the .NET one.

Related, async-http would need to always discard 1xx responses for HTTP/1.0 requests, even if it is willing to send them on h1.1 and h2.

The backwards compatibility concerns mean that both client and server have to treat 1xx as optional and discardable anyway, which leaves room to argue for simply discarding all the time.

[zarqman]

An exploration on using Client to upload using Expect: 100-continue. Uses a callback for the 100.

class Confirmable
  attr_accessor :released_to_send
  attr_accessor :canceled
  def read
    unless released_to_send
      begin
        # timeout/timer is pseudo-code
        task.with_timeout(wait_for_confirm, AutoRelease) do
          wait_for(released_to_send || canceled)
        end
      rescue AutoRelease
        released_to_send = true
      end
    end
    return nil if canceled
    @body.read
  end
end
body = Body::Confirmable.new(Body::Buffered.new('big-payload'), wait_for_confirm: 2.seconds)
body.released_to_send # => false
handle_1xx = proc do |resp|
  body.released_to_send = true if resp.status == 100
end
response = client.post('/upload', {'expect'=>'100-continue'}, body, info_responses: handle_1xx)
if response.status >= 400
  body.canceled = true
end
if response.status == 417
  # start over without Expect and with the direct payload
end
# continue normally, handling 401,403,200,etc

[ioquatix]

There is another weird edge case, for the sake of clients doing websockets, 101 Switching Protocols is, in practice, currently considered a "final response", in the sense we don't wait for some other status code, e.g. 200 OK. It's a bit weird because it's not purely "informational", and simply dropping it/ignoring it would break websockets, and not returning that as a response itself would break existing behaviour.