RFC: Type-Modeled Retryability

[rcoh]

Currently, retry behavior in the SDK is implicit. Retries are always enabled and not easy to disable. Making things slightly more complex, retries will never be enabled when the body is streaming because a generalized streaming body cannot be retried.

This RFC proposes several changes to improve on this behavior:

1. Client level retry opt-out: <servicename>::Config will expose a .retry_mode setting. Retry mode can also be read from environment variables.

   let conf = dynamodb::Config::builder().retry_mode(RetryMode::Never).build();
   let client = dynamodb::Client::from_conf(conf);
   client.list_tables().send().await // will not retry

2. Operation level retry opt-out: Operation builders will expose a .retry_mode setting:

   let client = dynamodb::Client::from_env();
   client.list_tables().retry_mode(RetryMode::Never).send();

3. Explicitly retryable bodies: Request bodies generated by the SDK via serialization will always be retryable. Internally, these bodies use Bytes which is cheap to clone. The issue arises with APIs like S3::put_object() where the user passes a byte stream directly. We propose introducing the following trait:

   trait RetryableByteStream {
     fn make_byte_stream(&self) -> ByteStream
   }

   Generated builders will add an additional builder method, <bodyname>_retryable:

    /* s3 put object builder */
    fn body(self, body: ByteStream) -> Self {
      /* .. */
    }
    fn body_retryable(self, body: impl RetryableByteStream) -> Self {
      /* .. */
    }

   Customers who rely on retryable inputs can use the body_retryable API to validate that their request will be retryable at compile time.

   let client = s3::Client::from_env();
   let (send, streaming_body) = SdkBody::channel();
   client.put_object().body(streaming_body); // OK
   client.put_object().body_retryable(streaming_body); // Not OK, channel bodies are not retryable

Retryable Byte Stream Internals

Internally, will introduce a Retryable variant onto SDK body that contains a Box<dyn RetryableByteStream>. There is a bit more design work here to determine the best way to wire the pieces together coherently, but these are exclusively private APIs that we can rework at any point.

RetryMode

RetryMode will be a non-exhaustive enum:

#[non_exhaustive]
pub enum RetryMode {
  /// Regardless of response from the service, do not retry, return the error immediately
  NeverRetry,
  /// Retry when indicated by the response with standard exponential backoff with Jitter
  Standard,
  // In the future:
  // Adaptive, 
}

[carllerche]

Something to consider, the names body and body_retryable will have users tend to default to using body. One option could be to switch the body fn to be retriable and have body_once or something like that.

Also, if a body is provided with the body fn (T: ByteStream) will that never retry?

[rcoh]

body will retry based the the retryability of body, which in practice means that all non-streaming bodies or bodies created from something retryable (like a file) will retry, whereas dynamically streaming bodies will not.

[carllerche]

Ah right, here is the relevant bit: https://github.com/awslabs/aws-sdk-rust/blob/20e8e1c81b637624cb17c4c62522877a06b69f75/sdk/smithy-http/src/body.rs#L33-L42

[carllerche]

What happens if retries are requested but a non-retriable body is provided? Will the request still be issued and fail on retry or will the entire request result in an error or panic?

[rcoh]

a retry won't be issued and the original error from the service will be returned (as if the client ran out of retries). Tower calls try_clone, clone returns None, then tower won't even talk to our retry policy:
https://github.com/awslabs/smithy-rs/blob/336670b4c10beb01521e195d6dbc2b9cba256ea3/aws/rust-runtime/aws-hyper/src/retry.rs#L269-L271

[rcoh]

I think most users will reach for .body(byte_stream: ByteStream)—it does exactly what the other SDKs do (retry when possible).

body_retryable() is really just for the advanced customer that wants an absolute guarantee that the request that they send will not be denied a retry on account of the request body.