ExportPartition - Backoff policy #1279
Description
Each part within a partition export task has retry_count and max_retry_count fields. Whenever max is reached, the entire task is failed. With the current implementation, it is possible that a temporary S3 failure (i.e, S3 goes out for a few seconds or minutes) leads to too many failed attempts within a short period of time causing the entire task to be failed. This becomes even more problematic with large parts / partitions.
It would be great to have a backoff policy.