From 9de4ec8401a0741a8a7de8df19f7c41570e04119 Mon Sep 17 00:00:00 2001 From: Samrat002 Date: Fri, 24 Jul 2026 20:03:06 +0530 Subject: [PATCH] [FLINK-40147] Improve Native S3 FileSystem docs when using S3-compatible servers that do not support AWS checksum-validation --- docs/content/docs/deployment/filesystems/s3.md | 8 ++++++++ flink-filesystems/flink-s3-fs-native/README.md | 5 ++++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/docs/content/docs/deployment/filesystems/s3.md b/docs/content/docs/deployment/filesystems/s3.md index 95e8f33d0828a3..a49ac14e678770 100644 --- a/docs/content/docs/deployment/filesystems/s3.md +++ b/docs/content/docs/deployment/filesystems/s3.md @@ -172,6 +172,14 @@ In addition to the [common configuration](#common-configuration) options (`s3.ac View the detailed configuration at [native-s3-fs](https://github.com/apache/flink/tree/master/flink-filesystems/flink-s3-fs-native#configuration-options) +For S3-compatible endpoints, the AWS SDK requires a region. Use the server's region, or `us-east-1` if the server ignores it. Disable checksum validation only if the server does not support it: + +```yaml +s3.endpoint: https://your-s3-compatible-server +s3.region: us-east-1 +s3.checksum-validation.enabled: false +``` + --- ### Presto S3 FileSystem diff --git a/flink-filesystems/flink-s3-fs-native/README.md b/flink-filesystems/flink-s3-fs-native/README.md index 96ac27a907ac04..292f87cd25aae3 100644 --- a/flink-filesystems/flink-s3-fs-native/README.md +++ b/flink-filesystems/flink-s3-fs-native/README.md @@ -277,6 +277,7 @@ For S3-compatible servers (SeaweedFS, LocalStack, Ceph RGW, etc.), set the endpo ```yaml s3.endpoint: http://localhost:8333 +s3.region: us-east-1 fs.s3.aws.credentials.provider: AnonymousCredentialsProvider # Required: SeaweedFS serves the S3 API in path-style and does not support @@ -289,7 +290,9 @@ s3.chunked-encoding.enabled: false s3.checksum-validation.enabled: false ``` -The exact subset of compatibility flags needed depends on the server and version — consult its documentation. The defaults in this filesystem (`path-style-access=false`, `chunked-encoding.enabled=true`, `checksum-validation.enabled=true`) are tuned for AWS S3. +The AWS SDK requires a region for custom endpoints. Use the server's region, or `us-east-1` if the server ignores it. + +Required compatibility flags vary by server and version. Disable checksum validation only if the server does not support it. The defaults (`path-style-access=false`, `chunked-encoding.enabled=true`, `checksum-validation.enabled=true`) target AWS S3. ## Memory Optimization for Large Files