Improve S3 error messages for credential and access issues#6669
Closed
pinin4fjords wants to merge 3 commits intomasterfrom
Closed
Improve S3 error messages for credential and access issues#6669pinin4fjords wants to merge 3 commits intomasterfrom
pinin4fjords wants to merge 3 commits intomasterfrom
Conversation
Translate AWS SDK exceptions to standard Java NIO exceptions with clear, actionable error messages in S3ObjectSummaryLookup.lookup(). - 401/403 status codes -> AccessDeniedException with credential hint - 404 status code -> NoSuchFileException - Credential loading errors -> AccessDeniedException - Other SDK errors -> IOException with original cause This allows downstream tools like nf-schema to distinguish between access errors (which can be skipped gracefully) and other validation failures, and provides users with meaningful error messages instead of cryptic AWS SDK exceptions. Fixes #6668 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Jonathan Manning <jonathan.manning@seqera.io>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
- Remove fragile 'Unable to marshall request' pattern matching (marshall errors now wrapped as IOException, not AccessDeniedException) - Apply consistent exception handling to getS3ObjectMetadata() - Update copyright years to 2025 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Jonathan Manning <jonathan.manning@seqera.io>
The 'Unable to marshall request' error is a symptom of credential issues and should be translated to AccessDeniedException so that downstream tools like nf-schema can distinguish access errors from other failures. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: Jonathan Manning <jonathan.manning@seqera.io>
pinin4fjords
force-pushed
the
improve-s3-error-messages
branch
from
eb01c4a to
1c27949
Compare
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR illustrates a potential approach to translating AWS SDK exceptions to standard Java NIO exceptions with clearer error messages. It is not intended to be merged as-is - we're looking for guidance on a better solution.
The Problem
Users see cryptic errors like:
When they should see something like:
What This PR Demonstrates
lookup()andgetS3ObjectMetadata()methodsAccessDeniedException, 404 →NoSuchFileException)IOExceptionThe current implementation relies on fragile string matching to detect credential errors:
This is not a viable solution because:
"Unable to marshall request"pattern catches serialization errors that coincidentally appear when credentials are misconfigured - this is not a reliable indicatorWhat We Need
We're looking for guidance on how to properly detect credential and access issues from AWS SDK exceptions. Is there a more robust approach using exception types, error codes, or the SDK's error handling facilities?
Context
See issue #6668 for the full problem description. The goal is to allow downstream tools like nf-schema to distinguish access errors from other failures.
🤖 Generated with Claude Code