Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alias blobstore breaks multipart uploads to S3 #655

Open
jrtcppv opened this issue Jun 19, 2024 · 1 comment
Open

Alias blobstore breaks multipart uploads to S3 #655

jrtcppv opened this issue Jun 19, 2024 · 1 comment

Comments

@jrtcppv
Copy link

jrtcppv commented Jun 19, 2024

Using the following s3proxy.conf for an AWS S3 bucket:

s3proxy.endpoint=http://0.0.0.0:8085
s3proxy.authorization=none
jclouds.provider=aws-s3
jclouds.identity=####################
jclouds.credential=####################################
jclouds.region=us-east-2

I am able to complete a multipart upload successfully with aws s3 cp ~/big_file.mp4 s3://test-bucket --endpoint-url http://localhost:8085

However, when I enable alias-blobstore and bucket-locator:

s3proxy.endpoint=http://0.0.0.0:8085
s3proxy.authorization=none
jclouds.provider=aws-s3
jclouds.identity=####################
jclouds.credential=####################################
jclouds.region=us-east-2
s3proxy.alias-blobstore.project-1=test-bucket
s3proxy.bucket-locator.1=project-1

And attempt to upload with aws s3 cp ~/big_file.mp4 s3://project-1 --endpoint-url http://localhost:8085 I receive the following error after uploading seemingly all parts:

upload failed: ../../big_file.mp4 to s3://project-1/big_file.mp4 An error occurred (NoSuchKey) when calling the CompleteMultipartUpload operation: The specified key does not exist.

Not sure if this is relevant but I am invoking s3proxy using the docker container with the latest tag, like so:

docker run --rm -v ~/s3proxy.conf:/s3proxy.conf --publish 8085:8085 --entrypoint /bin/bash andrewgaul/s3proxy -c "java -jar /opt/s3proxy/s3proxy --properties /s3proxy.conf"
@jrtcppv
Copy link
Author

jrtcppv commented Jun 19, 2024

I just tried with the bucket locator removed, so just using the alias-blobstore is sufficient to reproduce:

s3proxy.endpoint=http://0.0.0.0:8085
s3proxy.authorization=none
jclouds.provider=aws-s3
jclouds.identity=####################
jclouds.credential=####################################
jclouds.region=us-east-2
s3proxy.alias-blobstore.project-1=test-bucket

@jrtcppv jrtcppv changed the title Alias blobstore + Bucket Locator breaks multipart uploads to S3 Alias blobstore breaks multipart uploads to S3 Jun 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant