Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ChunkSignature added to the uploaded file #657

Open
aravinds502 opened this issue Jun 21, 2024 · 0 comments
Open

ChunkSignature added to the uploaded file #657

aravinds502 opened this issue Jun 21, 2024 · 0 comments

Comments

@aravinds502
Copy link

When a file is Uploaded to GCP via s3proxy, "chunk signature is added in the first and last lines

We are using s3client, Upload multipart Request to upload. The uploaded data was as follows for a sample.csv

69e;chunk-signature=e2d4edfb35ad945809e4c08c2ed8c8abf4406b1af8621b3b91d1224a4d24a613
"Year","Month","DayofMonth","DayOfWeek","DepTime","CRSDepTime","ArrTime","CRSArrTime","UniqueCarrier","FlightNum","TailNum","ActualElapsedTime","CRSElapsedTime","AirTime","ArrDelay","DepDelay","Origin","Dest","Distance","TaxiIn","TaxiOut","Cancelled","CancellationCode","Diverted","CarrierDelay","WeatherDelay","NASDelay","SecurityDelay","LateAircraftDelay"
"1993","1","29","5","1055","1055","1228","1212","US","66","","93","77","","16","0","EWR","BUF","282","","","0","","0","","","","",""
"1993","1","30","6","1052","1055","1214","1212","US","66","","82","77","","2","-3","EWR","BUF","282","","","0","","0","","","","",""
"1993","1","31","7","1103","1055","1213","1212","US","66","","70","77","","1","8","EWR","BUF","282","","","0","","0","","","","",""
"1993","1","1","5","1738","1730","1949","1941","US","68","","131","131","","8","8","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","3","7","1737","1730","1946","1941","US","68","","129","131","","5","7","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","4","1","1739","1730","1951","1941","US","68","","132","131","","10","9","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","5","2","1729","1730","1939","1941","US","68","","130","131","","-2","-1","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","6","3","1730","1730","1940","1943","US","68","","130","133","","-3","0","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","7","4","1734","1730","1942","1943","US","68","","128","133","","-1","4","MCO","PIT","834","","","0","","0","","","","",""
"1993","1","8","5","1812","1730","2015","1943","US","68","","123","133","","32","42","MCO","PIT","834","","","0","","0","","","","",""

0;chunk-signature=c300db3aca97bfed581d9c494fcb458faddc6eda7382701a5b08a89bd74340b8

How to avoid adding the chunk-signatures, while doing uploads with GCP via s3proxy? Is there any setting we can add?
The same thing works fine if we directly connect to GCP.

Thanks,
Aravind

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant