-
Notifications
You must be signed in to change notification settings - Fork 576
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PutObjectCommand
fails when passing a ReadableStream<Uint8Array>
with a TypeError only at runtime
#6428
Comments
Hi @Sophon96 - thanks for reaching out. The issue you're facing is caused by a type mismatch between the expected input type for the Body parameter of the PutObjectCommand constructor and the actual type of the The Body parameter of the PutObjectCommand constructor expects one of these types: The TypeScript compiler doesn't catch this error because the Body parameter is defined as a union type that includes any (or unknown in newer TypeScript versions). This means that any type of value can be assigned to Body without causing a compile-time error. Here's how you can convert the ReadableStream before passing it to PutObject in your current code: import { Readable } from 'stream';
// ...
async function fetchAndPut() {
const response = await fetch("https://picsum.photos/1600/900.webp");
// ...
const stream = response.body as Readable;
const uploadParams: PutObjectCommandInput = {
Bucket: "fetchandputrepro",
Key: "image1.webp",
Body: await streamToBuffer(stream), // Convert the stream to a Buffer
ContentType: "image/webp",
ContentLength: contentLength,
};
await s3Client.send(new PutObjectCommand(uploadParams)).then(
(data) => console.log(`Etag: ${data.ETag}`),
(err) => console.error(`response.body error: ${err}`)
);
}
async function streamToBuffer(stream: Readable): Promise<Buffer> {
const chunks: Buffer[] = [];
for await (const chunk of stream) {
chunks.push(Buffer.from(chunk));
}
return Buffer.concat(chunks);
} In the modified code, I introduced a new The streamToBuffer function is then used to convert the Note that this approach reads the entire stream into memory before uploading it to S3. If you need to handle large files or streams efficiently, you might want to consider using the Upload utility provided by the @aws-sdk/lib-storage package which allows you to upload data in chunks without reading the entire stream into memory. Hope it helps! |
Only a bit related, but when passing an ArrayBuffer to the PutObjectCommand we also get a typeerror in 3.645.0. It seems that the Smithy type Simple example below: import type { File } from "node:buffer";
const upload = async (file: File) => {
const buffer: ArrayBuffer = await file.arrayBuffer();
const command = new PutObjectCommand({
Bucket: "bucket",
Key: `key`,
Body: buffer, // Type 'ArrayBuffer' is not assignable to type 'StreamingBlobPayloadInputTypes | undefined
});
}; Its easily fixable by casting to Buffer, but still something that might needs to be looked at. const command = new PutObjectCommand({
// ...
Body: buffer as Buffer, // Works
});
`` |
Hi @aBurmeseDev, thanks for the reply! However, are you aware of a method of narrowing the types so that type checking works? I'd rather not have to guess-and-check which types work. (I suppose this is also related to Simon's concern.) Thanks again. |
Checkboxes for prior research
Describe the bug
Attempting to pass
Response.body
to theBody
parameter of thePutObjectCommand
constructor results in a TypeError at runtime, but not at compile time:SDK version number
@aws-sdk/[email protected]
Which JavaScript Runtime is this issue in?
Node.js
Details of the browser/Node.js/ReactNative version
v18.12.1
Reproduction Steps
Here's a minimal reproduction repo: https://github.com/Sophon96/s3fetchandputrepro
Observed Behavior
There are no errors at TS compile time, but the following error is produced at runtime:
Expected Behavior
PutObjectCommand
should accept theReadableStream<Uint8Array>
ofResponse.body
without error.Possible Solution
No response
Additional Information/Context
No response
The text was updated successfully, but these errors were encountered: