AWS s3 V3 Javascript SDK stream file from bucket (GetObjectCommand) - amazon-web-services

Question

Iv looked all over AWS docks and stack overflow (even went to page 4 of google!!!) but i cannot for the life of me work out how to stream a file from S3. The docs for V3 are pretty useless and all the examples i find are from V2.
The send commond that V3 uses only returns a promise so how do i get a stream and pipe it instead of waiting for the whole file (it needs to be piped into encryption algo then to a response stream)
this.s3.send(
new GetObjectCommand({
Bucket: '...',
Key: key,
}),
);
I was able to upload fine by passing the stream as the body, is there something i have to do similar here?
uploadToAws(key) {
const pass = new PassThrough();
return {
writeStream: pass,
promise: this.s3.send(
new PutObjectCommand({
Bucket: '...',
Key: key,
Body: pass,
ServerSideEncryption: '...',
ContentLength: 37,
}),
),
};
}

Answer 1

Body from the GetObjectCommand is a readable stream (https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/interfaces/getobjectcommandoutput.html#body).
So you can do:
const command = new GetObjectCommand({
Bucket
Key,
});
const item = await s3Client.send(command);
item.Body.pipe(createWriteStream(fileName));

Answer 2

docs links
For those landing here after googling because aws v3 sdk docs are missing details on getobjectcommandoutput interface, you can find the full getobjectcommandoutput definition at source or at "module" → https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-s3/modules/getobjectoutput.html

Answer 3

Figured it out,
s3Client.send(command) returns a type GetObjectCommandOutput.
const data: GetObjectCommandOutput = await s3Client.send(command)
data.Body is of type SdkStream<Readable | ReadableStream<any> | Blob | undefined> | undefined
Undefined is for error cases, you check there is no error case like this
if (!data.Body)//handle error
For success case, you can get ReadableStream like this
const readableStream: ReadableStream = data.Body!.transformToWebStream()
For aws-sdk V2, there was createReadStream(), this is seems to be the way in v3.
To pipe through ReadableStream, use
readableStream.pipeTo() or readableStream.pipeThrough()

Answer 4

Using transformToString() as per docs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/example_s3_GetObject_section.html.
export const handler = async event => {
try {
// Retrieve the object from S3
const data = await s3.getObject({ Bucket: BUCKET_NAME, Key: PATH_AND_FILE_NAME });
// Set the content type of the response
const contentType = data.ContentType;
// Convert to base64 string
const streamToString = await data.Body?.transformToString("base64");
// Return the object data in the response
return {
statusCode: 200,
headers: {
"Content-Type": contentType
},
body: streamToString,
isBase64Encoded: true
};
} catch (error) {
return {
statusCode: 500,
body: "An error occurred: " + error.message
};
}
};

Answer 5

In case you also run into a similar issue while using typescript, This helped me
(object.Body as any).pipe(res);
I did this because as mentioned by laconbass, aws v3 sdk docs are missing details on getobjectcommandoutput.