File Download from Amazon S3 via REST (Jax-RS) - amazon-web-services

Question

I am trying to download a file from Amazon S3.
I want the user to visit my app via a GET api.
The app in turn gets the content from S3 and give it back to the user as a downloadable file.
Note:
I dont want to store the file locally in my server, i want it to be streamed form amazon s3 directly to the end user
I tried with a file of around 300 MB, if I run it locally like below
the memory footprint is low, i.e. when the same file is present locally
#GET
#Path("/pdfdownload")
#Produces("application/pdf")
public Response getFile() {
File file = new File('/pathToFile'); // in local
ResponseBuilder response = Response.ok((Object) file);
response.header("Content-Disposition", "attachment; filename=file.pdf");
return response.build();
}
But when I download the same from Amazon s3, my tomcat server's memory quickly raises to around 600 MB, I think I am streaming the content, but when i look at the memory used i doubt it
Am i missing something ?
#GET
#Path("/pdfdownload")
#Produces("application/pdf")
public Response getFile2() {
final S3Object s3Object = getAmazonS3Object();// AWS S3
final S3ObjectInputStream s3is = s3Object.getObjectContent();
final StreamingOutput stream = new StreamingOutput() {
#Override
public void write(OutputStream os) throws IOException, WebApplicationException {
byte[] read_buf = new byte[1024];
int read_len = 0;
while ((read_len = s3is.read(read_buf)) > 0) {
os.write(read_buf, 0, read_len);
}
os.close();
s3is.close();
}
};
ResponseBuilder response = Response.ok(stream);
response.header("Content-Disposition", "attachment; filename=file.pdf");
return response.build();
}
private S3Object getAmazonS3Object() {
AWSCredentials credentials = new BasicAWSCredentials("accesskey",
"secretkey");
try {
AmazonS3 s3 = new AmazonS3Client(credentials);
S3Object s3object = s3.getObject(new GetObjectRequest("bucketName", "filename_WithExtension"));
return s3object;
} catch (AmazonServiceException e) {
System.err.println(e.getErrorMessage());
System.exit(1);
}
System.out.println("Done!");
return null;
}
Pom :
<dependency>
<groupId>com.sun.jersey</groupId>
<artifactId>jersey-server</artifactId>
<version>1.8</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk</artifactId>
<version>1.11.542</version>
</dependency>
Similar to this S3 download pdf - REST API
I dont want to use PreSignedURl: https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURLJavaSDK.html
Please see this article on streaming: https://memorynotfound.com/low-level-streaming-with-jax-rs-streamingoutput/
Could some please help as why the memory spikes up?

Answer 1

Thanks to all the post on stackoverflow and one of my colleague.
My colleague found the answer, actually the above code doesnt have a memory issue, when I was monitoring the jvm i saw a spike, but didnt realize garbage collection didnt kick in.
I tried downloading 6 files each fo 300 MB +, the server holds its ground