Not able to stream PDF, using AWS gateway/Lambda setup - amazon-web-services

Question

Problem Statement:- Not able to stream PDF, using AWS gateway/Lambda setup.
I've following Setup:-
Gateway API -> Lambda Function(java) -> S3
Api should stream a PDF back to the client via Lambda from S3 server.
I am unable to do so.
Things tried
1) On Lambda side
Use RequestStreamHandler, write PDF on OutputStream
outputStream.write();
Tried Setting Base64 encoding using following
AWS Base64 Utility
java.util Base64 Utility
apache Base64 utility
Send Byte[] without encoding
Use non-proxy handler, write PDF as string to one of the output variable.
Tried Setting Base64 encoding using following
AWS Base64 Utility
java.util Base64 Utility
apache Base64 utility
Send Byte[] without encoding
2) On Gateway Side
Set Proxy integration
Remove Proxy integration
Tried Set Content Handling using
Passthrough
Convert to Binary
Convert to String
Set headers
Content-Type = 'application/pdf'
Content-Disposition = 'attachment; filename="nameofpdffile.pdf"'
I've tried all the permutations/combinations of these 2. However, I am not able to get it to work and documentation around this area seems poor.
When I set Base64 encoding in Lambda and on gateway side specify content handling as convert to binary, i get error, fail to conert using Base64 decode.
For other combinations, I get binary data as output when I test it using gateway Test funciton, however my client(Postman, Chrome, Safari) fail to convert it to PDF
Note:- I've a workaround in place, where I create a signedURL to S3 object, and send it as redirect. However, I am trying to make it work by streaming a binary file using Lambda.
Thanks for reading such a long post. Any pointers, links in the right direction are highly appreciated.
- Frustated AWS user :-)

Answer 1

I had a similar problem with trying to offer up gzipped content: Client -> API-Gateway -> S3.
I couldn't figure out why the content I pulled couldn't get recognized by the client (either browser or code) until I inspected the header(the binary header, not http).
It appears that API Gateway by default assumes a string based encoding (utf-8) so what I was seeing is data that API Gateway was transforming on the fly and adding utf-8 headers in various places.
Late last year API Gateway started to support binary payloads. I started reading this article which may help in your case. It did not in mine, but your use case is slightly different and closer to the article so it might.
https://aws.amazon.com/blogs/compute/binary-support-for-api-integrations-with-amazon-api-gateway/
As for what I ended up doing... I realized I was just using the API Gateway as a passthrough (to take advantage of the Cognito based authentication I had already setup) so I bypassed API Gateway all together and used the aws js sdk to connect directly to s3 (and set IAM policies on the bucket separately)