I'm trying to return a 1px gif from an AWS API Gateway method.
Since binary data is now supported, I return an image/gif using the following 'Integration Response' mapping:
$util.base64Decode("R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7")
However, when I look at this in Chrome, I see the following binary being returned:
Instead of:
Could anyone help me understand why this is garbled and the wrong length? Or what I could do to return the correct binary? Is there some other what I could always return this 1px gif without using the base64Decode function?
Many thanks in advance, this has being causing me a lot of pain!
EDIT
This one gets stranger. It looks like the issue is not with base64Decode, but with the general handling of binary. I added a Lambda backend (previously I was using Firehose) following this blog post and this Stack Overflow question. I set images as binaryMediaType as per this documentation page.
This has let me pass the following image/bmp pixel from Lambda through the Gateway API, and it works correctly:
exports.handler = function(event, context) {
var imageHex = "\x42\x4d\x3c\x00\x00\x00\x00\x00\x00\x00\x36\x00\x00\x00\x28\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x18\x00\x00\x00\x00\x00\x06\x00\x00\x00\x27\x00\x00\x00\x27\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\x00\x00";
context.done(null, { "body":imageHex });
};
However the following images representing an image/png or a image/gif get garbled when passed through:
exports.handler = function(event, context) {
//var imageHex = "\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\x00\x00\x00\xff\xff\xff\x21\xf9\x04\x01\x00\x00\x00\x00\x2c\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x01\x44\x00\x3b";
//var imageHex = "\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\xff\xff\xff\x00\x00\x00\x21\xf9\x04\x01\x00\x00\x00\x00\x2c\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02\x44\x01\x00\x3b";
var imageHex = "\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x21\xf9\x04\x01\x00\x00\x00\x00\x2c\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02\x44\x01\x00\x3b\x0a"
context.done(null, { "body":imageHex });
};
This seems to be the same issue as another Stack Overflow question, but I was hoping this would be fixed with the Gateway API binary support. Unfortunately image/bmp doesn't work for my use case as it can't be transparent...
In case it helps anyone, this has been a good tool for converting between base64 and hex.
To anyone else having problems with this: I was also banging my head against the wall trying to retrieve a binary image over API Gateway proxy integration from lambda, but then I noticed that it says right there in the Binary Support section of Lambda Console:
API Gateway will look at the Content-Type and Accept HTTP headers to decide how to handle the body.
So I added Accept: image/png to the request headers and it worked. Oh the joy, and joyness!
No need to manually change content handling to CONVERT_TO_BINARY or muck about with the cli. Of course this rules out using, for example, <img src= directly (can't set headers).
So, in order to get a binary file over API Gateway from lambda with proxy integration:
List all supported binary content types in the lambda console (and deploy)
The request Accept header must include the Content-Type header returned from the lambda expression
The returned body must be base64 encoded
The result object must also have the isBase64Encoded property set to true
Code:
callback(null, {
statusCode: 200,
headers: { 'Content-Type': 'image/png' },
body: buffer.toString('base64'),
isBase64Encoded: true
}
It looks like this was a known issue previously:
https://forums.aws.amazon.com/thread.jspa?messageID=668306򣊒
But it should be possible now that they've added support for binary data:
http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings.html
It looks like this is the bit we need: "Set the contentHandling property of the IntegrationResponse resource to CONVERT_TO_BINARY to have the response payload converted from a Base64-encoded string to its binary blob". Then we shouldn't need the base64Decode() function.
Working on a test now to see if this works.
EDIT: I was finally able to get this working. You can see the binary image here:
https://chtskiuz10.execute-api.us-east-1.amazonaws.com/prod/rest/image
I updated the method response as follows:
I updated the integration response to include a hard-coded image/png header:
The last step was tricky: setting the contentHandling property to "CONVERT_TO_BINARY". I couldn't figure out how to do in the AWS console. I had to use the CLI API to accomplish this:
aws apigateway update-integration-response \
--profile davemaple \
--rest-api-id chtskiuzxx \
--resource-id ki1lxx \
--http-method GET \
--status-code 200 \
--patch-operations '[{"op" : "replace", "path" : "/contentHandling", "value" : "CONVERT_TO_BINARY"}]'
I hope this helps.
Check out this answer. It helped me with exposing PDF file for download through GET request without any additional headers.
Related
I have been trying to read the AWS Lambda#Edge documentation, but I still cannot figure out if the following option is possible.
Assume I have an object (image.jpg, with size 32922 bytes) and I have setup AWS as static website. So I can retrieve:
$ GET http://example.com/image.jpg
I would like to be able to also expose:
$ GET http://example.com/image
Where the response body would be a multipart/related file (for example). Something like this :
--myboundary
Content-Type: image/jpeg;
Content-Length: 32922
MIME-Version: 1.0
<actual binary jpeg data from 'image.jpg'>
--myboundary
Is this something supported out of the box in the AWS Lambda#Edge API ? or should I use another solution to create such response ? In particular it seems that the response only deal with text or base64 (I would need binary in my case).
I finally was able to find complete documentation. I eventually stumble upon:
API Gateway - PORT multipart/form-data
which refers to:
Enabling binary support using the API Gateway console
The above documentation specify the steps to handle binary data. Pay attention that you need to base64 encode the response from lambda to pass it to API Gateway.
I am trying to learn to use AWS Lambda, DynamoDB, and API Gateway.
I am able to set up a run a successful test of the Lambda that would take querystring parameters and returns the data from the database and I know I have the ability to access the Lambda from my API as I have created one to just return hello world. However, I am stuck as to how to send the parameters I need to the Lambda. These work fine from the Lambda test feature within Lambda.
{
"queryStringParameters": {
"TableName": "Exams",
"Key": {
"ExamName": "MyFirstExam"
}
}
}
However I can not seem to create a html query http://apiurl/route?parameters
as I have no idea how to send the Key.
I have tried everything I can think of here is just a sample of what I have tried
http://apiurl/route?TableName=Exams&Key=ExamName&ExamName=MyFirstExam
http://apiurl/route?TableName=Exams&key=ExamName&Keyvalue=MyFirstExam
http://apiurl/route?TableName=Exams&key=ExamName&value=MyFirstExam
http://apiurl/route?TableName=Exams&key=ExamName:MyFirstExam
but nothing has worked so I am wondering how do you send key = ExamName : MyfirstExam so that it is a part of the querystring parameters?
UPDATE:
I found that the rest api gave me more feed back and it looks like I need to use
````http://apiURL/Route?TableName= Exams&Key= { ExamName: MyFirstExam } ```
as this populates the querystring parameters.
"queryStringParameters":{"TableName":"Exams","Key":"{ ExamName: MyFirstExam }"}
However, I am still getting an error. That leads me to believe that I still may not have the format right. The problem is that when I used the same data in the test from within the lambda console it works and returns the data. When I use the querystring above which appears to be the same data as used in the lambda test it throws an error.
Lambda execution failed with status 200 due to customer function error: The provided key element does not match the schema. So my Key is getting there as I get a status 200 and if I leave the key off of the querystring it throws and error that the key is missing but it appears to not be formatted correctly apparently or is it something else that I am missing. It appears to be the same as the format that is working in the Lambda test.
This works from the lambda test
{ "queryStringParameters": { "TableName": "Exams", "Key": { "ExamName": "MyFirstExam" } } }
this does not work from the API test
"queryStringParameters":{"TableName":"Exams","Key":"{ ExamName: MyFirstExam }"}
This is how I am formatting the querystring
http://apiURL/Route?TableName= Exams&Key= { ExamName: MyFirstExam }
How do I format a querystring so that it works from the API gateway?
The easiest and most commonly used method of joining API Gateway and lambda is through AWS_PROXY integration between your API and your function.
With this type of integration:
API Gateway applies a default mapping template to send the entire request to the Lambda function and transforms the output from the Lambda function to HTTP responses.
This means that every query parameter you provide through your API endpoint is going to be delivered directly into lambda function:
This request data includes the request headers, query string parameters, URL path variables, payload, and API configuration data.
The event format your lambda function gets is described here.
I resolved this by having the key part already in the code and just now only need to pass a normal variable ExamName=Exam.
var queryParameters ={
TableName: request.TableName,
Key:{
"ExamName": request.ExamName
},
};```
I'm using AWS API Gateway integrated with Lambda.
Note: I am NOT using Lambda Proxy.
I need to return a binary response from API Gateway. I have successfully set this up as follows:
Encoded my binary data as base64 UTF-8 string and returning ONLY that from my lambda function return "base64 encoded binary data"
Enabled CONVERT_TO_BINARY on the API Gateway Integration Response
Mapped the Content-Type header on the API Gateway Method Response to the binary media type of my binary content
Added the media type of my binary content to API Gateway's list of Binary Media Types
The issue is that as well as sending the binary data (which I can do successfully from the above steps), I need to include a x-my-header custom header in the API Response.
I know how to set-up header mapping in the API Gateway, but the header has to be calculated from database data, and therefore this value needs to also be returned from lambda.
My understanding of lambda integration (remember, I'm not using lambda proxy here) is that API gateway makes a HTTP request to trigger lambda. Lambda then returns a HTTP response to API Gateway, adding the functions output to the body, and also adding internal aws headers to the response.
Now it is possible to map a header to the Method Response using:
integration.response.header.header-name
My question is...
Can I tell lambda to add my custom header to a binary response, when I'm
using custom lambda integration (not proxy) ?
Note: IF i was using lambda proxy, I know that the return object looks as below, and then I would be able to send custom headers. But for reasons out of my control, I cant use lambda proxy.
Lambda return object solution IF i was using lambda proxy:
return {
'body': "base64 encoded binary data",
'headers': 'x-my-header': 'my-value',
'isBase64Encoded': True
}
For Lambda Integration (not proxy) I have tried modifying my lambda output...
return {
"base64-data": "base64 encoded binary data",
"x-my-header: "some value"
}
And setting up a mapping template in the integration response...
$input.json("$.base64-data")
And setting up a header mapping using...
integration.response.body.x-my-header
But API Gateway returns an error:
Execution failed due to configuration error: Unable to transform response
I believe this error occurs because there cannot be a mapping template when you have CONVERT_TO_BINARY enable. From AWS docs:
When converting a text payload to a binary blob, API Gateway assumes that the text data is a Base64-encoded string and outputs the binary data as a Base64-decoded blob. If the conversion fails, it returns a 500 response indicating an API configuration error. You do not provide a mapping template for such a conversion, although you must enable the passthrough behaviors on the API.
I realize this is an old question, but I ran into a similar header mapping issue recently, even though I'm not using binary data.
To my mind the return from Lambda could look like this
{
"base64-data": "base64 encoded binary data",
"x-my-header: "some value"
}
Based on that Lambda Response, you could apply the following mapping (which I modified based on an AWS example)
$input.json("$.base64-data")
#set($context.responseOverride.header.x-my-header = "$input.json('$.x-my-header')")
$input.json("$") references your response and $.[key] is the right way to reference your sub-keys.
You also have to preconfigure your header "x-my-header" in your method response.
In the integration Response the header mapping can just be an empty value - e.g.
""
The real value will be provided by the override ($context.responseOverride.header) which is set by the mapping template.
Using the out the box Serverless template for C# .Net Core 2.0, I've created an S3 bucket that contains an image - I want to display this in a browser.
Running the webapi locally I can view the image fine and the content-length header isn't even included. When accessing via https://apidemo.boro2g.co.uk/media/mirrored-faces.jpg the content-length is getting changed to be bigger than it should. The original image is roughly 300kb but out of the API gateway its roughly 400kb.
I want to use the proxy to apply some custom security over the images so don't want them to be public, for testing you can see the original at https://s3-eu-west-1.amazonaws.com/boro2g-photogallery/mirrored-faces.jpg
I've tried adding the binary media types as well within the API but these don't seem to have any effect.
The code to 'get' the image is pretty simple:
[HttpGet("{key}")]
public async Task Get(string key)
{
try
{
var getResponse = await S3Client.GetObjectAsync(new GetObjectRequest
{
BucketName = BucketName,
Key = key
});
Response.ContentType = getResponse.Headers.ContentType;
getResponse.ResponseStream.CopyTo(Response.Body);
}
catch (AmazonS3Exception e)
{
Response.StatusCode = (int)e.StatusCode;
var writer = new StreamWriter(Response.Body);
writer.Write(e.Message);
}
}
Why would the API be updating the content-length incorrectly?
FYI static txt files load without problem: https://apidemo.boro2g.co.uk/media/test.txt
In the end the solution I found to work was:
In the settings for your api gateway, add the Binary Media Type: */*
On the {proxy+} method ‘Method Response’ add a 200 response header and add a header Content-Type. Finally publish your api
The second step here may not be necessary but I found the / didn’t kick in until I made the change.
In the api gateway settings, add */* (or the exact content types) to the Binary Media Type
Click the "Deploy API" button (didn't work without it)
I had to wait couple of minutes
Why is AWS Gateway setting the wrong content length for an image
streamed from S3?
Your API Gateway is not rendering the binary data properly, its assuming the data is string in a different encoding such as base64, following did fix the issue.
API Settings > Binary Media Types
*/*
API Method Response, Add HTTP Status 200, and set desired Content-Type header
I also had to apply following patch for the response body to be converted to binary.
aws apigateway update-integration-response \
--profile AWS_PROFILE_NAME \
--rest-api-id API_ID \
--resource-id RESOURCE_ID \
--region AWS_REGION \
--http-method GET \
--status-code 200 \
--patch-operations '[{"op" : "replace", "path" : "/contentHandling", "value" : "CONVERT_TO_BINARY"}]'
I have a lambda function which fetches a file from s3 using the input key in event and needs to send the same to client. I am using the following function to get the file from s3
function getObject(key){
var params = {
Bucket: "my_bucket",
Key: key
}
return new Promise(function (resolve, reject){
s3.getObject(params, function (err, data){
if(err){
reject(err);
}
resolve(data.Body)
})
})
}
If I send the response of this promise (buffer) to context.succeed, it is displayed as a JSON array on front end. How can I send it as a file ? The files can be either ZIP or HTTP Archive (HAR) files. The s3 keys contain the appropriate extension. I am guessing it has got something to do with the "Integration Response" in API Gateway. But not able to figure out where to change
Good news, you can now handle binary input and output for API Gateway (announcement and documentation).
Basically, nothing changes in your Lambda Function, but you can now set the contentHandling API Gateway Integration property to CONVERT_TO_BINARY.
Unfortunately, the official AWS examples showcase only the HTTP API Gateway backend, as the AWS Lambda support seems not complete yet. For example, I haven't managed to return gzipped content from AWS Lambda yet, although it should be possible thanks to the new binary support and the $util.base64Decode() mapping utility.