I have a lambda that needs to return a binary object and some http headers (e.g. content-type) through an api gateway (using lambda integration) OR redirect to another URL. In the binary support examples (e.g. https://aws.amazon.com/blogs/compute/binary-support-for-api-integrations-with-amazon-api-gateway/) the lambda only returns the (base64 of the) binary object (the image). In my case, I also need to return a status code and http headers (or something equivalent). I struggle with how I can make this work with binary support in api gateway.
The lambda returns a json on this form:
{
"statusCode": 200,
"headers": {
"content-type": "image/jpeg"
},
"body": "/9j/4AAQS...gLDAoKCAwZK",
"isBase64Encoded": true
}
In the integration response I add body mappings for image/jpeg (etc) of the form:
$input.json('$.body')
And header mapping for 'content-type' like so:
integration.response.body.headers['content-type']
I've tried many variations of the above, but the result is consistently
Execution failed due to configuration error: Unable to transform response
How do I transform the json from the lambda into a form that can be converted to binary by the api gateway, with http headers and all? Can I get more debug logging out of the api gateway to show more specific what it is unhappy with?
Is there perhaps a way to get more debug logging out of the api gateway?
I recently got this working after facing a similar problem.
In my case, I was missing two things:
First, I needed to change the list of types AWS will send to the upstream in the "Accept" header"
"x-amazon-apigateway-binary-media-types" : [
"image/jpeg"
]
Secondly, I needed to set the Integration Response to "Convert to binary (if needed)":
"contentHandling": "CONVERT_TO_BINARY"
See this answer for the details, and sample config.
I also found that I wasn't being patient enough. Whenever I deployed the API, I was checking immediately, instead of waiting a few minutes for the changes to propagate.
I tired returning binary data with base64 encoding, but I couldn't manage to return it from Lambda function through API Gateway.
Therefore, I decided to redirect the URL. I changed the Method Response to 302 and added "Location" Response header. I also deleted response code 200 from Integration Response, selected 302 as response code and mapped integration.response.body.location value with Location header. My Lambda code returned location of redirect URL in this format:
{"location":"www.google.com/"}
Hope this helps.
I've been struggling to do the same for days, but couldn't find any documentation to back up that it's possible, instead I found this
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings-workflow.html
When converting a text payload to a binary blob, API Gateway assumes that the > text data is a Base64-encoded string and outputs the binary data as a
Base64-decoded blob. If the conversion fails, it returns a 500 response
indicating an API configuration error. You do not provide a mapping template
for such a conversion, although you must enable the passthrough behaviors on
the API.
When converting a binary payload to a text string, API Gateway always applies > a Base64 encoding on the binary data. You can define a mapping template for
such a payload, but can only access the Base64-encoded string in the mapping
template through $input.body, as shown in the following excerpt of an example > mapping template.
which sounds to me that mapping like this is only possible the other way around; what I ended up doing is that just return Base64Encoded binary string and hard code the content-type and cache- control in header mapping; for the complete implementation see my blog post https://mesfinmoges.com/dynamic-image-resizing-using-amazon-s3-aws-lambda-mazon-api-gateway-amazon-cloudfront/
Related
This is similar to this post API-Gateway Integration Request HTTP Header not mapping query string to header but I can't see any answer for it and all of the answers are not related to what the question is intended.
I am trying to add the integration request parameters in API Gateway, but whenever I set 'Http Headers' as shown here and 'Mapping template' as passthrough, I could not see that header when I log inside the lambda.
Also in the integration response I cannot reference it inside the integration header response parameters. The "integration.response.header.cokers" will be returned blank when I invoke the API. This is how I configured the integration response
Ultimately the solution here is to implement a lambda proxy integration.
A proxy integration in API Gateway tells API Gateway to simply forward all headers to the integration for processing, which means you will see all of those values in your lambda function.
NOTE: Lambda has to return a specific response format back to API Gateway if it's a proxy integration. Ultimately, the response should be something like:
{
'statusCode': 200,
'headers': {
'echo-proxy': True
},
'body': 'some payload'
}
What you are trying to do right now is map everything manually, which is a deprecated approach and usually you don't want to do that unless you absolutely have to because it's kind of a pain.
If you have to map the headers manually start by mapping them in the the method request so it can carry on to the next step and so on. Basically like this:
Method Request -> Maps variable to Integration Request -> Maps variable to Body Mapping Template -> Maps variable to actual request header
What you have in your screenshot for the Integration Request -> HTTP Headers is:
Name: cokers
Mapped from: 'blah'
However, "mapped from" should look something like "method.request.header.coker" which is a standardized path (meaning to get the value from the Method Request Header field with name "coker").
Once you have added the coker header to the Method Request, and the Integration Request HTTP Headers are mapped correctly, you have to implement a mapping template. Set the content-type to application/json with passthrough set to "When there are no templates defined(recommended)" and a simple mapping template:
{
"headers": {
"coker" : "$input.params('coker')"
}
}
That is the way my API is setup and it returns the following to me because I had my lambda function return the event as a json object back to API GW:
{"body": "{\"headers\": {\"coker\": \"mapped\"}}", "statusCode": 200}
NOTE: the value of my header "coker" in the request on the client side is "mapped"
UPDATED ANSWER
To map the original "coker" header to "coker2" (or any other name you want to give it) you simply set the name of the header in your mapping template like so:
{
"headers": {
"coker2" : "$input.params('coker')"
}
}
Then edit your lambda function to return "coker2" header and you should get a response like this:
{"body": "{\"headers\": {\"coker2\": \"mapped\"}}", "statusCode": 200}
I have a service that returns image data as base64 encoded text in the body:e.g.
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAASwAAAEsCAYAAAB5fY51AAAgAE...etc
However, when we put the service behind the api-gateway behaving as a proxy, the same request will seemingly convert the response to binary
�PNG
IHDR,,y}�u IDA... etc
and has a response header
Content-Type: application/json
The documentation explains how to convert TO binary automatically, but makes no mention on how to leave the response unmodified. I've tried changing contentHandling to CONVERT_TO_TEXT with no success, so I'm wondering if there's another way of having the api gateway not mess with my service responses.
I believe you are using the same setup as me;
API gateway
V
Proxy (no contentHandling available)
V
Lambda (returns {body: Buffer.toString('base64')})
This solved it for me: https://stackoverflow.com/a/47780921/853015.
I'm using API Gateway Lambda proxy integration and trying to return a binary application/protobuf response. No matter what I do, the response body is always a base64 encoded string
I have application/protobuf setup as a binary media types in APIG
My client (javascript) is sending following headers in the POST:
Accept: application/protobuf
Content-Type: application/protobuf
My lambda is responing with content-type: application/protobuf, and correctly setting the IsBase64Encoded Lambda response to true
How do you get APIG to base64 decode the string? I swear I had this working a few months ago when I 1st tried this.
Note: I've also tried */* as a binary media types
Some related posts to add background:
https://github.com/twitchtv/twirp/issues/81
https://github.com/awslabs/aws-serverless-express/issues/39#issuecomment-276019222
Update:
Turns out I can only get it working if binary media type is set to */*. The client Accept header has no impact once it is set to this.
Many bad side effects of using */* because every response is attempted to get decoded (even when IsBase64Encoded is false or not set)
I thought it wasn't decoding because Chrome network inspect tools will always show binary data as base64 encoded in the Preview tab. You can see the protobuf in the Response tab.
The problem was I'm using CloudFront in front of API Gateway, and I was not passing the Accept header to the origin (APIG).
The docs on handling binary with Lambda proxy are not great, so here is a quick summary:
Your client must send an Accept header who's 1st media type matches what you have set as a binary media types in API Gateway
Your Lambda, if serving a binary media type, must set IsBase64Encoded to true AND the body must be base64 encoded
If the clients Accept header matches an entry in API Gateway's binary media types and these conditions are met, API Gateway will transform (base64 decode) before sending a response to the client.
This blog post walks you through step-by-step on how to get it working (without CloudFront).
This is a full blown aws-blueprint for getting a production grade ci/cd with CloudFront.
My API Gateway/Lambda setup returns an HTTP response header:
Lambda uses callback function to return the value as part of a JSON
and the Integration Response maps it into a HTTP header (using integration.response.body)
With this solution, the values are sent back both in the body and the header.
How can I map headers from the Lambda response without duplicating the values in the response body?
If you have Lambda proxy integration enabled, you can set the response headers as part of Lambda output and API Gateway will return them as part of the HTTP response to the client.
Node.js example:
callback(null, {
"isBase64Encoded": false, // Set to `true` for binary support.
"statusCode": 200,
"headers": {
"header1Name": "header1Value",
"header2Name": "header2Value",
},
"body": "...",
});
where headers can be null or unspecified if no extra response headers are to be returned.
See Output Format of a Lambda Function for Proxy Integration.
and, if you DON'T have Lamba proxy integration enabled, you can add (and map) the response headers in the amazon API gateway console:
go to resources -> method execution -> method response -> add 'Access-Control-Allow-Origin' (or whatever) header for http status 200. Then go back to method execution -> integration response -> http status 200 -> set header mapping for 'Access-Control-Allow-Origin' to '*' (or whatever).
Solved this error...: "No 'Access-Control-Allow-Origin' header is present on the requested resource"
Since the question states that custom mappings are being used (using integration.response.body), it means Lambda Proxy Integrations are NOT being used. So, the solution, in this case, is to map the headers the way you are already doing.
To remove the headers duplication from the body part, use mapping template in the integration response and ignore headers in the mapping. I think you might be using pass through responses, that's why you are seeing duplicate headers.
See more documentation here: https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html
So I have CORS enabled going through the basic setup given by AWS Gateway. However for this API I need to allow Control Origins for all requests and allow credentials.
Here is what it looks like
The issue as you may have guessed is this setup is not allowed by CORS, you can not have a wildcard for Origin and have credentials as true. Normally the work around for this is to just grab the requesting domain and add it into the Origin Header. Which is more or less what I want to do. But I don't know how to get that information and add it as a mapping value. Where does API Gateway store that information and how do i get it?
UPDATE:
I have to pass through HTTP Header Host to my Lambda Function which I should have mentioned earlier, I have tried implementing the Answer below but I cannot access the header to pass it to the Lambda function using the instructions provided. Any more assistance with this is greatly appreciated.
Okay After hours of research and finding bits of information across the internet I have a solution and hopefully it is useful for other people.
To pass an HTTP Header that is not a default value provided by AWS API Gateway and then access that data via a Lambda Function and return that data in the Response Header follow the steps below
In "Method Request" go to "HTTP Request Headers" and add your desired header to capture. ie. if we want to get the host value of the API url you can enter "Host" here. if you want to get the website host of the caller use "Origin"
In "Integration Request" go to mapping templates and create a new template if an "application/json" does not exist, if it does just update it.
This is the important part, pass the header value you set in step 1. To do that write something similar to the following in the Template Box.
{
"origin" : "$input.params().header.Origin",
"host" : "$input.params().header.Host"
}
You can also pass in any url parameters you have defined in the same JSON.
Access the data from Lambda, The integration request passed the information into the "Event" parameter if using Node as the Lambda Backend code. to retrieve the value of any header just use the following within your handler.
event.origin;
When sending back your response from Lambda to API Gateway, it is best to format the response in JSON. Something similar to this.
{
"origin" : event.origin,
"host" : event.host,
"nonHeaderOutput" : "Hello World"
}
In "Integration Response" go to "Header Mappings", if the header you need is not listed you may add it in "Method Response" and it will then appear here. For this example I used "Access-Control-Allow-Origin" and edited the Mapping Value to be integration.response.body.origin
now go to "Mapping Templates and select the content type you want to use, and then edit the template to access the non header responses by adding this to the Template Box
$input.path("$.nonHeaderOutput")
That is it now the header that was sent to the API can be used in your method Response.
Jurgen from API Gateway here.
Thanks for reporting this issue. There is currently no simple way to set it up via the "Enable CORS" feature in the console. However, we are looking into it to improve the user experience for this use case.
As a potential workaround, you could passthrough the Origin header from the client to your backend and parse / create the value for the Access-Control-Allow-Origin header there. Then you would map the Header in the Integration Response from 'integration.response.header.Access-Control-Allow-Origin' to Access-Control-Allow-Origin and return it to the client.
Hope this helps.
Best,
Jurgen