I have configured API gateway as a Kinesis proxy as described in Amazon’s tutorial for putting record into Kinesis stream.
The HTTP headers for integration request are:
stream-name is mapped to method.request.header.stream-name
partition-key is mapped to method.request.header.partition-key
Content-Type is mapped to application/x-amz-json-1.1
The body mapping template for content-type application/json looks like:
{
"StreamName": "$input.params('stream-name')",
"PartitionKey": "$input.params('partition-key')",
"Data": "$util.base64Encode({"rows": "$input.json('$')", "uuid": "$input.params('uuid')"})"
}
The data comes in as a JSON-request payload. We should add a uuid parameter as a JSON-object key into the data-payload for Kinesis, but the encoded data being sent to Kinesis isn’t a JSON object.
It is required to add a parameter uuid as a json-object key into the data-payload for kinesis. The problem I am facing is, that encoded data sent to kinesis is not a JSON object.
The data I expect to be sent to Kinesis should be a JSON object:
{
"rows": [{"id": 1, "name": "a"}, {"id": 2, "name": "b"}],
"uuid": "0001"
}
But the actual data sent to Kinesis looks like:
{
rows=[{"id": 1, "name": "a"}, {"id": 2, "name": "b"}],
uuid=0001
}
How can I send a JSON object from API gateway to Kinesis? I have been experimenting with $util.parseJson but couldn't find a solution for this scenario.
You need to escape double quotes in velocity templates
#set($event = "{
""rows"": ""$input.json('$')"",
""uuid"": ""$input.params('uuid')"",
}")
{
"StreamName": "$input.params('stream-name')",
"Data": "$util.base64Encode($event)",
"PartitionKey": "$input.params('partition-key')"
}
Here is a similar question on the aws forums for sending data to kinesis through api gateway https://forums.aws.amazon.com/thread.jspa?threadID=233060
Related
My Spring Cloud Function(v3.1.6) based AWS Lambda function (without Lambda proxy integration) returns a list of data via API Gateway in the following format:
{
"isBase64Encoded": false,
"headers": {
"id": "<some_id>",
"contentType": "application/json",
"timestamp": "1644307568294"
},
"body": "{\"resultList\":[{\"id\":\"1\",\"name\":\"item\",(...some other fields...)}]}",
"statusCode": 200
}
My problem here is I want to return response.body in a JSON (ofc I've also created a Model schema) :
{
"resultList": [
{
"id": "1", "name": "item ",(...some other fields...)
}
]
}
I've configured an application/json based Response Template Mapping to transform the response to the desired format:
$util.parseJson($input.json('$.body'))
which returned that I wanted (check the attached image):
But when I call it via Postman, I've got this:
{
"message": "Internal server error"
}
and in CloudWatch I can see this logs:
2022-02-08T08:56:00.688+01:00 (...) Endpoint response body before transformations: [Binary Data]
2022-02-08T08:56:00.694+01:00 (...) Execution failed due to configuration error: Unable to transform response
What can be the problem?
I'm trying to get my "queryStringParameters" in the event into the "body" key of the event so I can parse them in the same way that I parse post requests. Is this possible with a HTTP api? I tried using parameter mappings in the integration settings but this only allows me to append the body to the queryString.
The way that you get the values from a POST request using body and with query parameters are different. So both ca not be in the body property as you want.
The queryStringParameters is passed when you add a URL Query String Parameter in your Api gateway resource.
When you send a request to the event in your API the following event will be send to the Lamba function
{
"resource": "/{proxy+}",
"path": "/path/to/resource",
"httpMethod": "POST",
"isBase64Encoded": true,
"queryStringParameters": {
"foo": "bar"
},
"multiValueQueryStringParameters": {
"foo": [
"bar"
]
},
"pathParameters": {
"proxy": "/path/to/resource"
},
"stageVariables": {
"baz": "qux"
},
"body":"{ \"time\": \"evening\" }",
"headers": {
...
},
...
}
You be able to get the query parameters of event in your Lambda using the properties:
event.queryStringParameters && event.queryStringParameters.foo
And the body we need to get with the body property:
if (event.body) {
let body = JSON.parse(event.body)
if (body.time)
time = body.time;
}
In this way in your Lambda you need to parse them in different ways.
For more information of how to work with API Gateway and Lambda take a look here: Tutorial: Build a Hello World REST API with Lambda proxy integration
I am designing and implementing a backup plan to restore my client API keys. How to go about this ?
To fasten the recovery process, I am trying to create a backup plan for taking the backup of Client API keys, probably in s3 or local. I am scratching my head from past 2 days on how to achieve this ? May be some python script or something which will take the values from apigateway and dump into some new s3 bucket. But not sure how to implement this.
You can get all apigateway API keys list using apigateway get-api-keys. Here is the full AWS CLI command.
aws apigateway get-api-keys --include-values
Remember --include-values is must to use otherwise actual API Key will not be included in the result.
It will display the result in the below format.
"items": [
{
"id": "j90yk1111",
"value": "AAAAAAAABBBBBBBBBBBCCCCCCCCCC",
"name": "MyKey1",
"description": "My Key1",
"enabled": true,
"createdDate": 1528350587,
"lastUpdatedDate": 1528352704,
"stageKeys": []
},
{
"id": "rqi9xxxxx",
"value": "Kw6Oqo91nv5g5K7rrrrrrrrrrrrrrr",
"name": "MyKey2",
"description": "My Key 2",
"enabled": true,
"createdDate": 1528406927,
"lastUpdatedDate": 1528406927,
"stageKeys": []
},
{
"id": "lse3o7xxxx",
"value": "VGUfTNfM7v9uysBDrU1Pxxxxxx",
"name": "MyKey3",
"description": "My Key 3",
"enabled": true,
"createdDate": 1528406609,
"lastUpdatedDate": 1528406609,
"stageKeys": []
}
}
]
To get API Key detail of a single API Key, use below AWS CLI command.
aws apigateway get-api-key --include-value --api-key lse3o7xxxx
It should display the below result.
{
"id": "lse3o7xxxx",
"value": "VGUfTNfM7v9uysBDrU1Pxxxxxx",
"name": "MyKey3",
"description": "My Key 3",
"enabled": true,
"createdDate": 1528406609,
"lastUpdatedDate": 1528406609,
"stageKeys": []
}
Similar to get-api-keys call, --include-value is must here, otherwise actual API Key will not be included in the result
Now you need to convert the output in a format which can be saved on s3 and later can be imported to apigateway.
You can import keys with import-api-keys
aws apigateway import-api-keys --body <value> --format <value>
--body (blob)
The payload of the POST request to import API keys. For the payload
format
--format (string)
A query parameter to specify the input format to imported API keys.
Currently, only the CSV format is supported. --format csv
Simplest style is with two fields only e.g Key,name
Key,name
apikey1234abcdefghij0123456789,MyFirstApiKey
You can see the full detail of formats from API Gateway API Key File Format.
I have implemented it in python using a lambda for backing up APIs keys. Used boto3 APIs similar to the above answer.
However, I am looking for a way to trigger the lambda with an event of "API key added/removed" :-)
I am trying to validate an incoming XML payload via API gateway. To be specific, I actually don't even care about the schema, I just want to make sure that the body is not empty (and maybe that it is valid XML if I can get that functionality). I see a variety of posts from years ago stating that XML input validation is not yet supported in API Gateway.
Can somebody confirm if this is still the case? To provide a specific example, I have a model like this:
{
"$schema" : "http://json-schema.org/draft-04/schema#",
"title" : "Test Schema",
"type" : "object",
"minProperties": 1,
"properties": {
"account_id": {
"type": "string",
"pattern": "[A-Za-z]{6}[0-9]{6}"
}
},
"required": ["account_id"]
}
If I add request body validation using this model for content type "application/json" all is well, but if I do the same for content type "application/xml" no validation is performed.
Yes APIGW only supports json payload validation.
Is it possible to send request with: Content-Type: multipart/form-data to API Gateway?
In my case, I try to send form-data like below via Postman:
user[email]:extest829#ex.com
user[password]:password
user[password_confirmation]:password
user[username]:testUser
But It seems that API Gateway loses the content.
Everything works fine when I send it as: application/x-www-form-urlencoded or application/json.
Using mulipart/form-data is not fully supported by AWS API Gateway, especially when we try to send file via mulipart/form-data.
To send image along with other data from form, probably the best solution would be send it as JSON, where image is encoded with base64.
For example, when someone want to send:
Username (string)
Avatar (image)
Avatar should be encoded with base64. It gives an image as a text, like:
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyA...
The content of POST message would be:
{
"user_account": {
"avatar": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAyA...",
"username": "my name"
}
}
API Gateway
In API Gateway, under your API, open Models and create new model, for example UserAccount:
{
"$schema": "http://json-schema.org/draft-04/schema#",
"title": "UserAccountUpdate",
"type": "object",
"properties": {
"user": {
"type": "object",
"properties": {
"avatar": { "type": "string" },
"username": { "type": "string" }
}
}
}
}
In Method Request -> HTTP Request Headers set: Content-Type.
In Method Request -> Request Body set: application/json and as model use created UserAccount model.
In Integration Request -> Content Handling set as: Passthrough.
In Integration Request -> Body Mapping Templates choose: When no template matches the reuqest Content-Type header. (You can also use two other options here and set additional mapping templates).
Backend
Backend receives an image as a text encoded with base64. So probably before it uses futher, it needs to decode it from text to image.
Encoding/decoding images with base64 is quite popular, so you should find some appropriate tools / libs in your frameworks.
When you are sending request with Content-Type: multipart/form-data to API Gateway, you need to pass through your original Content-Type header to your integration endpoint.
aws apigateway update-integration \
--rest-api-id a1b2c3d4e5 \
--resource-id a1b2c3 \
--http-method POST \
--patch-operations op='replace',path='/requestParameters/integration.request.header.Content-Type',value='method.request.header.Content-Type'
Code example for managing binary data in AWS Gateway and Proxy+ Lambda. Or to upload and retrieve files from servers using AWS Gateway and Proxy Lambda+
Click here