As per aws tutorial the kinesis stream PutRecord expects base64encoded JSON string as data.
In my case I need to create an object inside the template mapping for integration request before it gets to the kinesis stream.
Code example from API gateway template mapper:
#set($inputRoot = $input.path('$'))
#set($data = {
"session": "$inputRoot.session",
"customer": "$inputRoot.customer",
"ip": "$context.identity.sourceIp"
})
{
"StreamName": "metrics-stream",
"Data": "$util.base64Encode($data)",
"PartitionKey": "$input.path('$.session')",
}
The problem: Above code doesn't work since $data is not valid JSON string. How one can convert the $data to valid JSON string? Could't find a $util to do that inside template mapper.
Thanks!
Related
AWS enhanced subscription filtering feature documentation recommends to add the following response mapping template:
## Response Mapping Template - onSpecialTicketCreated subscription
$extensions.setSubscriptionFilter($util.transform.toSubscriptionFilter($util.parseJson($ctx.args.filter)))
$util.toJson($context.result)
When using a simple request mapping template the subscription will not return any data:
{
"version": "2017-02-28"
}
The documentation does not mention the following (got it though one month of back and forth with support):
In the request template the payload must be set:
{
"version": "2017-02-28",
"payload": {}
}
If you have fieldLevel resolver note that for some reason they are executed when the subscription is set up and a payload is set. To accommodate this make sure they handle that i.e. event.source.id is not defined. For them to work pass the args as payload:
{
"version": "2017-02-28",
"payload": $util.toJson($ctx.args)
}
Using this frontend side filters like {"filter" : "{\"severity\":{\"le\":2}}"} will work again
I'm trying to do a mapping in an api gateway and I can't manage to access the children objects inside the returned json. This is my case:
When I test the endpoint directly in the api gateway I get this response:
{
"status": "FAIL",
"output": {
"errorCode": "my code",
"message": "my message"
}
}
And the api gateway integration response mapping is as follows:
#set($inputRoot = $input.path("$.output"))
$inputRoot
But I just want to return the json inside the output key, so I tried the following:
#set($inputRoot = $input.path("$.output"))
$inputRoot.output
And when I run it a get no data.
Before the transformation, the return value is
{
"output":"{\"status\":\"FAIL\",\"output\":{\"errorCode\":\"my code\",\"message\":\"my message\"}}"
}
I think that the fact that is returned as string might have something to do, but I've tried with $util.parseJson and $util.escapeJavaScript and I had no luck.
Does anyone know how can I solve this? I can't change the integration response, I have to do it through the api gateway mapping.
It should be JSON-like:
#set($inputRoot = $input.path("$.output"))
{
"output": "$inputRoot"
}
Is there a way to filter messages based on Regex or substring in AWS SNS?
AWS Documentation for filtering messages mentions three types of filtering for strings:
Exact matching (whitelisting)
Anything-but matching (blacklisting)
Prefix matching
I want to filter out messages based on substrings in the messages, for example
I have a S3 event that sends a message to SNS when a new object is added to S3, the contents of the message are as below:
{
"Records": [
{
"s3": {
"bucket": {
"name": "images-bucket"
},
"object": {
"key": "some-key/more-key/filteringText/additionaldata.png"
}
}
}
]
}
I want to keep the messages if only filteringText is present in key field.
Note: The entire message is sent as text by S3 notification service, so Records is not a json object but string.
From what I've seen in the documentation, you can't do regex matches or substrings, but you can match prefixes and create your own attributes in the MessageAttributes field.
To do this, I send the S3 event to a simple Lambda that adds MessageAttributes and then sends to SNS.
In effect, S3 -> Lambda -> SNS -> other consumers (with filtering).
The Lambda can do something like this (where you'll have to programmatically decide when to add the attribute):
let messageAttributes = {
myfilterkey: {DataType: "String", StringValue:"filteringText"}
};
let params = {
Message: JSON.stringify(payload),
MessageAttributes: messageAttributes,
MessageStructure: 'json',
TargetArn: SNS_ARN
};
await sns.publish(params).promise();
Then in SNS you can filter:
{"myfilterkey": ["filtertext"]}
It seems a little convoluted to put the Lambda in there, but I like the idea of being able to plug and unplug consumers from SNS on the fly and use filtering to determine who gets what.
I am trying to save data in S3 through firehose proxied by API gateway. I have create an API gateway endpoint that uses the AWS service integration type and PutRecord action for firehose. I have the mapping template as
{
"DeliveryStreamName": "test-stream",
"Records": [
#foreach($elem in $input.path('$.data'))
{
"Data": "$elem"
}
#if($foreach.hasNext),#end
#end
]
}
Now when I test the endpoint with below JSON
{
"data": [
{"ticker_symbol":"DemoAPIGTWY","sector":"FINANCIAL","change":-0.42,"price":50.43},{"ticker_symbol":"DemoAPIGTWY","sector":"FINANCIAL","change":-0.42,"price":50.43}
]
}
JSON gets modified and shows up as below after the transformation
{ticker_symbol=DemoAPIGTWY, sector=FINANCIAL, change=-0.42, price=50.43}
: is being converted to = which is not a valid JSON
Not sure if something is wrong in the above mapping template
The problem is, that $input.path() returns a json object and not a stringified version of the json. You can take a look at the documentation here.
The Data property expects the value to be a string and not a json object. So long story short - currently there is no built in function which can revert a json object into its stringified version. This means you need to re read the current element in the loop via $input.json(). This will return a json string representation of the element, which you then can add as Data.
Take a look at the answer here which illustrates this concept.
In your case, applying the concept described in the link above would result in a mapping like this:
{
"DeliveryStreamName": "test-stream",
"Records": [
#foreach($elem in $input.path('$.data'))
{
#set($json = $input.json("$[$foreach.index]"))
"Data":"$util.base64Encode($json)",
}
#if($foreach.hasNext),#end
#end
]
}
API Gateway considers the payload data as a text and not as a Json unless explicitly specified.
Kinesis also expects data to be in encoded format while proxying through API Gateway.
Try the following code and this should work, wondering why the for loop has been commented in the mapping template.
Assuming you are not looping through the record set, the following solution should work for you.
{
"DeliveryStreamName": "test-stream",
"Record": {
"Data": "$util.base64Encode($input.json('$.Data'))Cg=="
}
}
Thanks & Regards,
Srivignesh KN
I have created one AWS Lex bot and I am invoking one lambda function from that bot. When testing the lambda function I am getting proper response but at bot I am getting below error:
An error has occurred: Received invalid response from Lambda: Can not
construct instance of IntentResponse: no String-argument
constructor/factory method to deserialize from String value
('2017-06-22 10:23:55.0') at [Source: "2017-06-22 10:23:55.0"; line:
1, column: 1]
Not sure, what is wrong and where I am missing. Could anyone assist me please?
The solution to above problem is that we need to make sure response returned by lambda function, to be used at AWS lex chat bot should be in below format:
{
"sessionAttributes": {
"key1": "value1",
"key2": "value2"
...
},
"dialogAction": {
"type": "ElicitIntent, ElicitSlot, ConfirmIntent, Delegate, or Close",
Full structure based on the type field. See below for details.
}
}
By this, chat bot expectd DialogAction and corresponding elements in order to process the message i.e. IntentResponse.
Reference: http://docs.aws.amazon.com/lex/latest/dg/lambda-input-response-format.html
no String-argument constructor/factory method to deserialize from String value
You are getting this error because you must be passing string values in the response of lambda function. You have to pass a predefined json object blueprint in the response.
Because the communication between Lex and Lambda is not simple value passing like normal functions. Amazon Lex expects output from Lambda in a particular JSON format and data is sent to Lambda in a particular JSON. The examples are here: Lambda Function Input Event and Response Format.
And just copying and pasting the blueprint won't work because in some fields you have choose between some predefined values and in some fields you have to entry valid input.
For example in,
"dialogAction": {
"type": "Close",
"fulfillmentState": "Fulfilled or Failed",
"message": {
"contentType": "PlainText or SSML",
"content": "Thanks, your pizza has been ordered."
}
}
you have assign a value "Fulfilled" or "Failed" to field 'fulfillmentState'. And same goes for 'contentType'.