I am using EventBridge Pipes to connect DynamoDB Streams directly to EventBridge. However, I am having difficulty transforming DynamoDB list attributes to my desired format using the Pipes Target Input Transformer.
What I Want
When I insert an item into DynamoDB, I want to publish a UserCreated event to EventBridge without a Lambda Function. I also want to remove the DynamoDB formatting from the event payload.
My DDB Stream event payload looks like this:
{
"eventID": "c814968f8803051fa5700a2a0b9fe599",
"eventName": "INSERT",
"eventVersion": "1.1",
"eventSource": "aws:dynamodb",
"awsRegion": "us-east-1",
"dynamodb": {
"ApproximateCreationDateTime": 1672592664,
"Keys": {
"sk": {
"S": "PROFILE"
},
"pk": {
"S": "USER#testuser"
}
},
"NewImage": {
"username": {
"S": "testuser"
},
"roles": {
"L": [{
"S": "admin"
},{
"S": "otherrole"
}]
},
"SequenceNumber": "3044600000000017324272463",
"SizeBytes": 505,
"StreamViewType": "NEW_AND_OLD_IMAGES"
}
}
}
I want my EventBridge event payload to look like this:
{
data:{
username: "testuser"
roles: ["admin","otherrole"]
},
metadata:{
eventType: "UserCreated"
}
}
However, using the following Input Transformer
{
"data":{
"username": <$.dynamodb.NewImage.username.S>,
"roles": <$.dynamodb.NewImage.roles.L>
},
"metadata":{
"eventType":"UserCreated"
}
}
I get the following output (notice the roles field still contains DDB formatting)
{
"data": {
"username": "testuser",
"roles": [{
"S": "admin"
},{
"S": "otherrole"
}]
},
"metadata": {
"eventType": "UserCreated"
}
}
I do not want to publish DynamoDB formatted lists in my event payloads.
What I've Tried
If I list out each list index manually, I'm able to get close to what I want
{
"roles": [<$.dynamodb.NewImage.roles.L.0.S>,<$.dynamodb.NewImage.roles.L.1.S>]
}
Which gives me the following output:
{
"roles": ["admin","otherrole"]
}
However, the size of this list is dynamic, so this approach will not work for me.
I've also used JSONPath.com to verify the following JSONPath expression is valid. However, the Pipes Input Transformer tells me this is an invalid syntax.
$.dynamodb.NewImage.roles[*][*].S
I may be able to use a Lambda Function to format the data to my liking in the Enrichment step of my Pipe. However, introducing a Lambda Function would defeat the purpose of using Pipes for my use case.
Related
I am building a AWS LexV2 chat bot that I want to integrate with Facebook Messenger via Channel Integration provided by LexV2. I am also using Lambda codehook to validate my inputs and fulfil my intents.
Following the docs, everything works as expected, except for one thing. When I log the event in my lambda function, requestAttributes field is missing from the object
My lambda function code:
import logging
logger = logging.getLogger()
logger.setLevel(logging.DEBUG)
def lambda_handler(event, context):
logger.error(event)
return ...
When I send a message to my app/page on facebook, this is what is logged:
{
"sessionId": "session-id",
"inputTranscript": "Intent utterance",
"interpretations":
[
{
"intent":
{
"slots":
{
"slot1": null,
"slot2": null
},
"confirmationState": "null",
"name": "IntentName",
"state": "InProgress"
},
"nluConfidence": 1.0
},
{
"intent":
{
"slots":
{},
"confirmationState": "null",
"name": "FallbackIntent",
"state": "InProgress"
}
}
],
"responseContentType": "text/plain; charset=utf-8",
"invocationSource": "DialogCodeHook",
"messageVersion": "1.0",
"sessionState":
{
"intent":
{
"slots":
{
"slot1": null,
"slot2": null
},
"confirmationState": "null",
"name": "IntentName",
"state": "InProgress"
},
"originatingRequestId": "req-id"
},
"bot":
{
"aliasId": "ALIASID",
"aliasName": "AliasName",
"name": "BotName",
"version": "DRAFT",
"localeId": "en_US",
"id": "bot-id"
},
"inputMode": "Text"
}
As you can see, no requestAttributes.
What's weird to me is that when I do the same thing for LexV1 bot (same facebook page/messenger app), I get these fields, e.g.
{
"requestAttributes":
{
"x-amz-lex:facebook-page-id": "page-id",
"x-amz-lex:channel-id": "channel-id",
"x-amz-lex:webhook-endpoint-url": "webhook-endpoint",
"x-amz-lex:accept-content-types": "PlainText",
"x-amz-lex:user-id": "user-id",
"x-amz-lex:channel-name": "channel-name",
"x-amz-lex:channel-type": "Facebook"
}
}
If anyone has any tips (other than "switch to V1") I would be very grateful. Thanks :))
Notes:
Changed Python Nones to nulls to easier format JSON
Replaced IDs and names from the logs with generic-id-slash-names
I have a situation where I need to filter out certain events using eventpatterns in eventbridge.
I want to run the rule for all events except those where username starts with abc or xyz.
I have tried below 2 syntax but none worked :
"userIdentity": {
"sessionContext": {
"sessionIssuer": {
"userName": [
{
"anything-but": {
"prefix": [
"abc-",
"xyz-"
]
}
}
]
}
}
}
"userIdentity": {
"sessionContext": {
"sessionIssuer": {
"userName": [
{
"anything-but": [{
"prefix": "abc-",
"prefix": "xyz-"
}]
}
]
}
}
}
Getting following error on saving the rule :
"Event pattern is not valid. Reason: Inside anything but list, start|null|boolean is not supported."
Am I missing something in the syntax or if this is a limitation then is there any alternative to this problem?
You can use prefix within an array in event pattern. Here is an example pattern:
{
"detail": {
"alarmName": [{
"prefix": "DemoApp1"
},
{
"prefix": "DemoApp2"
}
],
"state": {
"value": [
"ALARM"
]
},
"previousState": {
"value": [
"OK"
]
}
}
}
This event will match alarm that has name starting with either DemoApp1 or DemoApp2
TLDR: user #samtoddler is sort of correct.
Prefix matches only work on values as called out in https://docs.aws.amazon.com/eventbridge/latest/userguide/content-filtering-with-event-patterns.html#filtering-prefix-matching. They do not work with arrays. You can file a feature request with AWS support but if you'd like to unblock yourself; you it's probably best to either control the prefixes you have for userName (guessing this is something IAM related and in your control).
If that's not possible; consider filtering as much as you can via other properties before sending over to a compute (probably lambda) to performing additional filtering.
I have been working with Alexa Skills Kit for some time now with my code deployed in AWS Lambda written in Node Js. Now i want to integrate chatbots into it via the amazon Lex service. So that i am able to control my device using both Amazon Alexa and Amazon lex. My question was that if i use the same intent and slots name in Amazon Lex as i have did in my Alexa Skill would the AWS Lambda code just work out of the box? Or would i have to modify the AWS Lambda code to accommodate the AWS Lex?
You will have to accommodate for the differences between Lex and Alexa. The most notable differences are the request and response formats.
Notable differences to be aware of:
Major differences between the formats and passing of sessionAttribtues and slots.
Lex has 4 built-in slotTypes that Alexa does not use (yet?): AMAZON.EmailAddress,, AMAZON.Percentage, AMAZON.PhoneNumber, AMAZON.SpeedUnit, and AMAZON.WeightUnit. (Reference.)
Lex always passes the full user input through inputTranscript. Alexa does not.
Alexa provides resolutions for slot values but fills the actual slot value with the raw data extracted from the input.
Lex will automatically resolve a slot value if you have synonyms set for that slotType.
After working with both of them quite a lot, and often dealing with this, I much prefer Lex to Alexa. I have found Lex to be simpler and provides more developer freedom and control, even though you do have to conform to the restrictions of each of Lex's output channels.
Compare Request / Response Formats:
Alexa JSON format
Lex JSON format
Example Alexa Request:
{
"version": "1.0",
"session": {
"new": true,
"sessionId": "amzn1.echo-api.session.[unique-value-here]",
"application": {
"applicationId": "amzn1.ask.skill.[unique-value-here]"
},
"attributes": {
"key": "string value"
},
"user": {
"userId": "amzn1.ask.account.[unique-value-here]",
"accessToken": "Atza|AAAAAAAA...",
"permissions": {
"consentToken": "ZZZZZZZ..."
}
}
},
"context": {
"System": {
"device": {
"deviceId": "string",
"supportedInterfaces": {
"AudioPlayer": {}
}
},
"application": {
"applicationId": "amzn1.ask.skill.[unique-value-here]"
},
"user": {
"userId": "amzn1.ask.account.[unique-value-here]",
"accessToken": "Atza|AAAAAAAA...",
"permissions": {
"consentToken": "ZZZZZZZ..."
}
},
"apiEndpoint": "https://api.amazonalexa.com",
"apiAccessToken": "AxThk..."
},
"AudioPlayer": {
"playerActivity": "PLAYING",
"token": "audioplayer-token",
"offsetInMilliseconds": 0
}
},
"request": {}
}
Example Lex Request:
{
"currentIntent": {
"name": "intent-name",
"slots": {
"slot name": "value",
"slot name": "value"
},
"slotDetails": {
"slot name": {
"resolutions" : [
{ "value": "resolved value" },
{ "value": "resolved value" }
],
"originalValue": "original text"
},
"slot name": {
"resolutions" : [
{ "value": "resolved value" },
{ "value": "resolved value" }
],
"originalValue": "original text"
}
},
"confirmationStatus": "None, Confirmed, or Denied (intent confirmation, if configured)"
},
"bot": {
"name": "bot name",
"alias": "bot alias",
"version": "bot version"
},
"userId": "User ID specified in the POST request to Amazon Lex.",
"inputTranscript": "Text used to process the request",
"invocationSource": "FulfillmentCodeHook or DialogCodeHook",
"outputDialogMode": "Text or Voice, based on ContentType request header in runtime API request",
"messageVersion": "1.0",
"sessionAttributes": {
"key": "value",
"key": "value"
},
"requestAttributes": {
"key": "value",
"key": "value"
}
}
I have created a subscription to an SNS topic where all of the events will be from S3 ObjectCreated:Put actions. I only want to receive the notifications where the S3 object key contains the string 'KLWX'. What should that filter policy look like? The notification data is below, however the 'Message' attribute value is given as a string, not a JSON object. I just expanded it for easier reading.
{
"SignatureVersion": "1",
"Type": "Notification",
"TopicArn": "xxx",
"Message": {
"Records": [{
"eventVersion": "2.0",
"eventSource": "aws:s3",
"awsRegion": "us-east-1",
"eventTime": "2018-01-18T20:16:27.590Z",
"eventName": "ObjectCreated:Put",
"userIdentity": {
"principalId": "xxx"
},
"requestParameters": {
"sourceIPAddress": "xxx"
},
"responseElements": {
"x-amz-request-id": "6CF3314E6D6B7671",
"x-amz-id-2": "tJdr3KDcAsp1tuGdo6y4jBLkYXsEDEeVPcvQ1SWQoLXWsZL81WUzbloDe1HxbhGes4u0tY9Jh+g="
},
"s3": {
"s3SchemaVersion": "1.0",
"configurationId": "NewNEXRADLevel2Object",
"bucket": {
"name": "xxx",
"ownerIdentity": {
"principalId": "xxx"
},
"arn": "xxx"
},
"object": {
"key": "KCBW/881/20180118-201329-015-I",
"size": 16063,
"eTag": "772cd2d2e82b22448792308755891350",
"sequencer": "005A61009B8EC82991"
}
}
}]
},
"UnsubscribeURL": "xxx",
"Signature": "xxx",
"Timestamp": "2018-01-18T20:16:27.626Z",
"SigningCertURL": "xxx",
"Subject": "Amazon S3 Notification",
"MessageId": "ed6a0365-4af2-5497-9be0-51be4829cdee"
}
You have to do this on the S3. When you create the event, you can use a combination of prefix/suffix to filter which object sends the notification to your SNS topic.
Assuming the bucket name is YourBucket, and your object key is KCBW/881/20180118-201329-015-I, you have to configure the S3 event on YourBucket with prefix = KLWX/
You can subscribe and add a target for an AWS Lambda function. In that function, you can code up your logic. If needed, you could send out another SNS message from there, or store/process the data as needed.
Filter policy similar to the one below worked for me. This requires FilterPolicyScope to be set to MessageBody.
"Records" : {
"s3" : {
"object" : {
"key" : [
{ "prefix" : "KLWX/" }
]
}
}
}
I have defined my intent schema and sample utterances and it works fine. I can also test it with Service Simulator and Amazon dot.
I want to write a code which gets a text and create the IntentRequest based on it (knowing the intentSchema). I can see that Service Simulator is doing something similar: it receives the text, make the IntentRequest and show it in the left box named Lambda Request.
How can do the same? receive the text and recognize the intent and slots and convert it to IntentRequest (obviously, not manually).
example:
input: "How is the weather in Austin?"
output: a IntentRequest object similar to this:
{
"session": {
"sessionId": "....",
"application": {
"applicationId": "... "
},
"attributes": {},
"user": {...},
"new": true
},
"request": {
"type": "IntentRequest",
"requestId": "reqid",
"locale": "en-US",
"timestamp": "...",
"intent": {
"name": "WeatherIntent",
"slots": {...},
}
},
"version": "1.0"
}
You should use a dedicated dialog service such as Lex, API.AI, or Watson
They will get text and return the intent