I want to send an unique id to my cloudwatch event.
The below image shows the request id. This unique id has to be changed.
Below is the serverless.yml code for my lambda function.
I tried to add an input (InputTransformers) but it didn't do much.
What can i do here???
functions:
stream-function:
handler: src/stream-handler/stream.handler
memorySize: 1024 #in MB, also control CPU throughput
timeout: 31 #in seconds, match with downstream to avoid concurrent request
events:
- stream:
type: kinesis
arn:
Fn::GetAtt: [HagoStream, Arn]
batchSize: 1
parallelizationFactor: 2
Id: "my cloud watch rule name"
InputTransformer: {
InputTemplate: '{"uniqueId": "96c80428-14fe-c8d0-f6e3-639384992391"}',
InputPathsMap: {
id: '$.uniqueId'
}
}
maximumRetryAttempts: 2
destinations:
onFailure:
arn:
Fn::GetAtt: [HagoFailedQueue, Arn]
type: sqs
environment:
DLQ_URL:
Ref: HagoDlqQueue
This is AWS assigned unique ID for your lambda function invocation. You can't change it.
RequestId – The unique request ID for the invocation.
Instead, in your function code, you can generate some extra ID that you output in CloudWatch Logs alongside with other data of yours.
Related
I am writing a SAM template for a business application that, among other things, processes device status updates.
Users post status updates to the API Gateway that, in turn, suitably sets the destSQS message attribute and publishes such requests to an SNS topic.
Several SQS queues are subscribed to the topic and messages are routed depending on the value of destSQS. The following is the AWS::SNS::Subscription resource definition for the SQS queue of interest for this question.
StatusUpdatesQueueSubscription: # accept msg iff destSQS is "StatusUpdates"
Type: AWS::SNS::Subscription
Properties:
Endpoint: !Join [ "-", [ !Sub "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:${AWS::StackName}", !Ref StatusUpdatesQueueName ] ]
Protocol: sqs
FilterPolicy:
destSQS:
- "StatusUpdate"
RawMessageDelivery: True
TopicArn: !Ref DispatcherSNSTopic
So far everything works with "simple" (i.e., non-FIFO) SNS/SQS.
The following is the relevant section of the current OpenAPI (3.0.0) configuration embedded in the SAM template.
/devices/{device_id}/status:
post:
parameters:
- in: path
name: device_id
description: String ID of a device
required: true
schema:
type: string
description: Device unique identifier
security:
- ApiKeyAuth: [ ]
requestBody:
description: A status update for a device
required: true
content:
application/json:
schema:
type: object
responses:
"202":
description: Accepted
x-amazon-apigateway-integration:
type: "aws"
httpMethod: "POST"
uri: !Sub "arn:aws:apigateway:${AWS::Region}:sns:action/Publish"
credentials: !GetAtt DispatcherSNSTopicAPIRole.Arn
requestParameters:
integration.request.querystring.Message: "method.request.body"
integration.request.querystring.TopicArn: !Sub "'${DispatcherSNSTopic}'"
integration.request.querystring.MessageAttributes.entry.1.Name: "'destSQS'"
integration.request.querystring.MessageAttributes.entry.1.Value.DataType: "'String'"
integration.request.querystring.MessageAttributes.entry.1.Value.StringValue: "'StatusUpdate'"
responses:
default:
statusCode: 202
Now, I want to enrich this configuration to group messages by device_id (so that status updates for the same device will be processed in order).
I've already modified the template Resources section so that SQS queues and the topic are FIFO and tried to add the following mapping:
integration.request.querystring.MessageAttributes.entry.2.Name: "MessageGroupId"
integration.request.querystring.MessageAttributes.entry.2.Value.DataType: "'String'"
integration.request.querystring.MessageAttributes.entry.2.Value.StringValue: method.request.path.device_id
but I get the following error:
Resource handler returned message: "Errors found during import:
Unable to put integration on 'POST' for resource at path '/devices/{device_id}/status': Invalid mapping expression specified: Validation Result: warnings : [], errors : [Invalid mapping expression specified: MessageGroupId]
What am I missing?
After lots of searches and attempts, I've found the solution.
Perhaps sharing is useful for someone in the future.
MessageGroupId has to be set as a child of querystring and not as a MessageAttribute, as follows:
integration.request.querystring.MessageGroupId: "method.request.path.device_id"
the error was due to a syntax error: static strings must be enclosed by single quotes (and possibly, in turn, by double ones)
integration.request.querystring.MessageAttributes.entry.2.Name: "'MyAttributeName'"
I want to trigger a lambda whenever a new EC2 instance is registred in SSM's Fleet Manager (meaning the instance can be connected to using SSM), however I can't find what pattern to use in EventBridge.
Within EventBridge, I tried using the following pattern I found in the docs (so far its looks like the closest thing to my goal):
{
"source": ["aws.ssm"],
"detail-type": ["Inventory Resource State Change"]
}
However when I create a new EC2 and wait for its SSM agent to become active, it still doesn't trigger the above pattern.
Any idea how to catch this kind of event?
I think you have to go through CloudTrail API call.
Please find below a CloudFormation template I used in the past that was working. Please note that it just provides the SSM resources. You need to add your own SQS queue as well (see SQS.ARN) and I've used the association with the tag registration set to enabled. So that if you have a lambda function connected, you can set it to false so if the instance connect again, it won't go to the same process again.
AWSTemplateFormatVersion: "2010-09-09"
Description: >
SSM Registration event
# Description of the resources to be created.
Resources:
RegistrationDocument:
Type: AWS::SSM::Document
Properties:
DocumentType: Command
Content:
schemaVersion: "2.2"
description: >
An Automation Document ran by registered instances that gathers their software inventory
and automatically updates their AWS SSM Agent to the latest version.
mainSteps:
- name: GatherSoftware
action: aws:softwareInventory
- name: Sleep
action: aws:runShellScript
inputs:
runCommand:
- sleep 20 || true
- name: UpdateAgent
action: aws:updateSsmAgent
inputs:
agentName: amazon-ssm-agent
source: https://s3.{Region}.amazonaws.com/amazon-ssm-{Region}/ssm-agent-manifest.json
allowDowngrade: "false"
RegistrationDocumentAssociation:
Type: AWS::SSM::Association
Properties:
AssociationName: !Sub registration-association-${AWS::StackName}
Name: !Ref RegistrationDocument
Targets:
- Key: tag:registration
Values:
- enabled
RegistrationEventRule:
Type: AWS::Events::Rule
Properties:
Description: >
Events Rule that monitors registration of AWS SSM instances
and logs them to an SQS queue.
EventPattern:
source:
- aws.ssm
detail-type:
- AWS API Call via CloudTrail
detail:
eventName:
- UpdateInstanceAssociationStatus
requestParameters:
associationId:
- !Ref RegistrationDocumentAssociation
executionResult:
status:
- Success
State: ENABLED
Targets:
- Arn: SQS.ARN
Id: SqsRegistrationSubscription
SqsParameters:
MessageGroupId: registration.events
I am trying to add some filter patterns on dynamodb trigger to reduce the invocation on my lambda. I am using serverless to deploy the infra. I followed this instruction: https://blog.theodo.com/2022/04/dynamodb-streams-serverless-filter-patterns-and-you/ and the code looks like below.
But after the change, there is no pattern information on the lambda console. And the lambda is triggered for all changes not only the prefix pattern rule defined below. What did I do wrong in this case?
onStreamHandler:
handler: index.onStreamHandler
name: onStreamHandler
events:
- stream:
type: dynamodb
arn: ${self:custom.tableStreamArn}
startingPosition: LATEST
batchSize: 100
maximumRetryAttempts: 30
maximumRecordAgeInSeconds: 120
filterPatterns:
- eventName: [INSERT, MODIFY]
dynamodb:
NewImage:
type:
S:
- prefix: ['user']
I need to trigger a lambda when a specific object registers on DynamoDB.
For example:
If I create a User with a POST /my-website/user and, I store this User on DynamoDB, I want to trigger
my Lambda.
I don't want to trigger the Lambda if the registered object is
different from the User.
For the management of my stack, I use Serverless (with a serverless.yml file) and CloudFormation syntax.
With the serverless documentation, I can't figure out how I can trigger my Lambda only when a specific entry is registered to DynamoDB ( https://www.serverless.com/framework/docs/providers/aws/events/streams ).
Thanks in advance,
EDIT:
Thank you for your answers :)
It's work:
statement:
handler: lambda/statement.php
layers:
- arn:aws:lambda:#{AWS::Region}:<account_id>:layer:php-73:1
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:ListStreams
- dynamodb:GetItem
events:
- stream:
type: dynamodb
arn: arn:aws:dynamodb:eu-west-3:<account_id>:table/dev-project/stream/2020-11-18T22:34:01.579
maximumRetryAttempts: 1
batchSize: 1
filterPatterns:
- eventName: [INSERT]
dynamodb:
NewImage:
__partitionKey:
S: [myPk]
You have to setup stream filters. The process is explained in:
NEW: DynamoDB Streams Filtering in Serverless Framework
You attach the DynamoDB stream event onto the lambda in your serverless file in the function's "events" sections. https://carova.io/snippets/serverless-aws-dynamodb-stream-to-lamba-function
If you're creating the Dynamo table using serverless, you could output the table's StreamArn as a stack variable.
https://carova.io/snippets/serverless-aws-cloudformation-output-stack-variables
If they're in the same file, you don't need the Output section, you could just set the "events" sections of the lambda and reference
arn: { Fn::GetAtt: [DynamoTable, StreamArn] }
in the arn section of the specific event.
Stream Filters worked (thank you). Here's the final configuration:
statement:
handler: lambda/statement.php
layers:
- arn:aws:lambda:#{AWS::Region}:<account_id>:layer:php-73:1
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:ListStreams
- dynamodb:GetItem
events:
- stream:
type: dynamodb
arn: arn:aws:dynamodb:eu-west-3:<account_id>:table/dev-project/stream/2020-11-18T22:34:01.579
maximumRetryAttempts: 1
batchSize: 1
filterPatterns:
- eventName: [INSERT]
dynamodb:
NewImage:
__partitionKey:
S: [myPk]
As described in the documentation, when creating a RDS Event subscription you can select any number Event categories that will produce specific Event messages.
Then you can choose to send a notification to an E-mail, SMS or, which is my case, to a SNS topic that triggers a Lambda execution.
How to access the RDS Event ID - i.e. RDS-EVENT-0006 - from the Lambda event parameter?
Add a trigger event like this cloudformation example :
DbRestoredEventRule:
Type: AWS::Events::Rule
Properties:
Name: "xyz-db-restored"
Description: "xyz restored"
EventPattern:
source:
- "aws.rds"
detail-type:
- "RDS DB Instance Event"
detail:
EventCategories:
- "availability"
Message:
- 'DB instance restarted'
Targets:
- Arn:
Fn::GetAtt:
- "MigrationDataFunction"
- "Arn"
Id: "TargetFunctionV1"