I'm attempting to use a cloudformation template to create a cloudwatch event rule that matches a glue event and targets an SNS topic to send a message to, I can create it in the cloudwatch console, but not via a cloud watch template. Here is my event rule:
NotifyEventRule:
Type: AWS::Events::Rule
Properties:
Name: JobNotifyEvent
Description: Notification event on job status change.
EventPattern:
source:
- aws.glue
account:
- !Ref AWS::AccountId
detail-type:
- Glue Job State Change
detail:
jobName:
- !Ref GlueJobName
Targets:
-
Arn:
Ref: "JobNotificationTopic"
Id:
Ref: "JobNotificationTopicName"
InputTransformer:
InputTemplate: "Job finished in the following state: <state>."
InputPathsMap:
state: "$.detail.state"
The problem is with InputTemplate. The error I get is:
Invalid InputTemplate for target JobNotificationTopic : [Source:
(String)"Job finished in the following state: null."; line: 1, column:
10]. (Service: AmazonCloudWatchEvents; Status Code: 400; Error Code:
ValidationException; Request ID: 12345678...)
It seems like <state> may be the problem.
The syntax for InputTemplate is for some reason quite strict in CloudFormation. It is of type string but it does not accept any form of valid YAML string.
In your case, you should use YAML Literal Block Scalar, |, before the input string.
InputTransformer:
InputPathsMap:
state: "$.detail.state"
InputTemplate: |
"Job finished in the following state: <state>."
If the input string is multiline, each line has to be enclosed in double quotes.
InputTransformer:
InputPathsMap:
state: $.detail.state
name: $.detail.name
InputTemplate: |
"Job <name> has just been run."
"Job finished in the following state: <state>."
Just to note that your string uses plain flow scalars, which is picky about the : character. Colon cannot appear before a space or newline. See Yaml multiline for further details. However, as I pointed out most of these YAML multiline rules does not apply to InputTemplate.
This is not an issue with YAML format as you get the same error using JSON.
The InputTemplate must be a string inside AWS:
InputTemplate=
"\"The Pipeline <pipeline> Started\""
This AWS Developer Forum topic gave me the answer: https://forums.aws.amazon.com/thread.jspa?messageID=798687
I believe the issue is that the InputTemplate is not a valid JSON. See: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-events-rule-inputtransformer.html#cfn-events-rule-inputtransformer-inputtemplate
The InputTemplate must be valid JSON.
Related
I am writing a SAM template for a business application that, among other things, processes device status updates.
Users post status updates to the API Gateway that, in turn, suitably sets the destSQS message attribute and publishes such requests to an SNS topic.
Several SQS queues are subscribed to the topic and messages are routed depending on the value of destSQS. The following is the AWS::SNS::Subscription resource definition for the SQS queue of interest for this question.
StatusUpdatesQueueSubscription: # accept msg iff destSQS is "StatusUpdates"
Type: AWS::SNS::Subscription
Properties:
Endpoint: !Join [ "-", [ !Sub "arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:${AWS::StackName}", !Ref StatusUpdatesQueueName ] ]
Protocol: sqs
FilterPolicy:
destSQS:
- "StatusUpdate"
RawMessageDelivery: True
TopicArn: !Ref DispatcherSNSTopic
So far everything works with "simple" (i.e., non-FIFO) SNS/SQS.
The following is the relevant section of the current OpenAPI (3.0.0) configuration embedded in the SAM template.
/devices/{device_id}/status:
post:
parameters:
- in: path
name: device_id
description: String ID of a device
required: true
schema:
type: string
description: Device unique identifier
security:
- ApiKeyAuth: [ ]
requestBody:
description: A status update for a device
required: true
content:
application/json:
schema:
type: object
responses:
"202":
description: Accepted
x-amazon-apigateway-integration:
type: "aws"
httpMethod: "POST"
uri: !Sub "arn:aws:apigateway:${AWS::Region}:sns:action/Publish"
credentials: !GetAtt DispatcherSNSTopicAPIRole.Arn
requestParameters:
integration.request.querystring.Message: "method.request.body"
integration.request.querystring.TopicArn: !Sub "'${DispatcherSNSTopic}'"
integration.request.querystring.MessageAttributes.entry.1.Name: "'destSQS'"
integration.request.querystring.MessageAttributes.entry.1.Value.DataType: "'String'"
integration.request.querystring.MessageAttributes.entry.1.Value.StringValue: "'StatusUpdate'"
responses:
default:
statusCode: 202
Now, I want to enrich this configuration to group messages by device_id (so that status updates for the same device will be processed in order).
I've already modified the template Resources section so that SQS queues and the topic are FIFO and tried to add the following mapping:
integration.request.querystring.MessageAttributes.entry.2.Name: "MessageGroupId"
integration.request.querystring.MessageAttributes.entry.2.Value.DataType: "'String'"
integration.request.querystring.MessageAttributes.entry.2.Value.StringValue: method.request.path.device_id
but I get the following error:
Resource handler returned message: "Errors found during import:
Unable to put integration on 'POST' for resource at path '/devices/{device_id}/status': Invalid mapping expression specified: Validation Result: warnings : [], errors : [Invalid mapping expression specified: MessageGroupId]
What am I missing?
After lots of searches and attempts, I've found the solution.
Perhaps sharing is useful for someone in the future.
MessageGroupId has to be set as a child of querystring and not as a MessageAttribute, as follows:
integration.request.querystring.MessageGroupId: "method.request.path.device_id"
the error was due to a syntax error: static strings must be enclosed by single quotes (and possibly, in turn, by double ones)
integration.request.querystring.MessageAttributes.entry.2.Name: "'MyAttributeName'"
I have the following step function in my AWS SAM template, it was defined using the syntax in the documentation. I'm using intrinsic functions to get some pseudoparameters but something is going wrong with them.
SfdcOrderEventsStepFunction:
Type: AWS::Serverless::StateMachine
Properties:
DefinitionSubstitutions:
Region: !Ref "AWS::Region"
AccountId: !Ref "AWS::AccountId"
EventBusPublishTarget: "order-events"
DefinitionUri: sfn-definition.asl.yml
Events:
EventBridgeEvent:
Type: EventBridgeRule
Properties:
EventBusName: sfdc-events
Pattern:
# TODO: Update pattern when the salesforce service is ready
source:
- prefix: salesforce.com
detail-type:
- Order
detail:
Payload__c:
attributes:
type:
- order
InputPath: $.detail
Name: sfdc-order-events
Role: !Sub 'arn:aws:iam::${AWS::AccountId}:role/stepfunction_execution_role'
Tracing:
Enabled: true
when I try to deploy it shows me the following error:
Resource template validation failed for resource
SfdcOrderEventsStepFunction as the template has invalid properties.
Please refer to the resource documentation to fix the template.
Properties validation failed for resource SfdcOrderEventsStepFunction
with message:
#/De finitionSubstitutions/ AccountId: 2 subschemas matched instead of one
#/DefinitionSubstitutions/AccountId: expected type: Boolean, found: String
At the end it deploys without problems. The step function and all of its components run as expected and without errors, but I wanted to know if there if something I can do to fix the template.
I am trying to add some filter patterns on dynamodb trigger to reduce the invocation on my lambda. I am using serverless to deploy the infra. I followed this instruction: https://blog.theodo.com/2022/04/dynamodb-streams-serverless-filter-patterns-and-you/ and the code looks like below.
But after the change, there is no pattern information on the lambda console. And the lambda is triggered for all changes not only the prefix pattern rule defined below. What did I do wrong in this case?
onStreamHandler:
handler: index.onStreamHandler
name: onStreamHandler
events:
- stream:
type: dynamodb
arn: ${self:custom.tableStreamArn}
startingPosition: LATEST
batchSize: 100
maximumRetryAttempts: 30
maximumRecordAgeInSeconds: 120
filterPatterns:
- eventName: [INSERT, MODIFY]
dynamodb:
NewImage:
type:
S:
- prefix: ['user']
I have the following resource in my SAM template.yml:
MetricsRule:
Type: AWS::Events::Rule
Properties:
Name: MetricsRule
Description: Puts metrics to the CloudWatch log group
EventBusName: !FindInMap [LambdaConfig, !Ref stage, eventBusName]
EventPattern:
detail:
status:
- GENERATED
State: !FindInMap [LambdaConfig, !Ref stage, eventEnabled]
Targets:
- Id: LogGroupTarget
Arn: !Sub "arn:aws:logs:${AWS::Region}:${AWS::AccountId}:log-group:${LogGroup}"
InputTransformer:
InputPathsMap:
customer-id: $.detail.customerId
destination-type: $.detail.destinationType
provider-id: $.detail.providerId
InputTemplate: '"cusId = <customer-id> - <destination-type> for <provider-id>"'
If I try to deploy this stack it fails with the exception Input for target LogGroupTarget is not a valid JSON text.
UPD:
If I put InputTemplate as '{"message":"cusId = <customer-id> - <destination-type> for <provider-id>"}' it will work. But, in the documentation, examples and even in the placeholder of the field in UI it stands that Input Template: A string containing placeholders which will be filled with values defined in Input Paths e.g. "The state of Instance <instance> is <state>"
Is there any option to specify InputTemplate as a string?
Thanks.
I hope author found answer.
But just to have answer here too.
This should work:
InputTemplate: |
"cusId = <customer-id> - <destination-type> for <provider-id>"
More information is here: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-events-rule-inputtransformer.html
I'm trying to set a Life cycle configuration for my S3 buckets to expire after 90 days. However, I'm getting an error saying "Property Status cannot be empty" when pushing my CFT stack.
I tried setting a lifestyle config, and putting the expiration in days onto that, but it seems to be failing.
AWSTemplateFormatVersion: '2010-09-09'
Description: Creates S3 Bucket
Resources:
TestBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub "${AWS::StackName}-test"
AccessControl: Private
LifecycleConfiguration:
Rules:
- Id: DeleteContentAfter90Days
Prefix: ''
Status: Enabled
ExpirationInDays: '90'
I'm getting "Property status cannot be empty" and an update rollback when I check my status in the console.
Status: 'Enabled'
Status should be string value as stated in the documentation
Here is a working example of LifecycleConfiguration:
LifecycleConfiguration:
Rules:
- Id: DeleteContentAfter1Day
Status: 'Enabled'
ExpirationInDays: 1
ExpirationInDays should be a number, not a string
Cloudformation can't take null values, remove this line - Prefix: ''