Here is the error.
An error occurred: SNSTopicPolicy - Invalid parameter:
Policy Error: null (Service: AmazonSNS;
Status Code: 400; Error Code: InvalidParameter;
Request ID: 38a567df-2cff-50bf-8f0e-33a91775cc6e).
I cannot find a place to look into logs to tell me what parameter is missing.?.? I'm pretty sure I have everything that is required. I'm using this site and their api pages.
SNSTopic:
Type: AWS::SNS::Topic
Properties:
DisplayName: Aura main Topic
TopicName: ${file(./env.yml):${opt:stage, self:provider.stage}.env.auraSnsTopicName}
SNSTopicPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
PolicyDocument:
Id: auraAllowSQSsendrobelrobel
Statement:
-
Effect: Allow
Action:
- sns: Publish
Resource: { "Fn::GetAtt":["SQSQueue" ,"Arn"]}
Principle:
AWS: "*"
Topics:
- { "Ref": "SNSTopic" }
SQSQueue:
Type: AWS::SQS::Queue
Properties:
QueueName: ${file(./env.yml):${opt:stage, self:provider.stage}.env.eeegPagesQueueName}
RedrivePolicy:
deadLetterTargetArn: {"Fn::GetAtt" : [ "SQSQueueDLQ", "Arn" ]}
maxReceiveCount: 2
SQSQueuePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
PolicyDocument:
Id: allowSNSSourceAndLambdaTrigger
Statement:
-
Effect: Allow
Action:
- SQS:ReceiveMessage
- SQS:SendMessage
- SQS:ChangeMessageVisibility
- SQS:ListDeadLetterSourceQueues
- SQS:GetQueueUrl
# - lambda:CreateEventSourceMapping
# - lambda:ListEventSourceMappings
# - lambda:ListFunction
Resource: {"Ref": "SNSTopic"}
Queues:
- { "Ref": "SQSQueue" }
I have an SQS Policy statement that is working just fine.
Your Environment Information ----------------------------
OS: Mac darwin
Node Version: 8.12.0
Serverless Version: 1.32.0
Turns out you need spelling is important and also the AWS sub object was not needed.
Principal: "*"
Working Policy below:
SNSTopicPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
PolicyDocument:
Statement:
- Sid: auraAllowSQSPublish
Effect: Allow
Principal: "*"
Action: "sns:Publish"
Resource: { "Ref": "SNSTopic" }
Topics:
- { "Ref": "SNSTopic" }
Related
I'm trying to give SNS the permission to publish to an SQS queue but I keep getting error:
Invalid parameter: Policy Error: null (Service: AmazonSNS; Status Code: 400; Error Code: InvalidParameter; Request ID: a5459d05-e37a-5906-92c2-c16c4813cca0; Proxy: null)
I am using the serverless framework, and have read multiple posts describing the same issue but none of the fixes seem to work, here is my current version:
MyTopicPolicy:
Type: AWS::SNS::TopicPolicy
Properties:
PolicyDocument:
Id: 'allowsSNSToSQS'
Version: '2012-10-17'
Statement:
Sid: AllowSNStoPublishToSQS
Effect: Allow
Principle: "*"
Action: 'sqs:SendMessage'
Resource:
Fn::GetAtt: [MyQueue , Arn ]
Condition:
ArnEquals:
aws:SourceArn: { "Ref" : "MyTopic" }
Topics:
- { "Ref" : "MyTopic" }
Does anyone have any ideas? thanks
Along with the typo I was also using the wrong policy type, I should've been using AWS::SQS::QueuePolicy as can be seen below:
MyTopicPolicy:
Type: AWS::SQS::QueuePolicy
Properties:
Queues:
- {"Ref": "MyQueue"}
PolicyDocument:
Id: 'allowsSNSToSQS'
Version: '2012-10-17'
Statement:
Sid: AllowSNStoPublishToSQS
Effect: Allow
Principal:
Service: "sns.amazonaws.com"
Action: "sqs:SendMessage"
Resource:
Fn::GetAtt: [MyQueue , Arn ]
Condition:
ArnEquals:
aws:SourceArn: { "Ref" : "MyTopic" }
I want CloudWatch to send CreateLogGroup messages to EventBridge.
I understand this is possible, but also that Cloudwatch doesn't seem to send these messages by default. It seems you have to configure CloudTrail to get it to forward the message. But I can't find a CloudTrail configuration which works - in general deployment fails with:
AWS::CloudTrail::Trail - "Invalid request provided: Incorrect S3 bucket policy is detected for bucket"
AWSTemplateFormatVersion: '2010-09-09'
Outputs:
HelloFunction:
Value:
Ref: HelloFunction
WatcherFunction:
Value:
Ref: WatcherFunction
WatcherTrailBucket:
Value:
Ref: WatcherTrailBucket
Parameters:
MemorySizeDefault:
Default: '512'
Type: String
RuntimeVersion:
Default: '3.8'
Type: String
TimeoutDefault:
Default: '5'
Type: String
Resources:
HelloFunction:
Properties:
Code:
ZipFile: |
def handler(event, context):
print (event)
Handler: index.handler
MemorySize:
Ref: MemorySizeDefault
Role:
Fn::GetAtt:
- HelloFunctionRole
- Arn
Runtime:
Fn::Sub: python${RuntimeVersion}
Timeout:
Ref: TimeoutDefault
Type: AWS::Lambda::Function
HelloFunctionRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName:
Fn::Sub: hello-function-role-policy-${AWS::StackName}
Type: AWS::IAM::Role
WatcherFunction:
Properties:
Code:
ZipFile: |
def handler(event, context):
print (event)
Handler: index.handler
MemorySize:
Ref: MemorySizeDefault
Role:
Fn::GetAtt:
- WatcherFunctionRole
- Arn
Runtime:
Fn::Sub: python${RuntimeVersion}
Timeout:
Ref: TimeoutDefault
Type: AWS::Lambda::Function
WatcherFunctionRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName:
Fn::Sub: watcher-function-role-policy-${AWS::StackName}
Type: AWS::IAM::Role
WatcherEventRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
source:
- aws.logs
detail-type:
- "AWS API Call via CloudTrail"
detail:
eventName:
- CreateLogGroup
Targets:
- Id:
Fn::Sub: watcher-event-rule-${AWS::StackName}
Arn:
Fn::GetAtt:
- WatcherFunction
- Arn
State: ENABLED
WatcherEventRulePermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
Principal: events.amazonaws.com
FunctionName:
Ref: WatcherFunction
SourceArn:
Fn::GetAtt:
- WatcherEventRule
- Arn
WatcherTrailBucket:
Type: AWS::S3::Bucket
WatcherTrailBucketPolicy:
DependsOn:
- WatcherTrailBucket
Type: AWS::S3::BucketPolicy
Properties:
Bucket:
Ref: WatcherTrailBucket
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: cloudtrail.amazonaws.com
Action: s3:GetBucketAcl
Resource: "*"
Condition: {}
- Effect: Allow
Principal:
Service: cloudtrail.amazonaws.com
Action: s3:PutObject
Resource: "*"
Condition:
StringEquals:
s3:x-amz-acl: bucket-owner-full-control
WatcherTrail:
Type: AWS::CloudTrail::Trail
Properties:
EventSelectors:
- ReadWriteType: All
IsLogging: true
S3BucketName:
Ref: WatcherTrailBucket
IsLogging: true
S3KeyPrefix: logs/
Your WatcherTrail runs before WatcherTrailBucketPolicy, so that's why it fails (CloudFormation does not deploy resource in the order of their definition in the template). Add explicit DependsOn dependency on the bucket policy. Also your WatcherTrailBucketPolicy is incorrect, and trial needs a name. So it should be:
WatcherTrailBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket:
Ref: WatcherTrailBucket
PolicyDocument: !Sub |
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck20150319",
"Effect": "Allow",
"Principal": {"Service": "cloudtrail.amazonaws.com"},
"Action": "s3:GetBucketAcl",
"Resource": "arn:aws:s3:::${WatcherTrailBucket}",
"Condition": {
"StringEquals": {
"aws:SourceArn": "arn:aws:cloudtrail:${AWS::Region}:${AWS::AccountId}:trail/MyTrial"
}
}
},
{
"Sid": "AWSCloudTrailWrite20150319",
"Effect": "Allow",
"Principal": {"Service": "cloudtrail.amazonaws.com"},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::${WatcherTrailBucket}/logs/AWSLogs/${AWS::AccountId}/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control",
"aws:SourceArn": "arn:aws:cloudtrail:${AWS::Region}:${AWS::AccountId}:trail/MyTrial"
}
}
}
]
}
WatcherTrail:
Type: AWS::CloudTrail::Trail
DependsOn: WatcherTrailBucketPolicy
Properties:
TrailName: MyTrial
EventSelectors:
- ReadWriteType: All
IsLogging: true
S3BucketName:
Ref: WatcherTrailBucket
IsLogging: true
S3KeyPrefix: logs/
I wanted to use the ARN as parameter input to cloudformation stack resources EventRuleRegion1 - Target as well as EventBridgeIAMrole , but it is not working. when i call with Ref function
Original ARN
arn:aws:events:ap-southeast-2:123456789123:event-bus/central-eventbus-sydney
When i give the arn directly in code its working fine.
Code
AWSTemplateFormatVersion: 2010-09-09
Parameters:
EventBridgeName:
Description: Enter the Event Bridge Name
Type: String
Default: ec2-lifecycle-events
EventBusName:
Description: Enter the Central Event Bus Name
Type: String
Default: central-eventbus-sydney
EventBusArn:
Description: Enter the ARN of Central Event Bus
Type: String
Default: arn:aws:events:ap-southeast-2:123456789123:event-bus/central-eventbus-sydney
Monitoringaccount:
Description: Enter the Monitoring AWS account number
Type: String
Default: 123456789123
Resources:
EventRuleRegion1:
Type: AWS::Events::Rule
Properties:
Description: Event rule to send events to monitoring account event bus
EventBusName: default
EventPattern:
source:
- aws.ec2
detail-type:
- "EC2 Instance State-change Notification"
detail:
state:
- "running"
- "stopped"
- "terminated"
Name: !Ref EventBridgeName
State: ENABLED
Targets:
- Arn: >-
- !Join [ "", [ !Sub "arn:aws:events:${AWS::Region}:123456789123:event-bus/",!Ref EventBusName ] ]
Id: !Ref EventBusName
RoleArn: !GetAtt
- EventBridgeIAMrole
- Arn
EventBridgeIAMrole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: !Sub events.amazonaws.com
Action: 'sts:AssumeRole'
Path: /
Policies:
- PolicyName: PutEventsDestinationBus
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'events:PutEvents'
Resource:
- >-
- !Join [ "", [ !Sub "arn:aws:events:${AWS::Region}:123456789123:event-bus/",!Ref EventBusName ] ]
Error
Parameter - !Join [ "", [ !Sub "arn:aws:events:${AWS::Region}:123456789123:event-bus/",!Ref EventBusName ] ] is not valid. Reason: Provided Arn is not in correct format. (Service: AmazonCloudWatchEvents; Status Code: 400; Error Code: ValidationException; Request ID: 0d52a1d6-095e-44f7-9455-b7481dc4fb8d; Proxy: null)
The use of >- will result in literal strings, not evaluation of your CFN functions (join, ref). It should be:
Targets:
- Arn: !Join [ "", [ !Sub "arn:aws:events:${AWS::Region}:123456789123:event-bus/",!Ref EventBusName ] ]
Considering this lambda function on a serverless.yml file:
functions:
s3toEc2Lambda:
handler: s3toec2lambda.S3toEc2Lambda
name: "${self:service}-s3toEc2Lambda"
role: S3toEc2LambdaRole
And considering this SNS created on resources section: Does someone knows how to inform the Sns ARN Endpoint from the lambda function s3toEc2Lambda ?
resources:
Resources:
WordpressFrontEndSnsS3toEc2:
Type: AWS::SNS::Topic
Properties:
TopicName: "wordpress-front-end-s3-ec2"
WordpressFrontEndSnsS3toEc2Lambda:
Type: AWS::SNS::Subscription
Properties:
Endpoint: { "Fn::GetAtt": ["s3toEc2Lambda", "Arn" ] } <------ HERE <------
#Endpoint: ${self:functions.s3toEc2Lambda} <------ OR HERE <------
#Endpoint: { "Fn::GetAtt": ["${self:functions.s3toEc2Lambda}", "Arn" ] } <------ OR HERE <------
Protocol: lambda
TopicArn: !Ref 'WordpressFrontEndSnsS3toEc2'
For me always appear a error message like this: "Template error: instance of Fn::GetAtt references undefined resource s3toEc2Lambda"
Thank You !
CloudFormation resources created by serverless have known format. For lambda function this is:
{normalizedFunctionName}LambdaFunction
Thus you should be able to reference your function using the following:
"Fn::GetAtt": [ S3toEc2LambdaLambdaFunction, Arn ]
More example about this are here
We can create Function Roles, Functions Policy and Lambda functions SAM template.yml file by this
Type: AWS::IAM::Role
Properties:
RoleName: UatAdminUserStatsLambda
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Principal:
Service:
- 'lambda.amazonaws.com'
Action:
- 'sts:AssumeRole'
FunctionPolicy:
Type: AWS::IAM::Policy
DependsOn: FunctionRole
Properties:
PolicyName: UserStatsPolicy
Roles:
- !Ref FunctionRole
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Action:
- 'iam:GetUser'
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:GetLogEvents'
- 'logs:PutLogEvents'
- 's3:GetObject'
- 's3:PutObject'
Resource: '*'
adminUsersList:
Type: AWS::Serverless::Function
Properties:
CodeUri: src/admin/
Role: !GetAtt FunctionRole.Arn
Handler: adminUsersList.adminUsersList
Layers:
- !Ref NodeDependenciesLayer
Events:
adminUsersListEvent:
Type: Api
Properties:
Path: /api/admins
Method: GET
I'm creating an ASG group which has a lifecyclehook for termination:
LifecycleHook:
Type: AWS::AutoScaling::LifecycleHook
Properties:
AutoScalingGroupName: !Ref NodeGroup
DefaultResult: CONTINUE
HeartbeatTimeout: 60
LifecycleHookName: !Sub "${AWS::StackName}-lifecycle-hook"
LifecycleTransition: autoscaling:EC2_INSTANCE_TERMINATING
Now I create a lambda function as well:
LambdaCreation:
Type: "AWS::Lambda::Function"
Properties:
Handler: "lambda_function.lambda_handler"
Environment:
Variables:
aws_region : !Ref AWSRegion
Role: !GetAtt LambdaExecutionRole.Arn
Code:
S3Bucket: !Ref LambdaCodeBucket
S3Key: "lambda-functions/function.zip"
Runtime: "python3.6"
Timeout: 60
On cloudwatch events, i created a rule for said event:
CloudwatchEvent:
Type: AWS::Events::Rule
Properties:
Description: ASG scale-in event to lambda
EventPattern: {
"source": [
"aws.autoscaling"
],
"detail-type": [
"EC2 Instance-terminate Lifecycle Action"
],
"detail": {
"AutoScalingGroupName":
[
{
"Fn::ImportValue" :
{
"Fn::Sub" : "${RootStackName}-nodes-asg-name"
}
}
]
}
}
State: ENABLED
Targets:
-
Arn:
!GetAtt LambdaCreation.Arn
Id:
!Ref LambdaCreation
But the lambda is never triggered.
Now, on AWS console I don't see a trigger on the designer. But if i add manually a cloudwatch trigger for the created rule, it starts working...
Why is the trigger on the lambda side not created? What am I missing?
Thanks all!
I faced the exact same frustration. Only difference is that I was using terraform but that's irrelavant.
You are missing this:
{
"Type" : "AWS::Lambda::Permission",
"Properties" : {
"Action" : String,
"EventSourceToken" : String,
"FunctionName" : String,
"Principal" : String,
"SourceAccount" : String,
"SourceArn" : String
}
}
The reason the "manual way" works because it creates the trigger AND the permission. When you provision stuff using IaC tools like Cloudformation/terraform, you need to explicitly specify this Lambda permission object.
The below code snippet creates a lambda function and creates a cloudwatch event to trigger the lambda function with necessary privileges.
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: root
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:*
Resource: arn:aws:logs:*:*:*
- Effect: Allow
Action:
- s3:ListBucket
Resource: !Join [ '', [ 'arn:aws:s3:::', !Ref LambdaS3Bucket ] ]
- Effect: Allow
Action:
- s3:GetObject
Resource: !Join [ '', [ 'arn:aws:s3:::', !Ref LambdaS3Bucket, '/*' ] ]
- Effect: Allow
Action:
- sts:GetCallerIdentity
Resource: '*'
LambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Description: "Lambda function"
FunctionName: !Ref LambdaFunctionName
Handler: !Ref LambdaHandler
Runtime: !Ref LambdaRuntime
Timeout: !Ref LambdaTimeout
MemorySize: !Ref LambdaMemorysize
Role: !GetAtt LambdaExecutionRole.Arn
Code:
S3Bucket: !Ref LambdaS3Bucket
S3Key: !Ref LambdaS3BucketKey
Environment:
Variables:
time_interval_in_hours: !Ref TimeIntervalInHours
DependsOn: LambdaExecutionRole
CleanupEventRule:
Type: AWS::Events::Rule
Properties:
Description: "Cloudwatch Rule"
ScheduleExpression: !Ref CloudwatchScheduleExpression
State: !Ref CloudWatchEventState
Targets:
- Arn: !Sub ${LambdaFunction.Arn}
Id: "CleanupEventRule"
DependsOn: LambdaFunction
LambdaSchedulePermission:
Type: AWS::Lambda::Permission
Properties:
Action: 'lambda:InvokeFunction'
FunctionName: !Sub ${LambdaFunction.Arn}
Principal: 'events.amazonaws.com'
SourceArn: !Sub ${CleanupEventRule.Arn}
DependsOn: LambdaFunction