I want to create a lambda function through cloudformation template, I have ConfigurationLambdaRole, ConfigurationLambdaFunction and ConfigurationLambdaInvokePermission, in ConfigurationLambdaInvokePermission section, what should be the SourceArn? And is there any incorrect thing with my template?
Resources:
ConfigurationLambdaRole:
Type: "AWS::IAM::Role"
Properties:
RoleName: 'configuration-lambda'
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- events.amazonaws.com
- s3.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AmazonSQSFullAccess
- arn:aws:iam::aws:policy/CloudWatchLogsFullAccess
ConfigurationLambdaFunction:
Type: AWS::Lambda::Function
Properties:
Description: 'configuration service with lambda'
FunctionName: 'configuration-lambda1'
Handler: lambda.handler.EventHandler::handleRequest
Runtime: Java 11
MemorySize: 128
Timeout: 120
Code:
S3Bucket: configurationlambda
S3Key: lambda-service-1.0.0-SNAPSHOT.jar
Role: !GetAtt ConfiguratioLambdaRole.Arn
ConfigurationLambdaInvokePermission:
Type: AWS::Lambda::Permission
Properties:
FunctionName:
Fn::GetAtt:
- ConfigurationLambdaFunction
- Arn
Action: 'lambda:InvokeFunction'
Principal: "s3.amazonaws.com"
SourceArn: 'arn of jar file in s3(configurationlambda)'
SourceArn is an arn of a resource which is going to invoke your function. For example, if your lambda would be invoked through S3 Events Notifications, the SourceArn would be the ARN of your bucket.
In your case I don't see why would you need AWS::Lambda::Permission. So I would just remove the entire resource.
Related
I couldn't get a valid IAM policy to work for a Lambda function to OpenSearch.
Replicate:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Sub ${AWS::StackName}-Replicate
Description: !Sub
- Stack ${StackTagName} Environment ${EnvironmentTagName} Function ${ResourceName}
- ResourceName: DBReplicate
CodeUri: ../src/Replicate
Handler: index.handler
Runtime: nodejs16.x
MemorySize: 3008
Timeout: 30
Tracing: Active
Policies:
- PolicyName: Access
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- es:*
Resource:
- arn:aws:es:eu-west-1:22222222222:domain/mynewdomain
- DomainName: mynewdomain
Events:
MyDynamoDBtable:
Type: DynamoDB
Properties:
Stream: !Ref TableStreamArn
StartingPosition: TRIM_HORIZON
BatchSize: 1
Running sam validate, I'm getting:
Policy at index 0 in the 'Policies' property is not valid
Got the answer from another post, the structure was wrong, the correct way is:
Add inline policy to aws SAM template
QueryFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: query/
Handler: app.lambda_handler
Policies:
- AmazonDynamoDBFullAccess
- AWSLambdaVPCAccessExecutionRole
- Version: '2012-10-17' # Policy Document
Statement:
- Effect: Allow
Action:
- dynamodb:*
Resource: 'arn:aws:dynamodb:*:*:table/dynamo_db_table_endpoint'
I have a Cloudformation stack which connects two Lambdas together via a Lambda destination config and an SQS queue.
The idea is that ErrorsFunction is fired when there is an error in HelloAddFunction.
This stack deploys fine, and HelloAddFunction works fine when I invoke it with some integer values.
I can see an error in HelloAddFunction when I invoke it with non- integer values, but no corresponding error seems to be received by ErrorsFunction.
The binding of ErrorsFunction to ErrorsQueue seems to be working - if I push a message onto ErrorsQueue via the console, it's received by ErrorsFunction.
So it feels like the Lambda destination config is somehow not working.
What am I missing here ?
TIA
AWSTemplateFormatVersion: '2010-09-09'
Outputs: {}
Parameters:
AppName:
Type: String
MemorySizeSmall:
Default: 512
Type: Number
RuntimeVersion:
Default: '3.8'
Type: String
TimeoutShort:
Default: 5
Type: Number
Resources:
HelloAddEventConfig:
Properties:
DestinationConfig:
OnFailure:
Destination:
Fn::GetAtt:
- ErrorsQueue
- Arn
FunctionName:
Ref: HelloAddFunction
MaximumRetryAttempts: 0
Qualifier: $LATEST
Type: AWS::Lambda::EventInvokeConfig
HelloAddFunction:
Properties:
Code:
ZipFile: |
def handler(event, context):
x, y = int(event["x"]), int(event["y"])
return x+y
Handler: index.handler
MemorySize:
Ref: MemorySizeSmall
Role:
Fn::GetAtt:
- HelloAddRole
- Arn
Runtime:
Fn::Sub: python${RuntimeVersion}
Timeout:
Ref: TimeoutShort
Type: AWS::Lambda::Function
HelloAddRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action: logs:*
Effect: Allow
Resource: '*'
- Action: sqs:*
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName:
Fn::Sub: hello-add-role-policy-${AWS::StackName}
Type: AWS::IAM::Role
ErrorsFunction:
Properties:
Code:
ZipFile: |
def handler(event, context):
print (event)
Handler: index.handler
MemorySize:
Ref: MemorySizeSmall
Role:
Fn::GetAtt:
- ErrorsRole
- Arn
Runtime:
Fn::Sub: python${RuntimeVersion}
Timeout:
Ref: TimeoutShort
Type: AWS::Lambda::Function
ErrorsQueue:
Properties: {}
Type: AWS::SQS::Queue
ErrorsQueueBinding:
Properties:
EventSourceArn:
Fn::GetAtt:
- ErrorsQueue
- Arn
FunctionName:
Ref: ErrorsFunction
Type: AWS::Lambda::EventSourceMapping
ErrorsRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action: logs:*
Effect: Allow
Resource: '*'
- Action: sqs:*
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName:
Fn::Sub: errors-role-policy-${AWS::StackName}
Type: AWS::IAM::Role
Your template is fine, but lambda destinations are for asynchronous invocations of your function only. So you have to make such an invocation, which can be done using AWS CLI (--invocation-type Event). AWS Console is synchronous only.
aws lambda invoke --function-name <HelloAddFunctionnaem> --invocation-type Event --payload '{"x": 3, "y": "SSS"}' /dev/stdout
I'm trying to setup an S3 bucket which notifies a Lambda function when a new object is created.
The stack below works fine but I want to added the SourceArn to the Lambda permission, following best practices.
There's some literature on this which suggests the way to do it is via a string rather than Fn::GetAtt/Arn -
https://aws.amazon.com/premiumsupport/knowledge-center/unable-validate-circular-dependency-cloudformation/
But if I uncomment the relevant SourceArn line and redeploy, I get -
Unable to validate the following destination configurations (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: TBZDEVZ1HVD9EN0Q; S3 Extended Request ID: gL3CRz6UayvHup5i5oC4+/RMm0p1oRaRrPVtfZrykeAaJ1BVuhNSKkqxQ8TL5sy749d9PtbMOEQ=; Proxy: null)
Ho hum. Looking at the article again, I see that MyBucket needs to depend on MyBucketFunctionPermission - but if I uncomment that and redeploy I now get -
An error occurred (ValidationError) when calling the CreateChangeSet operation: Circular dependency between resources: [MyBucketFunctionPermission, MyBucket]
This is some fresh circle of hell. Am I missing something from the article or is there some other combination of SourceArn format + DependsOn that would get this to work ?
TIA.
AWSTemplateFormatVersion: '2010-09-09'
Outputs: {}
Parameters: {}
Resources:
MyBucket:
# DependsOn:
# - MyBucketFunctionPermission
Properties:
NotificationConfiguration:
LambdaConfigurations:
- Event: s3:ObjectCreated:*
Function:
Fn::GetAtt:
- MyBucketFunction
- Arn
Type: AWS::S3::Bucket
MyBucketFunction:
Properties:
Code:
ZipFile: "def handler(event, context):\n print (event)"
Handler: index.handler
Role:
Fn::GetAtt:
- MyBucketRole
- Arn
Runtime: "python3.8"
Type: AWS::Lambda::Function
MyBucketFunctionPermission:
Properties:
Action: lambda:InvokeFunction
FunctionName:
Ref: MyBucketFunction
Principal: s3.amazonaws.com
# SourceArn:
# Fn::Sub: arn:aws:s3:::${MyBucket}
Type: AWS::Lambda::Permission
MyBucketRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action: logs:*
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName:
Fn::Sub: my-bucket-role-policy-1234567890
Type: AWS::IAM::Role
There is nothing seemingly wrong with your template. It works fine, at least for me. But a race condition is possible between MyBucket and MyBucketFunctionPermission. Thus, protect against this, you have to use DependsOn. But for that to work you have to explicitly set your bucket name. For example:
AWSTemplateFormatVersion: '2010-09-09'
Outputs: {}
Parameters: {}
Resources:
MyBucket:
DependsOn:
- MyBucketFunctionPermission
Properties:
BucketName: !Sub "my-bucket-323323-${AWS::StackName}-${AWS::Region}"
NotificationConfiguration:
LambdaConfigurations:
- Event: s3:ObjectCreated:*
Function:
Fn::GetAtt:
- MyBucketFunction
- Arn
Type: AWS::S3::Bucket
MyBucketFunction:
Properties:
Code:
ZipFile: "def handler(event, context):\n print (event)"
Handler: index.handler
Role:
Fn::GetAtt:
- MyBucketRole
- Arn
Runtime: "python3.8"
Type: AWS::Lambda::Function
MyBucketFunctionPermission:
Properties:
Action: lambda:InvokeFunction
FunctionName:
Ref: MyBucketFunction
Principal: s3.amazonaws.com
SourceArn: !Sub "arn:aws:s3:::my-bucket-323323-${AWS::StackName}-${AWS::Region}"
Type: AWS::Lambda::Permission
MyBucketRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action: logs:*
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName:
Fn::Sub: my-bucket-role-policy-1234567890
Type: AWS::IAM::Role
Considering this lambda function on a serverless.yml file:
functions:
s3toEc2Lambda:
handler: s3toec2lambda.S3toEc2Lambda
name: "${self:service}-s3toEc2Lambda"
role: S3toEc2LambdaRole
And considering this SNS created on resources section: Does someone knows how to inform the Sns ARN Endpoint from the lambda function s3toEc2Lambda ?
resources:
Resources:
WordpressFrontEndSnsS3toEc2:
Type: AWS::SNS::Topic
Properties:
TopicName: "wordpress-front-end-s3-ec2"
WordpressFrontEndSnsS3toEc2Lambda:
Type: AWS::SNS::Subscription
Properties:
Endpoint: { "Fn::GetAtt": ["s3toEc2Lambda", "Arn" ] } <------ HERE <------
#Endpoint: ${self:functions.s3toEc2Lambda} <------ OR HERE <------
#Endpoint: { "Fn::GetAtt": ["${self:functions.s3toEc2Lambda}", "Arn" ] } <------ OR HERE <------
Protocol: lambda
TopicArn: !Ref 'WordpressFrontEndSnsS3toEc2'
For me always appear a error message like this: "Template error: instance of Fn::GetAtt references undefined resource s3toEc2Lambda"
Thank You !
CloudFormation resources created by serverless have known format. For lambda function this is:
{normalizedFunctionName}LambdaFunction
Thus you should be able to reference your function using the following:
"Fn::GetAtt": [ S3toEc2LambdaLambdaFunction, Arn ]
More example about this are here
We can create Function Roles, Functions Policy and Lambda functions SAM template.yml file by this
Type: AWS::IAM::Role
Properties:
RoleName: UatAdminUserStatsLambda
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Principal:
Service:
- 'lambda.amazonaws.com'
Action:
- 'sts:AssumeRole'
FunctionPolicy:
Type: AWS::IAM::Policy
DependsOn: FunctionRole
Properties:
PolicyName: UserStatsPolicy
Roles:
- !Ref FunctionRole
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: 'Allow'
Action:
- 'iam:GetUser'
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:GetLogEvents'
- 'logs:PutLogEvents'
- 's3:GetObject'
- 's3:PutObject'
Resource: '*'
adminUsersList:
Type: AWS::Serverless::Function
Properties:
CodeUri: src/admin/
Role: !GetAtt FunctionRole.Arn
Handler: adminUsersList.adminUsersList
Layers:
- !Ref NodeDependenciesLayer
Events:
adminUsersListEvent:
Type: Api
Properties:
Path: /api/admins
Method: GET
I am looking for a Cloudformation template to push cloudwatch logs to elasticsearch in another account. Even to the same account would be ok and I can update that.
Seems like a standard problem but haven't seen any template which automates the steps described in https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_ES_Stream.html.
I should have updated the answer. This is what I wrote which is the minimalistic version of streaming logs from CW to Elastic Search.
Resources:
LambdaElasticSearchExecutionRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: ['sts:AssumeRole']
Effect: Allow
Principal:
Service: [lambda.amazonaws.com]
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action: ['es:ESHttpPost']
Effect: Allow
Resource: "*"
- Action: ['logs:CreateLogGroup', 'logs:CreateLogStream', 'logs:PutLogEvents']
Effect: Allow
Resource: "arn:aws:logs:*:*:*"
- Action: ['lambda:InvokeFunction']
Effect: Allow
Resource: "arn:aws:logs:*:*:*"
Version: '2012-10-17'
PolicyName: lambdaRoleElasticSearchStreaming
Type: AWS::IAM::Role
ESStreamingLambda:
Type: AWS::Lambda::Function
DependsOn: LambdaElasticSearchExecutionRole
Properties:
Handler: index.handler
Role:
Fn::GetAtt: [LambdaElasticSearchExecutionRole, Arn]
Code:
S3Bucket: {'Fn::Sub': 'do-not-delete-cw-es-log-streaming-lamda-${AWS::Region}'}
S3Key: LogsToElasticsearch.zip
Runtime: nodejs4.3
LambdaPermissionForCWInvokation:
DependsOn: ESStreamingLambda
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName:
Fn::GetAtt: [ESStreamingLambda, Arn]
Principal: {'Fn::Sub': 'logs.${AWS::Region}.amazonaws.com'}
APILogsToElasticSearchSubscriptionFilter:
Type: AWS::Logs::SubscriptionFilter
DependsOn: [ESStreamingLambda, LambdaPermissionForCWInvokation]
Properties:
DestinationArn:
Fn::GetAtt: [ESStreamingLambda, Arn]
FilterPattern: ''
LogGroupName: {Ref: LambdaLogGroup}
```java
I am currently trying to do the same thing. I have found these examples for now:
https://docs.aws.amazon.com/solutions/latest/centralized-logging/templates.html (seems to be the best bet)
https://github.com/amazon-archives/cloudwatch-logs-subscription-consumer (seems to me this is an old approach)
https://github.com/awslabs/hids-cloudwatchlogs-elasticsearch-template (just an example)