In a Cloudformation template, I define two S3 Buckets.
Bucket1:
Type: AWS::S3::Bucket
Properties:
...
Bucket2:
Type: AWS::S3::Bucket
Properties:
...
Outputs:
Bucket1:
Description: S3 Bucket
Value: !Ref Bucket1
Export:
Name: !Sub "${AWS::StackName}:Bucket1"
Bucket2:
Description: S3 Bucket
Value: !Ref Bucket2
Export:
Name: !Sub "${AWS::StackName}:Bucket2"
I use these exported buckets in two different cloudformation templates.
Template 1
Parameters:
LoaderCodeBucket:
Type: String
Resources:
MyLambdaFunction:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket:
Fn::ImportValue:
!Sub "${LoaderCodeBucket}:Bucket1"
Template 2
Parameters:
ProcessorCodeBucket:
Type: String
Resources:
MyOtherLambdaFunction:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket:
Fn::ImportValue:
!Sub "${ProcessorCodeBucket}:Bucket2"
Template 1 passes aws cloudformation validate-template --template-body ... while Template 2 fails due to
Template error: the attribute in Fn::ImportValue must not depend on any resources, imported values, or Fn::GetAZs.
The only difference is the lambda function in template 2 is used in an aws analytics application that is also defined in template 2.
I know for sure it's the S3 Bucket causing issues because when I remove that section of code, it passes the validation check.
I have been using this site to try to debug this issue, but none of the questions seem to answer this particular issue.
This is in same region/same account.
My question is:
Why is this particular section of code (template 2) throwing a template error when template 1 passes with no error?
This is a working example.
Template 1:
AWSTemplateFormatVersion: "2010-09-09"
Description: "Test"
Resources:
MyBucketOne:
Type: "AWS::S3::Bucket"
Properties:
BucketName: bucket-one-12341234
MyBucketTwo:
Type: "AWS::S3::Bucket"
Properties:
BucketName: bucket-two-12341234
Outputs:
MyBucketOneOutput:
Description: "Bucket Name of BucketOne"
Value: !Ref MyBucketOne
Export:
Name: !Sub "${AWS::StackName}-BucketOne"
MyBucketTwoOutput:
Description: "Bucket Name of BucketTwo"
Value: !Ref MyBucketTwo
Export:
Name: !Sub "${AWS::StackName}-BucketTwo"
Template 2: we can import it as !ImportValue my-s3-BucketOne
AWSTemplateFormatVersion: "2010-09-09"
Description: "Test"
Resources:
MyLambda:
Type: "AWS::Lambda::Function"
Properties:
Handler: index.handler
Runtime: nodejs12.x
FunctionName: "test-s3-import"
Code:
S3Bucket: !ImportValue my-s3-BucketOne
S3Key: "index.zip"
Description: "Test Lambda"
MemorySize: 128
Timeout: 60
Role: test-role-arn
If you do want to use from Parameter, it will be Fn::ImportValue: !Sub ${BucketExportNamePrefix}-BucketOne
AWSTemplateFormatVersion: "2010-09-09"
Description: "Test"
Parameters:
BucketExportNamePrefix:
Type: String
Default: "my-s3"
Resources:
MyLambda:
Type: "AWS::Lambda::Function"
Properties:
Handler: index.handler
Runtime: nodejs12.x
FunctionName: "test-s3-import"
Code:
S3Bucket:
Fn::ImportValue: !Sub ${BucketExportNamePrefix}-BucketOne
S3Key: "index.zip"
Description: "Test Lambda"
MemorySize: 128
Timeout: 60
Role: test-role-arn
Related
I am developping lambda with CloudFormation by SAM
My template.yaml is here.
It can be deployed, however this lambda is not set in VPC.
I want to put the lambda in default VPC (to access RDS)
Any setting can be used here or I should do something another??
(And, template makes IAmRole automatically, is there any way I can attach policy to it?? for example RDSFullAccess)
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
python3.9 Sample SAM Template for chatbot-sam
Parameters:
DBNAME:
Type: String
DBUSER:
Type: String
DBPASSWORD:
Type: String
DBHOST:
Type: String
DBPORT:
Type: String
LINELONGLIVETOKEN:
Type: String
Globals:
Function:
Timeout: 30
Environment:
Variables:
DBNAME: !Ref DBNAME
DBUSER: !Ref DBUSER
DBPASSWORD: !Ref DBPASSWORD
DBHOST: !Ref DBHOST
DBPORT: !Ref DBPORT
LINELONGLIVETOKEN: !Ref LINELONGLIVETOKEN
Resources:
WebhookFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
Architectures:
- x86_64
Events:
Webhook:
Type: Api
Properties:
Path: /webhook
Method: post
Metadata:
Dockerfile: Dockerfile.webhook
DockerContext: ./chatbotapp
DockerTag: python3.9-v1
Outputs:
WebhookApi:
Description: "API Gateway endpoint URL for Prod stage for Hello World function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/webhook/"
WebhookFunction:
Description: "Webhook Lambda Function ARN"
Value: !GetAtt WebhookFunction.Arn
WebhookFunctionIamRole:
Description: "Implicit IAM Role created for Webhook function"
Value: !GetAtt WebhookFunctionRole.Arn
I updated.
Attaches VpcConfig and Policies , however it doesn't look change.
lambda -> setting -> vpc, there is no vpc setting and can't find the clue it belongs to SecurityGroup and Subnet
Policies: AWSLambdaVPCAccessExecutionRole
VpcConfig:
SubnetIds:
- subnet-fb6fa4d0
- subnet-bf8ab8e4
SecurityGroupIds:
- sg-0641506b4ec3782de
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
python3.9 Sample SAM Template for chatbot-sam
Parameters:
DBNAME:
Type: String
DBUSER:
Type: String
DBPASSWORD:
Type: String
DBHOST:
Type: String
DBPORT:
Type: String
LINELONGLIVETOKEN:
Type: String
Globals:
Function:
Timeout: 30
Environment:
Variables:
DBNAME: !Ref DBNAME
DBUSER: !Ref DBUSER
DBPASSWORD: !Ref DBPASSWORD
DBHOST: !Ref DBHOST
DBPORT: !Ref DBPORT
LINELONGLIVETOKEN: !Ref LINELONGLIVETOKEN
Resources:
WebhookFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
Architectures:
- x86_64
Events:
Webhook:
Type: Api
Properties:
Path: /webhook
Method: post
Policies: AWSLambdaVPCAccessExecutionRole
VpcConfig:
SubnetIds:
- subnet-fb6fa4d0
- subnet-bf8ab8e4
SecurityGroupIds:
- sg-0641506b4ec3782de
Metadata:
Dockerfile: Dockerfile.webhook
DockerContext: ./chatbotapp
DockerTag: python3.9-v1
Outputs:
WebhookApi:
Description: "API Gateway endpoint URL for Prod stage for Hello World function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/webhook/"
WebhookFunction:
Description: "Webhook Lambda Function ARN"
Value: !GetAtt WebhookFunction.Arn
WebhookFunctionIamRole:
Description: "Implicit IAM Role created for Webhook function"
Value: !GetAtt WebhookFunctionRole.Arn
You'll need to add a VpcConfig to the properties of your function definition. You can see an example of how to use that here.
You can also add policies to the default role that is made for the function, or you can supply your own role, in which case the default role will not be created.
I have added one layer to my Lambda function through CloudFormation. Now I have a requirement to add one more layer to my function. Basically, I need two layers in my existing Lambda function. Is it possible? I tried searching the AWS docs but I don't see it.
Resources:
LambdaLayer:
Type: "AWS::Lambda::LayerVersion"
Properties:
CompatibleRuntimes:
- python3.8
Content:
S3Bucket: !Sub "hello-${AWS::Region}"
S3Key: !Sub "myapp/layer1.zip"
LayerName: "layer1"
LambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Code:
S3Bucket: hello
S3Key: myapp/function.zip"
FunctionName: "hello-function"
Handler: "hello-function.lambda_handler"
Layers:
- !Ref LambdaLayer
Yes, is it possible. Add additional layers in the same way that you did the first layer, only append numbers to the resource names to distinguish them:
Resources:
LambdaLayer1:
Type: "AWS::Lambda::LayerVersion"
Properties:
CompatibleRuntimes:
- python3.8
Content:
S3Bucket: !Sub "hello-${AWS::Region}"
S3Key: !Sub "myapp/layer1.zip"
LayerName: "layer1"
LambdaLayer2:
Type: "AWS::Lambda::LayerVersion"
...
LayerName: "layer2"
LambdaFunction:
Type: "AWS::Lambda::Function"
Properties:
Code:
S3Bucket: hello
S3Key: myapp/function.zip"
FunctionName: "hello-function"
Handler: "hello-function.lambda_handler"
Layers:
- !Ref LambdaLayer1
- !Ref LambdaLayer2
I am trying to achieve something similar to below in a AWS Cloudformation YAML file:
AWSTemplateFormatVersion: 2010-09-09
testAttribute = "test"
Resources:
Lambda:
Type: AWS::Lambda::Function
Properties:
Runtime: python3.7
Role: !GetAtt iam.Arn
MemorySize: 128
Timeout: 10
Handler: lambda_function.lambda_handler
FunctionName: "testName"+${testAttribute}
Description: 'This is my lambda'
Code:
S3Bucket: myBucket
S3Key: "lambda/testName"+${testAttribute}+".zip"
I know that above isn't quite correct, but I cant find a good answer when searching how to achieve it. Anyone who have some guidance on this matter?
It depends on the use case but if the "variable" would be static and you don't need the change it when deploying the stack, I would suggest an alternative solution, to use the Mappings section.
This allows you to define some static values without sending them when deploying the stack (you will have much cleaner deploy commands, and the logic would be on the template side instead of the deploy side).
In this case, I'm using !Sub intrinsic function with a mapping (you can set multiple variables to be substituted using !Sub):
AWSTemplateFormatVersion: 2010-09-09
Mappings:
attributes:
lambda:
testAttribute: "test"
Resources:
Lambda:
Type: AWS::Lambda::Function
Properties:
Runtime: python3.7
Role: !GetAtt iam.Arn
MemorySize: 128
Timeout: 10
Handler: lambda_function.lambda_handler
FunctionName: !Sub
- "testName${attr}"
- {attr: !FindInMap [attributes, lambda, testAttribute]}
Description: 'This is my lambda'
Code:
S3Bucket: myBucket
S3Key: !Sub
- "lambda/testName${attr}.zip"
- {attr: !FindInMap [attributes, lambda, testAttribute]}
Note: Mappings have a mandatory three-level nesting, take this into consideration while designing your solution
You could use Parameters with a default value, and Sub later in the template:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
testAttribute:
Type: String
Default: test
Resources:
Lambda:
Type: AWS::Lambda::Function
Properties:
Runtime: python3.7
Role: !GetAtt iam.Arn
MemorySize: 128
Timeout: 10
Handler: lambda_function.lambda_handler
FunctionName: !Sub "testName${testAttribute}"
Description: 'This is my lambda'
Code:
S3Bucket: myBucket
S3Key: !Sub "lambda/testName${testAttribute}.zip"
[Edited for typo]
I'm trying to deploy a parent and nested stacks to AWS with cloudformation. The parent stack looks like this
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
VPC:
Description: Choose which VPC the Lambda-functions should be deployed to
Type: AWS::EC2::VPC::Id
Default: vpc-sdjkfnsdjklfn
Subnets:
Description: Choose which subnets Lambda-functions should be deployed to
Type: CommaDelimitedList
Default: "subnet-sdoifno, subnet-sdofjnsdo"
SecurityGroup:
Description: Select the Security Group to use for the Lambda-functions
Type: AWS::EC2::SecurityGroup::Id
Default: sg-sdklfnsdkl
Role:
Description: Role for Lambda functions
Type: String
Default: arn:aws:iam::dlfksd:role/ssdfnsdo
Resources:
RestApi:
Type: AWS::ApiGateway::RestApi
Properties:
Name: "my-api"
Description: "SPP Lambda API"
Stack1:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: 'https://s3.amazonaws.com/bucket/template1.yml'
Parameters:
VPC: !Ref VPC
Subnets: !Join
- ','
- !Ref Subnets
SecurityGroup: !Ref SecurityGroup
Role: !Ref Role
RestApi: !Ref RestApi
ApiResourceParent: !GetAtt "RestApi.RootResourceId"
The child stack looks like this
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
VPC:
Type: AWS::EC2::VPC::Id
Subnets:
Type: CommaDelimitedList
SecurityGroup:
Type: AWS::EC2::SecurityGroup::Id
Role:
Type: String
RestApi:
Type: AWS::ApiGateway::RestApi
ApiResourceParent:
Type: AWS::ApiGateway::Resource
Resources:
Fucntion:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: bucket
S3Key: node_lambdas.zip
Handler: Function.handler
Role: !Ref Role
Runtime: nodejs6.10
Timeout: 300
VpcConfig:
SecurityGroupIds:
- !Ref SecurityGroup
SubnetIds: !Ref Subnets
#Policies: AWSLambdaDynamoDBExecutionRole
Permission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName: !GetAtt "Function.Arn"
Principal: "apigateway.amazonaws.com"
SourceArn: !Sub "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${RestApi}/*/*/*"
Resource:
Type: AWS::ApiGateway::Resource
Properties:
RestApiId: !Ref RestApi
ParentId: !Ref ApiResourceParent
PathPart: addadjustments
When I run aws cloudformation deploy --template-file parent-stack.yml --stack-name spp-lambda --region us-east-1 --capabilities CAPABILITY_IAM I get the following error
Embedded stack
arn:aws:cloudformation:us-east-1:771653148224:stack/spp-lambda-Stack1-97M9BLBUM3A5/4a454a50-c274-11e8-b49c-500c28903236
was not successfully created: Parameter validation failed: parameter
type AWS::ApiGateway::RestApi for parameter name RestApi does not
exist, parameter type AWS::ApiGateway::Resource for parameter name
ApiResourceParent does not exist
It doesn't complain about the parameters that are explicitly defined in the parent template. I want the parameters it is complaining about to be created and passed dynamically as I won't know the values before hand. What am I doing wrong?
Although some of the AWS resource type are supported as a cloudformation parameter type, it doesn't mean all resource type are supported.
You are trying to reference API gateway value as an AWS-specific parameter type, but it is not supported: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html#aws-specific-parameter-types
I believe using String as the type is sufficient.
I am trying to deploy my lambda functions using CloudFormation StackSets to multiple AWS accounts and regions. But failed because of the below error
ResourceLogicalId:OfficeHoursAutoScalingStart, ResourceType:AWS::Lambda::Function, ResourceStatusReason:Error occurred while GetObject. S3 Error Code: AuthorizationHeaderMalformed. S3 Error Message: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-southeast-1'
It seems like its a permissions thing? How do I resolve this?
My template:
AWSTemplateFormatVersion : '2010-09-09'
Description: 'Skynet. AWS Management Assistant'
Parameters:
AppName:
Type: String
Description: Prefix for resources
Default: skynet-lambda-stackset
ArtifactsBucket:
Type: String
Description: S3 bucket storing lambda function zip
ArtifactZipPath:
Type: String
Description: Path to lambda function zip
CostCenter:
Type: String
Description: Cost center
Default: Admin
Owner:
Type: String
Description: Owner
Default: Jiew Meng
Resources:
LambdaRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub '${AppName}-lambda'
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
- apigateway.amazonaws.com
Action:
- sts:AssumeRole
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/AmazonEC2FullAccess'
- 'arn:aws:iam::aws:policy/AWSLambdaFullAccess'
- 'arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess'
- 'arn:aws:iam::aws:policy/AmazonAPIGatewayInvokeFullAccess'
- 'arn:aws:iam::aws:policy/CloudWatchLogsFullAccess'
NewEc2AutoTag:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: !Ref ArtifactsBucket
S3Key: !Ref ArtifactZipPath
Handler: ec2/newEc2_autoTag.handler
Runtime: nodejs6.10
FunctionName: 'NewEC2_AutoTag'
Description: 'Auto tag new EC2 instances with Owner tag'
Timeout: 30
Role: !GetAtt LambdaRole.Arn
Tags:
- Key: Cost Center
Value: !Ref CostCenter
- Key: Owner
Value: !Ref Owner
NewEc2Event:
Type: AWS::Events::Rule
Properties:
Name: !Sub ${AppName}-newEc2
Description: On new EC2 instance created
EventPattern:
source:
- 'aws.ec2'
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventName:
- RunInstances
Targets:
- !Ref NewEc2AutoTag
AfterhoursEc2Shutdown:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: !Ref ArtifactsBucket
S3Key: !Ref ArtifactZipPath
Handler: ec2/afterHours_shutdown.handler
Runtime: nodejs6.10
FunctionName: 'Afterhours_Shutdown'
Description: 'Shutdown instances tagged Auto Shutdown: true'
Timeout: 30
Role: !GetAtt LambdaRole.Arn
Tags:
- Key: Cost Center
Value: !Ref CostCenter
- Key: Owner
Value: !Ref Owner
AfterHoursEvent:
Type: AWS::Events::Rule
Properties:
Name: !Sub ${AppName}-afterHours
Description: Triggered on weekdays 2400 SGT
ScheduleExpression: cron(0 16 ? * MON,TUE,WED,THUR,FRI *)
Targets:
- !Ref AfterhoursEc2Shutdown
- !Ref AfterhoursAutoScalingShutdown
OfficeHoursEc2Start:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: !Ref ArtifactsBucket
S3Key: !Ref ArtifactZipPath
Handler: ec2/officeHours_start.handler
Runtime: nodejs6.10
FunctionName: 'OfficeHours_Start'
Description: 'Starts instances with Auto Shutdown: true'
Timeout: 30
Role: !GetAtt LambdaRole.Arn
Tags:
- Key: Cost Center
Value: !Ref CostCenter
- Key: Owner
Value: !Ref Owner
OfficeHoursEvent:
Type: AWS::Events::Rule
Properties:
Name: !Sub ${AppName}-officeHours
Description: Triggered on 7AM SGT weekdays
ScheduleExpression: cron(0 23 ? * SUN,MON,TUE,WED,THU *)
Targets:
- !Ref OfficeHoursEc2Start
- !Ref OfficeHoursAutoScalingStart
StartedEc2ConfigureDns:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: !Ref ArtifactsBucket
S3Key: !Ref ArtifactZipPath
Handler: ec2/started_configureDns.handler
Runtime: nodejs6.10
FunctionName: 'StartedEc2_ConfigureDns'
Description: 'When EC2 started, configure DNS if required'
Timeout: 30
Role: !GetAtt LambdaRole.Arn
Tags:
- Key: Cost Center
Value: !Ref CostCenter
- Key: Owner
Value: !Ref Owner
Ec2StartedEvent:
Type: AWS::Events::Rule
Properties:
Name: !Sub ${AppName}-ec2-started
Description: Triggered on EC2 starts
EventPattern:
source:
- 'aws.ec2'
detail-type:
- 'EC2 Instance State-change Notification'
detail:
state:
- running
Targets:
- !Ref StartedEc2ConfigureDns
AfterhoursAutoScalingShutdown:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: !Ref ArtifactsBucket
S3Key: !Ref ArtifactZipPath
Handler: autoscaling/afterHours_shutdown.handler
Runtime: nodejs6.10
FunctionName: 'Afterhours_AutoScalingShutdown'
Description: 'Scales down autoscaling groups tagged Auto Shutdown: true'
Timeout: 30
Role: !GetAtt LambdaRole.Arn
Tags:
- Key: Cost Center
Value: !Ref CostCenter
- Key: Owner
Value: !Ref Owner
OfficeHoursAutoScalingStart:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: !Ref ArtifactsBucket
S3Key: !Ref ArtifactZipPath
Handler: autoscaling/officeHours_start.handler
Runtime: nodejs6.10
FunctionName: 'OfficeHours_AutoScalingStart'
Description: 'Scales up auto scaling groups that are scaled down to 0 and tagged autostart: true'
Timeout: 30
Role: !GetAtt LambdaRole.Arn
Tags:
- Key: Cost Center
Value: !Ref CostCenter
- Key: Owner
Value: !Ref Owner
NewAutoScalingGroupEvent:
Type: AWS::Events::Rule
Properties:
Name: !Sub ${AppName}-autoscaling-new
Description: Triggered when new autoscaling group created
EventPattern:
source:
- 'aws.autoscaling'
detail-type:
- 'AWS API Call via CloudTrail'
detail:
eventName:
- CreateAutoScalingGroup
Targets:
- !Ref NewAutoScalingGroupAutoTag
NewAutoScalingGroupAutoTag:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: !Ref ArtifactsBucket
S3Key: !Ref ArtifactZipPath
Handler: autoscaling/new_autoTag.handler
Runtime: nodejs6.10
FunctionName: 'NewAutoScalingGroup_AutoTag'
Description: 'Tags new autoscaling groups with owner and autoshutdown tags if not existing'
Timeout: 30
Role: !GetAtt LambdaRole.Arn
Tags:
- Key: Cost Center
Value: !Ref CostCenter
- Key: Owner
Value: !Ref Owner
Looks like you have created the s3 bucket (referenced by variable ArtifactsBucket in your template) in AWS region ap-southeast-1.
Using AWS Stacksets, You have selected us-east-1 as one of the regions in Deployment Order.
The AWS Stackset passes the SAME parameters to all the stacks which it tries to create in multiple regions/accounts.
So when it is trying to create the lambda function OfficeHoursAutoScalingStart in us-east-1 region, It is tryin to access the s3 bucket(GETObject request) in us-east-1 region itself, with the same bucket name.
ie. It is presuming that the s3 bucket with name passed by ArtifactsBucketparameter, is present in us-east-1 itself.But since the source code of the lambda function is actually in the bucket present in region ap-southeast-1,the header malformed error is thrown. In this case the bucket name is matching, but the region is not.
Currently, when you create lambda function using CloudFormation, there is a restriction that the S3 bucket that contains the source code of your Lambda function must be in the SAME region as the STACK which you are creating. Doc Reference Link
If this is the issue, then as a fix, you can think of creating s3 buckets (add region-name as a prefix to the bucket name) in the required regions and use them in the template based on the region.
Example:
us-east-1-lambdabkt
us-east-2-lambdabkt
ap-southeast-1-lambdabkt