I am creating my lambda function like following through cloud formation template.
My question is , after creating this lambda resource , I want to pass it few variables as an input and execute it immediately. Is there a way I can do it through cloud formation template?
AWSTemplateFormatVersion: '2010-09-09'
Description: Create a lambda function for chef rds read replica
Parameters:
Environment:
Description: Environment that will be built
Type: String
Default: q1
AllowedValues:
- q1
Mappings:
ChefEnvironmentMap:
q1:
IAMRole: CHEFAWS-RDS
Subnets:
- subnet-***
SecurityGroups:
- sg-***
NetCoreEnvironment: qa
Resources:
ChefRDSReadOnlyReplica:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket: cheftestbucket
S3Key: lambda/cheflambda.zip
Description: "Chef rds"
Environment:
Variables:
http_proxy: 'http://**'
https_proxy: 'http://**'
no_proxy: '169.254.169.254,127.0.0.1,localaddress,.localdomain.com'
Handler: createreadreplica.lambda_handler
MemorySize: 128
Role: arn:aws:iam::*****:role/CHEFAWS-RDS
Runtime: python2.7
Timeout: 60
VpcConfig:
SecurityGroupIds: !FindInMap [ChefEnvironmentMap, !Ref Environment, SecurityGroups]
SubnetIds: !FindInMap [ChefEnvironmentMap, !Ref Environment, Subnets]
I followed the approach to create a custom resource to execute the lambda fuction. I launched the custom resource through the cft as well, like this.
Lambdaresource:
DependsOn: ChefRDSReadOnlyReplica
Properties:
sourceregion: "us-east-1"
target_region: "us-west-2"
db_instance_id: "chef-tod-pg-rds"
dbsubnet_groupname: "******"
kms_keyid: "******"
ServiceToken: !GetAtt ChefRDSReadOnlyReplica.Arn
Type: AWS::CloudFormation::CustomResource
However, this lead to more complexity . As cloud formation of custom resource doesnt get notified after it triggers the lambda function. So it remains in “Creating Resource” State. To handle this , I had to create a new function in my lambda Python code that returned the success/failure. As mentioned here: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-code.html
Related
I am using aws-cli to deploy my stack across several environments and need to parametrize the subnets / security groups available to my stack.
I have a section in my SAM template defining the subnets and security groups as such:
EnvSubnets:
Description: Define subnet ids
Type: 'List<AWS::EC2::Subnet::Id>'
EnvSecGroups:
Description: Security Groups
Type: 'List<AWS::EC2::SecurityGroup::Id>'
I specify the arguments using `aws cloudformation deploy ... --parameter-overrides file://env.json' but cannot find a single format that passes the arrays to cloudformation.
I keep getting the followign errors:
#/VpcConfig/SecurityGroupIds: expected type: JSONArray, found: String #/VpcConfig/SubnetIds: expected type: JSONArray, found: String
Any hints?
It seems that at the current time this is not supported - I ended-up using a nested template driven by a user-overridable parameter:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: 'SAM Template for XXXXX XXXXX'
Parameters:
LambdaRole:
Description: Define exiting Lambda role to provide permissions
Type: String
LambdaImage:
Description: Define Lambda image URI
Type: String
LambdaVPCInclude:
Description: S3 URI of the YAML for the S3 VPC section
Type: String
Resources:
FOO:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
ImageUri: !Ref LambdaImage
Architectures:
- x86_64
MemorySize: 1024
Timeout: 900
Role: !Ref LambdaRole
'Fn::Transform':
Name: 'AWS::Include'
Parameters:
Location: !Ref LambdaVPCInclude
Metadata:
SamResourceId: FOO
Outputs:
QuantUniverse:
Description: FOO Lambda Function ARN
Value: !GetAtt FOO.Arn
and in an S3 bucket I have a file with my VPC config:
VpcConfig:
SubnetIds:
- subnet-*****************
- subnet-*****************
- subnet-*****************
SecurityGroupIds:
- sg-*****************
- sg-*****************
and pass the S3 URI of this file as the override for LambdaVPCInclude in aws cloudformation deploy
Hope this helps others.
Im trying to add auto scaling to the Provisioned Concurrency configuration for our Lambdas on AWS.
Im having trouble with the DependsOn: value for the Lambda target it doesnt seem recognize the resource im passing in and says its null. In the example from AWS its using AutoPublishAlias which isnt an option for me. https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/
Am I referencing incorrectly?
**LambdaAlias:**
Type: AWS::Lambda::Alias
Properties:
Description: Adds provisioned concurrency for lambda using alias on arn
FunctionName: !Ref Lambda
FunctionVersion: !GetAtt LambdaVersion.Version
Name: live
ProvisionedConcurrencyConfig:
ProvisionedConcurrentExecutions: !Ref ProvisionedConcurrentExecutions
LambdaTarget:
Type: AWS::ApplicationAutoScaling::ScalableTarget
Properties:
MaxCapacity: 100
MinCapacity: 1
ResourceId: !Sub function:${Lambda}:live # You need to specify an alis or version here
RoleARN: !Ref IamRoleArn
ScalableDimension: lambda:function:ProvisionedConcurrency
ServiceNamespace: lambda
**DependsOn: LambdaAlias** # This is your function logical ID + "Alias" + what you use for AutoPublishAlias
LambdaTrackingScalingPolicy:
Type: AWS::ApplicationAutoScaling::ScalingPolicy
Properties:
PolicyName: utilization
PolicyType: TargetTrackingScaling
ScalingTargetId: !Ref LambdaTarget
TargetTrackingScalingPolicyConfiguration:
TargetValue: 0.70 # Any value between 0.1 and 0.9 can be used here
PredefinedMetricSpecification:
PredefinedMetricType: LambdaProvisionedConcurrencyUtilization
Adding Resource Snippet
Resources:
Lambda:
Type: AWS::Lambda::Function
Properties:
!If
- EnableVpcConfig
- FunctionName: !Ref FunctionName
Description: !Ref FunctionDescription
Code:
S3Bucket: !Ref CodeSourceBucket
S3Key: !Sub 'api-packages/${CodeFile}'
Handler: !Ref Handler
Environment:
Variables:
APP_ID: !If [ UsesPublicKeyAPI, !Ref AppId, !Ref
Solution
Answer was to reference the logical id of the AWS::Lambda::Alias resource I created which in this case was
DependsOn: LambdaAlias
The doc was using AutoPublishAlias which didnt require the creation of a resource AWS::Lambda::Alias so config needed is slightly different
Have you tried just using the name of the resource for the LambdaAlias
I feel DependsOn: LambdaAlias should do the trick
You are using plain Cloudformation where the article you refer to is using AWS SAM.
since you are not using any AutoPublishAlias the format
{functionLogicalResourceName}Alias{aliasName}
should work for you..
In the sample you have provided. You have not provided the part which is deploying Lambda so if we assume your lambda logical id is MyLambda you should
DependsOn: MyLambdaAliaslive
I'm stuck in a weird issue. I have created an AWS S3 bucket using following cloud formation template:-
AWSTemplateFormatVersion: '2010-09-09'
Metadata:
License: Unlicensed
Description: >
This template creates a global unique S3 bucket in a specific region which is unique.
The bucket name is formed by the environment, account id and region
Parameters:
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html
Environment:
Description: This paramenter will accept the environment details from the user
Type: String
Default: sbx
AllowedValues:
- sbx
- dev
- qa
- e2e
- prod
ConstraintDescription: Invalid environment. Please select one of the given environments only
Resources:
# https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket.html
MyS3Bucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
Properties:
BucketName: !Sub 'global-bucket-${Environment}-${AWS::Region}-${AWS::AccountId}' # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html
AccessControl: Private
LoggingConfiguration:
DestinationBucketName: !Ref 'LoggingBucket'
LogFilePrefix: 'access-logs'
Tags:
- Key: name
Value: globalbucket
- Key: department
Value: engineering
LoggingBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
Properties:
BucketName: !Sub 'global-loggings-${Environment}-${AWS::Region}-${AWS::AccountId}'
AccessControl: LogDeliveryWrite
Outputs:
GlobalS3Bucket:
Description: A private S3 bucket with deletion policy as retain and logging configuration
Value: !Ref MyS3Bucket
Export:
Name: global-bucket
If you note in the template above then I'm exporting this S3 bucket in the Outputs section by the name called global-bucket.
Now, my intention is to refer to this existing bucket going forward in my AWS account whenever any new resource like Lambda, etc wants an S3 bucket. Here is an example using AWS SAM (Serverless Application Model), I'm trying to create an AWS Lambda and trying to refer to this existing S3 bucket using property !ImportValue and the export name as global-bucket as shown below:-
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
hellolambda
Sample SAM Template for hellolambda
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Runtime: nodejs12.x
Events:
HelloLambdaEvent:
Type: S3
Properties:
Bucket: !Ref SrcBucket
Events: s3:ObjectCreated:*
SrcBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !ImportValue global-bucket
Now, the problem is when I execute the command like sam build and then sam deploy --guided and select the same region (where my previous CloudFormation stack output is present) then I get the following error:-
global-bucket-sbx-ap-southeast-1-088853283839 already exists in stack arn:aws:cloudformation:ap-southeast-1:088853283839:stack/my-s3-global-bucket/aabd20e0-f57d-11ea-80bf-06f1487f6a64
The screenshot below:-
The problem is AWS CloudFormation is trying to create the S3 bucket rather than referring to the existing one.
But, if I try to update this SAM template like and then execute sam deploy, I get the following error:-
Waiting for changeset to be created..
Error: Failed to create changeset for the stack: my-lambda-stack, ex: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state Status: FAILED. Reason: Transform AWS::Serverless-2016-10-31 failed with: Invalid Serverless Application Specification document. Number of errors found: 1. Resource with id [HelloWorldFunction] is invalid. Event with id [HelloLambdaEvent] is invalid. S3 events must reference an S3 bucket in the same template.
I'm blocked by both ends. I would really appreciate it if someone can assist to guide me writing the SAM template correctly in my Lambda so that I can refer the existing bucket properly instead of creating the new one.
Thank you
Any items listed under the Resources section refer to the resources the stack is responsible for maintaining.
When you list SrcBucket you are asking for CloudFormation to create a new S3 bucket with the name being the value of !ImportValue global-bucket which is the name of an S3 bucket you have already created.
Assuming that this is the bucket name you can simply reference it in your template as shown below.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
hellolambda
Sample SAM Template for hellolambda
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Runtime: nodejs12.x
Events:
HelloLambdaEvent:
Type: S3
Properties:
Bucket: !ImportValue global-bucket
Events: s3:ObjectCreated:*
I am creating a cloudformation template, with few resources, couple of lambda functions , S3 bucket.see code below, it is work on progress and so far I have a S3 bucket and a lamda function triggered by S3. we have vpc defined in our team that we are supposed to use. I would like to add private subnet under that vpc for my lambda function and assign public subnet for the s3 bucket. how to get reference of the vpc , and pass it to my template and use it? sample code will be helpful.
AWSTemplateFormatVersion: 2010-09-09
Resources:
# S3 Bucket
S3Bucket:
Type: AWS::S3::Bucket
# Functions
S3-Lambda-trigger:
Type: AWS::Serverless::Function
Properties:
CodeUri: .
Handler: lambda.handler
Description: s3 object creation triggers lambda
Runtime: nodejs12.x
Events:
S3Bucket:
Type: S3
Properties:
Bucket: !Ref S3Bucket
Events: 's3:ObjectCreated:*'
# Permissions
Allow-lamda-invocation-s3:
Type: AWS::Lambda::Permission
Properties:
Action: 'lambda:InvokeFunction'
FunctionName: !Ref S3-Lambda-trigger
Principal: s3.amazonaws.com
SourceArn: !GetAtt S3Bucket.Arn
how to get reference of the vpc , and pass it to my template and use it?
One way would be through AWS-Specific Parameter Types, specifically AWS::EC2::VPC::Id, in a Parameters section.
For example:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
VPCId:
Type: AWS::EC2::VPC::Id
Resources:
MySubnet:
Type: AWS::EC2::Subnet
Properties:
# other properties
VpcId: !Ref VPCId
Thanks to this, when creating the stack in AWS Console, you would be able to choose existing VPCId to pass to the template.
I create CloudFormation template for my AWS Lambda function and I need to specify different values of environment variables for different lambda aliases.
My template looks like:
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Lambda function configuration
Resources:
EndpointLambda:
Type: "AWS::Lambda::Function"
Properties:
FunctionName: "endpoint-lambda"
Handler: "com.test.aws.RequestHandler::handleRequest"
Runtime: java8
Code:
S3Bucket: "lambda-functions"
S3Key: "test-endpoint-lambda-0.0.1.jar"
Description: Test Lambda function
MemorySize: 256
Timeout: 60
Environment:
Variables:
ES_HOST: test-es-host-url
ES_ON: true
ES_PORT: 443
ES_PROTOCOL: https
REDIS_URL: test-redis-host-url
QaLambdaAlias:
Type: "AWS::Lambda::Alias"
Properties:
FunctionName: !Ref EndpointLambda
FunctionVersion: 1
Name: "QA"
Description: "QA alias"
ProdLambdaAlias:
Type: "AWS::Lambda::Alias"
Properties:
FunctionName: !Ref EndpointLambda
FunctionVersion: 1
Name: "Prod"
Description: "Production alias"
As you see, I have two aliases - QA and Prod and bunch of environment variables. I specified variables with common values in lambda function declaration. But I need to declare for QA alias env. variable's values related to QA, and for Prod alias - values for Prod environment. Any ideas how can I do that?
You can use CloudFormation Parameters to do this. As a quick example:
Parameters:
LambdaRuntime:
Type: String
Default: 'java8'
Description: What Lambda runtime do we use?
Resources:
QaLambdaAlias:
Type: "AWS::Lambda::Alias"
Properties:
FunctionName:
Ref: EndpointLambda
FunctionVersion: 1
Name: "QA"
Description: "QA alias"
Runtime:
Ref: LambdaRuntime
Then, if you want to use a different parameter, when you deploy via CLI, you can override with parameter-overrides like this:
aws cloudformation deploy --stack-name MyStack --template-file \
CloudFormation/MyStack.yaml --capabilities CAPABILITY_IAM \
--parameter-overrides LambdaRuntime=nodejs8.10