Is there a way to do conditional template urls in cloudformation?
This fails because it is not evaluated until the aws cloudformation deploy step and it errors out saying that the templateURL must be an s3 link. When I hard code in one of the urls it will upload that relative file to s3 and in the packaged final template it will just have the s3 url in place.
This doesn't work (pre aws cloudformation package step)
Vpc:
Type: 'AWS::CloudFormation::Stack'
Properties:
Parameters:
AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
NatGateways: 'true'
TemplateURL: !If [IsProduction, './default-vpc-module.yml', './production-vpc-module.yml']
When a url is hardcoded in it will package into this (post aws cloudformation pacakge step)
VpcModule:
Fn::GetAtt:
- Vpc
- Outputs.StackName
AlertingModule:
Fn::GetAtt:
- Alerting
- Outputs.StackName
Priority: '2'
HealthCheckPath: /health-check.php
TemplateURL: https://s3.amazonaws.com/my-cicd-bucket-/fe556ff9386a28c063c4a110b31b.template
Yes you absolutely can achieve what your after, the way i did it was reference the s3 url, by using an aws cloudformaiton package but also a s3 cp in the codebuild stage to enforce naming conventions on the url. As long as your template url paths are distinguished differently.
To provide a very flexible example, you could use !Sub with an !If replacement statement on the dynamic name component, which will also allow your to use !Ref inside the !If statement:
Parameters:
StageProd:
Description: Environment
Default: "production"
Type: String
.......
......
TemplateURL: !Sub
- https://${CfnBucketName}.s3-ap-southeast-2.amazonaws.com/${CfnKeyPrefix}/SomeFileName-${Stage}.yaml
- { Stage: !If [ IsProduction, !Ref StageProd, "default"]}
The above should meet almost any sort of combination of dynamic naming you want to achieve; however, you could also very much simplify the above with a simple !Sub replacing the stage name.
Here is what I ended up doing, but I don't like it.
Vpc:
Condition: IsProduction
Type: 'AWS::CloudFormation::Stack'
Properties:
Parameters:
AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
NatGateways: 'true' # reduce costs
TemplateURL: ./production-vpc-module.yml
Vpc:
Condition: IsNotProduction
Type: 'AWS::CloudFormation::Stack'
Properties:
Parameters:
AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
NatGateways: 'true' # reduce costs
TemplateURL: ./default-vpc-module.yml
Related
I have the following parameter in my CloudFormation script:
CloudFormationURL:
Type: String
Description: S3 URL for nested CloudFormation templates
Default: ""
This parameter covers the CloudFormation scripts in a nested folder of my deployment config.
I use it with a resource like this:
CloudWatchDashboardStack:
Type: "AWS::CloudFormation::Stack"
Properties:
TemplateURL: !Sub "${CloudFormationURL}/cloudwatch-dashboard.cfn.yaml"
Parameters:
AppName: !Ref AppName
DeployPhase: !Ref DeployPhase
DeveloperPrefix: !Ref DeveloperPrefix
Environment: !Ref Environment
Which works fine, and has worked for months.
I needed to add another resource, so I added this:
BatchDNSResources:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
AppName: !Ref AppName
Environment: !Ref Environment
DeveloperPrefix: !Ref DeveloperPrefix
DeployPhase: !Ref DeployPhase
AppVersion: !Ref AppVersion
SharedBucketName: !Ref SharedBucketName
S3Version: !Ref S3Version
HostedZone: !Ref HostedZone
VPCStackName: !FindInMap
- EnvironmentMap
- !Ref Environment
- VpcStackName
Company: !Ref Company
CostCenter: !Ref CostCenter
Team: !Ref Team
TemplateURL: !Sub "${CloudFormationURL}/batch-dns.cfn.yaml"
CloudFormation throws this error and then fails:
Parameters: [CloudFormationURL] must have values
Checking the changeset for the stack I can see the following value for the CloudFormationURL:
s3://application-shared-dev/application-name/qa/cf/nested/KShyDj205UK8mz6W_XUA5TnEF8nqPWHS
Checking the application predeploy logs I can see:
upload: deploy/cloudformation/templates/nested/batch-dns.cfn.yaml to s3://application-shared-dev/application-name/qa/cf/nested/KShyDj205UK8mz6W_XUA5TnEF8nqPWHS/batch-dns.cfn.yaml
And I can see the file in the S3 bucket.
If I remove BatchDNSResource the stack completes successfully.
What the heck am I missing here?
Sometimes, the smallest things will get you.
I had copied the Parameters from the master CloudFormation script, including this one, into the nested script:
CloudFormationURL:
Type: String
Description: S3 URL for nested CloudFormation templates
Default: ""
If you look closely, you will see that I did not pass the parameter into the nested script when calling the resource:
BatchDNSResources:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
AppName: !Ref AppName
Environment: !Ref Environment
DeveloperPrefix: !Ref DeveloperPrefix
DeployPhase: !Ref DeployPhase
AppVersion: !Ref AppVersion
SharedBucketName: !Ref SharedBucketName
S3Version: !Ref S3Version
HostedZone: !Ref HostedZone
VPCStackName: !FindInMap
- EnvironmentMap
- !Ref Environment
- VpcStackName
Company: !Ref Company
CostCenter: !Ref CostCenter
Team: !Ref Team
TemplateURL: !Sub "${CloudFormationURL}/batch-dns.cfn.yaml"
Because the CloudFormation console was saying the issue was with the BatchDNSResources I kept looking at the master script for the problem and missing the reference in the other script. There are two ways to solve this problem:
Keep CloudFormationURL as a parameter in the nested script (if you need it for some reason) and pass the value from the master script.
Remove the parameter from the nested script (if it is not needed)
Sometimes just asking for an extra set of eyeballs and getting a little rest will help you to find the issues. I want to leave this question/answer in place here because when I was searching for the error here and elsewhere no one ever mentioned (probably out of embarrassment) that the mistake is simply overlooking something like this. I hope this answer prompts others to check everything when they run across this type of error.
I am trying to deploy this CloudFormation template across my organization that contains global resources as well as Region-specific. I have seen several methods pertaining to my issue but none seem to work as I was hoping. I pieced my script together with other StackOverflow answers that I can't find the links to right now, but most pointed to this handy link: https://garbe.io/blog/2017/07/17/cloudformation-hacks/
Here is how I have essentially pieced my script together (broken up for easier reading):
Parameters:
CommercialMaster:
Description: The Commercial Account ID for routing.
Type: String
Default: ############
GovCloudMaster:
Description: The GovCloud Account ID for routing.
Type: String
Default: ############
LambdaRoleName:
Type: String
Default: 'LambdaR53Role'
Conditions:
Commercial: !Equals [ !Ref AWS::Partition, 'aws' ]
RegionCheck: !Or [!Equals [!Ref AWS::Region, 'us-east-1'], !Equals [!Ref AWS::Region, 'us-gov-west-1']]
CreateLambdaRole: !Equals [ !Ref LambdaRoleName, 'false' ]
CreateLambdaRoleRegion: !And
- !Condition RegionCheck
- !Condition CreateLambdaRole
Resources:
LambdaPermissionsRole:
Type: "AWS::IAM::Role"
Condition: RegionCheck
Properties:
RoleName: LambdaR53Role
Description: Lambda permissions role for R53 association.
CreateRoleWaitHandle:
Condition: CreateLambdaRole
DependsOn: LambdaPermissionsRole
Type: AWS::CloudFormation::WaitConditionHandle
#added, since DependsOn: !If is not possible, trigger by WaitCondition if CreateLambdaRole is false
WaitHandle:
Type: AWS::CloudFormation::WaitConditionHandle
#added, since DependsOn: !If is not possible
WaitCondition:
Type: AWS::CloudFormation::WaitCondition
Properties:
Handle: !If [CreateLambdaRole, !Ref CreateRoleWaitHandle, !Ref WaitHandle]
Timeout: "1"
Count: 0
AssociateVPCs:
Type: 'AWS::Lambda::Function'
DependsOn: WaitCondition
Properties:
Obviously, because of the hardcoded AWS::Region this works fine when deploying out to us-east-1 for Commercial, but when deploying with a StackSet, it fails in any other region. If I remove the RegionCheck then the stack fails because it tries to create the Role in every region and realizes that it already exists (because IAM is global, obviously).
My question, the part I'm stuck on, is how I can create the global IAM role, and STILL deploy the lambda function out to each region I need it (4 regions)?
My temporary solution, which I don't like is to essentially hardcode the region on to the back of the role name, like so:
Resources:
LambdaPermissionsRole:
Type: "AWS::IAM::Role"
Condition: CreateLambdaRole
Properties:
RoleName: !Sub LambdaR53Role-${AWS::Region}
Description: Lambda permissions role for R53 association.
Any further guidance would be great, but this headache has given me the push to start examining Terraform as my way forward so things can be more consolidated.
I am trying to deploy lambda having a zip(contains jar file). Now if the static value of the artifact in CodeUri is provided, it works fine but the problem is that the artifact is not static in nature i.e the version of the jar file (along with its name ex: abc-<1.x.x>-prod.jar) will change whenever their is new build.
So, I want to pass the artifact name in CodeUri as dynamic value rather than static value.
I had tried splitting Bucket, Key & pass the value as parameter but it fails saying NoSuchKey while deployment.
Edit: Adding Sample Template
Transform: AWS::Serverless-2016-10-31
Description: engine-service
Parameters:
Environment:
Type: String
Default: ""
SecurityGroupIds:
Type: String
Default: ""
SubnetIds1:
Type: String
Default: ""
SubnetIds2:
Type: String
Default: ""
DBSubnetGroupName:
Type: String
Default: ""
RDSSecret:
Type: String
Default: ""
RDSInstance:
Type: String
Default: ""
API:
Type: String
Default: ""
Globals:
Function:
Timeout: 120
Resources:
TranslationEngineLambda:
Type: AWS::Serverless::Function
Properties:
FunctionName: !Sub "translation-engine-service-${Environment}"
CodeUri: target/abc-**1.0.0-SNAPSHOT**-prod.jar
Handler: com.abc.Main
Runtime: java11
MemorySize: 1024
Environment:
Variables:
BUCKET_NAME: "abc-dummy"
DB_SECRET: "abc-dummy"
FUNCTION_NAME: TranslateFunction
SPRING_PROFILES_ACTIVE: db
TEXT_EXTRACT_LAMBDA: !Sub arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:text-extract-service-${Environment}
TRANSLATE_OPTION: AWS
VpcConfig:
SecurityGroupIds:
- !Ref SecurityGroupIds
SubnetIds:
- !Ref SubnetIds1
- !Ref SubnetIds2
Policies:
- AWSLambda_FullAccess
- AmazonEC2FullAccess
- SecretsManagerReadWrite
- AmazonS3ReadOnlyAccess
- AmazonRDSFullAccess
TranslationEngineLambdaInvoke:
Type: "AWS::Lambda::Permission"
Properties:
Action: "lambda:InvokeFunction"
FunctionName: !GetAtt "TranslationEngineLambda.Arn"
Principal: "apigateway.amazonaws.com"
SourceArn: !Join ['', ['arn:aws:execute-api:MyRegion:MyAccountNumber:', Fn::ImportValue: !Ref API, '/*/POST/language-translator/v1/translate']]
Outputs:
TranslationEngineLambda:
Description: "Hello World Lambda Function ARN"
Value: !GetAtt TranslationEngineLambda.Arn
TranslationEngineLambdaIamRole:
Description: "Implicit IAM Role created for Hello World function"
Value: !GetAtt TranslationEngineLambdaRole.Arn`
Your question encompasses a few things.
First, if you're using CodeUri as you're doing in the template with a relative URL, AWS SAM will use that path to search from the directory in which the template resides to find the required files. If you're using Bucket/Key, AWS SAM will look in S3 to the specified Bucket and search for the Key. This is of course an entirely different way of working and assumes that you've already uploaded the artefact to that location yourself. You've presumably not done this, which results in the NoSuchKey error.
One of the more useful things about AWS SAM is that you can not only use it to deploy your code, but you can also use it to build your artefacts themselves. In that case, you have to point your CodeUri to the root of the folder in which your Lambda function code resides. AWS SAM will then - in the build step - create the necessary artefact (be it a jar of a zip). During the deployment, it will upload those artefacts to S3, update the CodeUris to reflect this and deploy the CloudFormation stack.
I don't think you can use CloudFormation parameters (with !Sub, !Join or similar) when using a relative CodeUri URL since the parameters are only interpreted in the cloud, and not during the AWS SAM build or package steps. So if you do not want to rely on AWS SAM to build your artefacts, you're probably better of also uploading them yourself.
I created a CloudFormation Stackset that deployed AWS Config Rules to two accounts. Now I want to create a stackset that deploys the Remediation. the bottom lines of code work when I have it all in one CFT. but I want to deploy te detection rules in one script first then the remediation rules second. How can I reference the S3BucketEncryptionEnabled Resource from a different scipt?
---------------------Detection --------------------------------------------------------
S3BucketEncryptionEnabled:
Type: AWS::Config::ConfigRule
Properties:
Description: Checks that your Amazon S3 bucket either has S3 default encryption enabled or that the S3 bucket policy explicitly denies put-object requests without server side encryption.
Source:
Owner: AWS
SourceIdentifier: S3_BUCKET_SERVER_SIDE_ENCRYPTION_ENABLED
Scope:
ComplianceResourceTypes:
- AWS::S3::Bucket
DependsOn: ConfigRecorder
----------------------Remediation Script-----------------------------------------------
BasicRemediationConfiguration:
Type: "AWS::Config::RemediationConfiguration"
Properties:
Automatic: True
MaximumAutomaticAttempts: 5
RetryAttemptSeconds: 60
ConfigRuleName: !Ref S3BucketEncryptionEnabled
Parameters:
AutomationAssumeRole:
StaticValue:
Values: [{"Fn::GetAtt" : ["S3Role","Arn"]}]
BucketName:
ResourceValue:
Value: RESOURCE_ID
SSEAlgorithm:
StaticValue:
Values: [AES256]
TargetId: "AWS-EnableS3BucketEncryption"
TargetType: "SSM_DOCUMENT"
TargetVersion: "1"
Normally, in your Detection template you would export the S3BucketEncryptionEnabled in your outputs.
For example:
Outputs:
S3BucketEncryptionEnabled:
Value: !Ref S3BucketEncryptionEnabled
Export:
Name: MyS3BucketEncryptionEnabled
Then in your Remediation template, you would use ImportValue to import the exported value.
For example:
BasicRemediationConfiguration:
Type: "AWS::Config::RemediationConfiguration"
Properties:
Automatic: True
MaximumAutomaticAttempts: 5
RetryAttemptSeconds: 60
ConfigRuleName: !ImportValue MyS3BucketEncryptionEnabled
# remaining properties
I'm having a strange behavior with cloudformation template. This my template, where I create a bucket and want to notification configuration depending on a condition :
AWSTemplateFormatVersion: '2010-09-09'
Description: "Setup Artifacts Bucket"
Parameters:
BucketName:
Description: Name of the pipeline setup arctifact bucket
Type: String
Default: "s3-pipeline-setup"
NotificationCondition:
Description: Conditionally add Notification configuration to the artifact bucket
Type: String
Default: false
Conditions:
AddNotificationConfiguration: !Equals [ !Ref NotificationCondition, true ]
Resources:
ArtifactBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref BucketName
Fn::If:
- AddNotificationConfiguration
-
NotificationConfiguration:
LambdaConfigurations:
-
Function: "arn:aws:lambda:eu-west-1:341292222222227:function:lambda-ops-trigger-pipeline-setup"
Event: "s3:ObjectCreated:*"
Filter:
S3Key:
Rules:
-
Name: prefix
Value: "appstackcodes/"
-
Name: suffix
Value: "txt"
- !Ref AWS::NoValue
When I try a deploy it fails with this error :
00:28:10
UTC+0200 CREATE_FAILED AWS::S3::Bucket ArtifactBucket Encountered
unsupported property Fn::If
I don't really understand the matter.. Can someone try and let me know the mistake there please?
Thanks
Unfortunately you can not do what you intended in cloudformation.
The Fn::If can basically just be used as a ternary expression. E.g.
key: Fn::If: [condition_name, value_if_true, value_if_false]
It can't be used as logic flow like you would in a programming language. There are ways around it. You actually already seemed to have discovered the AWS::NoValue, so it's just a matter of moving the NotificationConfiguration assignment to outside the if.
Resources:
ArtifactBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref BucketName
NotificationConfiguration:
Fn::If:
- AddNotificationConfiguration
- LambdaConfigurations:
-
Function: "arn:aws:lambda:eu-west-1:341294322147:function:lambda-itops-trigger-pipeline-setup"
Event: "s3:ObjectCreated:*"
Filter:
S3Key:
Rules:
-
Name: prefix
Value: "appstackcodes/"
-
Name: suffix
Value: "txt"
- !Ref AWS::NoValue
Effectively you are always assigning something to NotificationConfiguration, but sometimes it's the magic AWS::NoValue. This works in the majority of cases, although there are times when this just isn't sufficient and more creativity is required!