I have the following parameter in my CloudFormation script:
CloudFormationURL:
Type: String
Description: S3 URL for nested CloudFormation templates
Default: ""
This parameter covers the CloudFormation scripts in a nested folder of my deployment config.
I use it with a resource like this:
CloudWatchDashboardStack:
Type: "AWS::CloudFormation::Stack"
Properties:
TemplateURL: !Sub "${CloudFormationURL}/cloudwatch-dashboard.cfn.yaml"
Parameters:
AppName: !Ref AppName
DeployPhase: !Ref DeployPhase
DeveloperPrefix: !Ref DeveloperPrefix
Environment: !Ref Environment
Which works fine, and has worked for months.
I needed to add another resource, so I added this:
BatchDNSResources:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
AppName: !Ref AppName
Environment: !Ref Environment
DeveloperPrefix: !Ref DeveloperPrefix
DeployPhase: !Ref DeployPhase
AppVersion: !Ref AppVersion
SharedBucketName: !Ref SharedBucketName
S3Version: !Ref S3Version
HostedZone: !Ref HostedZone
VPCStackName: !FindInMap
- EnvironmentMap
- !Ref Environment
- VpcStackName
Company: !Ref Company
CostCenter: !Ref CostCenter
Team: !Ref Team
TemplateURL: !Sub "${CloudFormationURL}/batch-dns.cfn.yaml"
CloudFormation throws this error and then fails:
Parameters: [CloudFormationURL] must have values
Checking the changeset for the stack I can see the following value for the CloudFormationURL:
s3://application-shared-dev/application-name/qa/cf/nested/KShyDj205UK8mz6W_XUA5TnEF8nqPWHS
Checking the application predeploy logs I can see:
upload: deploy/cloudformation/templates/nested/batch-dns.cfn.yaml to s3://application-shared-dev/application-name/qa/cf/nested/KShyDj205UK8mz6W_XUA5TnEF8nqPWHS/batch-dns.cfn.yaml
And I can see the file in the S3 bucket.
If I remove BatchDNSResource the stack completes successfully.
What the heck am I missing here?
Sometimes, the smallest things will get you.
I had copied the Parameters from the master CloudFormation script, including this one, into the nested script:
CloudFormationURL:
Type: String
Description: S3 URL for nested CloudFormation templates
Default: ""
If you look closely, you will see that I did not pass the parameter into the nested script when calling the resource:
BatchDNSResources:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
AppName: !Ref AppName
Environment: !Ref Environment
DeveloperPrefix: !Ref DeveloperPrefix
DeployPhase: !Ref DeployPhase
AppVersion: !Ref AppVersion
SharedBucketName: !Ref SharedBucketName
S3Version: !Ref S3Version
HostedZone: !Ref HostedZone
VPCStackName: !FindInMap
- EnvironmentMap
- !Ref Environment
- VpcStackName
Company: !Ref Company
CostCenter: !Ref CostCenter
Team: !Ref Team
TemplateURL: !Sub "${CloudFormationURL}/batch-dns.cfn.yaml"
Because the CloudFormation console was saying the issue was with the BatchDNSResources I kept looking at the master script for the problem and missing the reference in the other script. There are two ways to solve this problem:
Keep CloudFormationURL as a parameter in the nested script (if you need it for some reason) and pass the value from the master script.
Remove the parameter from the nested script (if it is not needed)
Sometimes just asking for an extra set of eyeballs and getting a little rest will help you to find the issues. I want to leave this question/answer in place here because when I was searching for the error here and elsewhere no one ever mentioned (probably out of embarrassment) that the mistake is simply overlooking something like this. I hope this answer prompts others to check everything when they run across this type of error.
Related
I am trying to deploy this CloudFormation template across my organization that contains global resources as well as Region-specific. I have seen several methods pertaining to my issue but none seem to work as I was hoping. I pieced my script together with other StackOverflow answers that I can't find the links to right now, but most pointed to this handy link: https://garbe.io/blog/2017/07/17/cloudformation-hacks/
Here is how I have essentially pieced my script together (broken up for easier reading):
Parameters:
CommercialMaster:
Description: The Commercial Account ID for routing.
Type: String
Default: ############
GovCloudMaster:
Description: The GovCloud Account ID for routing.
Type: String
Default: ############
LambdaRoleName:
Type: String
Default: 'LambdaR53Role'
Conditions:
Commercial: !Equals [ !Ref AWS::Partition, 'aws' ]
RegionCheck: !Or [!Equals [!Ref AWS::Region, 'us-east-1'], !Equals [!Ref AWS::Region, 'us-gov-west-1']]
CreateLambdaRole: !Equals [ !Ref LambdaRoleName, 'false' ]
CreateLambdaRoleRegion: !And
- !Condition RegionCheck
- !Condition CreateLambdaRole
Resources:
LambdaPermissionsRole:
Type: "AWS::IAM::Role"
Condition: RegionCheck
Properties:
RoleName: LambdaR53Role
Description: Lambda permissions role for R53 association.
CreateRoleWaitHandle:
Condition: CreateLambdaRole
DependsOn: LambdaPermissionsRole
Type: AWS::CloudFormation::WaitConditionHandle
#added, since DependsOn: !If is not possible, trigger by WaitCondition if CreateLambdaRole is false
WaitHandle:
Type: AWS::CloudFormation::WaitConditionHandle
#added, since DependsOn: !If is not possible
WaitCondition:
Type: AWS::CloudFormation::WaitCondition
Properties:
Handle: !If [CreateLambdaRole, !Ref CreateRoleWaitHandle, !Ref WaitHandle]
Timeout: "1"
Count: 0
AssociateVPCs:
Type: 'AWS::Lambda::Function'
DependsOn: WaitCondition
Properties:
Obviously, because of the hardcoded AWS::Region this works fine when deploying out to us-east-1 for Commercial, but when deploying with a StackSet, it fails in any other region. If I remove the RegionCheck then the stack fails because it tries to create the Role in every region and realizes that it already exists (because IAM is global, obviously).
My question, the part I'm stuck on, is how I can create the global IAM role, and STILL deploy the lambda function out to each region I need it (4 regions)?
My temporary solution, which I don't like is to essentially hardcode the region on to the back of the role name, like so:
Resources:
LambdaPermissionsRole:
Type: "AWS::IAM::Role"
Condition: CreateLambdaRole
Properties:
RoleName: !Sub LambdaR53Role-${AWS::Region}
Description: Lambda permissions role for R53 association.
Any further guidance would be great, but this headache has given me the push to start examining Terraform as my way forward so things can be more consolidated.
I have AWS SAM template, part of which looks like this:
# .......
InternalApiKey:
Type: AWS::ApiGateway::ApiKey
Properties:
Enabled: true
Name: !Sub internal_api_key_${Env}
Value: !Ref InternalApiKeyValue
StageKeys:
- RestApiId: !Ref ServerlessRestApi
StageName: Prod
InternalUsagePlan:
Type: AWS::ApiGateway::UsagePlan
Properties:
ApiStages:
- ApiId: !Ref ServerlessRestApi
Stage: Prod
InternalUsagePlanKey:
Type: AWS::ApiGateway::UsagePlanKey
Properties:
KeyId: !Ref InternalApiKey
KeyType: API_KEY
UsagePlanId: !Ref InternalUsagePlan
#......
Deploying this template as two distinct application (CloudFormation stack) with sam deploy --stack-name=stack-a and sam deploy --stack-name=stack-b fails with the following error even when api key names are different:
API Key already exists (Service: AmazonApiGateway; Status Code: 409; Error Code: ConflictException; Request ID:
redacted; Proxy:
null)
How can I deploy two stacks from this template?
A different key value and your problem will go away. So if you are setting the api key value yourself ensure they are unique. Not sure why that's a constraint between different stages.
Do you really have to provide a value for your key? It's an optional parameter. You can remove it from your template and AWS will generate a unique key for you in each deployment.
Is there a way to do conditional template urls in cloudformation?
This fails because it is not evaluated until the aws cloudformation deploy step and it errors out saying that the templateURL must be an s3 link. When I hard code in one of the urls it will upload that relative file to s3 and in the packaged final template it will just have the s3 url in place.
This doesn't work (pre aws cloudformation package step)
Vpc:
Type: 'AWS::CloudFormation::Stack'
Properties:
Parameters:
AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
NatGateways: 'true'
TemplateURL: !If [IsProduction, './default-vpc-module.yml', './production-vpc-module.yml']
When a url is hardcoded in it will package into this (post aws cloudformation pacakge step)
VpcModule:
Fn::GetAtt:
- Vpc
- Outputs.StackName
AlertingModule:
Fn::GetAtt:
- Alerting
- Outputs.StackName
Priority: '2'
HealthCheckPath: /health-check.php
TemplateURL: https://s3.amazonaws.com/my-cicd-bucket-/fe556ff9386a28c063c4a110b31b.template
Yes you absolutely can achieve what your after, the way i did it was reference the s3 url, by using an aws cloudformaiton package but also a s3 cp in the codebuild stage to enforce naming conventions on the url. As long as your template url paths are distinguished differently.
To provide a very flexible example, you could use !Sub with an !If replacement statement on the dynamic name component, which will also allow your to use !Ref inside the !If statement:
Parameters:
StageProd:
Description: Environment
Default: "production"
Type: String
.......
......
TemplateURL: !Sub
- https://${CfnBucketName}.s3-ap-southeast-2.amazonaws.com/${CfnKeyPrefix}/SomeFileName-${Stage}.yaml
- { Stage: !If [ IsProduction, !Ref StageProd, "default"]}
The above should meet almost any sort of combination of dynamic naming you want to achieve; however, you could also very much simplify the above with a simple !Sub replacing the stage name.
Here is what I ended up doing, but I don't like it.
Vpc:
Condition: IsProduction
Type: 'AWS::CloudFormation::Stack'
Properties:
Parameters:
AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
NatGateways: 'true' # reduce costs
TemplateURL: ./production-vpc-module.yml
Vpc:
Condition: IsNotProduction
Type: 'AWS::CloudFormation::Stack'
Properties:
Parameters:
AlertingModule: !GetAtt 'Alerting.Outputs.StackName'
NatGateways: 'true' # reduce costs
TemplateURL: ./default-vpc-module.yml
I get an "Internal transform" error when deploying this template. I use a Mapping to transform dev and prod settings. This works for single values but cloudformation is choking on trying to transform the Events config. This passes aws cloudformation validate-template whats wrong with my config?:
I use a Mapping to transform dev and prod settings. This works for single values but SAM is choking on trying to transform the Events config, whats wrong with my config?:
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Parameters:
env:
Description: Config map for transforming template with dev or prod values
Type: String
Default: dev
# Template uses !FindInMap along with env parameter to reference dev or prod values defined here
Mappings:
ConfigMap:
dev:
awsAccount: 'dev'
configFile: 'dev_config.yaml'
cloudwatchEvents: !Ref "AWS::NoValue"
prod:
awsAccount: 'prod'
configFile: 'prod_config.yaml'
cloudwatchEvents:
Schedule1:
Type: Schedule
Properties:
Schedule: rate(3 minutes)
Resources:
myfunction:
Type: 'AWS::Serverless::Function'
Properties:
CodeUri: ../build
Handler: lambda_function.lambda_handler
Runtime: python3.6
MemorySize: 128
Timeout: 30
Events:
!FindInMap
- ConfigMap
- !Ref env
- cloudwatchEvents
Environment:
Variables:
config_file: !FindInMap
- ConfigMap
- !Ref env
- configFile
Tags:
account: !FindInMap
- ConfigMap
- !Ref env
- awsAccount
blah: derp
Looks like you have a typo after !FindInMap, you are missing a colon after that. It should be !FindInMap:
I'm having a strange behavior with cloudformation template. This my template, where I create a bucket and want to notification configuration depending on a condition :
AWSTemplateFormatVersion: '2010-09-09'
Description: "Setup Artifacts Bucket"
Parameters:
BucketName:
Description: Name of the pipeline setup arctifact bucket
Type: String
Default: "s3-pipeline-setup"
NotificationCondition:
Description: Conditionally add Notification configuration to the artifact bucket
Type: String
Default: false
Conditions:
AddNotificationConfiguration: !Equals [ !Ref NotificationCondition, true ]
Resources:
ArtifactBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref BucketName
Fn::If:
- AddNotificationConfiguration
-
NotificationConfiguration:
LambdaConfigurations:
-
Function: "arn:aws:lambda:eu-west-1:341292222222227:function:lambda-ops-trigger-pipeline-setup"
Event: "s3:ObjectCreated:*"
Filter:
S3Key:
Rules:
-
Name: prefix
Value: "appstackcodes/"
-
Name: suffix
Value: "txt"
- !Ref AWS::NoValue
When I try a deploy it fails with this error :
00:28:10
UTC+0200 CREATE_FAILED AWS::S3::Bucket ArtifactBucket Encountered
unsupported property Fn::If
I don't really understand the matter.. Can someone try and let me know the mistake there please?
Thanks
Unfortunately you can not do what you intended in cloudformation.
The Fn::If can basically just be used as a ternary expression. E.g.
key: Fn::If: [condition_name, value_if_true, value_if_false]
It can't be used as logic flow like you would in a programming language. There are ways around it. You actually already seemed to have discovered the AWS::NoValue, so it's just a matter of moving the NotificationConfiguration assignment to outside the if.
Resources:
ArtifactBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref BucketName
NotificationConfiguration:
Fn::If:
- AddNotificationConfiguration
- LambdaConfigurations:
-
Function: "arn:aws:lambda:eu-west-1:341294322147:function:lambda-itops-trigger-pipeline-setup"
Event: "s3:ObjectCreated:*"
Filter:
S3Key:
Rules:
-
Name: prefix
Value: "appstackcodes/"
-
Name: suffix
Value: "txt"
- !Ref AWS::NoValue
Effectively you are always assigning something to NotificationConfiguration, but sometimes it's the magic AWS::NoValue. This works in the majority of cases, although there are times when this just isn't sufficient and more creativity is required!