Get AWS::StackName without the generated random hash - amazon-web-services

How can one get the AWS::StackName without the random generate part?
I create a stack: aws cloudformation create-stack --stack-name test
The stack name returned when evaluated using AWS:StackName will included a random generated part, e.g. test-KB0IKRIHP9PH
What I really want returned is the parameter without the generated part, in this case test,
omitting -KB0IKRIHP9PH
My use case for this is, when my containers startup, they need to get database
credential from a pre created named secret. With the random part in place the service all fail to start initially until the secrets are created.
In the code below I assign the StackName to an environment variable.
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
- Name: website-service
Environment:
- Name: ENVIRONMENT_VAR
Value: !Join ["", ["CF_", {"Ref": "AWS::StackName"}]]
Here is an update as requested, to show how I create the stack. I am using a MakeFile...
create-test: s3
#ip_address=$$(dig #resolver1.opendns.com ANY myip.opendns.com +short); \
read -s -p "Enter DB Root Password: " pswd; \
[[ -z $$pswd ]] && exit 1 || \
aws cloudformation create-stack \
--capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM \
--stack-name test \
--template-body file://master.yaml \
--parameters ParameterKey=DBRootPassword,ParameterValue=$$pswd \
ParameterKey=DBHostAccessCidr,ParameterValue=$$ip_address/32

I test this with a simple template:
AWSTemplateFormatVersion: 2010-09-09
Resources:
Bucket:
Type: AWS::S3::Bucket
Outputs:
Stack:
Value: !Sub ${AWS::StackName}
The Stack output variable exactly matched the name of the stack that I created. There were no random characters.
I launched the stack via the console.

If AWS::StackName is in the form of test-KB0IKRIHP9PH, then you can get test and perform the Join as follows:
Environment:
- Name: ENVIRONMENT_VAR
Value: !Join ["", ["CD_", !Select [0, !Split ['-', !Ref "AWS::StackName"] ] ] ]

Nested Stack Names contain a random hash.
To overcome the problem, pass the AWS::StackName as a parameter to the nested stack, from the root/master stack.
In the example below, the AWS::StackName is passed as a parameter.
master.yaml
Resources:
S3:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: https://s3-ap-southeast-2.amazonaws.com/...s3.yaml
Parameters:
ParamStackName: !Ref AWS::StackName
s3.yaml Notice: !Ref AWS::StackName while nested, will include a random hash.
Parameters:
ParamStackName:
Type: String
Resources:
MyS3:
Type: AWS::S3::Bucket
Properties:
# Using !Ref AWS::StackName will include the random hash
BucketName: !Ref ParamStackName

Related

AWS Parameter Must Have Values Error (value exists)

I have the following parameter in my CloudFormation script:
CloudFormationURL:
Type: String
Description: S3 URL for nested CloudFormation templates
Default: ""
This parameter covers the CloudFormation scripts in a nested folder of my deployment config.
I use it with a resource like this:
CloudWatchDashboardStack:
Type: "AWS::CloudFormation::Stack"
Properties:
TemplateURL: !Sub "${CloudFormationURL}/cloudwatch-dashboard.cfn.yaml"
Parameters:
AppName: !Ref AppName
DeployPhase: !Ref DeployPhase
DeveloperPrefix: !Ref DeveloperPrefix
Environment: !Ref Environment
Which works fine, and has worked for months.
I needed to add another resource, so I added this:
BatchDNSResources:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
AppName: !Ref AppName
Environment: !Ref Environment
DeveloperPrefix: !Ref DeveloperPrefix
DeployPhase: !Ref DeployPhase
AppVersion: !Ref AppVersion
SharedBucketName: !Ref SharedBucketName
S3Version: !Ref S3Version
HostedZone: !Ref HostedZone
VPCStackName: !FindInMap
- EnvironmentMap
- !Ref Environment
- VpcStackName
Company: !Ref Company
CostCenter: !Ref CostCenter
Team: !Ref Team
TemplateURL: !Sub "${CloudFormationURL}/batch-dns.cfn.yaml"
CloudFormation throws this error and then fails:
Parameters: [CloudFormationURL] must have values
Checking the changeset for the stack I can see the following value for the CloudFormationURL:
s3://application-shared-dev/application-name/qa/cf/nested/KShyDj205UK8mz6W_XUA5TnEF8nqPWHS
Checking the application predeploy logs I can see:
upload: deploy/cloudformation/templates/nested/batch-dns.cfn.yaml to s3://application-shared-dev/application-name/qa/cf/nested/KShyDj205UK8mz6W_XUA5TnEF8nqPWHS/batch-dns.cfn.yaml
And I can see the file in the S3 bucket.
If I remove BatchDNSResource the stack completes successfully.
What the heck am I missing here?
Sometimes, the smallest things will get you.
I had copied the Parameters from the master CloudFormation script, including this one, into the nested script:
CloudFormationURL:
Type: String
Description: S3 URL for nested CloudFormation templates
Default: ""
If you look closely, you will see that I did not pass the parameter into the nested script when calling the resource:
BatchDNSResources:
Type: "AWS::CloudFormation::Stack"
Properties:
Parameters:
AppName: !Ref AppName
Environment: !Ref Environment
DeveloperPrefix: !Ref DeveloperPrefix
DeployPhase: !Ref DeployPhase
AppVersion: !Ref AppVersion
SharedBucketName: !Ref SharedBucketName
S3Version: !Ref S3Version
HostedZone: !Ref HostedZone
VPCStackName: !FindInMap
- EnvironmentMap
- !Ref Environment
- VpcStackName
Company: !Ref Company
CostCenter: !Ref CostCenter
Team: !Ref Team
TemplateURL: !Sub "${CloudFormationURL}/batch-dns.cfn.yaml"
Because the CloudFormation console was saying the issue was with the BatchDNSResources I kept looking at the master script for the problem and missing the reference in the other script. There are two ways to solve this problem:
Keep CloudFormationURL as a parameter in the nested script (if you need it for some reason) and pass the value from the master script.
Remove the parameter from the nested script (if it is not needed)
Sometimes just asking for an extra set of eyeballs and getting a little rest will help you to find the issues. I want to leave this question/answer in place here because when I was searching for the error here and elsewhere no one ever mentioned (probably out of embarrassment) that the mistake is simply overlooking something like this. I hope this answer prompts others to check everything when they run across this type of error.

Passing Security Group Ids and Subnet Ids in a Clould Formation template

Parameters:
ClusterName:
Type: String
ClusterVersion:
Type: Number
AllowedValues: [1.21, 1.20, 1.19, 1.18]
RoleArnValue:
Type: String
ListOfSubnetIDs:
Description: Array of Subnet IDs
Type: List<AWS::EC2::Subnet::Id>
ListOfSecurityGroupIDs:
Description: Array of security group ids
Type: List<AWS::EC2::SecurityGroup::Id>
Resources:
EKSCluster:
Type: AWS::EKS::Cluster
Properties:
Name: !Sub ${ClusterName}
Version: !Sub ${ClusterVersion}
RoleArn: !Sub ${RoleArnValue}
ResourcesVpcConfig:
SecurityGroupIds:
- !Sub ${ListOfSecurityGroupIDs}
SubnetIds:
- !Sub ${ListOfSubnetIDs}
Above is the .yaml clouldformation template I have created so i can spin up eks cluster. Then i am using aws cli to spin up the cluster using the following command.
aws cloudformation deploy --template-file eks.yaml --stack-name cluster-test --parameter-overrides ClusterName=Dev ClusterVersion=1.21 ListOfSubnetIDs=subnet-11111d11b11b011f4,subnet-99999d237f87f11d7,subnet-222222c110c7e4be7,subnet-88888884de8d25176 ListOfSecurityGroupIDs=sg-01111111a21221 RoleArnValue=arn:aws:iam::123456546456:role/cluster-ServiceRole-WMIC72AOWSP0 --capabilities CAPABILITY_NAMED_IAM
I get the following error
An error occurred (ValidationError) when calling the CreateChangeSet operation: Template error: variable ListOfSecurityGroupIDs in Fn::Sub expression does not resolve to a string
I am not sure why. Am i using !sub in correctly? Would really appreciate input on this.
Since you want to reference the parameters you provided the template as they are, you should use the Ref function.
Here's an example of a valid template:
Parameters:
ClusterName:
Type: String
RoleArnValue:
Type: String
ListOfSubnetIDs:
Description: Array of Subnet IDs
Type: List<AWS::EC2::Subnet::Id>
ListOfSecurityGroupIDs:
Description: Array of security group ids
Type: List<AWS::EC2::SecurityGroup::Id>
Resources:
EKSCluster:
Type: AWS::EKS::Cluster
Properties:
Name: !Ref ClusterName
RoleArn: !Ref RoleArnValue
ResourcesVpcConfig:
SecurityGroupIds: !Ref ListOfSecurityGroupIDs
SubnetIds: !Ref ListOfSubnetIDs
and here's how I deployed it:
aws cloudformation deploy --template-file eks.yml --stack-name cluster-test --parameter-overrides ClusterName=Dev ListOfSubnetIDs=subnet-be0a99c4,subnet-c71046ae ListOfSecurityGroupIDs=sg-009690ac6b3bff6df,sg-009a3f1cb63943941 -RoleArnValue=...
Sub should be used when you want to perform string manipulation. Checkout the examples from the documentation.

Cloudformation ImportValue within ParameterOverride CodePipeline

Problem:
I have a cloudformation template that is supposed to retrieve code found in CodeCommit and push it to a Lambda. The code in CodeCommit also contains a SAM template with a few parameters. The SAM template has the following setup
AWSTemplateFormatVersion: 2010-09-09
Transform: AWS::Serverless-2016-10-31
Description: SAM Template for Deploy Python code to Lambda
Parameters:
ArtifactsBucket:
Description: The artifact bucket to get the lambda code
Type: String
Name:
Description: Name of the lambda function
Type: String
SqsARN:
Description: AWS SQS Arn to act as a trigger for the lambda function
Type: String
...
and the CodePipeline Cloudformation template has the following to override the 3 parameters present in the SAM template.
...
- Name: Deploy
Actions:
- Name: Deploy
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: CloudFormation
Version: 1
Configuration:
ActionMode: CREATE_UPDATE
Capabilities: 'CAPABILITY_IAM,CAPABILITY_AUTO_EXPAND,CAPABILITY_NAMED_IAM'
ChangeSetName: !Join
- '-'
- - lambda
- !Ref CodeCommitRepoName
ParameterOverrides: !Sub |
{
"ArtifactsBucket": "${ArtifactsBucket}",
"Name": "${CodeCommitRepoName}",
"SqsARN": {"Fn::ImportValue": "My-queue:us-east-1:Queue:Arn"}
}
...
The ArtifactBucket and the Name parameters are easily changed by the !Sub function but I am not able to have a valid value for SqsARN which is an imported value.
QuestionIs there anyway to include ImportValue in conjungtion with a Sub function within ParametersOverride?
Attempts I also tried to switch from
{"Fn::ImportValue": "My-queue:us-east-1:Queue:Arn"}
to
!ImportValue": "My-queue:us-east-1:Queue:Arn"
which also did not work. Remove the !Sub function and using a !Ref function yields the same output/problem as with ImportValue.
Key thing to remember is, using JSON within YAML
From the documentation
You can't use the short form of !ImportValue when it contains a !Sub.
It is valid for AWS CloudFormation, but not valid
for YAML:
Assuming Environment as Parameter, Here is a working example of Sub >> ImportValue >> Sub
Value: !Sub
- '{
"sqsUrl": "${sqsUrl}",
}'
- {
sqsUrl: { 'Fn::ImportValue': { 'Fn::Sub': 'QueUrl-${Environment}' } }
}
Applying it to above example might look something like below(using ssm for testing)
Parameters:
Environment:
Type: String
Default: DV
ArtifactsBucket:
Type: String
Default: TestBucket
CodeCommitRepoName:
Type: String
Default: Test
Resources:
SmsLambdaParameter:
Type: 'AWS::SSM::Parameter'
Properties:
Name: !Sub
- '/${EnvFullUpper}/My-Param/Test'
- { EnvFullUpper: !Ref Environment }
Type: 'String'
Value: !Sub
- '{
"ArtifactsBucket": "${ArtifactsBucket}",
"Name": "${CodeCommitRepoName}",
"SqsARN": "${SqsArn}"
}'
- { SqsARN: { 'Fn::ImportValue': { 'Fn::Sub': 'QueueArn-${Environment}' } } }
You can use array form of Fn::Sub:
!Sub
- String
- Var1Name: Var1Value
Var2Name: Var2Value
which would result in:
ParameterOverrides: !Sub
- |
{
"ArtifactsBucket": "${ArtifactsBucket}",
"Name": "${CodeCommitRepoName}",
"SqsARN": "${SqsARNImport}"
}
- SqsARNImport: !ImportValue <name-of-your-queue-export>

Terraform cloudformation_stack unkown variable onto UserData

I have a terraform the create a stack on AWS from an yaml file.
My resource "aws_cloudformation_stack" "gitlab-runner" has the following parameters:
Token = "GAdt_YVHgcp5QM_Nms65"
IAMRoleName = "${module.gitlab-iam.iam_role_name}"
My yaml file has the following statements:
Parameters:
GitLabRunnerToken:
Description: >-
Registration token for GitLab Runner. Registration token must contain
exactly 20 alphanumeric characters
AllowedPattern: '^[-_a-zA-Z0-9]*$'
Type: String
MinLength: '20'
MaxLength: '20'
NoEcho: true
Resources:
...
LaunchConfiguration:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: !FindInMap [AWSRegionToAMI, !Ref 'AWS::Region', AMIID]
SecurityGroups:
- !Ref SecurityGroup
InstanceType: !Ref InstanceType
IamInstanceProfile: !Ref GitlabRunnerInstanceProfile
KeyName: !Ref KeyName
BlockDeviceMappings:
- DeviceName: /dev/xvdb
Ebs:
VolumeSize: !Ref 'VolumeSize'
VolumeType: !Ref 'VolumeType'
DeleteOnTermination: !Ref 'DeleteOnTermination'
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
docker run --rm -t -i -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
--non-interactive \
--executor "docker" \
--docker-image alpine:3 \
--url "https://gitlab.affinitas.de" \
--registration-token ${GitLabRunnerToken} \
--description "docker-runner" \
--tag-list "docker,aws" \
--run-untagged \
--locked="false"
I am not able to get the value of the parameter GitLabRunnerToken and inject inside UserData: Base64: !Sub |
I got error below:
<template_file>:160,34-51: Unknown variable; There is no variable named "GitLabRunnerToken".
I tried fetch the value using:
1. ${GitLabRunnerToken}
2. Ref: "GitLabRunnerToken"
3. !Ref: "GitLabRunnerToken"
4. !ImportValue "GitLabRunnerToken"
But I am still not able to get and pass the valeu for my UserData, LaunchConfiguration.
Any clue on it?
Thank you.
It's because you use ${variable} format. You pass variables to your cloud-init/userdata scripts this way. Terraform foolishly tires to replace anything "${something}" with values in the vars section of template_file. Lose {} and you'll be fine.

Configuration of environment variables for different AWS Lambda aliases in CloudFormation template

I create CloudFormation template for my AWS Lambda function and I need to specify different values of environment variables for different lambda aliases.
My template looks like:
AWSTemplateFormatVersion: "2010-09-09"
Transform: "AWS::Serverless-2016-10-31"
Description: Lambda function configuration
Resources:
EndpointLambda:
Type: "AWS::Lambda::Function"
Properties:
FunctionName: "endpoint-lambda"
Handler: "com.test.aws.RequestHandler::handleRequest"
Runtime: java8
Code:
S3Bucket: "lambda-functions"
S3Key: "test-endpoint-lambda-0.0.1.jar"
Description: Test Lambda function
MemorySize: 256
Timeout: 60
Environment:
Variables:
ES_HOST: test-es-host-url
ES_ON: true
ES_PORT: 443
ES_PROTOCOL: https
REDIS_URL: test-redis-host-url
QaLambdaAlias:
Type: "AWS::Lambda::Alias"
Properties:
FunctionName: !Ref EndpointLambda
FunctionVersion: 1
Name: "QA"
Description: "QA alias"
ProdLambdaAlias:
Type: "AWS::Lambda::Alias"
Properties:
FunctionName: !Ref EndpointLambda
FunctionVersion: 1
Name: "Prod"
Description: "Production alias"
As you see, I have two aliases - QA and Prod and bunch of environment variables. I specified variables with common values in lambda function declaration. But I need to declare for QA alias env. variable's values related to QA, and for Prod alias - values for Prod environment. Any ideas how can I do that?
You can use CloudFormation Parameters to do this. As a quick example:
Parameters:
LambdaRuntime:
Type: String
Default: 'java8'
Description: What Lambda runtime do we use?
Resources:
QaLambdaAlias:
Type: "AWS::Lambda::Alias"
Properties:
FunctionName:
Ref: EndpointLambda
FunctionVersion: 1
Name: "QA"
Description: "QA alias"
Runtime:
Ref: LambdaRuntime
Then, if you want to use a different parameter, when you deploy via CLI, you can override with parameter-overrides like this:
aws cloudformation deploy --stack-name MyStack --template-file \
CloudFormation/MyStack.yaml --capabilities CAPABILITY_IAM \
--parameter-overrides LambdaRuntime=nodejs8.10