I have a terraform the create a stack on AWS from an yaml file.
My resource "aws_cloudformation_stack" "gitlab-runner" has the following parameters:
Token = "GAdt_YVHgcp5QM_Nms65"
IAMRoleName = "${module.gitlab-iam.iam_role_name}"
My yaml file has the following statements:
Parameters:
GitLabRunnerToken:
Description: >-
Registration token for GitLab Runner. Registration token must contain
exactly 20 alphanumeric characters
AllowedPattern: '^[-_a-zA-Z0-9]*$'
Type: String
MinLength: '20'
MaxLength: '20'
NoEcho: true
Resources:
...
LaunchConfiguration:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: !FindInMap [AWSRegionToAMI, !Ref 'AWS::Region', AMIID]
SecurityGroups:
- !Ref SecurityGroup
InstanceType: !Ref InstanceType
IamInstanceProfile: !Ref GitlabRunnerInstanceProfile
KeyName: !Ref KeyName
BlockDeviceMappings:
- DeviceName: /dev/xvdb
Ebs:
VolumeSize: !Ref 'VolumeSize'
VolumeType: !Ref 'VolumeType'
DeleteOnTermination: !Ref 'DeleteOnTermination'
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
docker run --rm -t -i -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register \
--non-interactive \
--executor "docker" \
--docker-image alpine:3 \
--url "https://gitlab.affinitas.de" \
--registration-token ${GitLabRunnerToken} \
--description "docker-runner" \
--tag-list "docker,aws" \
--run-untagged \
--locked="false"
I am not able to get the value of the parameter GitLabRunnerToken and inject inside UserData: Base64: !Sub |
I got error below:
<template_file>:160,34-51: Unknown variable; There is no variable named "GitLabRunnerToken".
I tried fetch the value using:
1. ${GitLabRunnerToken}
2. Ref: "GitLabRunnerToken"
3. !Ref: "GitLabRunnerToken"
4. !ImportValue "GitLabRunnerToken"
But I am still not able to get and pass the valeu for my UserData, LaunchConfiguration.
Any clue on it?
Thank you.
It's because you use ${variable} format. You pass variables to your cloud-init/userdata scripts this way. Terraform foolishly tires to replace anything "${something}" with values in the vars section of template_file. Lose {} and you'll be fine.
Related
I'm creating a SpotFleet request using CloudFormation, but whenever I try to deploy it, fails with the message:
Unable to fetch parameters [ami-09bee01cc997a78a6] from parameter store for this account.
I'm using the following code (snippet):
SpotFleet:
Type: AWS::EC2::SpotFleet
Properties:
SpotFleetRequestConfigData:
ExcessCapacityTerminationPolicy: default
InstanceInterruptionBehavior: terminate
IamFleetRole: !GetAtt SpotFleetRole.Arn
TargetCapacity: !Ref ClusterSize
TerminateInstancesWithExpiration: false
LaunchSpecifications:
- IamInstanceProfile:
Arn: !Ref ECSInstanceProfile
ImageId: !Ref LatestAmiId
InstanceType: !Ref SpotFleetInstanceType
KeyName: !Ref KeyName
Monitoring:
Enabled: false
SecurityGroups:
- GroupId: !Ref ECSHostSecurityGroup
SubnetId: !Ref PublicSubnet
...
The ImageId is:
LatestAmiId:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: '/aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id'
When I manually fetch the image id from the AWS CLI I get:
aws ssm get-parameters --names /aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id --region us-east-1 --output json | jq -r ."Parameters[].Value"
Output:
ami-09bee01cc997a78a6
If a manually hardcode this value into the ImageId parameter, it returns the same error (Unable to fetch parameters...)
Why it is failing if I am able to fetch the id value from the CLI?
Thanks to #Marcin's comment and this answer, I found the problem.
The problem is that I was using the wrong parameter type for the Image Id. I was using:
LatestAmiId:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: '/aws/service/ecs/optimized-ami/amazon-linux-2/recommended/image_id'
and I should be using this:
LatestAmiId:
Description: Linux AMI
Type: AWS::EC2::Image::Id
I'm writing a cloudformation template in YAML format.
Now i'm stuck with appending the id of the ebs volume to ec2 user data.
Type: AWS::EC2::Volume
Properties:
Size: 50
AvailabilityZone: ap-southeast-1b
Tags:
- Key: Name
Value: Logstash Volume
LogStashMountPoint:
Type: AWS::EC2::VolumeAttachment
Properties:
InstanceId:
Ref: LogstashInstance
VolumeId:
Ref: LogstashVolume
Device: "/dev/xvdf"
LogstashInstance:
Type: AWS::EC2::Instance
Properties:
IamInstanceProfile:
Ref: LogstashInstanceProfile
InstanceType: t2.micro
KeyName: chuongtest
ImageId: ami-0cd31be676780afa7
UserData:
Fn::Base64:
Fn::Sub:
- |
#!/bin/bash -xe
echo ${LogstashVolume} >> /home/ec2-user/ebsid.txt
{LogstashVolume: Ref: LogstashVolume}
touch /home/ec2-user/ebscomplete.txt
curl "http://169.254.169.254/latest/meta-data/instance-id" >> /home/ec2-user/ec2id.txt
touch /home/ec2-user/ec2complete.txt
touch /home/ec2-user/complete.txt
- LogstashVolume: !Ref LogstashVolume
SecurityGroupIds:
- Ref: LogstashSecurityGroup
SubnetId: subnet-0d0e0989f57b96389
Tags:
- Key: Name
Value: Logstash Instance
UserData script with Resource Attribute CloudFormation
I'm following this link but it still doesn't work
When the new instance is launched. It has nothing in /home/ec2-user.
I looked everywhere and this is my final but it didn't work
Can anyone help me with it?
There is at least one syntax error in your UserData:
echo ${LogstashVolume}) >> /home/ec2-user/ebsid.txt
should be
echo ${LogstashVolume} >> /home/ec2-user/ebsid.txt
For further debugging of your UserData, login to the instance and check /var/log/could-init-output.log file.
p.s.
The following is also incorrect:
{LogstashVolume: Ref: LogstashVolume}
Its not a valid bash command.
How can one get the AWS::StackName without the random generate part?
I create a stack: aws cloudformation create-stack --stack-name test
The stack name returned when evaluated using AWS:StackName will included a random generated part, e.g. test-KB0IKRIHP9PH
What I really want returned is the parameter without the generated part, in this case test,
omitting -KB0IKRIHP9PH
My use case for this is, when my containers startup, they need to get database
credential from a pre created named secret. With the random part in place the service all fail to start initially until the secrets are created.
In the code below I assign the StackName to an environment variable.
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
ContainerDefinitions:
- Name: website-service
Environment:
- Name: ENVIRONMENT_VAR
Value: !Join ["", ["CF_", {"Ref": "AWS::StackName"}]]
Here is an update as requested, to show how I create the stack. I am using a MakeFile...
create-test: s3
#ip_address=$$(dig #resolver1.opendns.com ANY myip.opendns.com +short); \
read -s -p "Enter DB Root Password: " pswd; \
[[ -z $$pswd ]] && exit 1 || \
aws cloudformation create-stack \
--capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM \
--stack-name test \
--template-body file://master.yaml \
--parameters ParameterKey=DBRootPassword,ParameterValue=$$pswd \
ParameterKey=DBHostAccessCidr,ParameterValue=$$ip_address/32
I test this with a simple template:
AWSTemplateFormatVersion: 2010-09-09
Resources:
Bucket:
Type: AWS::S3::Bucket
Outputs:
Stack:
Value: !Sub ${AWS::StackName}
The Stack output variable exactly matched the name of the stack that I created. There were no random characters.
I launched the stack via the console.
If AWS::StackName is in the form of test-KB0IKRIHP9PH, then you can get test and perform the Join as follows:
Environment:
- Name: ENVIRONMENT_VAR
Value: !Join ["", ["CD_", !Select [0, !Split ['-', !Ref "AWS::StackName"] ] ] ]
Nested Stack Names contain a random hash.
To overcome the problem, pass the AWS::StackName as a parameter to the nested stack, from the root/master stack.
In the example below, the AWS::StackName is passed as a parameter.
master.yaml
Resources:
S3:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: https://s3-ap-southeast-2.amazonaws.com/...s3.yaml
Parameters:
ParamStackName: !Ref AWS::StackName
s3.yaml Notice: !Ref AWS::StackName while nested, will include a random hash.
Parameters:
ParamStackName:
Type: String
Resources:
MyS3:
Type: AWS::S3::Bucket
Properties:
# Using !Ref AWS::StackName will include the random hash
BucketName: !Ref ParamStackName
I'm working on an infrastructure with CloudFormation.
That is my own infrastructure code shown below .
AWSRegionArch2AMI:
us-east-1:
HVM64: ami-0ff8a91507f77f867
HVMG2: ami-0a584ac55a7631c0c
...
..
.
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType:
Ref: InstanceType
SecurityGroups:
- Ref: InstanceSecurityGroup
KeyName:
Ref: KeyName
UserData:
Fn::Base64: !Sub |
#!/bin/bash
export MY_AMI_NAME=${ !GetAtt AWSRegionArch2AMI.us-east-1.HVM64 }
echo $MY_AMI_NAME > $HOME/user_data.txt
I want to set the variable to the user_data file but it is empty, How I can get the environment variable to inside my user data field and use it my own application side How I can do it.
Please help !
Try this:
UserData:
Fn::Base64: !Sub
- |
#!/bin/bash
MY_AMI_NAME=${image_id}
echo $MY_AMI_NAME > $HOME/user_data.txt
- image_id: !GetAtt AWSRegionArch2AMI.us-east-1.HVM64
Explanation:
Use of export in Bash is to make variables available to subshells - "environment variables". You don't need it there.
See the docs for proper use of the !Sub function.
See also this related Stack Overflow answer.
I have this under parameter section ,
Parameters:
PlatformSelect:
Description: Cockpit platform Select.
Type: String
Default: qa-1
AllowedValues: [qa-1, qa-2, staging, production]
I need to reference this value in my UserData. I’m using Mappings in between.
Mappings:
bootstrap:
ubuntu:
print: echo ${PlatformSelect} >>test.txt
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref ‘InstanceType’
KeyName: !Ref ‘KeyName’
Tags:
- Key: Name
Value: Test
UserData:
Fn::Base64:
Fn::Join:
- ‘’
- - |
#!/bin/bash
- Fn::FindInMap:
- bootstrap
- ubuntu
- print
- |2+
This is not working. Not sure the way I refer it is wrong in first place!!
Should I use something before it like, ‘${AWS::Parameters:PlatformSelect}’ ?
Is there a reason why you are using Mapping in between?
You could easily use !Sub instead
Resources:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
InstanceType: !Ref InstanceType
KeyName: !Ref KeyName
Tags:
- Key: Name
Value: Test
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
${PlatformSelect}
What about a combination of Fn::Join and Ref
UserData:
Fn::Base64:
Fn::Join:
- ''
- - '#!/bin/bash\n'
- 'print: echo'
- !Ref 'PlatformSelect'
- '>>test.txt\n'