I'm writing the cloudformation template that includes ec2 instance. In userdata block, I want to create a file with some content. In the file, I'm initializing local variable MY_MESSAGE, but next, after the template is deployed this variable is not shown in the file.
original temlate:
EC2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-03368e982f317ae48
InstanceType: t2.micro
KeyName: ec2
UserData:
!Base64 |
#!/bin/bash
cat <<EOF > /etc/aws-kinesis/start.sh
#!/bin/sh
MY_MESSAGE="Hello World"
echo $MY_MESSAGE
output file in ec2 instance:
#!/bin/sh
MY_MESSAGE="Hello World"
echo
As you can see variable MY_MESSAGE does not exist in echo block.
You can put EOF in quotes: "EOF":
UserData:
!Base64 |
#!/bin/bash
cat <<"EOF" > /etc/aws-kinesis/start.sh
#!/bin/sh
MY_MESSAGE="Hello World"
echo $MY_MESSAGE
EOF
Related
I have a commadelimted list and template snippet like:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
IPWhitelist:
Description: Comma-delimited list of CIDR blocks.
Type: CommaDelimitedList
Default: '1.1.1.1/32, 2.2.2.2/32, 3.3.3.3/32'
Resources:
EC2Instance:
Type: 'AWS::EC2::Instance'
CreationPolicy:
ResourceSignal:
Timeout: PT5M
Metadata:
'AWS::CloudFormation::Init':
configSets:
foobar_setup:
- configure_foo
configure_foo:
commands:
01_config:
command: !Sub |
IFS=', ' read -r -a array <<< "$(echo ${IPWhitelist} | sed -e 's/[][]//g')"
for IP in "${!array[#]}";do echo $IP >> /foo/bar/allowed_ips.txt;done
I'd like to run the following command in the Init type commands key:
IFS=', ' read -r -a array <<< "$(echo ${IPWhitelist} | sed -e 's/[][]//g')"
for IP in "${array[#]}";do echo $IP >> /etc/squid/allowed_ips.txt;done
so for the array as the doc says
To write a dollar sign and curly braces (${}) literally, add an
exclamation point (!) after the open curly brace, such as ${!Literal}.
AWS CloudFormation resolves this text as ${Literal}.
What about the first line? how to substitute cloudformation parameter inside bash command substitution $()?
Error message:
Template contains errors.: Template error: variable IPWhitelist in
Fn::Sub expression does not resolve to a string
I am trying to deploy a Sagemaker Lifecycle with AWS CloudFormation.
The Lifecycle is importing ipynb notebooks from s3 bucket to the Sagemaker notebook instance.
the bucket name is specified in the parameters, I want to use it in a !Sub function inside the bash script of the Lifecycle.
The problem is that the CF runs first on the template and tries to complete its own functions (like !Sub) and then the scripts upload as bash script to the Lifecycle.
This is my code:
LifecycleConfig:
Type: AWS::SageMaker::NotebookInstanceLifecycleConfig
Properties:
NotebookInstanceLifecycleConfigName: !Sub
- ${NotebookInstanceName}LifecycleConfig
- NotebookInstanceName: !Ref NotebookInstanceName
OnStart:
- Content:
Fn::Base64: !Sub
- |
#!/bin/bash -xe
set -e
CP_SAMPLES=true
EXTRACT_CSV=false
s3region=s3.amazonaws.com
SRC_NOTEBOOK_DIR=${Consumer2BucketName}/sagemaker-notebooks
Sagedir=/home/ec2-user/SageMaker
industry=industry
notebooks=("notebook1.ipynb" "notebook2.ipynb" "notebook3.ipynb")
download_files(){
for notebook in "${notebooks[#]}"
do
printf "aws s3 cp s3://${SRC_NOTEBOOK_DIR}/${notebook} ${Sagedir}/${industry}\n"
aws s3 cp s3://"${SRC_NOTEBOOK_DIR}"/"${notebook}" ${Sagedir}/${industry}
done
}
if [ ${CP_SAMPLES} = true ]; then
sudo -u ec2-user mkdir -p ${Sagedir}/${industry}
mkdir -p ${Sagedir}/${industry}
download_files
chmod -R 755 ${Sagedir}/${industry}
chown -R ec2-user:ec2-user ${Sagedir}/${industry}/.
fi
- Consumer2BucketName: !Ref Consumer2BucketName
Raised the following error:
Template error: variable names in Fn::Sub syntax must contain only alphanumeric characters, underscores, periods, and colons
It seems that was a conflict with the Bash Vars and the !Sub CF function.
In the following template I changed the Bash Vars and removed the {}:
LifecycleConfig:
Type: AWS::SageMaker::NotebookInstanceLifecycleConfig
Properties:
NotebookInstanceLifecycleConfigName: !Sub
- ${NotebookInstanceName}LifecycleConfig
- NotebookInstanceName: !Ref NotebookInstanceName
OnStart:
- Content:
Fn::Base64:
!Sub
- |
#!/bin/bash -xe
set -e
CP_SAMPLES=true
EXTRACT_CSV=false
s3region=s3.amazonaws.com
SRC_NOTEBOOK_DIR=${Consumer2BucketName}/sagemaker-notebooks
Sagedir=/home/ec2-user/SageMaker
industry=industry
notebooks=("notebook1.ipynb" "notebook2.ipynb" "notebook3.ipynb")
download_files(){
for notebook in $notebooks
do
printf "aws s3 cp s3://$SRC_NOTEBOOK_DIR/${!notebook} $Sagedir/$industry\n"
aws s3 cp s3://"$SRC_NOTEBOOK_DIR"/"${!notebook}" $Sagedir/$industry
done
}
if [ $CP_SAMPLES = true ]; then
sudo -u ec2-user mkdir -p $Sagedir/$industry
mkdir -p $Sagedir/$industry
download_files
chmod -R 755 $Sagedir/$industry
chown -R ec2-user:ec2-user $Sagedir/$industry/.
fi
- Consumer2BucketName: !Ref Consumer2BucketName
The problem here is the for loop is not running through all the notebooks in the list but importing only the first one.
After going through some solutions I tried adding [#] to the notebooks:
for notebook in $notebooks[#]
and
for notebook in “$notebooks[#]“/”$notebooks[*]“/$notebooks[#]
I got the same error.
It seems that was a conflict with the Bash Vars and the !Sub CF function.
That's correct. Both bash and !Sub use ${} for variable substitution. You can escape the bash variables with ${!}. For example:
for notebook in "${!notebooks[#]}"
Also mentioned in the docs:
To write a dollar sign and curly braces (${}) literally, add an exclamation point (!) after the open curly brace, such as ${!Literal}. AWS CloudFormation resolves this text as ${Literal}.
How do I get public IP of EC2 instance and write to a text file in UserData. Tried the following, but as expected it wrote the text literally, rather than resolving it.
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
echo "Public IP: " !GetAtt Bastion.PublicIp > /home/ec2-user/readme.txt
Thanks in advance! (PS : It has to be yaml)
I am using an AWS CloudFormation template for IAM role-based access to an EC2 instance.
I getting permission denied error while running the template, and I am not able to access the EC2 machine with a username without a pem file.
Instance:
Type: 'AWS::EC2::Instance'
Metadata:
'AWS::CloudFormation::Init':
config:
files:
/opt/authorized_keys_command.sh:
content: >
#!/bin/bash -e
if [ -z "$1" ]; then
exit 1
fi
SaveUserName="$1"
SaveUserName=${SaveUserName//"+"/".plus."}
SaveUserName=${SaveUserName//"="/".equal."}
SaveUserName=${SaveUserName//","/".comma."}
SaveUserName=${SaveUserName//"#"/".at."}
aws iam list-ssh-public-keys --user-name "$SaveUserName" --query
"SSHPublicKeys[?Status == 'Active'].[SSHPublicKeyId]" --output
text | while read KeyId; do
aws iam get-ssh-public-key --user-name "$SaveUserName" --ssh-public-key-id "$KeyId" --encoding SSH --query "SSHPublicKey.SSHPublicKeyBody" --output text
done
mode: '000755'
owner: root
group: root
/opt/import_users.sh:
content: >
#!/bin/bash
aws iam list-users --query "Users[].[UserName]" --output text |
while read User; do
SaveUserName="$User"
SaveUserName=${SaveUserName//"+"/".plus."}
SaveUserName=${SaveUserName//"="/".equal."}
SaveUserName=${SaveUserName//","/".comma."}
SaveUserName=${SaveUserName//"#"/".at."}
if id -u "$SaveUserName" >/dev/null 2>&1; then
echo "$SaveUserName exists"
else
#sudo will read each file in /etc/sudoers.d, skipping file names that end in ?~? or contain a ?.? character to avoid causing problems with package manager or editor temporary/backup files.
SaveUserFileName=$(echo "$SaveUserName" | tr "." " ")
/usr/sbin/adduser "$SaveUserName"
echo "$SaveUserName ALL=(ALL) NOPASSWD:ALL" > "/etc/sudoers.d/$SaveUserFileName"
fi
done
mode: '000755' owner: root group: root
/etc/cron.d/import_users:
content: |
*/10 * * * * root /opt/import_users.sh
mode: '000644' owner: root
group: root
/etc/cfn/cfn-hup.conf:
content: !Sub |
[main]
stack=${AWS::StackId}
region=${AWS::Region}
interval=1
mode: '000400' owner: root group: root
/etc/cfn/hooks.d/cfn-auto-reloader.conf:
content: !Sub >
[cfn-auto-reloader-hook]
triggers=post.update
path=Resources.Instance.Metadata.AWS::CloudFormation::Init
action=/opt/aws/bin/cfn-init --verbose
--stack=${AWS::StackName} --region=${AWS::Region}
--resource=Instance
runas=root
commands:
a_configure_sshd_command:
command: >-
sed -i 's:#AuthorizedKeysCommand none:AuthorizedKeysCommand
/opt/authorized_keys_command.sh:g' /etc/ssh/sshd_config
b_configure_sshd_commanduser:
command: >-
sed -i 's:#AuthorizedKeysCommandUser
nobody:AuthorizedKeysCommandUser nobody:g' /etc/ssh/sshd_config
c_import_users:
command: ./import_users.sh
cwd: /opt
services:
sysvinit:
cfn-hup:
enabled: true
ensureRunning: true
files:
- /etc/cfn/cfn-hup.conf
- /etc/cfn/hooks.d/cfn-auto-reloader.conf
sshd:
enabled: true
ensureRunning: true
commands:
- a_configure_sshd_command
- b_configure_sshd_commanduser
'AWS::CloudFormation::Designer':
id: 85ddeee0-0623-4f50-8872-1872897c812f
Properties:
ImageId: !FindInMap
- RegionMap
- !Ref 'AWS::Region'
- AMI
IamInstanceProfile: !Ref InstanceProfile
InstanceType: t2.micro
UserData:
'Fn::Base64': !Sub >
#!/bin/bash -x
/opt/aws/bin/cfn-init --verbose --stack=${AWS::StackName}
--region=${AWS::Region} --resource=Instance
/opt/aws/bin/cfn-signal --exit-code=$? --stack=${AWS::StackName}
--region=${AWS::Region} --resource=Instance
This User Data script will configure a Linux instance to use password authentication.
While the password here is hard-coded, you could obtain it in other ways and set it to the appropriate value.
#!
echo 'secret-password' | passwd ec2-user --stdin
sed -i 's|[#]*PasswordAuthentication no|PasswordAuthentication yes|g' /etc/ssh/sshd_config
systemctl restart sshd.service
I have a bash script that I want to run on my instances but I only want the second portion of the script to run if a value is true. Also, I would prefer to not have the if statement take place in the script.
Parameters:
#TestParameter = TRUE
Resource:
UserData:
Fn::Sub: |
echo "This is a test example"
#If TestParameter is true:
echo "Only is parameter is true"
In my opinion, it's better to construct Sub the parameter into the script:
UserData:
Fn::Sub: |
echo "This is a test example"
if ${TestParameter}; then
echo "Only is parameter is true"
fi
But since you don't want to have the if inside the script, you'll have to construct the body of the script in the Cloudformation template. Building strings in CFN is always messy. Try something like this:
UserData:
Fn::Join: ["\n", ["echo 'This is a test example'",
['Fn::If': [!Equals [!Ref TestParameter, "true"], "echo 'Parameter is true'", ""]]]]