AWS Systems Manager using cloud formation template - amazon-web-services

I started writing something for AWS Systems Manager to:
Create custom Windows and Linux images and
How to apply Windows and Linux Updates to the AMIs that would be useful...
I'm following this example, but am not able to get results when I run it.
This is the error I receive:
Template format error: At least one Resources member must be defined.
Please let me know what I'm doing wrong here, I have gone through the code but not able to find what I am doing wrong.
#description: Create a Golden AMI with Linux distribution packages(ClamAV) and Amazon
#software(SSM & Inspector). For details,see https://github.com/miztiik/AWS-Demos/tree/master/How-To/setup-ami-lifecycle-management-using-ssm
Resources:
Ec2Instance:
Type: 'AWS::EC2::Instance'
assumeRole: "{{AutomationAssumeRole}}"
parameters:
SourceAmiId:
type: String
description: "(Required) The source Amazon Machine Image ID."
default: ami-0d2692b6acea72ee6
InstanceIamRole:
type: String
description: "(Required) The name of the role that enables Systems Manager (SSM)
to manage the instance."
default: ManagedInstanceRole
AutomationAssumeRole:
type: String
description: "(Required) The ARN of the role that allows Automation to perform
the actions on your behalf."
default: arn:aws:iam::{{global:ACCOUNT_ID}}:role/AutomationServiceRole
SubnetId:
type: String
description: "(Required) The subnet that the created instance will be placed into."
default: subnet-0a72519be8028a56c
TargetAmiName:
type: String
description: "(Optional) The name of the new AMI that will be created. Default
is a system-generated string including the source AMI id, and the creation time
and date."
default: GoldenAMI-RH-7_on_{{global:DATE_TIME}}
InstanceType:
type: String
description: "(Optional) Type of instance to launch as the workspace host. Instance
types vary by region. Default is t2.micro."
default: t2.micro
PreUpdateScript:
type: String
description: (Optional) URL of a script to run before updates are applied. Default
("none") is to not run a script.
default: none
PostUpdateScript:
type: String
description: (Optional) URL of a script to run after package updates are applied.
Default ("none") is to not run a script.
default: none
IncludePackages:
type: String
description: (Optional) Only update these named packages. By default ("all"),
all available updates are applied.
default: all
ExcludePackages:
type: String
description: (Optional) Names of packages to hold back from updates, under all
conditions. By default ("none"), no package is excluded.
default: none
lambdaFunctionName:
type: String
description: "(Required) The name of the lambda function. Default ('none') is
to not run a script."
default: Automation-UpdateSsmParam
mainSteps:
- name: launchInstance
action: aws:runInstances
maxAttempts: 5
timeoutSeconds: 1200
onFailure: Abort
inputs:
ImageId: "{{SourceAmiId}}"
InstanceType: "{{InstanceType}}"
SubnetId: "{{ SubnetId }}"
UserData: IyEvYmluL2Jhc2gNCg0KZnVuY3Rpb24gZ2V0X2NvbnRlbnRzKCkgew0KICAgIGlmIFsgLXggIiQod2hpY2ggY3VybCkiIF07IHRoZW4NCiAgICAgICAgY3VybCAtcyAtZiAiJDEiDQogICAgZWxpZiBbIC14ICIkKHdoaWNoIHdnZXQpIiBdOyB0aGVuDQogICAgICAgIHdnZXQgIiQxIiAtTyAtDQogICAgZWxzZQ0KICAgICAgICBkaWUgIk5vIGRvd25sb2FkIHV0aWxpdHkgKGN1cmwsIHdnZXQpIg0KICAgIGZpDQp9DQoNCnJlYWRvbmx5IElERU5USVRZX1VSTD0iaHR0cDovLzE2OS4yNTQuMTY5LjI1NC8yMDE2LTA2LTMwL2R5bmFtaWMvaW5zdGFuY2UtaWRlbnRpdHkvZG9jdW1lbnQvIg0KcmVhZG9ubHkgVFJVRV9SRUdJT049JChnZXRfY29udGVudHMgIiRJREVOVElUWV9VUkwiIHwgYXdrIC1GXCIgJy9yZWdpb24vIHsgcHJpbnQgJDQgfScpDQpyZWFkb25seSBERUZBVUxUX1JFR0lPTj0idXMtZWFzdC0xIg0KcmVhZG9ubHkgUkVHSU9OPSIke1RSVUVfUkVHSU9OOi0kREVGQVVMVF9SRUdJT059Ig0KDQpyZWFkb25seSBTQ1JJUFRfTkFNRT0iYXdzLWluc3RhbGwtc3NtLWFnZW50Ig0KIFNDUklQVF9VUkw9Imh0dHBzOi8vYXdzLXNzbS1kb3dubG9hZHMtJFJFR0lPTi5zMy5hbWF6b25hd3MuY29tL3NjcmlwdHMvJFNDUklQVF9OQU1FIg0KDQppZiBbICIkUkVHSU9OIiA9ICJjbi1ub3J0aC0xIiBdOyB0aGVuDQogIFNDUklQVF9VUkw9Imh0dHBzOi8vYXdzLXNzbS1kb3dubG9hZHMtJFJFR0lPTi5zMy5jbi1ub3J0aC0xLmFtYXpvbmF3cy5jb20uY24vc2NyaXB0cy8kU0NSSVBUX05BTUUiDQpmaQ0KDQppZiBbICIkUkVHSU9OIiA9ICJ1cy1nb3Ytd2VzdC0xIiBdOyB0aGVuDQogIFNDUklQVF9VUkw9Imh0dHBzOi8vYXdzLXNzbS1kb3dubG9hZHMtJFJFR0lPTi5zMy11cy1nb3Ytd2VzdC0xLmFtYXpvbmF3cy5jb20vc2NyaXB0cy8kU0NSSVBUX05BTUUiDQpmaQ0KDQpjZCAvdG1wDQpGSUxFX1NJWkU9MA0KTUFYX1JFVFJZX0NPVU5UPTMNClJFVFJZX0NPVU5UPTANCg0Kd2hpbGUgWyAkUkVUUllfQ09VTlQgLWx0ICRNQVhfUkVUUllfQ09VTlQgXSA7IGRvDQogIGVjaG8gQVdTLVVwZGF0ZUxpbnV4QW1pOiBEb3dubG9hZGluZyBzY3JpcHQgZnJvbSAkU0NSSVBUX1VSTA0KICBnZXRfY29udGVudHMgIiRTQ1JJUFRfVVJMIiA+ICIkU0NSSVBUX05BTUUiDQogIEZJTEVfU0laRT0kKGR1IC1rIC90bXAvJFNDUklQVF9OQU1FIHwgY3V0IC1mMSkNCiAgZWNobyBBV1MtVXBkYXRlTGludXhBbWk6IEZpbmlzaGVkIGRvd25sb2FkaW5nIHNjcmlwdCwgc2l6ZTogJEZJTEVfU0laRQ0KICBpZiBbICRGSUxFX1NJWkUgLWd0IDAgXTsgdGhlbg0KICAgIGJyZWFrDQogIGVsc2UNCiAgICBpZiBbWyAkUkVUUllfQ09VTlQgLWx0IE1BWF9SRVRSWV9DT1VOVCBdXTsgdGhlbg0KICAgICAgUkVUUllfQ09VTlQ9JCgoUkVUUllfQ09VTlQrMSkpOw0KICAgICAgZWNobyBBV1MtVXBkYXRlTGludXhBbWk6IEZpbGVTaXplIGlzIDAsIHJldHJ5Q291bnQ6ICRSRVRSWV9DT1VOVA0KICAgIGZpDQogIGZpIA0KZG9uZQ0KDQppZiBbICRGSUxFX1NJWkUgLWd0IDAgXTsgdGhlbg0KICBjaG1vZCAreCAiJFNDUklQVF9OQU1FIg0KICBlY2hvIEFXUy1VcGRhdGVMaW51eEFtaTogUnVubmluZyBVcGRhdGVTU01BZ2VudCBzY3JpcHQgbm93IC4uLi4NCiAgLi8iJFNDUklQVF9OQU1FIiAtLXJlZ2lvbiAiJFJFR0lPTiINCmVsc2UNCiAgZWNobyBBV1MtVXBkYXRlTGludXhBbWk6IFVuYWJsZSB0byBkb3dubG9hZCBzY3JpcHQsIHF1aXR0aW5nIC4uLi4NCmZp
MinInstanceCount: 1
MaxInstanceCount: 3
IamInstanceProfileName: "{{InstanceIamRole}}"
- name: updateOSSoftware
action: aws:runCommand
maxAttempts: 3
timeoutSeconds: 3600
onFailure: Abort
inputs:
DocumentName: AWS-RunShellScript
InstanceIds:
- "{{launchInstance.InstanceIds}}"
Parameters:
commands:
- set -e
- '[ -x "$(which wget)" ] && get_contents=''wget $1 -O -'''
- '[ -x "$(which curl)" ] && get_contents=''curl -s -f $1'''
- eval $get_contents https://aws-ssm-downloads-{{global:REGION}}.s3.amazonaws.com/scripts/aws-update-linux-instance
> /tmp/aws-update-linux-instance
- chmod +x /tmp/aws-update-linux-instance
- "/tmp/aws-update-linux-instance --pre-update-script '{{PreUpdateScript}}'
--post-update-script '{{PostUpdateScript}}' --include-packages '{{IncludePackages}}'
--exclude-packages '{{ExcludePackages}}' 2>&1 | tee /tmp/aws-update-linux-instance.log"
- name: installCustomizations
action: aws:runCommand
maxAttempts: 3
timeoutSeconds: 600
onFailure: Abort
inputs:
DocumentName: AWS-RunShellScript
InstanceIds:
- "{{launchInstance.InstanceIds}}"
Parameters:
commands:
- curl -O http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
- rpm -ivh epel-release-latest-7.noarch.rpm
- yum -y install httpd
- systemctl enable httpd
- systemctl restart httpd
- sudo yum --enablerepo=epel install -y clamav
- yum-config-manager --disable epel
- cat /etc/motd >> /var/www/html/index.html
- echo 'Welcome' >> /var/www/html/index.html
- cat > /etc/motd <<- EOF
- " __ __ _ _ _ _ "
- " /\\ | \\/ (_) /\\ | | | | (_) "
- " / \\ | \\ / |_ / \\ _ _| |_ ___ _ __ ___ __ _| |_ _ ___
\ _ __ "
- " / /\\ \\ | |\\/| | | / /\\ \\| | | | __/ _ \\| '_ ` _ \\ / _` | __|
|/ _ \\| '_ \\ "
- " / ____ \\| | | | | / ____ \\ |_| | || (_) | | | | | | (_| | |_| | (_)
| | | |"
- " /_/ \\_\\_| |_|_| /_/ \\_\\__,_|\\__\\___/|_| |_| |_|\\__,_|\\__|_|\\___/|_|
|_|"
- " "
- " "
- EOF
- name: installInspectorAgent
action: aws:runCommand
maxAttempts: 3
timeoutSeconds: 600
onFailure: Abort
inputs:
DocumentName: AmazonInspector-ManageAWSAgent
InstanceIds:
- "{{launchInstance.InstanceIds}}"
Parameters:
Operation: Install
- name: installUnifiedCloudWatchAgent
action: aws:runCommand
maxAttempts: 3
timeoutSeconds: 600
onFailure: Abort
inputs:
DocumentName: AWS-ConfigureAWSPackage
InstanceIds:
- "{{launchInstance.InstanceIds}}"
Parameters:
name: AmazonCloudWatchAgent
action: Install
- name: stopInstance
action: aws:changeInstanceState
maxAttempts: 3
timeoutSeconds: 1200
onFailure: Abort
inputs:
InstanceIds:
- "{{launchInstance.InstanceIds}}"
DesiredState: stopped
- name: createImage
action: aws:createImage
maxAttempts: 3
onFailure: Abort
inputs:
InstanceId: "{{launchInstance.InstanceIds}}"
ImageName: "{{TargetAmiName}}"
NoReboot: true
ImageDescription: AMI Generated by EC2 Automation on {{global:DATE_TIME}} from
{{SourceAmiId}}
- name: createEncryptedCopy
action: aws:copyImage
maxAttempts: 3
onFailure: Abort
inputs:
SourceImageId: "{{createImage.ImageId}}"
SourceRegion: "{{global:REGION}}"
ImageName: Encrypted-{{TargetAmiName}}
ImageDescription: Encrypted GoldenAMI by SSM Automation on {{global:DATE_TIME}}
from source AMI {{createImage.ImageId}}
Encrypted: true
- name: createTagsForEncryptedImage
action: aws:createTags
maxAttempts: 1
onFailure: Continue
inputs:
ResourceType: EC2
ResourceIds:
- "{{createEncryptedCopy.ImageId}}"
Tags:
- Key: Automation-Id
Value: "{{automation:EXECUTION_ID}}"
- Key: Owner
Value: Mystique
- Key: SourceAMI
Value: "{{SourceAmiId}}"
- Key: Amazon-Inspector
Value: 'true'
- Key: Amazon-SSM
Value: 'true'
- Key: Encrypted
Value: 'true'
- name: updateSsmParam
action: aws:invokeLambdaFunction
timeoutSeconds: 1200
maxAttempts: 1
onFailure: Abort
inputs:
FunctionName: Automation-UpdateSsmParam
Payload: '{"parameterName":"/GoldenAMI/Linux/RedHat-7/latest", "parameterValue":"{{createEncryptedCopy.ImageId}}"}'
- name: terminateInstance
action: aws:changeInstanceState
maxAttempts: 3
onFailure: Continue
inputs:
InstanceIds:
- "{{launchInstance.InstanceIds}}"
DesiredState: terminated
- name: deleteUnEcryptedImage
action: aws:deleteImage
maxAttempts: 3
timeoutSeconds: 180
onFailure: Abort
inputs:
ImageId: "{{createImage.ImageId}}"
outputs:
- createImage.ImageId
I expect it to run and create a parameter store and a document inside AWS SSM.

The code you pasted above is a Systems Manager (SSM) document rather than a CloudFormation template. That's why CF complains Template format error.
Similar to CloudFormation template, the SSM Document supports both JSON and YAML formats.
As you can see from the README description in that same GitHub repository, the JSON file is used to create a SSM Document.

Related

Azure devops: pass variable group as parameter to Template

I am using Azure devops yml pipeline at code base.
I have created Variable Group at pipeline (pipeline > Library > variable group > called 'MY_VG')
In my pipeline (yml) file i want to send this variable group MY_VG to template my_template.yml as parameter.
But this parameter MY_VG is not being expanded when i use it under 'Variable' (though while printing it gives me value)
How to access the value of this MY_VG in the template here group: ${{parameters.variable_group}} shown below?
(I am calling a template file my_template_iterator.yml which iterates the environments and call the my_template.yml)
azure-pipelines.yml
resources:
repositories:
- repository: templates
type: git
name: MY_PROJECT/GIT_REPO_FOR_TEMPLATE
stages:
- stage: "CheckOut"
displayName: Checkout
jobs:
- job: Checkout
displayName: Checkout Application
pool:
name: $(my_pool_name)
workspace:
clean: all
steps:
- checkout: self
- template: folder_name/my_template_iterator.yml#templates
parameters:
agent_pool_name: $(my_pool)
db_resource_path: $(System.DefaultWorkingDirectory)/src/main/resources/db
envs:
- env:
env_name: 'dev'
variable_group: MY_VG
pipeline_environment_name: DEV_ENV
is_master: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
## Iterator: my_template_iterator.yml
parameters:
agent_pool_name: ''
db_resource_path: ''
envs: {}
stages:
- ${{ each env in parameters.envs }}:
- template: my_template.yml
parameters:
agent_pool_name: ${{ parameters.agent_pool_name }}
db_resource_path: ${{ parameters.db_resource_path }}
env_name: ${{ env.env_name }}
variable_group: ${{ env.variable_group }}
pipeline_environment_name: ${{ env.pipeline_environment_name }}
is_master: ${{ env.is_master }}
## my_template.yml
parameters:
- name: 'variable_group'
type: string
default: 'default_variable_group'
stages:
- stage:
displayName: Read Parameters
jobs:
- job: READ
displayName: Reading Parameters
steps:
- script: |
echo variable_group: ${{parameters.variable_group}}
- stage:
displayName: Deployment
variables:
group: ${{parameters.variable_group}}
condition: ${{parameters.is_master}}
I tested with above your yaml files. It seems fine. However i found a small mistake that you missed a - when you define a variable group in my_template.yml of yours. Maybe that is the reason the variable group didnot expand for you.
variables:
# a variable group
- group: myvariablegroup
I modified your my_template.yml file a little bit. And i got it working. See below:
parameters:
- name: 'variable_group'
type: string
default: 'default_variable_group'
- name: agent_pool_name
default: ""
- name: env_name
default: ""
- name: db_resource_path
default: ""
- name: pipeline_environment_name
default: ""
- name: is_master
default: ""
stages:
- stage:
displayName: ${{parameters.env_name}}
## i changed here add '-' to group
variables:
- group: ${{parameters.variable_group}}
jobs:
- job: READ
displayName: Reading Parameters
steps:
- script: |
echo variable_group: ${{parameters.variable_group}}
- powershell: echo "$(variableName-in-variablegroup)"
Ok. I'm not sure if this doable in a way how you are trying. But I tried another approach which seems to be working.
First let's define template for simple stage:
parameters:
- name: env_name
type: string
default: 'dev'
- name: variable_group
type: string
default: 'MY_VG'
- name: pipeline_environment_name
type: string
default: 'DEV_ENV'
stages:
- stage: ${{ parameters.env_name }}
jobs:
- job: angularinstall
steps:
- script: echo "${{ parameters.env_name }}"
- script: echo "${{ parameters.variable_group }}"
- script: echo "${{ parameters.pipeline_environment_name }}"
then template for main build:
parameters:
- name: stages # the name of the parameter is stageList
type: stageList # data type is stageList
default: [] # default value of stageList
stages: ${{ parameters.stages }}
and then we can combine them together in main build file:
extends:
template: template.yaml
parameters:
stages:
- template: stageTemplate.yaml
parameters:
env_name: 'dev'
variable_group: 'MY_VG'
pipeline_environment_name: 'DEV_ENV'
- template: stageTemplate.yaml
parameters:
env_name: 'stage'
variable_group: 'MY_STAGE'
pipeline_environment_name: 'STAGE_ENV'
And for that I got:
Does it sth what bring you closer to your solution?

ssm automation document input in AWS-RunShellScript not substituting variable

I am trying to run a command in bash where part of the command is substituted from a variable that I created in a previous step, however the string substitution is not working. I have tried many variations of this with single, double quotes, etc but cant not get it to work.
mainSteps:
- name: getIps
action: 'aws:invokeLambdaFunction'
timeoutSeconds: 1200
maxAttempts: 1
onFailure: Abort
inputs:
FunctionName: Automation-GetIPs
Payload: '{"asg": "Staging_web_ASG"}'
outputs:
- Name: asg_ips
Selector: $.Payload.IPs
Type: StringList
- name: updatelsync
action: 'aws:runCommand'
timeoutSeconds: 1200
inputs:
DocumentName: AWS-RunShellScript
InstanceIds:
- '{{ InstanceID }}'
Parameters:
commands:
- 'echo {{getIps.asg_ips}} > /root/asg_ips.test'
In the above code. I set asg_ips in step1 who's OutputPayload is as follows :
{"Payload":{"IPs": ["172.xx.x.xxx", "172.xx.x.xxx"]},"StatusCode":200}
but for input in the 2nd step, it shows as follows...
{"commands":["echo {{getIps.asg_ips}} > /root/asg_ips.test"]}
I need to get it to show something like this...
{"commands":["echo ["172.xx.x.xxx", "172.xx.x.xxx"] > /root/asg_ips.test"]}
Based on the comments.
The issue was caused by incorrect use of outputs in aws:invokeLambdaFunction SSM action. Specifically, the lambda action does not have outputs attribute as shown in the linked documentation:
name: invokeMyLambdaFunction
action: aws:invokeLambdaFunction
maxAttempts: 3
timeoutSeconds: 120
onFailure: Abort
inputs:
FunctionName: MyLambdaFunction
In contrast, as a side note, outputs attribute is valid in aws:executeAwsApi.
Therefore, the solution is to directly refer to the Payload returned by the lambda action:
mainSteps:
- name: getIps
action: 'aws:invokeLambdaFunction'
timeoutSeconds: 1200
maxAttempts: 1
onFailure: Abort
inputs:
FunctionName: Automation-GetIPs
Payload: '{"asg": "Staging_web_ASG"}'
- name: updatelsync
action: 'aws:runCommand'
timeoutSeconds: 1200
inputs:
DocumentName: AWS-RunShellScript
InstanceIds:
- '{{ InstanceID }}'
Parameters:
commands:
- 'echo {{getIps.Payload}} > /root/asg_ips.test'
The side effect is that now asg_ips.test needs to be post-processed to get the IP ranges values.

IAM based ssh to EC2 instance using CloudFormation template

I am using an AWS CloudFormation template for IAM role-based access to an EC2 instance.
I getting permission denied error while running the template, and I am not able to access the EC2 machine with a username without a pem file.
Instance:
Type: 'AWS::EC2::Instance'
Metadata:
'AWS::CloudFormation::Init':
config:
files:
/opt/authorized_keys_command.sh:
content: >
#!/bin/bash -e
if [ -z "$1" ]; then
exit 1
fi
SaveUserName="$1"
SaveUserName=${SaveUserName//"+"/".plus."}
SaveUserName=${SaveUserName//"="/".equal."}
SaveUserName=${SaveUserName//","/".comma."}
SaveUserName=${SaveUserName//"#"/".at."}
aws iam list-ssh-public-keys --user-name "$SaveUserName" --query
"SSHPublicKeys[?Status == 'Active'].[SSHPublicKeyId]" --output
text | while read KeyId; do
aws iam get-ssh-public-key --user-name "$SaveUserName" --ssh-public-key-id "$KeyId" --encoding SSH --query "SSHPublicKey.SSHPublicKeyBody" --output text
done
mode: '000755'
owner: root
group: root
/opt/import_users.sh:
content: >
#!/bin/bash
aws iam list-users --query "Users[].[UserName]" --output text |
while read User; do
SaveUserName="$User"
SaveUserName=${SaveUserName//"+"/".plus."}
SaveUserName=${SaveUserName//"="/".equal."}
SaveUserName=${SaveUserName//","/".comma."}
SaveUserName=${SaveUserName//"#"/".at."}
if id -u "$SaveUserName" >/dev/null 2>&1; then
echo "$SaveUserName exists"
else
#sudo will read each file in /etc/sudoers.d, skipping file names that end in ?~? or contain a ?.? character to avoid causing problems with package manager or editor temporary/backup files.
SaveUserFileName=$(echo "$SaveUserName" | tr "." " ")
/usr/sbin/adduser "$SaveUserName"
echo "$SaveUserName ALL=(ALL) NOPASSWD:ALL" > "/etc/sudoers.d/$SaveUserFileName"
fi
done
mode: '000755' owner: root group: root
/etc/cron.d/import_users:
content: |
*/10 * * * * root /opt/import_users.sh
mode: '000644' owner: root
group: root
/etc/cfn/cfn-hup.conf:
content: !Sub |
[main]
stack=${AWS::StackId}
region=${AWS::Region}
interval=1
mode: '000400' owner: root group: root
/etc/cfn/hooks.d/cfn-auto-reloader.conf:
content: !Sub >
[cfn-auto-reloader-hook]
triggers=post.update
path=Resources.Instance.Metadata.AWS::CloudFormation::Init
action=/opt/aws/bin/cfn-init --verbose
--stack=${AWS::StackName} --region=${AWS::Region}
--resource=Instance
runas=root
commands:
a_configure_sshd_command:
command: >-
sed -i 's:#AuthorizedKeysCommand none:AuthorizedKeysCommand
/opt/authorized_keys_command.sh:g' /etc/ssh/sshd_config
b_configure_sshd_commanduser:
command: >-
sed -i 's:#AuthorizedKeysCommandUser
nobody:AuthorizedKeysCommandUser nobody:g' /etc/ssh/sshd_config
c_import_users:
command: ./import_users.sh
cwd: /opt
services:
sysvinit:
cfn-hup:
enabled: true
ensureRunning: true
files:
- /etc/cfn/cfn-hup.conf
- /etc/cfn/hooks.d/cfn-auto-reloader.conf
sshd:
enabled: true
ensureRunning: true
commands:
- a_configure_sshd_command
- b_configure_sshd_commanduser
'AWS::CloudFormation::Designer':
id: 85ddeee0-0623-4f50-8872-1872897c812f
Properties:
ImageId: !FindInMap
- RegionMap
- !Ref 'AWS::Region'
- AMI
IamInstanceProfile: !Ref InstanceProfile
InstanceType: t2.micro
UserData:
'Fn::Base64': !Sub >
#!/bin/bash -x
/opt/aws/bin/cfn-init --verbose --stack=${AWS::StackName}
--region=${AWS::Region} --resource=Instance
/opt/aws/bin/cfn-signal --exit-code=$? --stack=${AWS::StackName}
--region=${AWS::Region} --resource=Instance
This User Data script will configure a Linux instance to use password authentication.
While the password here is hard-coded, you could obtain it in other ways and set it to the appropriate value.
#!
echo 'secret-password' | passwd ec2-user --stdin
sed -i 's|[#]*PasswordAuthentication no|PasswordAuthentication yes|g' /etc/ssh/sshd_config
systemctl restart sshd.service

Userdata script not executed on AWS CloudFormation Template

I am trying to create a CloudFormation stack which has UserData script to install java, tomcat, httpd and java application on launch of an EC2 instance.
However, the stack gets created successfully with all the resources but when I connect to EC2 instance to check the configuration of above applications I don't find any. My usecase is to spin-up an instance with all the above applications/software to be installed with automation.
UserData:
Fn::Base64:
Fn::Join:
- ' '
- - '#!/bin/bash -xe\n'
- 'sudo yum update && install pip && pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n'
- 'date > /home/ec2-user/starttime\n'
- 'sudo yum update -y aws-cfn-bootstrap\n'
# Initialize CloudFormation bits\n
- ' '
- '/opt/aws/bin/cfn-init -v\n'
- ' --stack\n'
- '!Ref AWS::StackName\n'
- ' --resource LaunchConfig\n'
- 'ACCESS_KEY=${HostKeys}&SECRET_KEY=${HostKeys.SecretAccessKey}\n'
# Start servers\n
- 'service tomcat8 start\n'
- '/etc/init.d/httpd start\n'
- 'date > /home/ec2-user/stoptime\n'
Metadata:
AWS::CloudFormation::Init:
config:
packages:
yum:
- java-1.8.0-openjdk.x86_64: []
- tomcat8: []
- httpd: []
services:
sysvinit:
httpd:
enabled: 'true'
ensureRunning: 'true'
files:
- /usr/share/tomcat8/webapps/sample.war:
- source: https://s3-eu-west-1.amazonaws.com/testbucket/sample.war
- mode: 000500
- owner: tomcat
- group: tomcat
CfnUser:
Type: AWS::IAM::User
Properties:
Path: '/'
Policies:
- PolicyName: Admin
PolicyDocument:
Statement:
- Effect: Allow
Action: '*'
Resource: '*'
HostKeys:
Type: AWS::IAM::AccessKey
Properties:
UserName: !Ref CfnUser
The problem is in the way you have formatted your UserData. I would suggest that you launch the EC2 instance and manually test the script first. It has a number of problems in it.
Try formatting your UserData like this:
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -xe
# FIXME. This won't work either.
# sudo yum update && install pip && pip install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz
date > /home/ec2-user/starttime
sudo yum update -y aws-cfn-bootstrap
# Initialize CloudFormation bits
/opt/aws/bin/cfn-init -v \
--stack ${AWS::StackName} \
--resource LaunchConfig
# FIXME. Not sure why these are here.
# ACCESS_KEY=${HostKeys}
# SECRET_KEY=${HostKeys.SecretAccessKey}
# Start servers\n
service tomcat8 start
/etc/init.d/httpd start
date > /home/ec2-user/stoptime
Things to note:
You can't interpolate here using !Ref notation. Notice I changed it to ${AWS::StackName} and notice the whole block is inside !Sub.
As my comments indicate, the yum update line has invalid commands in it.
As noted in the comments, it is a bad practice to inject access keys. Also, the keys don't seem to be required for anything in this script.
Note also that the files section is specified incorrectly in the MetaData, as Arrays instead of Hash keys.
It should be:
files:
/usr/share/tomcat8/webapps/sample.war:
source: https://s3-eu-west-1.amazonaws.com/testbucket/sample.war
mode: '000500'
owner: tomcat
group: tomcat

Can't write the content of a file in UserData AWS EC2 cloudformation

I want to get the content of a file in the UserData without providing inline content as following, but I'm getting the path of the file in the content when the ec2 boots instead of the content of the file.
Here's a snippet of my template:
ServiceInstance:
Type: "AWS::EC2::Instance"
Properties:
. . .
UserData:
'Fn::Base64': !Sub |
#cloud-config
write_files:
- path: /etc/sysconfig/cloudformation
permissions: 0644
owner: root
content: |
STACK_NAME=${AWS::StackName}
AWS_REGION=${AWS::Region}
- path: /etc/path-to-file/conf.yaml
permissions: 0644
owner: root
content: "#file://./config/conf-${Env}.yaml"
runcmd:
## run some commands
when I ssh to ec2 and check the file content I get this:
[ec2-user#ec2ip ~]$ cat /etc/path-to-file/conf.yaml
#file://./config/conf-dev.yaml
I checked this cloud init docs, but can't find something related.
any idea what did I do wrong in here ?
Encode the file content in base64 and pass it as argument. Cloud Init will decode the string b64.
Be careful of the cloudformation size limits on template and variables.
Parameters:
ConfContent:
Type: String
Description: "Conf content in base64 format."
Resources:
ServiceInstance:
Type: "AWS::EC2::Instance"
Properties:
. . .
UserData:
'Fn::Base64': !Sub
- |
#cloud-config
write_files:
- path: /etc/sysconfig/cloudformation
permissions: 0644
owner: root
content: |
STACK_NAME=${AWS::StackName}
AWS_REGION=${AWS::Region}
- path: /etc/path-to-file/conf.yaml
permissions: 0644
owner: root
content: ${CONF_CONTENT}
encoding: b64
runcmd:
## run some commands
- CONF_CONTENT: !Ref ConfContent
Then expose the file content as property :
aws cloudformation create-stack \
--stack-name "mystack" \
--template-body "template.yaml" \
--parameters \
ParameterKey=ConfContent,ParameterValue=\"$(base64 -w0 ./config/conf-dev.yaml)\"