In aws cloud formation I know you can update the stack by updating the json file and those changes will take affect but how could I just update the stacks packages for example yum update or apt update etc ?
Thanks in advance
Here is the sample for you on how to handle your problem.
Update the code in Cloudformation template in userdata.
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"yum update -y \n",
"# Install the files and packages from the metadata\n",
"/opt/aws/bin/cfn-init -v ",
" --stack ", { "Ref" : "AWS::StackName" },
" --resource WebServerInstance ",
" --configsets InstallAndRun ",
" --region ", { "Ref" : "AWS::Region" }, "\n"
]]}}
If you need know cfn-init, read this url cfn-init
If you need a sample template, see here: Deploying Applications on Amazon EC2 with AWS CloudFormation
Related
In the CloudFormation template I am deploying, I am running a long running operation in the UserData block.
It looks as follows:
"UserData": {
"Fn::Base64" : {
"Fn::Join" : [
"",
[
"/usr/local/bin/mylongrunningscript.sh"
]
]
}
}
The contents of the script are:
echo "UserData starting!" > /var/log/mycustomlog.log
sleep 300
echo "UserData finished!" >> /var/log/mycustomlog.log
The issue I am seeing is that I believe the CloudFormation template is completing it's deployment before the UserData script finishes running. I believe this is the case because if I am quick enough and ssh into the instance, I will see something as follows:
$ cat /var/log/mycustomlog.log
UserData starting
which suggests that the UserData didn't finish running
How can I make sure that the UserData code execution is completed before the stack is in the "CreateComplete" status?
To ensure the CloudFormation template waits for the completion of the UserData script, you must do two things:
Add a CreationPolicy to the resource you are targeting (virtual machine in my case).
Add logic in the script to signal its completion. This custom logic uses the cfn-signal utility, which you might have to install in your instance.
Here's how the template looks now:
"UserData": {
"Fn::Base64" : {
"Fn::Join" : [
"",
[
"/usr/local/bin/mylongrunningscript.sh"
]
]
}
},
"CreationPolicy": {
"ResourceSignal" : {
"Count": "1",
"Timeout": "PT10M"
}
}
The cfn-signal utility is used to signal the termination of the script:
"/home/ubuntu/aws-cfn-bootstrap-*/bin/cfn-signal -e $? ",
" --stack ", { "Ref": "AWS::StackName" },
" --resource MyInstance" ,
" --region ", { "Ref" : "AWS::Region" }, "\n"
See here for a Windows example.
I have a cloudformation template, to deploy a windows server and run some powershell commands. I can get the server to deploy, but none of my powershell commands seem to run. They are getting passed over.
I have been focusing on cnit to install my apps, no luck
{
"AWSTemplateFormatVersion":"2010-09-09",
"Description":"CHOCO",
"Resources":{
"MyEC2Instance1":{
"Type":"AWS::EC2::Instance",
"Metadata" : {
"AWS::CloudFormation::Init": {
"configSet" : {
"config" : [
"extract",
"prereq",
"install"
]
},
"extract" : {
"command" : "powershell.exe -Command Set-ExecutionPolicy -
Force remotesigned"
},
"prereq" : {
"command" : "powershell.exe -Command Invoke-WebRequest -
Uri https://xxxxx.s3.us-east-2.amazonaws.com/chocoserverinstall.ps1 -
OutFile C:chocoserverinstall.ps1"
},
"install" : {
"command" : "powershell.exe -File chocoserverinstall.ps1"
}
}
},
"Properties":{
"AvailabilityZone":"us-east-1a",
"DisableApiTermination":false,
"ImageId":"ami-06bee8e1000e44ca4",
"InstanceType":"t3.medium",
"KeyName":"xxx",
"SecurityGroupIds":[
"sg-01d044cb1e6566ef0"
],
"SubnetId":"subnet-36c3a56b",
"Tags":[
{
"Key":"Name",
"Value":"CHOCOSERVER"
},
{
"Key":"Function",
"Value":"CRISPAPPSREPO"
}
],
"UserData":{
"Fn::Base64":{
"Fn::Join":[
"",
[
"<script>\n",
"cfn-init.exe -v ",
" --stack RDSstack",
" --configsets config ",
" --region us-east-1",
"\n",
"<script>"
]]}
}
}
}
}
}
I'm excepting cloudformation to run thru my metadata commands when provisioning this template
The cfn-init command requires the -c or --configsets command to specify "a comma-separated list of configsets to run (in order)".
See:
cfn-init - AWS CloudFormation
AWS::CloudFormation::Init - AWS CloudFormation
I am trying to learn Cloudformation im stuck with a senario where I need a second EC2 instance started after one EC2 is provisioned and good to go.
This is what i have in UserData of Instance one
"#!/bin/bash\n",
"#############################################################################################\n",
"sudo add-apt-repository ppa:fkrull/deadsnakes\n",
"sudo apt-get update\n",
"curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash -\n",
"sudo apt-get install build-essential libssl-dev python2.7 python-setuptools -y\n",
"#############################################################################################\n",
"Install Easy Install",
"#############################################################################################\n",
"easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz\n",
"#############################################################################################\n",
"#############################################################################################\n",
"GIT LFS Repo",
"#############################################################################################\n",
"curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash\n",
"#############################################################################################\n",
"cfn-init",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource UI",
" --configsets InstallAndRun ",
" --region ",
{
"Ref": "AWS::Region"
},
"\n",
"#############################################################################################\n",
"# Signal the status from cfn-init\n",
"cfn-signal -e 0 ",
" --stack ",
{
"Ref": "AWS::StackName"
},
" --resource UI",
" --region ",
{
"Ref": "AWS::Region"
},
" ",
{
"Ref": "WaitHandleUIConfig"
},
"\n"
I have a WaitCondition , which i think is whats used to do this
"WaitHandleUIConfig" : {
"Type" : "AWS::CloudFormation::WaitConditionHandle",
"Properties" : {}
},
"WaitConditionUIConfig" : {
"Type" : "AWS::CloudFormation::WaitCondition",
"DependsOn" : "UI",
"Properties" : {
"Handle" : { "Ref" : "WaitHandleUIConfig" },
"Timeout" : "500"
}
}
In the Instance i use the DependsOn in the second instance to wait for first instance.
"Service": {
"Type": "AWS::EC2::Instance",
"Properties": {
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "1ba546d0-2bad-4b68-af47-6e35159290ca"
},
},
"DependsOn":"WaitConditionUIConfig"
}
this isnt working. I keep getting the error
WaitCondition timed out. Received 0 conditions when expecting 1
Any help would be appreciated.
Thanks
Put quotes around the Handle
Change this
" ",
{
"Ref": "WaitHandleUIConfig"
},
"\n"
to this
" \"",
{
"Ref": "WaitHandleUIConfig"
},
"\"\n"
Remove --stack, --resource and --region from your cfn-signal command. These are only used when 'resource signaling', not when signaling using a Wait Condition Handle. (You might also need to add an --id option, but the documentation says this is not required.)
For further debugging, examine the /var/log/cloud-init-output.log file on the EC2 instance to view any further cloud-init errors that might fail to successfully send the signal to your wait condition.
You might also want to comment and newline the descriptions "Install Easy Install", and "GIT LFS Repo",, e.g., "# Install Easy Install\n",, these syntax issues shouldn't cause your script to fail but will output 'command not found' errors to appear in your log.
I am starting a Amazon Linux instance (ami-fb8e9292) using the web console, pasting data into the user data box to run a script upon startup. If I use the example given by amazon to start a web server, it works. But when I run my own script (also a #!/bin/bash script), it does not get run.
If I look in var/log/cloud-init.log, it gives no useful information on the topic:
May 22 21:06:12 cloud-init[1286]: util.py[DEBUG]: Running command ['/var/lib/cloud/instance/scripts/part-001'] with allowed return codes [0] (shell=True, capture=False)
May 22 21:06:16 cloud-init[1286]: util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/part-001 [2]
May 22 21:06:16 cloud-init[1286]: util.py[DEBUG]: Failed running /var/lib/cloud/instance/scripts/part-001 [2]
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cloudinit/util.py", line 637, in runparts
subp([exe_path], capture=False, shell=True)
File "/usr/lib/python2.6/site-packages/cloudinit/util.py", line 1528, in subp
cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['/var/lib/cloud/instance/scripts/part-001']
Exit code: 2
Reason: -
Stdout: ''
Stderr: ''
If I ssh into the instance and sudo su and execute the shell script directly:
/var/lib/cloud/instance/scripts/part-001
then it runs fine. Also, it works if I emulate the way cloud-init runs it:
python
>>> import cloudinit.util
>>> cloudinit.util.runparts("/var/lib/cloud/instance/scripts/")
Using either of those methods, if I intentionally introduce errors into the script then it produces error messages. How can I debug the selective absence of useful debugging output?
Instead of /var/log/cloud-init.log consider searcing for keywords like "Failed", "ERROR" "WARNING" or "/var/lib/cloud/instance/scripts/" inside /var/log/cloud-init-output.log - which in most cases, contains very clear error messages.
For example - running a bad command will produce the following error in /var/log/cloud-init-output.log:
/var/lib/cloud/instance/scripts/part-001: line 10: vncpasswd: command not found
cp: cannot stat '/lib/systemd/system/vncserver#.service': No such file or directory
sed: can't read /etc/systemd/system/vncserver#.service: No such file or directory
Failed to execute operation: No such file or directory
Failed to start vncserver#:1.service: Unit not found.
Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
Cleaning repos: amzn2-core amzn2extra-docker amzn2extra-epel
And at the end of /var/log/cloud-init.log you'll receive a quiet general error message:
Aug 31 15:14:00 cloud-init[3532]: util.py[DEBUG]: Failed running /var/lib/cloud/instance/scripts/part-001 [1]
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 910, in runparts
subp(prefix + [exe_path], capture=False, shell=True)
File "/usr/lib/python2.7/site-packages/cloudinit/util.py", line 2105, in subp
cmd=args)
ProcessExecutionError: Unexpected error while running command.
Command: ['/var/lib/cloud/instance/scripts/part-001']
Exit code: 1
Reason: -
Stdout: -
Stderr: -
cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
(*) Try to grep just the relevant error message with:
grep -C 10 '<search-keyword>' cloud-init-output.log
I'm not sure if this is going to be the case for everyone, but I was having this issue and was able to fix it by changing my first line from this:
#!/bin/bash -e -v
to just this:
#!/bin/bash
Of course, now my script is failing and I have no idea how far it's getting, but at least I got past it not running it at. :)
Hope it will reduce the debugging time for someone.
I didn't have any explicit error messages in my /var/log/cloud-init-output.log, just this:
2021-04-07 10:36:57,748 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
2021-04-07 10:36:57,748 - util.py[WARNING]: Running module scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_user.py'>) failed
After some investigation, I've realized that the cause was a typo in the shebang string: #!?bin/bash instead of #!/bin/bash.
I had a similar issue and I was able to get around it. I realized that the environment variables EC2_HOME would not be setup for the sudo. I was doing a bunch of stuff in my configset which uses aws cli and for these to work, the EC2_HOME needs to be setup. So, I went in and removed sudo everywhere in my configset and UserData.
Earlier when I was hitting the issue, my UserData looked like:
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash\n",
"sudo yum update -y aws-cfn-bootstrap\n",
"# Install the files and packages and run the commands from the metadata\n",
"sudo /opt/aws/bin/cfn-init -v --access-key ", { "Ref" : "IAMUserAccessKey" }, " --secret-key ", { "Ref" : "SecretAccessKey" },
" --stack ", { "Ref" : "AWS::StackName" },
" --resource NAT2 ",
" --configsets config ",
" --region ", { "Ref" : "AWS::Region" }, "\n"
]]}}
My UserData after the changes looked like:
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"#!/bin/bash -xe\n",
"yum update -y aws-cfn-bootstrap\n",
"# Install the files and packages and run the commands from the metadata\n",
"/opt/aws/bin/cfn-init -v --access-key ", { "Ref" : "IAMUserAccessKey" }, " --secret-key ", { "Ref" : "SecretAccessKey" },
" --stack ", { "Ref" : "AWS::StackName" },
" --resource NAT2 ",
" --configsets config ",
" --region ", { "Ref" : "AWS::Region" }, "\n"
]]}}
Similarly, I removed all the sudo calls I was doing in my configsets
In my case cloudinit could not start script because userdata must start with
#!bin/bash
without empty spaces in front of it!
Nice AWS bug, lot of time for troubleshooting :)
I've been through this, and in my case it was also an issue with spaces before the she-bang #!bin/bash.
I spun up an instance through python code, using boto3.
ec2 = boto3.resource('ec2', region_name='eu-south-1')
instance = ec2.create_instances(
image=AMI_IMAGE_ID,
InstanceType=INSTANCE_TYPE,
...
UserData=USER_DATA_SCRIPT
...
)
where the definition of USER_DATA_SCRIPT was:
USER_DATA_SCRIPT = """
#!/bin/bash
apt update -y
apt upgrade -y
...
"""
This contained spaces up front, and this caused the script to generate the error without further details in /var/log/cloud-init-output.log.
Changing it into:
USER_DATA_SCRIPT = """#!/bin/bash
apt update -y
apt upgrade -y
...
"""
solved the issue.
I am trying to combine the Cloud formation template multi-tier-web-app-in-vpc.template with the cloudformation template used by viusal studio to create Load Balanced instances. The goal is to create 2 application servers within a private subnet of a VPC. The template works fine but when I start plugging in windows instances they just fail.
Error Message
CREATE_FAILED WaitCondition timed out. Received 0 conditions when expecting 1
The following resource(s) failed to create: [FrontendWaitCondition]. . Rollback requested by user.
Template used to create the cloud formation
https://s3.amazonaws.com/hg-optimise/Windows-Multi-Tier.template
I am trying to use the following Amazon templates as guides.
Amazon Visual Studio Template
https://s3.amazonaws.com/hg-optimise/Visual-Studio.template
Amazon Multi-tier Web Sample - http://aws.amazon.com/cloudformation/aws-cloudformation-templates/
https://s3.amazonaws.com/cloudformation-templates-us-east-1/multi-tier-web-app-in-vpc.template
It looks like you are taking on too much, trying to get everything working all at once. I would try to take it one step at a time. Create a template that gets one instance up, then add auto scaling, then load balancer, then subnet, routing, etc. The problem that presents itself now is likely because you have not signaled success for the wait condition.
Below is the Properties section of an Instance resource. This snipet was taken from an AWS documentation page. Note that the "UserData" section has a call to cfn-init.exe in order to perform the actions specified in the Instance's Cloud Formation section, and has a call to cfn-signal.exe to signal to the WaitCondition that the instance is up.
"Properties": {
"InstanceType" : { "Ref" : "InstanceType" },
"ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" },
{ "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] },
"SecurityGroups" : [ {"Ref" : "SharePointFoundationSecurityGroup"} ],
"KeyName" : { "Ref" : "KeyPairName" },
"UserData" : { "Fn::Base64" : { "Fn::Join" : ["", [
"<script>\n",
"cfn-init.exe -v -s ", { "Ref" : "AWS::StackName" },
" -r SharePointFoundation",
" --region ", { "Ref" : "AWS::Region" }, "\n",
"cfn-signal.exe -e %ERRORLEVEL% ", { "Fn::Base64" : { "Ref" : "SharePointFoundationWaitHandle" }}, "\n",
"</script>"
]]}}
}
You have set the front end wait condition to basically wait until your FrontendFleet is up and running.
You should set a desired capacity for your front end fleet.
When you get this error, what is the state of your autoscaling group FrontendFleet? If this is still bringing up instances, then your timeout is simply to short.
I honestly wouldn't worry about the waitcondtions unless you really need them.