I'm using AWS::CloudFormation::Init and have a lengthy list of commands.
In my commands I'm using many intrinsic functions to drop in variables. For example:
commands:
'01':
command: !Sub 'echo ${AWS::StackName} > ~/cfn-stack.txt'
Once my stack is deployed, and the EC2 instance is created, is there a place on the instance where I can view the raw commands that were run with the resolved variables dropped in?
Related
I have created a template where I created EC2 instance and I used cfn-init to process the configsets, and in the Userdata section of the instance, I wrote some commands to be executed by cloud-init and some commands to be executed without cloud-init.
I am not sure which commands runs in which sequence?
What I mean is, in which order the commands are executed? For example:
Commands in the configsets
Commands in the cloud-init section of the userdata
Commands in the Userdata
Part of my code is below:
UserData:
Fn::If:
- DatadogAgentEnabled
-
Fn::Base64: !Sub |
#!/bin/bash -xe
yum update -y
yum update -y aws-cfn-bootstrap
/opt/aws/bin/cfn-init --stack ${AWS::StackName} --resource GhostServer --configsets prepare_instance_with_datadog --region ${AWS::Region}
/opt/aws/bin/cfn-signal -e $? --stack ${AWS::StackName} --resource GhostServer --region ${AWS::Region}
#cloud-config <----cloud-init section inside the Userdata
runcmd:
- [ sh, -c, "sed 's/api_key:.*/api_key: {DatadogAPIKey}/' /etc/datadog-agent/datadog.yaml.example > /etc/datadog-agent/datadog.yaml" ]
- systemctl restart datadog-agent
From the AWS documentation
UserData :
The user data to make available to the instance. For more
information, see Running commands on your Linux instance at launch
(Linux) and Adding User Data (Windows). If you are using a command
line tool, base64-encoding is performed for you, and you can load the
text from a file. Otherwise, you must provide base64-encoded text.
User data is limited to 16 KB.
So Basically we define UserData to execute some commands at the launch of our EC2 instance.
Reference : https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html
To Answer your question first of all UserData commands will be executed and all the commands specified in this section will execute sequentially so in your example you invoked cfn helper scripts first and then cloud init thus configsets will be applied first and then cloud init commands will be called.
Inside that UserData section we are invoking cfn helper scripts which reads cloudformation template metadata and executes all the configsets defined under AWS::CloudFormation::Init:
From the AWS Documentation :
The cfn-init helper script reads template metadata from the
AWS::CloudFormation::Init key and acts accordingly to:
Fetch and parse metadata from AWS CloudFormation
Install packages
Write files to disk
Enable/disable and start/stop services
Reference : https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-init.html
$ terraform version
Terraform v0.14.4
I'm using Terraform to create an AWS autoscaling group, and it successfully launches an EC2 via a launch template, also created by the same Terraform plan. I added the following user_data definition in the launch template. The AMI I'm using already has Docker configured, and has the Docker image that I need.
user_data = filebase64("${path.module/docker_run.sh}")
and the docker_run.sh file contains simple
docker run -p 80:3000 -d 1234567890.dkr.ecr.us-east-1.amazonaws.com/node-app:latest
However, when I ssh to the EC2 instance, the container is NOT running. What am I missing?
Update:
Per Marcin's comment, I see the following in in /var/log/cloud-init-output.log
Jan 11 22:11:45 cloud-init[3871]: __init__.py[WARNING]: Unhandled non-multipart (text/x-not-multipart) userdata: 'docker run -p 80:3000 -d...'
From AWS docs and what you've posted the likely reason is that you are missing /bin/bash in your docker_run.sh:
User data shell scripts must start with the #! characters and the path to the interpreter you want to read the script (commonly /bin/bash).
Thus your docker_run.sh should be:
#!/bin/bash
docker run -p 80:3000 -d 1234567890.dkr.ecr.us-east-1.amazonaws.com/node-app:latest
If this still fails, please check /var/log/cloud-init-output.log on the instance for errors.
Using Elastic beanstalk autoscaling for an application, recently added a new SQS queue as an environment variable.
I can see the variable using eb printenv *my-env* | grep "SQS":
SQS_QUEUE = https://sqs.xxxxxx.amazonaws.com/xxxxxxx/xxxxxxx
When I SSH to a newly created instance I can't see that config sudo /opt/elasticbeanstalk/bin/get-config environment --output YAML | grep "SQS"
If I make a code change and deploy then the instance gets the config - this isn't a practical solution when the scaling goes up / down multiple times a day.
I'm using a custom AMI but I can't find any documentation on AWS around environment variables - adding new ones should just work..!?
What am I missing??
Please double check. I think -output YAML is incorrect. The queue should be accessible through:
/opt/elasticbeanstalk/bin/get-config environment -key SQS_QUEUE
I have a python application that's using Flask, which I'm trying to deploy to Elasticbeanstalk using the EB CLI.
I'm following the deployment steps mentioned here:
https://docs.amazonaws.cn/en_us/elasticbeanstalk/latest/dg/create-deploy-python-flask.html#python-flask-deploy
I ran the following commands
eb init -p python-3.6 demo-v1 --region us-west-2
eb init
eb create demo-env -it t3a.large
I got the following error
ERROR: ServiceError - Create environment operation is complete, but with errors. For more information, see troubleshooting documentation.
I checked the eb-activity.log and found that there was a MemoryError, which occurred because there wasn't enough space to install all the packages (as specified in the requirements.txt).
I then checked the instance type and it was showing the instance type as t2.micro.
I had to manually go in to the ELB console configuration -> capacity -> changed the instance type from t2.micro to t3a.large
Seems like the CLI ignored the instance type flag and it didn't throw an error or a warning.
How do I specify the instance type for my application?
I believe the option you'd want to use is -i or --instance-type. The -it option is for instance types which expects a list of instance types rather than just one. Most likely what happened was that the ebcli did not know what to make of only one instance type when it expected something like eb create demov2 -it "t2.micro, t3.micro, t3a.large"
I'm posting a "workaround" since this question is still active.
I created a file called 01instance_setting.config. I added this to my .ebextensions folder. This is the content for the 01instance_setting.config
option_settings:
aws:autoscaling:asg:
MinSize: 1
MaxSize: 3
aws:autoscaling:launchconfiguration:
RootVolumeType: gp2
RootVolumeSize: "32"
EC2KeyName: my-key-pair
MonitoringInterval: "2 minute"
aws:ec2:instances:
InstanceTypes: t3a.large
I think this is a better way rather than specifying the instance while creating the environment from the CLI. Its much more flexible if you're creating multiple environments.
I understand the AWS::CloudFormation::Init block of the template is executed by cfn-init script.
Here is the syntax I use to do so:
/usr/local/bin/cfn-init -v --stack WebS<erver1 --resource WebServerInstance --region eu-central-1 --configset InstallAndRun && cat /var/log/cfn-init.log
I can do the call via UserData or via SSH.
Now I would like to have the possibility to modify the AWS::CloudFormation::Init part of the template and repeat the cnf-init call via ssh.
Question:
Where is the file with the cloud formation code located on my instance, which is executed by cfn-init?
The cfn-init command uses a configuration file located at /var/lib/cfn-init/data/metadata.json. The JSON file contains the config sets defined in the AWS::CloudFormation::Init block of a Cloud Formation template.
I've confirmed the path above on Amazon Linux 2. The location on other operating systems may be different.
All of the cloud init files are in /var/lib/cloud (on Centos 7) and the userdata script is in /var/lib/cloud/instance
See my other answer
cloud-init: What is the execution order of cloud-config directives?
for general hints on where the source to cloud-init is, execution ordering