Terraform pre-install applications - amazon-web-services

new to Terraform, trying somethings to get to know terraform a bit better.
i'm creating my infrastructure via AWS using an EC2 instance.
i managed to create the instance with SG and everything, but i came across some difficulties installing apps (such as docker and so on).
i was wondering if there's a way i can tell the terraform file to pre-install docker, is there any way?
i found some similar issues about the matter here:
Terraform plugins
but i can't figure out if it answers my question fully.
can anyone please advise?

Usually for EC2 instance you would define user_data. User data allows you to:
perform common automated configuration tasks and even run scripts after the instance starts.
Which means that you can write your user data to install any applications you want on the instance, and configure them how you wish. Alternatively, you can create custom AMI and lunch instance using that.

There are many approaches to this, of the somewhat more common and simple you can either:
(1) use a user_data script that will bootstrap the EC2 instance for you
A sample user_data script might look like below. Borrowed from github/gonzaloplaza.
#!/bin/bash
# Install docker
apt-get update
apt-get install -y cloud-utils apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
apt-get update
apt-get install -y docker-ce
usermod -aG docker ubuntu
# Install docker-compose
curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
And then embed it in your Terraform EC2 instance definition:
resource "aws_instance" "my-instance" {
ami = "ami-05210ba6bdf75db36" # Ubuntu 20.04LTS eu-central-1
instance_type = "m5.large"
key_name = "<key-path>"
user_data = "${file("install_docker.sh")}"
tags = {
Name = "Terraform"
}
}
(2) or use an AMI (virtual image) that has the requirement already met. AWS Marketplace enables you to use AMIs that other users built. You can check out for example Docker on Ubuntu AWS Marketplace link.
(3) And a more complex approach would be to build your own AMIs with for example Puppet and Packer. You can then upload those AMIs to your AWS Account and use with the instances you create.
References
Ubuntu AMI Locator for the Ubuntu 20.04LTS eu-central-1 AMI
github/gonzaloplaza for the userscript example

Related

Why is the aws cli not found on amazonlinux2 ami?

The AmazonLinux2 AMI I am using for my Docker hosts does not appear to have the AWS CLI installed. The log has an error from the user data script that tries to run an aws command:
/var/lib/cloud/instance/scripts/part-001: line 7: aws: command not found
Then I connected with SSH to the instance for a sanity check, and aws is definitely not found:
[ec2-user#ip-X-X-X-X ~]$ cat /etc/os-release
NAME="Amazon Linux"
VERSION="2"
ID="amzn"
ID_LIKE="centos rhel fedora"
VERSION_ID="2"
PRETTY_NAME="Amazon Linux 2"
ANSI_COLOR="0;33"
CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
HOME_URL="https://amazonlinux.com/"
[ec2-user#ip-X-X-X-X ~]$ aws --version
-bash: aws: command not found
I thought the AWS CLI was installed by default on all AmazonLinux AMIs? I don't remember ever having to install it myself before.
This article even says that the CLI v1 is installed by default on AmazonLinux2:
AWS Docs Link
So how is it possible that it's not found on this stock AMI? Do only some of the AmazonLinux2 AMIs have the CLI pre-installed? For reference, I am using this AMI:
amzn2-ami-minimal-hvm-2.0.20200917.0-x86_64-ebs (ami-0a6993b2978bd23cb)
From this post on AWS forum:
Minimal has a smaller set of packages installed by default. For example, a lot of AWS specific packages are installed on the default for easy integration to other AWS services. The minimal do not have these installed. This gives a much lower footprint for those who are not directly interacting with other AWS services, or who want to cherry-pick which ones they install.
If you want awscli, you can install it:
sudo yum install -y awscli
to install the latest version of the awscli (v2) see this doc
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

How do I get my EC2 Instance to connect to ECS Cluster?

I have an ECS cluster defined in AWS and an Auto Scaling Group that I use to add/remove instance to handle tasks as necessary. I have the ASG setup so that it is creating the EC2 instance at the appropriate time, but it won't connect to the ECS Cluster unless I manually go in and disable/enable the ECS service.
I am using the Amazon Linux 2 ami on the EC2 machines and everything is in the same region/account etc.
I have included my user data below.
#!/bin/bash
yum update -y
amazon-linux-extras disable docker
amazon-linux-extras install -y ecs
echo "ECS_CLUSTER={CLUSTERNAME}" >> /etc/ecs/ecs.config
systemctl enable --now ecs
As mentioned this installs the ECS service and sets the config file properly but the enable doesn't actually connect the machine, but running the same disable/enable commands on the machine once running connects without problem. What am I missing?
First thing, the correct syntax is
#!/usr/bin/env bash
echo "ECS_CLUSTER=CLUSTER_NAMe" >> /etc/ecs/ecs.config
Once you update the config better to restart the ECS agent.
#!/usr/bin/env bash
echo "ECS_CLUSTER=CLUSTER_NAME" >> /etc/ecs/ecs.config
sudo yum update -y ecs-init
#this will update ECS agent, better when using custom AMI
/usr/bin/docker pull amazon/amazon-ecs-agent:latest
#Restart docker and ECS agent
sudo service docker restart
sudo start ecs
I ended up solving this using the old adage, turn it off and on again.
e.g. I added shutdown -r 0 to the bottom of the user data script to restart the machine after it was "configured" and it connected right now.

How to run codedeploy agent installation script in AWS ECS?

I have an AWS ECS cluster defined with a service that uses Replica service type. It creates an EC2 isntance with a docker container. I can access it through browser and all this stuff...
The issue is that I have to connect through ssh to the EC2 instance and run:
sudo yum update -y
sudo yum install-y ruby
sudo yum install-y wget
cd /home/ec2-user
wget https://aws-codedeploy-eu-west-1.s3.eu-west-1.amazonaws.com/latest/install
chmod +x ./install
sudo ./install auto
It install codedeploy agent, so I can connect github to the instance and CI/CD code.
I would like to set up this automatically in every server that the ECS definition creates. For example if i stop the EC2 instance, the cluster raises a new EC2 instance, which doesn't have this agent...
I saw that I should configure your Amazon ECS container instance with user data, but first of all is that I am not able to find this option, and I am not quite sure if it runs into the EC2 isntance or in the docker itself.
Based on the comments.
The solution was to use Launch Template or Launch Configurations.

Unable to install Webgoat on AWS. I get error about Dockerfile and Dockerrun.aws.json

I am trying to install webgoat on AWS. I am following the instructions given on https://github.com/WebGoat/WebGoat
I can get it up and running on my local box. But when I try to deploy it on AWS it gives error and complains about Dockerfile and Dockerrun.aws.json.
I go to elastic beanstalk. Then I create an application (of docker type). It asks me for the code and I give it the zip file from github. After several minutes it gives errors about Dockerfile and Dockerrun.aws.json.
Webgoat has several Dockerfiles, but no Dockerrun.aws.json. I am not sure how to resolve this.
What is the best way to deploy webgoat in aws?
Will appreciate any help I can get.
Finally I was able to install it using the info provided on these two sources.
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/docker-basics.html
and https://github.com/WebGoat/WebGoat
Here are the steps:
sudo yum update -y
sudo yum install -y docker
sudo service docker start
sudo usermod -a -G docker ec2-user (Restart the server)
sudo docker pull webgoat/webgoat-8.0
sudo docker run -p 80:8080 -it webgoat/webgoat-8.0 /home/webgoat/start.sh
Make sure to modify the security group associated with the aws instance to allow http traffic. After that you should be able to access the app with this url:
http://:80/WebGoat/login

AWS code deploy agent not able to install?

Hi i am trying to install code deploy agent in my ec2 agent but not able to succeed
I m following below steps
sudo apt-get update
sudo apt-get install awscli
sudo apt-get install ruby2.0
cd /home/ubuntu
sudo aws s3 cp s3://bucket-name/latest/install . --region region-name
sudo chmod +x ./install
sudo ./install auto
but ./install file is missing for me .
But I dont think its a problem with AMI as I used same steps with same AMI in different ec2 instance. Any one has any idea. please help me.
You need to fill in the bucket name and region name in sudo aws s3 cp s3://bucket-name/latest/install . --region region-name. If you are in us-east-1 you would use: aws-codedeploy-us-east-1 and us-east-1.
All the buckets follow that pattern so you can fill in another region if you are there instead.
See http://docs.aws.amazon.com/codedeploy/latest/userguide/how-to-set-up-new-instance.html for a complete list of buckets for each region.