I have a Terraform script for make a deploy of Ubuntu.
resource "aws_instance" "runner" {
instance_type = "${var.instance_type}"
ami = "${var.ami}"
user_data = "${data.template_file.deploy.rendered}"
}
data "template_file" "deploy" {
template = "${file("cloudinit.tpl")}"
}
My cloudinit.tpl:
#cloud-config
runcmd:
- apt-get update
- sleep 30
- apt-get install -y awscli
I can't find any issue on cloud-init.log and can't find user-data.log file in /var/log to understand why user-data is not working.
Cloud-init has a special command for system update which carry on about consistency
#cloud-config
package_update: true
package_upgrade: true
packages: ['awscli']
runcmd:
- aws --version
Than you may see command output in the log file, for Ubuntu it is /var/log/cloud-init-output.log
Related
Description
Looking at this AWS EC2 Doc It should be possible to add exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1 to a user-data script that is run on EC2 initialization.
When running aws ec2 --region eu-west-1 get-console-output --instance-id i-<id> | grep "user-data" (or searching for other patterns that should be present) none are found after the ec2 initialization.
Goal
To read the results and debug information from this initialization script without needing to SSH into the ec2 instance and poll the logs for the shutdown statement. Using the Instance shutdown as the "finished" state has a significant simplification on the deployment process for this repository.
Question
What about this particular setup is not correct such that we are not getting logs out of the aws ec2 get-console-output command.
Alternative answer: What's a better method of retrieving the logs from an EC2 instance.
User-Data Script
#!/bin/bash -xe
# redirect output to log file and console
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
if [ -n "postgresql,jq" ]
then
yum -y -q update
echo "complete: yum update"
IFS=', ' read -r -a REQS <<< "postgresql,jq"
for REQ in "${REQS[#]}"
do
yum -y -q install $REQ
echo "complete: yum install $REQ"
done
fi
echo "EC2P: ENTERING BOOTSTRAP SCRIPT....."
export PGPASSWORD=$( aws secretsmanager get-secret-value --secret-id <secret_arn> --query SecretString --region eu-west-1 | jq -r . | jq -r .password)
aws s3 cp s3://<query_object> /tmp/postgres-query.sql
echo "EC2P: Starting PSQL Execution"
psql -h <host> \
-p 5432 \
-U <user> \
-o /tmp/postgres-query-result.txt \
<db_name>_db \
< /tmp/postgres-query.sql \
> /tmp/postgres-query-output.txt 2>&1
echo "psql exit code is $?"
echo "EC2P: PSQL Execution Complete"
# since instance_initiated_shutdown_behavior = "terminate"
echo "shutdown"
shutdown
Terraform Declaration Of EC2 Instance
resource "aws_instance" "ec2_provisioner" {
count = var.enabled ? 1 : 0
ami = data.aws_ami.ec2_provisioner.id
iam_instance_profile = var.iam_instance_profile
instance_initiated_shutdown_behavior = "terminate"
instance_type = var.instance_type
root_block_device {
volume_type = "gp2"
volume_size = "16"
delete_on_termination = "true"
}
subnet_id = var.subnet_id
tags = merge(
{
"es:global:component-name" = "${var.component_name}-ec2-provisioner",
"Name" = var.name
},
jsondecode(var.additional_tags)
)
user_data = templatefile(
"${path.module}/user_data.tpl",
{
BASH_SCRIPT = var.bash_script,
PACKAGES = join(",", var.packages)
}
)
volume_tags = {
"Name" = var.name
}
vpc_security_group_ids = [var.security_group_id]
}
The core issue here is that our initialization of the instance is producing too much console output
74295 # > 64 KBs
Which is hitting a size limit built into the get-console-output command
By default, the console output returns buffered information that was posted shortly after an instance transition state (start, stop, reboot, or terminate). This information is available for at least one hour after the most recent post. Only the most recent 64 KB of console output is available
The solution we're going with to fix this issue to to enable SSM and to log into parse the log output.
I am deploying an EC2 instance to AWS using terraform. I am using user data section of EC2 to install apache.
This is my template.
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 2.70"
}
}
}
provider "aws" {
region = "eu-west-1"
access_key = "xxxxxxxxx"
secret_key = "xxxxxxxxx"
}
resource "aws_instance" "myinstance" {
ami = "ami-047bb4163c506cd98"
instance_type = "t2.micro"
tags = {
Name = "My first terraform instance"
}
vpc_security_group_ids = [ "sg-0721b555cc402a3ad" ]
user_data = "${file("install_apache.sh")}"
key_name = "MyKey"
}
As you can see, I am running a shell script to install apache in the user_data section. This is my install_apache.sh file.
#!/bin/bash -xe
cd /tmp
yum update -y
yum install -y httpd24
echo "Hello from the EC2 instance." > /var/www/html/index.html
sudo -u root service httpd start
As you can see, I have assigned an existing security group to the instance. My security group white list the HTTP request on port 80 for inbound rules as follow.
Then I deploy the template running this command, "terraform apply".
Then I open the public IP address of the instance on the browser. It just keeps loading, loading and loading showing the blank screen. What is wrong with my code and how can I fix it?
Your script is form Amazon Linux 1. For AL2 it should be:
#!/bin/bash -xe
cd /tmp
yum update -y
yum install -y httpd
echo "Hello from the EC2 instance $(hostname -f)." > /var/www/html/index.html
systemctl start httpd
I added $(hostname -f) as enhancement.
I have asked this before, and I thought that I had solved it (here), but now it no longer works.
I'm setting up an EC2 instance with terraform:
resource "aws_instance" "bastion" {
ami = "${var.image}"
instance_type = "${var.inst_type}"
key_name = "Some Keys"
subnet_id = "${aws_subnet.jan_public_subnet.id}"
user_data = "${file("${path.module}/test")}"
vpc_security_group_ids = [
"${aws_security_group.jan_vpc_security_group.id}"
]
tags={
Name="${var.inst_name}"
}
}
And the test file (in user_data) is:
#!/bin/bash
cat >/var/lib/cloud/scripts/per-once/test <<!
runcmd:
- mkdir /run/test-per-once
!
chmod 755 /var/lib/cloud/scripts/per-once/test
I can see that the instructions in test are carried out - the file gets created and it has the permissions as specified. But cloud-init ignores it - this is what I see in /var/log/cloud-init.log
2019-09-27 15:25:01,110 - stages.py[DEBUG]: Running module scripts-per-once (<module 'cloudinit.config.cc_scripts_per_once' from '/usr/lib/python3/dist-packages/cloudinit/config/cc_scripts_per_once.py'>) with frequency once
2019-09-27 15:25:01,110 - handlers.py[DEBUG]: start: modules-final/config-scripts-per-once: running config-scripts-per-once with frequency once
2019-09-27 15:25:01,110 - util.py[DEBUG]: Writing to /var/lib/cloud/sem/config_scripts_per_once.once - wb: [420] 24 bytes
2019-09-27 15:25:01,110 - helpers.py[DEBUG]: Running config-scripts-per-once using lock (<FileLock using file '/var/lib/cloud/sem/config_scripts_per_once.once'>)
2019-09-27 15:25:01,110 - handlers.py[DEBUG]: finish: modules-final/config-scripts-per-once: SUCCESS: config-scripts-per-once ran successfully
I can only assume that I am doing something wrong - but what?
I am setting up an aws instance with wordpress installation and want to create an AMI using created instance. Below I attach my code.
provider "aws" {
region = "${var.region}"
access_key = "${var.access_key}"
secret_key = "${var.secret_key}"
}
resource "aws_instance" "test-wordpress" {
ami = "${var.image_id}"
instance_type = "${var.instance_type}"
key_name = "test-web"
#associate_public_ip_address = yes
user_data = <<-EOF
#!/bin/bash
sudo yum update -y
sudo amazon-linux-extras install -y lamp-mariadb10.2-php7.2 php7.2
sudo yum install -y httpd mariadb-server
cd /var/www/html
sudo echo "healthy" > healthy.html
sudo wget https://wordpress.org/latest.tar.gz
sudo tar -xzf latest.tar.gz
sudo cp -r wordpress/* /var/www/html/
sudo rm -rf wordpress
sudo rm -rf latest.tar.gz
sudo chmod -R 755 wp-content
sudo chown -R apache:apache wp-content
sudo service httpd start
sudo chkconfig httpd on
EOF
tags = {
Name = "test-Wordpress-Server"
}
}
resource "aws_ami_from_instance" "test-wordpress-ami" {
name = "test-wordpress-ami"
source_instance_id = "${aws_instance.test-wordpress.id}"
depends_on = [
aws_instance.test-wordpress,
]
tags = {
Name = "test-wordpress-ami"
}
}
AMI will be created but When I use that AMI to create an another instance wordpress installation not in there. How can I solve this issue?
The best way to create AMI images i think is using Packer, also from Hashicorp like terraform.
What is Packer?
Provision Infrastructure with Packer Packer is HashiCorp's open-source tool for creating machine images from source
configuration. You can configure Packer images with an operating
system and software for your specific use-case.
Packer creates an instance with temporary keypair, security_group and IAM roles. In the provisioner "shell" are custom inline commands possible. Afterwards you can use this ami with your terraform code.
A sample script could look like this:
packer {
required_plugins {
amazon = {
version = ">= 0.0.2"
source = "github.com/hashicorp/amazon"
}
}
}
source "amazon-ebs" "linux" {
# AMI Settings
ami_name = "ami-oracle-python3"
instance_type = "t2.micro"
source_ami = "ami-xxxxxxxx"
ssh_username = "ec2-user"
associate_public_ip_address = false
ami_virtualization_type = "hvm"
subnet_id = "subnet-xxxxxx"
launch_block_device_mappings {
device_name = "/dev/xvda"
volume_size = 8
volume_type = "gp2"
delete_on_termination = true
encrypted = false
}
# Profile Settings
profile = "xxxxxx"
region = "eu-central-1"
}
build {
sources = [
"source.amazon-ebs.linux"
]
provisioner "shell" {
inline = [
"export no_proxy=localhost"
]
}
}
You can find documentation here.
So you can search for AMI by your tag as described in documentation
In your case:
data "aws_ami" "example" {
executable_users = ["self"]
most_recent = true
owners = ["self"]
filter {
name = "tag:Name"
values = ["test-wordpress-ami"]
}
}
and then refer ID as ${data.aws_ami.example.image_id}
I'm using terraform to create mutiple ec2 nodes on aws:
resource "aws_instance" "myapp" {
count = "${var.count}"
ami = "${data.aws_ami.ubuntu.id}"
instance_type = "m4.large"
vpc_security_group_ids = ["${aws_security_group.myapp-security-group.id}"]
subnet_id = "${var.subnet_id}"
key_name = "${var.key_name}"
iam_instance_profile = "${aws_iam_instance_profile.myapp_instance_profile.id}"
connection {
user = "ubuntu"
private_key = "${file("${var.key_file_path}")}"
}
provisioner "remote-exec" {
inline = [
"sudo apt-get update",
"sudo apt-get upgrade -y",
"sudo apt-get install -f -y openjdk-7-jre-headless git awscli"
]
}
}
When I run this with say count=4, some nodes intermittently fail with apt-get errors like:
aws_instance.myapp.1 (remote-exec): E: Unable to locate package awscli
while the other 3 nodes found awscli just fine. Now all nodes are created from the same AMI, use the exact same provisioning commands, why would only some of them fail? The variation could potentially come from:
Multiple copies of AMIs on amazon, which aren't identical
Multiple apt-get mirrors which aren't identical
Which is more likely? Any other possibilities I'm missing?
Is there an apt-get "force" type flag I can use that will make the provisioning more repeatable?
The whole point of automating provisioning through scripts is to avoid this kind of variation between nodes :/
The remote-exec provisioner feature of Terraform just generates a shell script that is uploaded to the new instance and runs the commands you specify. Most likely you're actually running into problems with cloud-init which is configured to run on standard Ubuntu AMIs, and the provisioner is attempting to run while cloud-init is also running, so you're running into a timing/conflict.
You can make your script wait until after cloud-init has finished provisioning. cloud-init creates a file in /var/lib/cloud/instance/boot-finished, so you can put this inline with your provisioner:
until [[ -f /var/lib/cloud/instance/boot-finished ]]; do
sleep 1
done
Alternatively, you can take advantage of cloud-init and have it install arbitrary packages for you. You can specify user-data for your instance like so in Terraform (modified from your snippet above):
resource "aws_instance" "myapp" {
count = "${var.count}"
ami = "${data.aws_ami.ubuntu.id}"
instance_type = "m4.large"
vpc_security_group_ids = ["${aws_security_group.myapp-security-group.id}"]
subnet_id = "${var.subnet_id}"
key_name = "${var.key_name}"
iam_instance_profile = "${aws_iam_instance_profile.myapp_instance_profile.id}"
user_data = "${data.template_cloudinit_config.config.rendered}"
}
# Standard cloud-init stuff
data "template_cloudinit_config" "config" {
# I've
gzip = false
base64_encode = false
part {
content_type = "text/cloud-config"
content = <<EOF
packages:
- awscli
- git
- openjdk-7-headless
EOF
}
}