Avoid Update of resources in Terraform - digital-ocean

Currently we are using the Blue/Green Deployment Model for our Application using Terraform.
And our TF Files have resources for both Blue & Green as seen below -
resource "aws_instance" "green_node" {
count = "${var.node_count * var.keep_green * var.build}"
lifecycle = {
create_before_destroy = true
}
ami = "${var.green_ami_id}"
instance_type = "${lookup(var.instance_type,lower(var.env))}"
security_groups = "${split(",", lookup(var.security_groups, format("%s-%s", lower(var.env),var.region)))}"
subnet_id = "${element(split(",", lookup(var.subnets, format("%s-%s", lower(var.env),var.region))), count.index)}"
iam_instance_profile = "${var.iam_role}"
key_name = "${var.key_name}"
associate_public_ip_address = "false"
tags {
Name = "node-green-${var.env}-${count.index + 1}"
}
user_data = "${data.template_cloudinit_config.green_node.rendered}"
}
resource "aws_instance" "blue_node" {
count = "${var.node_count * var.keep_blue * var.build}"
lifecycle = {
create_before_destroy = true
}
ami = "${var.blue_ami_id}"
instance_type = "${lookup(var.instance_type,lower(var.env))}"
security_groups = "${split(",", lookup(var.security_groups, format("%s-%s", lower(var.env),var.region)))}"
subnet_id = "${element(split(",", lookup(var.subnets, format("%s-%s", lower(var.env),var.region))), count.index)}"
iam_instance_profile = "${var.iam_role}"
key_name = "${var.key_name}"
associate_public_ip_address = "false"
tags {
Name = "node-blue-${var.env}-${count.index + 1}"
}
user_data = "${data.template_cloudinit_config.blue_node.rendered}"
}
My question - Is there a way to update the Green Resources without updating the Blue Resources and vice versa Without Using Targeted Plan. For eg. If we update the Security Groups(var.security_groups) which is a common variable, the update will occur for both Blue and Green and i will have to do a targeted plan(seen below)to avoid Blue Resources from getting updated with the New Security Group's -
terraform plan -out=green.plan -target=<green_resource_name>

This is a good question.
If you need to make the blue/green stack work as your expect and reduce the complexity of the code, You can use terraform modules, and set a variable to control which color you will update.
So the stack shares the module when you need update blue or green resources. Define a variable, such as TF_VAR_stack_color to blue or green
Add ${var.stack_color} in the name of any resources you try to create/update in modules.
module "nodes" {
source = "modules/nodes"
name = "${var.name}-${var.stack_color}-${var.others}"
...
}
So you can deploy the blue resource with below command without impact the running green resources.
TF_VAR_stack_color=blue terraform plan
or
terraform plan -var stack_color=blue
With terraform modules, you needn't write resource aws_instance two times for blue and green nodes.
I will recommend splitting the resources into different state files by terraform init, so they will be the totally separate stacks.

Related

How to apply different TAGs for AWS EC2 in Terraform

I have applied the code for tagging AWS ec2 instances in Terraform, when the code runs it only created singe TAG.
How can we add multiple TAGs e.g
It add Auto creation DATE.
It add Auto OS detection (like it is windows or linux)
Please see TAG detail in Screenshot
Gurus, your kind support will be highly appreciated.
I have added the following code for Tagging.
# Block for create EC2 Instance
resource "aws_instance" "ec2" {
count = var.instance_count
ami = "ami-005835d578c62050d"
instance_type = "t2.micro"
vpc_security_group_ids = [var.security_group_id]
subnet_id = var.subnet_id
key_name = var.key
**tags = {
Name = "${var.name}-${count.index + 1}"**
}
}
tags attribute accepts a map of strings and you can also use terraform functions like merge to merge default tags if available in your used case with custom resource-specific tags.
# Block for create EC2 Instance
resource "aws_instance" "ec2" {
count = var.instance_count
ami = "ami-005835d578c62050d"
instance_type = "t2.micro"
vpc_security_group_ids = [var.security_group_id]
subnet_id = var.subnet_id
key_name = var.key
tags = merge(var.default_ec2_tags,
{
Name = "${var.name}-${count.index + 1}"
}
)
}
variable "default_ec2_tags" {
type = map(string)
description = "(optional) default tags for ec2 instances"
default = {
managed_by = "terraform"
environment = "dev"
}
}
Something very specific to terraform-aws-provider and a very handy feature is default_tags which you can configure on the provider level and these tags will be applied to all resources managed by the provider.
Click to view Tutorial from hashicorp on default-tags-in-the-terraform-aws-provider
It's not possible to get the OS type tag natively as mentioned by #Marcin already in the comments.
You can add other tags by simply adding to your Tags, For example:
tags = {
Name = "${var.name}-${count.index + 1}"
CreationDate = timestamp()
OS = "Linux"
}

Assign aws name identification to an ec2 instance using Terraform

I created an ec2 instance using Terraform code and added one tag. But I would like to add a default instance name as well to appear in the field shown in the below screenshot.
How do I do that using Terraform?.
Right now, I tried the default argument for the tag but did not seem to work. I find it hard to figure out from the doc.
Code that I currently used to create it:
resource "aws_instance" "webserver" {
ami = "ami-09a41e26df464c548"
instance_type = "t2.micro"
tags = {
default = "my-ec2-instance"
"Terraform" = "Yes"
}
}
You would want to add a tag with the key of Name:
resource "aws_instance" "webserver" {
ami = "ami-09a41e26df464c548"
instance_type = "t2.micro"
tags = {
Name = "my-ec2-instance"
"Terraform" = "Yes"
}
}
The name of the instance will appear as such:

Dinamically add resources in Terraform

I set up a jenkins pipeline that launches terraform to create a new EC2 instance in our VPC and register it to our private hosted zone on R53 (which is created at the same time) at every run.
I also managed to save the state into S3 so it doesn't fail with the hosted zone being re-created.
the main issue I have is that at every run terraform keeps replacing the previous instance with the new one and not adding it to the pool of instances.
How can avoid this?
here's a snippet of my code
terraform {
backend "s3" {
bucket = "<redacted>"
key = "<redacted>/terraform.tfstate"
region = "eu-west-1"
}
}
provider "aws" {
region = "${var.region}"
}
data "aws_ami" "image" {
# limit search criteria for performance
most_recent = "${var.ami_filter_most_recent}"
name_regex = "${var.ami_filter_name_regex}"
owners = ["${var.ami_filter_name_owners}"]
# filter on tag purpose
filter {
name = "tag:purpose"
values = ["${var.ami_filter_purpose}"]
}
# filter on tag os
filter {
name = "tag:os"
values = ["${var.ami_filter_os}"]
}
}
resource "aws_instance" "server" {
# use extracted ami from image data source
ami = data.aws_ami.image.id
availability_zone = data.aws_subnet.most_available.availability_zone
subnet_id = data.aws_subnet.most_available.id
instance_type = "${var.instance_type}"
vpc_security_group_ids = ["${var.security_group}"]
user_data = "${var.user_data}"
iam_instance_profile = "${var.iam_instance_profile}"
root_block_device {
volume_size = "${var.root_disk_size}"
}
ebs_block_device {
device_name = "${var.extra_disk_device_name}"
volume_size = "${var.extra_disk_size}"
}
tags = {
Name = "${local.available_name}"
}
}
resource "aws_route53_zone" "private" {
name = var.hosted_zone_name
vpc {
vpc_id = var.vpc_id
}
}
resource "aws_route53_record" "record" {
zone_id = aws_route53_zone.private.zone_id
name = "${local.available_name}.${var.hosted_zone_name}"
type = "A"
ttl = "300"
records = [aws_instance.server.private_ip]
depends_on = [
aws_route53_zone.private
]
}
the outcome is that my previously created instance is destroyed and a new one is created. what I want is to keep adding instances with this code.
thank you
Your code creates only one instance aws_instance.server, and any change to its properties will modify that one instance only as your backend is in S3, thus it acts as a global state for each pipeline. The same goes for aws_route53_record.record and anything else in your script.
If you want different pipelines to reuse the same exact script, you should either use different workspaces, or create different TF states for each pipeline. The other alternative is to redefine your TF script to take a map of instances as an input variable and use for_each to create different instances.
If those instances should be same, you should manage their count using using aws_autoscaling_group and desired capacity.

Terraform detach and re-attach ebs volume

I am using Terraform to spin up EC2's. After EC2 is created, I write some data to /myapp. How do I detach /myapp and re-attach it every time when EC2 gets destroyed and recreated again? I did some research and found following code may be the option:
resource "aws_instance" "my_ec2" {
ami = "${var.ami_id}"
instance_type = "${var.instance_type}"
count = "${var.node_count}"
subnet_id = "${var.subnet_id}"
key_name = "${var.key_pair}"
root_block_device = {
volume_type = "gp2"
volume_size = 20
delete_on_termination = false
}
vpc_security_group_ids = ["${var.security_group_ids}"]
}
resource "aws_ebs_volume" "my_vol" {
size = 120
count = "${var.node_count}"
type = "gp2"
}
resource "aws_volume_attachment" "my_vol_att" {
device_name = "/dev/xvdf"
volume_id = "${element(aws_ebs_volume.my_vol.*.id, count.index)}"
instance_id = "${element(aws_instance.my_ec2.*.id, count.index)}"
count = "${var.node_count}"
}
My questions are:
If my_ec2 get destroyed:
ec2 is gone
my_vol is gone?
my_vol_att stays? If so, where can I see it?
When I run above Terraform code again to re-create ec2 after it gets destroyed:
will it create new my_vol id?
what will happen to
my_vol_att?
What exactly is my_vol_att? A pointer or a copy of m_vol that is attached to device and never gets destroyed unless manually delete it?
Sorry, my questions might sound silly as I am very new to both Terraform and AWS.
I believe this happens when the user_data changes. For example you want to say install jenkins on an ebs volume which is persistent
resource "aws_ebs_volume" "jenkins-data" {
availability_zone = local.jenkins-az-location
size = 20
type = "gp2"
tags = {
Name = "jenkins-data"
}
}
resource "aws_volume_attachment" "jenkins-data-attachment" {
device_name = var.INSTANCE_DEVICE_NAME
volume_id = aws_ebs_volume.jenkins-data.id
instance_id = aws_instance.jenkins-instance.id
skip_destroy = true
}
then your user_data
#!/bin/bash
# volume setup
vgchange -ay
DEVICE_FS=`blkid -o value -s TYPE ${DEVICE}`
if [ "`echo -n $DEVICE_FS`" == "" ] ; then
# wait for the device to be attached
DEVICENAME=`echo "${DEVICE}" | awk -F '/' '{print $3}'`
DEVICEEXISTS=''
while [[ -z $DEVICEEXISTS ]]; do
echo "checking $DEVICENAME"
DEVICEEXISTS=`lsblk |grep "$DEVICENAME" |wc -l`
if [[ $DEVICEEXISTS != "1" ]]; then
sleep 15
fi
done
pvcreate ${DEVICE}
vgcreate data ${DEVICE}
lvcreate --name volume1 -l 100%FREE data
mkfs.ext4 /dev/data/volume1
fi
mkdir -p /var/lib/jenkins
echo '/dev/data/volume1 /var/lib/jenkins ext4 defaults 0 0' >> /etc/fstab
mount /var/lib/jenkins
now down the track you want to change the user data for whatever reason and you run terraform again and it will destroy the instance and build another instance and your drive along with it.
To be honest I never looked into this I just take a snapshot of the volume every time I need to do some invasive surgery and it the volume gets destroyed I re-create it from the snapshot then push the changes back into the terraform state file, can be a pain but generally in my case Jenkins is not completely blown away often.
Once I get a change to look into this more I'll probably update this but at the moment I'm sure there is a option in TF to not destroy assets via api or something.
If I capture the sentiment of your questions correctly, you would like to be able to destroy the EC2 instance independently of the EBS volume.
This is possible, but requires some dynamic resolution via data resources.
What I would do here is create a Terraform module that encapsulates the code you have written above, but replaces the aws_ebs_volume resource with a data source like so:
my_module/main.tf
resource "aws_instance" "my_ec2" {
ami = "${var.ami_id}"
instance_type = "${var.instance_type}"
count = "${var.node_count}"
subnet_id = "${var.subnet_id}"
key_name = "${var.key_pair}"
root_block_device = {
volume_type = "gp2"
volume_size = 20
delete_on_termination = false
}
vpc_security_group_ids = ["${var.security_group_ids}"]
}
data "aws_ebs_volume" "my_vol" {
most_recent = true
filter {
name = "volume-type"
values = ["gp2"]
}
filter {
name = "tag:Name"
values = [var.volume_name]
}
}
resource "aws_volume_attachment" "my_vol_att" {
device_name = "/dev/xvdf"
volume_id = "${element(aws_ebs_volume.my_vol.*.id, count.index)}"
instance_id = "${element(aws_instance.my_ec2.*.id, count.index)}"
count = "${var.node_count}"
}
variable "volume_name"{
type = string
default = "my_volume"
}
main.tf
module {
source = "./my-module"
volume_name = "MyManuallyCreatedVolume
}
Note: That the data source I use here is trying to match a volume via the Name tag, which you didn't set in your code but is set in the example here.
Architecting your code like this will allow you to create the volume manually, or via some higher level terraform code that calls this one. As long as your state files are separate, you will be able to destroy your Instance(s) and volumes independently of one-another.

Trying to create 2 ASGs in one terraform file

I'm trying to create an launch configuration, ELB and 2 ASG. I guess one ELB is fine to create 2 ASG (im not sure).
So I have a launch configuration and asg code in one file calling the as module. My question is, can I create 2 ASG using a single terraform file or with file in a single repo?
Also, Please let me know if this is a good configuration.
when I tried to put two different files calling same module I get following error.
Error downloading modules: Error loading modules: module asg: duplicated. module names must be unique
My Terraform code:
auto_scaling.tf
resource "aws_launch_configuration" "launch_config" {
image_id = "${var.ec2ami_id}"
instance_type = "${var.ec2_instance_type}"
security_groups = ["${aws_security_group.*******.id}"]
key_name = "${var.keypair}"
lifecycle {
create_before_destroy = true
}
}
module "asg" {
source = ****
name = "*****"
environment = "***"
service = "****"
product = "**"
team = "****"
owner = "*****"
ami = "${var.ec2_id}"
#instance_profile = "******"
instance_type = "t2.micro"
ebs_optimized = true
key_name = "${var.keypair}"
security_group = ["${aws_security_group.****.id}"]
user_data = "${path.root}/blank_user_data.sh"
load_balancer_names = "${module.elb.elb_name}"
associate_public_ip = false
asg_instances = 2
asg_min_instances = 2
asg_max_instances = 4
root_volume_size = 250
asg_wait_for_capacity_timeout = "5m"
vpc_zone_subnets = "${module.vpc.private_subnets}"
}
###elb.tf###
module "elb" {
source = "*****"
name = "***elb"
subnet_ids = "${element(split(",",
module.vpc.private_subnets), 0)}"
security_groups = "${aws_security_group.****.id}"
s3_access_logs_bucket = "****"
}
I want to create 2 ASGs in one subnet.
You can reuse your asg module - just give both instances different resource names, e.g.:
module "asg1" {
...
}
module "asg2" {
...
}