Using Count in Terraform to create Launch Configuration - amazon-web-services

I have 3 different version of an AMI, for 3 different nodes in a cluster.
data "aws_ami" "node1"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node1*"]
}
filter {
name = "tag:version"
values = ["${var.node1_version}"]
}
}
data "aws_ami" "node2"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node2*"]
}
filter {
name = "tag:version"
values = ["${var.node2_version}"]
}
}
data "aws_ami" "node3"
{
...
}
I would like to create 3 different Launch Configuration and Auto Scaling Group using each of the AMIs respectively.
resource "aws_launch_configuration" "node"
{
count = "${local.node_instance_count}"
# Name-prefix must be used otherwise terraform fails to perform updates to existing launch configurations due to
# a name conflict: LCs are immutable and the LC cannot be destroyed without destroying attached ASGs as well, which
# terraform will not do. Using name-prefix lets a new LC be created and swapped into the ASG.
name_prefix = "${var.environment_name}-node${count.index + 1}-"
image_id = "${data.aws_ami.node[count.index].image_id}"
instance_type = "${var.default_ec2_instance_type}"
...
}
However, I am not able use aws_ami.node1, aws_ami.node2, aws_ami.node3 using the cound.index the way I have shown above. I get the following error:
Error reading config for aws_launch_configuration[node]: parse error at 1:39: expected "}" but found "."
Is there another way I can do this in Terraform?

Indexing data sources isn't something that's doable; at the moment.
You're likely better off simply dropping the data sources you've defined and codifying the image IDs into a Terraform map variable.
variable "node_image_ids" {
type = "map"
default = {
"node1" = "1234434"
"node2" = "1233334"
"node3" = "1222434"
}
}
Then, consume it:
image_id = "${lookup(var.node_image_ids, concat("node", count.index), "some_default_image_id")}"
The downside of this is that you'll need to manually update the image id when images are upgraded.

Related

Terraform find latest ami via data

I'm trying to implement some sort of mechanism where someone can fill in a variable which defines if it's going to deploy an Amazon Linux machine or a self-created packer machine. But for some reason it's not picking up the AWS AMI while it does find my own. Here's the code:
Main.tf of the module:
data "aws_ami" "latest" {
most_recent = true
owners = [var.owner]
filter {
name = "name"
values = [lookup(var.default_ami, var.ami)]
}
}
resource "aws_instance" "test-ec2deployment" {
ami = data.aws_ami.latest.id
variables.tf:
variable "default_ami" {
type = map
description = "Choose windows 2016 or 2019"
default = {
"2016" = "WIN2016-CUSTOM*"
"2019" = "WIN2019-CUSTOM*"
"linux" = "ami-0fb02dcdd38048fb9"
}
}
#Above are the options, here you need to make a decision.
variable "ami" {
description = "You can either choose the 2019 or the 2016 image or the linux image."
default = "2019"
}
And finally the main.tf which calls on the module:
module "aws_instance" {
shared_credentials_file = ""
source = ""
env_name = ""
volume_size = "60"
ami = "linux"
}
Like i said, it does find the correct image when i'm entering either 2019 or 2016. The error message is as follows:
module.aws_instance.data.aws_ami.latest: Refreshing state...
Error: Your query returned no results. Please change your search criteria and try again.
Any idea's?
Thanks for the help!
PS: I've emptied some fields on purpose.
Marcin's answer is spot on. AMI name and AMI id are two different things, you cannot use AMI id to search AMI based on its name. For using AMI id, you need to use "image-id" filter. To get a complete list of filters that we can use for searching AMI, check this.
Now, coming to your problem. You can use "name" and "image-id" filter together to get the required AMI. If you don't have value for any of them use "*". I solved it in the following way -
variable "ami" {
description = "You can either choose the 2019 or the 2016 image or the linux image."
default = "2019"
}
variable "default_ami" {
type = map(any)
default = {
"linux" = {
name = "*"
ami_id = "ami-0fb02dcdd38048fb9"
},
"2016" = {
name = "WIN2016-CUSTOM*"
ami_id = "*"
},
"2019" = {
name = "WIN2019-CUSTOM*"
ami_id = "*"
}
}
}
data "aws_ami" "latest" {
most_recent = true
owners = [var.owner]
filter {
name = "name"
values = ["${var.default_ami[var.ami]["name"]}"]
}
filter {
name = "image-id"
values = ["${var.default_ami[var.ami]["ami_id"]}"]
}
}
resource "aws_instance" "test-ec2deployment" {
ami = data.aws_ami.latest.id
instance_type = var.instance_type
}
AMI name and AMI id are two different things. You are using AMI id (ami-0fb02dcdd38048fb9) to search AMI based on its name. This will not work obviously. You have to replace ami-0fb02dcdd38048fb9 with AMI name.
You haven't specified how you named your linux AMI, maybe Linux2019-CUSTOM*? Whatever the name you've used, you have to use that, not AMI id.
If you are searching by name, you may want to use wildcards. For example, there is an ecs-optimized ami named amzn2-ami-ecs-hvm-2.0.20220318-x86_64-ebs. However, terraform couldn't find it. When I used wildcards, it worked: amzn2-ami-ecs-hvm-2.0.202*-x86_64-ebs
The actual code looks as so:
data "aws_ami" "ecs_optimized_ami" {
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["amzn2-ami-ecs-hvm-2.0.202*-x86_64-ebs"]
}
}

terraform plan works one way but not the other?

Terraform v0.12.17
I just have 2 simple files
snapshot_id.tf = gets a list of my completed EBS volume snapshot ids
data "aws_ebs_snapshot_ids" "jenkins_master" {
filter {
name = "tag:Name"
values = ["jenkins-master"]
}
filter {
name = "status"
values = ["completed"]
}
}
ebs_volume_green.tf = use above data resource to create an EBS volume
resource "aws_ebs_volume" "jenkins_master_ebs_green" {
availability_zone = var.availability_zones.green
snapshot_id = data.aws_ebs_snapshot_ids.jenkins_master.id
size = data.aws_ebs_snapshot_ids.jenkins_master.volume_size
type = "gp2"
tags = {
Name = "jenkins-master-green"
Environment = "sandbox"
Product = "Jenkins"
Role = "master"
}
}
This passes, so obviously the resource has volume_size defined.
$ terraform plan -target ebs_volume_green.tf -out out.output
$ terraform apply out.output
But this fails, i.e., if I don't specify the -target option. Why?
$ terraform plan -out out.output
Error: Unsupported attribute
on ebs_volume_green.tf line 4, in resource "aws_ebs_volume" "jenkins_master_ebs_green":
4: size = data.aws_ebs_snapshot_ids.jenkins_master.volume_size
This object has no argument, nested block, or exported attribute named
"volume_size".
You have confused the aws_ebs_snapshot_ids and the aws_ebs_snapshot data sources.
You should only be using the plural data sources if you need to return multiple of something. You can then pass these IDs into the individual data source that returns more useful information or just pass the IDs into something that takes a list of IDs such as the aws_autoscaling_group resource's vpc_zone_identifier parameter.
In your case if you just want the most recent snapshot that matches your tags you would just use the following:
data "aws_ebs_snapshot" "jenkins_master" {
most_recent = true
filter {
name = "tag:Name"
values = ["jenkins-master"]
}
filter {
name = "status"
values = ["completed"]
}
}
This will then have the volume_size attribute from the data source that you are expecting.
In your question you can see that it plans and applies successfully when you target just the data source but it's only when your plan includes a usage of a resource or data source's output that doesn't exist that Terraform will complain because it's not evaluating everything in -target mode. In general if you have to use -target then something is wrong somewhere and you should see that as a red flag.

Terraform: Handle error if no EC2 Instance Type offerings found in AZ

We are spinning up G4 instances in AWS through Terraform and often encounter issues where one or two of the AZs in the given Region don't support G4 Instance type.
As of now I have hardcoded our TF configuration as per below where I am creating a map of Region and AZs as "azs" variable. From this map I can spin up clusters in targeted AZs of the Region where we have G4 Instance support.
I am using aws command line mentioned in this AWS article to find which AZs are supported in a given Region and updating our "azs" variable as we expand to other Regions.
variable "azs" {
default = {
"us-west-2" = "us-west-2a,us-west-2b,us-west-2c"
"us-east-1" = "us-east-1a,us-east-1b,us-east-1e"
"eu-west-1" = "eu-west-1a,eu-west-1b,eu-west-1c"
"eu-west-2" = "eu-west-2a,eu-west-2b,eu-west-2c"
"eu-west-3" = "eu-west-3a,eu-west-3c"
}
However the above approach required human intervention and updates frequently (If AWS adds support to non-supported AZs in a given region later on )
There is this stack overflow question where User is trying to do the same thing however he can use the fallback instance type lets say if any of the AZs are not supported for given instance type.
In my use-case , I can't use any other fall back instance type since our app-servers only runs on G4.
I have tried to use the workaround mentioned as an Answer in the above stack overflow question however its failing with the following error message.
Error: no EC2 Instance Type Offerings found matching criteria; try
different search
on main.tf line 8, in data "aws_ec2_instance_type_offering"
"example": 8: data "aws_ec2_instance_type_offering" "example" {
I am using the TF config as below where my preferred_instance_types is g4dn.xlarge.
provider "aws" {
version = "2.70"
}
data "aws_availability_zones" "all" {
state = "available"
}
data "aws_ec2_instance_type_offering" "example" {
for_each = toset(data.aws_availability_zones.all.names)
filter {
name = "instance-type"
values = ["g4dn.xlarge"]
}
filter {
name = "location"
values = [each.value]
}
location_type = "availability-zone"
preferred_instance_types = ["g4dn.xlarge"]
}
output "foo" {
value = { for az, details in data.aws_ec2_instance_type_offering.example : az => details.instance_type }
}
I would like to know how to handle this failure as Terraform is not able to find the g4 instance type in one of the AZs of a given region and failing.
Is there any Terraform Error handling I can do to by pass this error for now and get the supported AZs as an Output ?
I had checked that other question you mentioned earlier, but i could never get the output correctly. Thanks to #ydaetskcoR for this response in that post - I could learn a bit and get my loop working.
Here is one way to accomplish what you are looking for... Let me know if it works for you.
Instead of "aws_ec2_instance_type_offering", use "aws_ec2_instance_type_offerings" ... (there is a 's' in the end. they are different Data Sources...
I will just paste the code here and assume you will be able to decode the logic. I am filtering for one specific instance type and if its not supported, instance_types will be black and i make a list of AZ thats does not do not have blank values.
variable "az" {
default="us-east-1"
}
variable "my_inst" {
default="g4dn.xlarge"
}
data "aws_availability_zones" "example" {
filter {
name = "opt-in-status"
values = ["opt-in-not-required"]
}
}
data "aws_ec2_instance_type_offerings" "example" {
for_each=toset(data.aws_availability_zones.example.names)
filter {
name = "instance-type"
values = [var.my_inst]
}
filter {
name = "location"
values = ["${each.key}"]
}
location_type = "availability-zone"
}
output "az_where_inst_avail" {
value = keys({ for az, details in data.aws_ec2_instance_type_offerings.example :
az => details.instance_types if length(details.instance_types) != 0 })
}
The output will look like below. us-east-1e does not have the instance type and its not there in the Output. Do test a few cases to see if it works everytime.
Outputs:
az_where_inst_avail = [
"us-east-1a",
"us-east-1b",
"us-east-1c",
"us-east-1d",
"us-east-1f",
]
I think there's a cleaner way. The data source already filters for the availability zone based off of the given filter. There is an attribute -> locations that will produce a list of the desired location_type.
provider "aws" {
region = var.region
}
data "aws_ec2_instance_type_offerings" "available" {
filter {
name = "instance-type"
values = [var.instance_type]
}
location_type = "availability-zone"
}
output "azs" {
value = data.aws_ec2_instance_type_offerings.available.locations
}
Where the instance_type is t3.micro and region is us-east-1, this accurately produces:
azs = tolist([
"us-east-1d",
"us-east-1a",
"us-east-1c",
"us-east-1f",
"us-east-1b",
])
You don't need to feed it a list of availability zones because it already gets those from the supplied region.

Interpolate data source

I am trying to create some generic Terraform code which supports different MySQL version AMIs. AMIs of MySQL are not in my AWS account. They are in a different AWS account and shared with my account. I am using
a data source to get the latest MySQL AMIs from different account. Now I want to have some thing like this
terraform apply -var somevar=mysql5.6 (This should use mysql5.6 AMI for creating resources)
terraform apply -var somevar=mysql5.5 (This should use mysql5.5 AMI for creating resources)
But the problem is that I can't variablize/interpolate data source mentioned in resource section. Is there any other way to get what I am looking for?
This is the excerpt of what I tried till now
data "aws_ami" "mysql56" {
owners = ["xxxxxxxxxxxx"]
most_recent = true
filter {
name = "image-id"
values = ["${var.os_to_ami["mysql56"]}"]
}
}
data "aws_ami" "mysql55" {
owners = ["xxxxxxxxxxxx"]
most_recent = true
filter {
name = "image-id"
values = ["${var.os_to_ami["mysql55"]}"]
}
}
variable "os_to_ami" {
type = "map"
description = "OS to AMI mapping"
default = {
"mysql56" = "ami-xxxxxxxxxxxxxxxxx"
"mysql57" = "ami-yyyyyyyyyyyyyyyyy"
}
}
resource "aws_instance" "web" {
ami = "${data.aws_ami.mysql56}"
...
}
I am using Terraform v0.11.0.
Your data sources aren't doing anything useful because you're passing the AMI ID into them which defeats the point of them. Instead you should be using the aws_ami data source to fetch the AMI ID based on some criteria such as the owner and the name. This would then let you more simply look these up based on a variable.
Assuming the AMIs are being shared by account 123456789012 and have the names mysql/mysql-5.5/20200818T212500Z and mysql/mysql-5.6/20200818T212500Z then you'd do something like the following:
variable "mysql_version" {}
data "aws_ami" "mysql" {
most_recent = true
filter {
name = "name"
values = ["mysql/mysql-${var.mysql_version}/*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["123456789012"]
}
resource "aws_instance" "mysql" {
ami = data.aws_ami.mysql.id
instance_type = "t3.micro"
}
With the above you have a required mysql_version variable that must be provided. This is then interpolated into the AMI name to search for and will return the latest one matching those criteria so if a new AMI was to be published by the 123456789012 account owner for the same MySQL version with a different timestamp then it would want to recreate your instance with that new AMI.

Create snapshots of multiple EBS volumes using Terraform

I am trying to create snapshots of certain EBS volumes based on tags in a particular AWS region using Terraform.
I have tried filtering EBS volumes based on Tags. I can get a clear output of EBS volume id when only one tag value is specified in the filter attribute but for more than one values, i get the following error:
data.aws_ebs_volume.ebs_volume: data.aws_ebs_volume.ebs_volume: Your
query returned more than one result. Please try a more specific search
criteria, or set most_recent attribute to true.
Below is my terraform template:
data "aws_ebs_volume" "ebs_volume" {
filter {
name = "tag:Name"
values = ["EBS1","EBS2","EBS3"]
}
}
output "ebs_volume_id" {
value = "${data.aws_ebs_volume.ebs_volume.id}"
}
resource "aws_ebs_snapshot" "ebs_volume" {
volume_id = "${data.aws_ebs_volume.ebs_volume.id}"
}
Is there a clear way to create snapshots of multiple EBS volumes using any kind of looping statement in terraform?
You can use the count meta parameter to loop over lists, creating multiple resources or data sources.
In your case you could do something like this:
variable "ebs_volumes" {
default = [
"EBS1",
"EBS2",
"EBS3",
]
}
data "aws_ebs_volume" "ebs_volume" {
count = "${length(var.ebs_volumes)}"
filter {
name = "tag:Name"
values = ["${var.ebs_volumes[count.index]}"]
}
}
output "ebs_volume_ids" {
value = ["${data.aws_ebs_volume.ebs_volume.*.id}"]
}
resource "aws_ebs_snapshot" "ebs_volume" {
count = "${length(var.ebs_volumes)}"
volume_id = "${data.aws_ebs_volume.ebs_volume.*.id[count.index]}"
}