Create snapshots of multiple EBS volumes using Terraform - amazon-web-services

I am trying to create snapshots of certain EBS volumes based on tags in a particular AWS region using Terraform.
I have tried filtering EBS volumes based on Tags. I can get a clear output of EBS volume id when only one tag value is specified in the filter attribute but for more than one values, i get the following error:
data.aws_ebs_volume.ebs_volume: data.aws_ebs_volume.ebs_volume: Your
query returned more than one result. Please try a more specific search
criteria, or set most_recent attribute to true.
Below is my terraform template:
data "aws_ebs_volume" "ebs_volume" {
filter {
name = "tag:Name"
values = ["EBS1","EBS2","EBS3"]
}
}
output "ebs_volume_id" {
value = "${data.aws_ebs_volume.ebs_volume.id}"
}
resource "aws_ebs_snapshot" "ebs_volume" {
volume_id = "${data.aws_ebs_volume.ebs_volume.id}"
}
Is there a clear way to create snapshots of multiple EBS volumes using any kind of looping statement in terraform?

You can use the count meta parameter to loop over lists, creating multiple resources or data sources.
In your case you could do something like this:
variable "ebs_volumes" {
default = [
"EBS1",
"EBS2",
"EBS3",
]
}
data "aws_ebs_volume" "ebs_volume" {
count = "${length(var.ebs_volumes)}"
filter {
name = "tag:Name"
values = ["${var.ebs_volumes[count.index]}"]
}
}
output "ebs_volume_ids" {
value = ["${data.aws_ebs_volume.ebs_volume.*.id}"]
}
resource "aws_ebs_snapshot" "ebs_volume" {
count = "${length(var.ebs_volumes)}"
volume_id = "${data.aws_ebs_volume.ebs_volume.*.id[count.index]}"
}

Related

multiple EC2 Instances matched; use additional constraints to reduce matches to a single EC2 Instance

I am using the following script to query a particular instance. There will be only one running instance with the given name. It is possible that another instance with the same name may exists but in different instance state.
How do I filter instance on instance's state so it only retrieves instance that is running state?
data "aws_instance" "ec2" {
filter {
name = "tag:Name"
values = ["dev-us-west-2-myinstance"]
}
}
Currently I get the following error
multiple EC2 Instances matched; use additional constraints to reduce
matches to a single EC2 Instance
The terraform documentation, links to the AWS documentation for the describe-instances filter.
That documentation indicates you should do the following:
data "aws_instance" "ec2" {
filter {
name = "tag:Name"
values = ["dev-us-west-2-myinstance"]
}
filter {
name = "instance-state-name"
values = ["running"]
}
}

how to use != in terraform data block

I want to get a list of ec2 instance ids, but the following is wrong, how can I get all the ec2 except Role = ngx?
data "aws_instances" "ec2" {
filter {
name = "Role"
values != ["ngx"]
}
}
You can't do this. First Role is the wrong filter. Maybe you wanted iam-instance-profile.arn? Second, you can't do inverse searches.
You have to get all the instances, and the filter it yourself in locals.

terraform plan works one way but not the other?

Terraform v0.12.17
I just have 2 simple files
snapshot_id.tf = gets a list of my completed EBS volume snapshot ids
data "aws_ebs_snapshot_ids" "jenkins_master" {
filter {
name = "tag:Name"
values = ["jenkins-master"]
}
filter {
name = "status"
values = ["completed"]
}
}
ebs_volume_green.tf = use above data resource to create an EBS volume
resource "aws_ebs_volume" "jenkins_master_ebs_green" {
availability_zone = var.availability_zones.green
snapshot_id = data.aws_ebs_snapshot_ids.jenkins_master.id
size = data.aws_ebs_snapshot_ids.jenkins_master.volume_size
type = "gp2"
tags = {
Name = "jenkins-master-green"
Environment = "sandbox"
Product = "Jenkins"
Role = "master"
}
}
This passes, so obviously the resource has volume_size defined.
$ terraform plan -target ebs_volume_green.tf -out out.output
$ terraform apply out.output
But this fails, i.e., if I don't specify the -target option. Why?
$ terraform plan -out out.output
Error: Unsupported attribute
on ebs_volume_green.tf line 4, in resource "aws_ebs_volume" "jenkins_master_ebs_green":
4: size = data.aws_ebs_snapshot_ids.jenkins_master.volume_size
This object has no argument, nested block, or exported attribute named
"volume_size".
You have confused the aws_ebs_snapshot_ids and the aws_ebs_snapshot data sources.
You should only be using the plural data sources if you need to return multiple of something. You can then pass these IDs into the individual data source that returns more useful information or just pass the IDs into something that takes a list of IDs such as the aws_autoscaling_group resource's vpc_zone_identifier parameter.
In your case if you just want the most recent snapshot that matches your tags you would just use the following:
data "aws_ebs_snapshot" "jenkins_master" {
most_recent = true
filter {
name = "tag:Name"
values = ["jenkins-master"]
}
filter {
name = "status"
values = ["completed"]
}
}
This will then have the volume_size attribute from the data source that you are expecting.
In your question you can see that it plans and applies successfully when you target just the data source but it's only when your plan includes a usage of a resource or data source's output that doesn't exist that Terraform will complain because it's not evaluating everything in -target mode. In general if you have to use -target then something is wrong somewhere and you should see that as a red flag.

Interpolate data source

I am trying to create some generic Terraform code which supports different MySQL version AMIs. AMIs of MySQL are not in my AWS account. They are in a different AWS account and shared with my account. I am using
a data source to get the latest MySQL AMIs from different account. Now I want to have some thing like this
terraform apply -var somevar=mysql5.6 (This should use mysql5.6 AMI for creating resources)
terraform apply -var somevar=mysql5.5 (This should use mysql5.5 AMI for creating resources)
But the problem is that I can't variablize/interpolate data source mentioned in resource section. Is there any other way to get what I am looking for?
This is the excerpt of what I tried till now
data "aws_ami" "mysql56" {
owners = ["xxxxxxxxxxxx"]
most_recent = true
filter {
name = "image-id"
values = ["${var.os_to_ami["mysql56"]}"]
}
}
data "aws_ami" "mysql55" {
owners = ["xxxxxxxxxxxx"]
most_recent = true
filter {
name = "image-id"
values = ["${var.os_to_ami["mysql55"]}"]
}
}
variable "os_to_ami" {
type = "map"
description = "OS to AMI mapping"
default = {
"mysql56" = "ami-xxxxxxxxxxxxxxxxx"
"mysql57" = "ami-yyyyyyyyyyyyyyyyy"
}
}
resource "aws_instance" "web" {
ami = "${data.aws_ami.mysql56}"
...
}
I am using Terraform v0.11.0.
Your data sources aren't doing anything useful because you're passing the AMI ID into them which defeats the point of them. Instead you should be using the aws_ami data source to fetch the AMI ID based on some criteria such as the owner and the name. This would then let you more simply look these up based on a variable.
Assuming the AMIs are being shared by account 123456789012 and have the names mysql/mysql-5.5/20200818T212500Z and mysql/mysql-5.6/20200818T212500Z then you'd do something like the following:
variable "mysql_version" {}
data "aws_ami" "mysql" {
most_recent = true
filter {
name = "name"
values = ["mysql/mysql-${var.mysql_version}/*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["123456789012"]
}
resource "aws_instance" "mysql" {
ami = data.aws_ami.mysql.id
instance_type = "t3.micro"
}
With the above you have a required mysql_version variable that must be provided. This is then interpolated into the AMI name to search for and will return the latest one matching those criteria so if a new AMI was to be published by the 123456789012 account owner for the same MySQL version with a different timestamp then it would want to recreate your instance with that new AMI.

Using Count in Terraform to create Launch Configuration

I have 3 different version of an AMI, for 3 different nodes in a cluster.
data "aws_ami" "node1"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node1*"]
}
filter {
name = "tag:version"
values = ["${var.node1_version}"]
}
}
data "aws_ami" "node2"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node2*"]
}
filter {
name = "tag:version"
values = ["${var.node2_version}"]
}
}
data "aws_ami" "node3"
{
...
}
I would like to create 3 different Launch Configuration and Auto Scaling Group using each of the AMIs respectively.
resource "aws_launch_configuration" "node"
{
count = "${local.node_instance_count}"
# Name-prefix must be used otherwise terraform fails to perform updates to existing launch configurations due to
# a name conflict: LCs are immutable and the LC cannot be destroyed without destroying attached ASGs as well, which
# terraform will not do. Using name-prefix lets a new LC be created and swapped into the ASG.
name_prefix = "${var.environment_name}-node${count.index + 1}-"
image_id = "${data.aws_ami.node[count.index].image_id}"
instance_type = "${var.default_ec2_instance_type}"
...
}
However, I am not able use aws_ami.node1, aws_ami.node2, aws_ami.node3 using the cound.index the way I have shown above. I get the following error:
Error reading config for aws_launch_configuration[node]: parse error at 1:39: expected "}" but found "."
Is there another way I can do this in Terraform?
Indexing data sources isn't something that's doable; at the moment.
You're likely better off simply dropping the data sources you've defined and codifying the image IDs into a Terraform map variable.
variable "node_image_ids" {
type = "map"
default = {
"node1" = "1234434"
"node2" = "1233334"
"node3" = "1222434"
}
}
Then, consume it:
image_id = "${lookup(var.node_image_ids, concat("node", count.index), "some_default_image_id")}"
The downside of this is that you'll need to manually update the image id when images are upgraded.