Terraform v0.12.17
I just have 2 simple files
snapshot_id.tf = gets a list of my completed EBS volume snapshot ids
data "aws_ebs_snapshot_ids" "jenkins_master" {
filter {
name = "tag:Name"
values = ["jenkins-master"]
}
filter {
name = "status"
values = ["completed"]
}
}
ebs_volume_green.tf = use above data resource to create an EBS volume
resource "aws_ebs_volume" "jenkins_master_ebs_green" {
availability_zone = var.availability_zones.green
snapshot_id = data.aws_ebs_snapshot_ids.jenkins_master.id
size = data.aws_ebs_snapshot_ids.jenkins_master.volume_size
type = "gp2"
tags = {
Name = "jenkins-master-green"
Environment = "sandbox"
Product = "Jenkins"
Role = "master"
}
}
This passes, so obviously the resource has volume_size defined.
$ terraform plan -target ebs_volume_green.tf -out out.output
$ terraform apply out.output
But this fails, i.e., if I don't specify the -target option. Why?
$ terraform plan -out out.output
Error: Unsupported attribute
on ebs_volume_green.tf line 4, in resource "aws_ebs_volume" "jenkins_master_ebs_green":
4: size = data.aws_ebs_snapshot_ids.jenkins_master.volume_size
This object has no argument, nested block, or exported attribute named
"volume_size".
You have confused the aws_ebs_snapshot_ids and the aws_ebs_snapshot data sources.
You should only be using the plural data sources if you need to return multiple of something. You can then pass these IDs into the individual data source that returns more useful information or just pass the IDs into something that takes a list of IDs such as the aws_autoscaling_group resource's vpc_zone_identifier parameter.
In your case if you just want the most recent snapshot that matches your tags you would just use the following:
data "aws_ebs_snapshot" "jenkins_master" {
most_recent = true
filter {
name = "tag:Name"
values = ["jenkins-master"]
}
filter {
name = "status"
values = ["completed"]
}
}
This will then have the volume_size attribute from the data source that you are expecting.
In your question you can see that it plans and applies successfully when you target just the data source but it's only when your plan includes a usage of a resource or data source's output that doesn't exist that Terraform will complain because it's not evaluating everything in -target mode. In general if you have to use -target then something is wrong somewhere and you should see that as a red flag.
Related
I'm trying to implement some sort of mechanism where someone can fill in a variable which defines if it's going to deploy an Amazon Linux machine or a self-created packer machine. But for some reason it's not picking up the AWS AMI while it does find my own. Here's the code:
Main.tf of the module:
data "aws_ami" "latest" {
most_recent = true
owners = [var.owner]
filter {
name = "name"
values = [lookup(var.default_ami, var.ami)]
}
}
resource "aws_instance" "test-ec2deployment" {
ami = data.aws_ami.latest.id
variables.tf:
variable "default_ami" {
type = map
description = "Choose windows 2016 or 2019"
default = {
"2016" = "WIN2016-CUSTOM*"
"2019" = "WIN2019-CUSTOM*"
"linux" = "ami-0fb02dcdd38048fb9"
}
}
#Above are the options, here you need to make a decision.
variable "ami" {
description = "You can either choose the 2019 or the 2016 image or the linux image."
default = "2019"
}
And finally the main.tf which calls on the module:
module "aws_instance" {
shared_credentials_file = ""
source = ""
env_name = ""
volume_size = "60"
ami = "linux"
}
Like i said, it does find the correct image when i'm entering either 2019 or 2016. The error message is as follows:
module.aws_instance.data.aws_ami.latest: Refreshing state...
Error: Your query returned no results. Please change your search criteria and try again.
Any idea's?
Thanks for the help!
PS: I've emptied some fields on purpose.
Marcin's answer is spot on. AMI name and AMI id are two different things, you cannot use AMI id to search AMI based on its name. For using AMI id, you need to use "image-id" filter. To get a complete list of filters that we can use for searching AMI, check this.
Now, coming to your problem. You can use "name" and "image-id" filter together to get the required AMI. If you don't have value for any of them use "*". I solved it in the following way -
variable "ami" {
description = "You can either choose the 2019 or the 2016 image or the linux image."
default = "2019"
}
variable "default_ami" {
type = map(any)
default = {
"linux" = {
name = "*"
ami_id = "ami-0fb02dcdd38048fb9"
},
"2016" = {
name = "WIN2016-CUSTOM*"
ami_id = "*"
},
"2019" = {
name = "WIN2019-CUSTOM*"
ami_id = "*"
}
}
}
data "aws_ami" "latest" {
most_recent = true
owners = [var.owner]
filter {
name = "name"
values = ["${var.default_ami[var.ami]["name"]}"]
}
filter {
name = "image-id"
values = ["${var.default_ami[var.ami]["ami_id"]}"]
}
}
resource "aws_instance" "test-ec2deployment" {
ami = data.aws_ami.latest.id
instance_type = var.instance_type
}
AMI name and AMI id are two different things. You are using AMI id (ami-0fb02dcdd38048fb9) to search AMI based on its name. This will not work obviously. You have to replace ami-0fb02dcdd38048fb9 with AMI name.
You haven't specified how you named your linux AMI, maybe Linux2019-CUSTOM*? Whatever the name you've used, you have to use that, not AMI id.
If you are searching by name, you may want to use wildcards. For example, there is an ecs-optimized ami named amzn2-ami-ecs-hvm-2.0.20220318-x86_64-ebs. However, terraform couldn't find it. When I used wildcards, it worked: amzn2-ami-ecs-hvm-2.0.202*-x86_64-ebs
The actual code looks as so:
data "aws_ami" "ecs_optimized_ami" {
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["amzn2-ami-ecs-hvm-2.0.202*-x86_64-ebs"]
}
}
I am trying to create some generic Terraform code which supports different MySQL version AMIs. AMIs of MySQL are not in my AWS account. They are in a different AWS account and shared with my account. I am using
a data source to get the latest MySQL AMIs from different account. Now I want to have some thing like this
terraform apply -var somevar=mysql5.6 (This should use mysql5.6 AMI for creating resources)
terraform apply -var somevar=mysql5.5 (This should use mysql5.5 AMI for creating resources)
But the problem is that I can't variablize/interpolate data source mentioned in resource section. Is there any other way to get what I am looking for?
This is the excerpt of what I tried till now
data "aws_ami" "mysql56" {
owners = ["xxxxxxxxxxxx"]
most_recent = true
filter {
name = "image-id"
values = ["${var.os_to_ami["mysql56"]}"]
}
}
data "aws_ami" "mysql55" {
owners = ["xxxxxxxxxxxx"]
most_recent = true
filter {
name = "image-id"
values = ["${var.os_to_ami["mysql55"]}"]
}
}
variable "os_to_ami" {
type = "map"
description = "OS to AMI mapping"
default = {
"mysql56" = "ami-xxxxxxxxxxxxxxxxx"
"mysql57" = "ami-yyyyyyyyyyyyyyyyy"
}
}
resource "aws_instance" "web" {
ami = "${data.aws_ami.mysql56}"
...
}
I am using Terraform v0.11.0.
Your data sources aren't doing anything useful because you're passing the AMI ID into them which defeats the point of them. Instead you should be using the aws_ami data source to fetch the AMI ID based on some criteria such as the owner and the name. This would then let you more simply look these up based on a variable.
Assuming the AMIs are being shared by account 123456789012 and have the names mysql/mysql-5.5/20200818T212500Z and mysql/mysql-5.6/20200818T212500Z then you'd do something like the following:
variable "mysql_version" {}
data "aws_ami" "mysql" {
most_recent = true
filter {
name = "name"
values = ["mysql/mysql-${var.mysql_version}/*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["123456789012"]
}
resource "aws_instance" "mysql" {
ami = data.aws_ami.mysql.id
instance_type = "t3.micro"
}
With the above you have a required mysql_version variable that must be provided. This is then interpolated into the AMI name to search for and will return the latest one matching those criteria so if a new AMI was to be published by the 123456789012 account owner for the same MySQL version with a different timestamp then it would want to recreate your instance with that new AMI.
I'm getting the following error when trying to initially plan or apply a resource that is using the data values from the AWS environment to a count.
$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
Error: Invalid count argument
on main.tf line 24, in resource "aws_efs_mount_target" "target":
24: count = length(data.aws_subnet_ids.subnets.ids)
The "count" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the count depends on.
$ terraform --version
Terraform v0.12.9
+ provider.aws v2.30.0
I tried using the target option but doesn't seem to work on data type.
$ terraform apply -target aws_subnet_ids.subnets
Apply complete! Resources: 0 added, 0 changed, 0 destroyed.
The only solution I found that works is:
remove the resource
apply the project
add the resource back
apply again
Here is a terraform config I created for testing.
provider "aws" {
version = "~> 2.0"
}
locals {
project_id = "it_broke_like_3_collar_watch"
}
terraform {
required_version = ">= 0.12"
}
resource aws_default_vpc default {
}
data aws_subnet_ids subnets {
vpc_id = aws_default_vpc.default.id
}
resource aws_efs_file_system efs {
creation_token = local.project_id
encrypted = true
}
resource aws_efs_mount_target target {
depends_on = [ aws_efs_file_system.efs ]
count = length(data.aws_subnet_ids.subnets.ids)
file_system_id = aws_efs_file_system.efs.id
subnet_id = tolist(data.aws_subnet_ids.subnets.ids)[count.index]
}
Finally figured out the answer after researching the answer by Dude0001.
Short Answer. Use the aws_vpc data source with the default argument instead of the aws_default_vpc resource. Here is the working sample with comments on the changes.
locals {
project_id = "it_broke_like_3_collar_watch"
}
terraform {
required_version = ">= 0.12"
}
// Delete this --> resource aws_default_vpc default {}
// Add this
data aws_vpc default {
default = true
}
data "aws_subnet_ids" "subnets" {
// Update this from aws_default_vpc.default.id
vpc_id = "${data.aws_vpc.default.id}"
}
resource aws_efs_file_system efs {
creation_token = local.project_id
encrypted = true
}
resource aws_efs_mount_target target {
depends_on = [ aws_efs_file_system.efs ]
count = length(data.aws_subnet_ids.subnets.ids)
file_system_id = aws_efs_file_system.efs.id
subnet_id = tolist(data.aws_subnet_ids.subnets.ids)[count.index]
}
What I couldn't figure out was why my work around of removing aws_efs_mount_target on the first apply worked. It's because after the first apply the aws_default_vpc was loaded into the state file.
So an alternate solution without making change to the original tf file would be to use the target option on the first apply:
$ terraform apply --target aws_default_vpc.default
However, I don't like this as it requires a special case on first deployment which is pretty unique for the terraform deployments I've worked with.
The aws_default_vpc isn't a resource TF can create or destroy. It is the default VPC for your account in each region that AWS creates automatically for you that is protected from being destroyed. You can only (and need to) adopt it in to management and your TF state. This will allow you to begin managing and to inspect when you run plan or apply. Otherwise, TF doesn't know what the resource is or what state it is in, and it cannot create a new one for you as it s a special type of protected resource as described above.
With that said, go get the default VPC id from the correct region you are deploying in your account. Then import it into your TF state. It should then be able to inspect and count the number of subnets.
For example
terraform import aws_default_vpc.default vpc-xxxxxx
https://www.terraform.io/docs/providers/aws/r/default_vpc.html
Using the data element for this looks a little odd to me as well. Can you change your TF script to get the count directly through the aws_default_vpc resource?
I am trying to create snapshots of certain EBS volumes based on tags in a particular AWS region using Terraform.
I have tried filtering EBS volumes based on Tags. I can get a clear output of EBS volume id when only one tag value is specified in the filter attribute but for more than one values, i get the following error:
data.aws_ebs_volume.ebs_volume: data.aws_ebs_volume.ebs_volume: Your
query returned more than one result. Please try a more specific search
criteria, or set most_recent attribute to true.
Below is my terraform template:
data "aws_ebs_volume" "ebs_volume" {
filter {
name = "tag:Name"
values = ["EBS1","EBS2","EBS3"]
}
}
output "ebs_volume_id" {
value = "${data.aws_ebs_volume.ebs_volume.id}"
}
resource "aws_ebs_snapshot" "ebs_volume" {
volume_id = "${data.aws_ebs_volume.ebs_volume.id}"
}
Is there a clear way to create snapshots of multiple EBS volumes using any kind of looping statement in terraform?
You can use the count meta parameter to loop over lists, creating multiple resources or data sources.
In your case you could do something like this:
variable "ebs_volumes" {
default = [
"EBS1",
"EBS2",
"EBS3",
]
}
data "aws_ebs_volume" "ebs_volume" {
count = "${length(var.ebs_volumes)}"
filter {
name = "tag:Name"
values = ["${var.ebs_volumes[count.index]}"]
}
}
output "ebs_volume_ids" {
value = ["${data.aws_ebs_volume.ebs_volume.*.id}"]
}
resource "aws_ebs_snapshot" "ebs_volume" {
count = "${length(var.ebs_volumes)}"
volume_id = "${data.aws_ebs_volume.ebs_volume.*.id[count.index]}"
}
I have 3 different version of an AMI, for 3 different nodes in a cluster.
data "aws_ami" "node1"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node1*"]
}
filter {
name = "tag:version"
values = ["${var.node1_version}"]
}
}
data "aws_ami" "node2"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node2*"]
}
filter {
name = "tag:version"
values = ["${var.node2_version}"]
}
}
data "aws_ami" "node3"
{
...
}
I would like to create 3 different Launch Configuration and Auto Scaling Group using each of the AMIs respectively.
resource "aws_launch_configuration" "node"
{
count = "${local.node_instance_count}"
# Name-prefix must be used otherwise terraform fails to perform updates to existing launch configurations due to
# a name conflict: LCs are immutable and the LC cannot be destroyed without destroying attached ASGs as well, which
# terraform will not do. Using name-prefix lets a new LC be created and swapped into the ASG.
name_prefix = "${var.environment_name}-node${count.index + 1}-"
image_id = "${data.aws_ami.node[count.index].image_id}"
instance_type = "${var.default_ec2_instance_type}"
...
}
However, I am not able use aws_ami.node1, aws_ami.node2, aws_ami.node3 using the cound.index the way I have shown above. I get the following error:
Error reading config for aws_launch_configuration[node]: parse error at 1:39: expected "}" but found "."
Is there another way I can do this in Terraform?
Indexing data sources isn't something that's doable; at the moment.
You're likely better off simply dropping the data sources you've defined and codifying the image IDs into a Terraform map variable.
variable "node_image_ids" {
type = "map"
default = {
"node1" = "1234434"
"node2" = "1233334"
"node3" = "1222434"
}
}
Then, consume it:
image_id = "${lookup(var.node_image_ids, concat("node", count.index), "some_default_image_id")}"
The downside of this is that you'll need to manually update the image id when images are upgraded.