Dynamic data source for fetching AMI using Terraform - amazon-web-services

So the following data source will fetch the latest AMI with component:web tagged. Let's say I have a handful of components with their own AMI. Instead of creating this same block 5 times, is there a way to make this more dynamic where I can pass in the value of component? I can't seem to think of a unique value I can pass in. Do I need to refactor my code a bit?
data "aws_ami" "web" {
filter {
name = "state"
values = ["available"]
}
filter {
name = "tag:component"
values = ["web"]
}
most_recent = true
}
I have a defaults module that acts as a metadata lookup where it fetches and outputs basic things like AMI ID and VPC IDs.
Default Module
# defaults/main.tf
data "aws_ami" "web" {
filter {
name = "state"
values = ["available"]
}
filter {
name = "tag:component"
values = ["web"]
}
most_recent = true
}
output "web_ami" {
value = "${data.aws_ami.web.id}"
}
Main code
# service_name/main.tf
module "defaults" {
source = "../defaults"
region = "${var.region}"
environment = "${var.environment}"
}
module "ftpserver" {
source . = "../ec2_instance"
ami_id = "${module.defaults.web_ami}"
...
}

I'd move the aws_ami data source into the module and have it look up the AMI directly rather than have it passed in from outside.
So I would change the ec2_instance module to look like:
variable "ami_component" {}
data "aws_ami" "selected" {
filter {
name = "state"
values = ["available"]
}
filter {
name = "tag:component"
values = ["${var.ami_component"]
}
most_recent = true
}
resource "aws_instance" "instance" {
ami = "${data.aws_ami.selected.id}"
instance_type = "t2.micro"
tags {
Name = "HelloWorld"
}
}
If you then felt like you needed to be able to override the AMI in the ec2_instance module you could change that to instead be:
variable "ami_component" {}
variable "override_ami" {
default = ""
}
data "aws_ami" "selected" {
filter {
name = "state"
values = ["available"]
}
filter {
name = "tag:component"
values = ["${var.ami_component"]
}
most_recent = true
}
resource "aws_instance" "instance" {
ami = "${var.override_ami != "" ? var.override_ami : data.aws_ami.selected.id}"
instance_type = "t2.micro"
tags {
Name = "HelloWorld"
}
}
This uses a conditional to check if the override_ami variable has been set to something else in which case it will use that, otherwise it will use the ami_component variable to look up the appropriate AMI and use that instead.
This has the benefit of moving the AMI selection logic into the Terraform module making the interface to that module much simpler.

Related

Terraform lunch EC2 condition based on count

Trying to lunch 1 or 2 instances based on a condition ( "single-target" )
ERROR im getting :
132: target_id = var.ec2-2
var.ec2-2 is empty tuple
Inappropriate value for attribute "target_id": string required.
the use of that variable :
resource "aws_lb_target_group_attachment" "b" {
target_id = var.ec2-2
}
outputs :
output "ec2-2_ot" {
value = aws_instance.ec2V2[*].id
description = "the value of the network module ec2-2 id"
}
variable :
variable "single-target" {
type=bool
default=false
}
main.tf :
module "network_module" {
ec2-2 = module.compute_module.ec2-2_ot
}
ec2 launch :
resource "aws_instance" "ec2V2" {
count = var.single-target ? 0 : 1
ami = var.ec2_ami
instance_type = var.ec2_type
associate_public_ip_address = true
key_name = var.key_pair
vpc_security_group_ids = [ var.vpc-sg ]
subnet_id = var.subnet-2
user_data = file("PATH/user-data.sh")
tags = {
Name = var.ec2-2_name
}
}
As the count has been already set to the resource you are trying to create, the returning result list would need to be accessed via Splat Expression mentioned in this terraform documentation.
If you need to choose specific index from the list of the result , then you can use the element function to do so
So in your case, if you need to output EC2 id it would be as follows.
output "thisisoutput" {
value = aws_instance.ec2V2[*].arn
}

Error - Missing resource instance key : Because google_compute_instance has "count" set, its attributes must be accessed on specific instances

I am trying to pass mat_ip of google compute instances created in module "microservice-instance" to another module "database". Since I am creating more than one instance, I am getting following error for output variable in module "microservice-instance".
Error: Missing resource instance key
on modules/microservice-instance/ms-outputs.tf line 3, in output "nat_ip": 3: value = google_compute_instance.apps.network_interface[*].access_config[0].nat_ip
Because google_compute_instance.apps has "count" set, its attributes must be accessed on specific instances.
For example, to correlate with indices of a referring resource, use:
google_compute_instance.apps[count.index]
I have looked at following and using the same way of accessing attribute but its not working. Here is code -
main.tf
provider "google" {
credentials = "${file("../../service-account.json")}"
project = var.project
region =var.region
}
# Include modules
module "microservice-instance" {
count = var.appserver_count
source = "./modules/microservice-instance"
appserver_count = var.appserver_count
}
module "database" {
count = var.no_of_db_instances
source = "./modules/database"
nat_ip = module.microservice-instance.nat_ip
no_of_db_instances = var.no_of_db_instances
}
./modules/microservice-instance/microservice-instance.tf
resource "google_compute_instance" "apps" {
count = var.appserver_count
name = "apps-${count.index + 1}"
# name = "apps-${random_id.app_name_suffix.hex}"
machine_type = "f1-micro"
boot_disk {
initialize_params {
image = "ubuntu-os-cloud/ubuntu-1804-lts"
}
}
network_interface {
network = "default"
access_config {
// Ephemeral IP
}
}
}
./modules/microservice-instance/ms-outputs.tf
output "nat_ip" {
value = google_compute_instance.apps.network_interface[*].access_config[0].nat_ip
}
./modules/database/database.tf
resource "random_id" "db_name_suffix" {
byte_length = 4
}
resource "google_sql_database_instance" "postgres" {
name = "postgres-instance-${random_id.db_name_suffix.hex}"
database_version = "POSTGRES_11"
settings {
tier = "db-f1-micro"
ip_configuration {
dynamic "authorized_networks" {
for_each = var.nat_ip
# iterator = ip
content {
# value = ni.0.access_config.0.nat_ip
value = each.key
}
}
}
}
}
You are creating var.appserver_count number of google_compute_instance.apps resources. So you will have:
google_compute_instance.apps[0]
google_compute_instance.apps[1]
...
google_compute_instance.apps[var.appserver_count - 1]
Therefore, in your output, instead of:
output "nat_ip" {
value = google_compute_instance.apps.network_interface[*].access_config[0].nat_ip
}
you have to reference individual apps resources or all of them using [*], for example:
output "nat_ip" {
value = google_compute_instance.apps[*].network_interface[*].access_config[0].nat_ip
}

Use output value in terraform object

I have multiple output variable, I want to make one parent out variable and then put other outputs into it. I have searched about it and found that we can user terraform object for it but can't get the syntax right.
Output.tf
output "public_subnet" {
value = "${module.my_vpc.public_subnets_ids}"
}
output "vpc_id" {
value = "${module.my_vpc.vpc_id}"
}
output "private_subnet" {
value = "${module.my_vpc.private_subnets_ids}"
}
I want my output to be in a object or you can say parent output variable that have all child output vales, I have come up with few line which I know is not right syntax wise but will get you a picture of what I am thinking of.
output "vpc" {
value = {
vpc_id = "${module.my_vpc.vpc_id}"
public_subnet = "${module.my_vpc.public_subnets_ids}"
private_subnet = "${module.my_vpc.private_subnets_ids}"
}
type = object({ vpc_id = string, public_subnet = string, private_subnet = string })
}
Terraform output does not have type. Therefore, your vpc should be:
output "vpc" {
value = {
vpc_id = "${module.my_vpc.vpc_id}"
public_subnet = "${module.my_vpc.public_subnets_ids}"
private_subnet = "${module.my_vpc.private_subnets_ids}"
}
}
But the issue is that a child module has no access to its parrent's outputs. Thus, I'm not exactly sure what do you want to achieve with your outputs. Normally, you would pass variables from parent to child using variable, and then you could make new output from those variables in the child module.
Update
Based on your previous questions, there is main.tf with
module "my_vpc" {
source = "./modules/vpc"
vpc_cidr = var.vpc_cidr
public_subnet = var.public_subnet
private_subnet = var.private_subnet
availability_zone = data.aws_availability_zones.azs.names
}
Therefore, you must have a folder ./modules/vpc. In the folder, there may be a file called ./modules/vpc/vpc.tf. The file will have something like this in it (variables could be in separate file as well):
variable "vpc_cidr" {}
variable "public_subnet" {}
variable "private_subnet" {}
variable "availability_zone" {}
# the rest of the VPC definition. Since the file is not given,
# i can only speculate on the exact details of the content
resource "aws_subnet" "public" {
count = length(var.public_subnet)
vpc_id = aws_vpc.my_vpc.id
# other attributes
}
resource "aws_subnet" "private" {
count = length(var.private_subnet)
vpc_id = aws_vpc.my_vpc.id
# other attributes
}
If so, then you can create a new file, called ./modules/vpc/output.tf with the content:
output "vpc" {
value = {
vpc_id = my_vpc.vpc_id
public_subnet = aws_subnet.public.*.id
private_subnet = aws_subnet.private.*.id
}
}

Terraform assign volume to AWS instances list

I have created a AWS instance list using Terraform:
resource "aws_instance" "masters" {
count = 2
ami = "${var.aws_centos_ami}"
instance_type = "t2.micro"
availability_zone = "eu-west-1b"
tags {
Name = "master-${count.index}"
}
}
How can I assign volumes to that instances like in a loop?
I just trying using the next:
data "aws_ebs_volume" "masters_ebs_volume" {
most_recent = true
filter {
name = "attachment.instance-id"
values = ["${aws_instance.masters.*.id}"]
}
}
But I don't thing it is working fine, because when I try to write the AWS volumes in a file, it writes only one volume name.
provisioner "local-exec" {
command = "echo \"${join("\n", data.aws_ebs_volume.masters_ebs_volume.*.id)}\" >> volumes"
}
I have tried defining the volume like this:
data "aws_ebs_volume" "masters_ebs_volume" {
count = 2
# most_recent = true
filter {
name = "attachment.instance-id"
values = ["${aws_instance.masters.*.id}"]
}
}
But it throws the next error:
data.aws_ebs_volume.masters_ebs_volume.0: Your query returned more than one result. Please try a more specific search criteria, or set `most_recent` attribute to true.
You'll need to tell it which instance specifically maps to which volume. You can do that with element():
data "aws_ebs_volume" "masters_ebs_volume" {
count = 2
filter {
name = "attachment.instance-id"
values = ["${element(aws_instance.masters.*.id, count.index)}"]
}
}

Terraform - Conditional Data Source

In terraform, is there any way to conditionally use a data source? For example:
data "aws_ami" "application" {
most_recent = true
filter {
name = "tag:environment"
values = ["${var.environment}"]
}
owners = ["self"]
}
I'm hoping to be able to pass in an environment variable via the command line, and based on that, determine whether or not to fetch this data source.
I know with resources you can use the count property, but it doesn't seem you can use that with data sources.
I would consider tucking this code away in a module, but modules also can't use the count parameter.
Lastly, another option would be to provide a "Default" value for the data source, if it returned null, but I don't think that's doable either.
Are there any other potential solutions for this?
You can use a conditional on data sources the same as you can with resources and also from Terraform 0.13+ on modules as well:
variable "lookup_ami" {
default = true
}
data "aws_ami" "application" {
count = var.lookup_ami ? 1 : 0
most_recent = true
filter {
name = "tag:environment"
values = [var.environment]
}
owners = ["self"]
}
One use case for this in Terraform 0.12+ is to utilise the lazy evaluation of ternary statements like with the following:
variable "internal" {
default = true
}
data "aws_route53_zone" "private_zone" {
count = var.internal ? 1 : 0
name = var.domain
vpc_id = var.vpc_id
private_zone = var.internal
}
data "aws_route53_zone" "public_zone" {
count = var.internal ? 0 : 1
name = var.domain
private_zone = var.internal
}
resource "aws_route53_record" "www" {
zone_id = var.internal ? data.aws_route53_zone.private_zone.zone_id : data.aws_route53_zone.public_zone.zone_id
name = "www.${var.domain}"
type = "A"
alias {
name = aws_elb.lb.dns_name
zone_id = aws_elb.lb.zone_id
evaluate_target_health = false
}
}
This would create a record in the private zone when var.internal is true and instead create a record in the public zone when var.internal is false.
For this specific use case you could also use Terraform 0.12+'s null to rewrite this more simply:
variable "internal" {
default = true
}
data "aws_route53_zone" "zone" {
name = var.domain
vpc_id = var.internal ? var.vpc_id : null
private_zone = var.internal
}
resource "aws_route53_record" "www" {
zone_id = data.aws_route53_zone.zone.zone_id
name = "www.${data.aws_route53_zone.zone.name}"
type = "A"
alias {
name = aws_elb.lb.dns_name
zone_id = aws_elb.lb.zone_id
evaluate_target_health = false
}
}
This would only pass the vpc_id parameter to the aws_route53_zone data source if var.internal is set to true as you can't set vpc_id when private_zone is false.
Old Terraform 0.11 and earlier answer:
You can in fact use a conditional on the count of data sources but I've yet to manage to work out a good use case for it when I've tried.
As an example I successfully had this working:
data "aws_route53_zone" "private_zone" {
count = "${var.internal == "true" ? 1 : 0}"
name = "${var.domain}"
vpc_id = "${var.vpc_id}"
private_zone = "true"
}
data "aws_route53_zone" "public_zone" {
count = "${var.internal == "true" ? 0 : 1}"
name = "${var.domain}"
private_zone = "false"
}
But then had issues in how to then select the output of it because Terraform will evaluate any variables in the ternary conditional before deciding which side of the ternary to use (instead of lazy evaluation). So something like this doesn't work:
resource "aws_route53_record" "www" {
zone_id = "${var.internal ? data.aws_route53_zone.private_zone.zone_id : data.aws_route53_zone.public_zone.zone_id}"
name = "www.example.com"
type = "A"
alias {
name = "${aws_elb.lb.dns_name}"
zone_id = "${aws_elb.lb.zone_id }"
evaluate_target_health = "false"
}
}
Because if internal is true then you get the private_zone data source but not the public_zone data source and so the second half of the ternary fails to evaluate because data.aws_route53_zone.public_zone.zone_id isn't defined and equally with the other way around too.
In your case you probably just want to conditionally use the data source so might be able to do something like this:
variable "dynamic_ami" { default = "true" }
variable "default_ami" { default = "ami-123456" }
data "aws_ami" "application" {
most_recent = true
filter {
name = "tag:environment"
values = ["${var.environment}"]
}
owners = ["self"]
}
resource "aws_instance" "app" {
ami = "${var.dynamic_ami == "true" ? data.aws_ami.application.id : var.default_ami}"
instance_type = "t2.micro"
}