I am trying to create some generic Terraform code which supports different MySQL version AMIs. AMIs of MySQL are not in my AWS account. They are in a different AWS account and shared with my account. I am using
a data source to get the latest MySQL AMIs from different account. Now I want to have some thing like this
terraform apply -var somevar=mysql5.6 (This should use mysql5.6 AMI for creating resources)
terraform apply -var somevar=mysql5.5 (This should use mysql5.5 AMI for creating resources)
But the problem is that I can't variablize/interpolate data source mentioned in resource section. Is there any other way to get what I am looking for?
This is the excerpt of what I tried till now
data "aws_ami" "mysql56" {
owners = ["xxxxxxxxxxxx"]
most_recent = true
filter {
name = "image-id"
values = ["${var.os_to_ami["mysql56"]}"]
}
}
data "aws_ami" "mysql55" {
owners = ["xxxxxxxxxxxx"]
most_recent = true
filter {
name = "image-id"
values = ["${var.os_to_ami["mysql55"]}"]
}
}
variable "os_to_ami" {
type = "map"
description = "OS to AMI mapping"
default = {
"mysql56" = "ami-xxxxxxxxxxxxxxxxx"
"mysql57" = "ami-yyyyyyyyyyyyyyyyy"
}
}
resource "aws_instance" "web" {
ami = "${data.aws_ami.mysql56}"
...
}
I am using Terraform v0.11.0.
Your data sources aren't doing anything useful because you're passing the AMI ID into them which defeats the point of them. Instead you should be using the aws_ami data source to fetch the AMI ID based on some criteria such as the owner and the name. This would then let you more simply look these up based on a variable.
Assuming the AMIs are being shared by account 123456789012 and have the names mysql/mysql-5.5/20200818T212500Z and mysql/mysql-5.6/20200818T212500Z then you'd do something like the following:
variable "mysql_version" {}
data "aws_ami" "mysql" {
most_recent = true
filter {
name = "name"
values = ["mysql/mysql-${var.mysql_version}/*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["123456789012"]
}
resource "aws_instance" "mysql" {
ami = data.aws_ami.mysql.id
instance_type = "t3.micro"
}
With the above you have a required mysql_version variable that must be provided. This is then interpolated into the AMI name to search for and will return the latest one matching those criteria so if a new AMI was to be published by the 123456789012 account owner for the same MySQL version with a different timestamp then it would want to recreate your instance with that new AMI.
Related
I set up a jenkins pipeline that launches terraform to create a new EC2 instance in our VPC and register it to our private hosted zone on R53 (which is created at the same time) at every run.
I also managed to save the state into S3 so it doesn't fail with the hosted zone being re-created.
the main issue I have is that at every run terraform keeps replacing the previous instance with the new one and not adding it to the pool of instances.
How can avoid this?
here's a snippet of my code
terraform {
backend "s3" {
bucket = "<redacted>"
key = "<redacted>/terraform.tfstate"
region = "eu-west-1"
}
}
provider "aws" {
region = "${var.region}"
}
data "aws_ami" "image" {
# limit search criteria for performance
most_recent = "${var.ami_filter_most_recent}"
name_regex = "${var.ami_filter_name_regex}"
owners = ["${var.ami_filter_name_owners}"]
# filter on tag purpose
filter {
name = "tag:purpose"
values = ["${var.ami_filter_purpose}"]
}
# filter on tag os
filter {
name = "tag:os"
values = ["${var.ami_filter_os}"]
}
}
resource "aws_instance" "server" {
# use extracted ami from image data source
ami = data.aws_ami.image.id
availability_zone = data.aws_subnet.most_available.availability_zone
subnet_id = data.aws_subnet.most_available.id
instance_type = "${var.instance_type}"
vpc_security_group_ids = ["${var.security_group}"]
user_data = "${var.user_data}"
iam_instance_profile = "${var.iam_instance_profile}"
root_block_device {
volume_size = "${var.root_disk_size}"
}
ebs_block_device {
device_name = "${var.extra_disk_device_name}"
volume_size = "${var.extra_disk_size}"
}
tags = {
Name = "${local.available_name}"
}
}
resource "aws_route53_zone" "private" {
name = var.hosted_zone_name
vpc {
vpc_id = var.vpc_id
}
}
resource "aws_route53_record" "record" {
zone_id = aws_route53_zone.private.zone_id
name = "${local.available_name}.${var.hosted_zone_name}"
type = "A"
ttl = "300"
records = [aws_instance.server.private_ip]
depends_on = [
aws_route53_zone.private
]
}
the outcome is that my previously created instance is destroyed and a new one is created. what I want is to keep adding instances with this code.
thank you
Your code creates only one instance aws_instance.server, and any change to its properties will modify that one instance only as your backend is in S3, thus it acts as a global state for each pipeline. The same goes for aws_route53_record.record and anything else in your script.
If you want different pipelines to reuse the same exact script, you should either use different workspaces, or create different TF states for each pipeline. The other alternative is to redefine your TF script to take a map of instances as an input variable and use for_each to create different instances.
If those instances should be same, you should manage their count using using aws_autoscaling_group and desired capacity.
I'm trying to implement some sort of mechanism where someone can fill in a variable which defines if it's going to deploy an Amazon Linux machine or a self-created packer machine. But for some reason it's not picking up the AWS AMI while it does find my own. Here's the code:
Main.tf of the module:
data "aws_ami" "latest" {
most_recent = true
owners = [var.owner]
filter {
name = "name"
values = [lookup(var.default_ami, var.ami)]
}
}
resource "aws_instance" "test-ec2deployment" {
ami = data.aws_ami.latest.id
variables.tf:
variable "default_ami" {
type = map
description = "Choose windows 2016 or 2019"
default = {
"2016" = "WIN2016-CUSTOM*"
"2019" = "WIN2019-CUSTOM*"
"linux" = "ami-0fb02dcdd38048fb9"
}
}
#Above are the options, here you need to make a decision.
variable "ami" {
description = "You can either choose the 2019 or the 2016 image or the linux image."
default = "2019"
}
And finally the main.tf which calls on the module:
module "aws_instance" {
shared_credentials_file = ""
source = ""
env_name = ""
volume_size = "60"
ami = "linux"
}
Like i said, it does find the correct image when i'm entering either 2019 or 2016. The error message is as follows:
module.aws_instance.data.aws_ami.latest: Refreshing state...
Error: Your query returned no results. Please change your search criteria and try again.
Any idea's?
Thanks for the help!
PS: I've emptied some fields on purpose.
Marcin's answer is spot on. AMI name and AMI id are two different things, you cannot use AMI id to search AMI based on its name. For using AMI id, you need to use "image-id" filter. To get a complete list of filters that we can use for searching AMI, check this.
Now, coming to your problem. You can use "name" and "image-id" filter together to get the required AMI. If you don't have value for any of them use "*". I solved it in the following way -
variable "ami" {
description = "You can either choose the 2019 or the 2016 image or the linux image."
default = "2019"
}
variable "default_ami" {
type = map(any)
default = {
"linux" = {
name = "*"
ami_id = "ami-0fb02dcdd38048fb9"
},
"2016" = {
name = "WIN2016-CUSTOM*"
ami_id = "*"
},
"2019" = {
name = "WIN2019-CUSTOM*"
ami_id = "*"
}
}
}
data "aws_ami" "latest" {
most_recent = true
owners = [var.owner]
filter {
name = "name"
values = ["${var.default_ami[var.ami]["name"]}"]
}
filter {
name = "image-id"
values = ["${var.default_ami[var.ami]["ami_id"]}"]
}
}
resource "aws_instance" "test-ec2deployment" {
ami = data.aws_ami.latest.id
instance_type = var.instance_type
}
AMI name and AMI id are two different things. You are using AMI id (ami-0fb02dcdd38048fb9) to search AMI based on its name. This will not work obviously. You have to replace ami-0fb02dcdd38048fb9 with AMI name.
You haven't specified how you named your linux AMI, maybe Linux2019-CUSTOM*? Whatever the name you've used, you have to use that, not AMI id.
If you are searching by name, you may want to use wildcards. For example, there is an ecs-optimized ami named amzn2-ami-ecs-hvm-2.0.20220318-x86_64-ebs. However, terraform couldn't find it. When I used wildcards, it worked: amzn2-ami-ecs-hvm-2.0.202*-x86_64-ebs
The actual code looks as so:
data "aws_ami" "ecs_optimized_ami" {
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["amzn2-ami-ecs-hvm-2.0.202*-x86_64-ebs"]
}
}
Terraform v0.12.17
I just have 2 simple files
snapshot_id.tf = gets a list of my completed EBS volume snapshot ids
data "aws_ebs_snapshot_ids" "jenkins_master" {
filter {
name = "tag:Name"
values = ["jenkins-master"]
}
filter {
name = "status"
values = ["completed"]
}
}
ebs_volume_green.tf = use above data resource to create an EBS volume
resource "aws_ebs_volume" "jenkins_master_ebs_green" {
availability_zone = var.availability_zones.green
snapshot_id = data.aws_ebs_snapshot_ids.jenkins_master.id
size = data.aws_ebs_snapshot_ids.jenkins_master.volume_size
type = "gp2"
tags = {
Name = "jenkins-master-green"
Environment = "sandbox"
Product = "Jenkins"
Role = "master"
}
}
This passes, so obviously the resource has volume_size defined.
$ terraform plan -target ebs_volume_green.tf -out out.output
$ terraform apply out.output
But this fails, i.e., if I don't specify the -target option. Why?
$ terraform plan -out out.output
Error: Unsupported attribute
on ebs_volume_green.tf line 4, in resource "aws_ebs_volume" "jenkins_master_ebs_green":
4: size = data.aws_ebs_snapshot_ids.jenkins_master.volume_size
This object has no argument, nested block, or exported attribute named
"volume_size".
You have confused the aws_ebs_snapshot_ids and the aws_ebs_snapshot data sources.
You should only be using the plural data sources if you need to return multiple of something. You can then pass these IDs into the individual data source that returns more useful information or just pass the IDs into something that takes a list of IDs such as the aws_autoscaling_group resource's vpc_zone_identifier parameter.
In your case if you just want the most recent snapshot that matches your tags you would just use the following:
data "aws_ebs_snapshot" "jenkins_master" {
most_recent = true
filter {
name = "tag:Name"
values = ["jenkins-master"]
}
filter {
name = "status"
values = ["completed"]
}
}
This will then have the volume_size attribute from the data source that you are expecting.
In your question you can see that it plans and applies successfully when you target just the data source but it's only when your plan includes a usage of a resource or data source's output that doesn't exist that Terraform will complain because it's not evaluating everything in -target mode. In general if you have to use -target then something is wrong somewhere and you should see that as a red flag.
I am trying my hands on terraform and packer.
I have created a costume image with packer.
I know we can create an image with ami id.
I have tried:
resource "aws_instance" "packer-yellowpages" {
ami = "*******"
instance_type = "t3.micro"
tags ={
Name = "demo"
}
}
I was wondering if we can do the same with ami name?
the reason I am thinking about this is: I read somewhere that cloud provider scrap the AMI id. So is there a way I can do this some other way apart from id.
OR implement some storage plan to access the store and access the image?
The aws_ami data source can be used to fetch information on an AMI based on tags such as "name".
data "aws_ami" "example" {
executable_users = ["self"]
most_recent = true
owners = ["self"]
filter {
name = "name"
values = ["myami-*"]
}
}
resource "aws_instance" "packer-yellowpages" {
ami = data.aws_ami.example.id
instance_type = "t3.micro"
tags ={
Name = "demo"
}
}
Try to use these instead to see if it works?
data "aws_ami" "example" {
executable_users = ["self"]
most_recent = true
owners = ["self"]
name_regex = "yellowpages"
}
It may be AMI Name, not Name tag.
I have 3 different version of an AMI, for 3 different nodes in a cluster.
data "aws_ami" "node1"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node1*"]
}
filter {
name = "tag:version"
values = ["${var.node1_version}"]
}
}
data "aws_ami" "node2"
{
# Use the most recent AMI that matches the pattern below in 'values'.
most_recent = true
filter {
name = "name"
values = ["AMI_node2*"]
}
filter {
name = "tag:version"
values = ["${var.node2_version}"]
}
}
data "aws_ami" "node3"
{
...
}
I would like to create 3 different Launch Configuration and Auto Scaling Group using each of the AMIs respectively.
resource "aws_launch_configuration" "node"
{
count = "${local.node_instance_count}"
# Name-prefix must be used otherwise terraform fails to perform updates to existing launch configurations due to
# a name conflict: LCs are immutable and the LC cannot be destroyed without destroying attached ASGs as well, which
# terraform will not do. Using name-prefix lets a new LC be created and swapped into the ASG.
name_prefix = "${var.environment_name}-node${count.index + 1}-"
image_id = "${data.aws_ami.node[count.index].image_id}"
instance_type = "${var.default_ec2_instance_type}"
...
}
However, I am not able use aws_ami.node1, aws_ami.node2, aws_ami.node3 using the cound.index the way I have shown above. I get the following error:
Error reading config for aws_launch_configuration[node]: parse error at 1:39: expected "}" but found "."
Is there another way I can do this in Terraform?
Indexing data sources isn't something that's doable; at the moment.
You're likely better off simply dropping the data sources you've defined and codifying the image IDs into a Terraform map variable.
variable "node_image_ids" {
type = "map"
default = {
"node1" = "1234434"
"node2" = "1233334"
"node3" = "1222434"
}
}
Then, consume it:
image_id = "${lookup(var.node_image_ids, concat("node", count.index), "some_default_image_id")}"
The downside of this is that you'll need to manually update the image id when images are upgraded.