Terraform: How to read the volume ID of one instance? - amazon-web-services

I create instances with a default CentOS 7 AMI. This AMI creates automatically a volume and attached to the instance. Is it possible to read thats volume ID using terraform? I create the instance using the next code:
resource "aws_instance" "DCOS-master3" {
ami = "${var.aws_centos_ami}"
availability_zone = "eu-west-1b"
instance_type = "t2.medium"
key_name = "${var.aws_key_name}"
security_groups = ["${aws_security_group.bastion.id}"]
associate_public_ip_address = true
private_ip = "10.0.0.13"
source_dest_check = false
subnet_id = "${aws_subnet.eu-west-1b-public.id}"
tags {
Name = "master3"
}
}

You won't be able to extract EBS details from aws_instance since it's AWS side that provides an EBS volume to the resource.
But you can define a EBS data source with some filter.
data "aws_ebs_volume" "ebs_volume" {
most_recent = true
filter {
name = "attachment.instance-id"
values = ["${aws_instance.DCOS-master3.id}"]
}
}
output "ebs_volume_id" {
value = "${data.aws_ebs_volume.ebs_volume.id}"
}
You can refer EBS filters here:
http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-volumes.html

You can: aws_instance.DCOS-master3.root_block_device.0.volume_id
As described in Terraform docs:
For any root_block_device and ebs_block_device the volume_id is exported. e.g. aws_instance.web.root_block_device.0.volume_id

output "volume-id-C" {
description = "root volume-id"
#get the root volume id form the instance
value = element(tolist(data.aws_instance.DCOS-master3.root_block_device.*.volume_id),0)
}
output "volume-id-D" {
description = "ebs-volume-id"
#get the 1st esb volume id form the instance
value = element(tolist(data.aws_instance.DCOS-master3.ebs_block_device.*.volume_id),0)
}

You can get the volume name of an aws_instance like this:
output "instance" {
value = aws_instance.ec2_instance.volume_tags["Name"]
}
And you can set it as follows:
resource "aws_instance" "ec2_instance" {
ami = var.instance_ami
instance_type = var.instance_type
key_name = var.instance_key
...
tags = {
Name = "${var.server_name}_${var.instance_name[count.index]}"
}
volume_tags = {
Name = "local_${var.instance_name[count.index]}"
}
}

Related

Terraform - I can not create EC2 instance using SG module

Could u please let me know why I'm not able to create a EC2 using a SG module that I built?
I'm getting the following error
Error: creating EC2 Instance: VPCIdNotSpecified: No default VPC for this user. GroupName is only supported for EC2-Classic and default VPC.
│ status code: 400, request id: e91aa79f-0d8f-44ec-84df-ba22cd3307d8
Indeed I don't wanna use a default VPC, follow below my main code:
module "vpc" {
source = "../modules/vpc/"
region = var.region
awsprofile = var.awsprofile
vpcname = var.vpcname
subnetaz1 = var.subnetaz1
subnetaz2 = var.subnetaz2
subnetaz3 = var.subnetaz3
private1_cidr = var.private1_cidr
private2_cidr = var.private2_cidr
private3_cidr = var.private3_cidr
public1_cidr = var.public1_cidr
public2_cidr = var.public2_cidr
public3_cidr = var.public3_cidr
vpc_cidr = var.vpc_cidr
}
module "security" {
source = "../modules/security/"
public_sg_name = var.public_sg_name
ingress_ports = var.ingress_ports
internet_access = var.internet_access
vpc_id = module.vpc.aws_vpc_id
}
data "aws_ami" "ubuntu" {
most_recent = true
owners = ["099720109477"] # Canonical
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
}
resource "aws_instance" "web" {
count = var.instance_count
ami = data.aws_ami.ubuntu.id
instance_type = var.instance_size
associate_public_ip_address = "true"
key_name = var.key_name
security_groups = [module.security.sg_name]
tags = {
Name = "${var.instance_name}-1"
}
}
In the "security_groups" I'm trying to get the output from security group module however unsucessfully.
output "sg_id" {
value = aws_security_group.PublicSG.id
}
output "sg_name" {
value = aws_security_group.PublicSG.name
}
Does anyone has any idea why it is not working?
Instead of security_groups you should be using vpc_security_group_ids:
resource "aws_instance" "web" {
count = var.instance_count
ami = data.aws_ami.ubuntu.id
instance_type = var.instance_size
associate_public_ip_address = "true"
key_name = var.key_name
vpc_security_group_ids = [module.security.sg_id]
tags = {
Name = "${var.instance_name}-1"
}
}

How to create ansible inventory from terraform?

I relay stack with that simple question.
Assume i need create few instance resources so how can i iterate from from tf variables to gather all private ips and pass it to ansible inventory file.
As i found i have to use * like here:
k8s_master_name = "${join("\n", azurerm_virtual_machine.k8s-master.*.name)}"
But i as think for me it will look like:
inst_ip = "${join("\n", ${aws_instance.*.private_ip})}"
But i got error:
Error: Invalid reference
on crc.cloud.connect.tf line 72, in resource "local_file" "servers1":
72: inst_ip = "${join("\n", aws_instance.*.private_ip)}"
A reference to a resource type must be followed by at least one attribute
access, specifying the resource name.
Full tf file:
resource "aws_instance" "sp-1" {
ami = "cmi-993E674A"
instance_type = "c5.large"
monitoring = true
source_dest_check = false
user_data = file("user_data.sh")
subnet_id = "subnet-F6C45280"
private_ip = "172.31.16.18"
vpc_security_group_ids = ["sg-230C7615"]
key_name = "mmk-key"
#network_interface {
# network_interface_id = "${aws_network_interface.ni-sp-1.id}"
# device_index = 0
#}
tags = {
desc = "sp-1"
group_name = "sp"
}
}
resource "aws_instance" "sp-2" {
ami = "cmi-993E674A"
instance_type = "c5.large"
monitoring = true
source_dest_check = false
user_data = file("user_data.sh")
subnet_id = "subnet-F6C45280"
private_ip = "172.31.16.19"
vpc_security_group_ids = ["sg-230C7615"]
key_name = "mmk-key"
tags = {
desc = "sp-2"
group_name = "sp"
}
}
resource "local_file" "servers1" {
content = templatefile("${path.module}/templates/servers1.tpl",
{
inst_ip = "${join("\n", ${aws_instance.*.private_ip})}"
}
)
filename = "../ansible/inventory/servers1"
}
Per the Terraform documentation, you need to reference the resource type and its associated name.
In your configuration file, you have an aws_instance resource with the name sp-1. If you wish to access the private_ip attribute of the resource, you need to do it like so: aws_instance.sp-1[*].private_ip.
You are creating a single instance aws_instance.sp-1, not multiple instances. To create multiple instance you would need to use count or for_each, or provision instances through aws_autoscaling_group.
Therefore, to access private_ip you don't really need splat * and join in your case (but still can use them if you want) as you have only one instance and will have only one private_ip. The following should be enough instead:
inst_ip = aws_instance.sp-1.private_ip

How can I get current aws spot price through Terraform

I would like to create auto scaling group in Terraform and get the spot price through a data and create the launch template with the updated spot price, for example:
resource "aws_launch_template" "launch_cfg_spot" {
count = length(var.pricing)
name_prefix = "launch_cfg_spot_${count.index}"
instance_type = var.pricing[count.index].InstanceType
image_id = "ami-0ff8a91507f77f867"
instance_market_options {
market_type = "spot"
spot_options {
max_price = var.pricing[count.index].price
}
}
network_interfaces{
subnet_id = var.subnets[var.pricing[count.index].az]
}
}
I have implemented it with an external script for now using the describe_spot_price_history command in boto3 but I know for sure that there is a way to get the price through Terraform
Since terraform aws provider 3.1.0 got released, there is a data source called "aws_ec2_spot_price". I use construction based on desired subnet (spot prices are different from one availability zone to another), but you certainly can adjust it up to your needs. I also add two more percent to prevent an instance from termination due to price volatility:
data "aws_subnet" "selected" {
id = var.subnet_id
}
data "aws_ec2_spot_price" "current" {
instance_type = var.instance_type
availability_zone = data.aws_subnet.selected.availability_zone
filter {
name = "product-description"
values = ["Linux/UNIX"]
}
}
locals {
spot_price = data.aws_ec2_spot_price.current.spot_price + data.aws_ec2_spot_price.current.spot_price * 0.02
common_tags = {
ManagedBy = "terraform"
}
}

Terraform applying huge index value for instance EBS block store

I am using Terraform (called via Terragrunt, if that's relevant) to create an instance from an AMI and mount an existing volume:
resource "aws_instance" "jenkins_master_with_snap" {
count = "${var.master_with_snapshot}"
ami = "${var.jenkins_ami}"
instance_type = "${var.jenkins_instance_type}"
iam_instance_profile = "${data.terraform_remote_state.global.jenkins_profile_name}"
subnet_id = "${data.aws_subnet.jenkins_subnet_with_snap.id}"
key_name = "${var.key_name}"
vpc_security_group_ids = [
"${aws_security_group.jenkins_master_target_sg.id}",
"${data.terraform_remote_state.cicd.cicd_sg_ipa}"
]
ebs_block_device {
snapshot_id = "${var.master_snapshot_id}"
device_name = "${var.jenkins_volume_device}"
volume_type = "gp2"
}
}
It's worth noting that the AMI used to create this resource already has a snapshot mapped to it from the build process, so this resource basically just replaces it with a different snapshot. I'm not sure if this is why I'm having the problem or not.
I'm using the resulting resource attributes to populate a Python template that will be zipped and uploaded as a lambda function. The Python script requires the volume-id from this instance's EBS block device.
data "template_file" "ebs_backup_lambda_with_snapshot_template" {
count = "${var.master_with_snapshot}"
template = "${file("${path.module}/jenkins_lambda_ebs_backup.py.tpl")}"
vars {
volume_id = "${aws_instance.jenkins_master_with_snap.ebs_block_device.???.volume_id}"
}
}
Onto the actual problem: I do not know how to properly reference the volume ID in the vars section of the template_file resource above. Here is the resulting state:
ebs_block_device.# = 1
ebs_block_device.1440725774.delete_on_termination = true
ebs_block_device.1440725774.device_name = /dev/xvdf
ebs_block_device.1440725774.encrypted = true
ebs_block_device.1440725774.iops = 900
ebs_block_device.1440725774.snapshot_id = snap-1111111111111
ebs_block_device.1440725774.volume_id = vol-1111111111111
ebs_block_device.1440725774.volume_size = 300
ebs_block_device.1440725774.volume_type = gp2
ebs_optimized = false
root_block_device.# = 1
root_block_device.0.delete_on_termination = false
root_block_device.0.iops = 0
root_block_device.0.volume_id = vol-1111111111111
root_block_device.0.volume_size = 8
root_block_device.0.volume_type = standard
The problem is that the index for the EBS volume is that insane integer 1440725774. I have no idea why that is occuring. In the console, there's only a single map in the list I'm interested in:
> aws_instance.jenkins_master_with_snap.ebs_block_device
[
{ delete_on_termination = 1 device_name = /dev/xvdf encrypted = 1 iops = 900 snapshot_id = snap-1111111111111 volume_id = vol-1111111111111 volume_size = 300 volume_type = gp2}
]
And it appears the only way to reference any of those keys is to use that index value directly:
> aws_instance.jenkins_master_with_snap.ebs_block_device.1440725774.volume_id
vol-1111111111111
Is there any way to reliably reference a single element in a list like this when I have no idea what the index is going to be? I can't just hardcode that integer into the template_file resource above and assume it's going to be the same every time. Does anyone have any clues as to why this is occurring in the first place?
Perhaps instead of inlining ebs_block_device block, create a separate aws_ebs_volume resource, then attach it with an aws_volume_attachment. Then reference the aws_ebs_volume.name.id attribute to get the ID you need.
Example (extended from the example code in aws_volume_attachment):
resource "aws_volume_attachment" "ebs_att" {
device_name = "/dev/sdh"
volume_id = "${aws_ebs_volume.example.id}"
instance_id = "${aws_instance.web.id}"
}
resource "aws_instance" "web" {
ami = "ami-21f78e11"
availability_zone = "us-west-2a"
instance_type = "t1.micro"
tags {
Name = "HelloWorld"
}
subnet_id = "<REDACTED>"
}
resource "aws_ebs_volume" "example" {
availability_zone = "us-west-2a"
size = 1
}
data "template_file" "example" {
template = "Your volume ID is $${volume_id}"
vars {
volume_id = "${aws_ebs_volume.example.id}"
}
}
output "custom_template" {
value = "${data.template_file.example.rendered}"
}
The resultant output:
Outputs:
custom_template = Your volume ID is vol-0b1064d4ca6f89a15
You can then use ${aws_ebs_volume.example.id} in your template vars to populate your lambda.

How to correctly use Count and pick multiple az subnets in Terraform

I am trying to implement a module where i am trying to spin a number of instance in already created subnets (by terraform) , but i am not sure how to actually use count in modules and also how to pick values from s3 bucket datasource to spin instance in multi-az , here is what my resource in module dir looks like
resource "aws_instance" "ec2-instances" {
count = "${var.count_num }"
ami = "${data.aws_ami.ubuntu.id}"
instance_type = "${var.machine_type}"
key_name = "${var.key_name}"
#vpc_security_group_ids = ["${aws_security_group.jumpbox-sec-group.id}"]
vpc_security_group_ids = ["${var.sec-group}"]
disable_api_termination = "${var.is_production ? true : false}"
subnet_id = "${element(var.es_stg_subnets, count.index)}" <--- This won't work , i need to use data-source as s3
tags {
#Name = "${var.master_name}-${count.index+1}"
Name = "${var.instance-tag}-${count.index+1}"
Type = "${var.instance-type-tag}"
}
root_block_device {
volume_size = "${var.instance-vol-size}"
volume_type = "gp2"
}
}
And here is the actual module :
module "grafana-stg" {
source = "../../modules/services/gen-ec2"
#ami_id = "${data.aws_ami.ubuntu.id}"
instance_type = "${var.grafana_machine_type}"
key_name = "jumpbox"
vpc_security_group_ids = ["${aws_security_group.grafana-sec-group.id}"]
#subnets = "${data.terraform_remote_state.s3_bucket_state.subnet-public-prod-1a}"
subnet_id = ??????????????????
disable_api_termination = "${var.is_production ? true : false}"
}
I would look at retrieving your subnets utilising a data source.
Utilising Data Sources
Terraform has the concept of data sources. You can pull information from AWS that you require for resources. In your gen-ec2.tf file -
// In order to get subnets, you need the VPC they belong to.
// Note you can filter on a variety of different tags.
data "aws_vpc" "selected" {
tags {
Name = "NameOfVPC"
}
}
// This will then retrieve all subnet ids based on filter
data "aws_subnet_ids" "private" {
vpc_id = "${data.aws_vpc.selected.id}"
tags {
Tier = "private*"
}
}
resource "aws_instance" "ec2-instances" {
count = "${length(data.aws_subnet_ids.private.ids)}"
ami = "${data.aws_ami.ubuntu.id}"
instance_type = "${var.machine_type}"
key_name = "${var.key_name}"
vpc_security_group_ids = ["${var.sec-group}"]
disable_api_termination = "${var.is_production ? true : false}"
subnet_id = "${element(data.aws_subnet_ids.private.*.ids, count.index)}"
tags {
Name = "${var.instance-tag}-${count.index+1}"
Type = "${var.instance-type-tag}"
}
root_block_device {
volume_size = "${var.instance-vol-size}"
volume_type = "gp2"
}
}
Your module now looks like so -
module "grafana-stg" {
source = "../../modules/services/gen-ec2"
#ami_id = "${data.aws_ami.ubuntu.id}"
instance_type = "${var.grafana_machine_type}"
key_name = "jumpbox"
vpc_security_group_ids = ["${aws_security_group.grafana-sec-group.id}"]
disable_api_termination = "${var.is_production ? true : false}"
}
For me as I am using Terraform v0.12.5, the bellow snippet worked fine
data "aws_subnet_ids" "public_subnet_list" {
vpc_id = "${var.vpc_id}"
tags = {
Tier = "Public"
}
}
resource "aws_instance" "example" {
count = 2
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
subnet_id = tolist(data.aws_subnet_ids.public_subnet_list.ids)[count.index]
}