How can I get current aws spot price through Terraform - amazon-web-services

I would like to create auto scaling group in Terraform and get the spot price through a data and create the launch template with the updated spot price, for example:
resource "aws_launch_template" "launch_cfg_spot" {
count = length(var.pricing)
name_prefix = "launch_cfg_spot_${count.index}"
instance_type = var.pricing[count.index].InstanceType
image_id = "ami-0ff8a91507f77f867"
instance_market_options {
market_type = "spot"
spot_options {
max_price = var.pricing[count.index].price
}
}
network_interfaces{
subnet_id = var.subnets[var.pricing[count.index].az]
}
}
I have implemented it with an external script for now using the describe_spot_price_history command in boto3 but I know for sure that there is a way to get the price through Terraform

Since terraform aws provider 3.1.0 got released, there is a data source called "aws_ec2_spot_price". I use construction based on desired subnet (spot prices are different from one availability zone to another), but you certainly can adjust it up to your needs. I also add two more percent to prevent an instance from termination due to price volatility:
data "aws_subnet" "selected" {
id = var.subnet_id
}
data "aws_ec2_spot_price" "current" {
instance_type = var.instance_type
availability_zone = data.aws_subnet.selected.availability_zone
filter {
name = "product-description"
values = ["Linux/UNIX"]
}
}
locals {
spot_price = data.aws_ec2_spot_price.current.spot_price + data.aws_ec2_spot_price.current.spot_price * 0.02
common_tags = {
ManagedBy = "terraform"
}
}

Related

GCP Compute Engine using Terraform

How to provision multiple instances in GCP Compute Engine using Terraform. I've tried using 'count' parameter in the resource block. But terraform is not provisioning more than one instance because the VM with a particular name is created once when first count is executed.
provider "google" {
version = "3.5.0"
credentials = file("battleground01-5c86f5873d44.json")
project = "battleground01"
region = "us-east1"
zone = "us-east1-b"
}
variable "node_count" {
default = "3"
}
resource "google_compute_instance" "appserver" {
count = "${var.node_count}"
name = "battleground"
machine_type = "f1-micro"
boot_disk {
initialize_params {
image = "debian-cloud/debian-9"
}
}
network_interface {
network = "default"
}
}
In order for this to work, you would have to make a slight change in the way you are naming your compute instances:
provider "google" {
version = "3.5.0"
credentials = file("battleground01-5c86f5873d44.json")
project = "battleground01"
region = "us-east1"
zone = "us-east1-b"
}
variable "node_count" {
type = number
default = 3
}
resource "google_compute_instance" "appserver" {
count = var.node_count
name = "battleground-${count.index}"
machine_type = "f1-micro"
boot_disk {
initialize_params {
image = "debian-cloud/debian-9"
}
}
network_interface {
network = "default"
}
}
As you are using the count meta-argument, the way to access the array index is by using the count.index attribute [1]. You have also set the node_count variable default value to be a string and even though it would probably get converted to a number by Terraform, make sure to use the right variable types.
[1] https://www.terraform.io/language/meta-arguments/count#the-count-object

Linking a series of GCP disks to corresponding instances dynamically in terraform

I need to dynamically create a variable number of riak instances, each with an attached disk across multiple zones in GCP using terraform.
Each attached disk must live in the same zone as its instance.
Upon terraform plan everything looked good, but when I ran apply terraform responded with an error saying that zone was undefined.
Okay, I thought, let's set the zone to be the same as the linked instance. No dice. Cycle error, so I move everything such that the information flows from the disk to the instance, but the cycle error persists. Here's the error:
│ Error: Cycle: module.riak_instances.google_compute_instance.riak_instance, module.riak_instances.google_compute_disk.data-disk
And the code in it's current incarnation:
data "google_compute_zones" "zones" {
}
resource "google_compute_instance" "riak_instance" {
count = var.instance_count
name = "riak-${count.index + 1}"
machine_type = var.machine_type
zone = google_compute_disk.data-disk[count.index].zone
boot_disk {
initialize_params {
image = var.disk_image
size = var.instance_disk_size
}
}
network_interface {
network = "default"
}
attached_disk {
source = google_compute_disk.data-disk[count.index].self_link
}
labels = {
environment = var.environment
owner = var.owner
}
}
resource "google_compute_disk" "data-disk" {
count = var.instance_count
name = "riak-disk-${count.index + 1}"
type = "pd-balanced"
size = var.data_disk_size
zone = data.google_compute_zones.zones.names[count.index]
labels = {
environment = var.environment
owner = var.owner
}
}

Configure an AWS auto-scaling group for price in Terraform?

I have created an AMI that performs work when a machine is started using systemd. Since the work is not time-critical, I would like to optimize for cost using an AWS auto-scaling group. I am using Terraform to manage my infrastructure.
Here is what I have so far:
# ...
resource "aws_launch_template" "default" {
name = "autoscaling-launch-template"
capacity_reservation_specification {
capacity_reservation_preference = "open"
}
credit_specification {
cpu_credits = "standard"
}
iam_instance_profile {
name = aws_iam_instance_profile.default.name
}
image_id = data.aws_ami.default.id
instance_market_options {
market_type = "spot"
}
instance_type = "t2.small"
key_name = var.master_key
monitoring {
enabled = true
}
placement {
availability_zone = "us-east-1a"
}
vpc_security_group_ids = [ "${aws_security_group.default.id}" ]
tag_specifications {
resource_type = "instance"
}
user_data = base64encode(local.user_data)
}
resource "aws_autoscaling_group" "default" {
name = "my-autoscaling-group"
min_size = 1
max_size = 5
desired_capacity = 2
availability_zones = [ "us-east-1a" ]
launch_template {
id = aws_launch_template.default.id
version = "$Latest"
}
lifecycle {
create_before_destroy = true
}
}
# ...
What I would like to achieve is the following:
When the spot instance price is low, scale up to the maximum
When the spot instance price is high, scale down to the minimum
"high" and "low" price should be defined approximately using a rolling average or similar. I don't want to have to maintain minimum and maximum prices.
I always want to use t2.small.
How can I achieve this in Terraform?
The terraform registry has a nice verified module for creating autoscaling groups:
https://registry.terraform.io/modules/terraform-aws-modules/autoscaling/aws/3.4.0
You can use the spot_price variable to launch spot instances into the ASG.

Terraform assign volume to AWS instances list

I have created a AWS instance list using Terraform:
resource "aws_instance" "masters" {
count = 2
ami = "${var.aws_centos_ami}"
instance_type = "t2.micro"
availability_zone = "eu-west-1b"
tags {
Name = "master-${count.index}"
}
}
How can I assign volumes to that instances like in a loop?
I just trying using the next:
data "aws_ebs_volume" "masters_ebs_volume" {
most_recent = true
filter {
name = "attachment.instance-id"
values = ["${aws_instance.masters.*.id}"]
}
}
But I don't thing it is working fine, because when I try to write the AWS volumes in a file, it writes only one volume name.
provisioner "local-exec" {
command = "echo \"${join("\n", data.aws_ebs_volume.masters_ebs_volume.*.id)}\" >> volumes"
}
I have tried defining the volume like this:
data "aws_ebs_volume" "masters_ebs_volume" {
count = 2
# most_recent = true
filter {
name = "attachment.instance-id"
values = ["${aws_instance.masters.*.id}"]
}
}
But it throws the next error:
data.aws_ebs_volume.masters_ebs_volume.0: Your query returned more than one result. Please try a more specific search criteria, or set `most_recent` attribute to true.
You'll need to tell it which instance specifically maps to which volume. You can do that with element():
data "aws_ebs_volume" "masters_ebs_volume" {
count = 2
filter {
name = "attachment.instance-id"
values = ["${element(aws_instance.masters.*.id, count.index)}"]
}
}

Terraform: How to read the volume ID of one instance?

I create instances with a default CentOS 7 AMI. This AMI creates automatically a volume and attached to the instance. Is it possible to read thats volume ID using terraform? I create the instance using the next code:
resource "aws_instance" "DCOS-master3" {
ami = "${var.aws_centos_ami}"
availability_zone = "eu-west-1b"
instance_type = "t2.medium"
key_name = "${var.aws_key_name}"
security_groups = ["${aws_security_group.bastion.id}"]
associate_public_ip_address = true
private_ip = "10.0.0.13"
source_dest_check = false
subnet_id = "${aws_subnet.eu-west-1b-public.id}"
tags {
Name = "master3"
}
}
You won't be able to extract EBS details from aws_instance since it's AWS side that provides an EBS volume to the resource.
But you can define a EBS data source with some filter.
data "aws_ebs_volume" "ebs_volume" {
most_recent = true
filter {
name = "attachment.instance-id"
values = ["${aws_instance.DCOS-master3.id}"]
}
}
output "ebs_volume_id" {
value = "${data.aws_ebs_volume.ebs_volume.id}"
}
You can refer EBS filters here:
http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-volumes.html
You can: aws_instance.DCOS-master3.root_block_device.0.volume_id
As described in Terraform docs:
For any root_block_device and ebs_block_device the volume_id is exported. e.g. aws_instance.web.root_block_device.0.volume_id
output "volume-id-C" {
description = "root volume-id"
#get the root volume id form the instance
value = element(tolist(data.aws_instance.DCOS-master3.root_block_device.*.volume_id),0)
}
output "volume-id-D" {
description = "ebs-volume-id"
#get the 1st esb volume id form the instance
value = element(tolist(data.aws_instance.DCOS-master3.ebs_block_device.*.volume_id),0)
}
You can get the volume name of an aws_instance like this:
output "instance" {
value = aws_instance.ec2_instance.volume_tags["Name"]
}
And you can set it as follows:
resource "aws_instance" "ec2_instance" {
ami = var.instance_ami
instance_type = var.instance_type
key_name = var.instance_key
...
tags = {
Name = "${var.server_name}_${var.instance_name[count.index]}"
}
volume_tags = {
Name = "local_${var.instance_name[count.index]}"
}
}