We have many IAM users, all creating self-serve infrastructure on EC2 using Terraform. Users don't necessarily set the key for their instances, so it's hard to tie an instance to a particular user. I realize we could dig through CloudTrail to find out which users are creating instances, but it seems like it would be simpler to tag the instances with the current IAM username.
The problem is Terraform doesn't appear to expose this - I can use aws_caller_identity or aws_canonical_user_id, but they both appear to return the organization account, not the specific IAM username. Is there a data source in Terraform that will return the IAM user creating the instances?
It looks like aws_caller_identity doesn't actually call the STS GetCallerId endpoint which would be able to provide the information you need - specifically the UserId and the Arn of the user running the command.
Instead it takes the simpler option and simply uses the accountid that the AWS client has already defined and simply returns that.
So you have a couple of options here. You could raise a pull request to have the aws_caller_identity data source actually call the STS GetCallerId endpoint or you could shell out using a local provisioner and use that to tag your resources.
Obviously if people are writing Terraform to directly use the raw resources that Terraform provides then you can't really enforce this other than having something kill anything that's not tagged but that still leaves the issue of people tagging things using someone else's UserId or Arn.
If instead you have a bunch of modules that people then source to use those instead then you could do something ugly like this in the modules that create the EC2 instances:
resource "aws_instance" "instance" {
ami = "ami-123456"
instance_type = "t2.micro"
tags {
Name = "HelloWorld"
}
lifecycle {
ignore_changes = [ "tags.Owner" ]
}
provisioner "local-exec" {
command = <<EOF
owner=`aws sts get-caller-identity --output text --query 'Arn' | cut -d"/" -f2`
aws ec2 create-tags --resources ${self.id} --tags Key=Owner,Value=$${owner}
EOF
}
}
The above Terraform will create an EC2 instance as normal but then ignore the "Owner" tag. After creating the instance it will run a local shell script that fetches the IAM account name/role for the user and then create an "Owner" tag for the instance using that value.
To handle multiple instances (using count), you can refer the below code:
resource "aws_instance" "instance" {
count = "${var.instance_number}"
ami = "ami-xxxxxx"
instance_type = "${var.instance_type}"
security_groups = "${concat(list("sg-xxxxxx"),var.security_groups)}"
disable_api_termination = "${var.termination_protection}"
subnet_id = "${var.subnet_id}"
iam_instance_profile = "test_role"
tags {
Name = "prod-${var.cluster_name}-${var.service_name}-${count.index+1}"
Environment = "prod"
Product = "${var.cluster_name}"
}
lifecycle {
ignore_changes = [ "tags.LaunchedBy" ]
}
provisioner "local-exec" {
command = <<EOF
launched_by=`aws iam get-user --profile prod | python -mjson.tool | grep UserName | awk '{print $2;exit; }'`
aws ec2 create-tags --resources ${self.id} --tags Key=LaunchedBy,Value=$${launched_by}
EOF
}
}
Related
Im trying to create a Custom AMI for my AWS Deployment with terraform. Its working quite good also its possible to run a bash script. Problem is it's not possible to create the instance temporary and then to terminate the ec2 instance with terraform and all the depending resources.
First im building an "aws_instance" than I provide a bash script in my /tmp folder and let this be done via ssh connection in the terraform script. Looking like the following:
Fist the aws_instance is created based on a standard AWS Amazon Machine Image (AMI). This is used to later create an image from it.
resource "aws_instance" "custom_ami_image" {
tags = { Name = "custom_ami_image" }
ami = var.ami_id //base custom ami id
subnet_id = var.subnet_id
vpc_security_group_ids = [var.security_group_id]
iam_instance_profile = "ec2-instance-profile"
instance_type = "t2.micro"
ebs_block_device {
//...further configurations
}
Now a bash script is provided. The source is the location of the bash script on the local linux box you are executing terraform from. The destination is on the new AWS instance. In the file I install further stuff like python3, oracle drivers and so on...
provisioner "file" {
source = "../bash_file"
destination = "/tmp/bash_file"
}
Then I'll change the permissions on the bash script and execute it with a ssh-user:
provisioner "remote-exec" {
inline = [
"chmod +x /tmp/bash_file",
"sudo /tmp/bash_file",
]
}
No you can login to the ssh-user with the previous created key.
connection {
type = "ssh"
user = "ssh-user"
password = ""
private_key = file("${var.key_name}.pem")
host = self.private_ip
}
}
With the aws_ami_from_instance the ami can be modelled with the current created EC2 instance. And now is callable for further deployments, its also possible to share it in to further aws accounts.
resource "aws_ami_from_instance" "custom_ami_image {
name = "acustom_ami_image"
source_instance_id = aws_instance.custom_ami_image.id
}
Its working fine, but what bothers me is the resulting ec2 instance! Its running and its not possible to terminate it with terraform? Does anyone have an idea how I can handle this? Sure, the running costs are manageable, but I don't like creating datagarbage....
The best way to create AMI images i think is using Packer, also from Hashicorp like Terraform.
What is Packer?
Provision Infrastructure with Packer Packer is HashiCorp's open-source tool for creating machine images from source
configuration. You can configure Packer images with an operating
system and software for your specific use-case.
Packer creates an temporary instance with temporary keypair, security_group and IAM roles. In the provisioner "shell" are custom inline commands possible. Afterwards you can use this ami with your terraform code.
A sample script could look like this:
packer {
required_plugins {
amazon = {
version = ">= 0.0.2"
source = "github.com/hashicorp/amazon"
}
}
}
source "amazon-ebs" "linux" {
# AMI Settings
ami_name = "ami-oracle-python3"
instance_type = "t2.micro"
source_ami = "ami-xxxxxxxx"
ssh_username = "ec2-user"
associate_public_ip_address = false
ami_virtualization_type = "hvm"
subnet_id = "subnet-xxxxxx"
launch_block_device_mappings {
device_name = "/dev/xvda"
volume_size = 8
volume_type = "gp2"
delete_on_termination = true
encrypted = false
}
# Profile Settings
profile = "xxxxxx"
region = "eu-central-1"
}
build {
sources = [
"source.amazon-ebs.linux"
]
provisioner "shell" {
inline = [
"export no_proxy=localhost"
]
}
}
You can find documentation about packer here.
I'm creating multiple aws_instances using terraform's count looping ability. I wish each of these instances to run a script which requires a custom binary. The script calls the binary a user specified number of times with some fixed parameters but I don't think the specifics of what it's doing is relevant to the question
I have been able to create a bucket:
# Create an S3 bucket to hold foo binary and bar script
resource "aws_s3_bucket" "foobar-bucket" {
bucket = "foobar-bucket"
acl = "private"
tags = {
Name = "Foobar Bucket"
}
}
Upload the script and binary to said bucket:
# Upload foo binary to S3 bucket
resource "aws_s3_bucket_object" "foo-object" {
bucket = "foobar-bucket"
key = "foo"
source = "./misc/foo" # local file location
depends_on = [
aws_s3_bucket.foobar-bucket,
]
}
# Upload bar script to S3 bucket
resource "aws_s3_bucket_object" "bar-script" {
bucket = "foobar-bucket"
key = "bar.sh"
source = "./misc/bar.sh"
depends_on = [
aws_s3_bucket.foobar-bucket,
]
}
Then using remote-exec downloaded the script and binary and called the script:
resource "aws_instance" "default" {
count = 10
...
provisioner "remote-exec" {
inline = [
"aws s3 cp s3://foobar-bucket/foo ./",
"aws s3 cp s3://foobar-bucket/bar.sh ./",
"chmod +x foo bar.sh",
"sudo ./bar.sh 100",
]
}
...
}
This works as expected, all the s3 access etc is set up correctly however it doesn't seem like the correct solution, especially as terraform's docs suggest provisioner and remote-exec should be last resort options.
What is the correct way to provision files on an ec2 instance and run scripts using terraform? S3 seemed like a good solution to only upload the files once and allow as many ec2 instances to access as necessary, but maybe there's a better solution again?
How about using user_data option in aws_instance resource?
resource "aws_instance" "example" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
user_data = <<-EOF
#!/bin/bash
aws s3 cp s3://foobar-bucket/foo ./
aws s3 cp s3://foobar-bucket/bar.sh ./
chmod +x foo bar.sh
sudo ./bar.sh 100
EOF
tags = {
Name = "terraform-example"
}
}
If the script is a bit lengthy and you prefer to have that defined in a completely different file
run_commands.sh
#!/usr/bin/env bash
aws s3 cp s3://foobar-bucket/foo ./
aws s3 cp s3://foobar-bucket/bar.sh ./
chmod +x foo bar.sh
sudo ./bar.sh 100
resource "aws_instance" "my-instance" {
ami = "ami-04169656fea786776"
instance_type = "t2.nano"
key_name = "${aws_key_pair.terraform-demo.key_name}"
user_data = "${file("run_commands.sh")}"
tags = {
Name = "Terraform"
}
}
I am running a terraform code to create multiple EC2 instances. Is there a way to setup the hostname of the instance based on tag and a domain name . Currently i login and run hostnamectl set-hostname ..
here is my tf script i use the create the instance.
resource "aws_instance" "RR-TEMP-V-DB" {
ami = var.linux_ami[var.region]
availability_zone = var.availability_zone
instance_type = var.temp_instance_type
key_name = var.linux_key_name
vpc_security_group_ids = [var.vpc_security_group_ids[var.region]]
subnet_id = var.db_subnet_id
count = var.temp_count
tags = {
Name = "RR-TEMP-V-DB-${format("%02d", count.index + 1)}"
Environment = var.env_tag
}
}
Thanks
We accomplish as part of user data, looks similar to:
instance_name=$(aws ec2 describe-instances --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --query "Reservations[*].Instances[*].Tags[?Key=='Name'].Value" --region $(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone | sed -e "s/.$//") --output text)
sudo hostnamectl set-hostname --static $instance_name
you can accomplish that with running as user data as #duhaas suggested or using remote-exec provisioner of terraform. here is the provisioner documentation of terraform, as you will see recommended way is setting user data on instance provision:
https://www.terraform.io/docs/provisioners/
for more details on remote-exec:
https://www.terraform.io/docs/provisioners/remote-exec.html
I have tried following code in order to obtain ip's from ASG, which has been created using Terraform? Is this a good practice or a bad one? But i got the correct ouput as i expected.
data "aws_instances" "test" {
instance_tags {
Environment = "${var.environment}",
instance = "${var.db_instance_name}"
}
instance_state_names = ["running"]
depends_on = ["aws_sqs_queue.ansible", "aws_autoscaling_group.sample"]
}
output.tf
output "privateips" {
value = "${data.aws_instances.test.private_ips}"
}
When creating the ASG, add a local provisioner at the end to execute a local script that interacts with AWS using the cli, so that you can query the ASG IPs:
resource "aws_autoscaling_group" "artifactory" {
name_prefix = "${var.env}-Application-ASG-"
vpc_zone_identifier = ["${var.app_subnets}"]
max_size = "${var.asg_max}"
min_size = "${var.asg_min}"
desired_capacity = "${var.asg_desired}"
force_delete = true
launch_configuration = "${aws_launch_configuration.application.name}"
target_group_arns = ["${aws_alb_target_group.application.arn}"]
provisioner "local-exec" {
command = "./getips.sh"
}
}
script:
ips=""
ids=""
while [ "$ids" = "" ]; do
ids=$(aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names $ASG --region $REGION --query AutoScalingGroups[].Instances[].InstanceId --output text)
sleep 1
done
for ID in $ids;
do
IP=$(aws ec2 describe-instances --instance-ids $ID --region $REGION --query Reservations[].Instances[].PrivateIpAddress --output text)
ips="$ips,$IP"
done
You can get the IPs of the instances in a JSON structure with a single AWS command line execution:
aws ec2 describe-instances \
--filters Name=tag:aws:autoscaling:groupName,Values=$ASG \
--query 'Reservations[*].Instances[*].{"private_ip":PrivateIpAddress}' \
--output json
Sample output:
[
[
{
"private_ip": "10.24.2.120"
}
],
[
{
"private_ip": "10.24.1.147"
}
]
]
This script takes advantage of the fact that an autoscaling group adds a tag to each instance it launches, where the value of that tag is the name of the ASG.
You can place this code directly in the ASG resource definition using the local exec trick presented by #victorm:
resource "aws_autoscaling_group" "ecs" {
name = var.asg_name
...
provisioner "local-exec" {
command = "aws ec2 describe-instances --filters Name=tag:aws:autoscaling:groupName,Values=${var.asg_name} --query 'Reservations[*].Instances[*].{private_ip:PrivateIpAddress}' --output json"
}
}
I added this code to one of my own deployments to make sure it worked. It remains to be worked out how you'd grab and use the output of the execution. I was doing something slightly different with this code. I created an output that spits out the command without executing it. Then I just copy/paste it into my terminal window to run it. I haven't found the need to fully automate the process, without that one manual step. I use that trick (outputting a shell command that can be copy/pasted and executed) for a number of things.
You can create a data resource as such and output the private or public IPs
data "aws_instances" "web_instances" {
instance_state_names = ["running"]
}
output "instance_state_privip" {
description = "Instance Private IPs"
value = data.aws_instances.web_instances.private_ips
}
output "instance_state_pubip" {
description = "Instance Public IPs"
value = data.aws_instances.web_instances.public_ips
}
Is there a way to filter instances by IAM role?
Basically I want a script that terminates all the instances that I've launched, but doesn't touch instances launched with other IAM roles.
Method 1:
If it is just a one-time activity, you can consider using aws-cli itself.
Use the below aws-cli command to list all instances with a particular IAM Role.
aws ec2 describe-instances --region us-east-1 --query 'Reservations[*].Instances[?IamInstanceProfile.Arn==`<Enter you Instance Profile ARN here>`].{InstanceId: InstanceId}' --output text
Replace <Enter you Instance Profile ARN here> with the Instance Profile Arn.
NOTE:
You must enter the Instance Profile Arn and NOT the Role ARN.
Instance Profile Arn will be of the form:
arn:aws:iam::xxxxxxxxxxxx:instance-profile/Profile-ASDNSDLKJ
You can then pass the list of Instance-id's returned above to the terminate-instance cli command. The instance-ids must be separated by spaces.
aws ec2 terminate-instances --instance-ids i-1234567890abcdef0 i-1234567890jkefpq1
Method 2:
import boto3
client = boto3.client('ec2',region_name='us-east-1')
response = client.describe_instances(
Filters=[
{
'Name': 'iam-instance-profile.arn',
'Values': [
'arn:aws:iam::1234567890:instance-profile/MyProfile-ASDNSDLKJ',
]
},
]
)
terminate_instance_list = []
for resp in response['Reservations']:
for inst in resp['Instances']:
#print inst['InstanceId']
terminate_instance_list.append(inst['InstanceId'])
#print(terminate_instance_list)
if terminate_instance_list:
response = client.terminate_instances(
InstanceIds=terminate_instance_list
)
print(response)