How to apply security group rules in different regions? - amazon-web-services

I need help in writing a terraform script fro AWS as follows:
I have a list of security groups in multiple regions, for example,
- us-east-2
- us-west-1
- etc.
Now when I add a new instance in any of the region, I am applying an EIP.
I need to add that EIP all traffic in every region's security group.
So far what I tried:
Saving the EIP in a file called node_ips.txt
Read that file
Apply it to security group
Here is the script sample:
variable "list_eips" { type=list" }
resource "aws_eip_association" "eip_assoc" {
count = "${local.number_of_instances}"
instance_id = "${element(aws_instance.ec2_instance.*.id, count.index)}"
allocation_id = "${element(data.aws_eip.db_ip.*.id, count.index)}"
provisioner "local-exec" {
command = "echo ${self.public_ip} >> node_ips.txt"
}
}
data "template_file" "read_node_ips" {
template = "${file("${path.cwd}/node_ips.txt")}"
}
resource "aws_security_group_rule" "allow_db_communication" {
type = "ingress"
from_port = 0
to_port = 65535
protocol = "tcp"
cidr_blocks = ["${split(",", "${join("/32,", concat(compact(split("\n",data.template_file.read_node_ips.rendered)),var.list_eips) )}/32")}"]
security_group_id = "${data.aws_security_group.cassandra_sg.id}"
}
This is not working for me. It is adding rules only for list_eips.
Again, when I add a new instance in different region, the secruity group is different. So, I am not able to know what was my security group in previous region.
Please advise any idea.
Thanks.

Terraform has the idea of "Multiple Provider Instances", You can create providers for each region that need to access and manipulate resources.
# West coast region
provider "aws" {
alias = "west"
region = "us-west-2"
}
resource "aws_instance" "foo" {
provider = "aws.west"
# ...
}
https://www.terraform.io/docs/configuration/providers.html

Related

Terraform ec2 - Permission denied (publickey)

I try to learn Terraform.
Want to install some stuff on an EC2 and connect from ssh.
I have created a new ssh-key pair for this.
When i try to ssh -i ssh-keys/id_rsa_aws ubuntu#52.47.123.18 I got the error
Permission denied (publickey).
Here a sample of my .tf script.
resource "aws_instance" "airflow" {
ami = "ami-0d3f551818b21ed81"
instance_type = "t3a.xlarge"
key_name = "admin"
vpc_security_group_ids = [aws_security_group.ssh-group.id]
tags = {
"Name" = "airflow"
}
subnet_id = aws_subnet.ec2_subnet.id
}
resource "aws_key_pair" "admin" {
key_name = "admin"
public_key = "ssh-rsa ........" # I cat my key.pub fot this
}
EDIT
Thanks to #GrzegorzOledzki I see that the issue come from my subnet work. Here the files.
gateway.tf
resource "aws_internet_gateway" "my_gateway" {
vpc_id = aws_vpc.my_vpc.id
tags = {
Name = "my-gateway"
}
}
network.tf
resource "aws_vpc" "my_vpc" {
cidr_block = "10.0.0.0/16"
enable_dns_hostnames = true
enable_dns_support = true
tags = {
Name = "my-vpc"
}
}
resource "aws_eip" "airflow_ip" {
instance = aws_instance.airflow.id
vpc = true
}
security_group.tf
resource "aws_security_group" "ssh-group" {
name = "ssh-group"
vpc_id = aws_vpc.my_vpc.id
ingress {
# TLS (change to whatever ports you need)
from_port = 22
to_port = 22
protocol = "tcp"
# Please restrict your ingress to only necessary IPs and ports.
# Opening to 0.0.0.0/0 can lead to security vulnerabilities.
cidr_blocks = ["my.ip.from.home/32"]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}
subnet.tf
resource "aws_subnet" "ec2_subnet" {
cidr_block = cidrsubnet(aws_vpc.my_vpc.cidr_block, 3, 1)
vpc_id = aws_vpc.my_vpc.id
availability_zone = "eu-west-3c"
}
resource "aws_route_table" "my_route_table" {
vpc_id = aws_vpc.my_vpc.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.my_gateway.id
}
tags = {
Name = "my_route_table"
}
}
resource "aws_route_table_association" "subnet_association" {
route_table_id = aws_route_table.my_route_table.id
subnet_id = aws_subnet.ec2_subnet.id
}
EDIT 2
I destroy everything then rebuilt it and it's working. I'm not sure but I have create my EC2 instance before making and linking the custom-vpc, subnet and security group. It was like something (what ?) went wrong and it couldn't not reassign everything to my instance.
Currenly that pem key has read permission only for users (chmod 400 *.pem). In this case, you need to enable read permission for others also (chmod 404 / chmod o=r *.pem).
I ran in to the same issue and the fix for me too was to just terraform destroy and then rebuilding the infrastructure.
The root cause for my issues was that I had first created and set the key pair for the ec2 instance, but I didn't output/save the private key at all. After creating the ec2 instance with the key pair, I added output + save functionality to the generated ssh keys, but they were actually not the key pair that was set to the ec2 instance, but instead just a new generated one. Now that I generated the key pair set for the newly created instance and saved the output from the private key I could ssh successfully in to the instance

Terraform get IPs of vpc endpoint subnets

I am trying to setup AWS SFTP transfer in vpc endpoint mode but there is one think I can't manage with.
The problem I have is how to get target IPs for NLB target group.
The only output I found:
output "vpc_endpoint_transferserver_network_interface_ids" {
description = "One or more network interfaces for the VPC Endpoint for transferserver"
value = flatten(aws_vpc_endpoint.transfer_server.*.network_interface_ids)
}
gives network interface ids which cannot be used as targets:
Outputs:
api_url = https://12345.execute-api.eu-west-1.amazonaws.com/prod
vpc_endpoint_transferserver_network_interface_ids = [
"eni-12345",
"eni-67890",
"eni-abcde",
]
I went through:
terraform get subnet integration ips from vpc endpoint subnets tab
and
Terraform how to get IP address of aws_lb
but none of them seems to be working. The latter says:
on modules/sftp/main.tf line 134, in data "aws_network_interface" "ifs":
134: count = "${length(local.nlb_interface_ids)}"
The "count" value depends on resource attributes that cannot be determined
until apply, so Terraform cannot predict how many instances will be created.
To work around this, use the -target argument to first apply only the
resources that the count depends on.
You can create an Elastic IP
resource "aws_eip" "lb" {
instance = "${aws_instance.web.id}"
vpc = true
}
Then specify the Elastic IPs while creating Network LB
resource "aws_lb" "example" {
name = "example"
load_balancer_type = "network"
subnet_mapping {
subnet_id = "${aws_subnet.example1.id}"
allocation_id = "${aws_eip.example1.id}"
}
subnet_mapping {
subnet_id = "${aws_subnet.example2.id}"
allocation_id = "${aws_eip.example2.id}"
}
}

How do I add a new security rule to existing EC2 instance using terraform

I have an EC2 Instance already running on AWS with defined security group and rules.
If suppose,I need a port to be opened on that instance,how do I add a new security rule using terraform to the existing security group attached to that instance?
Every time I try changing the port,It is creating a new security group with the same name leading to a duplicate name error
here I am writing a rule to already existing security group:
resource "aws_security_group_rule" "rule1" {
count = "${var.count}"
type = "ingress"
from_port = "${var.portNumber}"
to_port = "${var.toPort}"
protocol = "${var.protocol}"
cidr_blocks = "${var.cidr_blocks}"
security_group_id = "sg-061e#####8"
}
resource "aws_security_group_rule" "rule2" {
type = "ingress"
from_port = "${var.portNumber2}"
to_port = "${var.toPort2}"
protocol = "${var.protocol2}"
# Opening to 0.0.0.0/0 can lead to security vulnerabilities.
cidr_blocks = "${var.cidr_blocks2}"
security_group_id = "sg-061e#####8"
}
resource "aws_instance" "example" {
ami = "ami-0375ca3842950ade6"
instance_type = "t2.micro"
key_name = "${aws_key_pair.deployer.id}"
associate_public_ip_address = true
#security_groups = ["sg-061e11f267df00668"]
vpc_security_group_ids = [ "sg-061e#####8"]
}
How do I add a security rule to the existing security group?
You can add a second security group rule with a different name other than "allow_all" That is why you are getting a duplicate name issue. Maybe something like "allow-all-https" for example. You should also create the rule in terraform, then reference it with in the rule resource creation block.

Unable to update security group rule across regions

Terraform Version: v0.11.8
I am trying to update the rule of security group which is in another region (Ireland) after deploying my current security group (Ohio).
Below is my code snippet:
variable "aws_account_id" {}
terraform {
backend "s3" {}
}
provider "aws" {
region = "us-east-2"
allowed_account_ids = ["${var.aws_account_id}"]
}
provider "aws" {
alias = "TestApp"
region = "eu-west-1"
}
data "aws_security_group" "test_sg" {
provider = "aws.TestApp"
name = "test-sg"
}
resource "aws_security_group" "test1_sg" {
name = "Test 1"
vpc_id = "VPC ID"
}
resource "aws_security_group_rule" "allow_test1_access_test_sg" {
provider = "aws.TestApp"
type = "ingress"
from_port = "80"
to_port = "80"
protocol = "tcp"
security_group_id = "${data.aws_security_group.test_sg.id}"
source_security_group_id = "${aws_security_group.test1_sg.id}"
}
I am getting circular dependency like below.
Case 1: Run above code, Getting error that test1_sg security group does not exist.
Case 2: Run above code without provider = "aws.TestApp" in aws_security_group_rule, Getting error that test_sg security group does not exist.
Case 3: Run above code without provider = "aws.TestApp" in data "aws_security_group" "test_sg" , Getting error that test_sg security group does not exist.
I am not sure this is 100% issue of inter-region communication but it seems like this is the case.
Note: The above error is coming while running terraform apply. Plan is showing correct change without any error.
Any help will be highly appreciated.
Thanks!
Ok so finally i found the answer.
As per Amazon documentation...
You cannot reference the security group of a peer VPC that's in a
different region. Instead, use the CIDR block of the peer VPC.
AWS Doc: Updating Your Security Groups to Reference Peer VPC Groups
I need to use cidr block instead of the security group while referencing in cross region.
As far as I can see with your case 1, your source security group should be "test1_sg". There is no security group with the name "test1_access_test2". Use this.
source_security_group_id = "${aws_security_group.test1_sg.id}"
Otherwise, go and check on your console whether that security group exists or not.

How to create ACL rules for an AWS S3 VPC Endpoint via Terraform?

I need to set up a VPC Endpoint for S3 via Terraform.
To do so, I need to add rules to my acl for all of the CIDR blocks used by S3 in the appropriate region.
I can get the CIDR blocks using the aws_prefix_list data source.
The problem is then adding the rules.
Rules need to be added one at a time via the aws_network_acl_rule resource.
However, without knowing in advance how many CIDR blocks will be returned, there is no way to enumerate through them (Terraform does not let us simply count the results).
I don't want to have to hardcode the number of results to expect (which differs between region and over time).
The following fails with the error * aws_network_acl_rule.private_s3: aws_network_acl_rule.private_s3: value of 'count' cannot be computed because it does not know at plan time how many results will be returned.
resource "aws_vpc_endpoint" "private_s3" {
vpc_id = "{var.vpc_id}"
service_name = "com.amazonaws.us-west-2.s3"
}
data "aws_prefix_list" "private_s3" {
prefix_list_id = "${aws_vpc_endpoint.private_s3.prefix_list_id}"
}
resource "aws_network_acl" "bar" {
vpc_id = "{var.vpc_id}"
}
resource "aws_network_acl_rule" "private_s3" {
count = "${length(data.aws_prefix_list.private_s3.cidr_blocks)}"
network_acl_id = "${aws_network_acl.bar.id}"
rule_number = 200
egress = false
protocol = "tcp"
rule_action = "allow"
cidr_block = "${data.aws_prefix_list.private_s3.cidr_blocks[count.index]}"
from_port = 443
to_port = 443
}
Any thoughts?