no matching EC2 Security Group found - amazon-web-services

I create modules for creating AWS vpc, security group and EC2. I make EC2 module depend on vpc and the security group module.
but I get that error:
no matching EC2 Security Group found
on .terraform/modules/main_ec2/modules/EC2Module/main.tf line 26, in data "aws_security_group" "security_group_id":
26: data "aws_security_group" "security_group_id" {
EC2 module
data "aws_subnet" "subnet_id" {
vpc_id = var.vpc_id
count = length(var.subnet_name)
depends_on = [var.subnet_id_depends_on]
filter {
name = "tag:Name"
values = ["VPC0${var.vpc_number}-${element(var.subnet_name, count.index)}"]
}
}
data "aws_security_group" "security_group_id" {
count = length(var.Server_Group_Name)
depends_on = [var.security_group_depends_on]
filter {
name = "tag:Name"
values = ["${local.name}-SG-${var.Server_Group_Name[count.index]}-${format("%02d", count.index + 1)}"]
}
}
resource "aws_instance" "create_instance" {
ami = "${element(data.aws_ami_ids.ami_id.*.id, count.index)}"
instance_type = var.EC2_Type[count.index]
subnet_id = "${element(data.aws_subnet.subnet_id.*.id, count.index)}"
associate_public_ip_address = "true"
vpc_security_group_ids = [ data.aws_security_group.security_group_id[count.index].id ]
key_name = "${local.name}-KP-${var.Server_Name[count.index]}"
depends_on = [var.EC2_depends_on]
}
sg module
resource "aws_security_group" "SGS" {
count = length(var.Server_Group_Name)
name = "${local.name}-SG-${var.Server_Group_Name[count.index]}-${format("%02d", count.index + 1)}"
description = "${local.name}-SG-${var.Server_Group_Name[count.index]}-${format("%02d", count.index + 1)}"
vpc_id = var.vpc_id
tags = {
name = "tag:Name"
values = "${local.name}-SG-${var.Server_Group_Name[count.index]}-${format("%02d", count.index + 1)}"
}
varibles.tf
variable "subnet_id_depends_on" {
type = list(string)
default = ["module.vpc_main"]
}
variable "security_group_depends_on" {
type = list(string)
default = ["module.sg"]
}
variable "EC2_depends_on" {
type = list(string)
default = [ "module.vpc_main", "module.sg", "module.key_pair" ]
}
for example, I call modules like that:
module "main_ec2" {
source = "/home/reham/Data/projectes/terraform//modules/EC2Module"
subnet_id_depends_on = var.subnet_id_depends_on
security_group_depends_on = var.security_group_depends_on
ami_name = var.ami_name
EC2_depends_on = var.EC2_depends_on
}
Outputs defined in the sg module:
output "vpc_id" {
description = "VPC ID"
value = aws_vpc.vpc.id
}
output "primary_public_subnets_id" {
description = "public subnet ID"
value = aws_subnet.primary_public_subnets.*.id
}
output "secondary_public_subnets_id" {
description = "public subnet ID"
value = aws_subnet.secondary_public_subnets.*.id
}
output "primary_private_subnets_id" {
description = "public subnet ID"
value = aws_subnet.primary_private_subnets.*.id
}
output "security-groups" {
value = aws_security_group.SGS[*].id
}
what must I do to solve this problem?

There is more than one issue with the code and it is a result of probably not understanding how the module outputs work [1]. The second part of the issue is that there are many explicit dependencies defined with depends_on which are not required to be there since implicit dependencies are good enough [2]. Furthermore, as per comments, data sources in the EC2 module are not required as information about VPC, subnets and security groups can be provided with the outputs. Additionally, the EC2 instance resource is using count, but there is no count meta-argument there, so it will not work even if all the inputs are fixed. So here are my suggestions:
Redefine variables in the EC2 module (no need for depdends_on) and remove data sources (except possibly the one for the AMI)
Decide which variable you are going to use for the count meta-argument in the EC2 instance resource
I am going to give you an example with one subnet and security groups and then you can build from there. The EC2 module fixes:
resource "aws_instance" "create_instance" {
ami = data.aws_ami_ids.ami_id[0].id
instance_type = var.EC2_Type
subnet_id = var.subnet_id
associate_public_ip_address = "true"
vpc_security_group_ids = var.security_group_ids
key_name = "${local.name}-KP-${var.Server_Name[0]}"
}
In EC2 module, the variables would have to be changed to:
variable "subnet_id" {}
variable "security_group_ids" {}
Then, in the root module:
module "main_ec2" {
source = "/home/reham/Data/projectes/terraform//modules/EC2Module"
subnet_id = module.sg.primary_public_subnets_id[0]
security_group_ids = module.sg.security-groups
ami_name = var.ami_name
}
Referencing the module outputs in the EC2 module will make sure that the SG module is created first (that is an implicit dependency). At best this should give you a result for one EC2 instance as it seems there are many resources that rely on using the count meta-argument and that code is not in the question.
[1] https://www.terraform.io/language/values/outputs#accessing-child-module-outputs
[2] https://www.terraform.io/language/resources/behavior#resource-dependencies

Related

Incorrect attribute value type. Inappropriate value for sttributes "subnets: set of string required."

I've passed subnet ids many times and haven't encountered this error before, I'm really not sure why or how to fix it.
I have my subnet resource block right here in my VPC module:
resource "aws_subnet" "subnet_1" {
vpc_id = aws_vpc.vpc_1.id
cidr_block = "10.0.0.0/24"
availability_zone = var.region_az
tags = {
Name = "pcp-subnet-1"
}
}
This is what the outputs file looks like for my subnet output:
output "subnet_1" {
value = aws_subnet.subnet_1.id
}
This is my root module calling the subnet id output
module "ecs" {
source = "./ecs"
service_subnets = module.vpc.subnet_1
pcp_service_sg = module.vpc.pcp_service_sg
}
In my ecs module I have the variable listed here in my variables file
variable "service_subnets" {
type = string
}
This is my ecs resource where the error is occurring
resource "aws_ecs_service" "ecs-service" {
name = "python-cloud-project"
cluster = aws_ecs_cluster.cluster.id
task_definition = aws_ecs_task_definition.pcp-ecs-task-definition.arn
launch_type = "FARGATE"
network_configuration {
subnets = var.service_subnets
security_groups = var.pcp_service_sg
assign_public_ip = true
}
desired_count = 1
}
network_configuration requires a list of subnets, you are passing only one subnet. What you can do is to create a list with one element:
module "ecs" {
source = "./ecs"
service_subnets = [module.vpc.subnet_1]
pcp_service_sg = module.vpc.pcp_service_sg
}

Get outputs instance_id from loop another module terraform

How to get instance id for my module ec2 when I create using for_each and I need instance_id for my module tgroup to assign or join new instance in specific target group.
My module ec2
resource "aws_instance" "ProductionServer" {
count = length(var.SubnetServer)
ami = module.CreateAMI.AMIId.id
instance_type = var.TypeServer
subnet_id = var.SubnetServer[count.index]
}
ec2/outputs.tf
output "InstanceId" {
value = aws_instance.ProductionServer
description = "get value id"
}
and my module tgroup
resource "aws_alb_target_group_attachment" "TgProdRegister" {
for_each = module.GetInstanceId.InstanceId.id
target_group_arn = var.TgroupArn
target_id = each.value
port = var.TgPort
depends_on = [ module.GetInstanceId ]
}
and GetInstanceId is refer to module ec2
Since you are using count for the instances, to return their ids as a list, you would have to do:
output "InstanceId" {
value = aws_instance.ProductionServer[*].id
description = "get value id"
}
Then for the target groups it would be:
resource "aws_alb_target_group_attachment" "TgProdRegister" {
count = length(module.GetInstanceId)
target_group_arn = var.TgroupArn
target_id = module.GetInstanceId.InstanceId[count.index]
port = var.TgPort
}
Buy the way you should consider placing your instances in auto-scaling groups (ASG) if possible.

Retrieve Subnet ids for map variables

I have created subnet as map variable for availability zone and CIDR block and
variable "public_subnets_list" {
type = map(any)
description = "Public Subnets"
default = {
"ap-south-1a" = "10.0.1.0/24"
"ap-south-1b" = "10.0.2.0/24"
}
}
This works fine for creating subnets under my custom VPC with below code
resource "aws_subnet" "public_subnet" {
depends_on = [
aws_vpc.terraform_vpc
]
for_each = tomap(var.public_subnets_list)
availability_zone = each.key
cidr_block = each.value
vpc_id = aws_vpc.terraform_vpc.id
tags = {
Name = "Public_Subnet_${each.key}"
}
}
How do I retrieve and display the subnet id created for respective AZs from output which i get from aws_subnet.public_subnet[*]
From the Terraform documentation, splat expressions do not work with resources that use the for_each argument.
To retrieve a list of subnet IDs using your Terraform configuration, you can do the following:
output "subnet_ids" {
value = [for subnet in aws_subnet.public_subnet : subnet.id]
}

Terraform multiple instances but by separate execution

I am trying to create AWS instances with load balancer, security group and three instances ---> GROUP 1
I can do this by declaring appropriate resources.
Now I want to create multiples of such instances which are independent of previous instances ---> GROUP 2
I want this because of security of the GROUPS that one group's information should not overlap with other's.
I tried to look up a lot, but couldn't find an approach.
Below is an example of instance:
resource "aws_instance" "node" {
ami = data.aws_ami.ubuntu.id
subnet_id = aws_subnet.development-private-1a.id
key_name = aws_key_pair.nodes.key_name
instance_type = var.instance_type
vpc_security_group_ids = [aws_security_group.dev-ec2-sg.id]
tags = {
Name = "${var.app_name}"
#Environment = "production"
}
root_block_device {
volume_type = "gp2"
volume_size = 8
delete_on_termination = true
}
user_data = file("install_apache.sh")
}
resource "aws_lb_target_group_attachment" "node" {
target_group_arn = aws_lb_target_group.dev.arn
target_id = aws_instance.node.id
port = 80
}
I want to add multiple of these instances with different security groups and load balancers and all other stuff. but I dont want to add additional copies of the same in the terraform file. I want that those instances are independent of this one but then the problem I am facing is that terraform manipulates this instance only.
Based on the comments, you could consider organization of your instance code and its dependents (e.g. target group attachment) as terraform (TF) modules. Also since you are wish to create multiple instance of the same type, you could consider using aws_autoscaling_group which would allow you to not only easily create multiple instance but also easily to manage them.
Subsequently, you could define a module as followed. Below is only partial example. I also do not use aws_autoscaling_group, but create multiple instance using count:
./module/ec2/main.tf
variable "subnet_id" {}
variable "app_name" {}
variable "key_pair" {}
variable "security_group_id" {}
variable "target_group_arn" {}
variable "instance_count" {
default = 1
}
data "aws_ami" "ubuntu" {
# ...
}
resource "aws_instance" "node" {
count = var.instance_count
ami = data.aws_ami.ubuntu.id
subnet_id = var.subnet_id
key_name = var.key_pair
instance_type = var.instance_type
vpc_security_group_ids = [var.security_group_id]
tags = {
Name = "${var.app_name}"
#Environment = "production"
}
root_block_device {
volume_type = "gp2"
volume_size = 8
delete_on_termination = true
}
user_data = file("install_apache.sh")
}
resource "aws_lb_target_group_attachment" "node" {
count = var.instance_count
target_group_arn = var.target_group_arn
target_id = aws_instance.node[count.index].id
port = 80
}
# some outputs skipped
Having such module, in your parent file/module you would create GROUP 1 and 2 instance as follows (again, just partial example):
./main.tf
# resoruces such as LB, SGs, subnets, etc.
module "group1" {
source = "./module/ec2/"
instance_count = 3
security_group_id = <security-group-id1>
target_group_arn = aws_lb_target_group.dev.arn
# other parameters
}
module "group2" {
source = "./module/ec2/"
instance_count = 3
security_group_id = <security-group-id2>
target_group_arn = aws_lb_target_group.dev.arn
# other parameters
}

Terraform AWS subnet_id list is treated as single value string for ec2 instance

I have code to create VPC, with 2 private subnets, 2xec2 instances in private and bastion in public.
ec2 code uses outputs.tf of VPC module subnet_ids. as there are 2 private subnets there are 2 subnet_ids being generated. when these generated subnet_ids are fed into ec2 instances instead of one subnet_id, it is feeding 2 subnet_ids at once as a single value.
As a result terraform couldn't find that subnet_ids value, creation is being failed.
error:
The subnet ID 'subnet-0***********,subnet-0*************' does not exist
editing subnets*
vpc.tf
private_subnets = "10.10.20.#/#,10.10.20.#/#"
instanceec2.tf
subnet_id = "${module.vpc.private_subnets}"
below are modules:
vpc_main.tf
// Private subnet/s
resource "aws_subnet" "private" {
vpc_id = "${aws_vpc.vpc.id}"
cidr_block = "${element(split(",", var.private_subnets), count.index)}"
availability_zone = "${element(split(",", var.azs), count.index)}"
count = "${length(split(",", var.private_subnets))}"
tags {
Name = "${var.name}-private-${element(split(",", var.azs), count.index)}"
Team = "${var.team}"
Environment = "${var.environment}"
Service = "${var.service}"
Product = "${var.product}"
Owner = "${var.owner}"
Description = "${var.description}"
managed_by = "terraform"
}
}
resource "aws_route_table" "private" {
vpc_id = "${aws_vpc.vpc.id}"
count = "${length(split(",", var.private_subnets))}"
tags {
Name = "${var.name}-private-${element(split(",", var.azs), count.index)}"
Team = "${var.team}"
Environment = "${var.environment}"
Service = "${var.service}"
Product = "${var.product}"
Owner = "${var.owner}"
Description = "${var.description}"
managed_by = "terraform"
}
}
resource "aws_route_table_association" "private" {
subnet_id = "${element(aws_subnet.private.*.id, count.index)}"
route_table_id = "${element(aws_route_table.private.*.id, count.index)}"
count = "${length(split(",", var.private_subnets))}"
}
``````
vpc_outputs.tf
```````
output "private_subnets" {
value = "${join(",", aws_subnet.private.*.id)}"
}
Expected value is only one subnet ID as value:
Error: supply 2 Subnet IDs as one value.
aws_instance.ec2-instance[0]: 1 error(s) occurred:
aws_instance.ec2-instance.0: Error launching source instance: InvalidSubnetID.NotFound: The subnet ID 'subnet-0**********,subnet-0***********' does not exist
you are joining the subnet IDs in your output variable:
output "private_subnets" {
value = "${join(",", aws_subnet.private.*.id)}"
}
When you access this output value from your instanceec2.tf you will only receive this joined string of IDs.
So, you again have to slipt the received value as you've done before and access the respective individual ID with your count index of the ec2 resource:
resource "aws_instance" "default" {
count = "${length(split(",", module.vpc.private_subnets))}"
subnet_id = "${element(split(",", module.vpc.private_subnets), count.index)}"
....
}
That should solve you're issue.
Alternatively, you can also output the subnet IDs directly as a list:
output "private_subnets" {
description = "The IDs of the private subnets as list"
value = ["${aws_subnet.private.*.id}"]
}
and then access them with:
subnet_id = "${element(module.vpc.private_subnets, count.index)}"
Since you have 'join'ed the result, you would have to split again if you require just one subnet value.
Something like:
element(split(",", var.private_subnets), 0)