I added a launch template to IaC and and when I run the batch job it stuck in runnable state
this my launch template code :
resource "aws_launch_template" "conversion_launch_template" {
name = "conversion-launch-template"
block_device_mappings {
device_name = "/dev/sda1"
ebs {
volume_size = 80
}
}
iam_instance_profile {
name = "conversion-pipeline-batch-iam-instance-profile"
}
}
and in "aws_batch_compute_environment" resource I reference the launch template inside the compute ressource block :
resource "aws_batch_compute_environment" "conversion_pipeline" {
compute_environment_name = "conversion-pipeline-batch-compute-environment"
compute_resources {
instance_role = aws_iam_instance_profile.conversion_pipeline_batch.arn
instance_type = var.conversion_pipeline_instance_type
max_vcpus = var.conversion_pipeline_max_vcpus
min_vcpus = 0
security_group_ids = [
aws_security_group.conversion_pipeline_batch.id
]
subnets = var.subnets
type = "EC2"
launch_template {
launch_template_id = aws_launch_template.conversion_launch_template.id
}
}
service_role = aws_iam_role.conversion_pipeline_batch_service_role.arn
type = "MANAGED"
tags = {
environment = var.env
}
}
Related
I am trying to create EKS cluster with maxpods limit =110
Creating node group using aws_eks_node_group
resource "aws_eks_node_group" "eks-node-group" {
cluster_name = var.cluster-name
node_group_name = var.node-group-name
node_role_arn = var.eks-nodes-role.arn
subnet_ids = var.subnet-ids
version = var.cluster-version
release_version = nonsensitive(data.aws_ssm_parameter.eks_ami_release_version.value)
capacity_type = "SPOT"
lifecycle {
create_before_destroy = true
}
scaling_config {
desired_size = var.scale-config.desired-size
max_size = var.scale-config.max-size
min_size = var.scale-config.min-size
}
instance_types = var.scale-config.instance-types
update_config {
max_unavailable = var.update-config.max-unavailable
}
depends_on = [var.depends-on]
launch_template {
id = aws_launch_template.node-group-launch-template.id
version = aws_launch_template.node-group-launch-template.latest_version
}
}
resource "aws_launch_template" "node-group-launch-template" {
name_prefix = "eks-node-group"
image_id = var.template-image-id
block_device_mappings {
device_name = "/dev/sda1"
ebs {
volume_size = var.ebs_size
}
}
ebs_optimized = true
user_data = base64encode(data.template_file.test.rendered)
# user_data = filebase64("${path.module}/example.sh")
}
data "template_file" "test" {
template = <<EOF
/etc/eks/bootstrap.sh ${var.cluster-name} --use-max-pods false --kubelet-extra-args '--max-pods=110'
EOF
}
Launch template is created just to provide bootstrap arguments. I have tried supplying the same in aws_eks_cluster resource as well
module "eks__user_data" {
source = "terraform-aws-modules/eks/aws//modules/_user_data"
version = "18.30.3"
cluster_name = aws_eks_cluster.metashape-eks.name
bootstrap_extra_args = "--use-max-pods false --kubelet-extra-args '--max-pods=110'"
}
but unable to achieve desired effect till now.
Trying to follow
https://docs.aws.amazon.com/eks/latest/userguide/cni-increase-ip-addresses.html. CNI driver is enabled 1.12 and all other configurations seems correct too.
I'm creating aws eks cluster via terraform, but when i create cluster nodegroup with launch template i got 2 launch templates - one is with name and settings that i specify and second is with random name but with setting that i specify. Only difference in this 2 launch templat is IAM instance profile that got 2nd group (that creates automatically).
If i trying to specify IAM instance profile in my group it gives error that i cannot use it here
Is i'm doing something wrong or it's normal that it's creates 2 launch template ?
# eks node launch template
resource "aws_launch_template" "this" {
name = "${var.eks_cluster_name}-node-launch-template"
instance_type = var.instance_types[0]
image_id = var.node_ami
block_device_mappings {
device_name = "/dev/xvda"
ebs {
volume_size = 80
volume_type = "gp3"
throughput = "125"
encrypted = false
iops = 3000
}
}
lifecycle {
create_before_destroy = true
}
network_interfaces {
security_groups = [data.aws_eks_cluster.this.vpc_config[0].cluster_security_group_id]
}
user_data = base64encode(templatefile("${path.module}/userdata.tpl", merge(local.userdata_vars, local.cluster_data)))
tags = {
"eks:cluster-name" = var.eks_cluster_name
"eks:nodegroup-name" = var.node_group_name
}
tag_specifications {
resource_type = "instance"
tags = {
Name = "${var.eks_cluster_name}-node"
"eks:cluster-name" = var.eks_cluster_name
"eks:nodegroup-name" = var.node_group_name
}
}
tag_specifications {
resource_type = "volume"
tags = {
"eks:cluster-name" = var.eks_cluster_name
"eks:nodegroup-name" = var.node_group_name
}
}
}
# eks nodes
resource "aws_eks_node_group" "this" {
cluster_name = aws_eks_cluster.this.name
node_group_name = var.node_group_name
node_role_arn = aws_iam_role.eksNodesGroup.arn
subnet_ids = data.aws_subnet_ids.private.ids
scaling_config {
desired_size = 1
max_size = 10
min_size = 1
}
update_config {
max_unavailable = 1
}
launch_template {
version = aws_launch_template.this.latest_version
id = aws_launch_template.this.id
}
lifecycle {
create_before_destroy = true
ignore_changes = [
scaling_config[0].desired_size
]
}
# Ensure that IAM Role permissions are created before and deleted after EKS Node Group handling.
# Otherwise, EKS will not be able to properly delete EC2 Instances and Elastic Network Interfaces.
depends_on = [
aws_iam_role_policy_attachment.AmazonEKSWorkerNodePolicy,
aws_iam_role_policy_attachment.AmazonEKS_CNI_Policy,
aws_iam_role_policy_attachment.AmazonEC2ContainerRegistryReadOnly,
aws_launch_template.this
]
}
Expecting that terraform will create one launch template
Change the attribute name by name_prefix
Use:
name_prefix = "${var.eks_cluster_name}-node-launch-template"
Instead of:
name = "${var.eks_cluster_name}-node-launch-template"
The best choice to create a Unique name for a launch template using prefix (in your case ${var.eks_cluster_name}) is the name_prefix attribute.
Read more here
I am unable to register the ec2 instance into the ecs cluster, I have created the cluster, service and registered the task into it. But the ec2 instance is not registered. I have given the userdata to register the instance into the cluster but unable to register it. I am attaching the files which are needed. Ec2 instance are provisioning just not registering to the ECS cluster. I am implementing module wise structure. I am attaching the screenshot at the end of the question
Autoscaling:
resource "aws_launch_configuration" "ec2" {
image_id = var.image_id
instance_type = var.instance_type
name = "ec2-${terraform.workspace}"
user_data = <<EOF
#!/bin/bash
echo 'ECS_CLUSTER=${var.cluster_name.name}' >> /etc/ecs/ecs.config
echo 'ECS_DISABLE_PRIVILEGED=true' >> /etc/ecs/ecs.config
EOF
key_name = var.key_name
iam_instance_profile = var.instance_profile
security_groups = [aws_security_group.webserver.id]
}
resource "aws_autoscaling_group" "asg" {
vpc_zone_identifier = var.public_subnet
desired_capacity = 2
max_size = 2
min_size = 2
health_check_grace_period = 300
launch_configuration = aws_launch_configuration.ec2.name
target_group_arns = [var.tg.arn]
}
resource "aws_security_group" "webserver" {
name = "webserver-${terraform.workspace}"
description = "Allow internet traffic"
vpc_id = var.vpc_id
ingress {
description = "incoming for ec2-instance"
from_port = 0
to_port = 0
protocol = -1
security_groups = [var.alb_sg]
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
tags = {
Name = "webserver-sg"
}
}
output "ec2_sg" {
value = aws_security_group.webserver.id
}
Cluster:
resource "aws_ecs_cluster" "cluster" {
name = "wordpress-${terraform.workspace}"
}
output "cluster" {
value = aws_ecs_cluster.cluster.id
}
output "cluster1" {
value = aws_ecs_cluster.cluster
}
Service:
resource "aws_ecs_service" "wordpress" {
name = "Wordpress-${terraform.workspace}"
cluster = var.cluster
task_definition = var.task.id
desired_count = 2
scheduling_strategy = "REPLICA"
load_balancer {
target_group_arn = var.tg.arn
container_name = "wordpress"
container_port = 80
}
deployment_controller {
type = "ECS"
}
}
Task:
data "template_file" "init" {
template = "${file("${path.module}/template/containerdef.json")}"
vars = {
rds_endpoint = "${var.rds_endpoint}"
name = "${var.name}"
username = "${var.username}"
password = "${var.password}"
}
}
resource "aws_ecs_task_definition" "task" {
family = "wordpress"
container_definitions = "${data.template_file.init.rendered}"
network_mode = "bridge"
requires_compatibilities = ["EC2"]
memory = "1GB"
cpu = "1 vCPU"
task_role_arn = var.task_execution.arn
}
main.tf
data "aws_availability_zones" "azs" {}
data "aws_ssm_parameter" "name" {
name = "Dbname"
}
data "aws_ssm_parameter" "password" {
name = "db_password"
}
module "my_vpc" {
source = "./modules/vpc"
vpc_cidr = var.vpc_cidr
public_subnet = var.public_subnet
private_subnet = var.private_subnet
availability_zone = data.aws_availability_zones.azs.names
}
module "db" {
source = "./modules/rds"
ec2_sg = "${module.autoscaling.ec2_sg}"
allocated_storage = var.db_allocated_storage
storage_type = var.db_storage_type
engine = var.db_engine
engine_version = var.db_engine_version
instance_class = var.db_instance_class
name = data.aws_ssm_parameter.name.value
username = data.aws_ssm_parameter.name.value
password = data.aws_ssm_parameter.password.value
vpc_id = "${module.my_vpc.vpc_id}"
public_subnet = "${module.my_vpc.public_subnets_ids}"
}
module "alb" {
source = "./modules/alb"
vpc_id = "${module.my_vpc.vpc_id}"
public_subnet = "${module.my_vpc.public_subnets_ids}"
}
module "task" {
source = "./modules/task"
name = data.aws_ssm_parameter.name.value
username = data.aws_ssm_parameter.name.value
password = data.aws_ssm_parameter.password.value
rds_endpoint = "${module.db.rds_endpoint}"
task_execution = "${module.role.task_execution}"
}
module "autoscaling" {
source = "./modules/autoscaling"
vpc_id = "${module.my_vpc.vpc_id}"
#public_subnet = "${module.my_vpc.public_subnets_ids}"
tg = "${module.alb.tg}"
image_id = var.image_id
instance_type = var.instance_type
alb_sg = "${module.alb.alb_sg}"
public_subnet = "${module.my_vpc.public_subnets_ids}"
instance_profile = "${module.role.instance_profile}"
key_name = var.key_name
cluster_name = "${module.cluster.cluster1}"
}
module "role" {
source = "./modules/Iam_role"
}
module "cluster" {
source = "./modules/Ecs-cluster"
}
module "service" {
source = "./modules/services"
cluster = "${module.cluster.cluster}"
tg = "${module.alb.tg}"
task = "${module.task.task}"
}
ec2-instance role:
resource "aws_iam_role" "container_instance" {
name = "container_instance-${terraform.workspace}"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "ec2.amazonaws.com"
},
"Effect": "Allow"
}
]
}
EOF
tags = {
tag-key = "tag-value"
}
}
resource "aws_iam_instance_profile" "ec2_instance_role" {
name = "iam_instance_profile-${terraform.workspace}"
role = aws_iam_role.container_instance.name
}
resource "aws_iam_role_policy_attachment" "ec2_instance" {
role = aws_iam_role.container_instance.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role"
}
Screenshot:
[![enter image description here][2]][2]
Based on the chat discussion.
The issue could be caused by using incorrect instance profile:
iam_instance_profile = var.instance_profile.name
The important thing is that, now the two instances are correctly registered with the cluster.
Sorry for long post but hope that will provide good background.
Do not know if that is a bug or my code is wrong. I want to create ECS cluster with EC2 spot instances with help of launch template and ASG. My code is as follows:
For ECS service, cluster, task definition:
resource "aws_ecs_cluster" "main" {
name = "test-ecs-cluster"
}
resource "aws_ecs_service" "ec2_service" {
for_each = data.aws_subnet_ids.all_subnets.ids
name = "myservice_${replace(timestamp(), ":", "-")}"
task_definition = aws_ecs_task_definition.task_definition.arn
cluster = aws_ecs_cluster.main.id
desired_count = 1
launch_type = "EC2"
health_check_grace_period_seconds = 10
load_balancer {
container_name = "test-container"
container_port = 80
target_group_arn = aws_lb_target_group.alb_ec2_ecs_tg.id
}
network_configuration {
security_groups = [aws_security_group.ecs_ec2.id]
subnets = [each.value]
assign_public_ip = "false"
}
ordered_placement_strategy {
type = "binpack"
field = "cpu"
}
}
resource "aws_ecs_task_definition" "task_definition" {
container_definitions = data.template_file.task_definition_template.rendered
family = "test-ec2-task-family"
execution_role_arn = aws_iam_role.ecs_task_exec_role_ec2_ecs.arn
task_role_arn = aws_iam_role.ecs_task_exec_role_ec2_ecs.arn
network_mode = "awsvpc"
memory = 1024
cpu = 1024
requires_compatibilities = ["EC2"]
lifecycle {
create_before_destroy = true
}
}
data "template_file" "task_definition_template" {
template = file("${path.module}/templates/task_definition.json.tpl")
vars = {
container_port = var.container_port
region = var.region
log_group = var.cloudwatch_log_group
}
}
Launch template:
resource "aws_launch_template" "template_for_spot" {
name = "test-spor-ecs-launch-template"
disable_api_termination = false
instance_type = "t3.small"
image_id = data.aws_ami.amazon_linux_2_ecs_optimized.id
key_name = "FrankfurtRegion"
user_data = data.template_file.user_data.rendered
vpc_security_group_ids = [aws_security_group.ecs_ec2.id]
monitoring {
enabled = var.enable_spot == "true" ? false : true
}
block_device_mappings {
device_name = "/dev/sda1"
ebs {
volume_size = 30
}
}
iam_instance_profile {
arn = aws_iam_instance_profile.ecs_instance_profile.arn
}
lifecycle {
create_before_destroy = true
}
}
data "template_file" "user_data" {
template = file("${path.module}/user_data.tpl")
vars = {
cluster_name = aws_ecs_cluster.main.name
}
}
ASG with scaling policy:
resource "aws_autoscaling_group" "ecs_spot_asg" {
name = "test-asg-for-ecs"
max_size = 4
min_size = 2
desired_capacity = 2
termination_policies = [
"OldestInstance"]
vpc_zone_identifier = data.aws_subnet_ids.all_subnets.ids
health_check_type = "ELB"
health_check_grace_period = 300
mixed_instances_policy {
instances_distribution {
on_demand_percentage_above_base_capacity = 0
spot_instance_pools = 2
spot_max_price = "0.03"
}
launch_template {
launch_template_specification {
launch_template_id = aws_launch_template.template_for_spot.id
version = "$Latest"
}
override {
instance_type = "t3.large"
}
override {
instance_type = "t3.medium"
}
override {
instance_type = "t3a.large"
}
override {
instance_type = "t3a.medium"
}
}
}
lifecycle {
create_before_destroy = true
}
}
resource "aws_autoscaling_policy" "ecs_cluster_scale_policy" {
autoscaling_group_name = aws_autoscaling_group.ecs_spot_asg.name
name = "test-ecs-cluster-scaling-policy"
policy_type = "TargetTrackingScaling"
adjustment_type = "ChangeInCapacity"
target_tracking_configuration {
target_value = 70
customized_metric_specification {
metric_name = "ECS-cluster-metric"
namespace = "AWS/ECS"
statistic = "Average"
metric_dimension {
name = aws_ecs_cluster.main.name
value = aws_ecs_cluster.main.name
}
}
}
}
EDIT:
I'm getting :
Error: InvalidParameterException: Creation of service was not idempotent. "test-ec2-service-qaz"
on ecs.tf line 5, in resource "aws_ecs_service" "ec2_service":
5: resource "aws_ecs_service" "ec2_service" {
EDIT2:
Changed ecs_service name to name = "myservice_${replace(timestamp(), ":", "-")}", still getting same error.
Was reading from other issues that it is becouse of usage lifecycle with create_before_destroy statement in ecs_service, but it is not declared in my code. Maybe it is something related to something else, can't say what.
Thanks #Marko E and #karnauskas on github with name = "myservice_${each.value}" was able to deploy three ECS services. With correction to sub-nets handling I was able to deploy all the "stuff" as required. Subnets:
data "aws_subnet_ids" "all_subnets" {
vpc_id = data.aws_vpc.default.id
}
data "aws_subnet" "subnets" {
for_each = data.aws_subnet_ids.all_subnets.ids
id = each.value
}
AWS EC2 instance creation is failing while creating a network interface in the aws_instance section. The configuration is following configuration as defined in Terraform Network Interfaces
Configuration.
On removing the network block the configuration works seamlessly. With network block the following error was logged
"Error: Error launching source instance: Unsupported: The requested configuration is currently not supported. Please check the documentation for supported configurations."
variable "aws_region" {}
variable "aws_access_key" {}
variable "aws_secret_key" {}
variable "vpc_cidr_block" {}
variable "environment" {}
variable "applicationtype" {}
variable "subnet_cidr_block" {}
variable "amiid" {}
variable "instancetype" {}
variable "bucketname" {}
variable "publickey-fe" {}
variable "publickey-be" {}
provider "aws" {
profile = "default"
region = "${var.aws_region}"
access_key = "${var.aws_access_key}"
secret_key = "${var.aws_secret_key}"
}
data "aws_availability_zones" "availability" {
state = "available"
}
resource "aws_vpc" "sitespeed_vpc" {
cidr_block = "${var.vpc_cidr_block}"
instance_tenancy = "dedicated"
tags = {
env = "${var.environment}"
application = "${var.applicationtype}"
Name = "site-speed-VPC"
}
}
resource "aws_subnet" "sitespeed_subnet" {
vpc_id = "${aws_vpc.sitespeed_vpc.id}"
cidr_block = "${var.subnet_cidr_block}"
availability_zone = "${data.aws_availability_zones.availability.names[0]}"
tags = {
env = "${var.environment}"
application = "${var.applicationtype}"
Name = "site-speed-Subnet"
}
}
resource "aws_network_interface" "sitespeed_frontend_NIC" {
subnet_id = "${aws_subnet.sitespeed_subnet.id}"
private_ips = ["192.168.10.100"]
tags = {
env = "${var.environment}"
application = "${var.applicationtype}"
Name = "site-speed-frontend-nic"
}
}
resource "aws_network_interface" "sitespeed_backend_NIC" {
subnet_id = "${aws_subnet.sitespeed_subnet.id}"
private_ips = ["192.168.10.110"]
tags = {
env = "${var.environment}"
application = "${var.applicationtype}"
Name = "site-speed-backend-nic"
}
}
resource "aws_key_pair" "sitespeed_front_key" {
key_name = "site_speed_front_key"
public_key = "${var.publickey-fe}"
}
resource "aws_key_pair" "sitespeed_back_key" {
key_name = "site_speed_back_key"
public_key = "${var.publickey-be}"
}
resource "aws_instance" "sitespeed_front" {
ami = "ami-00942d7cd4f3ca5c0"
instance_type = "t2.micro"
key_name = "site_speed_front_key"
availability_zone = "${data.aws_availability_zones.availability.names[0]}"
network_interface {
network_interface_id = "${aws_network_interface.sitespeed_frontend_NIC.id}"
device_index = 0
}
tags = {
env = "${var.environment}"
application = "${var.applicationtype}"
Name = "site-speed-frontend-server"
public = "yes"
}
}
resource "aws_instance" "sitespeed_backend" {
ami = "ami-00942d7cd4f3ca5c0"
instance_type = "t2.micro"
key_name = "site_speed_back_key"
network_interface {
network_interface_id = "${aws_network_interface.sitespeed_backend_NIC.id}"
device_index = 0
}
tags = {
env = "${var.environment}"
application = "${var.applicationtype}"
Name = "site-speed-backend-server"
public = "No"
}
}
resource "aws_s3_bucket" "b" {
bucket = "${var.bucketname}"
acl = "private"
tags = {
env = "${var.environment}"
application = "${var.applicationtype}"
}
}
The issue was due to the Terraform Version. Following is the updated script that supports Terraform V.0.12.16 to create an EC2 Instance on AWS.
// Variable Definition
variable "aws_region" {}
variable "aws_vpc_cidr_block" {}
variable "aws_subnet_cidr_block" {}
variable "aws_private_ip_fe" {}
variable "aws_Name" {}
variable "aws_Application" {}
variable "aws_ami" {}
variable "aws_instance_type" {}
// Provider Definition
provider "aws" {
version = "~> 2.40"
region = var.aws_region
}
// Adds a VPC
resource "aws_vpc" "aws_ec2_deployment_test-vpc" {
cidr_block = var.aws_vpc_cidr_block
tags = {
Name = join("-", [var.aws_Name, "vpc"])
Application = var.aws_Application
}
}
//Adds a subnet
resource "aws_subnet" "aws_ec2_deployment_test-subnet" {
vpc_id = aws_vpc.aws_ec2_deployment_test-vpc.id
cidr_block = var.aws_subnet_cidr_block
availability_zone = join("", [var.aws_region, "a"])
tags = {
Name = join("-", [var.aws_Name, "subnet"])
Application = var.aws_Application
}
}
//Adds a Network Interface
resource "aws_network_interface" "aws_ec2_deployment_test-fe" {
subnet_id = aws_subnet.aws_ec2_deployment_test-subnet.id
private_ips = [ var.aws_private_ip_fe ]
tags = {
Name = join("-", [var.aws_Name, "network-interface-fe"])
Application = var.aws_Application
}
}
//Adds an EC2 Instance
resource "aws_instance" "aws_ec2_deployment_test-fe"{
ami = var.aws_ami
instance_type = var.aws_instance_type
network_interface {
network_interface_id = aws_network_interface.aws_ec2_deployment_test-fe.id
device_index = 0
}
tags = {
Name = join("-", [var.aws_Name, "fe-ec2"])
Application = var.aws_Application
}
}
// Print Output Values
output "aws_ec2_deployment_test-vpc" {
description = "CIDR Block for the VPC: "
value = aws_vpc.aws_ec2_deployment_test-vpc.cidr_block
}
output "aws_ec2_deployment_test-subnet" {
description = "Subnet Block: "
value = aws_subnet.aws_ec2_deployment_test-subnet.cidr_block
}
output "aws_ec2_deployment_test-private-ip" {
description = "System Private IP: "
value = aws_network_interface.aws_ec2_deployment_test-fe.private_ip
}
output "aws_ec2_deployment_test-EC2-Details" {
description = "EC2 Details: "
value = aws_instance.aws_ec2_deployment_test-fe.public_ip
}
Gist link to the solution