I'm following terraform documentation to create Aurora serverless v2 by terraform. https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster#rds-serverless-v2-cluster
In terraform documentation, they have not mentioned how to create Aurora serverless v2 with multi AZ (read replica in other region for failover). Although, by aws console, I can create multi AZ Aurora serverless v2.
Any help is appreciated to create Aurora serverless v2 with multi AZ by terraform?
You can't set it up becasuse by default your data is stored across multiple AZs. From docs:
The storage for each Aurora DB cluster consists of six copies of all your data, spread across three AZs. This built-in data replication applies regardless of whether your DB cluster includes any readers in addition to the writer. That way, your data is safe, even from issues that affect the compute capacity of the cluster.
I found a solution. I am using a custom module created from https://github.com/terraform-aws-modules/terraform-aws-rds-aurora and also I saw one more link https://github.com/hashicorp/terraform-provider-aws/issues/24502. In this link, check the answer from msbainuk. You have to define additional instance, which will automatically create it as a Multi AZ. PFB the Code snippet from his answer.
resource "aws_rds_cluster_instance" "cluster_instances" {
cluster_identifier = aws_rds_cluster.this.id
instance_class = "db.serverless"
engine = aws_rds_cluster.this.engine
engine_version = aws_rds_cluster.this.engine_version
}
Hope this helps.
You should create multiple aws_rds_cluster_instances. Here's an example from the docs:
resource "aws_rds_cluster_instance" "cluster_instances" {
count = 2
identifier = "aurora-cluster-demo-${count.index}"
cluster_identifier = aws_rds_cluster.default.id
instance_class = "db.r4.large"
engine = aws_rds_cluster.default.engine
engine_version = aws_rds_cluster.default.engine_version
}
resource "aws_rds_cluster" "default" {
cluster_identifier = "aurora-cluster-demo"
availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]
database_name = "mydb"
master_username = "foo"
master_password = "barbut8chars"
}
I believe the AWS console UI is masking over that. It's going to create 2x instances if you select "multi-az".
Related
I want to create a EKS cluster using Terraform, build custom docker images and then perform Kubernetes deployments on the created cluster via terraform. I want to perform all the tasks with a single terraform apply. But I see that the kubernetes provider needs the details of cluster on initialization itself. Is there a way I can achieve both cluster creation and deployment using a single terraform apply, so that once the cluster is created, the cluster details can be passed to Kubernetes provider and then the pods are deployed.
Please let me know how I can achieve this?
I am doing this with ease , below is the pseudo code , you just need to be careful with the way you are using depends_on attribute with resources and try to encapsulate as much as possible
Kubernetes Provider in a separate file like kubernetes.tf
host = module.eks.cluster_endpoint
cluster_ca_certificate = base64decode(module.eks.cluster_certificate_authority_data)
exec {
api_version = "client.authentication.k8s.io/v1alpha1"
command = "aws"
args = [
"eks",
"get-token",
"--cluster-name",
module.eks.cluster_id
]
}
}
Assuming you've got the network setup, here I am relying on implicit dependencies rather than an explicit one.
module "eks" {
source = "./modules/ekscluster"
clustername = module.clustername.cluster_name
eks_version = var.eks_version
private_subnets = module.networking.private_subnets_id
vpc_id = module.networking.vpc_id
environment = var.environment
instance_types = var.instance_types
}
Creating k8s resources using the depends_on attribute.
resource "kubernetes_namespace_v1" "app" {
metadata {
annotations = {
name = var.org_name
}
name = var.org_name
}
depends_on = [module.eks]
}
I'm quite new to Terraform, and struggling with something.
I'm playing around with Redshift for a personal project, and I want to update the inbound security rules for the default security group which is applied to Redshift when it's created.
If I were doing it in AWS Console, I'd be adding a new inbound rule with Type being All Traffic and Source being Anywhere -IPv4 which adds 0.0.0.0/0.
Below in main.tf I've tried to create a new security group and apply that to Redshift, but I get a VPC-by-Default customers cannot use cluster security groups error.
What is it I'm doing wrong?
resource "aws_redshift_cluster" "redshift" {
cluster_identifier = "redshift-cluster-pipeline"
skip_final_snapshot = true terraform destroy
master_username = "awsuser"
master_password = var.db_password
node_type = "dc2.large"
cluster_type = "single-node"
publicly_accessible = "true"
iam_roles = [aws_iam_role.redshift_role.arn]
cluster_security_groups = [aws_redshift_security_group.redshift-sg.name]
}
resource "aws_redshift_security_group" "redshift-sg" {
name = "redshift-sg"
ingress {
cidr = "0.0.0.0/0"
}
The documentation for the Terraform resource aws_redshift_security_group states:
Creates a new Amazon Redshift security group. You use security groups
to control access to non-VPC clusters
The error message you are receiving is clearly staging that you are using the wrong type of security group, and you need to use a VPC security group instead. Once you create the appropriate VPC security group, you would set it in the aws_redshift_cluster resource via the vpc_security_group_ids property.
I'm new Terraform and I'm working on an infrastructure setup for deploying Docker Containers. I've based my ECS Cluster off Infrablocks/ECS-Cluster and my Base Networking on Infrablocks/Base-Network. I've opted to use these due to time constraints on the project.
The problem I'm having is that the two EC2 Container Instances that are created by Infrablocks/ECS-Cluster module are not associated with ECS Cluster that Infrablocks builds. I've had zero luck determining why. This is blocking my task definitions from being able to run containers in the ECS Cluster because there are no associated EC2 Instances. I've provided my two dependent module configurations below.
Thank you in advance for any help you can provide!
My Terraform thus far:
module "base_network" {
source = "infrablocks/base-networking/aws"
version = "2.3.0"
vpc_cidr = "10.0.0.0/16"
region = "us-east-1"
availability_zones = ["us-east-1a", "us-east-1b"]
component = "dev-base-network"
deployment_identifier = "development"
include_route53_zone_association = "true"
private_zone_id = module.route53.private_zone_id
include_nat_gateway = "true"}
module "ecs_cluster" {
source = "infrablocks/ecs-cluster/aws"
version = "2.2.0"
region = "us-east-1"
vpc_id = module.base_network.vpc_id
subnet_ids = module.base_network.public_subnet_ids
associate_public_ip_addresses = "yes"
component = "dev"
deployment_identifier = "devx"
cluster_name = "services"
cluster_instance_ssh_public_key_path = "~/.ssh/id_rsa.pub"
cluster_instance_type = "t2.small"
cluster_minimum_size = 2
cluster_maximum_size = 10
cluster_desired_capacity = 2 }
You'd have to troubleshoot the instance to see why it isn't joining the cluster. On your EC2 instances (which, I have not looked, but I would hope that the "infrablocks" ecs-cluster module uses an AMI with the ECS agent installed), you can look in /var/log/ecs/ecs-agent.log .
If the networking configuration is sound, my first guess would be to check the ECS configuration file. If your module is working properly, it should have populated the config with the cluster name. See here for more on that
(I would have commented instead of answered but this account doesn't have enough rep :shrug:)
I am trying to spin up an ECS cluster with Terraform, but can not make EC2 instances register as container instances in the cluster.
I first tried with the verified module from Terraform, but this seems out dated (ecs-instance-profile has wrong path).
Then I tried with another module from anrim, but still no container instances. Here is the script I used:
provider "aws" {
region = "us-east-1"
}
module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "2.21.0"
name = "ecs-alb-single-svc"
cidr = "10.10.10.0/24"
azs = ["us-east-1a", "us-east-1b", "us-east-1c"]
private_subnets = ["10.10.10.0/27", "10.10.10.32/27", "10.10.10.64/27"]
public_subnets = ["10.10.10.96/27", "10.10.10.128/27", "10.10.10.160/27"]
tags = {
Owner = "user"
Environment = "me"
}
}
module "ecs_cluster" {
source = "../../modules/cluster"
name = "ecs-alb-single-svc"
vpc_id = module.vpc.vpc_id
vpc_subnets = module.vpc.private_subnets
tags = {
Owner = "user"
Environment = "me"
}
}
I then created a new ecs cluster (from the aws console) on the same VPC and carefully compared the differences in resources. I managed to find some small differences, fixed them and tried again. But still no container instances!
A fork of the module is available here.
Can you see instances being created in the autoscaling group? If so, I'd suggest SSHing to one of them (either directly or using a bastion host, eg. see this module) and checking ECS agent logs. In my experience those problems are usually related to IAM policies, and that's pretty visible in logs but YMMV.
I'm brand new to Terraform so I'm sure i'm missing something, but the answers i'm finding don't seem to be asking the same question I have.
I have an AWS VPC/Security Group that we need our EC2 instances to be created under and this VPC/SG is already created. To create an EC2 instance, Terraform requires that if I don't have a default VPC, I must import my own. But once I import and apply my plan, when I wish to destroy it, its trying to destroy my VPC as well. How do I encapsulate my resources so when I run "terraform apply", I can create an EC2 instance with my imported VPC, but when I run "terraform destroy" I only destroy my EC2 instance?
In case anyone wants to mention, I understand that:
lifecycle = {
prevent_destroy = true
}
is not what I'm looking for.
Here is my current practice code.
resource "aws_vpc" "my_vpc" {
cidr_block = "xx.xx.xx.xx/24"
}
provider "aws" {
region = "us-west-2"
}
data "aws_ami" "ubuntu" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-*"]
}
owners = ["099720109477"] # Canonical
}
resource "aws_instance" "web" {
ami = "${data.aws_ami.ubuntu.id}"
instance_type = "t3.nano"
vpc_security_group_ids = ["sg-0e27d851dxxxxxxxxxx"]
subnet_id = "subnet-0755c2exxxxxxxx"
tags = {
Name = "HelloWorld"
}
}
Terraform should not require you to deploy or import a VPC in order to deploy an EC2 instance into it. You should be able to reference the VPC, subnets and security groups by id so TF is aware of your existing network infrastructure just like you've already done for SGs and subnets. All you should need to deploy the EC2 instance "aws_instance" is give it an existing subnet id in the existing VPC like you already did. Why do you say deploying or importing a VPC is required by Terraform? What error or issue do you have deploying without the VPC and just using the existing one?
You can protect the VPC through AWS if you really wanted to, but I don't think you really want to import the VPC into your Terraform state and let Terraform manage it here. Sounds like you want the VPC to service other resources, maybe applications manually deployed or through other TF stacks, and the VPC to live independent of anyone application deployment.
To answer the original question, you can use a data source and match your VPC by id or tag name :
data "aws_vpc" "main" {
tags = {
Name = "main_vpc"
}
}
Or
data "aws_vpc" "main" {
id = "vpc-nnnnnnnn"
}
Then refer to it with : data.aws_vpc.main
Also, if you already included your VPC but would like not to destroy it while remove it from your state, you can manage to do it with the terraform state command : https://www.terraform.io/docs/commands/state/index.html