Using terraform I'm provisioning infra in AWS for my K3S cluster. I have provisioned an NLB with two listeners on port 80 and 443, with appropriate self-signed certs. This works. I can access HTTP services in my cluster via the nlb.
resource "tls_private_key" "agents" {
algorithm = "RSA"
}
resource "tls_self_signed_cert" "agents" {
key_algorithm = "RSA"
private_key_pem = tls_private_key.agents.private_key_pem
validity_period_hours = 24
subject {
common_name = "my hostname"
organization = "My org"
}
allowed_uses = [
"key_encipherment",
"digital_signature",
"server_auth"
]
}
resource "aws_acm_certificate" "agents" {
private_key = tls_private_key.agents.private_key_pem
certificate_body = tls_self_signed_cert.agents.cert_pem
}
resource "aws_lb" "agents" {
name = "basic-load-balancer"
load_balancer_type = "network"
subnet_mapping {
subnet_id = aws_subnet.agents.id
allocation_id = aws_eip.agents.id
}
}
resource "aws_lb_listener" "agents_80" {
load_balancer_arn = aws_lb.agents.arn
protocol = "TCP"
port = 80
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.agents_80.arn
}
}
resource "aws_lb_listener" "agents_443" {
load_balancer_arn = aws_lb.agents.arn
protocol = "TLS"
port = 443
certificate_arn = aws_acm_certificate.agents.arn
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.agents_443.arn
}
}
resource "aws_lb_target_group" "agents_80" {
port = 30000
protocol = "TCP"
vpc_id = var.vpc.id
depends_on = [
aws_lb.agents
]
}
resource "aws_lb_target_group" "agents_443" {
port = 30001
protocol = "TCP"
vpc_id = var.vpc.id
depends_on = [
aws_lb.agents
]
}
resource "aws_autoscaling_attachment" "agents_80" {
autoscaling_group_name = aws_autoscaling_group.agents.name
alb_target_group_arn = aws_lb_target_group.agents_80.arn
}
resource "aws_autoscaling_attachment" "agents_443" {
autoscaling_group_name = aws_autoscaling_group.agents.name
alb_target_group_arn = aws_lb_target_group.agents_443.arn
}
That's a cutdown version of my code.
I have configured my ingress controller to listen for HTTP and HTTPS on NodePorts 30000 and 30001 respectively. This works too.
The thing that doesn't work is that the NLB is terminating TLS, but I need it to passthrough. I'm doing this so that I can access Kubernetes Dashboard (among other apps), but the dashboard requires https to sign-in, something I can't provide if tls is terminated at the nlb.
I need help configuring the nlb for passthrough. I have searched and searched and can't find any examples. If anyone knows how to configure this it would be good to get some tf code, or even just an idea of the appropriate way of achieving it in AWS so that I can implement it myself in tf.
Do you need TLS passthrough, or just TLS communication between the NLB and the server? Or do you just need to configure your server to be aware that the initial connection was TLS?
For TLS passthrough you would install an SSL certificate on the server, and delete the certificate from the load balancer. You would change the protocol of the port 443 listener on the load balancer from "TLS" to "TCP". This is not a very typical setup on AWS, and you can't use the free AWS ACM SSL certificates in this configuration, you would have to use something like Let's Encrypt on the server.
For TLS communication between the NLB and the server, you would install a certificate on the server, a self-signed cert is fine for this, and then just change the target group settings on the load balancer to point to the secure ports on the server.
If you just want to make the server aware that the initial connection protocol was TLS, you would configure the server to use the x-forwarded-proto header passed by the load balancer to determine if the connection is secure.
Related
I have windows Ec2 instance hosting a web site in IIS and also using ALB. I have following configuration in AWS
https://mywebsite.mydomain.com -> Application Load Balancer -> Listener (443) -> Target Group -> Windows Ec2 Instance (IIS)
I have SSL certificate configured on ALB so that all the requests from users are always on HTTPS. However, the internal traffic from ALB to Ec2 is on port 80 (HTTP)
I want to configure internal traffic (ALB -> EC2) on HTTPS using self signed certificate. I can create a self signed certificate and configure in IIS. However, I am not sure what base route the target group is using to forward the traffic to instance? Does it use private ip or machine name or something else? What should be the value for dns name for the certificate?
$dnsname= "xxxxxx"
$cert = New-SelfSignedCertificate -DnsName "$dnsname" -CertStoreLocation "cert:\LocalMachine\My"
Here is terraform for the target group
resource "aws_lb_target_group" "https" {
name = "my-tg-https"
port = 443
protocol = "HTTPS"
target_type = "instance"
vpc_id = var.vpc_id
deregistration_delay = var.deregistration_delay
health_check {
path = "/"
interval = 30
healthy_threshold = 2
unhealthy_threshold = 2
timeout = 5
matcher = 200
port = 443
protocol = "HTTPS"
}
lifecycle {
create_before_destroy = true
}
}
We are creating infrastructure on GCP for our application which uses SSL Porxy Load Balancer on GCP. We use Terraform for our deployments and are struggling to create SSL Proxy Load Balancer via terraform.
If anyone could point to a sample code or point me to a direction where I can find some resources to create the load balancer
You can try with the following example:
resource "google_compute_target_ssl_proxy" "default" {
name = "test-proxy"
backend_service = google_compute_backend_service.default.id
ssl_certificates = [google_compute_ssl_certificate.default.id]
}
resource "google_compute_ssl_certificate" "default" {
name = "default-cert"
private_key = file("path/to/private.key")
certificate = file("path/to/certificate.crt")
}
resource "google_compute_backend_service" "default" {
name = "backend-service"
protocol = "SSL"
health_checks = [google_compute_health_check.default.id]
}
resource "google_compute_health_check" "default" {
name = "health-check"
check_interval_sec = 1
timeout_sec = 1
tcp_health_check {
port = "443"
}
}
Take onto consideration that the Health Check is pointed to port 443/tcp, if you want a different port please change that on here.
I'm confused with configuring LoadBalancer(NLB) in AWS. When configuring the LB as below (it's the Terraform file), I never specified HTTPS protocol. However, after the LB gets spinned up, I can only reach my targets by https://LB_ARN:80 and nothing is shown when I hit http://LB_ARN:80. I am quite confused of the reason, and also, more confusing part is that using https://LB_ARN:80 as DNS, my Browser(Chrome) tells me the site is not secure (though it is HTTPS). Any help please ?
resource "aws_lb" "boundary" {
name = "boundary-nlb"
load_balancer_type = "network"
internal = false
subnets = data.terraform_remote_state.network.outputs.tokyo_vpc_main.public_subnet_ids
tags = merge(local.common_tags, {
Name = "boundary-${terraform.workspace}-controller-nlb"
})
}
resource "aws_lb_target_group" "boundary" {
name = "boundary-nlb"
port = 9200
protocol = "TCP"
vpc_id = data.terraform_remote_state.network.outputs.tokyo_vpc_main.vpc_id
stickiness {
enabled = false
type = "source_ip"
}
tags = merge(local.common_tags, {
Name = "boundary-${terraform.workspace}-controller-nlb-tg"
})
}
resource "aws_lb_target_group_attachment" "boundary" {
count = var.num_controllers
target_group_arn = aws_lb_target_group.boundary.arn
target_id = aws_instance.controller[count.index].id
port = 9200
}
resource "aws_lb_listener" "boundary" {
load_balancer_arn = aws_lb.boundary.arn
port = "80"
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.boundary.arn
}
}
resource "aws_security_group" "boundary_lb" {
vpc_id = data.terraform_remote_state.network.outputs.tokyo_vpc_main.vpc_id
tags = merge(local.common_tags, {
Name = "boundary-${terraform.workspace}-controller-nlb-sg"
})
}
resource "aws_security_group_rule" "allow_9200" {
type = "ingress"
from_port = 9200
to_port = 9200
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"]
security_group_id = aws_security_group.boundary_lb.id
}
This appears to me as misconfiguration of your backend servers. Specifically, they seem to server HTTPS connections on port 80.
Since you are using NLB with TCP protocol, any HTTPS connection is forwarded to your backend servers. Meaning, there is no SSL termination on your NLB. So even though you haven't specified HTTPS in your NLB settings, HTTPS connections are forwarded on top of TCP to your backend instances. The backend instance handle the HTTPS with maybe self-signed SSL certificate, not NLB, on the wrong port. This would explain warnings from browser.
I would recommend checking configuration of your backend services and making sure that are serving HTTP on port 80, not HTTPS as it seems to be the case now.
I am trying to provision an ECS cluster using Terraform along with an ALB. The targets come up as Unhealthy. The error code is 502 in the console Health checks failed with these codes: [502]
I checked through the AWS Troubleshooting guide and nothing helped there.
EDIT: I have no services/tasks running on the EC2 containers. Its a vanilla ECS cluster.
Here is my relevant code for the ALB:
# Target Group declaration
resource "aws_alb_target_group" "lb_target_group_somm" {
name = "${var.alb_name}-default"
port = 80
protocol = "HTTP"
vpc_id = "${var.vpc_id}"
deregistration_delay = "${var.deregistration_delay}"
health_check {
path = "/"
port = 80
protocol = "HTTP"
}
lifecycle {
create_before_destroy = true
}
tags = {
Environment = "${var.environment}"
}
depends_on = ["aws_alb.alb"]
}
# ALB Listener with default forward rule
resource "aws_alb_listener" "https_listener" {
load_balancer_arn = "${aws_alb.alb.id}"
port = "80"
protocol = "HTTP"
default_action {
target_group_arn = "${aws_alb_target_group.lb_target_group_somm.arn}"
type = "forward"
}
}
# The ALB has a security group with ingress rules on TCP port 80 and egress rules to anywhere.
# There is a security group rule for the EC2 instances that allows ingress traffic to the ECS cluster from the ALB:
resource "aws_security_group_rule" "alb_to_ecs" {
type = "ingress"
/*from_port = 32768 */
from_port = 80
to_port = 65535
protocol = "TCP"
source_security_group_id = "${module.alb.alb_security_group_id}"
security_group_id = "${module.ecs_cluster.ecs_instance_security_group_id}"
}
Has anyone hit this error and know how to debug/fix this ?
It looks like you're trying to be register the ECS cluster instances with the ALB target group. This isn't how you're meant to send traffic to an ECS service via an ALB.
Instead you should have your service join the tasks to the target group. This will mean that if you are using host networking then only the instances with the task deployed will be registered. If you are using bridge networking then it will add the ephemeral ports used by your task to your target group (including allowing for there to be multiple targets on a single instance). And if you are using awsvpc networking then it will register the ENIs of every task that the service spins up.
To do this you should use the load_balancer block in the aws_ecs_service resource. An example might look something like this:
resource "aws_ecs_service" "mongo" {
name = "mongodb"
cluster = "${aws_ecs_cluster.foo.id}"
task_definition = "${aws_ecs_task_definition.mongo.arn}"
desired_count = 3
iam_role = "${aws_iam_role.foo.arn}"
load_balancer {
target_group_arn = "${aws_lb_target_group.lb_target_group_somm.arn}"
container_name = "mongo"
container_port = 8080
}
}
If you were using bridge networking this would mean that the tasks are accessible on the ephemeral port range on the instances so your security group rule would need to look like this:
resource "aws_security_group_rule" "alb_to_ecs" {
type = "ingress"
from_port = 32768 # ephemeral port range for bridge networking tasks
to_port = 60999 # cat /proc/sys/net/ipv4/ip_local_port_range
protocol = "TCP"
source_security_group_id = "${module.alb.alb_security_group_id}"
security_group_id = "${module.ecs_cluster.ecs_instance_security_group_id}"
}
it looks like the http://ecsInstanceIp:80 is not returning HTTP 200 OK. I would check that first. It would be easy to check if the instance is public. It wont be the case most of the times. Otherwise I would create an EC2 instance and make a curl request to confirm that.
You may also check the container logs to see if its logging the health check response.
Hope this helps. good luck.
I am trying to use API Gateway’s VPC links to route traffic to an internal API on HTTPS.
But, VPC links forces me to change my API’s load balancer from "application" to "network".
I understand that the network load balancer is on layer 4 and as such will not know about HTTPS.
I am used to using the layer 7 application load balancer. As such, I am not sure how I should configure or indeed use the network load balancer in terraform.
Below is my attempt at configuring the network load balancer in Terraform.
The health check fails and I'm not sure what I am doing wrong.
resource "aws_ecs_service" “app” {
name = "${var.env}-${var.subenv}-${var.appname}"
cluster = "${aws_ecs_cluster.cluster.id}"
task_definition = "${aws_ecs_task_definition.app.arn}"
desired_count = "${var.desired_app_count}"
deployment_minimum_healthy_percent = 50
deployment_maximum_percent = 200
iam_role = "arn:aws:iam::${var.account}:role/ecsServiceRole"
load_balancer {
target_group_arn = "${aws_lb_target_group.app-lb-tg.arn}"
container_name = "${var.env}-${var.subenv}-${var.appname}"
container_port = 9000
}
depends_on = [
"aws_lb.app-lb"
]
}
resource "aws_lb" “app-lb" {
name = "${var.env}-${var.subenv}-${var.appname}"
internal = false
load_balancer_type = "network"
subnets = "${var.subnet_ids}"
idle_timeout = 600
tags {
Owner = ""
Env = "${var.env}"
}
}
resource "aws_lb_listener" “app-lb-listener" {
load_balancer_arn = "${aws_lb.app-lb.arn}"
port = 443
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = "${aws_lb_target_group.app-lb-tg.arn}"
}
}
resource "aws_lb_target_group" “app-lb-tg" {
name = "${var.env}-${var.subenv}-${var.appname}"
port = 443
stickiness = []
health_check {
path = "/actuator/health"
}
protocol = "TCP"
vpc_id = "${var.vpc_id}"
}
For reference, this is how I previously configured my application load balancer before attempting to switch to a network load balancer:
resource "aws_ecs_service" "app" {
name = "${var.env}-${var.subenv}-${var.appname}"
cluster = "${aws_ecs_cluster.cluster.id}"
task_definition = "${aws_ecs_task_definition.app.arn}"
desired_count = "${var.desired_app_count}"
deployment_minimum_healthy_percent = 50
deployment_maximum_percent = 200
iam_role = "arn:aws:iam::${var.account}:role/ecsServiceRole"
load_balancer {
target_group_arn = "${aws_lb_target_group.app-alb-tg.arn}"
container_name = "${var.env}-${var.subenv}-${var.appname}"
container_port = 9000
}
depends_on = [
"aws_alb.app-alb"]
}
resource "aws_alb" "app-alb" {
name = "${var.env}-${var.subenv}-${var.appname}"
subnets = "${var.subnet_ids}"
security_groups = [
"${var.vpc_default_sg}",
"${aws_security_group.app_internal.id}"]
internal = false
idle_timeout = 600
tags {
Owner = ""
Env = "${var.env}"
}
}
resource "aws_lb_listener" "app-alb-listener" {
load_balancer_arn = "${aws_alb.app-alb.arn}"
port = "443"
protocol = "HTTPS"
ssl_policy = "ELBSecurityPolicy-2015-05"
certificate_arn = "${var.certificate_arn}"
default_action {
type = "forward"
target_group_arn = "${aws_lb_target_group.app-alb-tg.arn}"
}
}
resource "aws_lb_target_group" "app-alb-tg" {
name = "${var.env}-${var.subenv}-${var.appname}"
port = 80
health_check {
path = "/actuator/health"
}
protocol = "HTTP"
vpc_id = "${var.vpc_id}"
}
A network load balancer automatically performs passive health checks on non UDP traffic that flows through it so if that's enough then you can just remove the active health check configuration.
If you want to enable active health checks then you can either use TCP health checks (the default) which will just check that the port is open or you can specify the HTTP/HTTPS protocol and specify a path. Ideally the AWS API would error when you try to specify a path for the health check but don't set the protocol to HTTP or HTTPS but apparently that's not the case right now.
With Terraform this would look something like this:
resource "aws_lb_target_group" "app-alb-tg" {
name = "${var.env}-${var.subenv}-${var.appname}"
port = 443
protocol = "TCP"
vpc_id = "${var.vpc_id}"
health_check {
path = "/actuator/health"
protocol = "HTTPS"
}
}
Remember that active health checks which will check that the port is open on the target from the network load balancer's perspective (not just the source traffic). This means that your target will need to allow traffic from the subnets that your NLBs reside in as well as security groups or CIDR ranges etc that your source traffic originates in.