AWS ECS Fargate dynamically register IP to target group - amazon-web-services

I use AWS at work and I am fairly new to this.
I have multiple Services with one Task/Container running. Each Container is fundamentally the same with a few changes, it's basically for different stages/deployments. I have one target group for each, so my load balancer routes requests from specific domains to each.
For example: if host is example1.com then forward to exampleTargetGroup1 and so on.
The Problem
As you may know each time a container is updated, its IP changes, hence I have to re-register the new IP to the target group
I have found several approaches to this problem. Most of them suggest to use a Network Load Balancer for a static IP, but this doesn't work because, as I understand it, it registers the containers automatically on updates.
Another solution is to trigger a Lambda function on a cloud watch events when the Task is being updated. The function grabs the IP and updates the Route53 record. My Idea was to take this approach and deregister the old IP in the target group and register the new one.
My Questions
Is there a better solution to this or did I understand the first solutions wrong? If the last solution is optimal for my problem is there maybe a code sample so I won't need to figure it out?
EDIT:
Thanks to Mark B I now know, you should preferably use the AWS API or a tool like Terraform to create an ECS Service and associating a target group to it.

"but this doesn't work because, as I understand it, it registers the
containers automatically on updates."
I think you are misunderstanding something here. Each ECS service should be associated with a load balancer Target Group. Whenever the service creates a task, the service will automatically add that task's IP to the target group. Whenever the service removes a task, it will also remove that task's IP from the target group. This works with both Network Load Balancers and Application Load Balancers.
You stated the following:
"I have multiple Services with one Task/Container running"
So you have one task per service, and one service per target group. From your description, your architecture should look like this:
One load Balancer with multiple domains pointing at it.
In the Load Balancer listener configuration, you have each domain configured to route to a different target group.
Each ECS service configured with a task count of 1
Load balancer -> domain name 1 -> target group 1 -> ECS service 1 -> ECS task 1
Load balancer -> domain name 2 -> target group 2 -> ECS service 2 -> ECS task 2
Load balancer -> domain name 3 -> target group 3 -> ECS service 3 -> ECS task 3
etc...
In the above scenario, as long as you have each ECS service configured with the appropriate target group, each time that service redeploys a task it will automatically update the target group to point to the updated task.
In other words ECS will "dynamically register the IP to target group", exactly like you are wanting.

Related

aws application load balancer metrics not showing

I have created an aws application load balancer. I am trying to test something out on it and I have configured the target group and everything. When I try to hit the load balancer I get a bad gateway error (502), which is expected. However these metrics are not showing up in the monitoring section of the load balancer. I submitted around 5 requests.
Furthermore even after registering an ECS service, I still get bad gatewawy. This is what I see on the load balancer/target groups after registering the service
I have also allowed all traffic inbound and outbound from the two security groups (the security group used by the ECS service and the security group used by the load balancer)
However under the registered target when creating the ECS service I specified two availability zones, but it shows only one registered.
figured it out and its kind of silly. My VPN/network was blocking the call going out to the ALB. I'm not sure why, maybe some sort of network policy. But the url looks something like this my-lb-1123366532.us-west-1.elb.amazonaws.com I wasted almost a day trying to figure this out. I'm just putting it out here in case it helps someone.

Is this possible to create GCP load balancer use same instance in two different ports using unmanaged instance groups

Run one more application server in single GCP instance(GCP compute engine based on Port).While Creating load balancer facing issue.
"Validation failed for instance 'projects/test': instance may belong to at most one load-balanced instance group."
Please let me know if there is an option for running two different services in a single compute engine instance.(May be services running with different ports).
like,
load balancer 1 path rule "/calander" -> test1 instance group -> Instance 1 : x.x.x.x:8080
load balancer 1 path rule "/sample" -> test1 instance group -> Instance 1 : x.x.x.x:8081
Of course, you can. You have to create 2 different backends which use the same compute instance group (CIG) or the same network endpoint group (NEG), but that redirect to differents ports
But, take care of your health-checks definitions. If one service is down, the VM will be considered as unhealthy and will be killed and another one deployed. The 2 services can come in conflict because of this.
You can't directly add the VM as backend. In addition, with a HTTPS loadbalancer (Global Loadbalancer) you can't expose the 8081 port on internet. I'm sure it's not your use case, but I prefer to remember this!

Running multiple web services on a single ECS cluster

If I have an ECS cluster with N distinct websites running as N services on said cluster - how do I go about setting up the load balancers?
The way I've done it currently is for each website X,
I create a new target group spanning all instances in the cluster
I create a new application load balancer
I attach the ALB to the service using the target group
It seems to work... but am want to make sure this is the correct way to do this
Thanks!
The way you are doing it is of course one way to do it and how most people accomplish this.
Application load balancers also support two other types of routing. Host based and path based.
http://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html#host-conditions
Host based routing will allow you to route based off of the incoming host from that website. So for instance if you have website1.com and website2.com you could send them both through the same ALB and route accordingly.
http://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html#path-conditions
Similarly you can do the same thing with the path. If you websites were website1.com/site1/index.html and website1.com/site2/index.html you could put both of those on the same ALB and route accordingly.

AWS ECS handling DNS subdomains across multiple instances

So I am trying to get my AWS setup working with DNS.
I have 2 instances (currently).
I have 4 task definitions. 3 of these need to run on port 80/443, however all on separate subdomains.
Currently if I stop/start a task, it can end up on either of my instances. This causes issues with the subdomain DNS potentially being pointed in the wrong places.
I imagine I need to setup some kind of load balancer to point the DNS at, but unsure how to get that to route through to the correct tasks.
So my questions:
Do I need a single load balancer, or one per 'task / subdomain'?
How do I handle the ports to go from a set source port, to one of any number of destination ports (if I end up having multiple containers running the same task)
Am I over complicating this massively, or is there a simpler way to achieve this?
Do I need a single load balancer, or one per 'task / subdomain'?
You can have a single application load balancer and three target groups for Api, Site and Web App. Then you can do a rule base routing in the load balancer listener as shown in the following screenshot.
Ref: http://docs.aws.amazon.com/elasticloadbalancing/latest/application/listener-update-rules.html
You can then map your domains www.domain.com and app.domain.com to the load balancer
How do I handle the ports to go from a set source port, to one of any number of destination ports (if I end up having multiple containers running the same task)
When you create services for your task definitions in ECS you can configure load balancing using the target groups you created.
Ref: http://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-service.html (Check on "Configuring Your Service to Use a Load Balancer")

ECS tasks desired count not working

I have three EC2 instances, with a classic load balancer. Ideally I should have two tasks running in two instances. So when creating the service I made the desired count of the tasks to 2.
My problem arises when I try to run new version of the task definition. I update the service to run the new task definition. So it should theoretically run two updated tasks replacing the old ones, since i have three ec2 running.
What happens actually is only one updated task is running together with the old tasks. So altogether 3 tasks running even though the desired count is set to 2, as you are able to see in the given image.
Does anyone know a solution for this ?
When using a classic load balancer, you can only map static ports on the ec2 instance.
Your deployment settings are:
min-health: 100%
max-healthy: 200%
The new version of the service would require two more hosts available with the free tcp port you requested. Since you only have 3 servers in the cluster, this condition will not be satisfied. You can either add more servers to your cluster, or use the Application Load Balancer (ALB) which will integrate with docker dynamic port mapping.
Update regarding security groups:
To manage security groups, you can tag a security group with another. For example, tag your ALB with 'app-gateway-alb' which allows specific ports from outside your network, then on the container have a security group which allows ANY TCP from 'app-gateway-alb' this is achieved by putting the security group ID in the text box where you would generally put the CIDR rule.