I have the following stack (deployed by CloudFormation):
SQS queue that trigger a lambda function
The lambda function is calling an ECS service (by http post request - which is an express.js server)
I am going to create an autoscaling configuration for the ECS service to scale up and down based on SQS metric.
If there will be multiple tasks in the service (means there were a lot of SQS messages) how will the service manage the calls to the tasks? will I need to add a load balancer for the service? or the service know how to balance it between the tasks without load balancer?
I think there are two ways to deal with this.
Use load balancer as you pointed out. It can be internal load balancer. In this case, lambda will be associated with a VPC and will call the balancer's private dns.
Use ECS service discovery. This can also server as sort-of load balancer, though not as feature reach as regular balancer. The reason is that the private url associated with the service will return multiple IPs of its tasks, and the distribution of incoming requests is based on DNS cache timeouts. After each timeout, the IPs will be in different order. For this lambda would also be in a vpc and use private url associated with the ECS service to invoke your service.
I have an on-premises load balancer that I wish to use to distribute traffic on EC2s in an Autoscaling group(ASG).
When AWS's Elastic Load Balancer(ELB) registers an ASG as a target, during a scale-out,new instances are automatically registered to ELB to route traffic to.
Can the same functionality be achieved in any way with an on-prem load balancer?
There is in-built integration between Amazon EC2 Auto-Scaling and Elastic Load Balancers. The Auto Scaling systems knows how to modify Target Groups to add/remove instances. However, it has no knowledge about your on-premises load balancer.
You could use Amazon EC2 Auto Scaling lifecycle hooks to trigger additional code (that you write) to add/remove the instances to your own load balancer, but that would require an AWS Lambda function to communicate with the on-premises load balancer to update the configuration.
If you wanted to get the IPs of your target group you can do that do with an API call, and put that into your LB, but I think you may as well proxy calls from your LB to an ELB sitting on AWS which does the actual LoadBalancing. IMO there is way too much work and a lot could go wrong if you were to try and re-implement ELB features using Lambda functions.
I know that with AWS ECS's service-discovery you can make one service talk to the other by using the service.cluster pattern at the url (and service discovery does the rest for you). But I want to know if there is a way of doing one instance of a service talk to other container in the same service on AWS Elastic Container Service.
I have searched for this for a while now and decided to ask here as I wasn't able to find any conclusive ideas on how to make it work.
You need to associate your ECS Service with an Application Load Balancer (ALB) which makes sure ECS will registers the tasks (i.e. one or more containers) with the ALB providing a uniform URL for the services to communicate with each other. ALB support path based routing to multiple Target Groups allowing micro-service style communication between tasks/services.
As per the AWS documentation1, you can use a single ALB for multiple services as ALB provides path-based routing and priority-rules. For further information you can also view this link 2 which explains how path-based routing can be used to forward request to different target groups with different ECS tasks/services.
My applications run on ElasticBeanstalk and communicate purely with internal services like Kinesis and DynamoDB. There is no web traffic needed? Do I need an ElasticLoadBalancer in order to scale my instances up and down. I want to add and remove instances purely based on some cloudwatch metrics? Do I need the ELB to do managed updates etc.?
If there is no traffic to the service then there is no need to have a load balancer.
In fact the load balancer is primarily to distribute inbound traffic such as web requests.
Autoscaling can still be accomplished without a load balancer with scaling based on the CloudWatch metric that you want to use. In fact this is generally how consumer based applications tend to work.
To create this without a load balancer you would want to configure you environment as a worker environment.
#Chris already anwsered, but I would like to complement his answer for the following:
There is no web traffic needed?
Even if you communicate with Kinesis and DynamoDB only, your instances still need to be able to access internet to communicate with the AWS services. So the web traffic is required from your instances. The direct inbound traffic to your instances is not needed.
To fully separate your EB env from the internet you should have a look at the following:
Using Elastic Beanstalk with Amazon VPC
The document describes what you can do and want can't be done when using private subnets.
We are having several microservices on AWS ECS. We have single ALB which has different target group for different microservices. We want to expose some endpoints externally while some endpoints just for internal communication.
The problem is that if we put our load balancer in public VPC than it means that we are exposing all register endpoints externally. If we move load balancer to private VPC, we have to use some sort of proxy in public VPC, which required additional infra/cost and custom implementation of all security concerns like D-DOS etc.
What possible approaches we can have or does AWS provide some sort of out of the box solution for this ?
I would strongly recommend running 2 albs for this. Sure, it will cost you more (not double because the traffic costs won't be doubled), but it's much more straight forward to have an internal load balancer and an external load balancer. Work hours cost money too! Running 2 albs will be the least admin and probably the cheapest overall.
Checkout WAF. It stands for web application firewall and is available as AWS service. Follow these steps as guidance:
Create a WAF ACL.
Add "String and regex matching" condition for your private endpoints.
Add "IP addresses" condition for your IP list/range that are allowed to access private endpoints.
Create a rule in your ACL to Allow access if both conditions above are met.
Assign ALB to your WAF ACL.
UPDATE:
In this case you have to use external facing ALB in a public subnet as mentioned by Dan Farrell in comment below.
I would suggest doing it like this:
one internal ALB
one target group per microservice, as limited by ECS.
one Network load balancer(NLB), with one ip based target group.
The Ip based target group will have the internal ALB ip addresses,as the private ip addresses for ALB are not static, you will need to setup cloudwatch cron rule with this lambda function(forked from aws documentation and modified to work on public endpoints as well):
https://github.com/talal-shobaita/populate-nlb-tg-withalb/
Both ALB and NLB are scalable and protected from DDOS by AWS, AWS WAF is another great tool that can be attached directly to your ALB listener for extended protection.
Alternatively, you can wait for AWS to support multiple target group registration per service, it is already in their roadmap:
https://github.com/aws/containers-roadmap/issues/104
This how we eventually solved.
Two LB one in private and one in public subnet.
Some APIs meant to be public, so directly exposed through public LB.
For some private APIs endpoints need to be exposed, added a proxy in public LB and routed those particular paths from public LB to private LB through this proxy.
These days API Gateway is the best way to do this. You can have your API serve a number of different endpoints while serving only the public ones via API Gateway and proxying back to the API.
I don't see it mentioned yet so I'll note that we use a CloudMap for internal routing and an ALB for "external" (in our case simply intra/inter-VPC) communication. I didn't read in depth, but I think this article describes it.
AWS Cloud Map is a managed solution that lets you map logical names to the components/resources for an application. It allows applications to discover the resources using one of the AWS SDKs, RESTful API calls, or DNS queries. AWS Cloud Map serves registered resources, which can be Amazon DynamoDB tables, Amazon Simple Queue Service (SQS) queues, any higher-level application services that are built using EC2 instances or ECS tasks, or using a serverless stack.
...
Amazon ECS is tightly integrated with AWS Cloud Map to enable service discovery for compute workloads running in ECS. When you enable service discovery for ECS services, it automatically keeps track of all task instances in AWS Cloud Map.
You want to look at AWS Security Groups.
A security group acts as a virtual firewall for your instance to control inbound and outbound traffic.
For each security group, you add rules that control the inbound traffic to instances, and a separate set of rules that control the outbound traffic.
Even more specific to your use-case though might be their doc on ELB Security Groups. These are, as you may expect, security groups that are applied at the ELB level rather than the Instance level.
Using security groups, you can specify who has access to which endpoints.