I have been looking into different ways of connecting multiple miscorservices within their own services/tasks using ECS Fargate.
Normally, if all microservices are defined in the same task definition, we can just use the local ip with corresponding ports but this means we cannot scale individual microservices. From what I can tell there are two 'main' ways of enabling this communication when we break these out into multiple services:
Add a load balancer to each service and use the loadbalancers public ip as the single point of access from one service to another.
Questions I have on this is:
a. Do all the services that need to communicate need to sit in the same VPC and have the
service's incoming rules set to the security group of the load balancer?
b. Say we have now provisioned the entire set up and need to set one of the loadbalancer public DNS's in one microservices code base, whats the best way of attaining this, im guessing some sort of terraform script that 'assumes' the public DNS that will be added to it?
Making use of AWS Service Discovery, meaning we can query service to service with a simple built up identifier.
Question I have for this is:
a. Can we still attach load balancers to the services and STILL use service discovery? Or
does service discovery have an under the hood load balancer already configured?
Many thanks in advance for any help!
1.a All services in the same VPC and their security groups (SGs)
I assume that you are talking about case where each service will have its own load balancer (LB). Since the LBs are public, they can be in any VPC, region or account.
SGs are generally setup so that incoming rules to services allow only connections from the SG of the LB.
1.b DNS
Each task can have environmental variables. This is a good way to pass the DNS values. If you are taking about terraform (TF), then TF would provision the LBs and then create the tasks and set the env variables with DNS values of the LBs. Thus, you would know the DNS of LBs as they would have been created before your services.
2.a Service discovery (SD)
SD is only for private communication between services. No internet is involved, so everything must be in same VPC or peered-VPCs. So its basically oposite of the first solution with LBs.
I think you should be able to also use public LB along with SD.
SD does not use a LB. Instead when you query a DNS of a service through SD you will get private IP addresses of the tasks in random order. So the random order approximates load balancing of connections between tasks in a service.
Related
My applications run on ElasticBeanstalk and communicate purely with internal services like Kinesis and DynamoDB. There is no web traffic needed? Do I need an ElasticLoadBalancer in order to scale my instances up and down. I want to add and remove instances purely based on some cloudwatch metrics? Do I need the ELB to do managed updates etc.?
If there is no traffic to the service then there is no need to have a load balancer.
In fact the load balancer is primarily to distribute inbound traffic such as web requests.
Autoscaling can still be accomplished without a load balancer with scaling based on the CloudWatch metric that you want to use. In fact this is generally how consumer based applications tend to work.
To create this without a load balancer you would want to configure you environment as a worker environment.
#Chris already anwsered, but I would like to complement his answer for the following:
There is no web traffic needed?
Even if you communicate with Kinesis and DynamoDB only, your instances still need to be able to access internet to communicate with the AWS services. So the web traffic is required from your instances. The direct inbound traffic to your instances is not needed.
To fully separate your EB env from the internet you should have a look at the following:
Using Elastic Beanstalk with Amazon VPC
The document describes what you can do and want can't be done when using private subnets.
Is there any way to make GKE clusters (in the same network) talk to each other using internal IP addresses?
I know GKE internal load balancers can handle traffic only from the same network and the same region. And I find this implementation very strange.
I understand pod IPs are routable but they are not static and can change anytime. Also, I know there is loadBalancerSourceRanges configuration option in external load balancers using which I can allow only the subnets I want but what if I want to keep every communication using internal and not using a public IP?
Is there any way to achieve what I am trying? Like configuring the Firewall Rules or anything else? or "Global routing mode" while creating the VPC network or anything?
If you have 2 clusters in 2 different regions and you want them to communicate using internal IPs, your best option is to use nodePort service in the clusters to expose your pods and then configure a VM instance to act as a proxy for each cluster.
This will have the same effect as using the LoadBalancer service as an internal Load Balancer but it has the benefit that it will work across multiple regions. It also allows the same Load Balancer to handle requests for all your services.
The one thing you need to be careful of is overloading the proxy instance. Depending on the number of requests, you may need to configure multiple proxy instances for the cluster, each one only handline a handful of services.
Thanks to Global Access, adding networking.gke.io/internal-load-balancer-allow-global-access: "true" to annotation will allow you to communicate across regions.
"Global access is available in Beta on GKE clusters 1.16+ and GA on 1.17.9-gke.600+."
https://cloud.google.com/kubernetes-engine/docs/how-to/internal-load-balancing#global_access
I have a scenario where I have to deploy multiple micro-services on AWS ECS. I want to make services able to communicate with each other via APIs developed in each micro-service. I want to deploy the front-end on AWS ECS as well that can be accessed publicly and can also communicate with other micro-services deployed on AWS ECS. How can I achieve this? Can I use AWS ECS service discovery by having all services in a private subnet to enable communication between each of them? Can I use Elastic Load Balancer to make front-end micro-service accessible to end-users over the internet only via HTTP/HTTPS protocols while keeping it in a private subnet?
The combination of both AWS load balancer ( for public access) and Amazon ECS Service Discovery ( for internal communication) is the perfect choice for the web application.
Built-in service discovery in ECS is another feature that makes it
easy to develop a dynamic container environment without needing to
manage as many resources outside of your application. ECS and Route 53
combine to provide highly available, fully managed, and secure service
discovery
Service discovery is a technique for getting traffic from one container to another using the containers direct IP address, instead of an intermediary like a load balancer. It is suitable for a variety of use cases:
Private, internal service discovery
Low latency communication between services
Long lived bidirectional connections, such as gRPC.
Yes, you can use AWS ECS service discovery having all services in a private subnet to enable communication between them.
This makes it possible for an ECS service to automatically register
itself with a predictable and friendly DNS name in Amazon Route 53. As
your services scale up or down in response to load or container
health, the Route 53 hosted zone is kept up to date, allowing other
services to lookup where they need to make connections based on the
state of each service.
Yes, you can use Load Balancer to make front-end micro-service accessible to end-users over the internet. You can look into this diagram that shows AWS LB and service discovery for a Web application in ECS.
You can see the backend container which is in private subnet, serve public request through ALB while rest of the container use AWS service discovery.
Amazon ECS Service Discovery
Let’s launch an application with service discovery! First, I’ll create
two task definitions: “flask-backend” and “flask-worker”. Both are
simple AWS Fargate tasks with a single container serving HTTP
requests. I’ll have flask-backend ask worker.corp to do some work and
I’ll return the response as well as the address Route 53 returned for
worker. Something like the code below:
#app.route("/")
namespace = os.getenv("namespace")
worker_host = "worker" + namespace
def backend():
r = requests.get("http://"+worker_host)
worker = socket.gethostbyname(worker_host)
return "Worker Message: {]\nFrom: {}".format(r.content, worker)
Note that in this private architecture there is no public subnet, just a private subnet. Containers inside the subnet can communicate to each other using their internal IP addresses. But they need some way to discover each other’s IP address.
AWS service discovery offers two approaches:
DNS based (Route 53 create and maintains a custom DNS name which
resolves to one or more IP addresses of other containers, for
example, http://nginx.service.production Then other containers can
send traffic to the destination by just opening a connection using
this DNS name)
API based (Containers can query an API to get the list of IP address
targets available, and then open a connection directly to one of the
other container.)
You can read more about AWS service discovery and use cases amazon-ecs-service-discovery and here
According to the documentation, "Amazon ECS does not support registering services into public DNS namespaces"
In other words, when it registers the DNS, it only uses the service's private IP address which would likely be problematic. The DNS for the "public" services would register to the private IP addresses which would only work, for example, if you were on a VPN to the private network, regardless of what your subnet rules were.
I think a better solution is to attach the services to one of two load balancers... one internet facing, and one internal. I think this works more naturally for scaling the services up anyway. Service discovery is cool, but really more for services talking to each other, not for external clients.
I want to deploy the front-end on AWS ECS as well that can be accessed publicly and can also communicate with other micro-services deployed on AWS ECS.
I would use Service Discovery to wire the services internally and the Elastic Load Balancer integration to make them accessible for the public.
The load balancer can do the load balancing on one side and the DNS SRV records can do the load balancing for your APIs internally.
There is a similar question here on Stack Overflow and the answer [1] to it outlines a possible solution using the load balancer and the service discovery integrations in ECS.
Can I use Elastic Load Balancer to make front-end micro-service accessible to end-users over the internet only via HTTP/HTTPS protocols while keeping it in a private subnet?
Yes, the load balancer can register targets in a private subnet.
References
[1] https://stackoverflow.com/a/57137451/10473469
We are having several microservices on AWS ECS. We have single ALB which has different target group for different microservices. We want to expose some endpoints externally while some endpoints just for internal communication.
The problem is that if we put our load balancer in public VPC than it means that we are exposing all register endpoints externally. If we move load balancer to private VPC, we have to use some sort of proxy in public VPC, which required additional infra/cost and custom implementation of all security concerns like D-DOS etc.
What possible approaches we can have or does AWS provide some sort of out of the box solution for this ?
I would strongly recommend running 2 albs for this. Sure, it will cost you more (not double because the traffic costs won't be doubled), but it's much more straight forward to have an internal load balancer and an external load balancer. Work hours cost money too! Running 2 albs will be the least admin and probably the cheapest overall.
Checkout WAF. It stands for web application firewall and is available as AWS service. Follow these steps as guidance:
Create a WAF ACL.
Add "String and regex matching" condition for your private endpoints.
Add "IP addresses" condition for your IP list/range that are allowed to access private endpoints.
Create a rule in your ACL to Allow access if both conditions above are met.
Assign ALB to your WAF ACL.
UPDATE:
In this case you have to use external facing ALB in a public subnet as mentioned by Dan Farrell in comment below.
I would suggest doing it like this:
one internal ALB
one target group per microservice, as limited by ECS.
one Network load balancer(NLB), with one ip based target group.
The Ip based target group will have the internal ALB ip addresses,as the private ip addresses for ALB are not static, you will need to setup cloudwatch cron rule with this lambda function(forked from aws documentation and modified to work on public endpoints as well):
https://github.com/talal-shobaita/populate-nlb-tg-withalb/
Both ALB and NLB are scalable and protected from DDOS by AWS, AWS WAF is another great tool that can be attached directly to your ALB listener for extended protection.
Alternatively, you can wait for AWS to support multiple target group registration per service, it is already in their roadmap:
https://github.com/aws/containers-roadmap/issues/104
This how we eventually solved.
Two LB one in private and one in public subnet.
Some APIs meant to be public, so directly exposed through public LB.
For some private APIs endpoints need to be exposed, added a proxy in public LB and routed those particular paths from public LB to private LB through this proxy.
These days API Gateway is the best way to do this. You can have your API serve a number of different endpoints while serving only the public ones via API Gateway and proxying back to the API.
I don't see it mentioned yet so I'll note that we use a CloudMap for internal routing and an ALB for "external" (in our case simply intra/inter-VPC) communication. I didn't read in depth, but I think this article describes it.
AWS Cloud Map is a managed solution that lets you map logical names to the components/resources for an application. It allows applications to discover the resources using one of the AWS SDKs, RESTful API calls, or DNS queries. AWS Cloud Map serves registered resources, which can be Amazon DynamoDB tables, Amazon Simple Queue Service (SQS) queues, any higher-level application services that are built using EC2 instances or ECS tasks, or using a serverless stack.
...
Amazon ECS is tightly integrated with AWS Cloud Map to enable service discovery for compute workloads running in ECS. When you enable service discovery for ECS services, it automatically keeps track of all task instances in AWS Cloud Map.
You want to look at AWS Security Groups.
A security group acts as a virtual firewall for your instance to control inbound and outbound traffic.
For each security group, you add rules that control the inbound traffic to instances, and a separate set of rules that control the outbound traffic.
Even more specific to your use-case though might be their doc on ELB Security Groups. These are, as you may expect, security groups that are applied at the ELB level rather than the Instance level.
Using security groups, you can specify who has access to which endpoints.
I have an ECS cluster which has an Auto Scaling group with 2 EC2 instances. Also I have 3 services, and each service has its own task definition. Each EC2 instance run one docker container per service. Therefore I have 3 docker containers in each EC2 instance.
Each docker container simply run a spring boot app. Since I have 3 services, then I have 3 spring boot apps. Again, one container only runs one of these 3 spring boot apps. Each app exposes a rest ful API, with services like POST, GET etc, under URLs like /service1 or /service1/resource1. One important point here is I'm using dynamic port mapping in the container's host.
I have an external (internet-facing) ALB on port 443, which has 3 target group. Depending on the URL, the request will go to one of the 3 apps (or containers).
My problem is sometimes app A needs to make an http request to app B. My EC2 instances live in private subnet, while my ALB lives in a public subnet. So if I use my ALB to make http requests from inside a container to another, what's going to happen is the request will go through a NAT, and since the public ip of the NAT is not part of the security group of the ALB then it can't connect on port 443. I have 2 ways to make this work:
In the security groups of the ALB whitelist 0.0.0.0/0. I don't want to do that since the entire world will have access.
In the security group of the ALB whitelist the public IP of the NAT. I'm not sure about this approach. Is it recommendable?
A third option I'm trying to implement is to have a third load balancer, an internal one. But I'm lost here, as per AWS docs you can only assign 1 load balancer to your service. And since we are using dynamic port mapping I don't see a way to create manually an ALB and use the dynamic automatically assigned port.
How do you guys make this kind of connectivity between containers, where one container consumes a service that other provides?
As last comment, I use cloud formation for all. No manual setup from console.
Thanks,
You may try to whitelist you nat gateways public IP as a host with /32 mask. It is a quite normal approach because you already have exposed endpoints to the public internet via ALB. You only need to care about security rules updates in case of destroying or changing the NAT gateway, because it's IP may change.