I have a use case:- where my I have a lot of traffic coming to my webservers so I need greater performs and better latency however there are 2 paths to which traffic is incoming.
as per my understanding this is achievable with aws NLB which scales to 1000's request per second and 100 ms sub latency.
however I have www.jatin.com and www.jatin.com/somepath which means It needs path based routing which Is supported by aws ALB.
I need performance as well as path based routing achievable with NLB?
achievable with NLB?
Sadly its not possible. Concepts of url, paths or dns hostnames are only defined for Layer 7 - Application of the OSI model. However, NLB operates at layer 4 - transport. Subsequently, NLB is not able to differentiate between any url domain names or paths.
The only distribution of incoming traffic you can obtained for NLB, is based on port number, as per my knowledge. So you can have one listener for port 80, other listener for port 88, 443 and so on. This will work because ports, just like IP addresses, are part of layer 4.
Only ALB operates at layer 7 and partially CLB, thus it can do path based routing. So you either have to use ALB, or maybe try to look for a third party load balancer which you can deploy on AWS.
Related
In AWS EC2, I have one Application Load Balancer (ALB) alb1-556083996.us-east-1.elb.amazonaws.com, and two target groups tg-blue and tg-green.
In AWS route53, I have two records api.myapp.example.com and api-blue.myapp.example.com.
The route53 record api.myapp.example.com is aliased to api-blue.myapp.example.com which is alieased to an AWS ALB alb1-123456789.us-east-1.elb.amazonaws.com.
Both of these two route53 records are latency based with Evaluate target health enabled.
The route53 record api.myapp.example.com also can be aliased to api-green.myapp.example.com at some point instead of aliasing to api-blue.myapp.example.com.
I'd like to create following ALB listener rules:
When the host api.* will be pointing to api-blue.*, the traffics will be forwarded to the target group tg-blue.
When the host 'api.* will be pointing to api-green.*, the traffics will be forwarded to the target group tg-green.
The default rule will return a FIXED_RESPONSE.
In addition, as Evaluate target health and latency based routing have been enabled on both level of aliases, I'd expect the traffics will never be dropped as long as at least one target group has targets registered to it.
I have LB access logs enabled. When I hit api.myapp.example.com or api-blue.myapp.example.com, the host api-blue.* does not appear in the access logs. So, I'm afraid that host-headers based rule won't work here. I will need a lot of experiments to be able to figure out if the above is working or not.
So, I'm trying to find out whether it is possible or not from someone knowledgable in this area. If it is not possible, how can I achieve this?
Recently I have been trying to catch up with the knowledge that I'm missing around Load Balancing internals and I have found this great article
But it made me think about more questions than before;)
Till now I understood that if we talk about L4 LB we can differentiate:
LB terminate type - that creates for each incoming connection, a new one to backend.
LB passthrough type - that might be split into NAT, or Direct routing one (eventually with tunneling)
Now one of my questions that came to my mind is that how does it fit into AWS world - what type of LB is AWS Network Load Balancer in that case?
The next thing is about L7 LB's.
Does layer 7 LB also relies on NAT, or Direct routing? Or it's completely beyond that? When it's quite a lot of materials around layer 4, typically layer 7 is really poor in terms of proper articles covering internals - I know only top products like: haproxy or nginx, but still don't get the difference between them :(
I will be very thankful for anyone who might provide me with at least some piece of advice how connect the dots there :)
what type of LB is AWS Network Load Balancer in that case?
Network Load Balancer is a layer 4 load balancing, check the information provided on the amazon docs:
A Network Load Balancer functions at the fourth layer of the Open Systems Interconnection (OSI) model. It can handle millions of requests per second. After the load balancer receives a connection request, it selects a target from the target group for the default rule. It attempts to open a TCP connection to the selected target on the port specified in the listener configuration.
https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html
For the second question.
Does layer 7 LB also relies on NAT, or Direct routing? Or it's completely beyond that?
In the amazon world application load balancer is the only wich support layer 7 LB and works like mention in the article
"The client makes a single HTTP/2 TCP connection to the load balancer. The load balancer then proceeds to make two backend connections"
So for the client is direct connection with the load balancer, and that connection is splited to be served at backend pools.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 days ago.
Improve this question
I want to know the major difference between Amazon application load balance (ALB) and classic load balance (CLB). I have searched that but they gave only examples, such as classic load balance contains only same content, but application load balance can contain different content and grouped with target group.
ALB has some features (e.g., host based routing, path based routing), but my question is why we use ALB instead of classic load balance. Please provide use cases for both.
ALB
The application load balancer (Elastic Load Balancer V2) lets you direct traffic to EC2 instances based on more complex criteria/rules, specifically URL paths. You can have users trying to access "/signup" go to a group of instances and users trying to access "/homepage" go to another set of instances. In comparison to the classic ELB, an ALB inspects traffic at the application level (OSI layer 7). Its at this level that URL paths can be parsed for example.
ELB/Classic
The classic load balancer routes traffic uniformly using information at the TCP level (OSI layer 4 Transport). It will either send request to each instances "round-robin" style or utilize sticky-sessions and send each users/client to the same instance they initially landed on.
Why ALB over ELB?
You could use an ALB if you decided to architect your system in such a way that each path had its own set of instances or its own service. So
/signup, /login, /info etc etc all go through one load balancer that is pinned to your domain name https//mysite.com but a different EC2 instance is servicing each. ALBs only support HTTP/HTTPS. If your system uses another protocol you would have to use an ELB/Classic load balancer. Websockets HTTP/2 are currently only supported on ALB.
Another reason you might choose ALB over ELB is that there are some additional newer features that have not yet been added to ELB or may never be added. As Michael points out below AWS WAF is not supported on the classic load balancer but is on ALB. I expanded on other features farther down.
Why ELB over ALB?
Architecturally speaking its much simpler to send every request to the same set of instances and then internally within your application delegate requests for certain paths to certain functions/classes/methods etc etc... This is essentially the monolith design most applications start out as. For most work loads dedicating traffic to certain instance groups (the ALB approach) would be a waste of EC2 power. You would have some instances doing lots of work and others barely used.
Similar to the ALB there are features of the classic ELB that have not yet arrived on ALB. I expand on that below.
Update - More on Feature Differences
From a product perspective they differ in other ways that aren't really related to how they operate and more about some features not being present yet.
HTTP to HTTPS Redirection - For example, in ALB each target group (group of instances your assigning a specific route) currently can only handle one protocol so if you were to implement HTTP to HTTPS redirects this requires two instances minimum. With ELB you can handle HTTP to HTTPS redirection on a single instance. I imagine ALB will have this feature soon.
https://forums.aws.amazon.com/thread.jspa?threadID=247546
Multiple SSL Certificates on One Load Balancer - With an ALB you can assign it multiple SSL certificates for different domains. This is not possible on a classic load balancer though the feature has been requested. For a classic load balancer you can use a wild card certificate but that is not the same thing. An ALB makes use of SNI server name identification to make this possible where as this has not been added to the classic ELB feature set.
I see that ECS services can use Application Load Balancers, and the dynamic port stuff works atuomagically. However, an ALB has a maximum of 10 rules other than default rules. Does that mean that I need a separate ALB for every 10 services unless I wish to access via a different port (in which case the default rules would kick in)? This seems obvious, but for something touted to be the solution to load balancing in a microservices environment, this would seem incredibly limiting. Am I missing something?
As far as I know and have experienced, this is indeed true, you are limited to 10 listeners per ALB. Take into account that this setup (ALB + ECS) is fairly new so it is possible that Amazon will adjust the limits as people are requesting this.
Take into account as well that a listener typically has multiple targets, in a microservice architecture this translates to multiple instances of the same service. So you can run 10 different services but you are able to run 10 instances of each service, balancing 100 containers with a single ALB.
Alternatively (to save costs) you could create one listener with multiple rules, but they have to be distinguished by path pattern and have to listen (not route to) the same port. Rules can forward to a target group of your choice. E.g. you can route /service1 to container 1 and /service2 to container 2 within one listener.
Yes, you are correct, and it is a low restriction. However if you are able to use different CNAMES for your services then having them in an ALB with single target group for each service won't behave differently to having one ALB and multiple target groups each with rules. Dynamic ports are probably the main part of their "microservices solution" argument.
I am reading about load balancing.
I understand the idea that load balancers transfer the load among several slave servers of any given app. However very few literature that I can find talks about what happens when the load balancers themselves start struggling with the huge amount of requests, to the point that the "simple" task of load balancing (distribute requests among slaves) becomes an impossible undertaking.
Take for example this picture where you see 3 Load Balancers (LB) and some slave servers.
Figure 1: Clients know one IP to which they connect, one load balancer is behind that IP and will have to handle all those requests, thus that first load balancer is the bottleneck (and the internet connection).
What happens when the first load balancer starts struggling? If I add a new load balancer to side with the first one, I must add even another one so that the clients only need to know one IP. So the dilema continues: I still have only one load balancer receiving all my requests...!
Figure 2: I added one load balancer, but for having clients to know just one IP I had to add another one to centralize the incoming connections, thus ending up with the same bottleneck.
Moreover, my internet connection will also reach its limit of clients it can handle so I probably will want to have my load balancers in remote places to avoid flooding my internet connection. However if I distribute my load balancers, and want to keep my clients knowing just one single IP they have to connect, I still need to have one central load balancer behind that IP carrying all the traffic once again...
How do real world companies like Google and Facebook handle these issues? Can this be done without giving the clients multiple IPs and expect them to choose one at random avoiding every client to connect to the same load balancer, thus flooding us?
Your question doesn't sound AWS specific, so here's a generic answer (elastic LB in AWS auto-scales depending on traffic):
You're right, you can overwhelm a loadbalancer with the number of requests coming in. If you deploy a LB on a standard build machine, you're likely to first exhaust/overload the network stack including max number of open connections and handling rate of incoming connections.
As a first step, you would fine tune the network stack of your LB machine. If that still does not provide you the required throughput, there are special purpose loadbalancer appliances on the market, that are built ground-up and highly optimized to handle a large number of incoming connections and routing them to several servers. Examples of these are F5 and netscaler
You can also design your application in ways that help you split traffic to different sub domains, thereby reducing the number of requests 1 LB has to handle.
It is also possible to implement a round-robin DNS, where you would have 1 DNS entry point to several client facing LBs instead of just one as you've depicted.
Advanced load balancers like Netscaler and similar also does GSLB with DNS not simple DNS-RR (to explain further scaling)
if you are to connect to i.e service.domain.com, you let the load balancers become Authorative DNS for the zone and you add all the load balancers as valid name servers.
When a client looks up "service.domain.com" any of your loadbalancers will answer the DNS request and reply with the IP of the correct data center for your client. You can then further make the loadbalancer reply on the DNS request based of geo location of your client, latency between clients dns server and netscaler, or you can answer based on the different data centers load.
In each datacenter you typically set up one node or several nodes in cluster. You can scale quite high using such a design.
Since you tagged Amazon, they have load balancers built in to their system so you don't need to. Just use ELB and Amazon will direct the traffic to your correct system.
If you are doing it yourself, load balancers typically have a very light processing load. They typically do little more than redirect a connection from one machine to another based on a shallow inspection (or no inspection) of the data. It is possible for them to be overwhelmed, but typically that requires a load that would saturate most connections.
If you are running it yourself, and if your load balancer is doing more work or your connection is getting saturated, the next step is to use Round-Robin DNS for looking up your load balancers, generally using a combination of NS and CNAME records so different name lookups give different IP addresses.
If you plan to use amazon elastic load balancer they claim that
Elastic Load Balancing automatically scales its request handling
capacity to meet the demands of application traffic. Additionally,
Elastic Load Balancing offers integration with Auto Scaling to ensure
that you have back-end capacity to meet varying levels of traffic
levels without requiring manual intervention.
so you can go with them and do not need to handle the Load Balancer using your own instance/product