If I have the following VPC in AWS:
10.0.0.0/16 and I provision an application load balancer (internal) and AWS selects the following ip addresses for me 10.0.0.9 and 10.0.0.12 inside the subnets I choose.
Question: Do the internal addresses (10.0.0.9 and 10.0.0.12) that are picked ever change for the life of the load balancer?
I understand if I delete the load balancer, it will pick new ones.
I also understand that an internet application load balancer IP changes regularly (and thats why people use Network load balancers for static ips) but not much is said about the private internal ips associated with the load balancers.
Any information would be great.
Yes, they could change for application load balancers.
As the application load balancer scales with traffic it will "launch" more instances behind the scenes and use more IPs in your subnets (ENI creation). You don't see those instances in the console but you can have a look at the elastic network interfaces in the console. When it scales down, it's not guaranteed that you get the same IPs. This is why you always need some free space in the subnets used by your application load balancer.
Network load balancers have static private IPs (one ENI per availability zone). Those can be used in the security group of your instances, see Target Security Groups.
Yes it may change with thin the range provided... when more instanaces are launched if ASG is configured or without it due to increased load on the application, there can be more IPs used instead of old ones and it is in the range of your Subnet parameters.
I'm looking for a way to allow private traffic over the public ELB for inter-region communication between the workers. One way to do this is by checking the private IPs from the ELB frequently and update the DNS record.
Getting the private IPs is possible with the CLI:
aws ec2 describe-network-interfaces --region "us-west-1" --filters Name=requester-id,Values='amazon-elb' | jq -r '.NetworkInterfaces[].PrivateIpAddresses[].PrivateIpAddress'
Will give you back a list of used IPs (number depends on the amount of availability zones selected when creating the loadbalancer).
Next step would be to update this in Route53 when changed.
Lambda might be an idea to do this but I noticed that getting these ips takes some time and it sometimes even hit the 3 seconds timeout of lambda. So looking for a better way to do this.
Related
I'll start with describing the architecture of Application Load Balancer:
We have the Application Load Balancer, which contains a list of one or more listeners. Each listener is associated with one specific port and protocol, so that traffic incoming to the load balancer with this port will be handled by this listener. Then, each listener holds a list of rules. A rule specify conditions and actions - basically where to route traffic to. A listener also must have a default action which will be taken if no other condition is met. Usually, this action will be to route traffic to a specific target group.
A target group is a group of either EC2 instances, IP addresses, other Application Load Balancer, or Lambda function.
Furthermore, when you create an Application Load Balancer, it asks you to specify a VPC and list of 2 or more Availability zones, and for each to specify a subnet.
My question now is: Why does AWS asks you to specify this? As far as I understand, the target groups and the registered targets are where you specify the backend of the load balancer, so why do we need to specify the subnets in the ALB configuration?
EDIT
After experimenting, I found that, if I have a EC2 instance running a web server in AZ 3 for example, and now I create a load balancer and selects AZs 1 and 2, then traffic will not reach the web server on AZ 3 until I add another AZ in the load balancer settings.
So if to accurate my question: If this setting of AZs in the load balancer settings means: these are the AZs in which the Load Balancer will send traffic to, in what scenario should I not choose ALL the availability zones there are?
An Application Load Balancer runs on Amazon EC2 infrastructure. Think of it as an Amazon EC2 instance with pre-loaded software, but you don't actually see the EC2 instance in your own account.
Instead, you will see the Network Interfaces where the Load Balancer connects into the network.
Traffic will come into your network through an Internet Gateway then route within the VPC to the Load Balancer. The Load Balancer will then determine a target to receive the traffic, and will send the request via the Network Interface to the resource in the VPC.
When such traffic flows through the VPC, it is subject to normal Security Groups and Network Access Control Lists (NACLs). Private IP addresses will be assigned for each Network Interface used by the Load Balancer.
The Load Balancer instance runs in multiple AZs. If one AZ has a failure, then the Load Balancer instance in the remaining AZ(s) will continue to run. This is why the Load Balancer wants a connection to the VPC in multiple subnets.
Bottom line: While you might normally think of the Load Balancer as a "blackbox service", it is still just some software running on a virtualized computer that needs logical connectivity to the VPC, and it obeys all the rules of networking within the VPC.
why do we need to specify the subnets in the ALB configuration?
Because the load balancer itself requires physical network interfaces, which are created in the subnets you specify (one network interface per subnet).
When you need ALB you must assign an IP address to it. A subnet is nothing more than a range of IP addresses. By placing an ALB inside a subnet, you define which range of IP you desire to be assigned to this resource. Moreover, These subnets allow you to configure different security layers regards the resource inside the subnet for traffic management.
Your target group is located in another subnet with a different range, whereas you can place your ALB in another subnet with a different range.
We are trying to use Elastic Load Balancing in AWS with auto-scaling so we can scale in and out as needed.
Our application consists of several smaller applications, they are all on the same subnet and the same VPC.
We want to put our ELB between one of our apps and the rest.
Problem is we want the load balancer to be working both internally between different apps using an API and also internet-facing because our application still has some usage that should be done externally and not through the API.
I've read this question but I could not figure out exactly how to do it from there, it does not really specify any steps or maybe I did understand it very well.
Can we have an ELB that is both internal and external?
For the record, I can only access this network through a VPN.
It is not possible to for an Elastic Load Balancer to have both a public IP address and a private IP address. It is one or the other, but not both.
If you want your ELB to have a private IP address, then it cannot listen to requests from the internet.
If your ELB is public-facing, you can still call to it from your internal EC2 instances using the public endpoint. However, there are some caveats that goes with this:
The traffic will exit your VPC and re-enter it. It will not be direct instance-to-ELB connection that a private IP address will afford you.
You also cannot use security groups in your security group rules.
There are 3 alternative scenarios:
Duplicate the ELB and EC2 instances, one dedicated to private traffic, one dedicated to public traffic.
Have 2 ELBs (one public, one private) that share the same back-end EC2 instances.
Don't use an ELB for either private or public traffic, and instead use an Elastic IP address (if public) or a private IP address (if private) on a single EC2 instance.
I disagree with #MattHouser answer. Actually, in a VPC, your ELB have all its internal interfaces listed in Network Interfaces with Public IP AND Primary private IP.
I've tested the private IP of my public ELB and it's working exactly like the external one.
The problem is : theses IPs are not listed anywhere in a up to date manner like on a private ELB DNS. So you have to do it by yourself.
I've made a little POC script on this, with an internal Route53 hosted zone : https://gist.github.com/darylounet/3c6253c60b7dc52da927b80a0ae8d428
I made a Lambda function that checks which private IPs are set to the loadbalancer and will update Route53 record when it changes: https://github.com/Bramzor/lambda-sync-private-elb-ips
Using this function, you can easily make use of the ELB for private traffic. I personally use it to connect multiple regions to each other over a VPC inter-region peering without needing an additional ELB.
The standard AWS solution would be to have an extra internal ELB for this.
Looks like #DaryL has an interesting workaround, but it could fail for 5 minutes if the DNS is not updated. Also there is no way to have a separate security group for the internal IPs since they share the ENI and security of the external IP of the ELB.
I faced the same challenge and I can confirm the best solution so far is to have two different ALBs, one internet-facing and the other internal. You can attach both ALBs to a single AutoScaling Group so you can access the same cluster.
Make sure the networking options (Subnets, security groups) of both ALBs are the same in order for both to access the same cluster instances. Autoscaling and Launch Configuration works seamlessly with both ALBs attached to the same AutoSacling Group. This is also working with ALBs created from ElasticBeanstalk environments.
As far as high availability goes, what is the difference between using an Elastic IP or an Elastic Network Interface to mask instance failure? Is the only difference because ENIs can be used for private instances and Elastic IPs can't?
I'm trying to explain the advantages of both, so if someone can help me with this, I would appreciate it!
To achieve High Availability, you need the ability to redirect traffic in the case of instance failure. There are several options:
1. Use an Elastic Load Balancer
This is the preferred way to provide High Availability.
Run multiple Amazon EC2 instances, preferably in different Availability Zones (AZs). Users connect to the ELB (via the supplied DNS name), which redirects traffic to the EC2 instances. If an instance fails, ELB notices this via regular Health Checks, and will only direct traffic to the healthy instances.
Auto Scaling can be used to create these multiple instances across multiple Availability Zones, and it can also update the Load Balancing service when it adds/removes instances.
2. Redirect an Elastic IP address
Run multiple instances (preferably across multiple Availability Zones). Point an Elastic IP address to the instance you desire. Users connect via the Elastic IP address and are directed to the instance. If the instance fails, reassociate the Elastic IP address to a different instance, which will then start receiving the traffic immediately.
This method is not recommended because only one instance is receiving all the traffic while the other instance(s) are sitting idle. It also requires a mechanism to detect failure and reassociate the Elastic IP (which you must do yourself).
3. Reassign an Elastic Network Interface (ENI)
All EC2 instances have a primary ENI. They can optionally have additional ENIs.
It is possible to direct traffic to a secondary ENI and then move that secondary ENI to another instance. This is similar to reassigning an Elastic IP address.
This method is not recommended for the same reason as reassociating an Elastic IP address (above), but also because an ENI can only be reassigned within the same AZ. It cannot be used to direct traffic to an EC2 instance in a different AZ.
Bottom line: Use an Elastic Load Balancer. It provides true High Availability and can do it automatically.
See documentation: What Is Elastic Load Balancing?
My question is simple. Does it make sense to have an Amazon Elastic Load Balancer (ELB) with just one EC2 instance?
If I understood right, ELB will switch traffic between EC2 instances. However, I have just one EC2 instance. So, does it make sense?
On the other hand, I´m using Route 53 to route my domain requests example.com, and www.example.com to my ELB, and I don´t see how to redirect directly to my EC2 instance. So, do I need an ELB for routing purposes?
Using an Elastic Load Balancer with a single instance can be useful. It can provide your instance with a front-end to cover for a disaster situation.
For example, if you use an auto-scaling group with min=max=1 instance, with an Elastic Load Balancer, then if your instance is terminated or otherwise fails:
auto-scaling will launch a new replacement instance
the new instance will appear behind the load balancer
your user's traffic will flow to the new instance
This will happen automatically: no need to change DNS, no need to manually re-assign an Elastic IP address.
Later on, if you need to add more horsepower to your application, you can simply increase your min/max values in your autoscaling group without needing to change your DNS structure.
It's much easier to configure your SSL on an ELB than an EC2, just a few clicks in the AWS console. You can even hand pick the SSL protocols and ciphers.
It's also useful that you can associate different security groups to the actual EC2 and the forefront ELB. You can leave the ELB in the DMZ and protect your EC2 from being accessible by public and potentially vulnerable to attacks.
There is no need to use a Load Balancer if you are only running an single Amazon EC2 instance.
To point your domain name to an EC2 instance:
In the EC2 Management Console, select Elastic IP
Allocate New Address
Associate the address with your EC2 instance
Copy the Elastic IP address and use it in your Route 53 sub-domain
The Elastic IP address can be re-associated with a different EC2 instance later if desired.
Later, if you wish to balance between multiple EC2 instances:
Create an Elastic Load Balancer
Add your instance(s) to the Load Balancer
Point your Route 53 sub-domain to the Load Balancer
With NO ELB :-
Less Secure (DOS Attacks possible as HTTP 80 will be open to all, instead of being open only to ELB)
You won't have the freedom of terminating an instance to save EC2 hrs without worrying about remapping your elastic IP(not a big deal tho)
If you don't use ELB and your ec2 instance becomes unhealthy/terminates/goesDown
Your site will remain down (It will remain up if you use ELB+Scaling Policies)
You will have to remap your elastic IP
You pay for the time your elastic IP is not pointing to an instance around $0.005/hr
You get 750 hours of Elastic Load Balancing plus 15 GB data processing with the free tier so why not use it along with a min=1,max=1 scaling policy
On top of the answer about making SSL support easier by putting a load balancer in front of your EC2 instance, another potential benefit is HTTP/2. An Application Load Balancer (ALB) will automatically handle HTTP/2 traffic and convert up to 128 parallel requests to individual HTTP/1.1 requests across all healthy targets.
For more information, see: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html#listener-configuration
It really depends on what are you running in the EC2 instance.
While with only one EC2 instance it's not necessary to use ELB (all your traffic will go to that instance anyways), if your EC2 service has to scale in the near future, is not a bad idea to invest some time now and get familiar with ELB.
This way, when you need to scale, it's just a matter of firing up additional instances, because you have the ELB part done.
If your EC2 service won't scale in the near future, don't worry too much!
About the second part, you definitely can route directly to your EC2 instance, you just need the EC2 instance IP. Take a look at the amazon route53 docs. Mind that if your IP is not static (you don't setup an Amazon Elastic IP), you'd need to change the IP mapping everytime the EC2 ip changes.
You can also use an ELB in front of EC2 if for example you want it to be publically reachable, without having to use up an Elastic IP address. As said previously they work well too with ASG's
I am working on using a elastic load balancer along with AWS Auto scaling. I do have a setup in which instances will be scaled up/down automatically based on NetworkIn and it is working fine. I have a couple of questions regarding ELB.
How a fresh auto-scaling launched instance is registered with the ELB automatically? I know we will give the load balancer name while creating the auto-scaling group; I need to know the real 'how'.
Can we have multiple private IPs of instances run different applications and all of them visible to ELB?
Explanation for the 2). Lets say I configure the instances so that they will have multiple private IPs at the time of launch. Could they be exposed to ELB rather than the Public IP of the machine? Can ELB read the private IPs of the instances that are launched under it?
How a fresh auto-scaling launched instance is registered with the ELB automatically? I know we will give the load balancer name while creating the auto-scaling group; I need to know the real 'how'.
My guess is it makes an API call RegisterInstancesWithLoadBalancer. You can do that too in your own code. It does not have to be through Auto Scaling
Can we have multiple private IPs of instances run different applications and all of them visible to ELB?
Well, ELB does not care about the ip address at all. It goes by the Instance Id. Unless it is on a VPC and uses ENI. However, ELB routes traffic only to ip address attached to eth0
Update:
Note:
When you register a multi-homed instance (an instance that has an elastic network interface (ENI) attached) with your load balancer, the load balancer will route traffic to the primary IP address of the instance (eth0).
Source: ELB Developer Guide