We have a setup where our Load Balancer is talking to one of our RDS instances at Amazon. For the Security Group of our RDS instance we have to fill in a IP Address. Only the IP address of the Load Balancer cant be used because it could change. So we should "create a CNAME record for the Load Balancer DNS name". But we can only fill in a IP address into the Security Groups, so there's the problem.
What should be do to keep it secure but also working? Because opening the RDS instance for all ip addresses doesn't seem safe to me.
I contacted AWS directly and was told that currently RDS doesn't support ELB since AWS considers ELB's use case for distributing web traffic only. Here are two links that were provided to me by AWS in case you haven't seen them:
Discussion about why it's not good to load balance to dbs for writes:
Can I use Amazon ELB for my RDS instance for load balancing?
Feature request to AWS - customers are using self managed HAProxy to accomplish:
https://forums.aws.amazon.com/thread.jspa?threadID=58633
The only work around I can think of - if you want to continue on an unsupported design - is to use the subnet IP range that the ELB's are serving assuming you are using VPC.
Related
So I have just setup an application load balancer but I need a static IP to whitelist my database connection, I found Global Accelerator can do the job so I have set it up and assigned it to the ALB. All showing fine in the console but when I ping my domain (www.example.com), I don't see either of the 2 static IP's assigned... and when I whitelist both IP's my application still cannot connect.
Am I doing something wrong?
Edit: My database is Mongo DB hosted on the Atlas Cloud. In my staging environment I have secured the connection to a single server instance using that servers IP address. Now I'm moving to a production environment with a load balancer, I'm not quite sure how I would achieve the same result, since I have multiple EC2 instances which can be created/destroyed via autoscaling. My thinking is that I need to whitelist the load balancer IP address rather than individual instances.
I am assuming that your architecture is:
Domain name pointing to an Application Load Balancer in AWS
Load Balancer points to an Auto Scaling group of Amazon EC2 instances
The EC2 instances point to your MongoDB database hosted on the Atlas Cloud
You want an static IP address so that the database can permit access from the Amazon EC2 instances
While incoming traffic to the EC2 instances goes through the Load Balancer, please note that the connection from an EC2 instance to the database is a separate outbound connection that is established to the database. This traffic does not go through the Load Balancer. The only traffic coming 'out' of a Load Balancer is the response to requests that came 'in'.
The typical way to implement this architecture is:
Load Balancer in public subnets
Auto-Scaled Amazon EC2 instances in private subnets
A NAT Gateway in the public subnet(s)
This way, the instances in the private subnets can access the Internet via the NAT Gateway, yet they are fully isolated from traffic coming in from the Internet. It has the additional benefit that the NAT Gateway has a static IP address. All traffic going through the NAT Gateway to the Internet will 'appear' to be coming from this IP address.
For fault tolerance, it is recommended to put a NAT Gateway in at least two Availability Zones. Each will have its own static IP address.
Oh, and you could consider moving your database to Amazon DocumentDB (with MongoDB Compatibility), which would reduce latency between the application servers and the database.
I have a load balancer configured to have an IPV4 Ip address. However, the provided IP is a DNS mapped IP address to the load balancer of the format *.ap-south-1.elb.amazonaws.com.
I need to configure IOT devices to send data to the load balancer and they do not support DNS. How can I assign a static IP address like ... to my load balancer so that I can configure my IOT devices to send data to it.
The Elastic IPs section does not provide a facility to allocate it to a load balancer and only supports ec2 instances.
Conclusion:
I have found a way to use DNS on my IOT device and working on this was vital. I am now aware of the option of manually hosting a load-balancer on an EC2 instance. A simper alternative is forwarding all requests at an elastic IP addressed EC2 instance to the load balancer. However, this will cause a bottleneck at the transparent proxy. Hence, I think using the DNS feature on the IOT device is the best option.
Elastic Load Balancers do not support static IP addresses. They only support DNS CNAMEs (or Aliases if you are using Route 53). This is because ELB DNS entries will resolve to different IP addresses depending on how it is scaling between availability zones. Also, over time, the IP addresses will/may change.
The AWS documentation also specifically states to create CNAME-records only when mapping custom DNS entries to your ELB. If you are using Route 53, you can create an Alias record, which look like an A-record to the outside world.
If you need a static IP address, then you cannot use ELB.
Instead, you will need to manage your own load balancer (HAProxy, nginx, etc.) on an EC2 instance using an Elastic IP address.
It would not be possible to assign a static IP with the elastic load balancer. You need to use DNS name only.
The only way I am aware of doing this is by setting up your instances within a VPC and having dedicated NAT instances by which all outbound traffic is routed.
Here is a link to the AWS documentation on how to set up NAT instances:
http://docs.amazonwebservices.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html
AWS Elastic Load Balancer does not support assigning a static IP address due to many reasons.
Looking at your problem, the issue you are facing is having large amount of data sources to pump data to AWS. I suggest you to use AWS Kinesis Firehose service instead of the current approach as Firehose specifically focus on streaming data into AWS.
How can I assign a static IP address to a ELB. Seems like I cannot.
Some articles online asks to create a Route 53 record but this requires changing CNAME of domain which also redirect email traffic. I just want to change A record not CNAME.
Some articles also mention that I can use a EC2 instance as a reverse proxy. But will a single proxy be able to handle a lot of traffic?
Any solution for this?
AWS' Elastic Load Balancer is actually elastic on two levels as described here:
http://shlomoswidler.com/2009/07/elastic-in-elastic-load-balancing-elb.html
The first level is the load balancer itself. In order to make sure that ELB can scale to whatever volume you have and burst to whatever volume you suddenly encounter, AWS assigns a 'static' DNS hostname (e.g. MyDomainELB-918273645.us-east-1.elb.amazonaws.com). That hostname points to multiple IP addresses. You can see that (from a command line) by running
$ host MyDomainELB-918273645.us-east-1.elb.amazonaws.com
MyDomainELB-918273645.us-east-1.elb.amazonaws.com 172.31.7.2
MyDomainELB-918273645.us-east-1.elb.amazonaws.com 172.31.11.33
The second form of elasticity within the ELB is obviously then ELB directing the query to one of your EC2 instances in the pool.
So, you can see that trying to assign a static IP address to the load balancer would be self-defeating.
Using an EC2 instance as a reverse proxy would also seem self-defeating as you would then create a bottleneck before even getting to the ELB. Might as well just create your own load balancer.
The recommended solution (which you've pointed out) is to create a CNAME that points to the ELB hostname (which won't change).
i.e. my-app.mycompany.com ->
MyDomainELB-918273645.us-east-1.elb.amazonaws.com
This would allow you to integrate your scalable application, behind the ELB within your domain.
I'm not sure I fully understand why you cannot create a CNAME in your DNS or what that has to do with directing email traffic, can you explain?
A new feature in AWS (I believe it was announced at Re:Invent 2017) allows for static IPs with Network Load Balancers (NLB). NLB can only handle layer 4 (TCP) and not HTTP specifics (layer 7).
You can assign one Elastic IP address per availability zone.
For details see the AWS blog post or the NLB documentation.
The "Classic Load Balancer" and "Application Load Balancer" do not support static IPs. If you need a feature only provided by those, you have to fall back to the CNAME solution described above.
A blog was recently published by AWS support on this topic leveraging NLB to provide static IP to Classic and Application load balancer - https://aws.amazon.com/blogs/networking-and-content-delivery/using-static-ip-addresses-for-application-load-balancers/
Summary of solution as described by the post
We end up with a TCP listener on a NLB that accepts traffic and forwards it to an internal ALB. The ALB terminates TLS, examines HTTP headers, and routes requests based on your configured rules to target groups with your instances, servers, or containers. The AWS Lambda function keeps everything in sync by watching the ALB for IP address changes and updating the NLB target group. In the end we’ll have a few static IP addresses that are easy for whitelisting, and we won’t lose any of the benefits of ALB. Note that we will be sending all of the traffic through two load balancers
I found setting up AWS Global Accelerator very straight forward and simple. It created 2 static IP Addresses and a static DNS pointing to my Application load balancer.
Configuring Global Accelerator
Set listeners as TCP port 80, 443
Select your load balancer endpoint (AWS Global Accelerator Configuration)
Add cname record for your dns pointing to the static dns it created
(mywebsite.com > globalacceleratorDNS.com). If any client needs to
whitelist, give them the 2 static IP it created
Pricing is $18 per month + a few pennies per GB of data transfer.
I'm pretty sure its cheaper than the NLB, Nat Gateway, Elastic IP setup.
https://docs.aws.amazon.com/global-accelerator/latest/dg/about-accelerators.html
For little traffic, it might be a solution to set up an EC2 Instance running Nginx as a forwarding proxy.
So you can use the EC2's static IP Address to forward your traffic resolving the ALB's DNS name.
However, it's a kind of a hack, but using a Global Accelerator or an NLB seems to me also like a hack :-)
Unlike the Network Load Balancer, the Application Load Balancer (ALB) does not support Elastic IPs, but that's not the worst part. If you use Route 53 together with the ALB, the DNS automatically sets the TTL to 60 seconds. This appears to be causing problems for our institutional - mainly government - customers running older Windows DNS servers. They just can't keep up with the ALB's Listener changing its public-facing IP on such a short notice. Older DNS infrastructure is either not respecting or is not capable of handling such aggressive TTL.
While I don't like it, AWS recommends to put a Network Load Balancer in front of the Application Load Balancer, per here: https://aws.amazon.com/blogs/networking-and-content-delivery/using-static-ip-addresses-for-application-load-balancers/
My question is simple. Does it make sense to have an Amazon Elastic Load Balancer (ELB) with just one EC2 instance?
If I understood right, ELB will switch traffic between EC2 instances. However, I have just one EC2 instance. So, does it make sense?
On the other hand, I´m using Route 53 to route my domain requests example.com, and www.example.com to my ELB, and I don´t see how to redirect directly to my EC2 instance. So, do I need an ELB for routing purposes?
Using an Elastic Load Balancer with a single instance can be useful. It can provide your instance with a front-end to cover for a disaster situation.
For example, if you use an auto-scaling group with min=max=1 instance, with an Elastic Load Balancer, then if your instance is terminated or otherwise fails:
auto-scaling will launch a new replacement instance
the new instance will appear behind the load balancer
your user's traffic will flow to the new instance
This will happen automatically: no need to change DNS, no need to manually re-assign an Elastic IP address.
Later on, if you need to add more horsepower to your application, you can simply increase your min/max values in your autoscaling group without needing to change your DNS structure.
It's much easier to configure your SSL on an ELB than an EC2, just a few clicks in the AWS console. You can even hand pick the SSL protocols and ciphers.
It's also useful that you can associate different security groups to the actual EC2 and the forefront ELB. You can leave the ELB in the DMZ and protect your EC2 from being accessible by public and potentially vulnerable to attacks.
There is no need to use a Load Balancer if you are only running an single Amazon EC2 instance.
To point your domain name to an EC2 instance:
In the EC2 Management Console, select Elastic IP
Allocate New Address
Associate the address with your EC2 instance
Copy the Elastic IP address and use it in your Route 53 sub-domain
The Elastic IP address can be re-associated with a different EC2 instance later if desired.
Later, if you wish to balance between multiple EC2 instances:
Create an Elastic Load Balancer
Add your instance(s) to the Load Balancer
Point your Route 53 sub-domain to the Load Balancer
With NO ELB :-
Less Secure (DOS Attacks possible as HTTP 80 will be open to all, instead of being open only to ELB)
You won't have the freedom of terminating an instance to save EC2 hrs without worrying about remapping your elastic IP(not a big deal tho)
If you don't use ELB and your ec2 instance becomes unhealthy/terminates/goesDown
Your site will remain down (It will remain up if you use ELB+Scaling Policies)
You will have to remap your elastic IP
You pay for the time your elastic IP is not pointing to an instance around $0.005/hr
You get 750 hours of Elastic Load Balancing plus 15 GB data processing with the free tier so why not use it along with a min=1,max=1 scaling policy
On top of the answer about making SSL support easier by putting a load balancer in front of your EC2 instance, another potential benefit is HTTP/2. An Application Load Balancer (ALB) will automatically handle HTTP/2 traffic and convert up to 128 parallel requests to individual HTTP/1.1 requests across all healthy targets.
For more information, see: https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html#listener-configuration
It really depends on what are you running in the EC2 instance.
While with only one EC2 instance it's not necessary to use ELB (all your traffic will go to that instance anyways), if your EC2 service has to scale in the near future, is not a bad idea to invest some time now and get familiar with ELB.
This way, when you need to scale, it's just a matter of firing up additional instances, because you have the ELB part done.
If your EC2 service won't scale in the near future, don't worry too much!
About the second part, you definitely can route directly to your EC2 instance, you just need the EC2 instance IP. Take a look at the amazon route53 docs. Mind that if your IP is not static (you don't setup an Amazon Elastic IP), you'd need to change the IP mapping everytime the EC2 ip changes.
You can also use an ELB in front of EC2 if for example you want it to be publically reachable, without having to use up an Elastic IP address. As said previously they work well too with ASG's
I have a set-up running on Amazon cloud with a couple of EC2 Instances running through a load balancer.
It is important that the site has a unique(static) IP or set of IPs as I'm plugging in 3rd party APIs which only accept requests made from IPs which have been added to their whitelist.
So basically unless we can give these 3rd parties a static IP or range of IPs that the requests from the site will always come from then we would be unable to make any calls to them.
Anyone knows how to achieve this as I know that Elastic IPs are not compatible with load balancers?
If I were to look up the IP of the load balancer DNS name (e.g. dualstack.awseb-BAMobile-ENV-xxxxxxxxx.eu-west-1.elb.amazonaws.com resolves to 200.200.200.200) would that IP be Static?
Any help/advise is greatly appreciated guys.
The ip addresses of your load balancer is not static. In any event, your incoming load balancer IP wouldn't be used for outgoing connections.
You could assign elastic IPs to the actual instances behind the load balancer, which would then be used for outgoing requests. You get 5 free elastic ips, and I believe you can apply for more if you need them.
Additionally if using a VPC and if your instances are in a private subnet then they will only be able to access the internet via the NAT instance(s) you setup, and you can of course assign an elastic IP to the NAT instances
This is an old question, but things have changed now.
Now you can create a Network ELB to get a LB with a static IP.
from https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html
Support for static IP addresses for the load balancer. You can also
assign one Elastic IP address per subnet enabled for the load
balancer.
https://aws.amazon.com/blogs/aws/new-network-load-balancer-effortless-scaling-to-millions-of-requests-per-second/
You can attache an additional ENI (Elastic Network Interface) to an instance in your VPC. This way the ELB (Elastic Load Balancer) routes the incoming Internet requests to the web server, and the additional ENI will be used to connect to your 3rd party (or internal) requests (Management network)
You can see more details about it in the VPC documentations
Really the only way I am aware of doing this is by setting up your instances within a VPC and having dedicated NAT instances by which all outbound traffic is routed.
Here is a link to the AWS documentation on how to set up NAT instances:
http://docs.amazonwebservices.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html
You CAN attach an elastic IP to the instances BUT NOT to the ELB (which is what the client sees).
You could use a full reverse proxy layer 7 load balancer like HAProxy:
Or a commercial implementation like Loadbalancer.org or Riverbed (Zeus)
They both are in the AWS Marketplace:
Your outbound requests to your 3rd party APIs will NOT go out via the ELB/ALB. That's for incoming connections. If you need an inbound static IP you'll probably need to forego the loadbalancer (or figure out how to implement Anshu's suggestion to attach an elastic IP to the loadbalancers, the doc is light on details). Update: I found some documentation that ALB use static addresses (and I just tried binding an elastic IP to one to be sure and that failed).
If you're talking about outbound connections see below:
If your server is deployed in a public subnet you can attach an
elastic IP to that host. Outbound communications will go out over
that address.
If your server is deployed in a private subnet there's
a NAT gateway attached to it. All outbound traffic from your private
subnet will go out over that interface.
You could use as already mentioned loadbalancer.org appliance in AWS. It would replace the AWS NAT instance and give greater functionality and include both Layer4 and Layer7, along with SSL termination and a WAF.
Best of all you get free support in your 30 day trial in AWS to help you get up and running.
Yes I am biased as I work for loadbalancer.org however I would say nothing ventured nothing gained.
You can use a DNS service like DNSMadeeasy that allows "ANAME" records. These act like an A Record but can be pointed at a FQDN or IP. So in this case you can point it to the ELB DNS.
Dave