I have an instance running on premise and its IP address is changed regularly. My other services are running on AWS and they are using IP to connect to the premise's services. I have to update the IP address saved on AWS services whenever the IP is changed on premise network. I have a thought about using DNS but it is still a need to update A record.
I am looking for a way to do some auto-detect instead of manual updating. I wonder whether I can use load balancer to do the check. I know there will be a range of IP addresses on premise network. Can load balancer do a health check on these IP within the range? So my AWS service can send request to the load balancer. Is there any side-effect on this approach?
You need to use hostname instead of IP address as you mentoned the IP addresses keeps changing. AWS VPC can use a DNS forwarder like Unbound, which can forward the requests to your on premise DNS server when VPC resolution is unable to resolve the hostnames. This appraoch is quite effective as you send only those DN resolution to on-premise DNS that are missed by AWS VPC DNS.
Unbound allows resolution of requests originating from AWS by
forwarding them to your on-premises environment—and vice versa. For
the purposes of this post, I will focus on a basic installation of
Amazon Linux with the configuration necessary to direct traffic to
on-premises environments or to the Amazon VPC–provided DNS, as
appropriate. Review the Unbound documentation for details and other
configuration options.
Further reading : How to setup DNS resolution from AWS to on premise servers
Related
I have successfully built a VPN connection between gcp and aws using the following guide(https://cloud.google.com/solutions/automated-network-deployment-multicloud).
I can currently ping the resources on the other cloud providers based on the private IP. However, I would like to use the dns resolution that resolves to private IP of the AWS resource DNS names. Can someone please help me with this?. Using DNS server policy may not be the best of options for me as it points to alternative name server only and not the gcp’s internal name servers anymore. So how can I use forwarding zones in gcp for DNS names such as database-test.c34fdgt1ascxz.us-west-1.rds.amazonaws.com so that it resolves to private IP. The above example is for database which I have not made public. Has someone done this already? Or does anyone have any idea on how to go about this. Any help is much appreciated, thank you so much.
It is possible.
If your goal is to configure outbound forwarding to AWS, then you should remove this policy you just need a Cloud DNS managed zone to accomplish this.
The DNS queries that are forwarded from GCP to AWS will come from the 35.199.192.0/19 address block.
The 35.199.192.0/19 traffic can be routed over a dynamic VPN tunnel dynamic (BGP), so you would just need to modify your AWS VPN gateway or router by adding a route that to reach 35.199.192.0/19.
It looks like a public address block, but Google uses this block only for forwarding, and does not announce it on the public Internet.
And finally, AWS needs to be configured so that responses to DNS queries from 35.199.192.0/19 are routed back to GCP using the VPN tunnel configured between AWS and GCP.
In other words, this traffic needs to go through the VPN tunnel.
To debug it you can use stackdriver logging and also by checking network captures on both endpoints.
Check this documentation guides: Creating Forward zones1 and DNS forwarding2.
You can't resolve AWS private IP addresses by submitting the AWS public endpoint to GCP's DNS. That just wont work.
AWS uses a service called Route53 resolver that will forward requests that can't be resolved internally to an external DNS server that you specify. We use this in our env's to resolve on-prem corp IP's that are not part of Route53. I have not tried this, but it's possible you can use that to point to GCP DNS.
I have a load balancer configured to have an IPV4 Ip address. However, the provided IP is a DNS mapped IP address to the load balancer of the format *.ap-south-1.elb.amazonaws.com.
I need to configure IOT devices to send data to the load balancer and they do not support DNS. How can I assign a static IP address like ... to my load balancer so that I can configure my IOT devices to send data to it.
The Elastic IPs section does not provide a facility to allocate it to a load balancer and only supports ec2 instances.
Conclusion:
I have found a way to use DNS on my IOT device and working on this was vital. I am now aware of the option of manually hosting a load-balancer on an EC2 instance. A simper alternative is forwarding all requests at an elastic IP addressed EC2 instance to the load balancer. However, this will cause a bottleneck at the transparent proxy. Hence, I think using the DNS feature on the IOT device is the best option.
Elastic Load Balancers do not support static IP addresses. They only support DNS CNAMEs (or Aliases if you are using Route 53). This is because ELB DNS entries will resolve to different IP addresses depending on how it is scaling between availability zones. Also, over time, the IP addresses will/may change.
The AWS documentation also specifically states to create CNAME-records only when mapping custom DNS entries to your ELB. If you are using Route 53, you can create an Alias record, which look like an A-record to the outside world.
If you need a static IP address, then you cannot use ELB.
Instead, you will need to manage your own load balancer (HAProxy, nginx, etc.) on an EC2 instance using an Elastic IP address.
It would not be possible to assign a static IP with the elastic load balancer. You need to use DNS name only.
The only way I am aware of doing this is by setting up your instances within a VPC and having dedicated NAT instances by which all outbound traffic is routed.
Here is a link to the AWS documentation on how to set up NAT instances:
http://docs.amazonwebservices.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html
AWS Elastic Load Balancer does not support assigning a static IP address due to many reasons.
Looking at your problem, the issue you are facing is having large amount of data sources to pump data to AWS. I suggest you to use AWS Kinesis Firehose service instead of the current approach as Firehose specifically focus on streaming data into AWS.
How can I assign a static IP address to a ELB. Seems like I cannot.
Some articles online asks to create a Route 53 record but this requires changing CNAME of domain which also redirect email traffic. I just want to change A record not CNAME.
Some articles also mention that I can use a EC2 instance as a reverse proxy. But will a single proxy be able to handle a lot of traffic?
Any solution for this?
AWS' Elastic Load Balancer is actually elastic on two levels as described here:
http://shlomoswidler.com/2009/07/elastic-in-elastic-load-balancing-elb.html
The first level is the load balancer itself. In order to make sure that ELB can scale to whatever volume you have and burst to whatever volume you suddenly encounter, AWS assigns a 'static' DNS hostname (e.g. MyDomainELB-918273645.us-east-1.elb.amazonaws.com). That hostname points to multiple IP addresses. You can see that (from a command line) by running
$ host MyDomainELB-918273645.us-east-1.elb.amazonaws.com
MyDomainELB-918273645.us-east-1.elb.amazonaws.com 172.31.7.2
MyDomainELB-918273645.us-east-1.elb.amazonaws.com 172.31.11.33
The second form of elasticity within the ELB is obviously then ELB directing the query to one of your EC2 instances in the pool.
So, you can see that trying to assign a static IP address to the load balancer would be self-defeating.
Using an EC2 instance as a reverse proxy would also seem self-defeating as you would then create a bottleneck before even getting to the ELB. Might as well just create your own load balancer.
The recommended solution (which you've pointed out) is to create a CNAME that points to the ELB hostname (which won't change).
i.e. my-app.mycompany.com ->
MyDomainELB-918273645.us-east-1.elb.amazonaws.com
This would allow you to integrate your scalable application, behind the ELB within your domain.
I'm not sure I fully understand why you cannot create a CNAME in your DNS or what that has to do with directing email traffic, can you explain?
A new feature in AWS (I believe it was announced at Re:Invent 2017) allows for static IPs with Network Load Balancers (NLB). NLB can only handle layer 4 (TCP) and not HTTP specifics (layer 7).
You can assign one Elastic IP address per availability zone.
For details see the AWS blog post or the NLB documentation.
The "Classic Load Balancer" and "Application Load Balancer" do not support static IPs. If you need a feature only provided by those, you have to fall back to the CNAME solution described above.
A blog was recently published by AWS support on this topic leveraging NLB to provide static IP to Classic and Application load balancer - https://aws.amazon.com/blogs/networking-and-content-delivery/using-static-ip-addresses-for-application-load-balancers/
Summary of solution as described by the post
We end up with a TCP listener on a NLB that accepts traffic and forwards it to an internal ALB. The ALB terminates TLS, examines HTTP headers, and routes requests based on your configured rules to target groups with your instances, servers, or containers. The AWS Lambda function keeps everything in sync by watching the ALB for IP address changes and updating the NLB target group. In the end we’ll have a few static IP addresses that are easy for whitelisting, and we won’t lose any of the benefits of ALB. Note that we will be sending all of the traffic through two load balancers
I found setting up AWS Global Accelerator very straight forward and simple. It created 2 static IP Addresses and a static DNS pointing to my Application load balancer.
Configuring Global Accelerator
Set listeners as TCP port 80, 443
Select your load balancer endpoint (AWS Global Accelerator Configuration)
Add cname record for your dns pointing to the static dns it created
(mywebsite.com > globalacceleratorDNS.com). If any client needs to
whitelist, give them the 2 static IP it created
Pricing is $18 per month + a few pennies per GB of data transfer.
I'm pretty sure its cheaper than the NLB, Nat Gateway, Elastic IP setup.
https://docs.aws.amazon.com/global-accelerator/latest/dg/about-accelerators.html
For little traffic, it might be a solution to set up an EC2 Instance running Nginx as a forwarding proxy.
So you can use the EC2's static IP Address to forward your traffic resolving the ALB's DNS name.
However, it's a kind of a hack, but using a Global Accelerator or an NLB seems to me also like a hack :-)
Unlike the Network Load Balancer, the Application Load Balancer (ALB) does not support Elastic IPs, but that's not the worst part. If you use Route 53 together with the ALB, the DNS automatically sets the TTL to 60 seconds. This appears to be causing problems for our institutional - mainly government - customers running older Windows DNS servers. They just can't keep up with the ALB's Listener changing its public-facing IP on such a short notice. Older DNS infrastructure is either not respecting or is not capable of handling such aggressive TTL.
While I don't like it, AWS recommends to put a Network Load Balancer in front of the Application Load Balancer, per here: https://aws.amazon.com/blogs/networking-and-content-delivery/using-static-ip-addresses-for-application-load-balancers/
I'm a backend developer who is a very beginner in Server Administrative here.
I have ec2 instance sitting on AWS with Elastic IP assigned to it.
There are several domains point to this Elastic IP (from my clients).
The server then, response with appropriate content.
At this point, I would like to start using load balance as the number of traffics is increasing.
The question is
Would I be able to apply Load Balancer while still not changing IP Address (Elastic IP) that domains pointing to?
I would be a bit difficult to get to all clients and ask them to update the DNS record point to new IP Address.
So having the same IP would be an ideal solution for me.
It will be a big help if you guys could shed light for me on this.
No. The ELB has a DNS entry that you can setup on your DNS provider or on Route 53. This DNS is the only external link to the ELB and ELBs don't have elastic IPs. Also, keep in mind that the DNS doesn't change on your ELB but the IP can change at discretion of Amazon.
Also, Amazon uses it's own internal mechanism to access instances behind load balancer and I'm pretty sure it doesn't use external addresses (most likely internal)
Another alternative is to setup your own HAproxy instance in EC2 and configure it with an Elastic IP, but that's a little bit more work. (Also, you have to monitor it closely as you would have to manage it in case of changes in traffic patterns)
Hope it helps.
With a webserver (apache or nginx) I am able to find the x-forwarded-for header and find the client IP rather than the ELB's IP.
Can I do the same thing using IPTables so that I am able to block certain IP addresses?
I can do this at the webserver level. However, I think this is a bit inefficient and I am hoping I can achieve this with IP Tables or something similar?
Since July 30th ELB supports Proxy Protocol. As stated in the end of this thread in the AWS forum:
Elastic Load Balancing (ELB) now supports Proxy Protocol version 1.
This feature allows you to identify the client’s connection
information when using TCP load balancing, providing additional
insight into visitors to your applications. Having this information
can be useful for analyzing traffic logs, gathering connection
statistics, troubleshooting, or managing whitelists of IP addresses.
You must enable Proxy Protocol in the ELB.
The developers guide has more information about Proxy Protocol.
You can't do this with iptables, because iptables will only ever see the IP address of the elastic load balancer, since the ELB is what is establishing the connections to your instance.
Using the web server to block certain x-forwarded-for values isn't particularly inefficient, but if you want to control who can access your ELB by IP address, that can also be accomplished with the Security Group attached to the ELB.
Update: Your comment is partially correct, because, at least as of now, ELB on EC2 "classic" does not support an inbound security group or network access control list, but ELB on VPC does.
Q: Can I configure a security group for the front-end of the Elastic Load Balancer?
If you are using Amazon Virtual Private Cloud, you can configure security groups for the front-end of your Elastic Load Balancer. — http://aws.amazon.com/ec2/faqs/#ELB6
Security groups are easiest to use when you need to allow a relatively small set of specific IP address ranges. If you want to allow most but block a few, then a VPC Network Access Control List is the easier approach.