I'm using aws ec2 instances as web servers.
There are more then 20 web servers and they has to connect to some external services. Those external services has ip based security rules. Because of this reason I have to use a only one or two public ip address to connect those services.
How can i route outgoing traffics to use only one public ip address ?
Yes, you could use a NAT instance for that.
Just make sure your instance is large enough to accommodate the desired throughput.
See: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html
Related
Lets say I have a service running clustered on N ec2 instances. On top of that I have Amazon EKS and Elastic Loadbalancer. There is a service not managed by me running outside of AWS where I have an account that my services in AWS are using via HTTP requests. When I made an account to this external service I was asked for an IP (range) of services which will be using this external service. There is my problem. Currently lets say I have 3 EC2 instances with Elastic IP addresses (which are static), so I can just give those three IP addresses to this external service provider and everything works just fine. But in the future I might add more EC2 instances to scale out and whitelisting new IP addresses in the external service is a pain. In some cases those whitelist change requests may take for a week to approve by the external service provider and I dont have that time. Even further, accessing this external service is the only reason I go for static IPs for the EC2 instances. So if possible I would ditch the Elastic IPs.
So my question is how could I act so that if I make requests outside of the AWS in a random instance in my cluster, external service providers would always see the same IP address for me as a service consumer?
Disclaimer: I actually dont have that setup running yet, but I am in the middle of doing research if that would be a feasible option. So forgive me if my question sounds dumb for some obvious reason
Something like Network address translation (NAT) can solve your problem. A NAT gateway with Elastic IP, used for rerouting all traffic through it.
NAT gateway provided by AWS as service can be expensive if your data traffic is big, so you can make your own NAT instance, but that is bit complicated to set up and maintain.
The main difference between NAT gateway and NAT instance are listed here
The example bellow is assumed that EC2 instances are in private subnet, but it doesn't have to be a case.
I believe you need a proxy server in your environment with an Elastic IP. Basically you can use something like NGINX/Apache and configure it with an elastic IP. Configure the webserver to provide an endpoint to your EC2 instances, and doing a proxy pass to the external endpoint.
For high availability, you can manage a proxy in each availability zone, ideally configured using an auto scaling group to keep at leaset one instance alive in each AZ. Going through this approach, you will need to make sure that you assign the public IP from your elastic IP pool.
Generally, hostnames are better alternative to the IP addresses to avoid such situations as they can provide a static endpoint no matter what is the IP behind. Not sure whether you can explore that path with your external API provider. It can be challenging when there is static IP based routing/whitelisting rules in place.
This is what a NAT Gateway is for. NAT Gateways have an Elastic IP attached and allow the instances inside a VPC to make outbound connections, transparently, using the gateway's static address.
After restarting AWS EC2, hostname & public IP gets changed.
Remote docker clients get affected as they rely(export DOCKER_HOST) on these public names.
How to resolve this dynamic IP(public) problem of EC2?
By default, AWS assigned public IP addresses as well as hostnames are ephemeral, meaning they will be released back to the pool if you restart the instance. If you really need a persistent IP address, you can use Elastic IPs, but bear in mind there’s a limit per region.
Note: I’d still recommend evaluating the need for using a public IP from the IPv4 pool, as they are a rare resource. Most of the times, one can get by well by using the correct combination of security groups and private IPs, along with Route53 hosted zones for friendly naming, assuming instances are in the same VPC or can communicate via VPC peering.
We are trying to use Elastic Load Balancing in AWS with auto-scaling so we can scale in and out as needed.
Our application consists of several smaller applications, they are all on the same subnet and the same VPC.
We want to put our ELB between one of our apps and the rest.
Problem is we want the load balancer to be working both internally between different apps using an API and also internet-facing because our application still has some usage that should be done externally and not through the API.
I've read this question but I could not figure out exactly how to do it from there, it does not really specify any steps or maybe I did understand it very well.
Can we have an ELB that is both internal and external?
For the record, I can only access this network through a VPN.
It is not possible to for an Elastic Load Balancer to have both a public IP address and a private IP address. It is one or the other, but not both.
If you want your ELB to have a private IP address, then it cannot listen to requests from the internet.
If your ELB is public-facing, you can still call to it from your internal EC2 instances using the public endpoint. However, there are some caveats that goes with this:
The traffic will exit your VPC and re-enter it. It will not be direct instance-to-ELB connection that a private IP address will afford you.
You also cannot use security groups in your security group rules.
There are 3 alternative scenarios:
Duplicate the ELB and EC2 instances, one dedicated to private traffic, one dedicated to public traffic.
Have 2 ELBs (one public, one private) that share the same back-end EC2 instances.
Don't use an ELB for either private or public traffic, and instead use an Elastic IP address (if public) or a private IP address (if private) on a single EC2 instance.
I disagree with #MattHouser answer. Actually, in a VPC, your ELB have all its internal interfaces listed in Network Interfaces with Public IP AND Primary private IP.
I've tested the private IP of my public ELB and it's working exactly like the external one.
The problem is : theses IPs are not listed anywhere in a up to date manner like on a private ELB DNS. So you have to do it by yourself.
I've made a little POC script on this, with an internal Route53 hosted zone : https://gist.github.com/darylounet/3c6253c60b7dc52da927b80a0ae8d428
I made a Lambda function that checks which private IPs are set to the loadbalancer and will update Route53 record when it changes: https://github.com/Bramzor/lambda-sync-private-elb-ips
Using this function, you can easily make use of the ELB for private traffic. I personally use it to connect multiple regions to each other over a VPC inter-region peering without needing an additional ELB.
The standard AWS solution would be to have an extra internal ELB for this.
Looks like #DaryL has an interesting workaround, but it could fail for 5 minutes if the DNS is not updated. Also there is no way to have a separate security group for the internal IPs since they share the ENI and security of the external IP of the ELB.
I faced the same challenge and I can confirm the best solution so far is to have two different ALBs, one internet-facing and the other internal. You can attach both ALBs to a single AutoScaling Group so you can access the same cluster.
Make sure the networking options (Subnets, security groups) of both ALBs are the same in order for both to access the same cluster instances. Autoscaling and Launch Configuration works seamlessly with both ALBs attached to the same AutoSacling Group. This is also working with ALBs created from ElasticBeanstalk environments.
This has probably been answered elsewhere but I can't seem to find it!
I have a number of AWS EC2 instances that I am using as part of a project being built and I am now looking into securing the setup a bit. I want to lock down access to some of the ports.
For example I want to have one of the instances act as a database server (hosting mysql). I want this to be closed to public access but open to access from my other EC2 instances on their private IP's.
I also use the AWS auto-scaler to add/remove instances as required and need these to be able to access the DB server without having to manually add its IP to a list.
Similarly if possible I want to lock down some instances so that they can only accept traffic from an AWS Load Balancer. So port 80 is open on the instance but only for traffic coming from the Load Balancer.
I've looked at specifying the IP's using CIDR notation but can't seem to get it working. From the look of the private IP's being assigned to my instances the first two octets remain the same and the last two vary. But opening it to all instances with the same first two octets doesn't seem that secure either?!
Thanks
What you want to do is all pretty standard stuff, and is extensively documented in the AWS VPC documentation for Virtual Private Clouds. If your EC2 instances are not running in a VPC, they should be.
The link below should help, it seems to be your scenario:
Scenario 2: VPC with Public and Private Subnets (NAT)
The configuration for this scenario includes a VPC with a public
subnet and private subnet, and a network address translation (NAT)
instance in the public subnet. A NAT instance enables instances in the
private subnet to initiate outbound traffic to the Internet. We
recommend this scenario if you want to run a public-facing web
application, while maintaining back-end servers that aren't publicly
accessible. A common example is a multi-tier website, with the web
servers in a public subnet and the database servers in a private
subnet. You can set up security and routing so that the web servers
can communicate with the database servers.
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Scenario2.html
I have several EC2 instances which need to have a white listed IP address to talk with an external service. Is it possible to have these all route through the same Elastic IP when they make external calls using strictly AWS features (not another nginx reverse proxy server)?
I need them to all go through 1 IP so that I can support auto scaling.
Update 12/18/2015:
There is now a managed NAT solution that can be used to solve this problem.
See: https://aws.amazon.com/about-aws/whats-new/2015/12/introducing-amazon-vpc-nat-gateway-a-managed-nat-service/
Original Answer:
You basically need a NAT host and you need to point all traffic that has a destination outside of the VPC to the NAT instance.
Give the NAT instance the whitelisted ip.
Here are the docs:
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html