It takes several minutes for a newly deployed version to ElasticBeanstalk to become available, so I am hoping that someone can spare me all the testing/experimenting :-)
Scenario 1:
I need to connect to an EC2 instance running MySQL from another EC2 instance but belonging to a different security group. Do I use the public DNS or the private IP to specify the MySQL host?
Scenario 2:
Same as above except both instances belong to the same security group. I believe that I need to use the private IP in this case, correct? Would the public DNS also work?
Thank you!
You should always use the private IP when possible for ec2 instances communicating with each other.
Among other reasons, you will get charged money for using the public ip even though the machines are started in the same availability zone.
Also, the security group is just a set of inbound and outbound rules, it doesn't matter that the two machines are in different groups with different rules so long as your mysql server can accept traffic on the port from the other ec2 instance based on the ruleset.
If you're going to be starting and stopping instances frequently, you might benefit from creating an elastic IP and attaching it to instances as needed instead of constantly changing configuration files.
Related
Does anyone know the correct AWS services needed to launch multiple instances of a Docker image on unique publicly accessible IP addresses?
Every path I have tried with Amazon's ECS seems to be set up for scaling instances locked away in a private network and / or behind a single IP.
The container has instances of a web application running on port 8080, but ideally the end user will connect via port 80.
The objective is to be able to launch around 20 identical copies of the container at once, with each accessible via its own public IP.
There is no need for the public IP to be known in advance, as on startup, I patch the data as needed with the current IP address.
The containers live in Amazon's ECR, and there are a couple of unique instances running in standalone EC2 machines, I was trying to use ECS to launch multiple instances at will, but can successfully launch a total of 1 at a time before getting errors about conflicting ports because things are not isolated enough.
You can do this with ECS:
Change your task definition to use the awsvpc networking mode.
Change your service network configuration to auto-assign a public IP.
If you're deploying onto EC2 instances, I think you may be limited in the number of either network interfaces or public IP addresses that you can use. Fargate does not have this restriction.
I have four EC2 instances, three of them running api services and another running user interface (UI). The UI instance obtains the data over api calls to another instances. Right now everthing works fine becouse im using the public IP provided for eeach EC2 service for api calling. But, mi cocern is about what happend if the public ip of service change (for any reason)? then miy application go down becouse UI cannot get the data from services. After a little researching i have found that appers to be a solution: use a vpc for connect EC2 instances over private ip (because is static) and associed the UI instance to an Elastic IP (no problem here). Sow, i have some issues:
1) I make a test putting all instances in the same vpc (and sub net) but when I do ping from one to another the pings faild. Its my approach right? or i missing some thing?
2) I read a couple of another options but im not sure what is best: Maybe i have to use an Api Gateway?. Or a NAT Gateway?
3) What is the standar practice to communicate EC2 instances in private way?
1) I make a test putting all instances in the same vpc (and sub net) but when I do ping from one to another the pings faild. Its my approach right? or i missing some thing?
For security reasons, AWS block the ICMP traffic using security group. Please enable Ping traffic (ICMP) in security group from the Ip's you are trying to connect, it's better to allow the entire CIDR block for the VPC for all traffic, will make your life a lot easy. Please make sure you do this in a test Environment only.
2) I read a couple of another options but im not sure what is best: Maybe i have to use an Api Gateway?. Or a NAT Gateway?
Also, as you mentioned that your concern is that the public IP of the Instance will change, (definitely if your Instance stop/starts for any reason), but why don't you use Elastic IP for all of your Instances, that could be on of the solution, but using this approach all of your instances will be exposed to internet, so going with private IP is the best option.
3) What is the standard practice to communicate EC2 instances in private way?
It depends on the use case, if your Instances are in the same vpc no extra configuration is required, you only need to make sure the security groups, Network Access Control List and firewall configuration are correct.
In case if your instances are in different VPC, then you can use VPC Peering/Transit gateway.
1.) You need to update security groups with the permission to ICMP traffic.
Go to your VPC -> Select Security Groups -> Select the relevant security group -> Add Inbound/Outbound rule for all traffic with CIDR of the instance subnet.
2.) Internal network is the better way as long as all your traffic gonna be internal.
Thanks
After restarting AWS EC2, hostname & public IP gets changed.
Remote docker clients get affected as they rely(export DOCKER_HOST) on these public names.
How to resolve this dynamic IP(public) problem of EC2?
By default, AWS assigned public IP addresses as well as hostnames are ephemeral, meaning they will be released back to the pool if you restart the instance. If you really need a persistent IP address, you can use Elastic IPs, but bear in mind there’s a limit per region.
Note: I’d still recommend evaluating the need for using a public IP from the IPv4 pool, as they are a rare resource. Most of the times, one can get by well by using the correct combination of security groups and private IPs, along with Route53 hosted zones for friendly naming, assuming instances are in the same VPC or can communicate via VPC peering.
I am trying to setup couchbase cluster on AWS. I want my nodes to be distributed across multiple availability zones.
Ec2 instances with in an availability zone are able to access each other using the ip (Private DNS) which is assigned to them during creation and which does not change even if we restart the machine.
I am not able to access an Ec2 instance from other AZ using this (Private DNS). One way this can be done is by using Elastic Ip which has a limit per region.
Question here is How to access one Ec2 instance from other EC2 instance in another AZ without elastic ip?
You do not want to use Elastic IP for this. Your statement that Elastic IP is a solution to your issue is not correct. You want to use the Private IP assigned to the instance when you created it.
The private IP will not change as long as the instances are deployed inside a VPC.
You have to use the private IP in order to keep all network traffic inside the VPC. Then you just need to make sure your Security Groups are configured correctly to allow traffic between the instances.
Amazon Web Services Operates Split-horizon DNS (AKA Split-Brain DNS). The best practice when deploying couchbase onto EC2 is to use hostnames not IP addresses, see http://developer.couchbase.com/documentation/server/current/install/cloud-deployment.html . Amazon will automatically give a different IP when resolving the hostname depending if the source of the request is internal or external.
We have a setup where 3 ec2 instances each are associated with an elastic ip on its primary network interface eth0 so incoming requests can be served by these instances.
Each of these instances has a secondary network interface eth1 where in the event of a failure/ crash/ reboot of an instance, the elastic ip associated with that instance would be associated to one of the remaining running ec2 instances on that interface. This is some sort of failover mechanism as we always want those elastic ips to be served by some running instance so we don't lose any incoming requests.
The problem I have experienced is specifically on reboot of an instance. When an instance reboots, it cannot get back the public ip it had where this public ip is that of the elastic ip that is now associated with another instance. Thus this instance cannot access the internet unless I manually re-assign the elastic ip back to this instance.
Is it possible to automatically reclaim/re-associate the elastic ip it once had onto its eth1 interface on reboot? If not, do you have suggestions for a workaround?
Reboot is necessary as we would be doing unattended upgrades on the instances.
Update:
Also note that I need to use these elastic ips as they are the ones allowed in the firewall of a partner company we integrate with. Using ELBs won't work as its IP changes over time.
So here's how I finally solved this problem. What I missed out on was that Amazon only provides a new public IP to an instance under two conditions.
Its elastic IP is detached
It has just one network interface
So based on this, on startup, i configure the instance with two instances but i detach the secondary eth1 interface. Hence this makes the instance eligible for getting a new public IP (if for any reason it reboots).
Now for failover, once one of the running instances detects an instance has gone offline from the cluster (in this case, lets say it rebooted), it will then on the fly attach the secondary interface and associate the elastic IP to it. Hence, the elastic IP is now being served by atleast one of the running instances. The effect is instant.
Now when the failed instance comes back up after reboot, amazon already provided it a new non-elastic public IP. This was because it fulfilled the two conditions of having just one network interface and also its elastic IP was disassociated and re-associated to another running instance. Hence, this rebooted instance now has a new public IP and can connect to the internet on startup and do the necessary tasks it needs to configure itself and re-join the cluster. After that it re-associates back the elastic ip it needed to have.
Also, when the running instance that took over the elastic IP detects a new instance or the rebooted instance has come online, it detaches the secondary interface again so it would be eligible to get a new public ip as well if it rebooted.
This is how i handle the failover and making sure the elastic ips are always served. However this solution is not perfect and can be improved. It can scale to handling N failed/rebooted instances provided N network interfaces can be used for failover!
However if the instance that attached secondary interface(s) during failover reboots, it will not get a new public IP and will remain disconnected from the cluster, but atleast the elastic IPs would still be served by remaining live instances. This is only in the case of reboots.
BTW, atleast from all that i read, these conditions of getting a new public ip wasn't clearly mentioned in the amazon docs.
It sounds like you would be better served by using an elastic load balancer (ELB). You could just use one ELB and it would serve requests to your 3 application servers.
If one goes down, the ELB detects that and stops routing requests there. When it comes back online, the ELB detects that and adds it to the routing group again.
http://aws.amazon.com/elasticloadbalancing/