Im using ECS to create a service that uses the awsvpc network type - thus creating ENIs on my host instances giving my ec2 instances two internal network configs and two internal Ips.
I cant seem to be able to pull out any ENI information out of the instance metadata http://169.254.169.254/latest/meta-data/ - only the original internal Ip.
Is there any way i can discover the ENI ip via querying the instance metadata or other method from within the docker container without resorting to aws-cli ?
it is possible to get the ip address of the ENI adapter with the instance meta-data:
curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/
then sub in the macs to get both private ips:
curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/<mac address>/local-ipv4s
Related
I created an elastic IP for my EC2 Instance and Route53 routes instance to my domain using that elastic IP.
However, someone mentioned the idea of connecting the Elastic IP to a network interface and then connecting the network interface to the EC2 Instance...
What's the point in adding the network interface in between and should I do it?
Elastic IP is always attached to a network interface.
However, when you attach address directly to an instance, AWS attaches it to instance's primary network interface, replacing any existing IP this interface had already attached to it. So the use case for attaching IP to an interface first, for example, includes the need to preserve the IP which is already associated with instance's primary NI.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/elastic-ip-addresses-eip.html#eip-basics
If you created multiple network interfaces, you could select to associate the EIP to a particular NI. You may have created multiple NI for load balancing, private communication a backup network between instances. You could configure connection to database servers as such.
I created a task definition that runs a Neo4j database, opened the ports 7474 and 7687 for the database http and bolt connectors, and launched the task definition in a ECS service of type Fargate in a private subnetwork.
How can I get the IP address of the container running the database without an ALB?
Thanks
There are a couple of ways to obtain the ip address of your running container. Through the console the private subnet should be accessible as a link, and when you click on the subnet you should be able to see the private ip address.
You can also make use of the command line to describe your tasks. The describe output should contain the ip address of your running conatiner:
https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_DescribeTasks.html#API_DescribeTasks_RequestSyntax
Finally, in Fargate there is also a metadata endpoint that is exposed in each running container, if need be, you can programmatically access that endpoint to obtain the ip address in your private subnet:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-metadata-endpoint.html#task-metadata-response
I am trying to setup couchbase cluster on AWS. I want my nodes to be distributed across multiple availability zones.
Ec2 instances with in an availability zone are able to access each other using the ip (Private DNS) which is assigned to them during creation and which does not change even if we restart the machine.
I am not able to access an Ec2 instance from other AZ using this (Private DNS). One way this can be done is by using Elastic Ip which has a limit per region.
Question here is How to access one Ec2 instance from other EC2 instance in another AZ without elastic ip?
You do not want to use Elastic IP for this. Your statement that Elastic IP is a solution to your issue is not correct. You want to use the Private IP assigned to the instance when you created it.
The private IP will not change as long as the instances are deployed inside a VPC.
You have to use the private IP in order to keep all network traffic inside the VPC. Then you just need to make sure your Security Groups are configured correctly to allow traffic between the instances.
Amazon Web Services Operates Split-horizon DNS (AKA Split-Brain DNS). The best practice when deploying couchbase onto EC2 is to use hostnames not IP addresses, see http://developer.couchbase.com/documentation/server/current/install/cloud-deployment.html . Amazon will automatically give a different IP when resolving the hostname depending if the source of the request is internal or external.
What should be the appropriate security group rules to allow all EC2 instances attached to a single SecurityGroup to talk to each other using their ElasticIPs instead of their PrivateIPs
ILLUSTRATING THE DIAGRAM ABOVE I have a single Public Subnet in a AmazonVPC. The subnet has 3 to many more EC2 instances: DevOps instance, PHP instance, Python instance, and several others. The DevOps instances is attached to 2 security groups SG1 & SG2. The PHP/Python instances are attached to only 1 security group SG2. Given the SG1 & SG2 rules shown in the table: the DevOps instance can be accessed from the world via SSH port:22 only; the PHP/Python instances can be accessed from the world via HTTP port:80 only; all ports in all the DevOps/PHP/Python instances can be accessed from all EC2 instances that are attached to the same security group only.
USE CASE: I would like to SSH into the DevOps instance from the internet using its ElasticIP 9.8.7.1. Then from the DevOps instance i would like to SSH into PHP/Python instance using their ElasticIPs 9.8.7.2 / 9.8.7.3 (not using their PrivateIPs). Also, my PHP app should be able to talk to the Python app via ElasticIP and vice versa - because the ElasticIPs are what we have in our source codes of the apps.
PROBLEM1: Given the security group rules shown in the table, currently i am able to SSH from DevOps instance to PHP/Python instances by using their PrivateIPs only and not by using their ElasticIPs.
PROBLEM2: My PHP/Python instances are also not able to communicate with each other via their ElasticIPs.
CONSTRAINT: We regularly launch new instances for our PHP/Python apps from new/updated AMIs. Every time we launch a new instance, the PrivateIP changes. But I need to perform regular communication via something which does not change (i.e. the ElasticIP). We keep a ~/.ssh/config file in the DevOps instance so that doing SSH into the app instances are easy. In the ssh config file we have kept ElasticIPs as the IP address for our PHP/Python/Other instances. It's not possible to change the IP address in the ssh config file to a new PrivateIP address every time an app instance is replaced by a new one.
The simplest solution is to use the IP addresses' hostnames, instead of IP addresses directly, because VPC does some magical mapping for you.
Take this elastic IP for example:
$ nslookup 203.0.113.50
50.113.0.203.in-addr.arpa name = ec2-203-0-113-50.compute-1.amazonaws.com.
From inside the same VPC, this hostname does not resolve back to the elastic IP. Instead, it resolves to the private IP.
$ nslookup ec2-203-0-113-50.compute-1.amazonaws.com.
Name: ec2-203-0-113-50.compute-1.amazonaws.com.
Address: 172.31.10.25 # returns the private IP of the attached instance
Yet from outside the VPC, it resolves more like you would expect:
$ nslookup ec2-203-0-113-50.compute-1.amazonaws.com.
Name: ec2-203-0-113-50.compute-1.amazonaws.com.
Address: 203.0.113.50 # returns the elastic IP
By extension, then, if you were to configure a DNS CNAME devops.example.com pointing to the hostname of the elastic address of the devops machine, you'd get the external IP if you accessed it externally, but the internal IP if you accessed it internally.
Since internal requests would use the private IPs, no unusual security group configuration is needed.
Additional motivation: you pay for the traffic to leave your VPC and come back if you configure your instances to communicate using Elastic IPs within the same availability zone, $0.01/GB each direction, but not when using private IPs. See data transfer pricing. Presumably the difference is because you're using more of the AWS network hardware to translate and retranslate the traffic, which traverses the Internet Gateway twice when you use elastic IPs.
The example above is a real example from my network, with my EIP replaced with a placeholder from RFC-5737.
1) you can manually add inbound rules for SG2 like: 1) AllPorts EIP1 2) AllPorts EIP2
2) Do the same with script as below:
Make your script get the instances in a security group using the aws cli or API.
Example:
aws ec2 describe-instances --filter "Name=instance.group-name,Values=SG2"
this will get all the instances in secuirty group SG2. Save the instance ids in a dict called SG2_Instances.
Then run: aws ec2 describe-addresses.
This will give for each elastic ip, the corresponding fields including the instances ids if associated. Put all the instance ids along with EIP in a list. The list will look like:
[(EIP1, instanceId1), (EIP2,InstanceId2),..].
Now iterate through the list and check whether the instanceId in list is present in your dict SG2_Instances. If yes, then you can add a rule like mentioned in 1) using the corresponding EIP.
We do continuous integration from Jenkins, and have Jenkins deploy to an EC2 instance. This EC2 instance exports an NFS share of the deployed code to EC2 processing nodes. The processing nodes mount the NFS share.
Jenkins needs to be able to "find" this code-sharing EC2 instance and scp freshly-built code, and the processing nodes need to "find" this code-sharing EC2 instance and mount its NFS share.
These communications happen over private IP space, with our on-premise Jenkins communicating with our EC2 in a Direct Connect VPC subnet, not using public IP addresses.
Is there a straightforward way to reliably "address" (by static private IP address, hostname, or some other method) this code-sharing EC2 that receives scp'd builds and exports them via NFS? We determine the subnet at launch, of course, but we don't know how to protect against changes of IP address if the instance is terminated and relaunched.
We're also eagerly considering other methods for deployment, such as the new EFS or S3, but those will have to wait a little bit until we have the bandwidth for them.
Thanks!
-Greg
If it is a single instance at any given time that is this "code-sharing" instance you can assign an Elastic IP to it when after you've launched it. This will give you a fixed public IP that you can target.
Elastic IPs are reserved and static until you release them. Keep in mind that they cost money when they are not reserved.
Further on you can use SecurityGroups to limit access to the instance.
In the end, we created & saved a network interface, assigning one of our private IPs to it. When recycling the EC2, and making a new one in the same role, we just assign that saved interface (with its IP) to the new EC2. Seems to get the job done for us!