I take create an AMI image from a Linux running instance [ centos ]. It has only private IP. Some of the random instance while restoring in a different VPC, retains the old IP or we just couldn't reach it.
Source AMI server has:
[centos#ip-xxxx ~]$ cat /etc/sysconfig/network
# Created by cloud-init on instance boot automatically, do not edit.
#
NETWORKING=yes
[centos#ip-xxxx ~]$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
[centos#ip-xxxx ~]$ cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script
search ec2.internal
nameserver xxxx
Has anyone faced this? Why is it happening? I couldn't find any pattern. Similar servers from same VPC when I restore the AMI, some works, some doesn't randomnly. Its so confusing! Any help appreciated. Thanks.
An Amazon EC2 instance should not be hard-coding its IP address.
Rather, the private IP address is allocated to the instance via DHCP. The VPC will always serve the same private IP address to an instance (similar to having a DHCP reservation). However, a new instance launched from the same AMI is not considered to be the same instance, so it will receive a 'new' private IP address via DHCP. (It could be the same IP address if both subnets have the same CIDR range, but it is unlikely.)
If you are seeing an incorrect IP address associated with an instance, it is likely because somebody hard-coded it into a configuration file (which is not a good idea).
Related
I have one VPC with two Subnets (SubnetA and SubnetB).
My team wants to have multiple IPs assigned to the Instance, each from one subnet.
The Instance already have one Private IP (from SubnetA, Primary one) when I launched it, then I attached another Private IP from another SubnetB via the Console Attach network Interface option.
I can see both of the IPs in the console under Managed IP Address option.
I rebooted the Instance, and I was expecting to see both of the IPs when I do ifconfig, but I can see only the Primary one.
To cross-check if the Private IP is actually attached to the Instance, I queried Instance Metadata using the following commands :
curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/02:14:46:91:bc:34/local-ipv4s
curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/02:1d:2a:75:ax:04/local-ipv4s
I can see both of the IPs in the output for the above two commands respectively.
I checked the status of NETWORKMANAGER systemctl status NetworkManager
It was stopped
I started the Service and enabled NetworkManager automatically at boot time, using following commands:
systemctl start NetworkManager
systemctl enable NetworkManager
Then I checked the output of ifconfig
This time it showed me both of the MAC addresses, with the only difference for the second one I was not able to see the IP address. So basically the interface is up, so the underlying device is found. There is no IP address associated with this interface.
So I tried both of the options to associate IP:
Assign an IP address manually:
sudo ifconfig ens6 w.x.y.z
Or contact the DHCP server, if it exists, and let it provides an IP address for the interface:
sudo dhclient -v ens6
Both of them worked and I can see both of the IPs under inet.
The last problem was I have to do this every time I reboot the Instance.
So I was trying to add a permanent route using the following command:
sudo /sbin/route add default gw 1xx.xx.2xx.193
Here the IP is the second IP from the SubnetB, but I am getting the error :
SIOCADDRT: Network is unreachable
To solve the above problem what I did is, I was already having a file with
/etc/sysconfig/network-scripts/ifcfg-ens5 with details for Primary IP, I added one more file
/etc/sysconfig/network-scripts/ifcfg-ens6 with the necessary details for secondary IP
This is what I referred.
Rebooted and it is working.
But I am not able to ping the secondary IP.
I think I have to add one more Gateway from the second subnet but not sure about this.
What else needs to be done so that I can route traffic, ping, ssh using the Secondary IP.
Please refer to my VPC Subnet CIDRS:
Subnet A: 1.7.2.128/26
Subnet B: 1.7.2.192/26
Output of ip route:
Update:
Today when I started the server I am able to ping the Secondary IP(200), but not the Primary one(136), from one of my test Instances. Also, ssh is done using Primary IP.
ip route add default via 1XX.XX.XXX.X9X dev ens6 table 2000;
ip route add 1XX.7X.2XX.X9X dev ens6 table 2000;
ip rule add from 1XX.7X.2XX.1XX lookup 2000;
The above command helps me to resolve this issue and I am able to ping my secondary IP.
To make this configuration persist after reboots, the same commands, I have added into rc.local
In the first line, the IP is the Gateway IP (Second IP in the Subnet Range)
The IP mentioned in the second and third lines in the command is the actual Secondary IP of my Server.
Our company has a github setup in an aws EC2 instance. This of course will have private/public ip.
I have setup two ec2 instances in a different region.
However, when I try connection to our company's github from one instance, it will use the public ip of the github instance. In this case, the connection is successful and I can clone and do stuff.
But the other will use private ip of the github instance. In this case, it fails to connect to the github instance.
BTW, the second one(the one that fails) was migrated from the region where github instance is setup, to a different region. Saved the AMI and relaunched it in the new region.
Here is an example describing my symptom.
I will assume:
instanceA: migrated ec2 instance to new region
instanceB: newly created ec2 instance in new region
github instance: has domain name "git.blahblah.com". It has private IP of 172.31.xx.xx , public IP of 52.78.xx.xx
instanceA
$ ping git.blahblah.com
PING git.blahblah.com (172.31.xx.xx) 56(84) bytes of data.
instanceB
$ ping git.blahblah.com
PING git.blahblah.com (52.78.xx.xx) 56(84) bytes of data.
What can be causing it? Any hints on tackling this issue?
I have checked DNS settings by `$ cat /etc/resolv.conf
instanceA
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 172.30.0.2
search us-west-2.compute.internal
instanceB
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 172.30.0.2
search us-west-2.compute.internal
the security group of instanceA and instanceB is identical.
VPC ID, subnet ID of instanceA and instanceB is identical.
How dumb of me.
The cause of this was that in instanceA(migrated from a different region) had some configurations in /etc/hosts file that forced git.blahblah.com to refer to the private IP of it.
$ cat /etc/hosts
127.0.0.1 localhost
172.31.xx.xx git.blahblah.com
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
After I deleted 172.31.xx.xx git.blahblah.com line from the hosts file, it worked.
Sorry for my stupidity people.
Because your instances are in different regions from the github instance then they are not over the same network, and hence they can't connect to the private ip address.
If you want them to connect together over the same ip address then they need to be in the same VPC and to get there they need to be on the same region (but not the same availability zone)
A private IPv4 address is an IP address that's not reachable over the
Internet. You can use private IPv4 addresses for communication between
instances in the same network (EC2-Classic or a VPC).
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html#concepts-private-addresses
Every EC2 comes with a default network ETH0 and if we associate an elastic IP address to the default network interface, the IP is reachable.
Now, I disassociated the elastic IP to the default ETH0. Created a new network interface (ETH1) and associated the elastic ip. Attached the new network interface (ETH1) to the instance. But the elastic ip is not reachable or pingable.
Any idea why?
While AWS does allocate a secondary IP address for your instance, I don't think DHCP can be used to automatically configure the IP address on your Ubuntu instance.
You will need to add the secondary IP address to your system configuration. I found a few resources that suggest ways to do this.
From How to assign a second IP address on AWS Plesk for Ubuntu?:
Connect to the instance via SSH and edit the /etc/network/interfaces.d/50-cloud-init.cfg file. Add the following lines at the bottom:
auto eth0:1 iface eth0:1 inet static address 172.31.16.199 << Your actual address
Execute the following command to apply the changes:
systemctl restart networking
Other similar pages:
ubuntu - adding 2nd private IP on AWS EC2 instance - Server Fault
14.04 - add additional ipv4 addressess to the interface - Ask Ubuntu
networking - Add second IP-address to interface in 17.10 - Ask Ubuntu
When I spin up an Amazon EC2 CentOS 7 server in, say, availability zone us-east-1a, the server is automatically assigned a primary private IP address on eth0, such as 172.31.8.244/20 and a gateway of 172.31.0.1. If I then attach a second interface on eth1, I can specify the address, which needs to be within the 172.31.0.0/20 subnet (or one will be assigned to me automatically within that subnet). Eth1 will have the same gateway as eth0. Let's say I am assigned 172.31.12.121/20. I use the same security group on both eth0 and eth1, which allows SSH only in and everything out.
The problem is that when I try to SSH to eth0 from a different server, it works fine. But when I try to SSH to eth1 I get a timeout. ip addr and ip route show that both interfaces are up and have the correct routes. I can even SSH locally to eth1 and the /var/log/secure log shows the correct entries as when I SSH to eth0 bound to eth1. What do I need to do to be able to SSH to either interface from a different server?
The problem is asymmetric routing. A request to eth1 comes in eth1 and goes out eth0. The reply coming out on eth0 has a different IP address than in the request, and so it is dropped on the client side. The solution is to set up rules that allow responses to route through eth1.
First, make sure you have created an AMI of your server, because if you enter the wrong thing in following steps, you may lose all connectivity to the server and be unable to do anything but reboot it from the Amazon console web page.
Start off by setting the default route for each interface in separate tables:
ip route add default via 172.31.0.1 dev eth0 tab 1
ip route add default via 172.31.0.1 dev eth1 tab 2
To check those were properly added use:
ip route show table 1
ip route show table 2
Now you need to add rules that say to use the different tables depending on the source IP address:
ip rule add from 172.31.8.244/32 tab 1
ip rule add from 172.31.12.121/32 tab 2
You can check all of the rules with:
ip rule
You should now be able to connect to either IP address from a client machine. You can also use the bind option of SSH to connect from either interface on this server to a client machine:
ssh centos#client_ip_address -i mykey.pem (uses the default, eth0)
ssh -b 172.31.12.121 centos#client_ip_address -i mykey.pem (uses eth1)
ssh -b 172.31.8.244 centos#client_ip_address -i mykey.pem (uses eth0)
You can use both interfaces to connect to other EC2 servers in the same availability zone and for any interface that has a Public IP assigned to it, you can connect to the outside world or to other EC2 servers in the same VPC, even if they are in different availability zones.
But what if you want to connect to other EC2 servers that are in the same VPC but different availability zones? In other words, servers in the same data center. The problem is that the Private IP address is masked at 20 bits, which confines you to one availability zone. So for datacenter us-east-1 you have:
us-east-1a: 172.31.0.0/20
us-east-1b: 172.31.16.0/20
us-east-1d: 172.31.48.0/20
us-east-1e: 172.31.32.0/20
To connect across availability zones in one VPC and in one datacenter you need a 16-bit mask. ip addr will show:
inet 172.31.12.121/20 brd 172.31.31.255 scope dynamic eth1
If losf -n | egrep 172.31.12.121 shows you that this address is not in use you can add the new mask and delete the old. Note that the broadcast address has to change at the same time the mask changes:
ip addr add 172.31.12.121/16 dev eth1 brd 172.31.255.255
ip addr del 172.31.12.121/20 dev eth1
Now you should be able to connect from an EC2 server in availability zone A to another host in availability zone B, so long as they are in the same VPC, even if they do not have Public IP addresses.
Troubleshooting:
If you are having problems, try resetting both interfaces, which will remove any manual twiddling you have done. First copy /etc/sysconfig/network-scripts/ifcfg-eth0 to /etc/sysconfig/network-scripts/ifcfg-eth1, editing the second file to change the DEVICE from eth0 to eth1. Then add a line to /etc/sysconfig/network which says GATEWAYDEV=eth0. Finally, run /etc/init.d/network restart (no, it should not disconnect you). Then start over with the above commands.
I got the same answer everywhere on the internet regarding my question of how to get the IP address of an RDS instance in AWS, but I don't know what dig is, and how to use it.
How to allocate IP address in VPC to RDS instance?
Please help me find the IP address of my RDS instance in AWS?
It won't display the ip address on the configuration page for the RDS instance because RDS IP's are dynamic in nature. You can dig for them if you need them but you would be better server using the DNS endpoint for the instance. This remains static and can be found on the details tab of the RDS instance within AWS.
The IP address of your AWS instance will be the IP address of the RDS instance.
If your hostname of your AWS instance is blah.blah.amazonaws.com then you can use the dig command (on Linux machines) or ping command on both Windows and Linux to find out the IP address of the host:
ping blah.blah.amazonaws.com
This will give you back the IP address of the host (something like this):
c:\ping www.google.com
Pinging www.google.com [216.58.210.100] with 32 bytes of data:
Reply from 216.58.210.100: bytes=32 time=14ms TTL=58
Reply from 216.58.210.100: bytes=32 time=15ms TTL=58
Reply from 216.58.210.100: bytes=32 time=14ms TTL=58
So the IP address of www.google.com is 216.58.210.100. dig will give you more output but the IP address is also in there. But sometimes the actual IP address of your AWS instance is actually in the hostname itself. For example:
ip-12-34-56-78.us-west-2.compute.internal
The IP address 12.34.56.78 is already in the name (depending on your instance). You can find the hostname in the AWS Console/Dashboard.
Use this:
SELECT inet_server_addr();
Simple method to find RDS ip address: Open command prompt from local machine and ping your RDS Endpoint. You will see the reply from destination host ip address. That is your RDS ip address! Regardless if it is public accessible or not.
The answer given by Paolo Bolla is good, but only works for Postgresql.
For Microsoft SQL Server, please use:
SELECT
local_net_address = ConnectionProperty('local_net_address')
, local_tcp_port = ConnectionProperty('local_tcp_port')
You can also use Nslookup <<RDS SQL Server>> command to figure out the IP address as suggested in this AWS blog post.
dig is command-line utility that finds an IP address a domain name leads to.
You can simply install it using :
apt-get update && apt-get install dnsutils -y
Then to find an IP of RDS instance (with endpoint xxx.yyy.us-east-1.rds.amazonaws.com), run the following command :
dig xxx.yyy.us-east-1.rds.amazonaws.com
In the output of dig, you will be able to find IP address in ANSWER SECTION :
;; ANSWER SECTION:
xxx.yyy.us-east-1.rds.amazonaws.com. 5 IN A 172.30.1.253
As you can see RDS instance endpoint, resolved to A type of DNS record with Private IP address
On AWS Management Console - Location : AWS Management Console -> EC2 -> Network & Security, Network interfaces
You can see all of IP addresses and its resource's details on description tab together.