Is there a way to duplicate a EC2 instance in the same VPC in different AZ for HA purpose, so that when the primary instance is bad (e.g. due to check status failure), I can shut it down and quickly switch over to the standby one.
For some reason, I need to keep the same private IP address, and public/EIP IP address.
For the EIP address, I can dis-associate it from the OLD instance, and re-associate it to the NEW instance, but how about the private IP address?
Any suggestion? Thanks in advance...
What is "an instance"?
It's worth thinking about what people mean when they talk about "duplicating an EC2 instance". An EC2 instance consists of:
The boot disk with Operating System
Any data disks
Network settings (eg public & private IP addresses)
Configurations (eg Instance Type, User Data, Tags, etc)
Duplicating an EC2 instance usually means launching another instance with the same configuration, but it's not necessarily a "duplicate". For example, a different Availability Zone means it will have a different IP address (see below) and the new instance will be booted from an AMI (Amazon Machine Image) rather than being an exact duplicate of the disk from the previous instance.
Duplicates for High Availability
Next comes the issue of High Availability. A new instance can take a few minutes to launch. If the requirement is for near-instant cut-over, then the only solution is to always run more than one instance, and then re-point an IP address or DNS name.
As you mentioned, it is easy and fast to associate an Elastic IP Address to an alternate EC2 instance. This change immediately redirects traffic sent to that IP address.
However, it is not possible to reassign an Internal IP Address to another instance (but see below).
Using Auto Scaling to launch another instance
If the requirement allows for a few minutes of outage, then more possibilities arise. The simplest would be to launch the EC2 instance within an Auto Scaling group. The group can be configured to always have a certain number of instances (eg a minimum of 1 instance). Thus, when an instance fails, Auto Scaling can automatically launch a replacement instance with the same configuration (boot disk, instance type, etc).
Further, Auto Scaling can automatically launch instances in another Availability Zone if a zone fails.
However, please note that internal IP address ranges are associated with Subnets within a VPC (Virtual Private Cloud). Each subnet is associated with a single Availability Zone. The hierarchy is:
VPC
Availability Zone
Subnet (with CIDR range of IP addresses)
Instance
Thus, launching an instance in a different Availability Zone (and thus a different Subnet) will require the instance to have a different Internal IP Address.
A hack for reassigning IP addresses
While Internal IP Addresses cannot be reassigned (and especially not between Subnets), an interesting hack was described in the ARC401 session at re:Invent 2014 (see slides 33 & 34, or on YouTube).
This involved associating an IP address with a secondary Elastic Network Interface (ENI), where the IP address falls outside the VPC range. Then, use routing rules to route the traffic destined for that address to the ENI (turning off Source/Dest Check). Effectively, the traffic can be re-routed to a different instance by modifying the routing rules. A bit of a hack, but it apparently works.
You may not be able to keep the same private, primary IP address, but you cano assign a secondary private IP address to an EC2 instance in a VPC, and they are reassignable:
Private IP addresses
When you launch an instance into a VPC, a primary private IP address
from the address range of the subnet is assigned to the default
network interface (eth0) of the instance. If you don't specify a
primary private IP address, we select an available IP address in the
subnet range for you.
You can assign additional private IP addresses, known as secondary
private IP addresses, to instances that are running in a VPC. Unlike a
primary private IP address, you can reassign a secondary private IP
address from one network interface to another.
From here: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-ip-addressing.html
Related
I have an instance inside a VPC with DNS hostnames & DNS resolution enabled
Today, each instances I spawn get the hostname & Private IP DNS name as
ip-xx-xx-xx-xx.ec2.internal
The problem with this is everytime my instance is terminated and I have to replace it, the IP address is going to change. Other machines in the same VPC need a deterministic way to access this instance. e.g. by connecting to mymasterinstance.dev. Any idea how do I give an internal / private DNS for my instance?
Private IP addresses on Amazon EC2 instances do not change. Therefore, the other machines could simply reference the private IP address.
Alternatively, you can create a "Private hosted zone" in Amazon Route 53 and create an A-Record to associate a domain name (eg mymasterinstance.dev) with the private IP address of the instance.
The public IP address on an instance will change when the instance is Stopped and Started (but will not change when the instance is Rebooted). If you require a static public IP address, you can create an Elastic IP Address and associate it with the instance.
Another perhaps simpler way to do this is to add a second IP address to your EC2 instance. When you terminate an instance, and add back a "replacement" instance, set its second IP address to what the terminated instance's second IP address was. Then you can do all of your referencing directly by IP address and you have full control over that second IP address. None of your configs needs to change.
I have a scenario where I want to setup an haproxy cluster with keepalived. I saw this blog post https://www.peternijssen.nl/high-availability-haproxy-keepalived-aws/ and its pretty detailed and easy to follow, but for my scenario I need the elastic IP's to be private. Is this doable ? As from AWS documentation Elastic IP's are only public IPs.
I need a floating IP which will act as a floating cluster IP. What can I use if Elastic IP is not an option.
An elastic IP within AWS is actually a static public IP address. This functionality is required as by default a public IP address is not reserved by an account, and when enabled limits the total IPv4 addresses you can keep.
For private address ranges however, your instances can be assigned any private IP address that is within any of your VPCs CIDR range(s).
In fact for this reason you can select almost any IP address found within your VPC when launching a new instance, in addition the IP addresses that are attached to an EC2 instance are static and will be attached to that instance until it is terminated.
Therefore as long as you don't terminate the instances the private IP(s) will be maintained. However, in the event the instance fails and is not recoverable you would still need to launch a new instance and use that IP.
It might be more beneficial using a DNS record to resolve the IP just in case of an emergency, or failing that use a load balancer to distribute traffic.
For this case you can provision an Elastic Network Interface (ENI) with a static private IP address which you can attach/re-attach to EC2 instances. As long as you don’t delete the ENI, it will retain the I as p address. Find out more here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html
I have a MEAN application where I am using Private IP to provide the connection in between the server or application. And I am using ec2 Instance Private IP for this inside my application.
Now I want to use Auto Scaling for my Production Environment. But I am not able to allocate predefined Private IPs when launching ec2 Instance using Auto Scaling Group.
Is there a better way to do this? Perhaps through Launch Configurations?
What is the best method to allow ec2 instances launched by an Auto Scaling Group to be assigned a private IP?
There's no way to have specific private IP's reserved for the auto scaling group in the launch configurations for ec2 Instance.
However three possible workaround here.
As per as Private IP is concern kindly create small subnet and choose it when creating Auto Scaling Group.
Create ENI (network interface) and assign it Private IP address of your choice, In the Auto Scaling Launch Configuration write user data script which uses the AWS cli tools to discover the pre created ENI and attach it to the ec2 Instance.
You may also try adding Secondary Private IP address to instance primary interface using user data.
Amazon EC2 Auto Scaling adjusts capacity by:
Launching new instances
Terminating existing instances
When a new instance is launched, it receives a private IP address in the CIDR range of the associated subnet. While it is normally possible in Amazon EC2 to request a specific IP address (which is done via a DHCP reservation), this is not possible with Auto Scaling. Instead, each instance will receive a random address from the subnet's CIDR range.
If your application really does require a range of pre-defined IP addresses, you could use a pool of Elastic IP addresses and have each new instance "grab" one of these addresses. However, traffic within a VPC that refers to a public IP address (and an Elastic IP address is a public IP address) has a charge of 1c/GB because it goes out, and then back into, the VPC. I wouldn't recommend this approach.
Given that there could be a variable number of instances operating at any time, and therefore assuming that your application can handle the fact that it might have an IP address registered but not in use, you could just give it every IP address in the CIDR range. You could even make a smaller subnet so the range is not very big.
I have created an EC2 instance behind an ELB and the hostname is
mysubdomain.domain.com
The instance is reachable via the internet from my local workstation (have opened all connections from My IP --> to the instance in the security group it belongs to);
However, when performing
curl mysubdomain.domain.com
from within the instance, it times out;
Do I need to assign a public (I assume I want it to be elastic so that I don't have to change it every now and then in my security group inbound rules) and add an allow rule in my security group (that the instance belongs to) from that specific IP?
Is there another way to go about it, given that I have reached the limit of Elastic IPs?
For an Amazon EC2 instance to access the Internet, it must either be:
In a public subnet with a public IP address, or
In a private subnet with a NAT Gateway or similar NAT service
By default, security groups allow all outbound traffic, so you will not need to modify the security groups.
Even if you have reached the limit of your Elastic IP addresses, you can launch the EC2 instance with a Public IP address via Auto-assign Public IP — this is different to an Elastic IP address, in that it is assigned when the instance is Started and might change when the instance is Stopped/Started. However, it will work perfectly fine to obtain Internet access.
If an instance is behind a load balancer, there is no reason to want to reach that instance directly from the Internet. Thus, there is no need for an Elastic IP address. In fact, in best-practice architectures, the instances should be in a Private Subnet so that they are better protected from the Internet. This will then require a NAT Gateway or NAT Instance to enable the instance to access the Internet.
I have a windows EC2 instance running a production website and DNS is configured to have my domain name point to its public IP. There is currently no Elastic IP (EIP) associated with the instance. I would like to start using a Elastic IP and have my domain name point to it instead of the public IP (which can change if I ever have to change the instance).
Reading the documentation I find this statement troubling:
When you associate an EIP with an instance, the instance's current
public IP address is released to the EC2-Classic public IP address
pool.
My fear is this:
I assign an EIP to the instance and the public IP is released.
Now my website no longer works, because the domain name points to the public IP, which is no longer associated with my EC2 instance.
I must then point DNS records to the EIP. But this could take up to 48 hours for propagation to take place (i.e. my site may be be unreachable for up to 48 hours).
How can I do this without having to live through DNS propagation?
If your EC2 instance is in a VPC, you can add a second network interface onto your EC2 instance. You can associate your Elastic IP address with that second network interface. This way, your EC2 instance could respond to both IP addresses.
Instructions
Create a new Network Interface in the same subnet as your EC2 instance.
Allocate a new Elastic IP for your VPC (if you haven't done so already).
Associate the Elastic IP address with your new Network Interface (eni).
Attach your new Network Interface to your EC2 instance.
Do not change your DNS yet.
You may need to RDP/SSH into your EC2 instance to make some configuration changes to ensure your EC2 instance responds correctly to the new IP address.
Modify the hosts file on your local computer to test connecting to your website via the new IP address.
When that works, do the DNS switch and restore your hosts file.
48 to 72 hours before your pre-determined switch-over time, reduce the time-to-live (TTL) on your DNS entry to 300 seconds (5 minutes).
At your designated switch-over time:
Attach the Elastic IP address
Update your DNS entry to point to your Elastic IP address
Doing this, your effective "downtime" is reduced to 5 minutes.
You can have two identical EC2 instances. One with the old public IP where DNS record is pointing to. One with the EIP assigned. Requests should be able to access anyone of the two instances without noticing it. Your application must be able to scale horizontally. Then you change DNS record to point to EIP. Eventually, when DNS is updated, all requests to your domain will end up going to the EC2 instance with the EIP. At that moment you can stop or terminate the old EC2 instance.
Other possibility if your application cannot scale horizontally and if it is a web application, the web server in the old EC2 instance can redirect requests to the EIP. It would redirect to an IP address but it's a possibility.
If you don't use EC2-Classic instances, you can freely move the Elastic IP to any other EC2 instance without losing the EIP.
aws ec2 associate-address --region us-east-1 --allocation-id eipalloc-xxxxxxxxxxx --allow-reassociation --network-interface-id eni-xxxxxxxxxx
where eipalloc-xxxxxxxxxxx is the id of the Elastic IP and eni-xxxxx is the id of the target EC2 instance.
https://aws.amazon.com/premiumsupport/knowledge-center/ec2-recover-ip-address/
Elastic IP addresses
It's a best practice to use an Elastic IP address. Elastic IP addresses are allocated to your account, instead of to the instance. You can associate your Elastic IP addresses to and from instances as needed.
If you release the Elastic IP address that was allocated to your account, you might be able to recover it. For more information, see Recovering an Elastic IP address.
All instances except EC2-Classic instances retain their associated Elastic IP addresses when stopped. AWS continues to bill for Elastic IP addresses associated with a stopped instance.
Note: Elastic IP addresses associated with EC2-Classic instances aren't recoverable.