I need to provide my customers with fixed urls that don't change when the EC2 instances are stopped/started because sometimes we need to change the size of the EC2 and when we restart the instance the public IP has changed.
I thought on using Elastic IPs so I can keep the same public IP when the instance is rebooted, but I've seen that Amazon tells you that you only have 5 Elastic IPs. If you ask them they say that they can give you more, but I guess they're not giving you 10.000 of them.
How can I use a single public Elastic IP to give each user different URLs for out service?
It would be something like this, being 11.22.33.44 the Elastic IP and 192.168.0.X two EC2 instances:
11.22.33.44:**1000** --> 192.168.0.**1**:22
11.22.33.44:**1001** --> 192.168.0.**1**:80
11.22.33.44:**1002** --> 192.168.0.**1**:443
11.22.33.44:**1003** --> 192.168.0.**2**:22
11.22.33.44:**1004** --> 192.168.0.**2**:80
11.22.33.44:**1005** --> 192.168.0.**2**:443
I need to make it work programmatically, as I'm creating EC2 instances from the SDK as needed.
Another way I thought is using subdomains from my .com domain that points to the current public IP of each EC2 instance, but using the IP as I described before sounds better.
The issue is that instances are receiving new (temporary) Public IP addresses after they are stopped and started.
A simple way to handle this is to add a script to each instance that runs during every boot. This script can update a DNS record to point it at the instance.
The script should go into the /var/lib/cloud/scripts/per-boot directory, which will cause Cloud-Init to automatically run the script each time the instance is started.
# Set these values based on your Route 53 Record Sets
ZONE_ID=Z3NAOAOAABC1XY
RECORD_SET=my-domain.com
# Extract information about the Instance
INSTANCE_ID=$(curl -s http://169.254.169.254/latest/meta-data/instance-id/)
AZ=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone/)
MY_IP=$(curl -s http://169.254.169.254/latest/meta-data/public-ipv4/)
# Extract Name tag associated with instance
NAME_TAG=$(aws ec2 describe-tags --region ${AZ::-1} --filters "Name=resource-id,Values=${INSTANCE_ID}" --query 'Tags[?Key==`Name`].Value' --output text)
# Update Route 53 Record Set based on the Name tag to the current Public IP address of the Instance
aws route53 change-resource-record-sets --hosted-zone-id $ZONE_ID --change-batch '{"Changes":[{"Action":"UPSERT","ResourceRecordSet":{"Name":"'$NAME_TAG.$RECORD_SET'","Type":"A","TTL":300,"ResourceRecords":[{"Value":"'$MY_IP'"}]}}]}'
The script will extract the Name tag of the instance and update the corresponding Record Set in Route 53. (Feel free to change this to use a different Tag.) The instance will also require IAM permissions for ec2 describe-tags and route53 change-resource-record-sets.
Update: I've turned this answer into a blog post: Amazon Route 53: How to automatically update IP addresses without using Elastic IPs
Related
I am using a YAML file in the CloudFormation service on AWS. This file creates an EC2 server with a public IP and a stack output within the CloudFormation using the IP of the EC2 server that it just created. The output however, is not something we use a lot. Therefore I would like to close the server from time to time and open it again whenever I need it. The problem is that every time I launch the EC2 server again, it changes its public IP and the IP of the stack output doesn't change with it.
I found a way to make it static, using an elastic IP address from the EC2 service. However, I can't seem to find a way to select that IP address when choosing properties in creating the stack. So I would like some assistance on this.
You cannot define the IP address yourself, but you can extract it after it has been generated.
In your Cloudformation template, add an Outputs section like the following:
Outputs:
myEc2IP: # just some identifier
Value: !GetAtt myInstance.PublicIp # assuming that "myInstance" is your resource
Then, after deploying your stack, you can use the AWS CLI to extract the value:
aws cloudformation describe-stacks --stack-name $YOUR_STACK \
--query 'Stacks[0].Outputs[?OutputKey==`myEc2IP`].OutputValue' \
--output text
You can even load this into a shell variable by something like
export MY_ROLE_ARN="$(aws cloudformation describe-stacks …)"
Learn more about Outputs: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html
See other possible output/return values for EC2 instances in Cloudformation: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-ec2-instance.html (in the “Return values” section).
I can't seem to find a way to select that IP address when choosing properties in creating the stack. So I would like some assistance on this.
You can't select the IP address that you will get. Elastic IP addresses are chosen from Amazon's pool of IP addresses.
If you want to specify a specific IP address that you own, you can use Bring your own IP addresses (BYOIP) in Amazon EC2. That will allow you to specify the IP that you own to be used by your EC2 instances.
There is a topic in EC2 documentation Changing the System Hostname. Why does one need to change it? Just for fun? Just to have some nice shell prompt?
// change this
ubuntu#ip-123-12-1-231 ~ $
// to this?
ubuntu#my-beautiful-hostname ~ $
I'm learning how AWS DNS work, where my EC2's DNS lives that resolves a default Public DNS name to Public IP address of my instance
Public DNS: ec2-xx-xx-xxx-xx.ap-southeast-2.compute.amazonaws.com
Public IP: xx-xx-xxx-xx
And how can I host multiple apps with real domain names (example1.com, example2.com, so on) in one EC2 instance, how to modify and manage DNS. And actually I don't know what to read about it in docs, and read everything related to hostnames and DNS, and found this topic Changing the System Hostname and don't understand why would one want to change a hostname and if it can be valuable info for me.
UPD:
And now a real a practical question for those specimens who like closing questions quietly.
Where does a DNS live in EC2 instance? How is Public DNS mapped to Public IP? Where is that record in my EC2 Ubuntu instance? Is Route53 involved in it?
Where does a DNS live in EC2 instance?
It doesn't, DNS resolution use by the server is set in /etc/resolv.conf and /etc/nsswitch.conf. The hostname domain name for that server is set (Redhat derived systems) in /etc/sysconfig/network
How is Public DNS mapped to Public IP?
With a DNS record
Where is that record in my EC2 Ubuntu instance?
In the DNS for the domain that you have attached it to
Is Route53 involved in it?
Only if you are using Route53 for DNS
EC2 DNS location (source):
In EC2-Classic, the Amazon DNS server is located at 172.16.0.23.
In EC2-VPC, the Amazon DNS server is located at the base of your VPC network range plus two.
For more information, see Amazon DNS Server in the Amazon VPC User Guide
Well i had the same issue as you did and someone replied me this
It isn't a huge deal if you are just running a single server, mostly
to help you identify a server with local networking. Some things like
mail servers will use your hostname unless you specify otherwise.
This is an example of somewhere I saw that done
My original query
why do some people set hostname and some dont? whats the use?
hostnamectl set-hostname
We have a vpc in aws, it there some feature provided by aws to automatically add private DNS to route53 when manually launch an instance.
If you have installed boto, you can add / update DNS settings like this:
DNS_ADDRESS="`ec2metadata | grep 'public-hostname:' | cut -d ' ' -f 2`"
/usr/local/bin/route53 change_record <Hosted zone ID> foo.bar.com. CNAME $DNS_ADDRESS 300
DNS_ADDRESS is the AWS DNS name of the instance such as ec2-1-2-3-4.eu-west-1.compute.amazonaws.com
The second line creates a CNAME with 5min TTL. You need to add the hosted zone ID of the domain you want to change.
If you put this snippet into /etc/rc.local, the instance will automatically update / create the CNAME when it's booting.
I'm not sure what you mean with "private" DNS. Route53 provides a (public) DNS service. However, if your instances are inside a VPC there's no harm to make the DNS entries public, since no one can access them anyway.
Sample Script:
/usr/local/bin/route53 change_record $Hosted_zone_ID $DNS-Name A $IP 300
$Hosted_zone_ID, $DNS-Name, $IP are place holders. Replace them with actual values to run script
There is no feature built-in to AWS that does this automatically, but AWS provides a feature that allows you to customize all kinds of things using simple workflows and a bit of Javascript.
For example, we use an auto scaling group to launch instances as needed, and we have used this blog article as a sample of how to set up an AWS Lambda function that receives SNS notifications from the auto scaling group when an instance goes up and down, and adds a custom DNS name to the route 53 zone. It should be pretty straight forward to customize it for your needs.
Currently there is a cluster of 3 rabbitmq nodes, with each one in a different aws zones than any of the others. I have been able to cluster them and prove that is working.
The challenge I am running into is using autoscaling and automatically connecting new nodes to the cluster. From what I understand all of the other nodes must know about each other and have the other's IP's in their /etc/hosts file.
Is there a way to just search for the cluster name and tell the nodes to connect?
I have tried setting nodes behind ELBs and pointing them to the ELBs instead of IP's; however, that did not to work.
for anyone else who is trying this the work around I did was to instal aws-cli. After doing that I wrote a script to change the /etc/hosts file based on the three different elb's I had (one for each node)
The script first finds that the ec2 instance behind the elb by running this command
aws elb describe-instance-health --load-balancer-name <NAME OF YOUR ELB>
The script then gets the ip of the ec2 behind the elb by running this command
aws ec2 describe-instances --instance-ids <THE VALUE RETURNED FROM ABOVE> --query 'Reservations[*].Instances[*].PrivateIpAddress' --output text
The trick that we are doing is running this in a script where it goes over each elb individually and there is only one elb infront of each node. This allows for a node to go down, be autoscalled back up and us to know the new ip without having to worry about manually edditing the file.
Thus we are just running over each one and telling it to save the output to the /etc/hosts file.
Hope that helps someone
I have an Amazon EC2 instance. Lets say my public DNS is
ec2-54-200-54-33.us-west-2.compute.amazonaws.com
I am using this instance to help develop a new website, and I want to map the host
dev01.company.com
to this instance. How do I do that? I think there are at most 2 steps, including mapping the host with the registrar, and possibly something in Amazon. What are the steps I would need to do to accomplish this? How do I take all the details of my instance and map the hosts as metioned above?
1) Assign an Elastic IP to your EC2 instance (you can do that through the AWS Management Console).
2) Add an "A" record for "dev01" pointing to that IP, using the tool provided by your domain registrar.
Amazon recommends using a CNAME record instead
dev01.company.com CNAME ec2-54-200-54-33.us-west-2.compute.amazonaws.com
You will also need to open up whatever ports you need accessible to the security group of the EC2 instance (ie TCP 80 open to 0.0.0.0/0 > this will open to everyone)