I have cloudera installed in one m1.large instance .
I was able install it successfully.
But once i stop the instance and start it again next morning the IP changes and cloudera still refers to old IP and none of the services run .
How can i solve this problem because i can not keep instance up whole day?
Since its for academic purpose and I don't have a budget .
This doesn't really have anything to do with Cloudera -- more the behavior of EC2 Instances. The feature you're looking for is called an Elastic IP Address, which is a static IP address that you can assign to your EC2 instance. Elastic IP behavior has some differences in EC2-Classic and EC2-VPC, noted here. Most notably for you will be instance startup behavior regarding Elastic IP addresses, which I'll get to shortly.
You can find details on Elastic IP pricing here. Note that while your first Elastic IP address (associated with a running instance is free), you will be charged by the hour for the time that your instance is offline and you still have the Elastic IP Address. This rate is "$0.005 per Elastic IP address not associated with a running instance per hour on a pro rata basis." So not very expensive, but there is some expense to be aware of.
Another recent question has a couple of good answers on how to allocate the Elastic IP address on instance startup. In EC2-Classic you will need to script it or do it manually. In VPC the Elastic IP will remain associated with the instance. (I think you're still charged the fee, though, per this documentation claiming that a small hourly fee is charged if an Elastic IP is associated with a stopped instance.)
If any expensive is prohibitive, I would recommend looking into AWS Command Line Interface to get the new public IP of the EC2 instance on startup and writing a script to change that IP address in your configuration.
You can setup your machine inside a VPC:
By launching your instances into a VPC instead of EC2-Classic, you
gain the ability to:
Assign static private IP addresses to your instances that persist
across starts and stops
Related
The question came from this question: Do you get charged for a 'stopped' instance on EC2?
Elastic IPs are heavily charged when not in use, according to the Amazon docs. Does that count (are you going to get charged for not using the elastic IP) if you associate elastic IPs to a stopped ec2 VM?
From the official documentation (emphasis mine):
To ensure efficient use of Elastic IP addresses, we impose a small
hourly charge if an Elastic IP address is not associated with a
running instance, or if it is associated with a stopped instance or an
unattached network interface.
So you can't avoid the charge by associating it with a stopped instance.
I am new to AWS and tried to create an EC2 instance.
I have a domain and ready to modify the A record to the associated EC2 instance.
I found an article that said an elastic IP is required for associating a production domain.
But AWS provides a public IP and it is accessible on the public internet too (i know it changes after restart, just assuming its okay to modify the A record after the machine is restarted - actually it is not restart very often).
In this case, is it a must that to assign an elastic IP to the instance (this instance contains no load balancing, it is only a simple single instance)?
If yes, why is it necessary?
An Elastic IP (EIP) is not necessary provided that you understand the limitations of public IPs. You may not reboot your instance, but AWS might for any number of reasons. This means that the public IP address could change when you are not expecting it.
When an EIP is assigned to a running EC2 instance, there are no charges for the EIP e.g. it is free. Therefore why go thru the hassle of needing to monitor your public IP address.
I have upgraded an ec2 instance from t2.micro to t2.medium, Now the public IP for my instance is changed.
And I have put the new public IP in DNS type A., But services are not accessible when I am hitting with the domain name in the URL. When I hit the services using public IP its working fine.
Note: When I check logs I am still getting traffic but don't know why I am not able to access these services.
It's a critical issue. Any help is really appreciated.
When you change IP address in DNS you need to wait for propagation of such change (what can take up to 24h) as DNS is distributed system using a lot of cache to reduce load. To reduce that time in future you can try to set TTL for your record to some small number like 60s but please note that TTL is more like hint for DNS clients than restriction so you can still experience problems.
I would suggest to attach Elastic IP address to your EC2 instance - that address is static and you can detach it and attach to new EC2 instance so change in DNS is not required and still you can update/upgrade your EC2 instance.
We have a setup where 3 ec2 instances each are associated with an elastic ip on its primary network interface eth0 so incoming requests can be served by these instances.
Each of these instances has a secondary network interface eth1 where in the event of a failure/ crash/ reboot of an instance, the elastic ip associated with that instance would be associated to one of the remaining running ec2 instances on that interface. This is some sort of failover mechanism as we always want those elastic ips to be served by some running instance so we don't lose any incoming requests.
The problem I have experienced is specifically on reboot of an instance. When an instance reboots, it cannot get back the public ip it had where this public ip is that of the elastic ip that is now associated with another instance. Thus this instance cannot access the internet unless I manually re-assign the elastic ip back to this instance.
Is it possible to automatically reclaim/re-associate the elastic ip it once had onto its eth1 interface on reboot? If not, do you have suggestions for a workaround?
Reboot is necessary as we would be doing unattended upgrades on the instances.
Update:
Also note that I need to use these elastic ips as they are the ones allowed in the firewall of a partner company we integrate with. Using ELBs won't work as its IP changes over time.
So here's how I finally solved this problem. What I missed out on was that Amazon only provides a new public IP to an instance under two conditions.
Its elastic IP is detached
It has just one network interface
So based on this, on startup, i configure the instance with two instances but i detach the secondary eth1 interface. Hence this makes the instance eligible for getting a new public IP (if for any reason it reboots).
Now for failover, once one of the running instances detects an instance has gone offline from the cluster (in this case, lets say it rebooted), it will then on the fly attach the secondary interface and associate the elastic IP to it. Hence, the elastic IP is now being served by atleast one of the running instances. The effect is instant.
Now when the failed instance comes back up after reboot, amazon already provided it a new non-elastic public IP. This was because it fulfilled the two conditions of having just one network interface and also its elastic IP was disassociated and re-associated to another running instance. Hence, this rebooted instance now has a new public IP and can connect to the internet on startup and do the necessary tasks it needs to configure itself and re-join the cluster. After that it re-associates back the elastic ip it needed to have.
Also, when the running instance that took over the elastic IP detects a new instance or the rebooted instance has come online, it detaches the secondary interface again so it would be eligible to get a new public ip as well if it rebooted.
This is how i handle the failover and making sure the elastic ips are always served. However this solution is not perfect and can be improved. It can scale to handling N failed/rebooted instances provided N network interfaces can be used for failover!
However if the instance that attached secondary interface(s) during failover reboots, it will not get a new public IP and will remain disconnected from the cluster, but atleast the elastic IPs would still be served by remaining live instances. This is only in the case of reboots.
BTW, atleast from all that i read, these conditions of getting a new public ip wasn't clearly mentioned in the amazon docs.
It sounds like you would be better served by using an elastic load balancer (ELB). You could just use one ELB and it would serve requests to your 3 application servers.
If one goes down, the ELB detects that and stops routing requests there. When it comes back online, the ELB detects that and adds it to the routing group again.
http://aws.amazon.com/elasticloadbalancing/
I had an issues with my AWS EC2 instance today. It was hanging with cpu at 100%...not sure why, but that's for another post.
I stopped the instance, and restarted it. After that, I couldn't get to my website. The Public DNS changed (I think this is normal as amazon may have moved me to another machine). I then looked at Elastic IP, and the entry is still there, but in the 'Instance' column, it was blank. I had to click 'Associate Address' and set the instance.
Is this normal? I was under the impression that Elastic IP should not need to be touched even after a stop/start. Did something go wrong? Is there something I can to find out why this happened?
Is the instance in EC2 classic or EC2 VPC?
The elastic IP is disassociated when you stop the instance in EC2-Classic, but is retained in EC2-VPC. So I assume your instance is in EC2-Classic, where the elastic IP is disassociated when the instance is stopped. You need to re-associate when you start the instance.
For more info
EC2-Classic: We disassociate any Elastic IP address (EIP) that's
associated with the instance. You're charged for Elastic IP addresses
that aren't associated with an instance. When you restart the
instance, you must associate the Elastic IP address with the instance;
we don't do this automatically.
EC2-VPC: The instance retains its associated Elastic IP addresses
(EIP). You're charged for any Elastic IP addresses associated with a
stopped instance.