I want to run data analysis using AWS that'll probably take a few days. I've setup a system where spot instances will reload the same EBS volume so it can resume the analysis after a termination. So far I've got it working where I can just generate a persistent spot request and have the analysis keep running and continue after spot instance terminations until it is done.
But I also want the spot instance to cancel the persistent spot instance request when the analysis is finished. Is there anyway to get the request ID from within the spot instance?
The spot instance information is not available from the EC2 metadata. However, the instance ID is.
Get the Instance ID from the EC2 metadata.
Use DescribeInstances (or ec2-describe-instances, or aws ec2 describe-instances to get the instance information for the instance.
Within the instance information is spotInstanceRequestId which is the spot instance request ID you're looking for.
Related
I would like to use AWS EC2 spot requests for machine learning that I do in spots.
If you don't configure any settings for spot requests, they will be set to "fleet" and an EC2 instance will be created that cannot be paused (it can only be terminated).
If you know how to create an instance with a spot request that can be paused, please let me know.
Spot instances launched in a fleet can’t be paused or stopped. To stop a Spot instance, it must be launched by a Spot instance request for a single instance, you can create such requests using Launch Instance wizard, you may follow steps at this link. Make sure to select "Persistent request" in the wizard.
This request will launch a Spot instance that can be Stopped but still can’t be Paused. Hibernate/Pause option is only available as a behavior at the event of interruption and can't be requested by the user.
I got the mail from Amazon regarding my EC2 as follows:
EC2 has detected degradation of the underlying hardware hosting your Amazon EC2 instance (instance-ID: xxxxxxxx) associated with your AWS account (AWS Account ID: xxxxxx) in the ap-south-1 region. Due to this degradation your instance could already be unreachable. We will stop your instance after 2020-12-29 22:00:00 UTC. Please take appropriate action before this time.
The affected instances are listed below:
xxxxxxx
So what would I do now to keep my data safe ?
AWS provides guides on what to do when an instance is about to be retired:
What do I need to know when my Amazon EC2 instance is scheduled for retirement?
The easiest way would be to stop/start the instance:
You are required to stop and then start the instance at your preferred time before the instance retirement date. Stopping and starting the instance moves the instance to another healthy host.
However, exact details depend on your instance type (e.g. EBS based or instance store based), thus please read the guide in the link provided to understand your options which are mostly determined on your actual EC2 instance setup.
I'm having an issue with AWS boxes (EC2) where I terminate the box and it re-spawns. To give context, there is no autoscaling group. Anywhere I can search for some config that might be triggering the launch?
I would make sure you don't have a persistent spot request active in your account, and also check to see if you perhaps installed the AWS Instance scheduler - either or both of those could be starting instances on your behalf - (double check the autoscaling group, that is the most obvious reason though)
If you terminate a running Spot Instance that was launched by a
persistent Spot request, the Spot request returns to the open state so
that a new Spot Instance can be launched. To cancel a persistent Spot
request and terminate its Spot Instances, you must cancel the Spot
request first and then terminate the Spot Instances. Otherwise, the
persistent Spot request can launch a new instance.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html#terminating-a-spot-instance
https://aws.amazon.com/solutions/instance-scheduler/
So I found out the culprit, maybe it can help more people. Apparently, there is a service from AWS called OpsWorks that automates things like Cheff of Puppet, which my company had configured some time ago. That would be checking for instances running and triggering re-provisioning when it didn't see the instance running. OpsWorks is here
I am running amazon AWS ECS container which creates one single instance of EC2. I made sure that it is 1 instance when I created ECS.
My issue is that I have another instance running in EC2 and amazon has sent me an email that I am using double of the free quota and will be charged.
But I am not sure why this second EC2 instance is coming from.
I have terminated it many times but it is recreated. When I terminate it, this is the prompt i receive which advises me that it is created from EBS but there is no app in EBS
On an EBS-backed instance, the default action is for the root EBS volume to be deleted when the instance is terminated.
Storage on any local drives will be lost.
This name of the instance is.
ECS Instance - amazon-ecs-cli-setup-ecs-cricketscorer
Please help.
check if you have any Auto Scaling Groups that you do not recognize. It is most probably being created because of it. If not, change your account password and deactivate/delete your existing AccessKeys.
Building off of this question:
https://unix.stackexchange.com/questions/24355/is-there-a-way-to-get-the-public-dns-address-of-an-instance
I know how to get an ec2 instance's own public DNS address. What I need is a way for this instance to get the public DNS instance of a second ec2 instance.
The idea is that I will have ~50 instances running, one or two of which will be a spot instance that is constantly running. All of the other worker instances need to know the spot, or master, instance's public DNS name to connect to it within my application. How can I do this?
On another note, is there a way I can create a backup of my spot master instance? In case it fails, I would like to have another spot instance that immediately takes its place, but my worker ec2 instances would have to update their information about the spot instance's public dns address.
I think the only way to get the public DNS of your other instance is by using the command line interface or Web API provided by amazon.
The concrete command you need is ec2-describe-instances which provides data about public DNS settings for each instance.
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeInstances.html
Of course you can do the same through the Web API:
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html
Regarding the backup you can map the spot instance to EBS (which is preferrable) and then make snapshot backups. The snapshot backups are still triggered manually in the amazon console (or again through command line tools and Web API). Snapshots should be good for regular backups.
You can also use a service like http://www.skeddly.com/ to automate your EC2 snapshot backups.
If you want to backup the full AMI image of your spot instance, so you can re-create it from scratch at a later time, or create multiple instances from the same image etc. go to the management console and do the following:
Click on Instances
Select the instance you want to create an AMI from
Click on "Actions" and select "Create Image"
Set the Image name and other info and save
An alternative is to use S3. When a spot instance comes up it will read its own public address and write it to a bucket in S3. The other instances will look up the bucket the first time they need it and use this value. If the spot instance goes down, the workers will poll the bucket periodically until a new spot instance comes up and updates the bucket.
Make sure to set the bucket to only allow authenticated access so only your applications can modify it.
This approach has a security advantage, as the VMs do not need access to your EC2 credentials. They only need access to a specific S3 bucket.