I'm trying to write a script to stop several instances in our test environment on Friday and have them start back on Monday, to save little cost.
Is there a way to stop instances by IP addresses (and not by instance ID), or some other way I don't know about? (The reason being that instance ID's may change if an instance had to be deleted and recreated.)
This is a zero code solution:
Put your instances into autoscale groups and add a shutdown and startup schedule on the autoscale group. This can be done in the AWS console.
This can also be automated using the AWS CLI.
Use EC2 Tags to give your instances key/value tag pairs, then write a script using Boto which searches for instances with the right tags, and then terminates them.
You could also use Boto to list instances matching the specific IP address, and terminate them that way.
But... IP addresses are dynamically assigned (unless you are using Elastic IPs). So why not make a note of the instance IDs when launching the instances, instead of the IP address?
Related
I would like to try to setup AWS Launch Template, or just Spot request (persistance) and I need automatically attach my specific volume.
The main idea - spot instance will be process data and store it in a separate volume. When Spot will die, another Spot should be requested automatically (which will be built from an image with predefined software) and data should continue processing automatically (and again, storing in my second volume).
But, I can`t setup it in AWS console, so, looks like it is not possible. Am I wrong? Is it possible in some another way?
The same according IP address - I would like to have the same IP address for any of "versions" of Spot (after recreating for example)
It is possible to attach and detach Elastic IP using AWS CLI to achieve what you want.
However there are other possible workarounds in case you do not want to script AWS CLI :
Using Route53 you can define an A record with TTL of 60 seconds if you can accept few minutes of transition. That way you get to use a domain to access your underlying instance instead.
Setup a ALB and forward the request to the EC2 fleet
I've an ECS cluster with running one task for my backend instance. I would like to be able to stop/start the EC2 instance whenever I want. Is it possible?? I was trying to stop instance directly but it terminates after few second when stopped and after that new instance is created automatically. I tried to change the Auto Scale Group to match desired=min=0 capacity but when I do that the instance gets auto terminated. I just want to turn off the Ec2 instance when its not needed to be used but at the same time I want data to persist betweet turning on and off. I am fighting with this for a few days now and wasn't able to achieve my goals.
Also how to link EBS volume with VOLUME /root/.local/share/XYZ from docker file image to persist the data from the XYZ folder
I would suggest you to do modifications in autoscaling group, when you want to turn off instance put 0 in auto scaling and when you want to turn on change value in autoscaling,
... you can do that with aws cli , and you can shcdule the period also by putting aws cli command in cron job
I would suggest using EFS. Here is an article from AWS on how to persist data from ECS containers using EFS.
Using Amazon EFS to Persist Data from Amazon ECS Containers
Start/Stop instances and auto-scale don't really fit together.
Auto-scale is specifically designed to solve scalein/scaleout.
One way to address this could be using customized termination policy (but I never tried this in ECS setup).
One note though, if your customized termination policy never terminates the instances and you continue adding instances to keep always, you might get good amount EC2 bill.
Is it possible to do AutoScaling with Static IPs in AWS ? The newly created instances should either have a pre-defined IP or pick from a pool of pre-defined IPs.
We are trying to setup ZooKeeper in production, with 5 zooKeeper instances. Each one should have a static-IP which are to hard-coded in the Kafka's AMI/Databag that we use. It should also support AutoScaling, so that if one of the zooKeeper node goes down, a new one is spawned with the same IP or from a pool of IPs. For this we have decided to go with 1 zoo-keeper instance per AutoScaling group, but the problem is with the IP.
If this is the wrong way, please suggest the right way. Thanks in advance !
One method would be to maintain a user data script on each instance, and have each instance assign itself an elastic IPs from a set of EIPs assigned for this purpose. This user data script would be referenced in the ASGs Launch Configuration, and would run on launch.
Say the user script is called "/scripts/assignEIP.sh", using the AWS CLI you would have it consult the pool to see which ones are available and which ones are not (already in use). Then it would assign itself one of the available EIPS.
For ease of IP management, you could keep the pool of IPs in a simple text properties file on S3, and have the instance download and consult that list when the instance starts.
Keep in mind that each instance will need an to be assigned IAM instance profile that will allow each instance to consult and assign EIPs to itself.
I'm trying to determine how to remove an instance from several applications (freeIPA, Chef, service discovery) from within an AWS autoscaling group but I'm finding that there's no reliable way to determine if an instance is simply stopping (sometimes our admins will take an instance out of the ASG for analysis) vs terminating. If the instance is stopped then I would like to retain the ability to have it stay connected to our LDAP and other systems. Anyone know a good way to do this?
Is the instance EBS backed or using instance store?
if instance store, you cannot really stop it (only terminate it)
Have you looked at the EC2 API (via aws-sdk maybe)? (looks like describe-instances and looking at reservations should do the trick here)
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html
I determined that the best way for me to do this was to use the ASG alarms (specifically the EC2_TERMINATE alarm). This effectively allows me to take no action if an instance is stopping and fire off a script if it is determined that the instance is terminating.
Building off of this question:
https://unix.stackexchange.com/questions/24355/is-there-a-way-to-get-the-public-dns-address-of-an-instance
I know how to get an ec2 instance's own public DNS address. What I need is a way for this instance to get the public DNS instance of a second ec2 instance.
The idea is that I will have ~50 instances running, one or two of which will be a spot instance that is constantly running. All of the other worker instances need to know the spot, or master, instance's public DNS name to connect to it within my application. How can I do this?
On another note, is there a way I can create a backup of my spot master instance? In case it fails, I would like to have another spot instance that immediately takes its place, but my worker ec2 instances would have to update their information about the spot instance's public dns address.
I think the only way to get the public DNS of your other instance is by using the command line interface or Web API provided by amazon.
The concrete command you need is ec2-describe-instances which provides data about public DNS settings for each instance.
http://docs.aws.amazon.com/AWSEC2/latest/CommandLineReference/ApiReference-cmd-DescribeInstances.html
Of course you can do the same through the Web API:
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html
Regarding the backup you can map the spot instance to EBS (which is preferrable) and then make snapshot backups. The snapshot backups are still triggered manually in the amazon console (or again through command line tools and Web API). Snapshots should be good for regular backups.
You can also use a service like http://www.skeddly.com/ to automate your EC2 snapshot backups.
If you want to backup the full AMI image of your spot instance, so you can re-create it from scratch at a later time, or create multiple instances from the same image etc. go to the management console and do the following:
Click on Instances
Select the instance you want to create an AMI from
Click on "Actions" and select "Create Image"
Set the Image name and other info and save
An alternative is to use S3. When a spot instance comes up it will read its own public address and write it to a bucket in S3. The other instances will look up the bucket the first time they need it and use this value. If the spot instance goes down, the workers will poll the bucket periodically until a new spot instance comes up and updates the bucket.
Make sure to set the bucket to only allow authenticated access so only your applications can modify it.
This approach has a security advantage, as the VMs do not need access to your EC2 credentials. They only need access to a specific S3 bucket.