I've configured AWS instance scheduler, everything is working as expected.
The issue I'm having is each instance has a autoscaling group in my dev environment and i'm unable to shutdown instances without them beign terminated by autoscale group when it does a health check and notices its down.
Has anyone figured out an automated solution to this without me having to manually suspend ASG? Since the whole purpose of this is to stop the instances after hours I'm unable to intervene to suspend/resume ASG.
Thanks in advance!
"Auto Scaling" and "AWS Instance Scheduler" don't really fit together nicely. Do you really need ELB for Dev environments? I feel this is overkill.
Anyway, if you still want to use ELB + AutoScaling and would like to shutdown the boxes during off hours, you can set "AutoScaling" to ZERO for the hours you want using Scheduled Scaling approach.
Related
I'm relatively new to ECS, for the most part its been fine, but lately I'm facing an issue that I can seem to find an intuitive solution for.
I'm running a ECS cluster with an EC2 capacity provider. EC2 has an AutoScalingGroup with a min_capacity: 1 & max_capacity: 5.
Each ECS service task has auto scaling enabled based upon CPU/Memory utilisation.
The issue I'm seeing is that when new Tasks and being deployed as part of our CI/CD, ECS returns unable to place a task because no container instance met all of its requirementsunable to place a task because no container instance met all of its requirements.
I'm wondering how I get ECS to trigger a scale out event for the ASG when this happens? Do I need a particular scaling policy for the ASG? (I feel its related to this)
I attempted to setup a EventBridge/Cloudwatch alarm to trigger a scale out event whenever this happens but seems hacky. It worked, not ideally but worked. Surely there is a nicer/simpler way of doing this?
Any advice or points from experience would be greatly appreciated!
(PS - let me know if you need any more information/screenshots/code examples etc.)
I have created ec2 instance and it's part of the auto scaling group and i want to stop this instance so i can save money while this server being not used.Whenever i want to use this server that time i can start the instance.
AutoScaling Group is able to take care of this process for you.
You can change the size of the group manually, or you could schedule custom configurations to scale-up or scale-down as per your needs.
This is all supported through AWS APIs, so you can do this through AWS Console or automating the configuration.
Instances in ASG can't be stopped. If you want to stop one instance you have to detach it from ASG. When you are ready to start it, you can attach it again.
But really, you should reconsider how you use ASG. Detaching and re-attaching instances from ASG is meant for mostly troubleshooting purposes, not something you do regulatory just to save cost. Its better to set minimum and desired capacity of ASG to 0 if you don't want any instance in it.
Ok, I have a strange situation. Can't find anything quite like it online.
Within a platform that I'm helping run, we have a couple of services that really can only run on a single node. Yes, our developers are working on fixing this, but in the meantime... We are currently using HA to handle failover to a hot standby, but we would like to try to use AWS Auto Scaling Groups, for consistency in our architecture.
We've tried setting the min/max/des to 1/1/1, with some success. However, we've had an issue arise where it takes about 3 minutes for the ASG to spin down a failed EC2, and spin up a replacement. During this time, havoc ensues within the platform.
My question is this, is there a way to make the ASG start the new EC2 instance, before stopping the unhealthy one?
From my current knowledge, the answer is no.
The ASG schedule almost immediately the replacement instance. Unfortunately, the instance needs some time to warm up and pass the health check.
https://docs.aws.amazon.com/autoscaling/ec2/userguide/healthcheck.html
I'm having an issue with AWS boxes (EC2) where I terminate the box and it re-spawns. To give context, there is no autoscaling group. Anywhere I can search for some config that might be triggering the launch?
I would make sure you don't have a persistent spot request active in your account, and also check to see if you perhaps installed the AWS Instance scheduler - either or both of those could be starting instances on your behalf - (double check the autoscaling group, that is the most obvious reason though)
If you terminate a running Spot Instance that was launched by a
persistent Spot request, the Spot request returns to the open state so
that a new Spot Instance can be launched. To cancel a persistent Spot
request and terminate its Spot Instances, you must cancel the Spot
request first and then terminate the Spot Instances. Otherwise, the
persistent Spot request can launch a new instance.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html#terminating-a-spot-instance
https://aws.amazon.com/solutions/instance-scheduler/
So I found out the culprit, maybe it can help more people. Apparently, there is a service from AWS called OpsWorks that automates things like Cheff of Puppet, which my company had configured some time ago. That would be checking for instances running and triggering re-provisioning when it didn't see the instance running. OpsWorks is here
I've an ECS cluster with running one task for my backend instance. I would like to be able to stop/start the EC2 instance whenever I want. Is it possible?? I was trying to stop instance directly but it terminates after few second when stopped and after that new instance is created automatically. I tried to change the Auto Scale Group to match desired=min=0 capacity but when I do that the instance gets auto terminated. I just want to turn off the Ec2 instance when its not needed to be used but at the same time I want data to persist betweet turning on and off. I am fighting with this for a few days now and wasn't able to achieve my goals.
Also how to link EBS volume with VOLUME /root/.local/share/XYZ from docker file image to persist the data from the XYZ folder
I would suggest you to do modifications in autoscaling group, when you want to turn off instance put 0 in auto scaling and when you want to turn on change value in autoscaling,
... you can do that with aws cli , and you can shcdule the period also by putting aws cli command in cron job
I would suggest using EFS. Here is an article from AWS on how to persist data from ECS containers using EFS.
Using Amazon EFS to Persist Data from Amazon ECS Containers
Start/Stop instances and auto-scale don't really fit together.
Auto-scale is specifically designed to solve scalein/scaleout.
One way to address this could be using customized termination policy (but I never tried this in ECS setup).
One note though, if your customized termination policy never terminates the instances and you continue adding instances to keep always, you might get good amount EC2 bill.