I am trying to figure out the way to create a custom autoscale policy for autoscaling in AWS using boto. I saw that the scale out and scale in policies are defined using system dependent resources like CPU utilization.
But I want the scale out/in policy to be defined in a way that it calls a REST API and compare the response with some values. How can I make it possible?
I am using CircleCi as CI tool. I have 2 ec2 instances running as CircleCi- builders. During the weekends we generally don't require 2 instances. So I need to autoscale with min 1 and max 3 ec2 instances. If there are builds in the queue I wan to spin up a new ec2 and if the queue is empty for more that say 2 hours I want to scale down to just 1 ec2 VM as circleci builder.
I can get the builds in queue information using CircleCi REST API.
Autoscaling doesnt do that for you. The reverse works though, you can execute a policy.
What you could also do is send your custom metrics to cloudwatch, configure an alarm on that and add an autoscaling action to the alarm.
Related
I have an application running in ECS with asg as the capacity provider and using code deploy for rolling out new images.
the scaling policy configured in ECS service triggers autoscaling based on the cpu/mem metrics through cloud-watch alarms (target tracking policy).
When i trigger blue/green rollout in code deploy, there is a need for exactly the double the instances in the asg in order to accommodate the replacement version before routing the traffic.
however at this point ASG wont trigger autoscale and hence code deploy will not get enough instances ready to get replacement version started.
i think there are few ways to achieve this (although not tried it) but I am looking for a simpler out of the box solution where i need not to maintain a lot of configuration
I am using GCP and I have a specific problem to solve where I want to use the metrics from a RabbitMQ instance to control the autoscaling requirements of a Managed Instance Group. Do note that this RabbitMQ instance is outside this group and is used only to maintain the messages in the queue.
I want to scale up the number of instances in the group, when the number of current messages in the queue exceeds the number of available consumers. I had implemented the same in AWS using Amazon MQ integrated with RabbitMQ to autoscale for an ECS Cluster of Instances.
I have installed an OPS agent on the RabbitMQ Instance so that I can monitor the queue-based stats in a dashboard, but I am not sure how these metrics can be used to scale the instance as no specification on the MIG config page seems to point to the accessibility of these metrics.
My question is that, is it possible to scale the instances in an MIG through the metrics of an external instance like in my case? This question arises because the documentation on GCP seems to point out that autoscaling can be used only on metrics of the instances within the group.
If not, I would like to understand other ways I can implement the same by perhaps monitoring a consumer-based metric.
Custom metrics can be used for triggering the auto scaling feature. This document outlines clearly how to configure custom metrics for triggering auto scaling of MIG instances. The configuration involves three simple steps.
Create a custom metric for the Rabbitmq queue in cloud monitoring.
Create a service account and give sufficient permissions for performing
auto scaling actions.
Create a trigger using these custom metrics for scaling the managed
instance group.
We have individual teams that are launching ec2/rds/s3 instances and they are not choosing right instances or leaving instances running for long time, so how do I build a simple alert system that tells me what was launched?
I am trying to looking into building something on top of cloudwatch but I have to use SNS to do it. Is this the right approach?
they are not choosing right instances or leaving instances running for long time
You can simply introduce an IAM policy that will restrict the instance launch with specific Instance types
Long-running Servers: you can make use of CW metrics and take necessary action
For RDS, S3 and other resources, I would recommend AWS Config so that you can keep track of all of your resources.
We have hosted a docker container on AWS ECS with EC2 instances and would like to terminate/showdown these EC2 instances in the night & weekend for Staging/QA to save the cost.
Thanks in advance :)
The AWS Instance Scheduler is a simple AWS-provided solution that enables customers to easily configure custom start and stop schedules for their Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Relational Database Service (Amazon RDS) instances. The solution is easy to deploy and can help reduce operational costs for both development and production environments.
https://aws.amazon.com/solutions/implementations/instance-scheduler/
If you run the instances in an AutoScaling Group (ASG) , you could use scheduled policy to set a desired capacity of the ASG to zero for the off-peak times. A second policy would start it for work time.
Alternative would be setup a CloudWatch Event scheduled rule using cron with target of lambda function. The function would do same as the scaling policy. But because this is lambda function, you could also do some other things there. For example, do some pre-shutdown checks or post-shutdown cleanup.
This will work, because if your tasks run in service, ECS will automatically relaunch your tasks when the instances are back.
You could also manage the number of tasks using scheduling capability of Amazon ECS.
currently am on the t2.micro & i read that amazon allow an auto scaling option to allow the server to expand/shrink according to the traffic which is perfect.
so my question is:
what exactly should i do in-order to enable the auto scaling/resizing
of the server when needed or when the traffic start to spike ?
is there an option to allow changing the instance type automatically ?
auto scaling i believe means adding more instances and balance the load in between them, so does this mean i need to have a background about load balancing and all that jargon that comes with it or does amazon take care of that automatically ?
am totaly new to the whole server maintenance/provisioning land, so please try to explain as simple as possible. also the only reason i went with amazon because of the automation capabilities it offer but sadly their docs are very complex and many things could go wrong.
If you want to scale your instance and you don't mind about uptime, I can suggest this workaround.
TL;DR: set an alert on AWS CloudWatch to "ping" SNS when a specific alert is triggered (i.e. CPU, RAM > %) and setup a Skeddly action to automatic scale your instance when a SNS endpoint is pinged.
Details:
subscribe to Skeddly, a service to automate actions on AWS.
It's free if you don't use it a lot;
setup a "Change EC2 Instances" action and activate the SNS feature,
then copy the SNS endpoint link; screenshot
be sure to clearly define the instance(s) affected by the action!
go on AWS Simple Notification Service dashboard and create a new
"topic", then select it and choose "subscribe to topic" from the
Actions menu;
you can paste here the SNS endpoint provided by Skeddly, then wait
until the subscription is confirmed (it takes a while);
now move to AWS CloudWatch and setup an alert for any metric that
you find meaningful for your instance up/downscaling, i.e. CPU >= 90%
for 1 day;
for each alarm add a notification selecting the "topic" previously
defined on SNS.
You are done!
Auto scaling with EC2 assumes "horizontal" scaling, adding more instances to a auto scaling group
There is no well used, standard pattern for "vertical" scaling of increasing an individual instance size automatically.
In order to effectively understand and use auto scaling for your application yes, you "need to have a background about load balancing and all that jargon that comes with it". See http://docs.aws.amazon.com/autoscaling/latest/userguide/GettingStartedTutorial.html
I'm assuming you are using the AWS management console. These operations are also possible using the Command Line Interface or AWS CloudFormation.
To resize an instance, you have to stop it then go to Actions > Instance Settings > Change Instance Type
As you can see, this operation is not automatic. In AWS you don't autoscale an instance but an autoscaling group which is a group of instances. So according to your memory/cpu usage, you can automatically start new instances (but not increase the size of the current ones)
To create an autoscaling group, go to Auto Scaling Groups in the EC2 menu:
To create an autoscaling group, you will need to create a Launch Configuration first which describes the properties of the instances you want to automatically scale. Then you will be able to define your scaling policies based on your Cloudwatch alarms (CPU usage, instance status...):