How to autoscale Servers in ECS? - amazon-web-services

Question

I recently started using ECS. I was able to deploy a container image in ECR and create task definition for my container with CPU/Memory limits. My use case is that each container will be a long running app (no webserver, no port mapping needed). The containers will be spawned on demand 1 at a time and deleted on demand 1 at a time.
I am able to create a cluster with N server instances. But I'd like to be able for the server instances to automatically scale up/down. For example if there isn't enough CPU/Memory in the cluster, I'd like a new instance to be created.
And if there is an instance with no containers running in it, I'd like that specific instance to be scaled down / deleted. This is to avoid auto scale down termination of a server instance that has running tasks in it.
What steps are needed to be able to achieve this?

Answer 1

Considering that you already have an ECS Cluster created, AWS provides instructions on Scaling cluster instances with CloudWatch Alarms.
Assuming that you want to scale the cluster based on the memory reservation, at a high level, you would need to do the following:
Create an Launch Configuration for your Auto Scaling Group. This
Create an Auto Scaling Group, so that the size of the cluster can be scaled up and down.
Create a CloudWatch Alarm to scale the cluster up if the memory reservation is over 70%
Create a CloudWatch Alarm to scale the cluster down if the memory reservation is under 30%
Because it's more of my specialty I wrote up an example CloudFormation template that should get you started for most of this:
Parameters:
MinInstances:
Type: Number
MaxInstances:
Type: Number
InstanceType:
Type: String
AllowedValues:
- t2.nano
- t2.micro
- t2.small
- t2.medium
- t2.large
VpcSubnetIds:
Type: String
Mappings:
EcsInstanceAmis:
us-east-2:
Ami: ami-1c002379
us-east-1:
Ami: ami-9eb4b1e5
us-west-2:
Ami: ami-1d668865
us-west-1:
Ami: ami-4a2c192a
eu-west-2:
Ami: ami-cb1101af
eu-west-1:
Ami: ami-8fcc32f6
eu-central-1:
Ami: ami-0460cb6b
ap-northeast-1:
Ami: ami-b743bed1
ap-southeast-2:
Ami: ami-c1a6bda2
ap-southeast-1:
Ami: ami-9d1f7efe
ca-central-1:
Ami: ami-b677c9d2
Resources:
Cluster:
Type: AWS::ECS::Cluster
Role:
Type: AWS::IAM::Role
Properties:
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AmazonEC2ContainerServiceforEC2Role
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Action:
- sts:AssumeRole
Principal:
Service:
- ec2.amazonaws.com
InstanceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles:
- !Ref Role
LaunchConfiguration:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
ImageId: !FindInMap [EcsInstanceAmis, !Ref "AWS::Region", Ami]
InstanceType: !Ref InstanceType
IamInstanceProfile: !Ref InstanceProfile
UserData:
Fn::Base64: !Sub |
#!/bin/bash
echo ECS_CLUSTER=${Cluster} >> /etc/ecs/ecs.config
AutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
MinSize: !Ref MinInstances
MaxSize: !Ref MaxInstances
LaunchConfigurationName: !Ref LaunchConfiguration
HealthCheckGracePeriod: 300
HealthCheckType: EC2
VPCZoneIdentifier: !Split [",", !Ref VpcSubnetIds]
ScaleUpPolicy:
Type: AWS::AutoScaling::ScalingPolicy
Properties:
AdjustmentType: ChangeInCapacity
AutoScalingGroupName: !Ref AutoScalingGroup
Cooldown: '1'
ScalingAdjustment: '1'
MemoryReservationAlarmHigh:
Type: AWS::CloudWatch::Alarm
Properties:
EvaluationPeriods: '2'
Statistic: Average
Threshold: '70'
AlarmDescription: Alarm if Cluster Memory Reservation is to high
Period: '60'
AlarmActions:
- Ref: ScaleUpPolicy
Namespace: AWS/ECS
Dimensions:
- Name: ClusterName
Value: !Ref Cluster
ComparisonOperator: GreaterThanThreshold
MetricName: MemoryReservation
ScaleDownPolicy:
Type: AWS::AutoScaling::ScalingPolicy
Properties:
AdjustmentType: ChangeInCapacity
AutoScalingGroupName: !Ref AutoScalingGroup
Cooldown: '1'
ScalingAdjustment: '-1'
MemoryReservationAlarmLow:
Type: AWS::CloudWatch::Alarm
Properties:
EvaluationPeriods: '2'
Statistic: Average
Threshold: '30'
AlarmDescription: Alarm if Cluster Memory Reservation is to Low
Period: '60'
AlarmActions:
- Ref: ScaleDownPolicy
Namespace: AWS/ECS
Dimensions:
- Name: ClusterName
Value: !Ref Cluster
ComparisonOperator: LessThanThreshold
MetricName: MemoryReservation
This creates an ECS Cluster, a Launch Configuration, An AutoScaling Group, As well as the Alarms based on the ECS Memory Reservation.
Now we can get to the interesting discussions.
Why can't we scale up based on the CPU Utilization And Memory Reservation?
The short answer is you totally can But you're likely to pay a lot for it. EC2 has a known property that when you create an instance, you pay for a minimum of 1 hour, because partial instance hours are charged as full hours. Why that's relevant is, imagine you have multiple alarms. Say you have a bunch of services that are currently running idle, and you fill the cluster. Either the CPU Alarm scales down the cluster, or the Memory Alarm scales up the cluster. One of these will likely scale the cluster to the point that it's alarm is no longer triggered. After the cooldown, period, the other alarm will undo it's last action, After the next cooldown, the action will likely be redone. Thus instances are created then destroyed repeatedly on every other cooldown.
After giving a bunch of thought to this, the strategy that I came up with was to use Application Autoscaling for ECS Services based on CPU Utilization, and Memory Reservation based on the cluster. So if one service is running hot, an extra task will be added to share the load. This will slowly fill the cluster memory reservation capacity. When the memory gets full, the cluster scales up. When a service is cooling down, the services will start shutting down tasks. As the memory reservation on the cluster drops, the cluster will be scaled down.
The thresholds for the CloudWatch Alarms might need to be experimented with, based on your task definitions. The reason for this is that if you put the scale up threshold too high, it may not scale up as the memory gets consumed, and then when autoscaling goes to place another task, it will find that there isn't enough memory available on any instance in the cluster, and therefore be unable to place another task.

Answer 2

As part of this year's re:Invent conference, AWS announced cluster auto scaling for Amazon ECS. Clusters configured with auto scaling can now add more capacity when needed and remove capacity that is not necessary. You can find more information about this in the documentation.
However, depending on what you're trying to run, AWS Fargate could be a better option. Fargate allows you to run containers without provisioning and managing the underlying infrastructure; i.e., you don't have to deal with any EC2 instances. With Fargate, you can make an API call to run your container, the container can run, and then there's nothing to clean up once the container stops running. Fargate is billed per-second (with a 1-minute minimum) and is priced based on the amount of CPU and memory allocated (see here for details).