We can scale docker containers using service auto scaling feature in AWS with the help of Cloud-watch Alarms -
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/service_autoscaling_tutorial.html
Is there any other options are available to scale the docker container if the CPU/Memory utilization reaches 80% without using Cloud-watch?
Note: We can achieve the same in Kubernetes using horizontal pod auto scaling. I want to achieve the same in AWS without Cloudwatch support.
You can use AWS ECS to scale docker containers. It provides a native orchestration platform to AWS as well as support for Kubernetes.
If you decides to use ECS native container orchestration, it involves a learning curve where you need to understand ECS specific terms such as Tasks, Services and etc. The same goes with Kubernetes where you need to understand Pods, Services and etc.
When using ECS, it manages the underlying complexities such as placing containers across multiple EC2s which powers the container cluster, supporting Load Balancer intergration for container level load balancing, Supporting fault tolerance by replacing unhealthy containers and etc.
It is also possible to use AWS Fargate which also comes with ECS, where the underlying nodes in the cluster is entirely managed by AWS without even exposing the number of EC2s powering the cluster. It is more like you can scale up and down to large number of containers without worrying about allocating EC2s to the cluster. However, it is expensive in comparison which limits its usage for more specific workloads that requires higher levels of scalability with least predictability which justifies the pricing.
Related
I'm not seeing the point in using a capacity provider to scale the ECS cluster if I have automatic scaling at the ECS service level:
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-auto-scaling.html
Am I missing something? Why would I use a capacity provider to scale the auto scaling group if I already can scale it at the service level?
Your auto scaling group scaling works on a service level only. An ECS cluster can have many services running. Therefore, capacity provider runs at cluster level and can scale your container instances based on all the services in the cluster, not only one service.
Capacity Providers is the compute interface that links your Amazon ECS cluster with your ASG. With Capacity Providers, you can define flexible rules for how containerized workloads run on different types of compute capacity, and manage the scaling of the capacity. Capacity Providers improve the availability, scalability, and cost of running tasks and services on ECS.
In other words, with Capacity provider, you can apply flexible rule to scales your app that saves the cost.
Example - By simply setting the base and weights in capacity provider strategy that to set the default strategy to be a mix of Fargate and Fargate Spot to ensure that if Spot tasks are terminated, we still have our minimum, desired amount of tasks running on Fargate. In this way we can take advantage of the cost savings of Fargate Spot in our every day workloads.
(for more information, check out the official AWS documentation.)
this is my first time using amazon ecs service.
I have searched online for awhile to understand auto-scaling with ecs services.
I found there are two options to auto-scale my application. But, There are some I don't understand.
First is Service auto scaling which track the cpu/memory metric from cloudWatch and increase the task number accordingly.
Second is cluster auto scaling which needs to create auto scaling resource, create capacity provider and so on. But, in Tutorial: Using cluster auto scaling, it can run the task definition without service. But it also seems increasing the task number in the end.
So what's the different and 'pros and cons' between them?
I will try to explain briefly.
A Task is a container which is running our code(from a docker image).
As Service is making sure that given desired no of tasks are maintained.
We will be running these services in ECS backed by EC2 or Fargate. Ec2 is machines managed by us. Fargate is machines managed by AWS.
Scaling:
Ultimately, We will be scaling the tasks just by setting desired no of tasks between min and max tasks, based on CPU or any other metric of individual task. This is called service auto scaling.
Fargate: Since AWS will manage necessary VMs behind the scenes, we can set any no of desired tasks we want and seamlessly scale without worrying about any infrastructure.
EC2: We can't seamlessly scale services because we need to add/remove EC2 instances behind the scenes too. We need to auto scale these instances also based on cpu or any other metrics of the Ec2 machines, which is called Cluster scaling.
Amazon EKS managed node groups automate the provisioning and lifecycle management of nodes (Amazon EC2 instances) for Amazon EKS Kubernetes clusters.
AWS Fargate is a technology that provides on-demand, right-sized compute capacity for containers. With AWS Fargate, you no longer have to provision, configure, or scale groups of virtual machines to run containers. This removes the need to choose server types, decide when to scale your node groups, or optimize cluster packing.
So, Is AWS Fargate better than Amazon EKS managed node groups? When should I choose managed node groups?
We chose to go with AWS Managed Node groups for the following reasons:
Daemonsets are not supported in EKS Fargate, so observability tools like Splunk and Datadog have to run in sidecar containers in each pod instead of a daemonset per node
In EKS Fargate each pod is run in its own VM and container images are not cached on nodes, making the startup times for pods 1-2 minutes long
All replies are on point. There isn't really "better" here. It's a trade-off. I am part of the container team at AWS and we recently wrote about the potential advantages of using Fargate over EC2. Faster pod start time, images caching, large pods configurations, special hw requirements e.g. GPUs) are all good reasons for needing to use EC2. We are working hard to make Fargate a better place to be though by filling some of the gaps so you could appreciate only the advantages.
There is no better than other. Your requirements (and skills) makes a product better than another!
The real difference in Fargate is that it's serverless, so you don't need for example to care about the EC2 instances right-sizing, you won't pay the idle time.
To go straight to the point: unless you are a K8S expert I would suggest Fargate.
Can AWS Fargate scale on demand like Lambda?
Suppose we set 10 container instances in scaling settings.
But what if certain times 10 instances is not enough, how do we scale further (11,12...)?
Yes, Fargate can scale on-demand.
You can actually use target tracking scaling policies and configure scaling limits (i.e. instance/task count). You will also have to configure Service AutoDiscovery in order to distribute load on the new instances.
More info here
Fargate supports auto-scaling, you can enable this within your configuration.
You will need to set it to scale against a specific metric (such as average CPU or average network in).
By using auto scaling you can set scale-out (add more containers) and scale-in (remove containers) rules that will affect how many containers you need running at the same time.
There is more information regarding how to configure this available here.
Since you can enable autoscaling of containers through DC/OS, when running this on an EC2 cluster, is it still necessary to, or redundant to run your cluster in an AutoScaling cluster?
There are two (orthogonal) concepts here at play and unfortunately the term 'auto-scale' is ambiguous here:
Certain IaaS platforms (incl. AWS) support dynamically adding VMs to a cluster.
The other is the capability of a container orchestrator to scale the number of copies of a service—in case of Marathon this is called instances or replicas in the context of Kubernetes—as long as there are sufficient resources (CPU, RAM, etc.) available in the cluster,
In the simplest case you'd auto-scale the services up to the point where the overall cluster utilization is high (>60%? >70%? >80%?) and the use the IaaS-level auto-scaling functionality to add further nodes. Turns out scaling back is the trickier thing.
So, complementary rather than redundant.