App Deployment Container Size vs Quantity - amazon-web-services

I am learning how to deploy a meteor app on galaxy and I am really confused by all this container stuff.
I am trying to understand when it would be better to scale an app by increasing container size rather than by adding more containers.
If I had lightweight chat room website for example. Why would I ever need to upgrade the container size if I can just add more small containers. In the end isn't the sum of processing power what matters?
2 x 0.5 containers = 1 x 1 container
The cost of doing it either way is the same.
Also, if a user modifies the database while using the app in one container, won't the other instances of the app running on other containers take a while to notice the change? If users on different containers were chatting together it would be a problem wouldn't it? How would you avoid it?
The only way I can make sense of this is:
Either lack of CPU and RAM, or capacity to handle parallel requests are going to create a need to scale.
If the app receives too much traffic you get more containers.
If the app uses too much CPU and RAM you get a bigger container.
But how can app ever get too big to fit in one container? Won't the CPU and RAM used by the app be related to how many users are using that instance of the app. Couldn't you just solve the problem by adding more containers and spreading out the users and decreasing CPU and RAM usage this way.
And why would you need to get more containers to handle more requests. Won't a bigger container also handle more requests?

The question you are asking is too broad to answer. In your case both strategies increasing container size (or vertical scaling) and adding more container (or horizontal scaling) will work if implemented effectively.
But preferring horizontal scaling is the best option. When you launch a cluster of containers they run behind AWS Elastic Loadbalancer and if you enable sticky sessions there will be no any problem in chat rooms.
Read this
http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-sticky-sessions.html
Also this is quiet good to read.
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/cloudwatch_alarm_autoscaling.html
https://aws.amazon.com/blogs/compute/powering-your-amazon-ecs-clusters-with-spot-fleet/
Then the question of database, I assume you will be using a parent database for your app so all the containers will be reading from same DB so do not worry about the changes applied from one container and seeing those changes applied from other container if proper DB optimization is in place there will be no any issue.

Related

large number of docker containers and aws resources

My application allow users to create multiple nodered instances using docker containers , each docker container handle one instance , the number of containers could reach 65000 container , so:
how can i configure host resources to handle that number of containers
if my host memory is 16 gb how to check if it can handle all those instances
if No should i increase memroy size or should i create another instance (aws instance) and use it
You don't, 16GB / 65000 = 0.25mb per container, there is no way a NodeJS process will start, run and do anything useful. So you are never going to run 65000 containers on a single machine.
As I said in the comment on your previous question, you need to spend some time running tests and determine how much memory/CPU your typical use case will need and you then set resource limits on the container when started to limit it to those values. How to set resource limits are in the Docker docs
If your work load is variable then you may need to have different sizes of container depending on the workload.
You then do the basic maths that is CPU in the box/ %CPU per container and memory in the box / memory per container and which ever is the smaller number is the max number of containers you can run. (You will also need to include some overhead for monitoring/management and other house keeping tasks)
It's up to you to decide what approach to sizing/choosing cloud infrastructure to support that work load and what will be economically viable.
(or you could outsource this to people that have already done a bunch of this e.g. FlowForge inc [full disclosure, I am lead developer at FlowForge])

Recommended way to run a web server on demand, with auto-shutdown (on AWS)

I am trying to find the best way to architect a low cost solution to provide an on-demand web server for a certain amount of time.
The context is as follows: I have some large amount of data sitting on S3. From time to time, users will want to consult that data. I've written a Flask app that can display the data in a nice way for them. Beign poorly written, it really only accepts a single user session at the time. Currently therefore they have to download the Flask app and run it on their own machine.
I would like to find a way for users to request a cloud-based web server that would run the Flask app (through a docker container for example) on-demand, and give them access to it quickly, without having to do much if anything on their own machine.
Every user wanting to view the data would have their own web server created on demand (to avoid multiple users sharing the same web server, which wouldn't work with my Flask app)
Critically, and in order to avoid cost, the web server would terminate itself automatically after some (configurable) idle time (possibly with the Flask app informing the user that it's about to shut down, so that they can "renew" the lease).
Initially I thought that maybe AWS Fargate would be good: it can run docker instances, is quite configurable in terms of CPU/disk it can get (my Flask app is resource-hungry), and at least on paper could be used in a way that there is zero cost when users are not consulting the data (bar S3 costs naturally). But it's when it comes to the detail that I'm not sure...
How to ensure that every new user gets their own Fargate instance?
How to shut-down the instance automatically after idle time?
Is Fargate quick enough in terms of boot time?
The closest I can think is AWS App Runner. It's built on top of Fargate and it provides an intelligent scale out mechanism (probably you are not interested in this) as well as a scale to (almost) 0 capability. The way it works is that when the endpoint is solicited and it's doing work you pay for the entire fargate task (cpu/memory) you have selected in the configuration. If the endpoint is doing nothing you only pay for the memory (note the memory cost is roughly 20% of the entire cost so it's not scale to 0 but "quasi"). Checkout the pricing examples at the bottom of this page.
Please note you can further optimize costs by pausing/starting the endpoint (when it's paused you pay nothing) but in that case you need to create the logic that pauses/restarts it.
Another option you may want to explore is using Lambda this way (which would allow you to use the same container image and benefit from the intrinsic scale to 0 of Lambda). But given your comment "Lambda doesn’t have enough power, and the timeout is not flexible" may be a show stopper.

AWS K8S web application architecture choice

I am trying to build a web application with Kubernetes and Amazon Web Service.
I believe there are many different approaches but I would like to ask for your opinions!
To make it simple, the app would be a simple web page that display information. User logs in, and based on filters, get specific information.
My reflection is on the k8s inside architecture:
Can I put my whole application as a Pod? That way, cluster and node scalability would make the app available for each user by allocating each of them 1 pod. Is that good practice?
By following that logic, every different elements of my app would be a container. For instance, in a simple way, 1 container that contains the front of the app, 1 container that has the data access/management, 1 container for backend/auth etc
So 1 user would "consume" 1 pod which containers' are discussing together to give the required data from the user. k8s would create a pod for every users, scaling up/down the nodes number etc...
But then, except for the data itself, everything would be dockerized and stored on ECR (Elastic Container Registry) right? And so no need for any S3/EBS/EFS in my opinion.
I am quite new at AWS and k8s, so please feel free to give honest opinions :) Feedback, good or bad, is always good to take.
Thanks in advance!
I'd recommend a layout where:
Any container can perform its unit of work for any user
One container per pod
Each component of your application has its own (stateless) deployment
It probably won't work well to try to put your entire application into a single multi-container pod. This means the entire application would need to fit on a single node; some applications are larger than that, and even if it fits, it can lead to trouble with scheduling. It also means that, if you update the image for any single container, you need to delete and recreate all of the containers, which could be more disruptive than you want.
Trying to create a pod per user also will present some practical problems. You need to figure out how to route inbound requests to a particular user's pod, and keep requests within that user's set of containers; Kubernetes doesn't have any sort of native support for this. A per-user pod will also keep using resources even if it's overnight or the weekend for that user and they're not using the application. You would also need something with access to the Kubernetes API to create and destroy resources as new users joined your platform.
In an AWS-specific context you might consider using RDS (hosted PostgreSQL/MySQL) or S3 (object storage) for data storage (and again one database or S3 bucket shared across all customers). ECR is useful, but as a place to store your Docker images; that is, your built code, but not any of the persisted data or running containers.

Use of redis cluster vs standalone redis

I have a question about when it makes sense to use a Redis cluster versus standalone Redis.
Suppose one has a real-time gaming application that will allow multiple instances of the game and wish to implement
real time leaderboard for each instance. (Games are created by communities of users).
Suppose at any time we have say 100 simultaneous matches running.
Based on the use cases outlined here :
https://d0.awsstatic.com/whitepapers/performance-at-scale-with-amazon-elasticache.pdf
https://redislabs.com/solutions/use-cases/leaderboards/
https://aws.amazon.com/blogs/database/building-a-real-time-gaming-leaderboard-with-amazon-elasticache-for-redis/
We can implement each leaderboard using a Sorted Set dataset in memory.
Now I would like to implement some sort of persistence where leaderboard state is saved at the end of each
game as a snapshot. Thus each of these independent Sorted Sets are saved as a snapshot file.
I have a question about design choices:
Would a redis cluster make sense for this scenario ? Or would it make more sense to have standalone redis instances and create a new database for each game ?
As far as I know there is only a single database 0 for a single redis cluster.(https://redis.io/topics/cluster-spec)
In that case, how would one be able to snapshot datasets for each leaderboard at different times work ?
https://redis.io/topics/cluster-spec
From what I can see using a Redis cluster only makes sense for large-scale monolithic applications and may not be the best approach for the scenario described above. Is that the case ?
Or if one goes with AWS Elasticache for Redis Cluster mode can I configure snapshotting for individual datasets ?
You are correct, clustering is a way of scaling out to handle really high request loads and store tons of data.
It really doesn't quite sound like you need to bother with a cluster.
I'd quite be very surprised if a standalone Redis setup would be your bottleneck before having several tens of thousands of simultaneous players.
If you are unsure, you can probably mock some simulated load and see what it can handle. My guess is that you are better off focusing on other complexities of your game until you start reaching quite serious usage. Which is a good problem to have. :)
You might however want to consider having one or two replica instances, which is a different thing.
Secondly, regardless of cluster or not, why do you want to use snap-shots (SAVE or BGSAVE) to persist your scoreboard?
If you want to have individual snapshots per game, and its only a few keys per game, why don't you just have your application read and persist those keys when needed to a traditional db? You can for example use MULTI, DUMP and RESTORE to achieve something that is very similar to snapshotting, but on the specific keys you want.
It doesn't sound like multiple databases is warranted for this.
Multiple databases on clustered Redis is only supported in the Enterprise version, so not on ElastiCache. But the above mentioned approach should work just fine.

Separate server for Memcache/Redis?

I am using Django for my project and I ll be hosting it on Linode or any other hosting service. Plus if I want to use memcache will I require a new Linode for it? Means just one server will be ok or I ll have to host my site on 2 servers, one for memcache and one for django? And is it the same for Redis? Also will I require a separate server for Mysql?
I don't think you understand that nobody is a fortune telling wizard. Nobody knows how many requests you will receive per second, nor how cpu/memory intensive each request will be. Nobody knows how optimized your code is. Nobody knows if your application is read heavy or write heavy. Your use case is your own, and your probably the only one who estimate it.
My only actual advice to you is to try to estimate your server data and sever load and benchmark your setup on one machine. If you are unsatisfied with the performance then scale up. You can either scale up vertically, by increasing the size of your linode, or scale horizontally by adding more linode instances. In the latter case, you will most likely put your DB on a machine of it's own and have multiple django instances fed by a load balancer. These Django instances could each share the same memcache on a machine, or they can each have their own memcaches on their own machine. Which one is better? I can't tell you. It again depends on your use case.
If I were you, I would set it all up on one linode instance. I would create test data that I assume would be close to real world. Then I would try to test my response times with an estimated number of requests per second. I would measure response times, cache hits, and memory usage. I would then decide based on that if my use case is satisfied with this level of performance or not because I'm really the only one who would know what is satisfactory performance. Additionally, adding more linode resources is not necessarily where I would first try and improve performance.
Some great tips on optimizing and benchmarking can be found here:
https://docs.djangoproject.com/en/1.8/topics/performance/
http://blog.disqus.com/post/62187806135/scaling-django-to-8-billion-page-views
http://scottbarnham.com/blog/2008/04/28/django-performance-testing-a-real-world-example/
Late night reading about scaling up Django can be found in many books, I like this one:
https://highperformancedjango.com/
Sorry if I sound a bit blunt, I just want you to understand that nobody can walk in here and give you an answer with a large degree of confidence. This question doesn't have a straight-forward answer.
TL;DR Start with one instance and scale up only if you've convinced yourself you need to.
You say Memcached or Redis, so I assume Redis would be deployed without persistence, with a purely in-memory configuration.
In such case both Memcached and Redis are unlikely to get saturated even if you run them in one server, since the limiting factor is more likely to be a single Django instance if your requests/second go high.
However you should make sure to have enough memory and to configure an appropriate max memory usage for Memcached / Redis (different ways to accomplish this in the two different services). Note that under memory pressure, the Linux OOM killer may kill your cache otherwise, so if you go for a single instance, which seems to me a sensible first step, make sure your Django memory usage plus the memory you allocate for caching, are not enough to go near the limits of the instance free memory.
CPU is hardly going to be an issue as I said since Memcached / Redis are pretty good at using little CPU, so I can't foresee a setup where Django is ok serving pages but the instance is in trouble since the CPU is burned by the cache.