I am at the beginning of a large migration from a single monolithic web service to a collection of microservices using Spring Cloud/Spring Cloud Netflix. Through my research of microservices I understand that the lines of demarcation between services should mirror the separations of concerns between them. An additional factor affecting separation is which services are required to scale individually.
As a concrete example, depending on the level of granularity desired, a microservice environment could end up like this:
Accounts (containing Signup, Login, Profiles, etc.)
Store (containing Products, Payments, Reporting, Inventories, etc.)
Chat/Social (containing chat rooms, user statuses, etc.)
...
Or it could end up with each of the areas of concern in brackets represented by their own microservice, e.g:
Accounts
Signup
Login
...
I believe there is a preference in the microservices community for the second approach, and I tend to agree. However, the issue I have is one of hosting and resource limitations.
In the migration I would like to streamline the provisioning of resources and the installation of updated services. Since we use the AWS stack, Elastic Beanstalk seemed like the perfect choice. While researching Elastic Beanstalk though I was rather disheartened to discover that there was a limit of 25 applications per account. Not only that, but EC2 has a limit of 20 instances per region per account. It seems like a microservice architecture will hit that limit very quickly, especially when you add multiple environments (staging and production) for each service into the mix, let alone websites and internal tooling.
With all of the amazing content that I've seen around the web regarding microservices, I'm surprised and somewhat disappointed at the lack of information regarding the actual hosting of microservices beyond the development of them. Have I missed something? Is there any information about deploying more than a couple of microservices on AWS?
It is my understanding that Netflix use AWS for their own microservice hosting, beyond requesting additional resources from Amazon and throwing money at it, are there other solutions? Would their Asgard tool help with this issue (possibly by handling the sharing of instances between services) or would it result in the same outcome?
As mentioned in the above comments, AWS will raise your limits if you have a legit use case - why wouldn't they? they are in the business of selling you services.
But since you have asked for suggestion other than increasing those limits, and since you are in the early stages of designing your solution, you should consider basing part of your micro-services architecture on Docker or another container/container like service (my own preference would be for the AWS's container service). Depending on the nature of you solution, even within the limits of 20 EC2 instances (per region), if you had large enough instances running you could fit dozens (or even hundreds of lightweight) docker images running on each of those allocated 20 instances - so potentially hundres or thousands of walled off micro-services running on those 20 EC2 instances.
Using an entire EC2 image for each of many micro-services you may have may end up being a lot more expensive than it needs to be.
You should also consider the use of AWS Lamba for at least portions of your micro-service architecture - its the 'ultra-micro service' tool also offered by AWS.
Related
To automatically manage Cloud resources in order to meet the felt need of my infrastructure, I need to increase VM resources. But this is possible only while the machine status is TERMINATED.
The problem is that I have got applications on the VMs that must not stop running. Do you have any suggestions about how I could proceed, like increasing my machines resources without interrupting its services? (database, web, etc...)
The purpose of that is to automate my whole infrastructure, to ensure its quality of service even if I'm not monitoring it by myself.
I suggest to take a look to Managed Instance Groups. It will offer some of the characteristics you need: high availability, scalability and the ability to add the instance group to a load balancer.
According to the Google official documentation about MIGs:
Make your workloads scalable and highly available by taking advantage of automated MIG services, including: autoscaling, autohealing, regional (multiple zone) deployment, and automatic updating.
Regarding the need of automation of the services you want, I suggest to generally use fully managed services. You can check a summary of GCP services and you can always inspect if they fit your demands.
I have a general question regarding how APIs are scaled. I have a basic RESTful API powered by Django rest framework, the backend uses RDS for database management. Right now, I'm deploying my django application to a digitalocean droplet but thinking of switching over to EC2 or potentially EKS. Is my understanding correct that I can effectively point my application to the RDS endpoint and spin up several EC2 instances with the same Django application fronted by an ELB? Would this take care of incoming traffic and the scalability of the django application?
This isn't exactly a coding question so I'm not sure if this is the best stackexchange site to ask this.
My two cents here:
I`ve been using lambda to serve Django and Flask apis for quite some time now, and it works great. You don't need to worry about scalability at all, unless there is a chance that your API would receive more than 10,000 requests per second (very unlikely on most scenarios). It will be way cheaper than EKS, even cheaper than EC2. I have a app with 400k active users which is served by an API running on lambda, I never paid more than $25 on invokations.
You can use Zappa (which is exclusively for python, I recommend) or Serverless framework, they will take care of most of the heavy work and make the deployment very easy.
But have in mind that lambda is not very good for long running tasks, like cronjobs. If you have have crons that might take some time to be executed your invokations can get a little expensive if you invoke it ofteen (lambdas can run up to 15 minutes, but those 15 minutes will be much more expensive than EC2). Also, the apigateway in front of the lambda function have a 30 seconds timeout, so your requests must be processed before that. If you think your requests will take longer, you will need to leverage some async requests. I think it is a very small price to have a full service without having to worry about the infrastructure.
You are right, but you can think not only about ec2 and EKS. You can also look into ECS and Fargate options. ELB distribute traffic across compute resources inside Target Group and it can be Autoscaling Group for EC2. Also, with RDS you can scale read replicas for handling mor read traffic independent from master node
I started last month in a new company. Where I will be responsible for the infrastructure and the backend of the SAAS.
We currently have one droplet/instance per customer. In the current phase of the company it is a good choice. But in the future when the number of instances grow, it will be difficult to maintain. At the moment there are 150 instances online, each with 1CPU and 1GB memory.
Our customers only use the environments for moments a week, a month or a year. So most of the time, they do nothing. So we want to change that. I am thinking of Kubernetes, Docker Swarm or another tool.
What advice can you give us? Should we make the step to Kubernetes or Docker Swarm, or stay with the droplets / VMs at DigitalOcean, AWS or GCP?
If we move to AWS or GCP our average price will go up from 5$ p/m to above the 10$ p/m.
We want to make the next step to lower the waste of resources but also thinking about the monthly bill. In my mind, it will be better to have 2 our 3 bigger VMs running Kubernetes or Docker Swarm to lower the monthly bill and lower our reserved resources.
What do you think?
If you are serious about scaling, then you should rethink your application architecture. The most expensive part of computing is memory (RAM), so having dedicated memory per-customer will not allow you to scale.
Rather than keeping customers separate by using droplets, you should move this logical separation to the data layer. So, every customer can use the same horizontally-scaled compute servers and databases, but the software separates their data and access based on a User Identifier in the database.
Think for a moment... does Gmail keep RAM around for each specific customer? No, everybody uses the same compute and database, but the software separates their messages from other users. This allows them to scale to huge numbers of customers without assigning per-customer resources.
Here's another couple of examples...
Atlassian used to have exactly what you have. Each JIRA Cloud customer would be assigned their own virtual machine with CPU, RAM and a database. They had to grow their data center to a crazy size, and it was Expensive!
They then embarked on a journey to move to multi-tenancy, first by separating the databases from each customer (and using a common pool of databases), then by moving to shared microservices and eventually they removed all per-customer resources.
See:
Atlassian’s two-year cloud journey | TechCrunch
How Atlassian moved Jira and Confluence users to Amazon Web Services, and what it learned along the way – GeekWire
Atlassian cloud architecture - Atlassian Documentation
Salesforce chose to go multi-tenant from the very beginning. They defined the concept of SaaS and used to call themselves the "cloud" (before Cloud Computing as we know it now). While their systems are sharded to allow scale, multiple customers share the same resources within a shard. The separation of customer data is done at the database-level.
See:
The Magic of Multitenancy - Salesforce Engineering
Multi Tenant Architecture - developer.force.com
Bottom line: Sure, you can try to optimize around the current architecture by using containers, but if you want to get serious about scale (I'm talking 10x or 100x), then you need to re-think the architecture.
We are trying to evaluate the best ways to scale our J2EE web application and use hosting services with AWS. Are there reasons why we would use the Lambda service over Kubernetes (EKS)? Although it seems that Lambda can scale functional units, I'm not clear why anyone would use that as a substitute for Kubernetes, given Kubernetes can replicate containers based on performance metrics.
They serve different purposes. If you want to have horizontal scalability on a "ec2/pod/container" level and handle the availability yourself (through k8s of course), go for Kubernetes.
If you have a straight forward function doing a particular thing and you don't want to bother yourself with operating costs of having to manage a cluster or packaging it, then you can let Lambda administer it for you (at the time of writing, you would pay 20 US cents per million call). It is just another layer of abstraction on top of a system that is probably similar to Kubernetes, scaling your function per needs.
The goal of these technologies is to remove as much overhead as possible between you and the code and infrastructure can be painful. To summarize, serverless is to Kubernetes what Kubernetes is to containers.
To make a conscious decision, take the following into account:
Does your budget covers operation and maintenance of infrastructure
Do you have the expertise in Kubernetes
How much would it cost to redesign your J2EE app into serverless
ready code
Your timeline (of course...)
Based on the AWS resources you will use, how much do you save or not
by implementing a k8s cluster (database service?, EBS, EC2s, etc.)
every once in a while i read/hear about AWS and now i tried reading the docs.
But such docs seem to be written for people who already know which AWS they need to use and only search for how it can be used.
So, for myself, to understand AWS better i try to sketch a hypothetical Webapplication with a few questions.
The apps purpose is to modify content like videos or images. So a user has some kind of webinterface where he can upload his files, do some settings and a server grabs the file and modifies it (e.g. reencoding). The Service also extracts the audio track of a video and trys to index the spoken words so the customer can search within his videos. (well its just hypothetical)
So my questions:
given my own domain 'oneofmydomains.com' is it possible to host the complete webinterface on AWS? i thought about using GWT to create the interface and just deliver the JS/images via AWS, but which one, simple storage? what about some kind of index.html, is there an EC2 instance needed to host a webserver which has to run 24/7 causing costs?
now the user has the interface with a login form, is it possible to manage logins with an AWS? here i also think about an EC2 instance hosting a database, but it would also cause costs and im not sure if there is a better way?
the user has logged in and uploads a file. which storage solution could be used to save the customers original and modified content?
now the user wants to browse the status of his uploads, this means i need some kind of ACL, so that the customer only sees his own files. do i need to use a database (e.g. EC2) for this, or does amazon provide some kind of ACL, so the GWT webinterface will be secure without any EC2?
the customers files are reencoded and the audio track is indexed. so he wants to search for a video. Which service could be used to create and maintain the index for each customer?
hope someone can give a few answers so i understand AWS better on how one could use it
thx!
Amazon AWS offers a whole ecosystem of services which should cover all aspects of a given architecture, from hosting to data storage, or messaging, etc. Whether they're the best fit for purpose will have to be decided on a case by case basis. Seeing as your question is quite broad I'll just cover some of the basics of what AWS has to offer and what the different types of services are for:
EC2 (Elastic Cloud Computing)
Amazon's cloud solution, which is basically the same as older virtual machine technology but the 'cloud' offers additional knots and bots such as automated provisioning, scaling, billing etc.
you pay for what your use (by hour), for the basic (single CPU, 1.7GB ram) would prob cost you just under $3 a day if you run it 24/7 (on a windows instance that is)
there's a number of different OS to choose from including linux and windows, linux instances are cheaper to run without the license cost associated with windows
once you're set up the server to be the way you want, including any server updates/patches, you can create your own AMI (Amazon machine image) which you can then use to bring up another identical instance
however, if all your html are baked into the image it'll make updates difficult, so normal approach is to include a service (windows service for instance) which will pull the latest deployment package from a storage (see S3 later) service and update the site at start up and at intervals
there's the Elastic Load Balancer (which has its own cost but only one is needed in most cases) which you can put in front of all your web servers
there's also the Cloud Watch (again, extra cost) service which you can enable on a per instance basis to help you monitor the CPU, network in/out, etc. of your running instance
you can set up AutoScalers which can automatically bring up or terminate instances based on some metric, e.g. terminate 1 instance at a time if average CPU utilization is less than 50% for 5 mins, bring up 1 instance at a time if average CPU goes beyond 70% for 5 mins
you can use the instances as web servers, use them to run a DB, or a Memcache cluster, etc. choice is yours
typically, I wouldn't recommend having Amazon instances talk to a DB outside of Amazon because of the round trip is much longer, the usual approach is to use SimpleDB (see below) as the database
the AmazonSDK contains enough classes to help you write some custom monitor/scaling service if you ever need to, but the AWS console allows you to do most of your configuration anyway
SimpleDB
Amazon's non-relational, key-value data store, compared to a traditional database you tend to pay a penalty on per query performance but get high scalability without having to do any extra work.
you pay for usage, i.e. how much work it takes to execute your query
extremely scalable by default, Amazon scales up SimpleDB instances based on traffic without you having to do anything, AND any control for that matter
data are partitioned in to 'domains' (equivalent to a table in normal SQL DB)
data are non-relational, if you need a relational model then check out Amazon RDB, I don't have any experience with it so not the best person to comment on it..
you can execute SQL like query against the database still, usually through some plugin or tool, Amazon doesn't provide a front end for this at the moment
be aware of 'eventual consistency', data are duplicated on multiple instances after Amazon scales up your database, and synchronization is not guaranteed when you do an update so it's possible (though highly unlikely) to update some data then read it back straight away and get the old data back
there's 'Consistent Read' and 'Conditional Update' mechanisms available to guard against the eventual consistency problem, if you're developing in .Net, I suggest using SimpleSavant client to talk to SimpleDB
S3 (Simple Storage Service)
Amazon's storage service, again, extremely scalable, and safe too - when you save a file on S3 it's replicated across multiple nodes so you get some DR ability straight away.
you only pay for data transfer
files are stored against a key
you create 'buckets' to hold your files, and each bucket has a unique url (unique across all of Amazon, and therefore S3 accounts)
CloudBerry S3 Explorer is the best UI client I've used in Windows
using the AmazonSDK you can write your own repository layer which utilizes S3
Sorry if this is a bit long winded, but that's the 3 most popular web services that Amazon provides and should cover all the requirements you've mentioned. We've been using Amazon AWS for some time now and there's still some kinks and bugs there but it's generally moving forward and pretty stable.
One downside to using something like aws is being vendor locked-in, whilst you could run your services outside of amazon and in your own datacenter or moving files out of S3 (at a cost though), getting out of SimpleDB will likely to represent the bulk of the work during migration.