I started last month in a new company. Where I will be responsible for the infrastructure and the backend of the SAAS.
We currently have one droplet/instance per customer. In the current phase of the company it is a good choice. But in the future when the number of instances grow, it will be difficult to maintain. At the moment there are 150 instances online, each with 1CPU and 1GB memory.
Our customers only use the environments for moments a week, a month or a year. So most of the time, they do nothing. So we want to change that. I am thinking of Kubernetes, Docker Swarm or another tool.
What advice can you give us? Should we make the step to Kubernetes or Docker Swarm, or stay with the droplets / VMs at DigitalOcean, AWS or GCP?
If we move to AWS or GCP our average price will go up from 5$ p/m to above the 10$ p/m.
We want to make the next step to lower the waste of resources but also thinking about the monthly bill. In my mind, it will be better to have 2 our 3 bigger VMs running Kubernetes or Docker Swarm to lower the monthly bill and lower our reserved resources.
What do you think?
If you are serious about scaling, then you should rethink your application architecture. The most expensive part of computing is memory (RAM), so having dedicated memory per-customer will not allow you to scale.
Rather than keeping customers separate by using droplets, you should move this logical separation to the data layer. So, every customer can use the same horizontally-scaled compute servers and databases, but the software separates their data and access based on a User Identifier in the database.
Think for a moment... does Gmail keep RAM around for each specific customer? No, everybody uses the same compute and database, but the software separates their messages from other users. This allows them to scale to huge numbers of customers without assigning per-customer resources.
Here's another couple of examples...
Atlassian used to have exactly what you have. Each JIRA Cloud customer would be assigned their own virtual machine with CPU, RAM and a database. They had to grow their data center to a crazy size, and it was Expensive!
They then embarked on a journey to move to multi-tenancy, first by separating the databases from each customer (and using a common pool of databases), then by moving to shared microservices and eventually they removed all per-customer resources.
See:
Atlassian’s two-year cloud journey | TechCrunch
How Atlassian moved Jira and Confluence users to Amazon Web Services, and what it learned along the way – GeekWire
Atlassian cloud architecture - Atlassian Documentation
Salesforce chose to go multi-tenant from the very beginning. They defined the concept of SaaS and used to call themselves the "cloud" (before Cloud Computing as we know it now). While their systems are sharded to allow scale, multiple customers share the same resources within a shard. The separation of customer data is done at the database-level.
See:
The Magic of Multitenancy - Salesforce Engineering
Multi Tenant Architecture - developer.force.com
Bottom line: Sure, you can try to optimize around the current architecture by using containers, but if you want to get serious about scale (I'm talking 10x or 100x), then you need to re-think the architecture.
Related
I am completely new to the implementation of cloud solutions. I've just started taking AWS training courses.
But I already have a very fundamental question about the flow of development in cloud projects:
How do you go about developing solutions without incurring costs? I know that there are free tiers, but in practice you need a lot of unfree elements. Especially when working with infrastructure-as-code approaches (e.g. CloudFormation), it can happen that every time you try out the templates, costs can be incurred immediately.
Is there maybe something like a sandbox mode or how else do you go about it in practice?
Outside of the AWS Free Tier you will be billed for creating services.
The best way to keep costs as low as possible is to combing the lowest priced settings (such as instance class) with removing resources you're not using after you're complete. I understand that this will cost, however many resources are now moving to per second billing (where you normally have to pay for at least the first minute) so the cost is kept low.
Additionally when dealing with some services (such as EC2, ECS, Fargate and ECR) you can make use of spot instances to pay sometimes as low as 10% of the original cost which will help to reduce these resources.
To ensure you can recreate resources when you want them use infrastructure as code to reroll out as you need the resources (CloudFormation or Terraform are great offerings for this).
Finally be on the lookout for AWS conferences, they are a great way to pickup AWS credits for attending which will offset your bill against most AWS services.
I need to make a cost model for AWS vs GCP. Currently, our organization is using AWS. Our biggest services used are:
EC2
RDS
Labda
AWS Gateway
S3
Elasticache
Cloudfront
Kinesis
I have very limited knowledge of cloud platforms. However, I have access to:
AWS Simple Monthly Calculator
Google Cloud Platform Pricing Calculator
MAP AWS services to GCP products
I also have access to CloudHealth so that I can get a breakdown of costs per services within our organization.
Of the 8 major services listed above are main usage and costs go to EC2, S3, and RDS.
Our director of engineering mentioned that I should be most concerned with vCPU and memory.
I would appreciate any insight (big or small) that people have into how I can go about creating this model, any other factors I should consider, which functionalities of the two providers for the services are considered historically "better" or cheaper, etc.
Thanks in advance, and any questions people may have, I am more than happy to answer.
-M
You should certainly cost-optimize your resources. It's so easy to create cloud resources that people don't always think about turning things off or right-sizing them.
Looking at your Top 5...
Amazon EC2
The simplest way to save money with Amazon EC2 is to turn off unused resources. You can even stop instances overnight and on the weekend. If they are only used 8 hours per workday, then that is only 40 out of 168 hours, so you can save 75% by turning them off when unused! For example, Dev and Test instances. People have written various types of automated utilities to turn instances on and off based on tags. Try search the Internet for AWS Stopinator.
Another way to save money on Amazon EC2 is to use spot instances. They are a fraction of the price, but have a risk that they might be turned off when demand increases. They are great where it is okay for systems to be terminated sometimes, such as automated testing systems. They are also a great way to supplement existing capacity at a fraction of the price.
If you definitely need the Amazon EC2 instances to keep running all the time, purchase Amazon EC2 Reserved Instances, which also offer a price saving.
Chat with your AWS Account Manager for help with the above options.
Amazon Relational Database Service (RDS)
Again, Amazon RDS instances can be stopped overnight/on weekends and turned on again when needed. You only pay while the instance is running (plus storage costs).
Examine the CloudWatch metrics for your RDS instances and determine whether they can be downsized without impacting applications. You can even resize them when they are used less (eg over weekends). Everything can be scripted, so you could trigger such downsizing and upsizing on a schedule.
Also look at the Engine used with RDS. Commercial offerings such as Oracle and Microsoft SQL Server are more expensive than open-source offerings like MySQL and PostgreSQL. Yes, your applications might need some changes, but the cost savings can be significant.
AWS Lambda
It is most unusual that Lambda is #3 in your list. In fact, some customers never get a charge for Lambda because it falls in the monthly free usage tier. Having high charges means you're making good use of Lambda (which is saving you EC2 costs), but take a look at which applications are using it the most and see whether they are using it wisely.
When correctly used, a Lambda function should only ever run for a few seconds, so check whether any application seem to be using it outside this pattern.
AWS API Gateway
Once again, these costs tend to be low ($3.50/million calls) so again I'd recommend trying to figure out how this is being used. If you really need that many calls, it would also explain the high Lambda costs. It would probably be more expensive if you were providing such functionality via Amazon EC2.
Amazon S3
Consider using different Storage Classes to reduce your costs. Costs can be reduced by:
Moving infrequently-accessed data to a different storage class
Moving data to One-Zone (if you have a copy of the data elsewhere, so don't need the redundancy)
Archiving infrequently-accessed data to Amazon Glacier, which offers much cheaper storage but does not have instant access
With GCP, you can benefit by receiving discounts such as the Committed Use Discount and the Sustained Use Discount.
With a Committed Use Discount, you can receive a discount of up to 70% if your usage is predictable.
With the Sustained Use Discount, there is an incremental discount if you reach certain usage thresholds.
On your concern with vCPU and memory, you may use predefined machine types. They are cheaper than custom machine types.
Lastly, you can also test the charges by trying out the Google Cloud Platform Free Tier.
The site I'm working on will potentially get 20,000 visitors per day. It's no guarantee, but it's supposed to be used everyday by each employee in an organisation.
In the past I've just used a single t2.micro EC2 instance with an attached EBS volume to host sites, which has always been enough because these sites don't get a lot of traffic. But with 20,000 visitors a day how could I improve my AWS architecture to scale?
The site is going to have a profile for each user, including a profile picture - so potentially 20,000 image files. Should I be writing these to an S3 bucket instead of to the EBS?
Would a t2.micro ec2 instance cope with the scale, or should I be using a t2.small, t2.medium or even t2.large?
My MySQL databases are currently on the EBS volume, but should I use RDS?
All the users are in the UK, so I'm assuming using CloudFront is overkill?
You're right to assume CloudFront is overkill since all your users are localized to UK.
Update: using a CDN will take a lot of stress off your servers by caching the files rather than processing them each time a call is made.
Look at it this way, if you get 100,000 hits a day, and 90% of those hits are cached and served by the CDN, then your server only has to process 10,000 hits a day. That could be the difference between needing a m4.xlarge versus just needing a t2.small.
#mark-b
Use the Ireland region (and soon you can copy over to the UK region)
If you want to keep your database on your instance I would highly recommend a bit bigger one. As for a quick and easy solution, start up the smallest T series instance with EBS, beta test with 1000-5000 users, see how that goes. Notify the select group all their crap will disappear so don't invest a bunch o' time.
Next, get your analytics on the system and see if that will work times 4-5 more users. For SQL DB stuff you'll eventually want a M series instance I believe.
Also, you could always create a load balanced fleet. You do this in EBS by hitting Load Balanced instead of Single Instance. Create an auto scaling group and boom sauce - check that off.
As for the images, yeah I would recommend S3. Don't really want to dump the whole amount in i/o cause DB, hits, i/o, all on one instance is a lot.
Lastly, if you do plan on going to the UK region (not positive if that's been rolled out yet) I would recommend sectioning all the pieces of your application. This is really good practice to use all the resources they provide.
For a very fault tolerant system:
EC2 fleet (m or c series) with an ELB
S3 the images
RDS the users
CloudWatch the stats
Tenecy the users with IAM groups
Authenticate with STS or AD or whatever (kinda been in the cognito only recently)
Store their session and authenticated crap in ElastiCache - Redis
Keep tabs on them with Kinesis (optional)
And let them search each other with CloudSearch (also optional)
Boss system right there!
And that's if you want to spend a bunch o' cash but have a sweet sweet system. If you want to spend next to nothing, make it serverless. A broad question asked with hundreds of combinations so this is up to interpretation.
Hope this helps!
I am at the beginning of a large migration from a single monolithic web service to a collection of microservices using Spring Cloud/Spring Cloud Netflix. Through my research of microservices I understand that the lines of demarcation between services should mirror the separations of concerns between them. An additional factor affecting separation is which services are required to scale individually.
As a concrete example, depending on the level of granularity desired, a microservice environment could end up like this:
Accounts (containing Signup, Login, Profiles, etc.)
Store (containing Products, Payments, Reporting, Inventories, etc.)
Chat/Social (containing chat rooms, user statuses, etc.)
...
Or it could end up with each of the areas of concern in brackets represented by their own microservice, e.g:
Accounts
Signup
Login
...
I believe there is a preference in the microservices community for the second approach, and I tend to agree. However, the issue I have is one of hosting and resource limitations.
In the migration I would like to streamline the provisioning of resources and the installation of updated services. Since we use the AWS stack, Elastic Beanstalk seemed like the perfect choice. While researching Elastic Beanstalk though I was rather disheartened to discover that there was a limit of 25 applications per account. Not only that, but EC2 has a limit of 20 instances per region per account. It seems like a microservice architecture will hit that limit very quickly, especially when you add multiple environments (staging and production) for each service into the mix, let alone websites and internal tooling.
With all of the amazing content that I've seen around the web regarding microservices, I'm surprised and somewhat disappointed at the lack of information regarding the actual hosting of microservices beyond the development of them. Have I missed something? Is there any information about deploying more than a couple of microservices on AWS?
It is my understanding that Netflix use AWS for their own microservice hosting, beyond requesting additional resources from Amazon and throwing money at it, are there other solutions? Would their Asgard tool help with this issue (possibly by handling the sharing of instances between services) or would it result in the same outcome?
As mentioned in the above comments, AWS will raise your limits if you have a legit use case - why wouldn't they? they are in the business of selling you services.
But since you have asked for suggestion other than increasing those limits, and since you are in the early stages of designing your solution, you should consider basing part of your micro-services architecture on Docker or another container/container like service (my own preference would be for the AWS's container service). Depending on the nature of you solution, even within the limits of 20 EC2 instances (per region), if you had large enough instances running you could fit dozens (or even hundreds of lightweight) docker images running on each of those allocated 20 instances - so potentially hundres or thousands of walled off micro-services running on those 20 EC2 instances.
Using an entire EC2 image for each of many micro-services you may have may end up being a lot more expensive than it needs to be.
You should also consider the use of AWS Lamba for at least portions of your micro-service architecture - its the 'ultra-micro service' tool also offered by AWS.
every once in a while i read/hear about AWS and now i tried reading the docs.
But such docs seem to be written for people who already know which AWS they need to use and only search for how it can be used.
So, for myself, to understand AWS better i try to sketch a hypothetical Webapplication with a few questions.
The apps purpose is to modify content like videos or images. So a user has some kind of webinterface where he can upload his files, do some settings and a server grabs the file and modifies it (e.g. reencoding). The Service also extracts the audio track of a video and trys to index the spoken words so the customer can search within his videos. (well its just hypothetical)
So my questions:
given my own domain 'oneofmydomains.com' is it possible to host the complete webinterface on AWS? i thought about using GWT to create the interface and just deliver the JS/images via AWS, but which one, simple storage? what about some kind of index.html, is there an EC2 instance needed to host a webserver which has to run 24/7 causing costs?
now the user has the interface with a login form, is it possible to manage logins with an AWS? here i also think about an EC2 instance hosting a database, but it would also cause costs and im not sure if there is a better way?
the user has logged in and uploads a file. which storage solution could be used to save the customers original and modified content?
now the user wants to browse the status of his uploads, this means i need some kind of ACL, so that the customer only sees his own files. do i need to use a database (e.g. EC2) for this, or does amazon provide some kind of ACL, so the GWT webinterface will be secure without any EC2?
the customers files are reencoded and the audio track is indexed. so he wants to search for a video. Which service could be used to create and maintain the index for each customer?
hope someone can give a few answers so i understand AWS better on how one could use it
thx!
Amazon AWS offers a whole ecosystem of services which should cover all aspects of a given architecture, from hosting to data storage, or messaging, etc. Whether they're the best fit for purpose will have to be decided on a case by case basis. Seeing as your question is quite broad I'll just cover some of the basics of what AWS has to offer and what the different types of services are for:
EC2 (Elastic Cloud Computing)
Amazon's cloud solution, which is basically the same as older virtual machine technology but the 'cloud' offers additional knots and bots such as automated provisioning, scaling, billing etc.
you pay for what your use (by hour), for the basic (single CPU, 1.7GB ram) would prob cost you just under $3 a day if you run it 24/7 (on a windows instance that is)
there's a number of different OS to choose from including linux and windows, linux instances are cheaper to run without the license cost associated with windows
once you're set up the server to be the way you want, including any server updates/patches, you can create your own AMI (Amazon machine image) which you can then use to bring up another identical instance
however, if all your html are baked into the image it'll make updates difficult, so normal approach is to include a service (windows service for instance) which will pull the latest deployment package from a storage (see S3 later) service and update the site at start up and at intervals
there's the Elastic Load Balancer (which has its own cost but only one is needed in most cases) which you can put in front of all your web servers
there's also the Cloud Watch (again, extra cost) service which you can enable on a per instance basis to help you monitor the CPU, network in/out, etc. of your running instance
you can set up AutoScalers which can automatically bring up or terminate instances based on some metric, e.g. terminate 1 instance at a time if average CPU utilization is less than 50% for 5 mins, bring up 1 instance at a time if average CPU goes beyond 70% for 5 mins
you can use the instances as web servers, use them to run a DB, or a Memcache cluster, etc. choice is yours
typically, I wouldn't recommend having Amazon instances talk to a DB outside of Amazon because of the round trip is much longer, the usual approach is to use SimpleDB (see below) as the database
the AmazonSDK contains enough classes to help you write some custom monitor/scaling service if you ever need to, but the AWS console allows you to do most of your configuration anyway
SimpleDB
Amazon's non-relational, key-value data store, compared to a traditional database you tend to pay a penalty on per query performance but get high scalability without having to do any extra work.
you pay for usage, i.e. how much work it takes to execute your query
extremely scalable by default, Amazon scales up SimpleDB instances based on traffic without you having to do anything, AND any control for that matter
data are partitioned in to 'domains' (equivalent to a table in normal SQL DB)
data are non-relational, if you need a relational model then check out Amazon RDB, I don't have any experience with it so not the best person to comment on it..
you can execute SQL like query against the database still, usually through some plugin or tool, Amazon doesn't provide a front end for this at the moment
be aware of 'eventual consistency', data are duplicated on multiple instances after Amazon scales up your database, and synchronization is not guaranteed when you do an update so it's possible (though highly unlikely) to update some data then read it back straight away and get the old data back
there's 'Consistent Read' and 'Conditional Update' mechanisms available to guard against the eventual consistency problem, if you're developing in .Net, I suggest using SimpleSavant client to talk to SimpleDB
S3 (Simple Storage Service)
Amazon's storage service, again, extremely scalable, and safe too - when you save a file on S3 it's replicated across multiple nodes so you get some DR ability straight away.
you only pay for data transfer
files are stored against a key
you create 'buckets' to hold your files, and each bucket has a unique url (unique across all of Amazon, and therefore S3 accounts)
CloudBerry S3 Explorer is the best UI client I've used in Windows
using the AmazonSDK you can write your own repository layer which utilizes S3
Sorry if this is a bit long winded, but that's the 3 most popular web services that Amazon provides and should cover all the requirements you've mentioned. We've been using Amazon AWS for some time now and there's still some kinks and bugs there but it's generally moving forward and pretty stable.
One downside to using something like aws is being vendor locked-in, whilst you could run your services outside of amazon and in your own datacenter or moving files out of S3 (at a cost though), getting out of SimpleDB will likely to represent the bulk of the work during migration.