I'm working on a project for a web app where each enterprise customer will have separate resources independent to themselves (i.e. users and user data unique to a single paying customer, content unique for each paying customer, etc). It's somewhat similar to a normal CRM. We're building on AWS and planning to use RDS for the database. Given the fact that each customer does not share data across any different region, would it be most effective to:
Upon enterprise customer sign up, a new VPC is created with new RDS and EC2 instances, so that each customer has siloed databased and services
Have a single VPC with RDS and EC2 instances shared across all customers and use an indicator field in the database to determine what data belongs to each customer
We don't anticipate having over 10000 users for any given enterprise customer, and right now the project is small with only a single enterprise customer.
This answer all depends on how you anticipate your application growth (not just in terms of number of customers but also performance), however personally I would say if you have a single enterprise customer at the moment create a single VPC.
Perhaps separate out a seperate database on the RDS for each customer initially assuming that cost would become a concern, with a seperate db user for each customer to limit exposure.
If the financials allow then you would look at separating out to a seperate RDS database where the performance demands (maybe you have tiers for SLAs etc).
If you get larger customers (who may have bespoke needs, or stringent requirements such as no multi tenant hosts) I would suggest you look at an organizations setup with a account per large customer to separate out the resources and a single account for any shared or multi-customer resources.
The key is plan ahead for how you would want to but if you're still in the early midst of onboarding clients then do not over complicate your setup yet.
Related
I'm working for a german company, therefor we're bound to GDPR. We're selling our product as a software as a service offering and are hosting the systems in AWS. Our customers are spread over Europe, USA and Asia. So we're running multiple VPCs in AWS in the Regions EU-West, US-Northeast and APAC. Our plan is to implement Keycloak as SSO backend.
Up to this point our initial idea was to imlement Keycloak with the so called Cross Data Center Replication. This would mean one Keycloak-Cluster per VPC with a load balancer in front, the Infispan cluster for inter VPC caching/communication and an Aurora RDS cluster as the centralised database, but we are not pinned to that. The problem is, as mentioned above, we're bound to the GDPR and so the data of european users must not leave the EU except the customer orders us to do so. All I've read is saying that Keycloak is expecting that all data is synced accross the database cluster.
Information about our topology and the issue itself:
Every customer has dedicated EC2 instances in the best suitable geographical region. Additionally there are centralized services hosted in the EU. So user from the USA or APAC need to have access to systems in the EU but EU users don't need to have access to instances/services outside of the EU, except the customers explicitely orders it.
So how do we achieve this?
My only idea atm would be to build up a database cluster (likely not AWS Aurora RDS) and configure on the database itself to not sync all of the data. But this sounds very gross to me and I don't think, that Keycloak is doing well with this. Any ideas or tips would be appreciated!
Ok, in case someone is interested in our solution:
It seems, that we will accept the fact, that not EU customer have a latency under some circumstances. The Infinispan cluster will serve as a cache, so these users will only have once in a while this latency. The DB will reside in the EU.
I'm not sure, if the Infinispan servers will act as a cache, if the connection to the nodes in the other datacenters is lost, but maybe I'll find something about that.
I plan to setup autoscaling on my api nodes, but want them to connect to the same RDS instance (with Multi AZ).
Will i notice any performance impacts if the api nodes all share the same db account?
Otherwise, what alternatives do i have that wont complicate the deployment process significantly?
Thanks
A database user account is used to control security (eg Is the user allowed to perform a command) and logging (eg Which user deleted the table?).
Having applications on multiple Amazon EC2 instances use the same database user account will not impact performance. Two requests coming from different database users will use the same amount of resource as two requests coming from the same database user.
An exception to this would be if any database configurations restrict resources by user, such as processing queues. However, this would not be a default behavior.
I am about to launch an iOS app that will be communicating with my custom REST API. Right now I am running a single EC2 t2.micro instance running an Apache web server with MySQLi. Before I go ahead and launch it for the public, I want to hear what proper steps should be taken regarding the following.
Should I run two separate EC2 instances? One only for the web server and the other to handle only the database?
How should I approach setting up the database? Should I still use MySQLi or should I start using Amazon's RDS?
In relationship to number two, when the database and/or web server runs out of space, how is this issue handled so that it seamlessly adds space to allow the database/web server to continue growth? I also read something regarding auto-scale.
I will be expecting many requests per minute to my web server and want to take precaution.
The answer to these questions largely depends on the requirements of your application, your budget, and on what you decide to manage vs. what you'd prefer to allow AWS to manage. However, I'll answer these as best I can.
1) Yes. Separating the database from the web server (that is, 2 different EC2 instances) makes sense for a lot of reasons. This will allow you to tailor resources like memory, CPU, etc. to each layer of your application separately. You do not want your web and database competing for the same resources. Additionally, an issue that forces you to take down one (web or database) will not force you to also take down the other. If your database lives on one of the web servers and you need to perform maintenance, your app will effectively become offline, since down goes your database as you perform updates. Also, ideally you would protect your database server within a private subnet in your VPC. If you have the web and database on the same server, they will both be in a public subnet, since you're web will require access to an internet gateway.
2) Depends. If you want to maintain total control of the database server, than use an EC2 instance where you retain operating system control. If you want to take advantage of features like Multi-AZ for high availability or allowing AWS to manage things like updates for you, RDS can be a great option. Cost also plays a role. For things like read-replicas and Multi-AZ, you will pay more, but you are purchasing performance and high availability. Thus, depends on your requirements. You can find the features of RDS here: RDS Product Details
3) For anything running on an EC2 instance (database or web) or if you decide to use RDS, you may provision and attach additional storage volumes as necessary. The type of storage you select will depend on the performance requirements, your budget, and the kind of workload you expect your database to face. Amazon provides the storage options available to you as well as a section for adding more storage here: RDS Storage Options
If you are worried about too many requests overwhelming your EC2 t2.micro instance, consider creating an ELB load balancer and setting up an auto-scaling group which will allow you to expand your capacity as necessary while distributing traffic such that no one server gets overwhelmed.
We host our clients' infrastructure. We are looking at AWS for some services. However, our business model is to provide the hosting included in our fee so to that end we need to be the account holder. We therefore need to have one account which is billed but clear delineation between different separate client infrastructure. Ideally with a separate console etc. Is this possible? Thanks.
You should take a look at consolidated billing, as it sounds like it will meet your needs:
http://docs.aws.amazon.com/awsaccountbilling/latest/about/consolidatedbilling.html
This would enable you to set up unique "linked accounts" per customer, each fully separate and independent of each other (separate console logins, separate security configurations, etc.).
every once in a while i read/hear about AWS and now i tried reading the docs.
But such docs seem to be written for people who already know which AWS they need to use and only search for how it can be used.
So, for myself, to understand AWS better i try to sketch a hypothetical Webapplication with a few questions.
The apps purpose is to modify content like videos or images. So a user has some kind of webinterface where he can upload his files, do some settings and a server grabs the file and modifies it (e.g. reencoding). The Service also extracts the audio track of a video and trys to index the spoken words so the customer can search within his videos. (well its just hypothetical)
So my questions:
given my own domain 'oneofmydomains.com' is it possible to host the complete webinterface on AWS? i thought about using GWT to create the interface and just deliver the JS/images via AWS, but which one, simple storage? what about some kind of index.html, is there an EC2 instance needed to host a webserver which has to run 24/7 causing costs?
now the user has the interface with a login form, is it possible to manage logins with an AWS? here i also think about an EC2 instance hosting a database, but it would also cause costs and im not sure if there is a better way?
the user has logged in and uploads a file. which storage solution could be used to save the customers original and modified content?
now the user wants to browse the status of his uploads, this means i need some kind of ACL, so that the customer only sees his own files. do i need to use a database (e.g. EC2) for this, or does amazon provide some kind of ACL, so the GWT webinterface will be secure without any EC2?
the customers files are reencoded and the audio track is indexed. so he wants to search for a video. Which service could be used to create and maintain the index for each customer?
hope someone can give a few answers so i understand AWS better on how one could use it
thx!
Amazon AWS offers a whole ecosystem of services which should cover all aspects of a given architecture, from hosting to data storage, or messaging, etc. Whether they're the best fit for purpose will have to be decided on a case by case basis. Seeing as your question is quite broad I'll just cover some of the basics of what AWS has to offer and what the different types of services are for:
EC2 (Elastic Cloud Computing)
Amazon's cloud solution, which is basically the same as older virtual machine technology but the 'cloud' offers additional knots and bots such as automated provisioning, scaling, billing etc.
you pay for what your use (by hour), for the basic (single CPU, 1.7GB ram) would prob cost you just under $3 a day if you run it 24/7 (on a windows instance that is)
there's a number of different OS to choose from including linux and windows, linux instances are cheaper to run without the license cost associated with windows
once you're set up the server to be the way you want, including any server updates/patches, you can create your own AMI (Amazon machine image) which you can then use to bring up another identical instance
however, if all your html are baked into the image it'll make updates difficult, so normal approach is to include a service (windows service for instance) which will pull the latest deployment package from a storage (see S3 later) service and update the site at start up and at intervals
there's the Elastic Load Balancer (which has its own cost but only one is needed in most cases) which you can put in front of all your web servers
there's also the Cloud Watch (again, extra cost) service which you can enable on a per instance basis to help you monitor the CPU, network in/out, etc. of your running instance
you can set up AutoScalers which can automatically bring up or terminate instances based on some metric, e.g. terminate 1 instance at a time if average CPU utilization is less than 50% for 5 mins, bring up 1 instance at a time if average CPU goes beyond 70% for 5 mins
you can use the instances as web servers, use them to run a DB, or a Memcache cluster, etc. choice is yours
typically, I wouldn't recommend having Amazon instances talk to a DB outside of Amazon because of the round trip is much longer, the usual approach is to use SimpleDB (see below) as the database
the AmazonSDK contains enough classes to help you write some custom monitor/scaling service if you ever need to, but the AWS console allows you to do most of your configuration anyway
SimpleDB
Amazon's non-relational, key-value data store, compared to a traditional database you tend to pay a penalty on per query performance but get high scalability without having to do any extra work.
you pay for usage, i.e. how much work it takes to execute your query
extremely scalable by default, Amazon scales up SimpleDB instances based on traffic without you having to do anything, AND any control for that matter
data are partitioned in to 'domains' (equivalent to a table in normal SQL DB)
data are non-relational, if you need a relational model then check out Amazon RDB, I don't have any experience with it so not the best person to comment on it..
you can execute SQL like query against the database still, usually through some plugin or tool, Amazon doesn't provide a front end for this at the moment
be aware of 'eventual consistency', data are duplicated on multiple instances after Amazon scales up your database, and synchronization is not guaranteed when you do an update so it's possible (though highly unlikely) to update some data then read it back straight away and get the old data back
there's 'Consistent Read' and 'Conditional Update' mechanisms available to guard against the eventual consistency problem, if you're developing in .Net, I suggest using SimpleSavant client to talk to SimpleDB
S3 (Simple Storage Service)
Amazon's storage service, again, extremely scalable, and safe too - when you save a file on S3 it's replicated across multiple nodes so you get some DR ability straight away.
you only pay for data transfer
files are stored against a key
you create 'buckets' to hold your files, and each bucket has a unique url (unique across all of Amazon, and therefore S3 accounts)
CloudBerry S3 Explorer is the best UI client I've used in Windows
using the AmazonSDK you can write your own repository layer which utilizes S3
Sorry if this is a bit long winded, but that's the 3 most popular web services that Amazon provides and should cover all the requirements you've mentioned. We've been using Amazon AWS for some time now and there's still some kinks and bugs there but it's generally moving forward and pretty stable.
One downside to using something like aws is being vendor locked-in, whilst you could run your services outside of amazon and in your own datacenter or moving files out of S3 (at a cost though), getting out of SimpleDB will likely to represent the bulk of the work during migration.