Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
AWS provides services like Elasticache, redis, databases and all are charged on hourly basis. But these services are also available in form of docker containers in docker hub. All the AWS services liste above use an instance. Meaning, an independent instance for databases and all. But what if one starts using an ec2 instance, and start downloading all the images for all the dependancies on databases. That would save them a lot of money right?
I have used docker before and it has almost all the images for the services aws provides.
EC2 is not free. You can run, for example, MySQL on an EC2 instance. It will be cheaper than using RDS, but you still need to pay for the compute and storage resources it consumes. Even if you run a database on a larger shared EC2 instance you need to account for its storage and CPU cycles, and you might need more or larger instances to run more tasks there.
(As of right now, in the us-east-1 region, a MySQL db.m5.large instance is US$0.171 per hour or US$895 per year paid up front, plus US$0.115 per GB of capacity per month; the same m5.large EC2 instance is US$0.096 per hour or US$501 per year, and storage is US$0.10 per GB per month. [Assuming 1-year, all-up-front, non-convertible reserved instances.])
There are good reasons to run databases not-in-Docker. Particularly in a microservice environment, application Docker containers are stateless, replicated, update their images routinely, can be freely deleted, and can be moved across hosts (by deleting and recreating them somewhere else). (In Kubernetes/EKS, look at how a Deployment object works.) None of these are true of databases, which are all about keeping state, cannot be deleted, cannot be moved (the data has to come with), and must be backed up.
RDS has some useful features. You can change the size of your database instance with some downtime, but no data loss. AWS will keep managed snapshots for you, and it's straightforward (if slow) to create a new database from a snapshot of your existing database. Patch updates to the database are automatically applied for you. You can pay Amazon for these features, in effect, or pay your own DBA to do the same tasks for a database running on an EC2 instance.
None of this is to say you have to use RDS, you do in fact save on AWS by running the same software on EC2, and it may or may not be in Docker. RDS is a reasonable choice in an otherwise all-Docker world though. The same basic tradeoffs apply to other services like Elasticache (for Redis).
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
Hey guys I need advice in order to make the right architectural decision.
I need to be able to run the console application (or docker container in the future) on different locations (Countries/Citys) without paying for hundreds always running virtual machines.
In other words, I need to press the button and run the application for a couple of hours on a server in New York, next press, and the same application will be run in Stambul.
The straight forward approach is to buy hundreds of virtual machines, but there are two problems with it:
It's too expensive.
Probably only a couple of them will be used but I'll have to pay for all of them.
What can you recommend?
Does Azure support it? Or maybe AWS?
First thing, cloud service provider work base on the region instead of a city like you mentioned new york etc but you can choose always nearest region to the country/city in which you want to run your application. you can also try cloudping or aws cloudping for nearest region.
In other words, I need to press the button and run the application for
a couple of hours on a server in New York, next press, and the same
application will be run in Stambul.
So I will recommend docker container as you want to run the same application in a different region so instead of mainain AMI better to go with the container.
AWS fargate is designing for pay as you go purpose along with zero server maintenance mean you just need to specify the docker image and run your application, rest AWS will take care of the resources.
AWS Fargate is a serverless compute engine for containers that works
with both Amazon Elastic Container Service (ECS) and Amazon Elastic
Kubernetes Service (EKS). Fargate makes it easy for you to focus on
building your applications. Fargate removes the need to provision and
manage servers, lets you specify and pay for resources per
application, and improves security through application isolation by
design.
like you mentioned
without paying for hundreds always running virtual machines.
So you do not need pay, you will only pay for the compute hours that used by your application when you start/run the container.
With AWS Fargate, there are no upfront payments and you only pay for
the resources that you use. You pay for the amount of vCPU and memory
resources consumed by your containerized applications.
AWS Fargate pricing
For deployment purpose, I will recommend terraform so you will only need to create resources for region and for the rest of the region you can make it parameterized.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 days ago.
Improve this question
I have a small mobile application which is based on microservice architecture. Two of the microservices are in java, one is in node. My application uses a MySQL database.
Presently I am using a VPS to host my services, all software (MySQL, tomcat and pm2) are installed in the same VPS.
Now I am planning to move to AWS and (as I have no prior AWS experience) I am overwhelmed by the services provided by AWS.
Can anyone please help me decide on this?
Since the usage is going to be very low at this point, I have to get this setup with the least monthly costs to be incurred.
For that I am thinking to get 1 EC2 instance and install all software in different docker containers (including the DB). Will this approach work? Or will I have to get another RDS instance? Is docker required? Or can I directly install all the software?
Such question is opinion based and community discourage such question but there is few things that are very clear to explain.
Will I have to get another RDS instance
I will not recommend to go with the container for DB, it will be hard to scale and maintain backup etc in container also there is a risk to lost data if no proper mounting were set in the container configuration.
I will recommend going with RDS free tier which is free for one year ( Term and condition apply)
It will be easy in future to upgrade, scale and maintain backup with RDS.
AWS Free Tier with Amazon RDS
750 hours of Amazon RDS Single-AZ db.t2.micro Instance usage running
MySQL, MariaDB, PostgreSQL, Oracle BYOL or SQL Server (running SQL
Server Express Edition) – enough hours to run a DB Instance
continuously each month
rds-free-tier
I am thinking to get 1 ec2 install and install all softwares in
different docker containers (including the DB),
At the initial level, it's fine to go with 1 instance. but here is the flow
Create ECS cluster
Create ECR registry and push your image to ECR
Create Task definition against each docker iamge
Create service for each task definition
As mentioned in the comment you can explore EKS, but in AWS I will prefer ECs.
You can explore this to start with gentle-introduction-to-how-aws-ecs-works-with-example
High-level look will be
It's been a while since I asked this question - and though I agree this is more of an opinion-based question - I still think this answer is something that would be a good start into cloud deployments.
Application hosting
For startups and small projects like these, the best approach would be to go with serverless lambda functions - and though it would add an overhead of lambda functions in code its worth the effort as it keeps the cost to almost zero until you start to get some tangible traffic.
Another approach for the application microservices could be docker - but docker containers are more for containerized deployment - to make sure code runs the same in the prod environment as it does in the dev- rather than this one should go with a small EC2 instance and deploy the microservices as separate processes (PM2 for node js). Though differential scaling would be tough at this point it doesn't matter - as soon as you start seeing CPU metrics touch the roof you can start decoupling the more used Microservices to another machine - and have a load balancer in front of it.
K8s is overkill at this point as again with one worker node even though the control plane is free to use - its just no point - until you have a sizable number of worker nodes
Database Deployment
Stateful deployments are trickier comparatively as there is a chance of data loss. Easier would be to go with managed DB hosting at this point such as AWS aurora/RDS or if you plan to use NoSQL then mongoDB atlas. Managing DB along with backups would be a painful task especially when you are saving every penny in infra costs.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 days ago.
Improve this question
We would like the following :
Dev Env will reside on AWS free tier account (both EC2 & RDS )
Once Dev env is ready and QA are finished, we will image this instance and open an additional 1 EC2 reserved instance.
Questions :
Can we benefit / use AWS free tier account for DEV and reserved instance for PROD at the same time, meaning we won't pay for Dev (1yr) and Prod we will pay for the 1 instance reserved price ?
If We would like to use PROD with reserved instance and activate autoscale (using on-demand or additional paid reserved instances) is it possible ?
To be honest I don't think it's worth the hassle as you can spin very cheap instances for development purposes. You can even automatically shut them down after office hours and on weekends when nobody is using them.
To address your questions:
It is possible to share an AMI with another account. See Sharing an AMI with Specific AWS Accounts
So once your instance has been approved on the dev account you can create an image, make it available to your prod account only, without making it public, and launch an instance based on that image.
According to AWS documentation it's possible to use auto-scaling with reserved instance benefits:
You can use Auto Scaling or other AWS services to launch the On-Demand instances that use your Reserved Instance benefits. For information about launching On-Demand instances, see Launch Your Instance.
Source: Reserved Instances
Let's look at each element individually...
Dev Env will reside on AWS free tier account (both EC2 & RDS )
The AWS Free Usage Tier provides free access to specific services within specific limits for the first 12 months of an AWS Account. In the case of Amazon EC2:
750 hours of Amazon EC2 Linux t2.micro instance usage (1 GiB of memory and 32-bit and 64-bit platform support) – enough hours to run continuously each month*
750 hours of Amazon EC2 Microsoft Windows Server t2.micro instance usage (1 GiB of memory and 32-bit and 64-bit platform support) – enough hours to run continuously each month*
A t2.micro instance in the USA is 1.3c per hour. So, the benefit gained from one instance (eg Linux) in the free tier is worth $113 (0.013*24*365).
For Amazon RDS, the value is approximately $150 (depending upon the chosen database):
750 hours of Amazon RDS Single-AZ db.t2.micro Instances,for running MySQL, PostgreSQL, MariaDB, Oracle BYOL or SQL Server (running SQL Server Express Edition) – enough hours to run a DB Instance continuously each month*
So, you really need to ask yourself whether doing strange things is really worth saving $263, given that it will be limiting you to micro-sized EC2 and RDS instances. A few hours of your time is presumably worth more than this, so it would be better for you to create infrastructure that is needed to successfully deliver your system rather than trying to focus on how to avoid spending money.
As an example, it is possible that these instance types might be too small for your development work and you will spend more time trying to get it to work (or waiting for it if it is slow), rather than productively spending your time on delivering a successful project.
Once Dev env is ready and QA are finished, we will image this instance and open an additional 1 EC2 reserved instance.
Reserved Instances are a great way to reduce your Amazon EC2 costs. However, you do not want to lock yourself into a particular instance type which might later prove inappropriate. For example, you might purchase a Reserved Instance for a medium instance, but later find that you actually need a large instance. This would mean your Reserved Instance is wasted.
It is much better to run your system for 2-3 months to understand your usage patterns before committing yourself to a Reserved Instance.
If We would like to use PROD with reserved instance and activate autoscale (using on-demand or additional paid reserved instances) is it possible ?
You can certainly use Auto Scaling to add additional instances. These additional instances would be charged at on-demand rates. If you purchase additional Reserved Instances, then you should run those instances all the time since you are already paying for them. There would be no benefit in reducing your number of instances to below the number of Reserved Instances you have purchased.
Bottom line: The cost of a couple of Amazon EC2 instances is not very high. Spend your time producing a great app rather than focussing on how you can reduce your expenditure by a couple of hundred dollars.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 days ago.
Improve this question
I am extremely new to Amazon Web Services. I would sincerely appreciate any help on finalising the architecture and arriving at a costing schedule. I am working on designing a AWS based solution for a dynamic website that we are designing. To begin with I need to have 2 High CPU Medium Utilisation EC2 instances (both will act as web servers), 1 High CPU Medium Utilisation EC2 instance for my database which will be Postgre SQL and 1 High CPU Medium Utilisation EC2 instance to serve as a read replica for my database. I will also be having a considerable volume of static content like images, videos or just .doc files for which I am contemplating using an S3 bucket. So, my website will be a dynamic + static kind of a website. I am expecting a rapid exponential scale up of users from 0 to say for example 1 million in a year. Hence I will need to scale up my EC2 combos (as described earlier) according to traffic. I am contemplatng using a CloudFormation stack for rapidly scaling my deployment. Also, to efficiently route traffic I will be using a single ELB to start with. Also, I would want to vertically partition my database based on user id's. For example, user ID 1 - 2000 on one EC2 database instance users 2001 to 4000 on second EC2 database instance etc... I will be auto scaling only the web server EC2 instances while my database EC2 instances will have a 100% uptime
My questions are:
What should be my auto scaling strategy for the web server EC2 instances and how do I know what will be the monthly costing when the scaling is so dynamic. I mean is there any way to predict so that I can do a cost break even analysis?
Do all the EC2 instances (web server and database) necessarily need an EBS backing or will Ephemeral storage suffice? I believe that for the database EC2 instances I will need an EBS backing. What about the web server Ec2 instances?
Suppose I end up scaling up to 100 EC2 instances. Will just one ELB suffice or do I need multiple ELBs?
How do I analyse as to how many HTTP requests can one High CPU Medium Utilisation EC2 instance handle before a breakdown?
Can CloudFront be used to host this kind of a dynamic + static site or is it used only for static sites?
Please help me with these questions as I have no clue on cloud solution architecting...
Thanks...
Vikram.
Here is a white paper we (RunSignUp) wrote about how we scaled on Amazon. Our use case was handling opening of an online race registration where 50,000 runners would want to sign up in less than 10 minutes. This shows how we configured everything including settings as well as some of the code we developed. We basically had the same issues as you and did not find anything that had a full use case of how to build a scalable app and how Amazon was used to scale with it. Hope it is helpful.
A quick attempt at an answer to get you started;
#1 is too big to answer without writing a book, it has too much to do with the system and its design to answer in a generic way.
#2 You'll definitely want backing for your database. The web servers on the other hand are probably better off with ephemeral storage, the more stateless/setup free they are, the easier to create/destroy instances dynamically.
#3 As far as I know, there's no upper limit on the number of machines "attached" to a single ELB. Under extreme load, you may experience slightly lesser latency if you split your site over multiple load balancers. You'll find more info that you can map against your actual architecture here.
#4 Only load testing of your system can tell you that.
#5 From the Cloudfront page;
Amazon CloudFront can be used to deliver your entire website, including dynamic, static and streaming content using a global network of edge locations.
In other words, its architecture does not prevent you from mixing static and dynamic content on your site.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
I am running a web-app on an AWS EC2 instance that uses EBS storage as its local drive. The web app runs on an Apache/Tomcat server, handles uploaded files in local storage and uses a MySQL database, all on this local drive. Does AWS guarantee the integrity and availability of EBS data or should I back it up to S3?
If so, how do I do that? I need to have daily incremental backups (i.e. I can only afford to loose recent transactions/files performed today).
Note: I am not worried about human caused errors (accidental deletes, etc.) rather system crashes, underlying service failure, etc.
Thanks..
Amazon does not guarantee the integrity of your EBS volumes, but they are very easy to back up. Simply take a daily snapshot (You could set up a cron using ec2-api-tools to do the daily snapshot).
EBS snapshots are stored in S3. They are not in your own bucket and the details are handled by Amazon, but the infrastructure that the snapshots are stored on is S3.
The snapshots are incremental, and back up the entire volume. Each snapshot stores the changes on the device since the last snapshot, so taking them often will reduce how long they take to make, but you can only have a limited number of snapshots at once per AWS account. I think it is 250. You need to delete your old snapshots eventually to deal with that. You could also do that with a cron job. Deleting old snapshots does not invalidate the newer ones even though they are stored as incremental, because it will actually update the next newest snapshot to contain the information from the previous one upon deletion.