I need to run few php scripts twice a day, and script execution won't last more than 60s per script.
AWS EB is working really fine to do the job, but I don't see any reason to pay for an EC2 instance running all day if I can pay half an hour a day.
Is there any solution to schedule application deployment, script execution and terminate application ?
It depends on what you want to pay for. Elastic Beanstalk provides deployment and management tools to run your application code, rather than just a plain instance. How valuable is that to you?
If you can figure out your own deployment pipeline, you might look at setting up an Autoscaling group with scheduled scaling actions.
While Elastic Beanstalk environments can also do scheduled scaling actions, I believe it only does so with load-balanced environments, where you would have to pay for the load balancer (~$20/month).
Running a t2-micro instance in a single-instance EB environment without a load balancer is only ~$10/month. You could spend even less on an annual basis with reserved instances. If the alternative is spending time designing, configuring, deploying, and managing an alternate solution, you could do a lot worse. I certainly recommend starting with EB.
If you want to run scripts, I would look at the Lambda serverless model. You only pay for activity which means idle code won't cost you.
Related
I cannot seem to find any evidence that supports whether this is a true statement or not. I want to know if AWS Beanstalk deploys are faster with more expensive instance types. Any thoughts are welcomed! My assumption is that it shouldn't matter, but my deploys are incredibly slow and I want to see how this can be improved.
The underlying EC2 instance definitely matters when it comes to the speed of deployment. Elastic Beanstalk is just an API service that receives and processes API calls, and provisions the underlying infrastructure. However, all the installation and configurations are done on the instances at the end of the day. Beanstalk runs a bunch of wrapper scripts in addition to user data on the instance and this is where the compute power comes into play. Moreover, some instances spin up faster than the others.
Having said that, this is definitely not the only factor. Your source bundle size, what region you are deploying to and a few other factors also play a role.
This really wasn't clear for me in the Docs. And the console configuration is very confusing.
Will a Docker Cluster running in Fargate mode behind a Load Balancer shutdown and not charge me while it's not being used?
What about cold starts? Do I need to care about this in Fargate like in Lambda?
Is it less horizontal than Lambda? A lambda hooked to API Gateway will spawn a new function for every concurrent request, will Fargate do this too? Or will the load balancer decide it?
I've been running Flask/Django applications in Lambda for some time (Using Serverless/Zappa), are there any benefits in migrating them to Fargate?
It seems to be that it is more expensive than Lambda but if the Lambda limitations are not a problem then Lambda should always be the better choice right?
Will a Docker Cluster running in Fargate mode behind a Load Balancer shutdown and not charge me while it's not being used?
This will depend on how you configure your AutoScaling Group. If you allow it to scale down to 0 then yes.
What about cold starts? Do I need to care about this in Fargate like in Lambda?
Some good research has been done on this here: https://blog.cribl.io/2018/05/29/analyzing-aws-fargate/
But the takeaway is for smaller instances you shouldnt notice any more and ~40seconds time to get to a running state. For bigger ones this will take longer.
Is it less horizontal than Lambda? A lambda hooked to API Gateway will spawn a new function for every concurrent request, will Fargate do this too? Or will the load balancer decide it?
ECS will not create a new instance for every concurrent request,any scaling will be done off the AutoScaling group. The load balancer doesnt have any control over scaling, it will exclusively just balance load. However the metrics which it can give can be used to help determine if scaling is needed
I've been running Flask/Django applications in Lambda for some time (Using Serverless/Zappa), are there any benefits in migrating them to Fargate?
I havent used Flask or Django, but the main reason people tend to migrate over to serverless is to remove the need to maintain the scaling of servers, this inc managing instance types, cluster scheduling, optimizing cluster utilization
#abdullahkhawer i agree to his view on sticking to lambdas. Unless you require something to always be running and always being used 99% of the time lambdas will be cheaper than running a VM.
For a pricing example
1 t2.medium on demand EC2 instance = ~$36/month
2 Million invocations of a 256MB 3 second running lambda = $0.42/month
With AWS Fargate, you pay only for the amount of vCPU and memory resources that your containerized application requests from the time your container images are pulled until the AWS ECS Task (running in Fargate mode) terminates. A minimum charge of 1 minute applies. So, you pay until your Task (a group of containers) is running, more like AWS EC2 but on a per-minute basis and unlike AWS Lambda where you pay per request/invocation.
AWS Fargate doesn't spawn containers on every request as in AWS Lambda. AWS Fargate works by simply running containers on a fleet of AWS EC2 instances internally managed by AWS.
AWS Fargate now supports the ability to run tasks on a scheduled basis and in response to AWS CloudWatch Events. This makes it easier to launch and stop container services that you need to run only at a certain time to save money.
Keeping in mind your use case, if your applications are not making any problems in the production environment due to any AWS Lambda limitations then AWS Lambda is the better choice. If the AWS Lambda is being invoked too much (e.g., more than 1K concurrent invocations at every point of time) in the production environment, then go for AWS EKS or AWS Fargate as AWS Lambda might cost you more.
I am using an Amazon EC2 instance with instance type m3.medium and an Amazon RDS database instance.
In my working hours the website goes down because CPU utilization reaches 100%, and at night (not working hours) the CPU utilization is 60%.
So please give me right solution for this site down issue. I am not sure why I am experiencing this problem.
Once I had set a cron job for every minutes, but I was removed it because of slow down issue, but still I have site down issue.
When i try to use "top" command, i had shows below images for cpu usage, in which httpd command consume more cpu usage, so any suggestion for settings to reduce cpu usage with httpd command
Without website use by any user below two images:
http://screencast.com/t/1jV98WqhCLvV
http://screencast.com/t/PbXF5EYI
After website access simultaneously 5 users
http://screencast.com/t/QZgZsiNgdCUl
If you are CPU Utilization is reaching 100% you have two options.
Increase your EC2 Instance Type to large.
Use AutoScaling to launch one more EC2 Instance of same Instance Type.
Looks like you need some scheduled actions as you donot need 100% CPU Utilization during non-working hours.
The best possible option is to use AWS AutoScaling with Scheduled actions.
http://docs.aws.amazon.com/autoscaling/latest/userguide/schedule_time.html
AWS AutoScaling can launch new EC2 instances based on your CPU Utilization (or other metrics like Network Load, Disk read/write etc). This way you can always keep your site alive.
Using the AutoScaling scheduled actions you can specify metrics such that you stop your autoscaled instances during non-working hours and autoscale instances during working hours according to CPU Utilization(or other metrics).
You can even stop your severs if you donot need them at some point of time.
If you are not familiar with AWS AutoScaling you can follow the Documentation which is very precise and easy.
http://docs.aws.amazon.com/autoscaling/latest/userguide/GettingStartedTutorial.html
If the cpu utilization reach 100% bacause of the number of visitors your site have, you must consider to change the instance type, Auto Scaling or AWS CloudFront in order to cache as many http requests as posible (static and dynamic content).
If visitors are not the problem and there are other scheduled tasks on the EC2 isntance, I strongly recomend to decouple these workload via AWS SQS & AWS Elasticbeanstalk - Worker type
In other words I would like to temporarily turn off an environment (and its associated billing costs) but not delete it entirely.
It seems if I set the [Configuration > Web Tier > Scaling > Minimum instance count] to 0 along with the related "Maximum instance count", AWS rejects those settings as invalid. Ditto for questionable values like 0.1.
Any ideas for temporarily taking an Elastic Beanstalk environment out of service?
You can achieve this by defining time periods in your environment.
from your Elastic Load blanancer dashboard navigate through: Configuration -> Capacity -> Time-based scaling -> Scheduled actions.
You may choose to use multiple scheduled actions to start and stop your environment.
i.e. One to start your environment and another to stop it, or scale out during high load periods and back in after.
In the background, Elastic Beanstalk will terminate and recreate the EC2. This means you'll lose any data on the instances local storage.
I have used this to turn my environments off when they are no longer needed. For example, I have a workload that crawls looking for new content; and I pause this crawling after it has once completed. I also use this feature to spin down my nonproduction environments outside of working hours.
I prefer this method to others as it is supported by both single instance and load-balanced environments.
How about eb scale 0 using the Elastic Beanstalk CLI tools?
I don't believe this is possible. I any case, you would still have a load balancer in front for which you pay, so you're not really reducing the costs for this environment.
It makes more sense to delete the environment completely create a script that will deploy a version from your application versions to a new environment.
I now use the ParkMyCloud service to suspend Elastic Beanstalk environments (and also RDS database instances) that I'm not using, both manually and in scheduled fashion.
Although ParkMyCloud is non-free, it's saved me a lot more money than I've spent using it. It's worked quite well over the last few years so I'm comfortable recommending it.
What is the best way to deal with traffic spikes on elastic beanstalk? In my experience this does not seem to scale quickly enough i.e. the new instances take a few minutes to get going.
Should I be doing some more calculations to optimise the scaling process?
Is there a formula for working these thing out?
Yeah, it takes 5-10 minutes (depending on the stack you're using; not counting Windows instances) to launch a new Beanstalk instance via CloudFormation, install and configure the environment software, add the instance to the load balanced cluster, deploy your application code, and run any of your .ebextensions. (All of which you can follow along with by watching the event log for the environment, or the log for the CloudFormation template executing in the background.)
There are a couple of ways to handle this:
Use larger instances that won't need to scale-out so quickly.
Tweak your Auto Scaling triggers for your environment (via the AWS Console, web service API, or CLI tools) so that scale-up happens sooner. That way you'll have the extra capacity by the time the existing servers get maxed-out.