I am currently signed up to the free tier of AWS. I am enjoying experimenting with various services including those not affording by said free tier. Can AWS's enhanced budgets be used to stop services like EC2 instances if I accidentally spend too much? Or do they merely act as alerts?
This is available for EC2, I don't think it is available for all of the AWS resources.
http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/UsingAlarmActions.html
Hope it helps.
There are several posts which looks it from different perspectives, such as this and this.
Having a cost cap might be a crucial requirement based on the usage, especially when considering how complex it is to set the things up properly and keeping everything secure on the cloud for an average user. At least we can expect to have a feature to switch on/off a cost-cap service, so a user can decide their own scenario easily.
Closest solution that I found is here:
Serverless Automated Cost Controls
https://aws.amazon.com/blogs/compute/serverless-automated-cost-controls-part1
It explains how to trigger AWS Lambda function to change IAM permission from EC2FullAccess to EC2ReadOnly when the budget exceeds the limit.
There is no built-in way to terminate services based on budgets or billing alarms.
You can get notified automatically, but it is then up to you to determine how to handle it.
Would you really want AWS automatically terminating your production infrastructure because you went $1 over your estimated monthly spending?
Edit: There is now a way to monitor and alert on free tier usage, and when your predicted usage will exceed the free tier. See here for details. You could probably come up with a way to terminate infrastructure based on an alert using SNS & lambda.
Edit 2: In Oct. 2020, AWS released Budget Actions - the ability to trigger an action when a budget thresholds are reached. This should give you the ability to automate a response - you can shut down servers, change IAM permissions to prevent additional infrastructure from being created, etc.
Recently, Amazon has given "budget action" to carry out actions like stop services automatically if the budget has exceeded.
https://aws.amazon.com/about-aws/whats-new/2020/10/announcing-aws-budgets-actions/
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/budgets-controls.html#:~:text=select%20Configure%20thresholds.-,To%20configure%20a%20budget%20action,-Under%20Configure%20thresholds
Related
Recently i discovered my bill raising without using anything above free tier of with very minor charges.
On the bill management page it was clear that the charges are coming from CloudWatch Alarms as you can see in the picture below.
My question is why and how can i stop them? I can see that the alarms are been created by DynamoDB
auto-scaling, but i can't continue being charged for such a simple thing, i'm sure there is an option to disable it but i can't figure out.
Edited I checked "hide auto-scaling alarms" box but i think it's not the fix, fingers crossed to be:P
This is part of DynamoDB AutoScaling. For a small project, you should consider using DynamoDB without Provisionned Throughput.
AWS Free Tier includes 10 Alarm metrics (not applicable to high-resolution alarms).
See How can I determine why I was charged for CloudWatch usage, and then how can I reduce future charges?
Because of business needs, I now need to monitor the service deduction amount of AWS. When the service deduction amount is less than $5000, I will give an alarm to relevant management personnel. Is there a way to realize this? I checked the documentation of AWS cost Explorer API on the official website, but I didn't see the API related to the service deduction amount.
Personally, I recommend AWS cloud watch and Grafana dashboard.
https://grafana.com/grafana/dashboards/139
You can use AWS Budgets to create an alert where your credits are at a certain threshold. You can use the advance filters to alert only on credits
You can also check https://aws.amazon.com/blogs/startups/how-to-set-aws-budget-when-paying-with-aws-credits/
I have newly created an API service that is going to be deployed as a pilot to a customer. It has been built with AWS API Gateway, AWS Lambda, and AWS S3. With a SaaS pricing model, what's the best way for me to monitor this customer's usage and cost? At the moment, I have made a unique API Gateway, Lambda function, and S3 bucket specific to this customer. Is there a good way to create a dashboard that allows me (and perhaps the customer) to detail this monitoring?
Additional question, what's the best way to streamline this process when expanding to multiple different customers? Each customer would have a unique API token — what's the better approach than the naive way of making unique AWS resources per customer?
I am new (a college student), but any insights/resources would help me a long way. Thanks.
Full disclosure: I work for Lumigo, a company that does exactly that.
Regarding your question,
As #gusto2 said, there are many tools that you can use, and the best tool depends on your specific requirements.
The main difference between the tools is the level of configuration that you need to apply.
cloudwatch default metrics - The first tool that you should use. This is an out-of-the-box solution that provides you many metrics on the services, such as: duration, number of invocations and errors, memory. You can configure metrics over different timeslots and aggregators (P99, average, max, etc.)
This tool is great for basic monitoring.
Its limitation is its greatest strength - it provides monitoring which is common to all the services, thus nothing tailored-fit to serverless applications. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/working_with_metrics.html
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html
cloudwatch custom metrics - The other side of the scale - getting much more precise metrics, which allows you to upload any metric data and monitor it: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html
This is a great tool if you know exactly what you want to monitor, and you already familiar with your architecture limitations and pain points.
And, of course, you can configure alarms over this data:
Lumigo - 3rd party company (again, as a disclosure, this is my workplace). Provides out-of-the-box monitoring, specifically created for serverless applications, such as an abnormal number of invocations, costs, etc.. This tool also provides troubleshooting capabilities to enable deeper observability.
Of course, there are more 3rd party tools that you can find online. All are great- just find the one that suits your requirement the best.
Is there a good way to create a dashboard
There a are multiple ways and options depending in your scaling, amount of data and requirements. So you could start small and simple, but check if any option is feasible or not.
You can start with the CloudWatch. You can monitor basic metrics, create dashboards and even share with other accounts.
naive way of making unique AWS resources per customer
For the start I would consider creating custom cloudwatch metrics with the customer id as a metric and put the metrics from the Lambda functions.
Looks simple, but you should do the math and a PoC about the number of requested datapoints and the dashboards to prevent a nasty surprise on the billing.
Another option is sending metrics/events to DynamoDB, using atomic functions you could directly build some basic aggregations (kind of naïve stream processing).
When scaling to a lot of events, clients, maybe you will need some serious api analytics, but that may be a different topic.
We are using AWS Organisations service (with Consolidated Billing feature) with a large member of accounts. Can we set a max limit somehow for the usage cost ($) of the Member accounts?
Would you add a little more detailed description about how can I hook an activity on a billing alert?
I am not sure about your exact requirements, you may check the documentation:
Monitor Your AWS Charges with Billing Alerts.
Managing Your Costs with Budgets
In both cases you can setup alerts on cost/budget, where an alert event is sent to an SNS topic. SNS topic can further send a mail, but the same way you can implement any functionality listening on the topic (e.g. a lambda function)
This way you may monitor total cost/budget usage
usage cost ($) of the Member accounts
You can have a look at Cost Explorer where you can create cost reports per tag or as well per account
(I am not sure if you can create an alert for cost per account)
There is no out-of-the-box enforcement to limit the usage of your AWS account. Alerts can only notify you that you're not in your forecasted budget, but that's it mostly, the rest you should implement on your own.
Couple of thoughts:
Enforcing max limit on your production account is not best practice. You don't want a production interruption or user impact because of a needed scale up in infra.
What you can do is to map your needs and usage, and set quota limits to services. https://docs.aws.amazon.com/general/latest/gr/aws_service_limits.html - again, this is not really recommended. Just make sure you have visibility on your growth, set alerts and monitor your usage. Optimization is a key in the could, and the financial responsibility is yours, not AWS.
I was attracted by the AWS free tier to give EC2/S3 a try. However, one thing I'm worried about is the payment process. There's quite a few management menus and it doesn't seem entirely transparent when I would break the free usage tier (or if I decide to pay, when I break that usage tier).
You can download .csv usage reports, but I wish the billing/usage monitoring was a little more interactive so I don't get unpleasantly surprised. Does anyone have experiences EC2, is there some aspect of the management interface that makes this a easier/less worrisome?
You can monitor your AWS resource usage and the resulting fees here:
AWS Account Activity
https://aws-portal.amazon.com/gp/aws/developer/account/
You can see how current the report is at the top. In my experience it lags by a few hours, which is pretty amazing if you think of how many different customers AWS has and how many little things they have to keep track of to calculate your fees (e.g., every disk I/O request and network byte sent).
Click "Expand All Services" to see the usage/fees broken down even more.
Note: You don't "decide to pay". You already gave AWS your credit card and agreed to pay according to their fee structure. If your resource usage goes over the free tier, AWS will automatically charge your credit card at the end of the month. Monitor the above page regularly to make sure your charges are accumulating as expected.
Use AWS Billing Alerts to notify when you exceeds the fee tier,
If you currently use the AWS Free Tier, you can set a billing alert to notify you if you exceed the free tier by setting a threshold of $0.00.
refer to,
AWS Billing Alerts