Because of business needs, I now need to monitor the service deduction amount of AWS. When the service deduction amount is less than $5000, I will give an alarm to relevant management personnel. Is there a way to realize this? I checked the documentation of AWS cost Explorer API on the official website, but I didn't see the API related to the service deduction amount.
Personally, I recommend AWS cloud watch and Grafana dashboard.
https://grafana.com/grafana/dashboards/139
You can use AWS Budgets to create an alert where your credits are at a certain threshold. You can use the advance filters to alert only on credits
You can also check https://aws.amazon.com/blogs/startups/how-to-set-aws-budget-when-paying-with-aws-credits/
Related
I am working on a project and all the project resource is on 'me-south-1'
I have other resources things in other regions.
I need to send a detailed bill to the client.
could anyone suggest me,How can I filter it according to the region?
For detailed billing (with the ability to filter) the best approach is to use Cost Explorer.
By using this service you can apply a range of filters (including region), this can also be done programmatically.
Be aware that using this service does charge $0.01 per request.
My cloudwatch bill has a $400 spike from August to September.
$0.01 per 1,000 requests(blended rate: 0.000010)161,204,177 Requests
$0.01 per 1,000 requests(blended rate: 0.000010)206,899,565 Requests
how can we check why requests have increased so much and determine their end points
or where the requests are coming from
This is likely a monitoring tool, DataDog is one example, but any tool that queries AWS continually to get stats on all of your servers. Are you importing your data to any third party tools, or internally built tools?
CloudTrail does NOT log this request, per the answer below from AWS to a similar question - This would be my answer too in general, this is a special case :-(
"Unfortunately, there is no tool from your side to check the source of GetMetricData. The reason being, this API call is not logged in CloudTrail." -- akash-aws from forums
Additional Info: https://forums.aws.amazon.com/thread.jspa?messageID=865061
Good luck finding the cause, I'm looking into a similar problem for a client and tracked down when it started, now to hopefully find more details.
See if you're getting billed for GetMetricData requests. If yes, see if you're running any 3rd party monitoring solutions that get metrics from your AWS account. Finally, to confirm it's caused by the 3rd party monitoring tool, turn off the tool if you're allowed to, at least for an hour or so, to see if there's indeed a dent in your bill at that hour.
I wrote detailed steps here: https://ystoneman.medium.com/a-hidden-cost-to-monitoring-aws-with-3rd-party-tools-e23dd9e38c75
You can Log Amazon CloudWatch API Calls with AWS CloudTrail - Amazon CloudWatch:
Amazon CloudWatch is integrated with AWS CloudTrail, a service that provides a record of actions taken by a user, role, or an AWS service in CloudWatch. CloudTrail captures API calls made by or on behalf of your AWS account. The calls captured include calls from the CloudWatch console and code calls to the CloudWatch API operations.
If you create a trail, you can enable continuous delivery of CloudTrail events to an Amazon S3 bucket, including events for CloudWatch. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in Event history.
Using the information collected by CloudTrail, you can determine the request that was made to CloudWatch, the IP address from which the request was made, who made the request, when it was made, and additional details.
I am currently signed up to the free tier of AWS. I am enjoying experimenting with various services including those not affording by said free tier. Can AWS's enhanced budgets be used to stop services like EC2 instances if I accidentally spend too much? Or do they merely act as alerts?
This is available for EC2, I don't think it is available for all of the AWS resources.
http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/UsingAlarmActions.html
Hope it helps.
There are several posts which looks it from different perspectives, such as this and this.
Having a cost cap might be a crucial requirement based on the usage, especially when considering how complex it is to set the things up properly and keeping everything secure on the cloud for an average user. At least we can expect to have a feature to switch on/off a cost-cap service, so a user can decide their own scenario easily.
Closest solution that I found is here:
Serverless Automated Cost Controls
https://aws.amazon.com/blogs/compute/serverless-automated-cost-controls-part1
It explains how to trigger AWS Lambda function to change IAM permission from EC2FullAccess to EC2ReadOnly when the budget exceeds the limit.
There is no built-in way to terminate services based on budgets or billing alarms.
You can get notified automatically, but it is then up to you to determine how to handle it.
Would you really want AWS automatically terminating your production infrastructure because you went $1 over your estimated monthly spending?
Edit: There is now a way to monitor and alert on free tier usage, and when your predicted usage will exceed the free tier. See here for details. You could probably come up with a way to terminate infrastructure based on an alert using SNS & lambda.
Edit 2: In Oct. 2020, AWS released Budget Actions - the ability to trigger an action when a budget thresholds are reached. This should give you the ability to automate a response - you can shut down servers, change IAM permissions to prevent additional infrastructure from being created, etc.
Recently, Amazon has given "budget action" to carry out actions like stop services automatically if the budget has exceeded.
https://aws.amazon.com/about-aws/whats-new/2020/10/announcing-aws-budgets-actions/
https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/budgets-controls.html#:~:text=select%20Configure%20thresholds.-,To%20configure%20a%20budget%20action,-Under%20Configure%20thresholds
It looks like CloudWatch gives customers 10 custom metrics under the free plan, then each additional one costs $0.50. Does anyone know how to enforce PutMetric accept only a set of custom metrics?
I'm interested in limiting the custom metrics coming from mobile clients or possibly adding a layer of protection against abuse.
Is the only solution to implement my own service which does the validation against a whitelist?
One option you could look at is placing AWS Gateway in front of Cloudwatch and making the calls through the api.
This example shows you how to do this for S3, but there's not reason why you couldn't do something similar for Cloudwatch.
This shows you how to do it for dynamo: https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-as-a-proxy-for-dynamodb/
I ended up running a simple tomcat service which validates metrics against a whitelist (stored in s3) and publishes them to CloudWatch.
what is the maximum number of rules for cloud watch I can create on my AWS account. I might have a lot of different rules that will invoke lambda function on schedule. Is it unlimited?
The basic limits are documented at http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_limits.html - currently 50 rules per account.
If you need more, reach out through your AWS contact and these can be expanded.
This is no longer 50 and has been increased to 100 per region per account.
As per this link:
https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/cloudwatch_limits_cwe.html
And as mentioned by johnny: this can be increased further on request (if amazon approves the request).
After talking to AWS cloud watch team I found out that the rule limit can be increased as per your need.
If you're willing to use a non-AWS service, then you might check out Microsoft Azure. Azure offers a great job scheduler that doesn't pose any limits. You could use this service to invoke your lambda functions.