I get into AWS Cost Explorer, then into service EC2, and immediately see the cost per day of EC2 instances, and that is great.
However I need to monitor storage costs (EBS) separately, to compare allocation strategies with respect to cost, and can't find a way.
Best regards
In the filter list on the right hand side, you can filter on "Usage Type Group". There are seven usage type groups for EBS: you can check all of them and see only costs related to EBS.
Related
I need to make a cost model for AWS vs GCP. Currently, our organization is using AWS. Our biggest services used are:
EC2
RDS
Labda
AWS Gateway
S3
Elasticache
Cloudfront
Kinesis
I have very limited knowledge of cloud platforms. However, I have access to:
AWS Simple Monthly Calculator
Google Cloud Platform Pricing Calculator
MAP AWS services to GCP products
I also have access to CloudHealth so that I can get a breakdown of costs per services within our organization.
Of the 8 major services listed above are main usage and costs go to EC2, S3, and RDS.
Our director of engineering mentioned that I should be most concerned with vCPU and memory.
I would appreciate any insight (big or small) that people have into how I can go about creating this model, any other factors I should consider, which functionalities of the two providers for the services are considered historically "better" or cheaper, etc.
Thanks in advance, and any questions people may have, I am more than happy to answer.
-M
You should certainly cost-optimize your resources. It's so easy to create cloud resources that people don't always think about turning things off or right-sizing them.
Looking at your Top 5...
Amazon EC2
The simplest way to save money with Amazon EC2 is to turn off unused resources. You can even stop instances overnight and on the weekend. If they are only used 8 hours per workday, then that is only 40 out of 168 hours, so you can save 75% by turning them off when unused! For example, Dev and Test instances. People have written various types of automated utilities to turn instances on and off based on tags. Try search the Internet for AWS Stopinator.
Another way to save money on Amazon EC2 is to use spot instances. They are a fraction of the price, but have a risk that they might be turned off when demand increases. They are great where it is okay for systems to be terminated sometimes, such as automated testing systems. They are also a great way to supplement existing capacity at a fraction of the price.
If you definitely need the Amazon EC2 instances to keep running all the time, purchase Amazon EC2 Reserved Instances, which also offer a price saving.
Chat with your AWS Account Manager for help with the above options.
Amazon Relational Database Service (RDS)
Again, Amazon RDS instances can be stopped overnight/on weekends and turned on again when needed. You only pay while the instance is running (plus storage costs).
Examine the CloudWatch metrics for your RDS instances and determine whether they can be downsized without impacting applications. You can even resize them when they are used less (eg over weekends). Everything can be scripted, so you could trigger such downsizing and upsizing on a schedule.
Also look at the Engine used with RDS. Commercial offerings such as Oracle and Microsoft SQL Server are more expensive than open-source offerings like MySQL and PostgreSQL. Yes, your applications might need some changes, but the cost savings can be significant.
AWS Lambda
It is most unusual that Lambda is #3 in your list. In fact, some customers never get a charge for Lambda because it falls in the monthly free usage tier. Having high charges means you're making good use of Lambda (which is saving you EC2 costs), but take a look at which applications are using it the most and see whether they are using it wisely.
When correctly used, a Lambda function should only ever run for a few seconds, so check whether any application seem to be using it outside this pattern.
AWS API Gateway
Once again, these costs tend to be low ($3.50/million calls) so again I'd recommend trying to figure out how this is being used. If you really need that many calls, it would also explain the high Lambda costs. It would probably be more expensive if you were providing such functionality via Amazon EC2.
Amazon S3
Consider using different Storage Classes to reduce your costs. Costs can be reduced by:
Moving infrequently-accessed data to a different storage class
Moving data to One-Zone (if you have a copy of the data elsewhere, so don't need the redundancy)
Archiving infrequently-accessed data to Amazon Glacier, which offers much cheaper storage but does not have instant access
With GCP, you can benefit by receiving discounts such as the Committed Use Discount and the Sustained Use Discount.
With a Committed Use Discount, you can receive a discount of up to 70% if your usage is predictable.
With the Sustained Use Discount, there is an incremental discount if you reach certain usage thresholds.
On your concern with vCPU and memory, you may use predefined machine types. They are cheaper than custom machine types.
Lastly, you can also test the charges by trying out the Google Cloud Platform Free Tier.
My team got billing for AWS EBS and we have no idea what is it about.
Going to our console and try to open EBS section though we failed to get one as the snapshots below.
So my question is how to get breakdown details on AWS EBS usage billing?
p.s.
The EBS usage cost is viewed under EC2 section in billing tab in the management console
EBS is under EC2 in the AWS Management Console. If you look at the size of your volumes, the type of volumes and also the size of your snapshots that should help identify the cost.
E.g. faster volumes types, provisioned IOPS, large volumes/snapshots are more expensive.
You can put this information into the AWS Simple Montly Calculator to work out the cost https://calculator.s3.amazonaws.com/index.html
You can also look at the usage in CloudWatch, however it's under EBS there and shows per volume statistics.
For AWS billing you can use cost explorer for generating handsome reports.
Billing Break-Down:
In aws they have grouped multiple offerings under one service. For ex: EBS, ELB, EIP all grouped under EC2.
Same when they provide billing breakdown they provide it on service basis then region basis and then offering.
If you want better breakdown best way is to use cost tags and cost explorer.
Like in your case you can give your service_tag = ebs and application_tag = app1.
Then by using cost explorer : filter_by = app_tag and region and etc. and group_by = service_tag. By performing multiple combinations you can get a clear view of your cost and usage.
I have reservation of type t2.micro and us-ease-1d availability zone, also I have multiple instances running, including the one with mentioned type and zone.
As a result I expect billing for this one instance will take into account the reservation. The problem is that I haven't found any link between reservation and actual instances.
I've used aws ec2 describe-instances and aws ec2 describe-reserved-instances CLI commands but I wasn't able to find any link.
Is it possible to realise which billing approach will be used for each instance using Amazon SDK?
So f.e. I will see that some instance is linked to some reserved instance (reservation)
Is it possible to realise which billing approach will be used for each instance using Amazon SDK?
No, it isn't.
The "link" between reserved instances and running instances is not something the EC2 operational infrastructure knows about. It's all done after the fact, in billing.
Each hour, for a given instance type and availability zone placement, you're billed for your reserved instances (depending on the terms of the reservation, this happens whether you have this many instances running or not, though in some cases, the amount billed here is $0 since you've already paid). Then, if the number of running on-demand instances for that type and placement exceeds exceeds the number of reserved instances for that type and placement, the difference for that hour is billed at the on-demand rate.
So if you had purchased one reserved instance matching a certain spec and during a given hour you had two such instances running, it's not really the case that one of your instances "is the reserved instance" and the other one isn't. If you stop either one, then the next hour, the reserved instance pricing applies to the one that remains running... but EC2 can't tell you which is which and, in fact, the billing logic is such that it doesn't matter.
There isn't really a link between reservations and specific instances. Think of it more like a discount that gets applied to your bill, after you have incurred some instance charges.
You can use the Reserved Instance Utilization Report to see how your reservations have been applied to the instance hours you have been charged for.
I have two ec2 instances running in aws which are currently stopped (I am using the free tier just to experiment with Azure). I noticed that even though the instances are in the stopped state I seem to be incurring a charge for (this is all I have)
S3 - Puts (This contained the sample applications which I uploaded to test)
EBS - Volumes
Is this the case? or am I missing something here.
It depends on how big the EC2 instances and their storage are.
Stopped instances themselves don't cost any further money, but EBS storage, S3 and other moving parts like ELB's will still cause charges.
You get 30GB of EBS storage free per month in the free tier, so if the total of the EBS in use is more than that, you pay.
https://aws.amazon.com/free/ nicely describes the limits of the free tier.
For further cost calculations, check out https://calculator.s3.amazonaws.com/index.html
I want to store around 10 GB of data on AWS services and use map reduce to process the data.
Is using EC2 the best option ? I want to use free tier service, it says maximum of 613 MB for free services on EC2 and that does not satisfy my requirement. I am doing a hobby project and my expenses are limited.
The free tier FAQ also talks about using AWS EBS with free 30 GB of data. Can I use Map Reduce services on EBS too, since AFAIK EMR is only available on EC2 ?
Does anyone know of any other alternatives that I can use for the same ?
Try the AWS Simply Monthly Calculator, located at http://calculator.s3.amazonaws.com/calc5.html#s=EMR, to get a feel for how much your project will cost using AWS.
The recommended workflow for EMR is to store data in an S3 bucket. So in the calculator, click S3 on the left. In the form enter 10G. The price for S3 storage is about $0.10 per gb/mo, so 10G costs about $1.00/mo.
Then, click on Amazon Elastic Map Reduce on the left. The form allows you to select predicted number of instances, hours/week or hours/mo expected usage, and expected instance type needed for your project. For example, for a project that requires 20 hrs/week using 1 Small EC2 is estimated to cost around $6.00. Micro instances do not seem to be offered with EMR.
Therefore, if you think you can get by with a Small Instance, and you plan to use it infrequently, your expenses might be under $10 per mo.
To reduce expenses even further, you could use spot instances rather than standard instances, as explained here: http://aws.amazon.com/ec2/spot-instances/#7.