How to monitor daily costs of Amazon S3 - amazon-web-services

I was looking around to learn about Do and Don'ts of Amazon S3 and was looking at options on how to monitor usage daily. I have couple of questions here:
I see on the Amazon S3 pricing page that PUT, COPY, POST, or LIST Requests are $0.005 per 1,000 requests
I have 5 buckets in S3 and each of these buckets have sub folders having 100k files.
If I do aws s3 ls --recursive will I be charged
5*100k=500000/1000 = 500*0.005 = $2.50?
Any suggestions on tools that can be used to email daily usage rate of my S3 bucket?

A LIST request can return up to 1000 objects. Therefore, to obtain the full list of 100k objects, it would require 100 LIST requests. That would consume a tenth of the initial $0.005 charge.
If you were to repeat this for 5 buckets, it would require 500 LIST requests, which would consume of half of the initial $0.005 charge.
Alternatively you could use Amazon S3 Storage Inventory, which can provide a daily/weekly CSV file that contains a list of all your objects. It is charged at $0.0025 per million objects listed (per bucket per run).
I sense that you are worried about your Amazon S3 costs. You can configure billing alerts and notifications to remain aware of when your costs rise above expected levels. This works for all AWS services.
See:
Monitoring Charges with Alerts and Notifications
Create a Billing Alarm to Monitor Your Estimated AWS Charges

Related

How can I pin-point the source of AWS S3 costs?

I'm currently surprised by a pretty high daily AWS S3 cost of over 31 USD per day (I expected 9 USD - 12 USD per month):
I'm using eu-central-1
All buckets combined are less than 400 GB
No replication
The best explanation I have is that the number of requests was way higher than expected. But I don't know how I can confirm this. How can I narrow down the source of AWS S3 cost?
Is it possible to see the costs by bucket?
Is it possible to see a breakdown by storage / requests / transfers / other features like replication?
First pay attention to the factors on which AWS S3 charges - i.e. based on storage, how many requests s3 is getting, data transfer and retrieval.
Some of the ways for cutting and keep track on the cost -
Delete the previous version of the buckets if you don't need that.
Move the data to different s3 storage based on frequency of data retrieval.
activate the cost allocation tags on your buckets so that you can review the cost on individual bucket.
create an S3 Storage Lens dashboard for all the buckets in your account.

S3 and large files backup

I have large files and I am looking for a place where I could store files (databases dumps). AWS S3 is good for backups? I have already exceeded all limits.
I have a few questions:
I am using API and CLI. Which solution is cheaper to send files via API? "aws s3api put-object" or "aws s3 cp"?
"2,000 Put, Copy, Post or List Requests of Amazon S3". How is consumption calculated? In HTTP requests or bytes? Ac Currently, Currently, I have level of consumption for 20 files per day: 2,000.00/2,000 Requests.
Are there any paid plans?
Everything you need to know is at the Request Pricing section of the S3 Pricing page.
Amazon S3 request costs are based on the request type, and are charged
on the quantity of requests or the volume of data retrieved as listed
in the table below. When you use the Amazon S3 console to browse your
storage, you incur charges for GET, LIST, and other requests that are
made to facilitate browsing. Charges are accrued at the same rate as
requests that are made using the API/SDK.
Specific pricing is available at that page (not included here because it will change over time).

Am I charged for creating and keeping AWS S3 buckets?

I created two buckets back in 2018 but never removed them. Rest assured that they are empty buckets with no files at all.
I didn't see any fee charged, so I presume Amazon/AWS didn't charge for simply creating S3 buckets?
S3 pricing is based on object storage, not buckets. You can read more about S3 pricing on the AWS S3 pricing page here.
Well, even though you're not charged for the bucket itself, you can still incur some charges related to them.
There are six Amazon S3 cost components to consider when storing and managing your data—storage pricing, request and data retrieval pricing, data transfer and transfer acceleration pricing, data management and analytics pricing, replication pricing, and the price to process your data with S3 Object Lambda. For my details about the pricing model, check it out here

Cloudwatch log store costing vs S3 costing

I have an ec2 instance which is running apache application.
I have to store my apache log somewhere. For this, I have used two approaches:
Cloudwatch Agent to push logs to cloudwatch
CronJob to push log file to s3
I have used both of the methods. Both methods suit fine for me. But, here I am little worried about the costing.
Which of these will have minimum cost?
S3 Pricing is basically is based upon three factors:
The amount of storage.
The amount of data transferred every month.
The number of requests made monthly.
The cost for data transfer between S3 and AWS resources within the same region is zero.
According to Cloudwatch pricing for logs :
All log types. There is no Data Transfer IN charge for any of CloudWatch.Data Transfer OUT from CloudWatch Logs is priced.
Pricing details for Cloudwatch logs:
Collect (Data Ingestion) :$0.50/GB
Store (Archival) :$0.03/GB
Analyze (Logs Insights queries) :$0.005/GB of data scanned
Refer CloudWatch pricing for more details.
Similarly, according to AWS, S3 pricing differs region wise.
e.g For N.Virginia :
S3 Standard Storage
First 50 TB / Month :$0.023 per GB
Next 450 TB / Month :$0.022 per GB
Over 500 TB / Month :$0.021 per GB
Refer S3 pricing for more details.
Hence, we can conclude that sending logs to S3 will be more cost effective than sending them to CloudWatch.
They both have similar storage costs, but CloudWatch Logs has an additional ingest charge.
Therefore, it would be lower cost to send straight to Amazon S3.
See: Amazon CloudWatch Pricing – Amazon Web Services (AWS)

Number of Buckets in Amazon Free Tier

I am trying out Amazon S3 for my file uploads and would like to store different buckets for development, test and production environments. In amazon documentation it was mentioned the following statement
As part of the AWS Free Usage Tier,
you can get started with Amazon S3 for
free. Upon sign-up, new AWS customers
receive 5 GB of Amazon S3 storage,
20,000 Get Requests, 2,000 Put
Requests, 15GB of data transfer in,
and 15GB of data transfer out each
month for one year.
Is there any limitation about the number of buckets. I mean if I have three buckets and If I use within the overall storage limit, will I be charged.
Each account in AWS is limited to 100 buckets -- even if you are paying the normal usage rates.
Buckets are not billable items in S3.
If the limit of 100 is not enough you can create virtual folders in your buckets and structure your environment that way.