Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 months ago.
Improve this question
I want to store audit data of some events happening within my API service.
I am planning to have 3 columns for the audit table.
The daily record/rows addition to this audit table won't exceed more than 100.
This audit table will be rarely accessed and will have very little write operation per day.
I want to minimize the cost, and I think that DynamoDB would be overkill here.
Is there any other choice of storage with RDS, or some other AWS storage service, that I can use to achieve my goal.
I would be writing the data to the audit table through a lambda service.
What makes you think DynamoDB would be overkill? It scales down as well as it scales up. In fact, when your activity rate is low that’s a great time to pick a serverless model because you don’t have to pay except for what you use.
Your usage would even fit within the free tier.
If you have a workload that's about 99% writes and 1% reads and doesn't have extreme read performance requirements as tends to be the case with audit workloads, I suggest you rethink your approach to storing this data.
Consider having the Lambda function(s) write the audit events to a Kinesis Firehose Stream. Kinesis Firehose can aggregate these records in JSON/Parquet files and store them on S3. When you need to access that data, you can use Athena to query the data in S3.
This should be significantly cheaper at scale than having a database around that's not going to be queried most of the time.
Pricing:
Kinesis Data Firehose
Athena
S3
Note: DynamoDB has a free tier of 25GB in storage costs per month - depending on what's going on in your account, that may be something to factor in.
Related
Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 3 months ago.
Improve this question
I am interested in setting up an AWS host that needs to occasionally pull data from another Internet site.
According to the AWS pricing calculator, inbound Internet data transfers are free but outbound data transfers are not. Does this mean I am only charged for the size of the HTTP request but the HTTP response is free? I searched but cannot find an answer to this question. Thanks.
Yes, you are correct. In your scenario, only requests incur charges. Also note that the first 100GB is free. From the docs:
AWS customers receive 100GB of data transfer out to the internet free each month, aggregated across all
AWS Services and Regions (except China and GovCloud). The 100 GB free tier for data transfer out to the internet
is global and does not apply separately or individually to AWS Regions.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
If we have a requirement on 24 hours max for RPO on data stored in Amazon S3, would S3IA be able to satisfy that?
Yes, you can say it could because Amazon S3 data is automatically replicated to 3 AZs in the region, so having a 24 hrs max RPO can ba handled. But what if disaster happens in the entire region?
Would we need a cross region replicate to satisfy this RPO requirement?
Part of your business's recovery objective goals should include identifying what type of outages are worth spending the money to guard against. Do you want to protect your business from a temporary regional AWS outage? Do you want to protect your business from a huge natural disaster that permanently destroys an entire AWS region? Do you want to protect your business from some sort of social or political upheaval that causes AWS to be shutdown in an entire country? Your business has to evaluate the level of outage that they anticipate to be an actual threat, as well as the level of outage that is worth the investment of capital to guard against.
I'm saying all this to explain why asking random strangers on the Internet if you need to implement cross region replication makes no sense. Only your business can answer that question, and simply staging an RPO number does not provide enough information.
Cross region replication, not only to a different region but also to an S3 bucket in a different account, is always a good idea. This gives you protection not only from an AWS regional outage, but also from your AWS account being hacked. Depending on what sort of outages you want to guard against, you may want that bucket to be in a different country, or even in a different hemisphere.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 2 years ago.
Improve this question
S3 has a storage pricing some $ per GB. Do I get unlimited retrievals or Am I charged when I retrieve or download an object from S3 bucket?
#hephalump only mentioned data transfer costs in the answer. Just so you are aware, you are also charged for object retrieval API calls. The detailed S3 pricing is listed on Amazon S3 pricing page. You can click the "Requests and data retrievals" tab and read the details.
Yes, you pay data transfer charges based on the amount of data transferred. As per the AWS documentation here:
You pay for all bandwidth into and out of Amazon S3, except for the following:
- Data transferred in from the internet.
- Data transferred out to an
Amazon Elastic Compute Cloud (Amazon EC2) instance, when the instance is in the same AWS Region as the S3 bucket.
- Data transferred out to
Amazon CloudFront (CloudFront).
Pricing varies based on the destination and volume but data transfer out to the Internet range from $0.00 on the free tier to $0.09/GB for the first 9.999TB/month.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I'm just wondering if there is any way to join data within Amazon Quicksight. I have several data sets each created from a separate database. I was wondering if there is any way to join these data sets together.
Thanks
Unfortunately, as of May 2019, there is not a way to join datasets from different databases or different schemas hosted in the same database.
A few options to consider that I have done to work around this:
1) If your data sources are all hosted in same database but are from different schemas, you could create a view in the database that joins the data there and then pull the data from the view
2) Use the DMS service to move your data all into the same spot. We ended up creating a datalake (i.e. S3 bucket) where we used DMS to dump a nightly snapshot of our RDS database tables from different schemas into S3. We then also have other processes that put emailed reports and other streams of data into S3. Once everything is in S3, you can use Glue Crawlers to put the S3 data into a catalog which can then be imported into QuickSight via Athena tables.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 5 years ago.
Improve this question
I am studying first time about Amazon Web Services. I want to know what is the difference or relation between Amazon s3 and Amazon Redshift.
Amazon S3 is storage service.Amazon S3 a simple web services interface to store and retrieve any amount of data from anywhere on the web. With Amazon S3, you pay only for the storage you actually use.
Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse.
Amazon Redshift gives you fast querying capabilities over structured data using familiar SQL-based clients and business intelligence (BI) tools using standard ODBC and JDBC connections. Queries are distributed and parallelized across multiple physical resources.
The relationship between Redshift and S3 is that data can be pumped into your warehouse from s3. More instructions can be found here.