With Redshift's Pause and Resume feature, do we pay for both the storage used on the cluster and for the snapshots? If yes to the storage on the cluster, then how do we calculate the storage cost because today we pay combined for both the computing and the storage?
Thanks!
From Overview of managing clusters in Amazon Redshift - Amazon Redshift:
When you pause a cluster, Amazon Redshift creates a snapshot
...
When you pause a cluster, billing is suspended.
From Amazon Redshift Pricing - Cloud Data Warehouse - Amazon Web Services:
The pause and resume feature allows you to suspend on-demand billing during the time the cluster is paused. During the time that a cluster is paused you only pay for backup storage.
Therefore, it seems that you would only be charged for the snapshot.
Related
Setting AWS Redshift disaster recovery plan. Ideally I would like to have the ability to restore table data to a new cluster (provisioned by IAC) in my DR account.
Sharing snapshots with the DR account I don't believe will work as restoring tables needs to take place within the cluster the snapshots were created from.
Restoring snapshots to a new provisioned cluster isn't ideal as that cluster creation takes place outside of our IAC.
I believe my only other option would be to use the COPY/ UNLOAD sql commands?
You can have your Redshift cluster automatically backed up to a second AWS Region. In case of failover to the second Region, you can restore the Redshift cluster there
If your IaC is CloudFormation, you can then bring the newly restored cluster into your IaC stack
According to AWS ElastiCache documentation, a cluster loses all its data upon reboot: https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/Clusters.Rebooting.html
When you reboot a cluster, the cluster flushes all its data and
restarts its engine. During this process you cannot access the
cluster. Because the cluster flushed all its data, when the cluster is
available again, you are starting with an empty cluster.
The automatic daily backups can only be used in a new cluster to get a warm-started cluster:
https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/backups-automatic.html
AOF file option has been disabled in Redis 2.8+ in AWS ElastiCache: https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/RedisAOF.html
How do we make AWS ElastiCache use the most recent daily backup data upon hardware failure/restarts
I am looking for same thing but as of now it is not possible. Backup can only be used to build new cluster.
I have a AWS Redshift Cluster dc2.8xlarge and currently I am paying huge bill each month for running the cluster 24/7.
Is there a way I can automate the cluster uptime so that the cluster will be running in day time and I can stop the cluster at 8PM in evening and again start it in 8AM in morning.
Update: Stop/Start is now available. See: Amazon Redshift launches pause and resume
Amazon Redshift does not have a Start/Stop concept. However, there are a few options...
You could resize the cluster so that it is a lower-cost. A Redshift Cluster is sized for Compute and for Storage. You could reduce the number of nodes as long as you retain enough nodes for your Storage needs.
Also, Amazon Redshift has introduced RA3 nodes with managed storage enabling independent compute and storage scaling, which means you might be able scale-down to a single node. (This is a new node type, I'm not sure of how it works.)
Another option is to take a Snapshot and Shutdown the cluster. This will result in no costs for the cluster (but the Snapshot will be charged). Then, create a new cluster from the Snapshot when you want the cluster again.
Scheduling the above can be done in Amazon CloudWatch Events, which can trigger an AWS Lambda function. Within the function, you can make the necessary API calls to the Amazon Redshift service.
If you are concerned with the general cost of your cluster, you might want to downside from the dc2.8xlarge. You could either use multiple dc2.large nodes, or even consider a move to ds2.xlarge, which is a lower cost per TB of data stored.
good news :)
Now we can able to pause and resume the Redshift cluster (both Console and CLI)
check out the link:
https://aws.amazon.com/blogs/big-data/lower-your-costs-with-the-new-pause-and-resume-actions-on-amazon-redshift/
Now we can pause and resume an AWS Redshift cluster.
We can also schedule the pause and the resume, which is a very important feature to check on the costs.
Link: https://aws.amazon.com/blogs/big-data/lower-your-costs-with-the-new-pause-and-resume-actions-on-amazon-redshift/
This will help you in automating the cluster uptime & downtime so that the cluster will be running in day time and is paused automatically at a specific time in the evening and again start in the morning automatically.
its pretty easy to use opensource https://cloudcustodian.io to automate nightime/weekend off hours on redshift and other aws resources.
I have an app that starts and stops an RDS instance on-demand. I understand how AWS charges for a stopped RDS instance.
Does AWS charge for startup/shutdown compute time?
The pricing documentation for RDS indicates:
Pricing is ... from the time a DB instance
is launched until it is stopped or deleted.
For RDS, more specifically:
RDS is billed in one-second increments for database instances and
attached storage. Pricing is still listed on a per-hour basis, but
bills are now calculated down to the second and show usage in decimal
form. There is a 10 minute minimum charge when an instance is created,
restored or started.
FYI, for regular EC2 instances:
Each time you start a stopped instance we charge a minimum of one
minute for usage. After one minute, we charge only for the seconds you
use.
The most cost efficient solution would be to use aurora serverless. This way you dont have to start and stop the instance.
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html
When you use Aurora Serverless, you pay for only the database resources that you consume, on a per-second basis.
I am running few experiments on AWS Redshift in the free tier with a single node dc2.large cluster. I keep a snapshot as I do not need it to run the cluster at night and again restoring from that snapshot the next morning.
I can see my EC2 bill is slowly rising up with the utility but not a single documentation or blog I could find to understand if a running Redshift cluster uses EC2 instance or taking and keeping a snapshot of a Redshift cluster does the same.
Can anyone help me understanding the behavior?
Your usage of Amazon Redshift (including running a cluster and creating/keeping snapshots) should not create a charge for Amazon EC2 resources.
It might generate traffic within the VPC depending upon how you are connecting to it (eg cross-AZ traffic).