I need a database, which is idle most of the time, because it is a test db. The devs normally use their local DB. Is there some function like: when 30 minutes of inactivity -> then stop db
When you are talking about an RDBMS, then there is Aurora Serverless, with either MySQL or PostgreSQL Compatibility.
You can configure it to scale-down to zero "capacity units" after a certain period of inactivity.
One key difference between an Aurora Serverless DB cluster and a provisioned DB instance is that with a Aurora Serverless DB cluster you cannot directly make the DB endpoint publicly accessible (from the Internet).
Related
I would like to setup a PostgreSQL database with read-only standby with automatic failover ability.
Does Amazon RDS with Multi-AZ have automatic failover ability?
If yes, will the endpoint/DNS automatically point to the new standby database?
There are 2 scenarios to this question as it was little ambiguous in the question.
scenario 1 :- rds with multi AZ.
Here standby instance will have automatic failover.
Here the endpoint/DNS automatically point to the new standby database, no manual intervention is needed.
But read and write is being taken by only the master DB and sync replication takes with standby database.
scenario 2 :- rds with multi AZ where read replicas is also there.
In this scenario read replicas are not treated like standby database database by default.howver they can be used like standby.
failover is not automatic.
You will need to manually update your dns.
article from aws says
A Read Replica in a different region than the source database can be used as a standby database and promoted to become the new production database in case of a regional disruption.
also a similar question on stack should also help you Difference between "Multi-AZ Deployment" and "Read Replica Verison Multi-AZ Deployment"
Scenario 2 seems to have slightly evolved since 2021.
Now this has become Multi-AZ DB Cluster
Reader DB instances act as automatic failover targets and also serve
read traffic to increase application read throughput. If an outage
occurs on your writer DB instance, RDS manages failover to one of the
reader DB instances. RDS does this based on which reader DB instance
has the most recent change record.
I'm trying to create an Aurora read replica from a RDS MySQL database, but I'm getting the following error:
You must enable backups on the master DB instance in order to set up DB instance to DB cluster replication...
I double checked the source instance and I already have backups enabled.
What else could be the problem?
It turns out that the problem was that my Backup retention period was set to 0 days.
I am having an AWS RDS Aurora PostgreSQL cluster with four instances with a Multi-AZ deployment serving in Production. Encryption-at-rest hasn't been enabled on this cluster. Now I have to enable the encryption on this existing cluster. AWS docs suggest me to create a snapshot of that cluster and then restore the cluster again with the encryption enabled this time. Ref: Here
Since my cluster is serving in production and no downtime or I/O suspension is acceptable to me. Here are some questions that I would like to get answered before I plan about encrypting the existing cluster:
Is there any downtime during the creation of the snapshot assuming there is a lot of data and a snapshot will take time.
What about the new data that is being written on to the database during the snapshot creation? Is the snapshot creation real-time or I will lose my new data during the time till the snapshot is being taken?
Is this the only way for me to enable encryption on the production cluster knowing that it will result in some database outage?
There is a way to encrypt your AWS RDS Amazon Aurora with PostgreSQL compatibility Cluster with no or minimum downtime, but it will take a bit of effort.
You need to take the following steps:
For the source DB, you have to take snapshot.
Then copy that snapshot, and check Enable Encryption and select Default Encryption Key or select your Custom AWS KMS CMK, now you have an encrypted copy of your DB snapshot.
Restore this encrypted snapshot to the new DB instance, and you can enable Multi-AZ and add Read Replicas now or modify them after migration.
Now you have two DB instances Encrypted and Unencrypted, but the data mismatched as it is a production database.
We will use AWS DMS to make synchronous replication of data, or ou can use PostgreSQL logical replication with Aurora instead of AWS DMS, it will be better, both will works.
Go to AWS DMS console, create an AWS DMS task.
For migration type, choose Migrate existing data and replicate ongoing changes.
For target table preparation mode, choose Truncate.
Under Advanced Task Settings, enable the awsdms_status table if you want to verify replication status.
Run the migration task and wait until all the records are updated. AWS DMS will then determine the size of the data to migrate.
Then, you need to verify the data in the Encrypted DB instance after migration is the same as the Unencrypted DB instance.
Check replication status in AWS DMS, by checking the migration task and awsdms_status.
You can now route traffic to the new endpoint.
For a smooth cutover, use Amazon Route 53 to route traffic by changing the DNS TTL to a short value, and eventually replacing the endpoint names in Route 53.
Now replying to your questions,
Is there any downtime during the creation of the snapshot assuming there is a lot of data and a snapshot will take time.
According to you cluster setup, you are running a Multi-AZ deployment, automated backups and DB Snapshots are simply taken from the standby to avoid I/O suspension on the primary. Please note that you may experience increased I/O latency (typically lasting a few minutes) during backups for both Single-AZ and Multi-AZ deployments.
What about the new data that is being written on to the database during the snapshot creation? Is the snapshot creation real-time or I
will lose my new data during the time till the snapshot is being
taken?
You will lose your data written after the snapshot has been taken, so you will use AWS DMS to replicate synchronous data to your encrypted DB instances.
Is this the only way for me to enable encryption on the production cluster knowing that it will result in some database outage?
Yes this is the only way, but it will result in no or little downtime.
When I restore a MySQL snapshot, I'm given the option to make the new instance Multi-AZ. However for some reason when I restore to Aurora, the "Multi-AZ Deployment" selection is disabled.
I thought possibly this meant it was automatically enabled. However when I go to the running instance details, it specifically lists Multi-AZ: "No".
Is there something I'm missing? Is there some other step I need to do to make Aurora Multi-AZ?
Multi-AZ doesn't mean the same thing in Aurora as it does for MySQL and MariaDB. With Aurora, any replica in the cluster can take over for the master on failure (though the selection is based on priority, so it isn't just a randomly selected replica, if there is more than one replica), and the storage is already/always multi-AZ in Aurora because that's a core part of the Aurora design. So, Aurora can be "converted" to Multi-AZ.
Apparently, that's the only way it is done when creating an Aurora instance from a snapshot, based on this:
You can migrate a DB snapshot of an Amazon RDS MySQL DB instance to create an Aurora DB cluster.
...
You can migrate either a manual or automated DB snapshot. After the DB cluster is created, you can then create optional Aurora Replicas.
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora.Migrate.RDSMySQL.html
If you create a multi-AZ Aurora cluster without a snapshot, that's all that happens -- a master and a replica are created. In that case, you can actually delete the replica and make it non-multi-AZ, which is a different process than what's involved with MySQL or MariaDB.
"Create a cluster" -- as used in the quote, above -- is a potentially confusing term, since you would naturally assume a cluster means two or more, but in fact an Aurora cluster can technically be a "cluster" of just one instance. Every Aurora instance is part of exactly one cluster. One instance is the primary (master) and any additional instances are replicas.
If the DB cluster doesn't contain any Aurora Replicas, then the primary instance is recreated during a failure event.
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora.Managing.html#Aurora.Managing.FaultTolerance
I am trying Amazon Aurora instance and I can not see an option to stop it. The only options are Delete and Reboot.
Am I missing something.
Edit: 2018/09/25 - Amazon Aurora Now Supports Stopping and Starting of Database Clusters
Per this announcement, Aurora now supports starting and stopping the db instance. This feature was released for other RDS configurations last year and generally behaves the same in this implementation. Stopping the database cluster stops the primary instance and any Aurora replicas. The database will remain stopped for 7 days, after which it will be automatically restarted.
Additionally, see:
AWS Documentation - Stopping and Starting a DB Instance
AWS CLI Documentation - aws rds start-db-instance
AWS CLI Documentation - aws rds stop-db-instance
Edit: 2018/08/13 - Serverless Aurora supports infrequent access workloads
Per this announcement, Serverless Aurora has been released and has a feature to pause compute capacity after N consecutive minutes of inactivity. Here's an example of this configuration from the announcement blog:
This value is configurable up to 1440 minutes (24 hours), and would be comparable to "stopping" an Aurora Instance because you are not paying for compute capacity while compute capacity is paused, only database storage.
If your use case includes infrequent access (example: a dev instance that is not used after-hours), then this is currently the easiest option available.
Edit: 2017/06/01 - RDS can now be stopped/started, per this announcement, but does not support Aurora yet.
In the AWS console, a 'Stop' option has been mentioned to the same menu described in the question body. A stopped instance retains its storage but cannot be modified until it is started again. Instances cannot be permanently stopped -- after seven days, a stopped instance will be automatically started again.
Unfortunately, some configurations still don't support this feature. See below the break for an alternative.
Additionally, see:
AWS Documentation - Stopping and Starting a DB Instance
AWS CLI Documentation - aws rds start-db-instance
AWS CLI Documentation - aws rds stop-db-instance
Alternative suggestion for incompatible configurations:
This includes:
Aurora via old SDKs prior to September 2018
Old SDKs prior to June 2017
RDS with Multi-AZ
RDS with read replica
RDS with SQL Server Mirroring
RDS instances in these situations cannot be stopped and started at-will like an EC2 instance. You can, however, approximate this behavior by deleting the RDS instance with a final snapshot, and then returning later to restore an RDS instance from that DB snapshot.
This is approximate to "stopping" because no new data will be written once your final snapshot is taken, and you also won't be paying for a running RDS instance. In the same way that you would pay for any EBS volumes attached to your stopped EC2 instance, you will likewise continue paying for the storage of the DB snapshot while it exists.
This behavior diverges from stopping an EC2 instance inthat the underlying infrastructure will no longer exist. When you restore from snapshot, that will be on a new RDS instance and that restoration will take some time to complete, depending on the size of both your infrastructure and snapshot.
Further Reading
AWS RDS Documentation - Delete Instance with Final Snapshot
AWS RDS Documentation - Restoring From a DB Snapshot
They did add support for stopping/starting RDS instances this month (June 2017). The key bit of info from #Venkata's link is: "The stop/start feature is available for database instances running in a Single-AZ deployment which are not part of a Read Replica (both source and replica) configuration."
This means it does not work for Aurora because it is Multi-AZ by default. You do not even receive the option in the interface.
It also will not work for MySQL if a read replica is configured. You will not get the option on the slave and you get the option on the master but it fails if you try to use it.
You can achieve the same thing with a restore from a snapshot... but the down side is that takes roughly an hour with our large db instance whereas stopping it and starting it can happen in ~5 minutes.
They just added support for this today:
https://aws.amazon.com/about-aws/whats-new/2017/06/amazon-rds-supports-stopping-and-starting-of-database-instances/
Enjoy! :)