I have a Amazon RDS MySQL database in the us-east-1 region. I have a few read replicas in that region for the master db as well. I'm working on a new app which will require the same data from that db but I would like to host the app and create a new db instance in us-west-2 for latency reasons. I still want to host the original application in us-east 1 though.
Is it possible in Amazon RDS to create a cross region bidirectional database so the db in us-west-2 is a replica of the one in us-east-1 that I can also write to?
I rather not use Amazon Aurora right now if possible but if that's the only answer then I guess it's something I will have to do.
Amazon RDS does not support multi-master clusters across regions.
From Working with Aurora Multi-Master Clusters - Amazon Aurora:
You can't enable cross-region replicas from multi-master clusters.
Both Aurora and the non-Aurora databases in Amazon RDS can provide multi-region read replicas, but these are one-way replicas.
If you happen to be using Aurora MySQL - take a look at Write Forwarding.
This allows replicas to send writes to the primary cluster and then (depending on your selected consistency option) waits for relevant changes to be replicated back to itself and other region replicas.
This can help get around the cross-region limitations of Multi-Master and lets you at least simulate a 2-way replication across AWS regions.
Related
I'm researching about AWS RDS bidirectional replication. I know that RDS has the read replica, but I need the bidirectional replication for disaster recovery.
Would anyone have the experience with this before? I'm very appreciated for your help.
AWS RDS does have multi master capability with Aurora. See documentation below:
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-multi-master.html
It supports replication across multiple master across multiple AZ. This is for a region only. Since you mentioned disaster recovery, it might not satisfy your requirement but you can evaluate it to decide as per your requirements.
See the blog post below which talks about replication architecture of multi master Auora
https://aws.amazon.com/blogs/database/building-highly-available-mysql-applications-using-amazon-aurora-mmsr/
For cross region, you can use Auora Global database but that supports only 1 writer instance.
What db engine are you using in RDS? is that mysql, pgsql , sql server or oracle? Aurora multi-master only supports MySQL, and even multi-master we are talking about both master in same AWS region, which may not satisfy your requirement of "DR". Why do you need bidirectional in first place? If the writes from "master A" and "master B" are isolated, i.e. A and B did not update same portion of same table at the same time, then you may consider to use AWS DMS bi-directional replication to setup a DR db for your primary DB, but notice that AWS DMS bi-directional replication is not a multi-master db solution, it won't handle any data conflict / inconsistency if primary and replica db update same rows. For detail please refer to AWS doc: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Task.CDC.html#CHAP_Task.CDC.Bidirectional
I noticed when a Cross Region Read Replica was created for my AWS Aurora cluster - it has both a Write and Reader instance (Similar to my Primary which naturally has a writer in addition to a reader instance). In the Cluster Configuration for the Cross Region Replica cluster - I can see this "Replica" cluster indeed has the Replication source tag and its correctly getting all data flowed asynchronously.
Couple of questions I need help understanding this:
should a Cross Region Replica have a Writer?
Should I write to it in case of a disaster in the source region?
I had to explicitly make the read-replica instance (in replica region) as read_only. Otherwise read-replica accepts writes to it.
Found relevant answer in AWS re:Post:
"You're replicating to a new cluster, and each cluster needs a writer
instance. It is recommended that customers apply the read_only
parameter to the replica, but by default they are able to write to
this instance. This allows for architectures where the replica is read
& writable.
https://repost.aws/questions/QUrCbnj0u4TWaz-A1uR-QDPQ/aurora-create-cross-region-read-replica-vs-add-region
I think the name "writer" is a bit misleading.
In the doc for Aurora endpoint:
A cluster endpoint (or writer endpoint) for an Aurora DB cluster connects to the current primary DB instance for that DB cluster. This endpoint is the only one that can perform write operations such as DDL statements. Because of this, the cluster endpoint is the one that you connect to when you first set up a cluster or when your cluster only contains a single DB instance. Each Aurora DB cluster has one cluster endpoint and one primary DB instance.
So the writer instance is the same entity as the primary instance of the cluster.
In the doc for Cross-Region Replication:
When you create a cross-Region read replica for Aurora MySQL by using the AWS Management Console, Amazon RDS creates a DB cluster in the target AWS Region, and then automatically creates a DB instance that is the primary instance for that DB cluster.
For cross region replication, the new cluster (writer instance) will be created.
You don't need to take care of the writer instance. For the disaster recovery, promote read replica.
I am reading this doc: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.html about high available for Aurora. It supports Multi-AZ configuration for db instance in Aurora cluster. Based on the doc, Amazon RDS automatically provisions and maintains a synchronous standby replica in a different Availability Zone, does this mean it creates one db instance per AZ in the region?
In my current configuration, I have three db instance AWS::RDS::DBInstance and each one is provisioned in one AZ. Do I still need to create three instance? Can one instance do the job? I understand multiple readers help on high throughput and performance. But in terms of High available, do I still need to provide three instances for three AZ?
You are reading the wrong link for MultiAZ. Your link is for MultiAZ in RDS, not Aurora. This is important because in Aurora there is no standby replica. MultiAZ in Aurora works differently then in RDS.
The correct links are High availability for Amazon Aurora and Replication with Amazon Aurora.
Basically, in Aurora, your Replicas are the source of its HA and MultiAZ. If Aurora writer fails, Aurora will automatically promote one of its Replicas to be new writer.
Can one instance do the job?
No it can't. For Aurora you need at least one Replcia.
I understand multiple readers help on high throughput and performance.
Not only this. Replicas provide fail-over capabilities for Aurora.
But in terms of High available, do I still need to provide three instances for three AZ?
Three are not needed. Minimum is two - writer in one AZ, and replica in a second AZ.
On aws rds postgresql (not aurora) cluster, there is a writer instance and a reader instance. Couple of questions -
In the describe-db-cluster command output it shows "MultiAZ": true, but describe-db-instances of this cluster's instances says "MultiAZ": false.
(a) Are the cluster's instances configured to be multi-az?
(b) Whenever writer instance fails over I see reader becomes writer, a bit confused here because I thought the difference between multi-az and read replica setup is that in case of multi-az's automatic failover happens and in case of read replica's failover needs to be triggered manually. How is it that read replica is becoming primary in case of failover?
(c) Why multi-az shows to be true at cluster level. Is it
showing multi-az because of reader and writer being in different
az's?
How do I know what storage class (provisioned iops, gp2) is being used by the cluster's instances? I am unable to figure it out from the describe-db-instances or describe-db-cluster command output.
I think there is some confusion here what is cluster, aurora, multi-az, and so on.
"Cluster" only applies to Aurora. Thus there is no such thing as "postgresql (not aurora) cluster" and command describe-db-cluster will not work on non-aurora databases. If you try to run it on non-aurora db instance you will get:
DBCluster <name> not found
In Aurora, MultiAZ is provided through Aurora replicas (i.e. readers), while in RDS databases, MultiAZ is provided through a Stand-by db instance, which is different from read-replicas.
How is it that read replica is becoming primary in case of failover?
Because this is exactly what happens when you run Aurora. In RDS, the Stand-by becomes the new primary, not read-replica.
Why multi-az shows to be true at cluster level. Is it showing multi-az because of reader and writer being in different az's?
Yes. In aurora Multi-AZ is enabled through readers in different AZs than primary.
How do I know what storage class (provisioned iops, gp2) is being used by the cluster's instances?
In Aurora there is no such notion as different storage classes. AWS uses different and in-house develop storage solution for it.
can someone help me with these questions please :
1- the documentation stated that Aurora will automatically fail-over to the read replicas, my question is how does it select the replica which will be promoted if you have more than one with different instances class?
2- can I disable this automatic fail-over (just asking, not stating that I will do it)
3- what is the purpose of multi-az in Aurora if you can have the same effect with much more control on instances classes while creating replicas and let Aurora do the auto fail-over for you. please correct me if I am wrong with this assumption.
thanks in advance
The algorithm for election of a new master in case of failure is not really documented... but it doesn't seem to matter, because Aurora replicas seem to be different than other RDS replicas, because all the instances in the cluster are necessarily of the same instance class.
Unlike other RDS offerings, read replicas in Aurora do not appear to have an independent copy of the backing store -- instead, the backing store itself provides redundancy, being replicated at the storage level with two copies in each of three availability zones.
The cluster volume is made up of multiple copies of the data for the DB cluster, but the data in the cluster volume is represented as a single, logical volume to the primary and Aurora Replicas in the DB cluster.
Because the cluster volume is shared among all instances in your DB cluster, no additional work is required to replicate a copy of the data for each Aurora Replica.
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora.Replication.html
Multi-AZ in Aurora is also not the same thing -- with other RDS products, multi-AZ involves a second, invisible instance, running in parallel with the master. The Aurora literature uses the phrase "multi-AZ technology," but the meaning appears to be different. Note that the Aurora pricing tables don't show a separate pricing rate for "multi-AZ" the way MySQL and MariaDB do.
Failover doesn't appear to be something that can be disabled. Even if you have no replicas, Aurora will still "fail over" if the master fails -- but it does it by spinning up a replacement master using the existing cluster volume as the backing store.
The above answer is no longer valid anymore.
Multi AZ = Aurora Cluster with at least one Read Replica in a
different AZ.
You can still create multiple read replicas for a cluster but if you create them within the same AZ of your writer, cluster will not be multi AZ.
Within each AWS Region, Availability Zones (AZs) represent locations that are distinct from each other to provide isolation in case of outages. We recommend that you distribute the primary instance and reader instances in your DB cluster over multiple Availability Zones to improve the availability of your DB cluster. That way, an issue that affects an entire Availability Zone doesn't cause an outage for your cluster.
You can set up a Multi-AZ cluster by making a simple choice when you create the cluster. The choice is simple whether you use the AWS Management Console, the AWS CLI, or the Amazon RDS API. You can also make an existing Aurora cluster into a Multi-AZ cluster by adding a new reader instance and specifying a different Availability Zone.
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Concepts.AuroraHighAvailability.html