I have an AWS Backup plan which do a snapshot every day from a RDS instance, created by terraform. (This AWS Backup plan is not created by terraform).
The point is that I have to restore some of this snapshot without my tfstate lose the state of de whole deployment, because if I restore by hand, a new instance will be created and I would have to change the database endpoint in my application pointing to the new rds instance..
Using the aws_db_instance only allow to restore backups created by the own rds and I haven't found any resource to restore using terraform this snapshot..
Is there any way to restore an AWS backup snapshot without lose my tfstate? Could be the solution create the backups in the own RDS service using terraform?
Thanks in advance.
Sadly, you can't do this. You have to restore by hand, and then manually update your TF code to reflect the changes. This is the same situation when your RDS fail-overs due to failures. TF will not be aware of this, and you have to manually correct the drift.
We handled this situation in the following way in AWS to avoid changing any TF configuration:
Restore snapshot to new database db-new, ensuring that the configuration matches the original db
Modify original db to have a new identifier like db-old. Note that this will cause your endpoint to change, so your app will lose connectivity when AWS completes the modification.
Modify the db-new to have the original db identifier. This will change the endpoint to match the original endpoint.
Run terraform plan and see whether there are any configuration changes. Make sure to update the new db to match the config in the current TF state.
You should see that TF only updates a security group in place, and it sees the new db as matching the old db.
Related
I am working with a Sprint boot application hosted on AWS. Application data gets stored in AWS RDS. Now I have set up another AWS account where I would like to migrate the data from the RDS instance associated with the current AWS account.
In this regard, after going through several posts here, it seems, the steps to do the same are as follows:
1. Create a snapshot of the database in the RDS database instance associated with the current AWS account.
2. Share the snapshot with the new AWS account.
3. Import the data from this snapshot to the RDS instance associated with the new AWS account.
One problem with this approach is: I have to make the application unavailable to the end-users until all the steps are completed as well as the application with the database hosted in the new AWS account starts running, otherwise there will be always some delta remaining to be migrated.
But, I would like to migrate the data from an existing instance to the new instance without making the application unavailable to end-users, I did not find any suitable approach to achieve the same.
Could anyone please help here? Thanks.
create a read replica
stop slave in read replica using stop MySQL.rds_stop_replication
capture mysql_binary_log_file_name,mysql_binary_log_file_location
stop instance
take snapshot
restore in another AWS account
create replication user name in master
run below command in newly created RDS with required values
CALL mysql.rds_set_external_master (
host_name
, host_port
, replication_user_name
, replication_user_password
, mysql_binary_log_file_name
, mysql_binary_log_file_location
);
move connection and plan cut over properly
Good article on how to setup cross-account replication:
https://aws.amazon.com/premiumsupport/knowledge-center/rds-mysql-cross-region-replica/
Another way is using AWS Database Migration Services (AWS DMS).
You can create a migration task, and set the original database as the source endpoint, and the new database as a destination endpoint.
Next create a task with "Full load, ongoing replication" settings.
Some caveats: make sure to enable replication on the source database and it uses RAW bin log format for MySQL/MariaDB. More details here: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MySQL.html
It also damaged the destination tables structure in my case.
I want to enable automatic backups on my redis cluster on AWS.
Looking at the documentation I have to specify SnapshotRetentionLimit and SnapshotWindow. This kind of works and it creates the backup for me.
Now the problem is how can I use the latest backup and restore my redis cluster when it's restarted for example ?
I see there's a parameter SnapshotName:
SnapshotName
The name of a Redis snapshot from which to restore data into the new node group (shard). The snapshot status changes to restoring while the new node group (shard) is being created.
Which name should I include? Automatic backup creates backups with dynamic names (prefixes them with date and time). Can I somehow get the latest backup name through cloudformation template? Is this even possible via cloudformation template?
Thank you
I have a database hosted on Amazon RDS , which i need to take backup and create new database out of it. Being new, not sure how to do it. Tried doing from SSMS but it didnt work.
This is one of the top google results but there is some outdated information here.
You can enable sql native backup / restore for sql server in RDS, backup a single database to S3 and then restore that database to the same RDS instance with a new database name.
Steps for environment setup and performing the backup/restore:
Create an S3 bucket to store your native backups.
Enable Sql native backup by configuring a new option group for your RDS instance and adding the "SQLSERVER_BACKUP_RESTORE" option.
Executing a series of procedures in SSMS to run the backup and restore tasks:
exec msdb.dbo.rds_backup_database
#source_db_name='database_name', #s3_arn_to_backup_to='arn:aws:s3:::bucket_name/file_name_and_extension',
#overwrite_S3_backup_file=1;
exec msdb.dbo.rds_restore_database
#restore_db_name='database_name',
#s3_arn_to_restore_from='arn:aws:s3:::bucket_name/file_name_and_extension';
exec msdb.dbo.rds_task_status #db_name='database_name'
Note: I cannot find any announcement about removing this prior limitation:
You can't restore a backup file to the same DB instance that was used to create the backup file. Instead, restore the backup file to a new DB instance.
However, as of today, I can confirm restoring a native backup to the same instance works as you would expect.
You can do this in couple of easy steps using AWS console as well
Take RDS database snapshot. You might have already RDS snapshots. Check in AWS Console --> RDS --> Snapshots.
If you do not have snapshot, then RDS Instances --> Select the required instance--> Click on "Instance Action"--> Take Snapshot.
Then next Item is you have to create new RDS instance from this snapshot.
Go snapshots--> Select the snapshot you want to create instance. --> Click on "Snapshot actions" --> Restore snapshot.
In restore screen, for "DB Instance Identifier*", enter the name of new RDS instance.
There are certain restriction in restoring from snapshot like you can not change the size of the DB, version of software etc. New instance inherit these attributes from original database.
Take a look at the documentation for creating a DB snapshot and restoring from a DB snapshot.
I suppose you use SQL server RDS.
It is not clear if you want to restore database to the same instance or not.
Restoring backup on same instance under different name is not available in Amazon RDS
You can't restore a backup file to the same DB instance that was used to create the >backup file. Instead, restore the backup file to a new DB instance.
Troubleshooting part at
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.html#SQLServer.Procedural.Importing.Native.Using.Poll
Also SQL Server native backup is not supported from SSMS in Amazon RDS since it requires to choose location for backup but in RDS you could not access OS resources.
There are few options:
1) Create another instance from snapshot,
2) if there are many databases and you want to restore only one, you need to enable SQLSERVER_BACKUP_RESTORE and use rds_backup_database to create a backup and rds_restore_database to restore it.
Prerequisite are to have S3 bucket and IAM account has access to S3 bucket
Steps should be:
change parameter SQLSERVER_BACKUP_RESTORE in option group. Be careful, it might require server reboot.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.SQLServer.Options.BackupRestore.html
call rds_backup_database in msdb database. Required parameters are database name and S3 bucket, optional are if you want to encrypt backup, overwrite backup with same name in S3 bucket, and back up type full or differential
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.html#SQLServer.Procedural.Importing.Native.Using.Backup
Output of the procedure is task id and it can be used to check status of backup task.
exec msdb..rds_task_status #task_id= 5
After backup has been created, login to another instance and run rds_restore_database. Parameters are name of the database to restore and S3 bucket where the backup is located.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.html#SQLServer.Procedural.Importing.Native.Using.Restore
If you want you can download backup file from S3 and restore it to SQL server on premise.
3) And the longest one if you want to have both database on same instance, to script database, create under different name and export data to the new database
The whole process is described here
UPDATE: Amazon allowed to restore database on the same instance where backup was created.
https://aws.amazon.com/about-aws/whats-new/2018/10/amazon-rds-for-sql-server-enhances-backup-and-restore-capabilities/
I've my application setup on AWS (EB and EC2). My database is PostgreSQL and it is stored in the EBS service provided by AWS.
I'm going to push a major change to my application (including invasive migrations), to ensure that I don't end up losing data I want to create a copy of my whole application and update the code for that.
The steps I have till now:
Clone an EB instance
Create a snapshot of my EBS and use that to create a new volume
Update the configuration settings of my EB instance to point to the new volume and deploy the new code to the EB instance
I can't find proper documentation for how to do these things on AWS so I'm looking for some confirmation about the steps I have ensure that I don't end up wrecking something.
So the way it works, you create snapshot, create a new EC2 with Disk restored from that snapshot and you have a new EC2 running instance with same DB.
But I would suggest if possible stop the postgresql or the instance before taking a snapshot, this will ensure the state of the DB is intact.
The two EC2 instances will have no relation and changes made in one DB will not impact the other.
I am new to AWS EB and I am trying to figure out how to backup and restore an entire EB environment. I created an AMI based on the EC2 instance generated by EB, and took a snapshot of RDS, also created by EB.
The problem I have is, how do I restore it, assuming that this is the correct approach of backup. Also, I am doing it manually, shouldn't there be an automated way of doing this within EB? By the way, when I created the AMI, it destroyed the source and the EB just created a new EC2 instance without all my changes.
How do I save & restore configuration changes to my application that impact both filesystem and database?
Unfortunately, Amazon AWS Elastic Beanstalk (EB) does not support restoring databases that contain live data, if those databases were created with EB. If you reload (AKA AWS "deploy") the EB saved configuration, you get a blank database!
I called them and they told me to create the RDS DB separately and update the application code to connect to the DB once you know it's name. If you restore the RDS DB it will have a new name too! So you have to update your code again to connect to it.
Also, if you code and environment is fine, but you want to restore your database, again it will have a new name and you will need to change your code.
How to change your code easily and automatically deploy it is a whole other question for which I don't have an answer yet.
So basically the RDS DB provisioning within Elastic Beanstalk has very limited uses, maybe coding and debugging and testing, but not live production use. :(
This is as of Jan 2015.
First go into your EB environment and save the current config. You should go to a running EC2 instance created by EB and make an Image. Then use that new AMI ID by going to the EB configuration and setting it. It will rebuild the environment tearing down all running instances and creating new ones.
For your RDS instance you should make a backup and restore with a new instance name as the docs say you will lose it if the environment is destroyed. You should probably just manually set the environment variables like RDS does and setup the proper security groups between RDS and EC2.
One option I think could work is just renaming the RDS instance name as the environment seems to break and then destroy the environment and create a new one with an attached RDS instance and then destroy that one and rename the old one to the new one's name which may work.
As always make proper backups before proceeding with any of these ideas.