I made a big technical mistake when I launched my mysql database by selecting the largest storage setting by my database. As a result I deleted the instance, but I do have a snapshot of the last instance. I don't want to reinstate the snapshot and get charged, so I was wondering if there was a way for me to export the data from my last snapshot into a csv file and then create a new instance with smaller storage. Is this possible?
The only thing you can do with an RDS snapshot is restore it into a fresh RDS instance. Once you have the instance, you can export the data as CSV or using mysqldump.
From that, you can terminate the temporary instance and create your new, smaller instance and import the data.
But you cannot get at the data directly with the snapshot.
Related
I am managing an RDS Database in an CDK Project. From time to time it is necasary to recover an old state of the database from snapshot.
To make this possible, I have CDK Context Parameter specifying the Snapshot from which the Database should be recovered. If it is given, the snapshot is bassed to the RDS Construct. This leads to a replacement of the database (which is fine).
Now, if the database has been recovered from snapshot and I do some other update in the CDK Project and want to update the project (which is done via CD Pipeline) I usally start the update without giving any snapshot. This leads to the database to be replaced by an empty one, which is of course not what I want.
To summarize, this is how it currently works:
Current state of the stack
Deployment Paramter
Result
Not created from snapshot
No snapshot
DB is not replaced
Not created from snapshot
snapshot
DB is replaced
created from snapshot
No snapshot
DB is replaced
created from snapshot
snapshot (same)
DB is not replaced
created from snapshot
snapshot (different)
DB is replaced
This is IMHO not managable.
But what I want is something like this:
Current state of the stack
Deployment Paramter
Result
Not created from snapshot
No snapshot
DB is not replaced
Not created from snapshot
snapshot
DB is replaced
created from snapshot
No snapshot
DB is not replaced
created from snapshot
snapshot (same)
DB is replaced
created from snapshot
snapshot (different)
DB is replaced
Or, to put it in other words:
If I specify no snapshot, don't replace the DB
If I specify a snapshot, repalce the DB
Is this possible? Or is there another good way to manage an RDS Database using CDK?
To me I care more about the schema, functions and triggers, and less on the data itself. From AWS document(https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CreateSnapshot.html), it doesn't address my question clearly.
Amazon RDS creates a storage volume snapshot of your DB instance,
backing up the entire DB instance and not just individual databases.
... Since the snapshot includes the entire storage volume, the size of
files, such as temporary files, also affects the amount of time it
takes to create the snapshot.
As far as I am aware, the schema, data, functions and triggers are all included. It includes all databases in the instance.
If you specifically don't want data to be backed up then you would need to create a manual backup process.
Being block level backups, AWS RDS snapshots include schema and data.
If you want a schema-only backup then use pg_dump with the --schema-only directive for PostgreSQL or mysqldump with --no-data for MySQL.
On AWS RDS, a Database Server (MySQL or SQL Server) is made of 2 things.
Compute Power (RAM, CPU, etc.)
Storage (EBS Volume)
Storage is the main Hard Disk that stores all the data files.
So, when you take a snapshot or backup on AWS RDS or Aurora, it creates a copy of the attached EBS volume (storage) and stores it somewhere in its data centers.
For example, you created 3, 4, or N SQL Server databases on an RDS SQL Server and you took the database snapshot or backup, then a copy of the complete EBS volume where database datafile (MFD, LDF, etc) are residing will be created. And when you restore it on a new instance, all databases will be restored, not an individual one.
Now answering your question, for SCHEMA backup, you can generate scripts from respective Database Management Tools.
SQL Server - SQL Server Management Studio
MySQL - SQLYug, WorkBench, etc.
Currently I am taking manual backup of our EC2 instance by zipping the data and downloading it locally as well as on DropBox.
But I am wondering, can I have an option where I just take a complete copy of the whole system automatically daily so if something goes wrong/crashes, I can replace it with previous copy immediately rather than spending hours installing and configuring things ?
I can see there is an option of take "Image" but can I automated them to have just 1 latest image and replace the system with single click ?
You can create a single Image of your instance as Backup of your instance Configuration.
And
To keep back up of your data you can use snapshots of your volumes.
snapshots store data in incremental format whenever you make any changes.
When ever needed you can just attach the volume from the snapshot to your Instance.
It is not a good idea to do "external backup" for EC2 instance snapshot, before you read AWS pricing details.
First, AWS is charging every GB of data your transfer OUTside AWS cloud. Check out this pricing. Generally speaking, after the 1st GB, the rest will be charge at least $0.09/GB, against S3-standard pricing ~ $0.023/GB.
Second, the snapshot created is actually charges as S3 pricing(Check :
Copying an Amazon EBS Snapshot), not EBS pricing. After offset the transfer cost, perhaps you should consider create multiple snapshot than keep doing the data transfer out backup.
HOWEVER, if you happens to use an instance that use ephemeral storage, snapshot will not help. You need to copy the data out from ephemeral storage yourself. Then it is your choice to store under S3 or other place.
Third. If you worry the AWS region going down, check the multiple AZ option. Or checkout alternate AWS region option.
Fourth. When storing backup data in S3, you can always store them under Infrequent-Access, which save you some bucks, and you don't need to face an insane Glacier bills during emergency restore(Avoid Glacier, unless you are pretty sure about your own requirement).
Fifth, after done your plan of doing everything inside AWS, you can write bash script (AWS CLI) or use boto3, etc API to do the automatic backup.
Lastly , here is way of AWS create and maintain snapshot. Though each snapshot are deem "incremental", when u delete old snap shot :
the snapshot deletion process is designed so that you need to retain
only the most recent snapshot in order to restore the volume.
You can always "test" restore by create another EC2 instance that load the backup snapshot. Or you can mount the snapshot volume from another EC2 instance to check the contents.
We are working with an EC2 instance in Amazon Web Services but we have lost our .pem.
In order to create a new one, we are following this guidance:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html#replacing-lost-key-pair
However, we are a bit worried because of this warning:
When you stop an instance, the data on any instance store volumes is
erased. Therefore, if you have any data on instance store volumes that
you want to keep, be sure to back it up to persistent storage.
We cannot access the instance, therefore we cannot really make a proper backup. Instead, we have make a snapshot of the volumes in Elastic Block Store.
We are wondering if this is enough and we can indeed stop the instance to proceed to the pair key recover or we need to do something else in order not to lose any data.
It depends on the type of instance.
If it's EBS backed you are probably safe to proceed as the volume will be reattached.
If it's instance store backed and you lost access to it you basically have lost what's on that machine.
By the sounds of it it's EBS backed. If it's instance store backed and you later created and attached an EBS volume and used that, you're going to be able to restore/reattach that volume just fine - but it's going to be to another machine.
Depending on how many instances we are talking about you should also be able to take an AMI Image of the running instance which will take snapshot of the EBS but also the exact state of the instance.
However if the instance's root device type is using a EBS backed store all the data should be safe so saving a snapshot and relaunching a new instance with the snapshot should have the data.
Good luck.
I want to do something very simple, I want to take a snapshot of a RDS instance (OK!) , and restore it on another instance but with a different size (duh!).
If I restore the snapshot It doesn't allow me to choose the capacity of the new instance.
If I create a DB instance, I dont know how to "attach" the snapshot.
RDS is a bit different. If you want to reduce the size of the volume, you need to export and reimport your data. When working from a snapshot, you have to choose the same volume size.