I am somewhat new to AWS RDS and their terminology. What they call a "database" seems to me to be an SQL Server instance. I have a database (as defined by SSMS--with tables, data, stored procedures, etc.) on RDS named "prod" and I want to duplicate it for testing purposes to be named "test" with all the content, and leave "prod" as-is.
All the instructions I've found by doing many, many searches seem to be related to duplicating the entire instance. Can someone help me with instructions on how to create a duplicate of just the (ssms term) database?
Thanks in advance for any help!
P.S. What does AWS/RDS call the object that is equivalent to an SSMS database?
I've found multiple posts here about duplicating an entire instance. It could be that I don't fully understand the terminology because I know this must be a common task but I am not understanding how to do it.
This is a production environment so I am proceeding very cautiously. I do have nightly snapshots made so I know I could recover but would rather do it right the first time.
I usually use a command like this to backup a single database to s3:
exec msdb.dbo.rds_backup_database #source_db_name='<mydatabasename>', #S3_arn_to_backup_to='arn:aws:s3:::<mys3objectname>', #type='FULL'
There is a bit of one-time configuration you need to do first, see the link below , and then its as simple as executing commands from SSMS to backup a database to S3 and then restoring it from S3 - maybe not exactly what you are looking for, but it works great.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/SQLServer.Procedural.Importing.html
What you are looking for can't really be done via any RDS commands/tools/interface. RDS only concerns itself with the database server itself, and isn't really even aware of the different databases, schemas, tables, etc. you may have created on the server.
You will need to use the tools for the DBMS you are using, in this case it sounds like you are using Microsoft SQL Server, so you will need to use MS SQL Server tools (perhaps running on an EC2 instance) to dump a single database to a file, and then load it into another database.
P.S. What does AWS/RDS call the object that is equivalent to an SSMS database?
An RDS database is a "database server". You might also see it called a "database instance". AWS/RDS calls the object equivalent to an SSMS database simply a database. The terminology is confusing because both the physical server/computer running the database software, and the logical grouping of tables/logs/data inside the software are both generally referred to as a "database".
Related
Since there is a nightly backup of SQL we are wondering of a good way to restore this backup to a different database in the same MySQL server instance. We have prod_xxxx for all our production databases AND we have staging_xxxx for all our staging databases (yes not that good in that they are all on the same mysql instance right now).
Anyways, we would love to restore all tables/constraints/etc and data from prod_incomingdb to staging_incomingdb. Is this possible in cloud SQL?
Since this is over a productive instance I recommend you to perform a backup before start, in order to avoid any data corruption.
To clone a database within the same instance, there is not a direct way to perform the task (this is a missing feature on MySQL).
I followed this path in order to successfully clone a database within same MySQL Cloud SQL instance.
1.- Create a dump of the desired database using the Google Cloud Console (Web UI) by follow these steps
*it is very important to only dump the desired database in format SQL, please not select multiple databases on the dump.
After finish the process, the dump will be available in a Google Cloud Storage Bucket.
2.- Download the dump file to a Compute Engine VM or to any local machine with linux.
3.- please replace the database name (the old one) in the USE clauses.
I used this sed command over my downloaded dump to change the names of the databases
sed -i 's/USE `employees`;/USE `emp2`;/g' employees.sql
*this can take some seconds depending the size of your file.
4.- Upload the updated file to the Cloud storage bucket.
5.- Create a new empty database on your Cloud SQL instance, in this case my target instance is called emp2.
6.- Import the modified dump by following these steps
I could not figure out the nightly backups as it seems to restore an entire instance. I think the answer to the above is no. I did find out that I can export and then import (not exactly what I wanted though as I didn't want to be exporting our DB during the day but for now, we may go with that and automate a nightly export later).
I have multiple questions about this issue since AWS is still new to me.
What I am trying to accomplish: I want to create a test/sandbox db for my live db instance. I want to do this to create a sandbox environment for my developers, so they can add, edit, delete whatever they want on the test db without effecting the companies live db that's being used for our site.
Specifications: I want my test db to be continually updated alongside my live db. So I want a parent and child relationship or master/slave. So basically whatever the master has the slave will have but the slave cannot edit the master in any way whatsoever. Also I want the slave to have a different security group than the masters.
Things I have tried: first I thought I could copy a db snapshot of my live db and transfer it to a new db instance. But how would the new instance be continually updated. I couldn't even find what to do with the copied snapshot so that was a dead end.
Then I read that I could create a read replica of my master db, so I did that, I promoted it as per the documentation. Now can I read and write to this replica? Is this the correct path for what I want? Can I assign the replica a different security group from the master? And will the replica get updated data from master?
Is what I'm asking possible? Could someone please explain to me what the correct course of action is here or if I'm going about it incorrectly.
Again a bit lost as I'm new to AWS, any help would be greatly appreciated! :)
Documentation for reference:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CopySnapshot.html
Not possible with RDS. Either you use replication (and can't write to replica) or make a clone that will not update.
You can, however, setup you own MySQL/MariaDB master-slave configuration and write to slave. Should work fine as long as you don't have conflicting values in indexes (usually because of autoincrementing primary key).
When it comes to databases, we want to leave managing them to the pros, which is why we went for a managed solution in the form of a CloudSQL 2nd gen db instance. Today the instance stopped responding, I clicked restart, it has been restarting for hours and is not responding, I have tried clone the instance, also not responding.
I don't know what else to do, our db is crippled and the service that uses it is down. These things happen, fine.
The thing that shocked me is that I am unable to contact anybody to resolve this problem. I understand that I can pay for a support subscription, $150p/m and up. This confuses me though, the GCloud console UI is not responding, am I incorrect in assuming I should not have to pay for support for the core product to at least work?
This leads me to my main question, if I want to continue using Google Cloud products in production, do I NEED a support subscription?
Same happened to us yesterday. The cloud SQL instance did not respond for an hour and a half (from 18h to 19:30h GTM+1).
We couldn't do absolutely nothing, we tried to backup the instance to a bucket but the command was returning an error saying that another operation is in progress.
We are a small startup and we can't pay for a support plan, but when we hired the cloud SQL service we thought that this kind of situations doesn't happen.
Honestly, after this I believe that Cloud SQL is not a good option if you do not contract at the same time a gold or platinum support plan. It is frustrating that something fails and you can not do anything, or even report the error.
Try the gcloud command line tool in your active shell, instead of the console UI. Try exporting the data from your SQL instance to google cloud storage bucket by using this command:
gcloud sql instances \
export <sql-instnace-name> \
gs://<bucket-name>/backup.sql
The SQL instance's service account by default has read and write access to google cloud storage bucket.
Create a new SQL instance using this command:
gcloud sql instances \
create <new-sql-instance-name>
Now, add the data to the new SQL instance using this command:
gcloud sql instances \
import <new-sql-instance-name> \
gs://<bucket-name>/backup.sql
You can get free or premium support here. You do not need a subscription to get help; it all depends on your needs and the level of urgency you estimate for eventual future problems.
If you have a recent backup of your database, you may consider re-creating it in another instance, from there.
You may consider posting your issue in the Google Cloud SQL Product Issue Tracker. This way, it will enjoy much better visibility from developers and Google support, without attracting any extra costs.
I'm very new to Amazon web services, especially using their RDS system. I have set up an Oracle database (11.2) and I now want to import a dump we made locally from our server using expdp. Apparently, the ability to use expdp/impdp on AWS is quite new. From what I understand, when creating an ORACLE database on RDS, a DATA_PUMP_DIR is automatically created. What is less obvious is how to access this directory and made our local dump available to RDS. I've tried to read the following information http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Oracle.Procedural.Importing.html on their website. But there is a lot of things I don't understand:
Why do I have to setup an EC2 instance when the dump file is actually on my local computer (and I can access remotely the RDS database using sqlplus or sql developper)
They are often using the 'sys' or 'system' user in their examples but, when reading the security settings for Oracle, it said that these users are made unavailable on RDS => you cannot connect to a database as Sysdba.
Could someone please point me to a simple and clear tutorial on how to use impdp on AWS ?
Thanks
It is possible to use Data Pump on RDS now.
duduklein's answer was correct when he wrote it. But the RDS docs now have details about using Oracle Data Pump. The doc page url is unmodified from the link as originally posted in the question (nice job, Amazon!) but it has new content on using Data Pump now.
It's not possible for now. I have just contacted amazon (through the premium support) for the same issue and they just told me that this is a feature request that was already passed to the RDS team, but there is no estimation of when this will be available.
The only way you can import files dumps is using the "exp" utility instead of the "expdp". In this case, you can use the "imp" utility to import data to RDS
I have a database instance on RDS with 2 databases on it. Is there a good way using the RDS command line tools to copy the one database to the other? If not, what is the recommended way of doing it?
This is not an exact solution to the OP, but if all you need is to clone an existing database for a new purpose, there's an easier way. You can take a snapshot from the original RDS instance, then restore it to a new instance. You can even use the web console.
I'd use mysqldump to get the tables and then mysql to import them.
Update 2014/07/08: Depending on what you're planning to do here, another solution today is to setup replication and then to promote the slave to be the master. That is for example if you want to update your database's release/version:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html
If you're looking to backup externally, there's also replication:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/MySQL.Procedural.Exporting.NonRDSRepl.html
RDS has come a long way.
it depends on which database you are hosting there - for SQL Server I have used the SQL Azure Migration wizard (free download from CodePlex).
To get full RDBMS functionality the trick is to use the DNS name of your SQL Server instance in the wizard, but select 'SQL Server v2008' (or eventually v2012 after AWS RDS makes instances with 2012 available) and do NOT select to-->'SQL Azure'. I did a short screencast on this on my blog as well.