If I try to take a snapshot
rhc snapshot save -a django
It is not saving all the data and codes in the server.
Here is the link of my app
http://django-appspot.rhcloud.com
It is running on:
Django-1.5.1
python-2.7
Mysql
The size without /tmp , /ssh , /sandbox is 866M
I think I am exceeding the disk quota.
Currently I am unable to take the media folder backup. Is there any way around??
It is Solved
https://www.openshift.com/forums/openshift/rhc-snapshot-not-saving-all-the-data#comment-33800
Use WinFTP to download the backup files
Related
Hi this is my first experience trying to deploy a Python app on cloud using CF. I am having issues deploying my app; I sincerely appreciate if anyone can help me or point me to the right direction to solve the issue.
The main problem is the app that I am trying to deploy is large size due to a lot of python dependencies. The size of my app directory is 200 Kb. The first error I observed was: Staging fails due to "Failed to upload payload for droplet" . I think the reason is when all Python dependencies are downloaded from requirements.txt file and finally the droplet is created its size is too large for upload. The droplet size=982. 3 Mb.
The first solution I tried was vendoring app where I created a vendor directory containing all python dependencies but the size of vendor directory was greater that 1Gb, which causes the upload size exceed 1Gb limit and leads to failure in uploading app files.
The second solution I am working on is to upload all installed Python libraries on an object store (in my case S3 bucket which is bounded to my app) and then download the dependencies folder called Pypackages to the app's root directory: /home/vcap/app, so I want to have /home/vcap/app/Pypackages exist before my app starts on the cloud. But I couldn't do it successfully yet. I have included a python script in my app directory which downloads files from S3 bucket successfully. (I have put the correct absolute path for download in downloadS3.py script ie, /home/vcap/app/Pypackages) I want to run this script using "python downloadS3.py" as a one-off task. First I tried the solution here : Can I have multiple commands run in a manifest.yml file?
and although I can see the status of the task is SUCCEED via '$cf tasks my-app-name' , /home/vcap/app/Pypackages does not exist.
I also tried to run one-off task as the steps below:
1-
$ cf push -c 'python downloadS3.py && sleep infinity' -i 1 --no-route
2-
$ cf push -c 'null'
I have printed the contents of /home/vcap/app on my app, ie when app is started and I enter the url in my browser (I don't know what is the right way to see the contents of root directory). Anyway, the problem is Pypackages are not downloaded to the correct root directory. I am not sure if I am running the one-off task in a wrong way or if there is a better solution to make my app work.
I appreciate any helps! (edited)
Diego Cells stage apps and upload droplet to blobstore via cloud controller, the max file can be uploaded is configurable at Ops Manager > TAS for VMs > Application Developer Control > Maximum File Upload Size (MB), default is 1024MB. Seems this is causing restriction, if you can get it increased with your admin help...
Tasks run in their own containers so possibly not an option. I think Python buildpack collects and install the packages before creating the droplet, so don't think copying packages directly to /app directory will be of much help.
If you have data files then you can use .profile file and do some scripting to copy them from S3 or server/NFS location into the /app directory. Something like
wget http://s3.location.com/data_files
cp data_files /home/vcap/app/
But if all these are packages and increasing the size is not feasible then you may need to look to break the app..
I was using the jupyterlab notebook instance at AI platform at GCP. You can access this by 1) entering GCP console, 2) search notebook instance and choose the entry with the subtitle of AI platform. 3) create one.
When I upload a zip to the jupyterlab, the speed is very very slow.
Don't know what to do. It is very frustrating when cost a day just to upload the data.
The Davic at GCP 24/7 chat support is helpful. After checking a bunch of things such as network speed (http://speedtest.net)
I found the speed of uploading a single file is pretty fast. And the network is pretty good too. Since my dataset is available at Kaggle, I just thought why not download directly from kaggle.
So I used the following commands:
pip install kaggle
mv kaggle.json /home/jupyter/.kaggle. # download your kaggle.json from profile page, upload it to jupyterlab, and move this place
chmod 600 /home/jupyter/.kaggle
kaggle download datasets {username/dataset name}
It is done!! Just 5 seconds, I guess, the dataset is deployed!!
I am trying to mount a persistent disk with data to a VM to use it in Google Datalab. So far no success, ideally I would like to see my files in the Datalab notebook.
First, I added the disk in VM settings with Read/Write mode.
Second, I ran $lsblk to see what disks there are.
Then tried this: sudo mount -o ro /dev/sdd /mnt/disks/z
I got this error:
wrong fs type, bad option, bad superblock on /dev/sdd,
P.S. I used the disk I want to mount on another VM and downloaded some data on it. It was formatted as NTFS disk.
Any ideas?
I would need to create a file system as noted in the following document
$ mkfs.vfat /dev/sdX2
Creates a new partition, without creating a new file system on that partition.
Command: mkpart [part-type fs-type name] start end
I have a Django app live on Heroku. I'm migrating it to Azure, taking advantage of the $120K/yr credit they recently offered me. Here's what I've done so far:
i) I created an Azure VM with Ubuntu (Standard_D1).
ii) I installed postgresql on it (my db of choice)
iii) I pulled my Heroku app's files from my github onto the Azure VM.
iv) I created a postgres DB on the Azure VM, and then ran syncdb to create the required tables.
v) I tweaked postgresql.conf and pg_hba.conf to cater to some tuning requirements and such.
vi) I took a backup from my Heroku app's dashboard, and downloaded it. This backup file's name is a random uuid, without a file format (e.g. f0af6457-1a24-47d0-881c-434f9bef7c92).
vii) I'm now gearing up to use pg_restore to fit the backup in the newly created+synced app on Azure VM.
Does all this sound about right so far? I have 3 questions:
1) Will pg_restore work with the backup I got off Heroku? This backup doesn't have a file format at all; whereas I'm under the impression it has to be a .tar archive to be compatible with pg_restore.
2) My database is called mydbname. The data backup is saved at /datadrive/backup/filename. Thus, in my case is the correct pg_restore command something like: pg_restore -d mydbname /datadrive/backup/filename?
3) Once I successfully load the correct data in my Azure app, the final step, in my opinion, is to route traffic going to the Heroku app instead to the Azure app. For that, I'll tweak DNS entries. Am I missing anything else here, in your opinion?
Essentially the extension shouldn't matter, your restore should work but frankly haven't tested myself with a heroku backup.
However what I would suggest is lets make it a valid .dump file
curl -o latest.dump heroku pg:backups public-url --app <yourappname>
this should be your valid .dump file, though its not any different from the backup you already have..
We started using Redmine at work. I know it uses MySQL as the database, and Apache 2 as the web server. How can Redmine be properly backed up so that it can be reloaded quickly when anything goes wrong?
This will do just fine:
mysqldump --single-transaction --user=user_name --password=your_password redmine_database > backup.sql
It will dump the entire contents of the redmine_database to the backup.sql file.
Update:
As far as backing up "apache", as I state in my comment below - you don't need or want to back up your apache installation. If you ever need to recover your system, apache would need to be reinstalled as with any other application. If you are referring to the actual files and directories within your redmine installation, those as well don't need to be backed up except for the files/ directory which contains user uploaded files to redmine. You can backup your entire redmine installation (to be safe) with the following command:
tar czvf redmine_backup.tar.gz /path/too/redmine/installation
Run it as a VM (JumpBox has a quickstartable one, I believe) then periodically pause or shutdown the VM and backup/copy the entire virtual disk.
I know this doesn't help with an existing installation, but it's what I'd recommend to anyone planning backups before they implement. That's not meant to be snide, just helpful to anyone else reading this thread.
Bitnami apps are self contained, so another option if you can afford some downtime, is simply to shutdown the server, and zip the directory contents ... You may want to do this maybe once a week, in addition to your mysqldump backups. This way you also capture any changes that may have happened in Apache, etc.
Read the Redmine user guide (look at the bottom).
Also, don't forget to backup the attached files.
Redmine backups should include:
Data (stored in your redmine database)
attachments (stored in the files directory of your Redmine install)
Here is a simple shell script that can be used for daily backups (assuming you're using a MySQL database):
# Database
/usr/bin/mysqldump -u <username> -p<password> <redmine_database> | gzip > /path/to/backup/db/redmine_`date +%y_%m_%d`.gz
# Attachments
rsync -a /path/to/redmine/files /path/to/backup/files
Redmine sets table charset as "latin1".
So, if you use non-latin1 charset (CJK in UTF-8 or something), you should give following option to backup script.
mysqldump -u root -p --default-character-set=latin1 --skip-set-charset bitnami_redmine -r backup.sql
It skips "set charset blah-blah-blah" on sql dump and you would get a clean(=dump without interpretation) dump.
By the way, you have to back up the files directory as well; it holds all uploaded files. I installed the Bitnami Redmine stack on Windows.
For MySQL, I use MySQLAdmin to schedule database backup every day.
And I use aceBackup to automatic backup database dump files and Redmine uploaded files to a remote FTP server.
When the server is something wrong, I can just reinstall the Bitnami Redmine stack, and import early dumped database file, then cover Redmine's files directory with backup files.
And that's OK.
This separate program (Bitnami Redmine stack) and data (database & uploaded files) perfectly.