So I have a client who cloned an old project in Django to start working on a brand new theme and features. Now they're needing to move the data from the old project to the new one (User and other similar tables). Some of the main models on the old project have significant changes on the newer version. I want to ask if there's any good route of doing this kind of migration?
Sure, here it is in several easy steps using south.
Clone the old project and database (or at least, make a backup)
Put the old models into south.
Create migrations, step by step for your changes with South, until your models match the new models.
(on old/converted project) python manage.py dumpdata myapp > myapp.json
Move the json fixture to the new project.
(on new project) python manage.py loaddata myapp.json
Related
I'm new to Django but one concern I'm running into is whether running makemigrations as part of the production deployment process is safe/deterministic?
The main reason is some 3rd party apps will create new migrations beyond the pre-defined ones, i.e. hordak for example, when a new currency is added via CURRENCIES in settings.py.
Since a currency can be added anytime later and these are part of an 3rd party app the migrations are not checked in to the repo.
This requires makemigrations to be run on deploy? This seems dangerous since there will be untraceable changes to the DB?
Is a way to explicitly check-in to the repo 3rd party migrations?
I found this question: How to store third party apps migrations in django
Using MIGRATION_MODULES in settings.py you can relocate 3rd party migration files removing the need to perform makemigrations in production.
While developing a Django project, all your migrations are stored within each app folder, however, in production I don't want those migrations, I want to keep a Production database, and a Development database:
How do I handle Django migrations in a Production and Development environment?
I'm asking this question because it's been really hard to update my deployed project with new additions in the development one, my ideal scenario would be to keep each set of migrations in a folder outside my source code, just like the databases.
The best idea is to keep production and development migrations the same and while developing you clean migrations before pushing the code and you should push migrations into your Version Control System too.
In development, you might end up deleting a table and re-creating it so make sure you don't push the un-intended migrations. The thing is you should treat migrations as code, not an automated script. I have done a lot of mistakes in the past, so, I came to the conclusion of including migrations in code. and that's effective and gives more control.
Moreover you might have to do data migrations in production, how will you do if you wont push the code?
I'm having trouble with my Django-app that's been deployed.
It was working fine but I had to do a minor modification (augmented the max_length) of a Charfield of some model. I did migrations and everything was working fine in the local version.
Then I commited the changes without a problem and the mentioned field of the web version now accepts more characters, as expected, but whenever I click the save button a Server Error rises.
I assume I have to do some kind of migration/DB update for the web version but I don't seem to find how.
(I'm working with Django 1.11, postgresql 9.6, and DigitalOcean).
EDIT
I've just realized that the 'minor modification' also included a field deletion in the model.
Short answer
You have to run
python manage.py migrate
on the server, too. Before you do that, make sure all migration scripts you have locally are also present on the server.
Explanation
After changing the model, you probably locally ran
python manage.py makemigrations
This creates migration scripts that'll transform database schema accordingly. Hopefully, you've committed these newly created scripts to Git, together with the changed model. (If not, you can still do so now.)
after running makemigrations (either before or after committing, that shouldn't matter), you've probably locally ran
python manage.py migrate
This applies the migration scripts to the database that haven't been applied to it, yet. (The information which ones have already been applied is stored in the database itself.)
You probably (and hopefully) haven't checked in your local database into Git, so when you pushed your tracked changes to a remote repo and pulled them down on your server (or however else the new Git revisions got there), the changes to the server database haven't happened, yet. So you have to repeat the last local step (migrate) on the server.
Further reading
For more information, refer to the Django 1.11 documentation w.r.t. migrations. (You can e.g. limit migration creation or migration application to a single Django app, instead of the whole Django project.) To get the grip of these things, I can recomment the free Django Girls tutorial.
We have migrated the Django 1.4 application to Django 1.8 successfully. The Django 1.4 version of applicaiton is still in use in production until we go live with Django 1.8. The problem is that lots of data have been updated on production server which needs to be migrated to version 1.8. Is there any way I can migrate the data from database of 1.4 to 1.8 except manually doing that? Note that the model/database columns are different in both the version.
Can anybody suggest some good options ?
Thanks.
Required pre-reading
Django migrations:
https://docs.djangoproject.com/en/1.8/ref/django-admin/#makemigrations-app-label
Assuming you were using South:
https://docs.djangoproject.com/en/1.8/topics/migrations/#upgrading-from-south
Getting Started
Firstly dump your local database I prefer using mysql/postgres/whatever docs for this rather than using ./manage.py dumpdata.
You will also want to dump your production database as well just for safekeeping.
Next in your local environment I would dump the database and create a new database.
Then I would test that all your migrations actually work on a blank database.
These are the instructions for django 1.8
./manage.py makemirgrations
./manage.py migrate
That will help show up if any migrations are in an inconsistent state operating from a blank slate. If you encounter any errors they should be fixed first.
Given that works, now I would test that your migrations actually work against your production data.
So drop your local database, create a fresh one then load the production dump in.
If the tables are already configured correctly (i.e. they are in an up to date state in your production database) then you will need to fake all migrations.
./manage.py migrate --fake <appname>
However given that you have changed some models since upgrading to 1.8 in your local environment, then you might need to fake only some of the migrations. This could be the tricky part depending on the timing of when you upgraded and when you created the migrations.
Because django 1.7 will just create an initial migration for each app, you may need to actually break up the migration for some apps. That is, instead of 0001_initial you might need to manually break up that migration into 2 components:
1. a migration to match the current state of your production database
2. a migration to match any additional changes you have made to your model since then.
One way to do this is to checkout your first commit once you had django 1.8 working correctly locally then run
./manage.py makemigrations
then commit that
then go forward to your latest commit then run
./manage.py makemigrations
Now you should have 2+ migrations in each app that you have modified since upgrading to django 1.8.
Then you can fake initial on those apps that have 2+ new migrations for django 1.8
./manage.py migrate --fake-initial app1 app2
and the rest just
./manage.py migrate app3 app4
Now run your tests to confirm everything is working locally.
If you have changed migrations you will again want to test locally against a blank database to test that they work smoothly
Once that is working, record the 'migrate' commands that you used - then deploy your app to production and run just those migration commands once you have upgraded to django 1.8 on your server.
After Successful Completion
Take new dumps of your local and production databases
Uninstall South (assuming you had it installed before) from local and production environments
I'm sure there are a couple of holes in the above but hopefully that gives you the gist of what you need to do.
I am developing a Django app that can be installed via pip for use within a larger project.
I have DB models in this app. I plan to use South in case I need to do any migrations in future versions.
Should I make South an install requirement and then have the user manually run migration(s) or should I do it automatically in the setup.py?
Running migrations is a part of an application setup/startup routine, in my opinion it is out of the scope of installation.
If I was the user, I prefer one setup script can figure everything out. And it is not a complex function, why not do it automatically.