I've got an app which has grown too big and I'd like to split it into two. It's not an easy task code-wise, but I'm more afraid of the database migrations I'll have to perform. Is there a proven way of doing such kind of thing?
Currently my thoughts are something like this:
Use south
Copy models to another app
Make a data migration to copy relevant models from one app to the other
Make another migration that removes the now unused models from the old app
Will have to make a script that does those operations in correct order
Has anybody done anything like this?
No need to do any database migrations if you don't want to. Just set the db_table Meta attribute on your models to refer to the old names, and everything will be perfectly happy.
Related
I'm just beginning my journey with Django framework and I read that Django developers have made using migrations mandatory beginning from version 2.0. I might be old school but I like my database separate from my code. I have always kept my database separate from my code models. I think that the migrations won't scale with the engineering team size.
So my question is 2 fold.
Can you not use Django 2.0 without the migrations as I don't think it will scale well and won't fit the CI/CD pipeline?
If we can't avoid the db migrations then how can we integrate them in a robust CI/CD pipeline where a model can be changed by different developers from different teams.
Yes, you can. You can create your tables manually and set Django to not manage your tables.
After your Django project is configured, just run on your terminal python manage.py inspectdb > models.py, and django will pick the models on the configured database. This is particularly good if your project will use a already existing or legacy database
Then, you can tell django to not manage your tables on the meta options of the model:
class MyModel(models.Model):
# your fields here
class Meta:
managed = False
See the docs here
But, unless you have a very good way to keep track of your table changes, I must say this is a mistake. Django migrations help you to keep track on your models changes along the way. It is really helpful if you need to rollback or understand your database history.
Migrations are not mandatory, it's not clear what you think has changed in 2.0 to make them so.
Migrations are intended for large teams. If you avoid them, you'll make things much much harder for yourself and your fellow team members.
I can think of three reasons why:
providing users with the flexibility on "when" to commit model changes
debugging modularity
perhaps resource consumption in larger
databases
However, it does seem that migrate always follows shortly after migration (tutorials/youtube videos).
so is there a philosophy behind this that I'm missing?
Ofcourse there are some reasons.
First of all, 'makemigrations' doesn't touch real DB, it just tells django how models(db scheme) have changed so you can see what's going on when you do 'migrate'.
and this makes django more safe.
This also provides to make default options for new fields or db changes..
Other reason is 'revert'.
If you want to roll-back db scehme with specific migrations, you can just tell django to roll back to specific migration file.
Another reason is 'reusable-app' principle.
If you create app with django and it could be reusable with no-db-interaction. It means if you deploy your app(or project, too!) to another project or server, it just needs 'migrations' files not real db.
Our product has a restful API and a server rendered app (the CMS). Both share the database. Both are written in django
The fields and the models needed in both are not mutually exclusive, there are some only particular to the API, some particular to the CMS, and some which are common.
My question is if I run migrations on one of the repos will they try to drop the fields that aren't present in the models of that particular repo, and needed by the other. Will running the migrations individually in both repos keep the database up to date and not pose a problem.
The only other valid option IMHO (besides merging projects) is turning off automation of Django migrations on common models (Meta.managed = False) and taking table creation & versioning into your own hands. You still can write migration scripts using django.db.migrations but makemigrations command won't do anything for these tables.
This was solved by using a schema migration tool external to Django's own. We use
yoyo migrations to migrate our schema now.
Will running the migrations individually in both repos keep the database up to
date and not pose a problem.
Unfortunately, no. As you suspected, changes in one will attempt to override the other.
The easiest thing to do is merge the two projects into one so this problem goes away entirely.
If this isn't an option, can the code be organised in such a way that both projects share the same models.py files? You could do this by perhaps having the models.py files and migrations folders only exist in one project. The second project could have a symlink across to each models.py file it uses. The trick (and the difficult part) will be to make sure you never create migrations for the app which uses the symlinks.
I think the best things to do would be to have one repo that contains all the fields. This project will be responsible to apply the migrations.
In the other projects, you'll need a db_router containing a function allow_migrate which will return False on your model classes.
Also having different db user with different db permissions can prevent from altering the tables.
Noob here... :)
I'm working on a small and personal project that is already in "production", but development still is under way. In the last weeks I've managed to handle the updates in a hacky way. Usually, I make a dump of the (still small) database into json files, separed by app or sometimes by table, drop everything in the database, implement the model's changes in json level trough scripts, syncdb a new database, and put everything back on. I known, it's dumb, but I'm lacking knowledge of a better alternative. So, now that I'm borderline insane with this strategy I come to you guys.
I've looked into South, but I failed to understand how exactly is it's workflow regarding the Data migration (in opposition of it's schema migration that is obvious).
So, how do you guys do it?
Thanks in advance.
South creates python scripts. So you can use South to create schema migrations, and then change these scripts to include your own data migration.
If you just add models and fields you don't need to do this, you can just use plain South.
Due to my little confidence with Django and my sheer horror at the thought of seriously messing up my beautiful project, I shall ask for proper advice/instructions here.
The database that my Django project is sitting on top of has been changed (a few field types have been changed) and my models are now out-of-sync. Funnily enough, my Django still works (God knows how) but I still want to update the models. How do I go about doing this the proper way. Thank you very much indeed in advance.
Marked as answered. My actual discover was:
./manage.py inspectdb > <file>
//Hands you all the tables from the database.
//Then you update the models accordingly.
SIMPLE! :)
It's probably a bit late, but you might want to take a look at South, which is a migrations system for Django.
The normal practice for your situation would be to run manage.py reset appname, where appname is the name of the app which contains the models you've changed. You'll obviously want to dump the data in the affected tables first (find out what tables are going to be affected by running manage.py sqlreset appname).
Finally, it's quite possible your site is still running happily because you've not restarted the webserver (I'm assuming you're talking about a production environment, the development server reloads most changes automatically).
If you've already made the changes to the live database, you can probably just change the models and restart your webserver.
As long as your Field names match between the database and the models you shouldn't have any issues.
That being said, it is a much better idea to use a migration tool like south (as Dominic suggested already)