In my company's Django project, our models are currently spread across about 20 "app" folders with little rhyme or reason. We'd like to consolidate them into a single new app (along with the rest of our code) so that in the future, we can refactor all parts of our system at-will without worrying about violating an "app" boundary.
Due to how Django works, simply moving the model breaks everything. I've spent hours reading everything I can about the various proposed solutions to this problem, with the best article being:
https://realpython.com/move-django-model/
So far, I'm not happy with any of the solutions I've come across. They're all just too onerous.
If I do the following:
Move all models to the new app. (Let's call it "newapp".)
Update all references in the code to the previous locations. (Including migration files.)
Take down the site for a minute.
Run a script to rename all database tables from someprevapp_model to newapp_model.
Run a script to update app_label for all rows in the django_content_type table.
Deploy latest codebase version, with updated model locations and references.
Turn site back on.
Have I succeeded, or am I missing something?
UPDATE
Just remembered - I'd also have to:
5.5) Run a script to update app for all rows in the in django_migrations table (easy part), and to renumber all migrations to form a single sequence (hard part), as well as modifying all migration files to match new sequence.
UPDATE - V2 - Full Reset
Given the need to re-sequence migrations, combined with having already been wanting to squash our huge migration stack, lead me to this new and improved recipe:
Move models to new app. (Let’s call it “newapp”.)
Delete previous migrations.
Update code to reflect new model locations.
Run makemigrations to generate single migration in new app with all models.
Turn site off.
Run script to rename my tables from prevapp_model to newapp_model.
Run script to update django_content_type.app_label.
Run script to replace contents of django_migrations with single row indicating migration newapp-0001 has already run.
Deploy new version.
Turn site back on.
Related
I migrated a database that was manipulated via SQL to use Django's migration module: https://docs.djangoproject.com/en/3.0/topics/migrations/.
Initially I thought of using migrations only for changes in the model such as changes in columns or new tables, performing deletes, inserts and updates through SQL, as they are constant and writing a migration for each case would be a little impractical.
Could using the migration module without using data migrations cause me some kind of problem or inconsistency in the future?
You can think about data migration if you made a change that required also a manual fix on the data.
Maybe you decided to normalize something in the database, e.g. to split a name column to the first name and the last name. If you have only one instance of the application with one database and you are the only developer then you will also not write a data migration, but if you want to change it on a live production site with 24 x 7 hours traffic or you cooperate with other developers then you probably prepare a data migration for their databases or you will thoroughly test the migration on a copy of live data that the update will work on the production site correctly without issues and with minimal shutdown. If you don't write a data migration and you had no problem immediately then it is OK and will be not worse then a possible ill-conceived data migration.
I am currently working on a Django project with around 30 models and there are lots of relations(For example, foreign key relations) between the models.
My doubt is "After 6 months, let's say I want to add a new field(s) in one of the model/table(s) in models.py, and make migrations, the new migration files will get created without affecting the initial migration files which were created 6 months ago."
Will the relations be maintained after adding new columns in different tables? (or) do I have to go to pgadmin console and tweak them accordingly?
One way is to erase all the migrations and start fresh, but this is not recommended often especially if there is production data (or) there are frequent changes in the database schema.
#Mahesh, You may use --fake-initial to avoid the existing tables error at the time of new migrations(When you want to add new column).
Relation will be maintained since you already declared it and unless you change it in a new migration.
How to add a new field to a model with new Django migrations?
And in Docs: https://docs.djangoproject.com/en/3.0/ref/django-admin/#cmdoption-migrate-fake-initial
If you don't change Django version, adding new fields on models will not create any problem, even after many years. But, there are some situations this might create problems. For example, if Django is updated and you have installed the latest version.
Situation: a project I'm working in had a file corruption or something. Models are the latest version, but the SQLite DB had to be rolled back before some columns were added/removed/modified. Migration files are all gone. Trying to create migrations anew results in the new columns being present in the initial migration file, so I can neither migrate for real (due to the table existing) nor fake it (since the columns are missing in the DB).
Given those circumstances, how can I make an initial migration matching the columns currently present in the DB so I can fake it and then make a real second migration to bring the tables in line with the models? The only thing that comes to mind is manually tweaking the models to match the DB schema, making the initial migration, faking it and then restoring the new version of the models, but I'd much prefer having this done automatically.
django inpsectdb to the rescue.
But first, learn how to use git if you had used proper version control, you would not be facing this difficulty now.
First step, add the code to version control.
Delete the existing models files
Use inspectdb to generate a models.py from the tables in the database. This is not perfect, you will have to edit the file manually and you may have to spread it out between different models files manually.
Now delete the contents of the migrations table
do a ./manage.py makemigrations (yourapp)
Do the fake migration you mentioned
replace the generated models.py with your current models.py (a git checkout of that file will do the trick nicely)
do a makemigrations and migrate again.
good luck.
I have the following situation:
more than one schema migration
one data migration
It would be simple enough but I encountered a problem with the data migration. It sends a query for a specific ContentType which I need for django-taggit. The problem is that the model I want to query didn't exist until the migration that preceded it. That errors out with an empty result from that query.
However, when I run all migrations up to the data migration and then I run the data migration itself, everything works well. I've noticed that a migration process doesn't save changes until all of the migrations are finished which doesn't work for this.
One of the solutions I got to was to manually commit/save changes to the database however I haven't been able to find a way to do it. Of course, if there are any other ideas/better solution I'd be happy to hear them.
This is the code where the data migration errors out:
# ChallengeContest ContentType
challenge_contest_ct = ContentType.objects.get(model='challengecontest')
As you can see the model challengecontest is the one that was created in a migration preceeding data migration.
I have found data migrations to be more trouble than they're worth. In my last two jobs we abandoned them, replacing them with writing one-off management commands.
All I did was add a field to a model, and now I get an error that says this column does not exist.
In an attempt to rebuild the database I used -flush (i dont care about losing the data), thinking this would rebuild the database, but I still get the same error.
I was told by someone else to use South because I'm running Django 1.6.
I followed the tutorial and literally the first time I ran syncdb, I got the following (probably unrelated) error:
dist-packages/easy-thumbnails/
raise Improperly_Configured(SOUTH_ERROR_MESSAGE)
django.core.exceptions.ImproperlyConfigured:
For South Support, customize the SOUTH_MIGRATION_MODULES setting like so:
South_Migration_Module = {
'easy_thumbnails': 'easy_thumbnails.south_migrations',
}
Ultimately all I want to do is have my db reflect my models. Back when I was working on this project in my dev environment I would literally just drag my sqlite file to the trash and then run syncdb, but I cannot do that now because I'm using postgres.
So my question is how can I accomplish this seemingly simple task? Whether that means addressing the South error, or just not using South altogether (which I would prefer), I would appreciated any help.
You can manually open up a db shell (>>> python manage.py dbshell) and drop the table with DROP TABLE <table_name>;. In order for syncdb to recreate a table with the new field, it is not enough to empty the table.
To solve the error, just do exactly what it says: add the SOUTH_MIGRATION_MODULES setting to your settings.
With that said, I'd definitely advice you to use south. It eases making changes to your models, and it allows you to preserve test data in your development environment that you'd otherwise have to recreate. That's all nice, but when your project goes live, it is absolutely mandatory that your data is preserved when making a change to your models. South is the best tool for that.
South has been such an integral part of pretty much any Django project, that Django has worked together with the developers of South to include it as a core feature from Django 1.7 onwards.