how can i migrate entiry db at one step? south`s startmigration command can work only with single application
If you want to create a migration, manage.py startmigration is deprecated (see manage.py help startmigration), you should be using manage.py schemamigration (as described in the docs), and you should definitely be doing this individually on each app.
If you want to run migrations (in other words do the actual altering of the database, which I'm guessing is the case), The command for that is manage.py migrate which, if run without any arguments, will migrate all of your apps to the latest migration available.
My deployment script just has manage.py migrate, and it works fine without manual intervention, no matter how many apps have new migrations that need to be run.
Even with raw SQL, you wont be able to migrate an entire database in a single step as you need a query per table. You can however, create migrations for all apps, and then run them all at once. That's the closest thing you'll come to a one step migration.
Related
I have a django REST project and a PostgreSQL database deployed to DigitalOcean. When I develop locally, I have a separate dockerized REST server and a separate PostgreSQL database to test backend features without touching production data.
My question arises when I'm adding/modifying model fields that require me to make migrations using python [manage.py](https://manage.py) makemigrations and python [manage.py](https://manage.py) migrate command. Here is my current situation so far:
What I was supposed to do
IN LOCAL ENV, to create the migration files,
python manage.py makemigrations
python manage.py migrate
Now commit these newly created files, something like below.
git add app/migrations/...
git commit -m 'add migration files' app/migrations/...
IN PRODUCTION ENV, run only the below command.
python manage.py migrate
What I did so far
IN LOCAL ENV, created the migration files,
python manage.py makemigrations
python manage.py migrate
I committed & pushed the changes to production WITHOUT the created migration file
IN PRODUCTION ENV, ran BOTH commands.
python manage.py makemigrations
python manage.py migrate
The production server successfully added the isActive field to the database and is working fine, but I still have a 0011_user_isActive.py migration file in my local changes that hasn't been staged/committed/pushed to github repo.
And because I ran makemigrations command in production env, it probably created the same migration file that I haven't pushed from local env.
My questions are:
What happens if I push the local migration file to production? Wouldn't it create a conflict when I run migration command on digitalocean console in the future?
How should I fix this situation?
I am just scared I'm going to corrupt/conflict my production database as I'm very inexperienced in databases and have too much to risk at the moment. Would appreciate any tips on best practices when dealing with such situations!
As docs says:
The migration files for each app live in a “migrations” directory inside of that app, and are designed to be committed to, and distributed as part of, its codebase. You should be making them once on your development machine and then running the same migrations on your colleagues’ machines, your staging machines, and eventually your production machines.
So it's best practice is to push your migration files and make your local and production migration files sync.
And if you got conflict when pushing migraions files and pulling them, the makemigrations --merge command is for solving that.
Also docs says:
Because migrations are stored in version control, you’ll occasionally come across situations where you and another developer have both committed a migration to the same app at the same time, resulting in two migrations with the same number.
Don’t worry - the numbers are just there for developers’ reference, Django just cares that each migration has a different name. Migrations specify which other migrations they depend on - including earlier migrations in the same app - in the file, so it’s possible to detect when there’s two new migrations for the same app that aren’t ordered.
When this happens, Django will prompt you and give you some options. If it thinks it’s safe enough, it will offer to automatically linearize the two migrations for you. If not, you’ll have to go in and modify the migrations yourself - don’t worry, this isn’t difficult, and is explained more in Migration files below.
Also be aware that in case of updating existed data in production, you can use RunPython in migration file. Read about it here.
So I've created a new model in Django, then executed both python manage.py makemigrations and python manage.py migrate in the right order. But then for some reason I accidentally dropped the table(relation) in PgAdmin (I know it sounds silly). So I tried deleting all the files in migrations folder but the init.py file. Then I ran the two commands above again, but could see no table in the PgAdmin. What should I do, aside from creating a table myself in PgAdmin?
Thanks in advance.
It won't work, because entry for all the migrations are already stored inside a table named django_migrations. So even if you run makemigrations after deleting all the migration files, it won't create a new one. So here is three ways you can fix it.
One: Drop DB(if data is not important)
Drop database and create a new one. Run makemigrations and migrate command.
Two: Create that table manually
If you already deleted all the migration files, you better restore them. You can use git for this: git checkout /path/to/migration/folder. Then you can manually create the table.
Three: Delete entries from django_migrations
I assumed you have deleted all the migration files. So this part covers the whole project. But #DenizKaplan has explained better way to do this.
If you are not using git, or no way to restore these files, then you can follow these steps:
Backup your database
Delete all entries from djang_migrations table
Run ./manage.py makemigrations to generate migration file
Run ./manage.py migrate <your lost app> to migrate your app(which you have lost in DB).
Run ./manage.py migrate --fake to fake the rest of apps.
There are some steps you need to check. If you have an empty output after makemigrations operations, you may need to check for django_migrations table to remove rows related to apps that you have working with. If this won't help at first place, you need check INSTALLED_APPS, maybe you may accidently delete apps. If all these not work, please give some detailed information about the error.
I deleted that table (whoops), and now I'd like to regenerate it. Is there a command that allows me to do that? Looking online, everyone is saying to use the command ./manage.py syncdb, but that command is no longer available in the most recent version of Django. So I tried ./manage.py migrate, but that didn't generate the table. I also tried ./manage.py --run-syncdb, but that didn't do it either.
I'm pretty sure I can do it by hand, but I'm hoping there's a way to do this with a built-in command.
Since the admin app only has one table, django_admin_log, you can revert all migrations for the admin app by running
python manage.py migrate admin zero
then re-apply the the admin app migrations by running
python manage.py migrate
You might want to create a backup before doing this (or any migration, really) :)
When running python manage.py migrate I encounter this error:
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration
<appname>.0016_auto_<date2>_<time2> is applied before its dependency
<appname>.0001_squashed_0015_auto_<date1>_<time1>
running showmigrations returns:
<appname>
[X] 0001_squashed_0015_auto_<date1>_<time1> (15 squashed migrations)
[X] 0016_auto_<date2>_<time2>
[ ] 0017_<modelname>_squashed_0019_auto_<date3>_<time3> (3 squashed migrations)
I was trying out django-extensions yesterday, when it all got messed up after me running some direct SQL queries and I reset hard using git. I'm still learning about migrations, so I don't understand what is wrong, since it seems to me that both migrations already have been applied.
Thank you for your help!
This worked for me. I thank my coworker for sharing this knowledge after I searched online for many hours.
Start your db shell
python manage.py dbshell
Use the database you want. If you don't know, run .databases (SQLite) or SHOW databases
mysql>use <database_name>;
Retrieve all the migrations under your app
mysql> select * from django_migrations where app='<app>';
You will see the output with ids next to all migrations. Look at the migration you want to drop. Say the id is 361
mysql> delete from django_migrations where id=361;
You have squashed the migrations, so one of the dependencies that 0016_auto_<date2>_<time2> had is now part of the newly created squashed migrations. Meanwhile the 0016_auto_<date2>_<time2> has already been run and now you're trying to run the squashed migration.
I personally don't know if there's any way to fix this automatically. You will need to fix the issues yourself. If you have version control, revert these changes and try to rethink how you should squash the migration without affecting old ones.
I have solved this problem when i did (custom user model) by this steps:
delete this file :
migrations\0001_initial.py
delete this :
db.sqlite3
put this code in settings.py :
AUTH_USER_MODEL = 'users.CustomUser'
Then do (makemigrations) then (migrate )
run server .. the problem solved :)
i have used this link it is help me to solve the problem of dependency :
https://docs.djangoproject.com/en/3.1/topics/auth/customizing/
Due to limitations of Django’s dynamic dependency feature for swappable models, the model referenced by AUTH_USER_MODEL must be created in the first migration of its app (usually called 0001_initial); otherwise, you’ll have dependency issues.
In addition, you may run into a CircularDependencyError when running your migrations as Django won’t be able to automatically break the dependency loop due to the dynamic dependency. If you see this error, you should break the loop by moving the models depended on by your user model into a second migration. (You can try making two normal models that have a ForeignKey to each other and seeing how makemigrations resolves that circular dependency if you want to see how it’s usually done.)
run this python manage.py dbshell
INSERT INTO public.django_migrations(app, name, applied)
VALUES ('YOUR_APP_NAME, '0017_<modelname>_squashed_0019_auto_<date3>_<time3>', now());
and you should be fine. If Your migration was changing a lot to the database, then I am afraid it won't be that easy to fix it.
you need to fake migrations and migrate again
just make sure that you have a backup from your data because when you migrate again you need to delete apps table.
make sure that you look at show migrations and migrate un migrated apps by its sequence
Edit the dependencies of the conflicting migration, so that it no longer references the already applied migration.
Then run python manage.py migrate again and it should be fixed.
Warning: this only work suppossing that the state of the database matchs the state you get having applied the conflicting migration.
I had the same issue on 2020 with Django 3.0.6.
I tried all the relevant answers with no success. So I went in my database and deleted all the tables. You must export the relevant tables if you have done lot of work. I mainly delete django files in my database. And after, run:
python manage.py makemigrations <my-app>
And:
python manage.py migrate
Export your relevant tables if any.
First back up your database before resolving the conflicts, (Use "python manage.py dumpdata > db.json" for SQLite).
Execute python manage.py dbshell, to access the database.
Delete the migrations rows that are having conflicts from the django_migrations table.
Rename the tables conflicting in the database
Execute the makemigrations and migrate commands
After successful migrations, Drop the newly readded tables and finally restore the previously renamed tables to match the migrations need
I had the same problem, and here's how I solved it.
The following is my error message
File "/usr/local/lib/python3.11/site-packages/django/db/migrations/loader.py", line 327, in check_consistent_history
raise InconsistentMigrationHistory(
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration aaaa.0024_campaign_template is applied before its dependency bbbb.0005_templatemodel_from_template on database 'default'.
My solution
python manage.py migrate bbbb
python manage.py migrate
Because I changed the Django's app name in batches, the application order was not consistent when applied to the database. The bbbb that aaaa relies on was not created first, so I manually created the bbbb first
Migration file is not created for all app:
step 1:
create migration folder and add __init__.py file for all app
step 2:
delete db.sqlite3 database
step 3:
python manage.py migrate
python manage.py makemigrations
Delete all of your migrations folder
Delete the database(sqlite3)
Then run the makemigrations and migrate command
Delete the migration files.
Run:
python manage.py migrate
python manage.py makemigrations
python manage.py migrate
python manage.pyrunserver
I am using django-south for migrating database tables in a django project. And I am renaming a model as discussed in a previous question:
# Renaming model from 'Foo' to 'Bar'
db.rename_table('myapp_foo', 'myapp_bar')
db.send_create_signal('myapp', ['Bar'])
However, I use fabric to automatically deploy my application to production servers, and I want the migrations to run without any user input. For this, I run the migration command with the noinput option as follows
python manage.py migrate --noinput
This works fine except that the send_create_signal does not remove stale contenttypes in this mode.
This is because the django contenttype managament command update_contenttypes only removes the stale contenttypes if input is given.
I could replicate the update_contenttypes command directly in the south migration, but that does not seem like a good solution. Does anyone have suggestions on how to trigger the removal of the contenttypes without repeating what is in the django command?
In my experience, running manage.py syncdb --all works some, but not all of the time when South is involved. You might try giving it a go, as it has worked for me in the past, certainly when removing stale models from the content-types table.