So I've created a new model in Django, then executed both python manage.py makemigrations and python manage.py migrate in the right order. But then for some reason I accidentally dropped the table(relation) in PgAdmin (I know it sounds silly). So I tried deleting all the files in migrations folder but the init.py file. Then I ran the two commands above again, but could see no table in the PgAdmin. What should I do, aside from creating a table myself in PgAdmin?
Thanks in advance.
It won't work, because entry for all the migrations are already stored inside a table named django_migrations. So even if you run makemigrations after deleting all the migration files, it won't create a new one. So here is three ways you can fix it.
One: Drop DB(if data is not important)
Drop database and create a new one. Run makemigrations and migrate command.
Two: Create that table manually
If you already deleted all the migration files, you better restore them. You can use git for this: git checkout /path/to/migration/folder. Then you can manually create the table.
Three: Delete entries from django_migrations
I assumed you have deleted all the migration files. So this part covers the whole project. But #DenizKaplan has explained better way to do this.
If you are not using git, or no way to restore these files, then you can follow these steps:
Backup your database
Delete all entries from djang_migrations table
Run ./manage.py makemigrations to generate migration file
Run ./manage.py migrate <your lost app> to migrate your app(which you have lost in DB).
Run ./manage.py migrate --fake to fake the rest of apps.
There are some steps you need to check. If you have an empty output after makemigrations operations, you may need to check for django_migrations table to remove rows related to apps that you have working with. If this won't help at first place, you need check INSTALLED_APPS, maybe you may accidently delete apps. If all these not work, please give some detailed information about the error.
Related
I've got a Django app with a lot of out-of-date migrations. I'd like to remove the old migrations and start fresh.
The app has 14 different "migrations" folders.
Here is what a few of them look like:
Is it safe to remove all the contents from each of these folders? Or, do I have to make sure to only remove some of the files -- and if so which files?
You should never just delete migrations before unapplying them, or it will be a nightmare when you want to apply new migrations.
To unapply migrations you should do the following:
Use the python manage.py migrate your_app_name XXXX in case you want to unapply migrations after the XXXX migration. Otherwise use python manage.py migrate your_app_name zero to completely unapply all migrations.
Remove the .pyc files under /migrations/_pycache_/ that you have unapplied.
Remove the .py files under migrations/ that you have unapplied.
Now you can create new migrations without any headaches.
If what you're looking for is to squash all the migrations into one, do the steps above removing all migrations and then run python manage.py makemigrations your_app_name to create a single migration file. After that just run python manage.py migrate your_app_name and you're done.
That depends. If you have a production database (or any database you cannot simply drop and recreate), then the answer is no, you cannot safely remove migrations.
If you do not have any permanent databases, then yes, you can remove all migrations, run python manage.py makemigrations --initial and it will create fresh migrations based on your current models.
Also, you should check if any of the migrations are custom data migrations written by hand. If there are any, you might want to keep those.
The .pyc files are generally safe to remove, provided the related .py files are still there.
your first screenshot is not Django and looks like a JS project of some sort.
The json and js files are unrelated to the django migrations as well as __pycache__ folder. You can delete all off them.
If you mean "previously applied and no longer needed as the project only needs the latest version of the migrations" you don't want to remove but squash them instead with squashmigrations which reduces the files you have to two, init file and the initial migration file, this way your project still works.
If by remove you mean you no longer need them because you already changed the models so much that the previous migrations aren't even used other than being applied and unapplied without ever being used, doesn't matter, go to step 2 and do that instead of deleting the files manually. When you create migrations on your applications one by one, you also create migration dependency tree, well, django does. And it is really hard to keep track of after some point, if you try to delete everything thinking you can create new migration files with ease, trust me as someone who experienced otherwise, it does not work like that. It is way simpler to let django handle the migration squashing, it optimizes the migration meaning that it also deletes the unused ones in your final state.
More to read at: https://docs.djangoproject.com/en/2.2/topics/migrations/#migration-squashing
Having marked one of the answers provided previously as being accepted, here is a summary of a few things I learned:
Deleting Django migrations is generally a bad idea.
Django keeps track of what's in your db through these migration files, as well as through a table it creates in your db, and if you delete any of this Django will start throwing errors on migrate that can be hard to fix.
I was getting some of those hard-to-fix errors. Here is what I did to fix it:
Ran migrate on the production server.
When I got an error, it would tell me how the db was out of sync with what Django expected. I corrected that manually by directly editing the db with an sql client.
E.g. If it said a key existed that wasn't supposed to exist, I deleted the relevant index from the indicated table.
Or if it said a table existed that wasn't supposed to exist, I backed up the table to a file, and deleted the table. Migrate then created the table, and then I repopulated it with data from the backup.
In the case of many-to-many tables, once Django had re-created them, I deleted all the new Django-created tables, and restored them from a backup created on my local dev system, which had already had all the latest migrations run on it.
Eventually I was able to complete all migrations successfully.
I have a feeling I lucked out and the above won't work in all cases! I've learned a lot about Django and migrations and will be much more careful about this in the future.
when you import from third app:
there are 2 step uninstall it
there are use the 'django_celery_beat' app for example.
step1: clean table
python .\manage.py migrate django_celery_beat zero
step2: remove app from INSTALLED_APPS
there are done!!!
this is django document on this.
How to Reset Migrations
if you are using linux/unix os then you can fire this command. delete all migration directory.
find . -path "/migrations/.py" -not -name "init.py" -delete
find . -path "/migrations/.pyc" -delete
one of my fellow developers checked out from master and created new models for our website. He ran makemigrations, then ran migrate which obviously created the tables we wanted.
However, he never committed his changes to github and he altered the production database. So when I went in to add a table today, when I ran makemigrations it the terminal listed several tables that I knew already existed...I was like "YOLO!" and ran the migrate command anyways and it puked.
So, since the migrations file isn't in my migrate folder, django thinks it needs to create those tables...then it goes to create them and pukes because they're already there.
The other developer is out of town visiting family and can't commit the file.
How do I get this set straight? I think I need to run ./manage.py migrate my_app --fake
But I don't completely understand what that does so I don't want to take the YOLO route and really mess things up...
OK, I promise everybody out there that I have been working on this problem for 9.5 hours today. Turns out this was the answer:
Django migrations : relation already exists
However, there were some spelling/syntax errors that made it difficult to understand that this other person had the same problem as me.
to reiterate the solution:
type: ./manage.py makemigrations your_app
Navigate to the my_app/migrations folder and open the migrations file that was just created (usually looks like '0005_modelsandstuff_blablabla.py"
Delete the models that DON'T ALREADY EXIST. Save the file and close
type: ./manage.py migrate your_app --fake
this will then sync what is in the database now with your models schema without altering any of the actual database
type: ./manage.py makemigrations your_app
type: ./manage.py migrate your_app
And that's it! Everything is all synced up again. Just as a quick jab I would like to say JavaScript sucks. Thanks.
Following these steps should solve your problem.
Backup your database
Stash your changes (so that only the missing schema changes are picked up)
Create the migrations (this creates the already applied schema changes)
Run migrate with --fake (this will fake apply the already done schema changes)
Apply your changes
Create the migrations
Run migrate
I cloned a postgres db and added a new model to one of the apps. Our project contains many apps.
Now When I ran migrations, migrate it would fail. So I commented that model out, deleted migrations.py file from folder and ran fake migrations. Than again put that model in and ran migrations, migrate. Things were fine.
But now I deleted this model table from db manually and when would run migrations it would show model doesn't exist.
Basically I would need to again and again tweak the model, delete, update table.
So I searched for migrating from scratch. Did delete some apps from django_migrations table. But it is not working out it shows relations already existing.
This is all becoming confusing, --fake, delete, squash what to do?
Basically if I drop table django_migrations, delete migrations folder from app. Can't django automatically sync with db and understand what model exist and what don't and figure it out itself.
If you want to create a clone of a Django database, without the data, there's one table you should always get the data from: django_migrations. This is the table that holds the state of the database: which migrations have been applied and as such what models exist already.
The fact that Django tries to create one it's core models tells me you didn't have the data from django_migrations in the clone.
This solved the problem.
deleted all the data from 'django_migrations' table.
deleted all the migrations file from all migrations folder in different apps
Commented out the models whose table are still not in database
ran python manage.py makemigrations
ran python manage.py migrate --fake
Put the comments off the model
ran python manage.py makemigrations
ran python manage.py migrate
Thing is to understand what does makemigrations and migrate does. And then you can delete/update manually or created new models and migrate/clone things etc. it would work. For a new starter these commands are little confusing.
Let's says everything is empty i.e. no files in migration folder of app.
Than makemigrations makes these files which contains say sqls/syntax on how to create your table. With each change it will create a new one, with only the changes listed in that.
migrate creates actual empty tables in the database. And creates an entry in django_migrations against the app that model was part of. This entry tells it when was the latest entries from that file were applied to the database.
Scenario 1:
I deleted one table manually from db. Now django doesn't know about it. So if you would run makemigrations or migrate it would do nothing.
Scenario 2:
I deleted all entries from django_migrations, deleted migrations files from app. Tables are still in db. Now when I do makemigrations it works, but when I migrate it throws error that already existing.
Scenario 3:
Have deleted entries from django_migrations and migration files. Django has one option of migrate --fake, this tells it that tables are already existing just make entries in django_migrations table. so makemigrations and migrate --fake works now without error. But now the table which I deleted manually is still deleted, django hadn't made it. So will throw error when I try to access it.
Scenario 4:
The one I described at start. I faked the entries which were in db and then migrated the model which wasn't in db.
So once one understands what's it's doing behind the scenes many approaches could have been followed, create the table schema manually or selectively pick django_migrations or migrations file entries and delete them. But best is to delete everything and start from scratch.
It would have been best that django understands that these are tables already created, these are models I have, create the missing one, leave the existing ones. If any discrepancy show them. I see so many questions related to migrations and people are confused here and there.
When running python manage.py migrate I encounter this error:
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration
<appname>.0016_auto_<date2>_<time2> is applied before its dependency
<appname>.0001_squashed_0015_auto_<date1>_<time1>
running showmigrations returns:
<appname>
[X] 0001_squashed_0015_auto_<date1>_<time1> (15 squashed migrations)
[X] 0016_auto_<date2>_<time2>
[ ] 0017_<modelname>_squashed_0019_auto_<date3>_<time3> (3 squashed migrations)
I was trying out django-extensions yesterday, when it all got messed up after me running some direct SQL queries and I reset hard using git. I'm still learning about migrations, so I don't understand what is wrong, since it seems to me that both migrations already have been applied.
Thank you for your help!
This worked for me. I thank my coworker for sharing this knowledge after I searched online for many hours.
Start your db shell
python manage.py dbshell
Use the database you want. If you don't know, run .databases (SQLite) or SHOW databases
mysql>use <database_name>;
Retrieve all the migrations under your app
mysql> select * from django_migrations where app='<app>';
You will see the output with ids next to all migrations. Look at the migration you want to drop. Say the id is 361
mysql> delete from django_migrations where id=361;
You have squashed the migrations, so one of the dependencies that 0016_auto_<date2>_<time2> had is now part of the newly created squashed migrations. Meanwhile the 0016_auto_<date2>_<time2> has already been run and now you're trying to run the squashed migration.
I personally don't know if there's any way to fix this automatically. You will need to fix the issues yourself. If you have version control, revert these changes and try to rethink how you should squash the migration without affecting old ones.
I have solved this problem when i did (custom user model) by this steps:
delete this file :
migrations\0001_initial.py
delete this :
db.sqlite3
put this code in settings.py :
AUTH_USER_MODEL = 'users.CustomUser'
Then do (makemigrations) then (migrate )
run server .. the problem solved :)
i have used this link it is help me to solve the problem of dependency :
https://docs.djangoproject.com/en/3.1/topics/auth/customizing/
Due to limitations of Django’s dynamic dependency feature for swappable models, the model referenced by AUTH_USER_MODEL must be created in the first migration of its app (usually called 0001_initial); otherwise, you’ll have dependency issues.
In addition, you may run into a CircularDependencyError when running your migrations as Django won’t be able to automatically break the dependency loop due to the dynamic dependency. If you see this error, you should break the loop by moving the models depended on by your user model into a second migration. (You can try making two normal models that have a ForeignKey to each other and seeing how makemigrations resolves that circular dependency if you want to see how it’s usually done.)
run this python manage.py dbshell
INSERT INTO public.django_migrations(app, name, applied)
VALUES ('YOUR_APP_NAME, '0017_<modelname>_squashed_0019_auto_<date3>_<time3>', now());
and you should be fine. If Your migration was changing a lot to the database, then I am afraid it won't be that easy to fix it.
you need to fake migrations and migrate again
just make sure that you have a backup from your data because when you migrate again you need to delete apps table.
make sure that you look at show migrations and migrate un migrated apps by its sequence
Edit the dependencies of the conflicting migration, so that it no longer references the already applied migration.
Then run python manage.py migrate again and it should be fixed.
Warning: this only work suppossing that the state of the database matchs the state you get having applied the conflicting migration.
I had the same issue on 2020 with Django 3.0.6.
I tried all the relevant answers with no success. So I went in my database and deleted all the tables. You must export the relevant tables if you have done lot of work. I mainly delete django files in my database. And after, run:
python manage.py makemigrations <my-app>
And:
python manage.py migrate
Export your relevant tables if any.
First back up your database before resolving the conflicts, (Use "python manage.py dumpdata > db.json" for SQLite).
Execute python manage.py dbshell, to access the database.
Delete the migrations rows that are having conflicts from the django_migrations table.
Rename the tables conflicting in the database
Execute the makemigrations and migrate commands
After successful migrations, Drop the newly readded tables and finally restore the previously renamed tables to match the migrations need
I had the same problem, and here's how I solved it.
The following is my error message
File "/usr/local/lib/python3.11/site-packages/django/db/migrations/loader.py", line 327, in check_consistent_history
raise InconsistentMigrationHistory(
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration aaaa.0024_campaign_template is applied before its dependency bbbb.0005_templatemodel_from_template on database 'default'.
My solution
python manage.py migrate bbbb
python manage.py migrate
Because I changed the Django's app name in batches, the application order was not consistent when applied to the database. The bbbb that aaaa relies on was not created first, so I manually created the bbbb first
Migration file is not created for all app:
step 1:
create migration folder and add __init__.py file for all app
step 2:
delete db.sqlite3 database
step 3:
python manage.py migrate
python manage.py makemigrations
Delete all of your migrations folder
Delete the database(sqlite3)
Then run the makemigrations and migrate command
Delete the migration files.
Run:
python manage.py migrate
python manage.py makemigrations
python manage.py migrate
python manage.pyrunserver
Actually we are group of 3 people working on the same project, and each one individually make changes in django database. After running migrations in individual machine, it creates migration file for each migration. When someone pushes updated code in remote git repository, it creates conflict with others' migrations of same name.
Because of this reason, I lost my whole data once. Kindly give suggestions what should I do with this migration thing?
here's what i do: whenever I want to fetch from remote i check if a duplicate migration will be fetched. (we have a script that checks for all migration directories if there are filenames which have the same starting number.) if that is the case, I 'merge' the migrations, usually like this:
Find the last migration before the duplicate, let's say it's migration 000X
Make sure you are on your local source version, before the duplicate is added.
migrate back to after migration n:
python manage migrate app 000X
pull the new version including the duplicates.
remove your duplicate migrations
run schemamigration
python manage schemamigration --auto
Now you should get a new migration adding your model changes on top of the changes that were made in the migration you pulled.