If I run python -Wall manage.py test this warning (and similar) occurs:
/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py:1453:
RuntimeWarning: DateTimeField SignUpUser.signup_time received a naive datetime (2018-03-17 21:27:22.620074) while time zone support is active.RuntimeWarning)
But there is no such field for the model SignUpUser anymore. It's called signup_timestamp. The same error occurs on other fields. To fix these I changed datetime.now to the django built-in timezone considering timezone.now. But the error messages also doesn't disappear. I think this occurs because of an old migration.
The site is already in production, but only me is developing. How should I fix this? Resetting all migration files and redo the migration with --fake-initial?
I ran into a similar problem. I had defined the default value for a DateTimeField with datetime.now and switched to timezone.now.
I had created migrations after each model change (proably to test them). This lead to one migration defining the default value without the timezone (field=models.DateTimeField(default=datetime.datetime(2018, 8, 2, 22, 15, 4, 702061)),) and the next migration would fix this issue and add the timezone (field=models.DateTimeField(default=datetime.datetime(2018, 8, 3, 19, 22, 32, 951341, tzinfo=utc)),).
When running the tests for the model (python manage.py test app_name) then resulted in: [...]/django/db/models/fields/__init__.py:1423: RuntimeWarning: DateTimeField Job.sub_date received a naive datetime (2018-08-02 22:15:04.702061) while time zone support is active.
It seems like Django tries to apply the migrations to a database in which the timezone awareness is already defined. Not sure why it does that but that seems to be going on.
To prevent the warning occurring I had to squash the migrations up to the migration that changed the default value of the DateTimeFiled to be timezone aware.
In my case this was migration 0007, while 0006 added the default value without the timezone awareness.
So the command to create the squash was:
$ python manage.py squashmigrations app_name 0007
This created a new migration file 0001_squashed_0007_auto_20180803_2122.py which contains all the changes done by the migrations 0001 through 0007. As the documentation says, this happens in an optimized manner so that only the final result of a series of migrations is build without the intermediate steps.
On new installs, instead of running the separate migrations 0001 through 0007 only the new squashed migration will be run.
You should keep the original migrations around for some time, to make sure the squashing did not introduce any issues:
Once you’ve squashed your migration, you should then commit it alongside the migrations it replaces and distribute this change to all running instances of your application, making sure that they run migrate to store the change in their database.
You must then transition the squashed migration to a normal migration by:
Deleting all the migration files it replaces.
Updating all migrations that depend on the deleted migrations to depend on the squashed migration instead.
Removing the replaces attribute in the Migration class of the squashed migration (this is how Django tells that it is a squashed migration).
Related
I have django app already online and running. I created new model and if I do the makemigrations, it is working fine, but if I want to migrate it, then I receive Running migrations:No migrations to apply. Even though that I made a change. The change is also in the migration file. I am using postgresql with it. I tried to delete the migration file and create it once more, but it did not help. Any idea what might be wrong?
Looks like some migrations are marked as applied incorrectly in the database. This can be solved by using the --fake flag [Django docs] to mark an appropriate migration as applied / some migrations as unapplied.
Suppose that your app is named myapp and the migration 0004 is applied properly to the database (you are trying to apply 0005 or further which doesn't work) then you will run the following command:
python manage.py migrate myapp 0004 --fake
Note: myapp and 0004 is an example, do this according to your apps name and which migrations are applied.
Edit: It appears that you have deleted the migrations. But thankfully have some version control in place. Best would be to get the migration files back from the version control. If for some reason you are unable to do that, revert the models to how they were before their changes and make sure no migration files exist for now. Next generate the migration files by running makemigrations (check that they are as expected and your database state corresponds to these generated files). Next run the following two commands:
python manage.py migrate <appname> zero --fake
python manage.py migrate <appname> --fake
The first one will mark all migrations as unapplied for the app. The second one will mark all the migrations as applied. Next you will add the changes you wanted to make back to the models and migrate again.
I am trying to simply add a boolean field to a model in my database following the simple rules:
- add field to model
- python manage.py makemigrations app
- python manage.py migrate app
Works all but 99% of the time. So during the second step (makemigrations), with my newly added field in my model raring to go, i get an error:
django.db.utils.OperationalError: (1054, "Unknown column 'model.field' in 'field list'")
Excellent. its not letting me make migrations by adding a new field..... because it cant find the field that I am trying to newly add... makes perfect sense!
Anyway, I have gone as far as deleting all my migrations, removing my new field, making migrations again, migrating... all fine - so now i have only 1 migration file (0001)...
Follow the same steps as above... ERROR
Am i missing something ridiculous here? I mean, adding a field to a model is very simple, and I have done it probably 1000 times. Why does Django tease me so
EDIT: Answer:
OK I have done it.
After deleting the migrations file, truncating the migrations table and migrating with 0001_initial.py, I made an empty migrations file (python manage.py makemigrations --empty app) and added the field in manually.... then I migrated and it worked! Baffled at this to be honest, but at least the change has been made:
Delete all migration files
Truncate the django_migrations table
comment the new boolean field
run python manage.py makemigrations
run python manage.py migrate --fake
run python manage.py makemigrations --empty app
add field in manually to the empty migrations file in the operations:
migrations.AddField('modelName', 'fieldName', models.BooleanField(default=False)),
run python manage.py migrate
uncomment the new boolean field so it represents what you made in the migrations operations
Disclaimer- Follow this only on local system, for production do understand the steps and then execute.
Kindly follow these steps:
Delete all migration files
Truncate the django_migrations table
comment the new boolean field
run python manage.py makemigrations
run python manage.py migrate --fake
Uncomment the boolean field
run python manage.py makemigrations
run python manage.py migrate
Generally these steps solve any kind of migration problem
An another reason can be if you are using django_rest_framework then the serialiser too needs to be updated as per your model change.
This is an issue that has persisted with me and lead me down a lot of rabbit holes dropping tables etc. A simple solution I have found is answering "N" when django asks you if you are renaming a field of that model (when running makemigrations). What that then essentially performs is a deletion of your previous field and creates the new field. Be careful as you may lose data on existing field so this works with fields which are either new or relatively easy to 'refill' their data required. You may need to run --fake if you get an error with regards to not being able to 'drop field' when migrating after makemigrations. If you would like to go back to your migration state before the problem you may need to delete the migrations you have done since then and try the above solution.
Update:
I did the above for a Boolean field and my data was kept. Even though I said N, it seems as if it is essentially a renaming.
Another source of the problem: I have a ModelForm based on the model. The ModelForm definition defines two extra fields. I had to comment out those two fields from the Form definition before doing the migration steps. Once the migration was done, I un-commented the two fields.
I have some questions on Squashing in Django 1.8. I've squashed a few migration files into one but not quite sure how to do the 2 actions as specified in the djangoprojects docs.
After this has been done, you must then transition the squashed migration to a normal initial migration, by:
-Deleting all the migration files it replaces
-Removing the replaces argument in the Migration class of the squashed migration (this is how Django tells that it is a squashed migration)
Do I just delete the migration files in the migration file using normal rm command?
Do I just vim the migration file "0001_squashed_xxx" and remove the part "replaces = []"?
Also, after I run the squashed migration, the migration (original is 0010) became 0001 automatically. Is this the expected behavior?
Any help would be great. Thanks!
Do I just delete the migration files in the migration file using normal rm command?
Do I just vim the migration file "0001_squashed_xxx" and remove the part "replaces = []"?
Yes and yes, but only AFTER you've run the squashed migrations on all your production sites.
Also, you should ensure that none of your other migrations depend on the migrations you're about to delete. If they do, you should change those dependencies to point at the corresponding squashed migrations instead.
Also, after I run the squashed migration, the migration (original is 0010) became 0001 automatically. Is this the expected behavior?
This is because squashed migrations are given the name <start_migration>_squashed_<end_migration>. So if you squashed from 0001_initial to 0010_blah, the squashed migration will have the name 0001_initial_squashed_0010_blah and thus its code will start with 0001. But the numbering doesn't really mean anything, it's just there so that your migrations are nicely ordered when listed in a directory.
Django documentation says we could delete migrations after squashing them:
You should commit this migration but leave the old ones in place; the
new migration will be used for new installs. Once you are sure all
instances of the code base have applied the migrations you squashed,
you can delete them.
Here, does deleting means deleting only the migration files, or the entries in the django_migrations table as well?
Here is some background: I have only the development machine, so just one code base. After squashing some of the migrations that I had already applied, I deleted the files and the database entries. Tested if this is OK by making migrations, it did not find anything. So, everything looked good. Next day, I had to change something, and made migration. When I tried to migrate, it tried to apply the squashed migration too (which was applied part by part before being squashed). So, I had to go back and recreate the entries in the django_migrations table. So, it seems like I had to keep the database entries. I am trying to make sure before I mess up anything again, and understand why it looked fine first, and then tried to apply the squashed migration.
Squashed migrations are never marked as applied, which will be fixed in 1.8.3 (see #24628).
The steps to remove the old migrations are:
Make sure all replaced migrations are applied (or none of them).
Remove the old migration files, remove the replaces attribute from the squashed migrations.
(Workaround) Run ./manage.py migrate <app_label> <squashed_migration> --fake.
The last step won't be necessary when 1.8.3 arrives.
Converting squashed migrations has gotten easier since the question was posted. I posted a small sample project that shows how to squash migrations with circular dependencies, and it also shows how to convert the squashed migration into a regular migration after all the installations have migrated past the squash point.
As the Django documentation says:
You must then transition the squashed migration to a normal migration by:
Deleting all the migration files it replaces.
Updating all migrations that depend on the deleted migrations to depend on the squashed migration instead.
Removing the replaces attribute in the Migration class of the squashed migration (this is how Django tells that it is a squashed migration).
I'm no expert by any means, but I just squashed my migrations, and ended up doing the following:
Ran this query to removed the old migrations (squashed)
DELETE FROM south_migrationhistory;
Run this management command to remove the ghosted migrations
./manage.py migrate --fake --delete-ghost-migrations
Django 1.7 also has squashmigrations
I started a django 1.8 project, which uses the migrations system.
Somehow along the way things got messy, so I erased the migrations folders and table from the DB, and now I'm trying to reconstruct them, with no success.
I have three apps (3 models.py files), and the models reflect the tables EXACTLY!
The best approach that I've found so far was:
Erase all migrations folders. Done!
Delete everything from the django_migrations table. Done!
Run python manage.py makemigrations --empty <app> for every app. Done!
Run python manage.py migrate --fake. Done! (although it works only if I run it after every makemigrations command.
Now I add a new field, run the makemigrations command, and I receive the following error:
django.db.utils.OperationalError: (1054, "Unknown column 'accounts_plan.max_item_size' in 'field list'")
I've been burning HOURS on this thing. How the h**l can I initialize the migrations so I can continue working without migration interruptions every time?
Why is it so complicated? Why isn't there a simple one-liner: initiate_migrations_from_schema?
EDIT:
Now things get even nastier. I truncated the django_migrations table and deleted all the migrations folder.
Now I try to run python manage.py migrate --fake-initial (something I found in the DEV docs), just so it sets up all of Django's 'internal' apps (auth, session, etc) and I'm getting:
(1054, "Unknown column 'name' in 'django_content_type'").
Now, this "column" is not a real column. It's a #property defined in Django's contenttypes app. WHAT IS GOING ON HERE? Why is it identifying the name property as a real column?
Finally got it to work, although I don't know why and I hope it will work in the future.
After doing numerous trials and going through Django's dev site (link).
Here are the steps (for whoever runs into this problem):
Empty the django_migrations table: delete from django_migrations;
For every app, delete its migrations folder: rm -rf <app>/migrations/
Reset the migrations for the "built-in" apps: python manage.py migrate --fake
For each app run: python manage.py makemigrations <app>. Take care of dependencies (models with ForeignKey's should run after their parent model).
Finally: python manage.py migrate --fake-initial
After that I ran the last command without the --fake-initial flag, just to make sure.
Now everything works and I can use the migrations system normally.
I'm sure I'm not the only one who encounters this issue. It must be documented better and even simplified.
Update for Django 1.9 users:
I had this scenario again with a Django 1.9.4, and step 5 failed.
All I had to do is replace --fake-initial with --fake to make it work.
django ..., 1.8, 1.9, ...
What you want to achieve is squashing existing migrations and use replacement for them.
How to do it right without using any command when releasing (a case without impact on database and coworkers).
For every app, get rid of its migrations folder:
mv <app>/migrations/ <app>/migrationsOLD/
For each that app run: python manage.py makemigrations <app>.
Customize each new migration:
if you have a complex app, or more apps and related models between them, to avoid CircularDependencyError or ValueError: Unhandled pending operations for models:
prepare second empty migration in <app> 0002_initial2.py (put there dependency to app_other::0001_initial.py and <app>::0001_initial.py as well - all ForeignKey, M2M related to models created in 0001 migration step in other apps)
All must be in order - sometimes it will require more migrations to prepare. Take care of dependencies attribute here in each Migration.
take care of initial values - verify yourself all RunPython actions from migrationsOLD and copy the code to new initial migration if needed.
(optional for --fake-initial) Add initial=True to all new Migration classes (0002 too if was added).
Add replaces attribute in new Migration class. (like own custom a squashmigrations). Put there all old migrations from <app>
Verify everything with makemigrations.
assert "No changes detected"
Check if migrate -l show [x] everywhere
assert similar:
[X] 0001_initial
[X] 0002_initial2 (102 squashed migrations)
Example:
For old:
0001_initial.py
0002_auto.py
...
0103_auto.py
prepare:
0001_initial.py
0002_initial2.py (optional but sometimes required to satisfy dependency)
and add to replacesto last one (0002 here, can be 0001):
replaces = [(b'<app>', '0002_auto.py'), ..., (b'<app>', '0103_auto.py')]
0001_initial.py should be named the same way as old one.
0002_initial2.py is new one, but it's a replacement for old migrations so Django will treat it as loaded.
I've run into this scenario but I've never had to drop the database to solve it. Typically I delete the migrations folder from the app's, and remove the migration entries from the database.
I would try to run make migrations one app at a time. If any of the app's rely upon other tables obviously add them last.
Also I usually just run, python manage.py makemigrations then just python manage.py migrate Even with the initial migration it should work fine with Django 1.7 and 1.8.
If you are using routers, might be a problem there. Check method allow_migrate if it is executed in a right way in routers.py. Try to set return value always to be True, and check whether it resolves problem,
def allow_migrate(self, db, app_label, model_name=None, **hints):
return True