Django finding non-existent changes to a model after each migration - django

After running python manage.py migrate, Django will consistently return:
Running migrations:
No migrations to apply.
Your models have changes that are not yet reflected in a migration, and so won't be applied.
Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
Hmm that's funny, I don't remember making a change.
python manage.py makemigrations --dry-run --verbosity=3
Generated by Django 1.11 on 2017-0x-xx xx:xx
from __future__ import unicode_literals
from django.db import migrations, models
import django_cryptography.fields
class Migration(migrations.Migration):
dependencies = [
('myApp', '0019_auto_xxxxxxx_xx'),
]
operations = [
migrations.AlterField(
model_name='myModel',
name='encrypted_field',
field=django_cryptography.fields.encrypt(models.CharField(verbose_name=models.CharField(verbose_name=models.CharField(max_length=50, verbose_name='Encrypted Field')))),
),
migrations.AlterField(
model_name='myModel',
name='second_encrypted_field',
field=django_cryptography.fields.encrypt(models.CharField(verbose_name=models.CharField(verbose_name=models.CharField(max_length=50, verbose_name='Second encrypted field')))),
),
]
That would be fine, but the exact same AlterField() was run in 0018_auto. I've tested it out on a couple systems, even tried making these migrations for real and applying but, but the warning keeps coming back in 0020_auto.
myModel.__str__ does reference one of those encrypted fields, which I think is what's causing the recursion in the migration. Still, I can't understand why that would make Django keep picking them up.
Any idea what could be causing this? Is it a bug? Something having to do with the nature of django_cryptography? Something else?

Related

Django database migration not working, new table not created

I initially created a database in mysql and used inspectdb to create all the models. So far so good, but then I started trying to make changes to the models and migrating them. I added columns to no avail, there were no changes in the database even though the migrations folder showed the changes. I began to wonder if it was something to do with case, so I tried again with a second model TestTwoT with table name and all fields in lower case. I then ran the make migrations which produced this output:
Migrations for 'testdb':
testdb/migrations/0010_testtwot.py
- Create model TestTwoT
The generated .py file in the migrations folder (which looks fine to me):
# Generated by Django 3.0.1 on 2020-02-10 22:46
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('testdb', '0009_auto_20200210_2241'),
]
operations = [
migrations.CreateModel(
name='TestTwoT',
fields=[
('id', models.AutoField(db_column='id', primary_key=True, serialize=False)),
('name', models.CharField(blank=True, db_column='name', max_length=1000, null=True)),
],
options={
'db_table': 'test_two_t',
'managed': True,
},
),
]
I then ran the migrate script which produced this output suggesting there were no problems creating the table:
Operations to perform:
Apply all migrations: admin, auth, contenttypes, testdb, sessions
Running migrations:
Applying testdb.0010_testtwot... OK
However, still no sign of my new table when I run the 'show tables' sql command or the 'describe test_two_t' command.
There is another complication in my scenario in that I am not using the default database, but my setup seems entirely OK and my application is working fine pointing to the database.
EDIT : Following Borut's advice I changed the commands to (adding the --database parameter):
python3 manage.py makemigrations
python3 manage.py migrate --database=testdb
EDIT2 : It's all working perfectly now (after I also deleted all the previous migration .py files and started from scratch with the corrected above commands).
Following Borut's advice I changed the commands to (adding the --database parameter):
python3 manage.py makemigrations
python3 manage.py migrate --database=testdb
It's all working perfectly now (after I also deleted all the previous migration .py files and started from scratch with the corrected above commands).

How would you create a 'manual' django migration?

I've discovered that I can set defaults for columns on a postgres database in a django project using migrations.RunSQL('some sql').
I'm currently doing this by adding a column, makemigrations, then removing the column, makemigrations, and then manually modifying the migration file that is produced.
I tried copying an old migration file and then removing the old code so just the new sql could be run and got some strange errors.
CommandError: Conflicting migrations detected; multiple leaf nodes in the migration graph: (0067_auto_20180509_2327, 0068_auto_20180514_0707 in csmu).
To fix them run python manage.py makemigrations --merge
How would you create a 'manual' django migration?
You can create a manual migration by running the command:
python manage.py makemigrations app_name --name migration_name --empty
Where app_name corresponds to the app within your project you want to add the migration. Remember Django manages both Project and Apps (A project is a collection of configuration and apps for a particular website. A project can contain multiple apps. An app can be in multiple projects.)
And --empty flag is to create a migration file where you will have to add your manual migrations.
For example, on a project where you have an app named api that only has one migration file 0001_initial.py running:
python manage.py makemigrations api --name migration_example --empty
will create a file named 0002_migration_example.py under the directory api/migrations/ that will look like:
# Generated by Django 2.2.10 on 2020-05-26 20:37
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('api', '0001_initial'),
]
operations = [
]
And you should add migrations.RunSQL('some sql'). inside operations brackets, like:
operations = [
migrations.RunSQL('some sql')
]
You can learn how to make migrations by investigating the automatically generated migrations, for example:
class Migration(migrations.Migration):
dependencies = [
('app_details', '0001_initial'),
]
operations = [
migrations.AddField(
...,
),
migrations.CreateModel(
...,
),
migrations.RenameModel(
...,
),
migrations.RenameField(
...,
),
migrations.RemoveModel(
...,
),
# and so on
]
Create a manual migration file
use this command in the terminal: python manage.py makemigrations --empty.
Then add what you want in it.
notice: you have to compose between the "models.py" and the manual migrations.

Django 1.9 - OperationalError: no such table: products_category_artists; due to ManyToManyField?

I tried Googling this problem, but I mostly saw answers that were from versions of Django where South was needed for migration and/or syncdb was still a thing. The ones that were relevant matched the symptom, but not the cause.
I started with the default project/app that Django creates when you do django-admin startproject project-name and django-admin startapp products.
models.py
from django.db import models
from django.contrib.auth.models import User
from django.core.exceptions import ObjectDoesNotExist
# Create your models here.
class Artist(models.Model):
user = models.OneToOneField(User, null=True, blank=True)
name = models.CharField(max_length=50)
def __str__(self): return "<Artist: %s>"%self.name
class Category(models.Model):
artists = models.ManyToManyField(Artist)
name = models.CharField(max_length=50)
desc = models.CharField(max_length=200)
My process:
python manage.py flush to clear the database
rm -rf products\migrations to delete prior migrations
python manage.py migrate
Output:
Operations to perform:
Apply all migrations: sessions, contenttypes, auth, admin
Running migrations:
No migrations to apply.
python manage.py makemigrations products
Output:
Migrations for 'products':
0001_initial.py:
- Create model Artist
- Create model Category
python manage.py migrate
Output:
Operations to perform:
Apply all migrations: contenttypes, admin, sessions, products, auth
Running migrations:
No migrations to apply.
At this point, I'm confused. Aren't there migrations to be applied, and yet there aren't any? I press on nonetheless.
python manage.py shell
In this shell, I enter these commands:
>>> from products.models import *
>>> artist = Artist(name="foo")
>>> artist.save()
>>> categ = Category(name="bar", desc="baz")
>>> categ.save()
>>> categ.artists.add(artist)
At this point, I get a huge error traceback. The problem seems to be this:
django.db.utils.OperationalError: no such table: products_category_artists
I see the same error if I try to use the built-in admin site.
My guess is that the problem is that migrations aren't actually being applied or that a particular table isn't being created, but I don't know how to fix either of these problems.
There is a table generated by django called django_migrations which keeps track of what migrations have been applied. If you delete your migrations, re-generate them and try to migrate without deleting the records from the table than django will think it already applied them. You should never delete your migrations, it will cause django to get confused.
If you are actively developing and want to skip the entire migrations system you can, but once you start using migrations, never delete them. Here is what I use while developing a new project:
dropdb mydb && createdb mydb && python manage.py migrate --run-syncdb && python manage.py loaddata initial
First, it drops the database and all data. Then it creates an empty one. The --run-syncdb generates the schema and the loaddata loads data from a fixtures file.
So, if you are still developing and can delete all your data and move what you care about to a fixtures file than you can delete all your migration folders and run the command above each time you change your model.

ValueError: Related model u'app.model' cannot be resolved

I have two applications (ook and eek say) and I want to use a foreign key to a model in ook from a model in eek. Both are in INSTALLED_APPS with ook first.
In ook.models.py, i have:
class Fubar(models.Model):
...
In eek.models.py, I have:
class monkey(models.Model):
external = models.ForeignKey('ook.Fubar', blank=True, null=True)
...
The migration generated is:
class Migration(migrations.Migration):
dependencies = [
('eek', '0002_auto_20151029_1040'),
]
operations = [
migrations.AlterField(
model_name='monkey',
name='external',
field=models.ForeignKey(blank=True, to='ook.Fubar', null=True),
),
]
When I run the migration, I get this error:
...
1595 raise ValueError('Foreign Object from and to fields must be
the same non-zero length')
1596 if isinstance(self.rel.to, six.string_types):
-> 1597 raise ValueError('Related model %r cannot be resolved' % self.rel.to)
1598 related_fields = []
1599 for index in range(len(self.from_fields)):
ValueError: Related model u'ook.Fubar' cannot be resolved
What am I doing wrong?
Because You have ForeignKey in operations, You must add a ook to dependencies:
dependencies = [
('ook', '__first__'),
('eek', '0002_auto_20151029_1040'),
]
Django migrations have two "magic" values:
__first__ - get module first migration
__latest__ - get module latest migration
Try running migrations one by one for every model.
This way you can debug the app you are facing problem with
python manage.py migrate appmname
I just got the same error, but referring to a model that was declared as part of the same migration. It turned out that the first migrations.CreateModel(...) referred to a not yet declared model. I manually moved this below the declaration of the referred model and then everything worked fine.
In my case, It was the cache and previous migrations that resulted in this error. I removed __pycache__ and migrations folder and then re-run the migrations command and it worked.
Remember, when you'll do python manage.py makemigrations it won't see any new migrations and will console output no changes detected. You'll have to do python manage.py makemigrations your_app_name instead to make things work.
I encountered this error when trying to use a child model of a base model as a foreign key. It makes sense that it didn't work because there's not an id field on the child model. My fix was to use the parent on the key. Unfortunately this was not immediately intuitive and set me back a couple hours.
I have found that it looks like this bug was not fixed yet when you scroll down to the bottom.
Django ValueError: Related model cannot be resolved Bug
I am using 1.11.7, they are talking about 1.9.3.
It worked everything on localhost, but was always failing on Heroku, so I tested all the options/answers above and nothing worked.
Then I have noticed, localhost DB in Admin I had 1 profile created (1 DB record), went to Heroku and DB has 0 records for Profile table so I have added 1, pushed the migration, python manage.py migrate and all it went OK.
That validates that I did not need to change any of those migrations manually that all is working.
Maybe it will help to someone.
migrations
# -*- coding: utf-8 -*-
# Generated by Django 1.11.7 on 2017-11-23 21:26
from __future__ import unicode_literals
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('blog', '0005_blog_author'),
]
operations = [
migrations.AlterField(
model_name='blog',
name='author',
field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE,
to='core.Profile'),
),
]
order of dependencies is too important.
in your case, ook must be created first then depend eek on it.
dependencies= [
('ook', '0001_initial'),
('eek', '0002_auto_20151029_1040'),
]

Django-migrations in Django 1.7 detects model changes but does not apply them on migrate

I have been trying to synchronize changes to a model in a Django app using migrations in 1.7 (postgres 9.1 - let me know if you need more details of my environment), but manage.py migrate doesn't seem to do anything, and sqlmigrate doesn't emit any SQL.
I thought Django 1.7 - "No migrations to apply" when run migrate after makemigrations might be applicable to my situation, and I did find some history in the django_migrations table in my database. I deleted the records for the app I am trying to migrate.
Recently I gave up on getting the alter table statements to generate/run and dropped the original version of the table. And while manage.py migrate states it is applying the migration, nothing happens to the database.
Here are the steps I've been trying:
Delete the history.
rm -r myapp/migrations
../manage.py dbshell
myapp_db=> delete from django_migrations where app='myapp'
Create an initial migration.
cp myapp/models.py.orig myapp/models.py
../manage.py makemigrations myapp
../manage.py migrate
manage.py migrate returns the following:
....
Running migrations:
Applying myapp.0001_initial... FAKED
Then I swap in the new models and generate a new migration.
cp myapp/models.py.new myapp/models.py
../manage.py makemigrations myapp
The result of makemigrations is in myapp/migrations/0002_notificationlog.py:
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models, migrations
class Migration(migrations.Migration):
dependencies = [
('myapp', '0001_initial'),
]
operations = [
migrations.CreateModel(
name='NotificationLog',
fields=[
('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)),
('tstamp', models.DateTimeField(help_text=b'Log time', auto_now_add=True)),
('recipient', models.CharField(max_length=100)),
('subject', models.TextField()),
],
options={
},
bases=(models.Model,),
),
]
Run this migration:
../manage.py migrate
manage.py migrate acts like everything is OK:
....
Running migrations:
Applying myapp.0002_notificationlog... OK
I can see the log entries appear in django_migrations, but the table is not created.
I'm lost. Any idea what to try next?
Update
When running migrate -v 3 as requested, I see
Running pre-migrate handlers for application auth
followed by a similar line for each installed app.
Then
Loading 'initial_data' fixtures...
Checking '/var/www/environment/default/myproj/myproj' for fixtures...
No fixture 'initial_data' in '/var/www/environment/default/myproj/myproj'.
repeated a total of 13 times, the number of unmanaged apps.
Then
Running migrations:
Applying myapp.0001_initial... FAKED
followed by
Running post-migrate handlers for application auth
with a similar line for each installed app.
For migration 0002, the output is the same, except for
Running migrations:
Applying myapp.0002_notificationlog... OK
Note also that sqlmigrate doesn't output anything either:
../manage.py sqlmigrate myapp 0002 -v 3
Produces nothing at all.
Update 2
I copied myapp into a new project and was able to run migrations on it, but migrations stopped working when I imported my main project settings. Are there settings I should be aware of that could affect migration execution, particularly if I've been using South with previous versions of Django?
The problem disappeared with generic project settings and reappeared with my old, complex project settings. I tracked the problem down to a database Router class that was missing an allow_migrate method.
DATABASE_ROUTERS = [ 'myproj.routers.DatabaseAppsRouter', ]
I use this router to handle queries for a separate app in the project (readonly/MySQL).
Sadly I can't blame anyone but myself, since the Django documentation states clearly:
Note that migrations will just silently not perform any operations on a model for which [allow_migrate] returns False. (link)
I had created this router some time ago and didn't add the allow_migrate method to my router class when I upgraded to Django 1.7. When I added the method and made sure it returned True when needed, migrations run and the problem is solved.