django keeps creating new migrations even when model has not changed - django

The following is my model:
class Biovariable(models.Model):
bioid = models.AutoField(primary_key=True, unique=True)
name = models.CharField(max_length=200)
description = models.CharField(max_length=2000)
units = models.ForeignKey(unit, on_delete=models.CASCADE)
def __str__(self):
return self.name
class Biovariable_data(models.Model):
biovar = models.ForeignKey(Biovariable, on_delete=models.CASCADE)
value = models.CharField(max_length=2000)
evdate = models.CharField(max_length=30)
evtime = models.CharField(max_length=30)
linkedcustomer = models.ForeignKey(customer, on_delete=models.CASCADE)
linkeddoctor = models.ForeignKey(doctor, on_delete=models.CASCADE)
linkedclinic = models.ForeignKey(Clinic, on_delete=models.CASCADE)
class Meta:
unique_together = [
"biovar", "value", "evdate", "evtime", "linkedcustomer",
"linkeddoctor", "linkedclinic"
]
Each time I run makemigrations, django keeps creating new migrations, and I cant understand why.
joel#hp:~/myappointments$ ./manage.py makemigrations
Migrations for 'appointments':
appointments/migrations/0010_auto_20190310_2138.py
- Add field dob to customer
- Alter field biovar on biovariable_data
joel#hp:~/myappointments$ ./manage.py migrate
Operations to perform:
Apply all migrations: admin, appointments, auth, clinic, contenttypes, sessions
Running migrations:
Applying appointments.0010_auto_20190310_2138... OK
joel#hp:~/myappointments$ ./manage.py makemigrations
Migrations for 'appointments':
appointments/migrations/0011_auto_20190310_2139.py
- Alter field biovar on biovariable_data
joel#hp:~/myappointments$ ./manage.py makemigrations
Migrations for 'appointments':
appointments/migrations/0012_auto_20190310_2139.py
- Alter field biovar on biovariable_data
joel#hp:~/myappointments$ ./manage.py migrate
Operations to perform:
Apply all migrations: admin, appointments, auth, clinic, contenttypes, sessions
Running migrations:
Applying appointments.0011_auto_20190310_2139... OK
Applying appointments.0012_auto_20190310_2139... OK
joel#hp:~/myappointments$ ./manage.py makemigrations
Migrations for 'appointments':
appointments/migrations/0013_auto_20190310_2140.py
- Alter field biovar on biovariable_data
joel#hp:~/myappointments$
One of these migration files looks like this:
# Generated by Django 2.1.3 on 2019-03-10 16:09
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('appointments', '0010_auto_20190310_2138'),
]
operations = [
migrations.AlterField(
model_name='biovariable_data',
name='biovar',
field=models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='appointments.Biovariable'),
),
]

I had the same problem but nothing in my model was dynamic. This occurred for me after changing the name of the model in question. (For reference it was a many to many through model which I renamed from "Enrollment" to "Registration").
Answering to provide a more explicit solution which was:
# 1 - Roll back migrations
python manage.py migrate <app_name> zero>
# 2 - Delete the migration files
# 3 - Make a new migration
python manage.py makemigrations <app_name>

As mentioned in the comments and documented in similar questions, this is usually because of something dynamic in the model such as a datetime or set of choices that's coming up in an undetermined order.
The issue could also be related to a minor typo or change such as lower vs. upper case characters. In my case, my FK's model initial migration had the name set to a lowercased word rather than the proper upper case name. The referencing model was getting an alter field every time I ran makemigrations. Fixing the FK model's initial migration file by simply capitalizing the name solved the problem:
operations = [
migrations.CreateModel(
name='sentence',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
to
operations = [
migrations.CreateModel(
name='Sentence',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
Not sure how the lower case character got in there in the first place, though...
It might be a bit difficult to look through every migration file for something like this, however, so I'd recommend just removing all of your migration files, and doing a fresh makemigrations to start over from scratch.
If you'd rather not do this for some reason (migrations have already been synced to git, for example) you could move your migration files to a temporary location, run makemigrations and then see if a git diff shows anything inconsistent in your original and new initial migration files. If you're able to find the issue, restore your original migration files and fix the problem there.

Related

Django - Adding a primary key to an existing model with data

This is the existing model in a djnago app:
class Task_Master_Data(models.Model):
project_id = models.ForeignKey(Project_Master_Data, on_delete=models.CASCADE)
task_created_at = models.DateTimeField(auto_now_add=True)
task_updated_at = models.DateTimeField(auto_now=True)
task_name = models.CharField(max_length=200, null=True)
I want to add a new field (which will be a primary key):
task_id = models.AutoField(primary_key=True, null=False)
When I am making migrations from the terminal, it will provide me with the option of adding a default value, but it will, understandably, bring an error of:
UNIQUE constraint failed: new__main_task_master_data.task_id
What would be the best way forward for this kind of a scenario without having to delete all the data.
Note, I am using the default sqlite3 database.
This is the result of an old bug that is still unfixed. See here for the ticket.
I have created a workaround.
First create an empty migration (change taskapp to the name of the app where your models.py with Task_Master_Data lives):
python manage.py makemigrations taskapp --empty
Then copy the below migrations into this file and run python manage.py migrate but be sure to adjust the name of the app (replace taskapp with the name of your app) and the dependencies (0010_task_master_data should be replaced by the name of the migration that comes before it).
First the existing rows in the database need values for the new task_id field. I solved this by creating an IntegerField with null=True, which I then manually fill using a RunPython command. I then remove the id field (my previous primary key), and use AlterField to change the field into the AutoField. A default value is required, but it should never be used. When you create a new Task_Master_Data the task_id should be auto-incrementing.
# Generated by Django 3.0.4 on 2022-05-11 07:13
from django.db import migrations, models
def fill_taskid(apps, schema_editor):
# be sure to change it to your app here
Task_Master_Data = apps.get_model("taskapp", "Task_Master_Data")
db_alias = schema_editor.connection.alias
tasks = Task_Master_Data.objects.using(db_alias).all()
for i, task in enumerate(tasks):
task.task_id = i
Task_Master_Data.objects.using(db_alias).bulk_update(tasks, ["task_id"])
class Migration(migrations.Migration):
dependencies = [
('taskapp', '0010_task_master_data'),
]
operations = [
migrations.AddField(
model_name='task_master_data',
name='task_id',
field=models.IntegerField(null=True),
),
migrations.RunPython(fill_taskid),
migrations.RemoveField(
model_name='task_master_data',
name='id',
),
migrations.AlterField(
model_name='task_master_data',
name='task_id',
field=models.AutoField(default=-1, primary_key=True, serialize=False),
preserve_default=False,
),
]
Note that depending on the state of your database you might need to rollback some of your migrations. The migration I created should work if your database is in a state where your task_id field does not exist yet.

How to fake migrations for not to create a specific existing intermediary table

I have following models
class VucutBolgesi(models.Model):
site = models.ForeignKey(Site)
bolge = models.CharField(verbose_name="Bölge", max_length=75)
hareketler = models.ManyToManyField("Hareket", verbose_name="Hareketler", null=True, blank=True, help_text="Bölgeyi çalıştıran hareketler")
class Hareket(models.Model):
site = models.ForeignKey(Site)
hareket = models.CharField(verbose_name="Hareket", max_length=75 )
bolgeler = models.ManyToManyField(VucutBolgesi, verbose_name="Çalıştırdığı Bölgeler", null=True, blank=True,
help_text="Hareketin çalıştırdığı bölgeler")
I have the same M2M on both table since I wish to display same intermediate table on both admin forms. They also have to use the same table (not create two separate tables) since one change in one admin form must be reflected to the other. Like, If I add a new Hareket to VucutBolgesi through HareketAdmin then the same result shoudl be visible on VucutBolgesiAdmin too.
For achieving this, I first remove hareketler M2M field from VucutBolgesi so Hareketler model would create the intermediate table. I migrate this and then add hareketler to VucutBolgesi with db_table attribute so it will recognize the same intermediary table.
final look of the field is as folows
hareketler = models.ManyToManyField("Hareket", verbose_name="Hareketler", db_table="antrenman_hareket_bolgeler",
null=True, blank=True, help_text="Bölgeyi çalıştıran hareketler")
When I try to migrate this, django throw following exception
django.db.utils.OperationalError: table "antrenman_hareket_bolgeler" already exists
How should I fake this migration?
Following is the migration django creates each time I run makemigrations
dependencies = [
('antrenman', '0005_vucutbolgesi_hareketler'),
]
operations = [
migrations.AddField(
model_name='vucutbolgesi',
name='hareketler',
field=models.ManyToManyField(to='antrenman.Hareket', db_table=b'antrenman_hareket_bolgeler', blank=True, help_text=b'B\xc3\xb6lgeyi \xc3\xa7al\xc4\xb1\xc5\x9ft\xc4\xb1ran hareketler', null=True, verbose_name=b'Hareketler'),
preserve_default=True,
),
]
Note: Editing related migration file and removing migrations.AddField fo not work since django creates the same migrations.AddField with each makemigrations
Is it possible to make a migration always to be faked, just override the apply and unapply methods. The consequences of this are not sufficiently investigated, but this far it works for me.
In the following example we create a migration that reuses django.contrib.auth.User.group's M2M table b'profile_user_groups:
from __future__ import unicode_literals
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('profile', '0001_initial'),
]
operations = [
migrations.AddField(
model_name='user',
name='organizations',
field=models.ManyToManyField(db_column=b'group_id', db_table=b'profile_user_groups', related_name='members', to='profile.Organization'),
),
]
def apply(self, project_state, schema_editor, collect_sql=False):
return project_state.clone()
def unapply(self, project_state, schema_editor, collect_sql=False):
return project_state.clone()
Solution was so simple.
You must be sure related migration is the only migration operation that needed to be faked. You must first create the migration with
python manage.py makemigrations antrenman
Then apply that migration with --fake
python manage.py migrate --fake antrenman
Handicap is, other developers should know that they have to fake related migration. If there are others migrations alongside with this one, they should make them first and then fake this one.
It is too bad there is no parameter that tells related migration should be real or fake.

Django migration error :you cannot alter to or from M2M fields, or add or remove through= on M2M fields

I'm trying to modify a M2M field to a ForeignKey field. The command validate shows me no issues and when I run syncdb :
ValueError: Cannot alter field xxx into yyy they are not compatible types (you cannot alter to or from M2M fields, or add or remove through= on M2M fields)
So I can't make the migration.
class InstituteStaff(Person):
user = models.OneToOneField(User, blank=True, null=True)
investigation_area = models.ManyToManyField(InvestigationArea, blank=True,)
investigation_group = models.ManyToManyField(InvestigationGroup, blank=True)
council_group = models.ForeignKey(CouncilGroup, null=True, blank=True)
#profiles = models.ManyToManyField(Profiles, null = True, blank = True)
profiles = models.ForeignKey(Profiles, null = True, blank = True)
Any suggestions?
I stumbled upon this and although I didn't care about my data much, I still didn't want to delete the whole DB. So I opened the migration file and changed the AlterField() command to a RemoveField() and an AddField() command that worked well. I lost my data on the specific field, but nothing else.
I.e.
migrations.AlterField(
model_name='player',
name='teams',
field=models.ManyToManyField(related_name='players', through='players.TeamPlayer', to='players.Team'),
),
to
migrations.RemoveField(
model_name='player',
name='teams',
),
migrations.AddField(
model_name='player',
name='teams',
field=models.ManyToManyField(related_name='players', through='players.TeamPlayer', to='players.Team'),
),
NO DATA LOSS EXAMPLE
I would say: If machine cannot do something for us, then let's help it!
Because the problem that OP put here can have multiple mutations, I will try to explain how to struggle with that kind of problem in a simple way.
Let's assume we have a model (in the app called users) like this:
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person)
def __str__(self):
return self.name
but after some while we need to add a date of a member join. So we want this:
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through='Membership') # <-- through model
def __str__(self):
return self.name
# and through Model itself
class Membership(models.Model):
person = models.ForeignKey(Person, on_delete=models.CASCADE)
group = models.ForeignKey(Group, on_delete=models.CASCADE)
date_joined = models.DateField()
Now, normally you will hit the same problem as OP wrote. To solve it, follow these steps:
start from this point:
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person)
def __str__(self):
return self.name
create through model and run python manage.py makemigrations (but don't put through property in the Group.members field yet):
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person) # <-- no through property yet!
def __str__(self):
return self.name
class Membership(models.Model): # <--- through model
person = models.ForeignKey(Person, on_delete=models.CASCADE)
group = models.ForeignKey(Group, on_delete=models.CASCADE)
date_joined = models.DateField()
create an empty migration using python manage.py makemigrations users --empty command and create conversion script in python (more about the python migrations here) which creates new relations (Membership) for an old field (Group.members). It could look like this:
# Generated by Django A.B on YYYY-MM-DD HH:MM
import datetime
from django.db import migrations
def create_through_relations(apps, schema_editor):
Group = apps.get_model('users', 'Group')
Membership = apps.get_model('users', 'Membership')
for group in Group.objects.all():
for member in group.members.all():
Membership(
person=member,
group=group,
date_joined=datetime.date.today()
).save()
class Migration(migrations.Migration):
dependencies = [
('myapp', '0005_create_models'),
]
operations = [
migrations.RunPython(create_through_relations, reverse_code=migrations.RunPython.noop),
]
remove members field in the Group model and run python manage.py makemigrations, so our Group will look like this:
class Group(models.Model):
name = models.CharField(max_length=128)
add members field the the Group model, but now with through property and run python manage.py makemigrations:
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through='Membership')
and that's it!
Now you need to change creation of members in a new way in your code - by through model. More about here.
You can also optionally tidy it up, by squashing these migrations.
Potential workarounds:
Create a new field with the ForeignKey relationship called profiles1 and DO NOT modify profiles. Make and run the migration. You might need a related_name parameter to prevent conflicts. Do a subsequent migration that drops the original field. Then do another migration that renames profiles1 back to profiles. Obviously, you won't have data in the new ForeignKey field.
Write a custom migration: https://docs.djangoproject.com/en/1.7/ref/migration-operations/
You might want to use makemigration and migration rather than syncdb.
Does your InstituteStaff have data that you want to retain?
If you're still developing the application, and don't need to preserve your existing data, you can get around this issue by doing the following:
Delete and re-create the db.
go to your project/app/migrations folder
Delete everything in that folder with the exception of the init.py file. Make sure you also delete the pycache dir.
Run syncdb, makemigrations, and migrate.
Another approach that worked for me:
Delete the existing M2M field and run migrations.
Add the FK field and run migrations again.
FK field added in this case has no relation to the previously used M2M field and hence should not create any problems.
This link helps you resolve all problems related to this
The one which worked for me is python3 backend/manage.py migrate --fake "app_name"
I literally had the same error for days and i had tried everything i saw here but still didn'y work.
This is what worked for me:
I deleted all the files in migrations folder exceps init.py
I also deleted my database in my case it was the preinstalled db.sqlite3
After this, i wrote my models from the scratch, although i didn't change anything but i did write it again.
Apply migrations then on the models and this time it worked and no errors.
This worked for Me as well
Delete last migrations
run command python manage.py migrate --fake <application name>
run command 'python manage.py makemigrations '
run command 'python manage.py migrate'
Hope this will solve your problem with deleting database/migrations
First delete the migrations in your app (the folders/ files under 'migrations'
folder)
Showing the 'migrations' folder
Then delete the 'db.sqlite3' file
Showing the 'db.sqlite3' file
And run python manage.py makemigrations name_of_app
Finally run python manage.py migrate
I had the same problem and found this How to Migrate a ‘through’ to a many to many relation in Django article which is really really helped me to solve this problem. Please have a look. I'll summarize his answer here,
There is three model and one(CollectionProduct) is going to connect as many-to-many relationship.
This is the final output,
class Product(models.Model):
pass
class Collection(models.Model):
products = models.ManyToManyField(
Product,
blank=True,
related_name="collections",
through="CollectionProduct",
through_fields=["collection", "product"],
)
class CollectionProduct(models.Model):
collection = models.ForeignKey(Collection, on_delete=models.CASCADE)
product = models.ForeignKey(Product, on_delete=models.CASCADE)
class Meta:
db_table = "product_collection_products"
and here is the solution,
The solution
Take your app label (the package name, e.g. ‘product’) and your M2M field name, and combine them together with and underscore:
APPLABEL + _ + M2M TABLE NAME + _ + M2M FIELD NAME
For example in our case, it’s this:
product_collection_products
This is your M2M’s through database table name. Now you need to edit your M2M’s through model to this:
Also found another solution in In Django you cannot add or remove through= on M2M fields article which is going to edit migration files. I didn't try this, but have a look if you don't have any other solution.
this happens when adding 'through' attribute to an existing M2M field:
as M2M fields are by default handled by model they are defined in (if through is set).
although when through is set to new model the M2M field is handled by that new model, hence the error in alter
solutions:-
you can reset db or
remove those m2m fields and run migration as explained above then create them again
*IF YOU ARE IN THE INITIAL STAGES OF DEVELOPMENT AND CAN AFFORD TO LOOSE DATA :)
delete all the migration files except init.py
then apply the migrations.
python manage.py makemigrations
python manage.py migrate
this will create new tables.

Adding indexes to model fields in Django with migrations

I am trying to add indexes on model fields using Field.db_index for an app that has migrations. Looking at Django's documentation all I need to do is to set db_index=True:
class Person(models.Model):
first_name = models.CharField()
last_name = models.CharField(db_index=True)
and then I first tried the new Django's Migration:
./manage.py makemigrations app-name
but Migration does not seem to notice the change and does not add the sql command for creating an index. So I tried django-admin.py as explained here:
django-admin.py sqlindexes app-name
But that does not print the sql command either and it exits with the following error:
CommandError: App 'app-name' has migrations. Only the sqlmigrate and sqlflush commands can be used when an app has migrations.
This problem still exists in django2.1.
I solved it by using the indexes Meta option. This is a bit cleaner than the index_together solution.
class Person(models.Model):
first_name = models.CharField()
last_name = models.CharField()
class Meta:
indexes = [
models.Index(fields=['last_name']),
]
OK, I managed to create the indexes using Meta.index_together. It is not the cleanest way, since I am not actually indexing multiple fields together but it works with makemigrations:
class Person(models.Model):
class Meta():
index_together = [['last_name']]
first_name = models.CharField()
last_name = models.CharField()
Now makemigrations does make a new migration:
./manage.py makemigrations app-name
>>Migrations for 'app-name':
>> 0005_auto_20140929_1540.py:
>> - Alter index_together for Person (1 constraint(s))
And the corresponding sql command is actually CREATE INDEX.
./manage.py sqlmigrate app-name 0005_auto_20140929_1540
>>BEGIN;
>>CREATE INDEX app-name_person_last_name_7...4_idx ON `app-name_person` (`last_name`);
>>COMMIT;
You can do this explicitly in your migration using Django's AddIndex and Index classes.
First create an empty migration with manage.py makemigrations --empty and then simply fill it out as follows:
from django.db import migrations
from django.db.models.indexes import Index
from django.db.migrations import AddIndex
class Migration(migrations.Migration):
dependencies = [
('app_name', 'ref_to_previous_migration'),
]
operations = [
AddIndex('ModelName', Index(fields=['field_name'], name='my_index'))
]
You can use options on the Index class to specify fields, add a name, and do special custom things like index only part of the table, etc. Check the doc links above.

Django South "myapp.foo" already exists" error with initial migration

I have an already existing app with alot of database entries.
class Foo(models.Model):
value = models.TextField(u"Value")
For this I do this:
python manage.py schemamigration myapp --initial
python manage.py migrate myapp
I change the model to such:
class Foo(models.Model):
value = models.TextField(u"Value")
live = models.BooleanField(u"Live", default=False)
creation_time = models.DateTimeField("Creation Time", auto_now_add=True, null=True, blank=True)
and migrate:
python manage.py schemamigration myapp --auto
python manage.py migrate myapp
I get django.db.utils.DatabaseError: relation "myapp.foo" already exists error.
I have already checked this question but --fake doesn't seem to be supported via South anymore.
Your models look invalid to me, though I'd be surprised if that's what's actually causing the problem.
It looks like your first argument is intended to be the verbose_name attribute, your model should probably look like this:
class Foo(models.Model):
value = models.TextField(verbose_name = u"Value")
live = models.BooleanField(verbose_name = u"Live", default=False)
creation_time = models.DateTimeField(verbose_name = u"Creation Time", auto_now_add=True, null=True, blank=True)
(you also forgot the u before the verbose_name for creation_time).
Meanwhile, --fake is definitely still supported (see the docs), what error are you getting when you try and run it?