I have one model in my app running in a server with a few entries. I need to add a SlugField, unique and not-null for this model. The SlugField will be populated based on trading_name. I've changed my model in order to add this new field and modified save method:
class Supplier(StatusModel):
SLUG_MAX_LENGTH = 210
slug = models.SlugField(unique=True, max_length=SLUG_MAX_LENGTH)
trading_name = models.CharField(max_length=200, verbose_name=_('trading name'))
...
def save(self, *args, **kwargs):
self.slug = orig = slugify(self.trading_name)[:Supplier.SLUG_MAX_LENGTH]
for x in itertools.count(1):
if not Supplier.objects.filter(slug=self.slug).exists():
break
# Truncate the original slug dynamically. Minus 1 for the hyphen.
self.slug = "%s-%d" % (orig[:Supplier.SLUG_MAX_LENGTH - len(str(x)) - 1], x)
self.full_clean()
super(Supplier, self).save(*args, **kwargs)
After changing the model, I've run manage.py makemigrations and got this migration as output:
class Migration(migrations.Migration):
dependencies = [
('opti', '0003_auto_20141226_1755'),
]
operations = [
migrations.AddField(
model_name='supplier',
name='slug',
field=models.SlugField(unique=True, default='', max_length=210),
preserve_default=False,
),
]
I can't run manage.py migrate because the default value wont work due to the unique constrant.
My question is: How can I do this with Django 1.7? I need to apply the schema change and keep the current entries in my database.
Unfortunatelly, I found no answer but I could create one solution:
First I have created a migration that allows the slug field to be nullable;
Then I have created another migration that populates the slug column with proper values to every row in the model;
Then another migration which adds the not-null constraint in the column.
You do your model changes (add field, change, etc), then you call manage.py makemigrations, then apply the migrations with manage.py migrate
You can add the field with null=True, then you e.g. make a script to populate it one time
Otherwise, if you need to populate the field within the migration you can write a custom one, see https://docs.djangoproject.com/en/1.7/ref/migration-operations/#writing-your-own
Related
I have a model (devices) with a CharField primary key containing integers that I would like to convert to an IntegerField. The problem is the devices are also used as a foreign key in another tables meaning when I change the field in my models.py and run the makemigrations/migrate I keep getting an error prompting me to drop the dependent objects first. How can I change the device primary key to an IntegerField without dropping the dependent objects?
models.py
class Device(models.Model):
id = models.CharField(primary_key=True, unique=True, max_length=255)
date_created = models.DateField()
class Station(models.Model):
id = models.AutoField(primary_key=True)
device = models.ForeignKey('Device', on_delete=models.PROTECT)
error when running the migration
constraint api_station_device_id_117642ec_fk_api_device_id on table api_station depends on index api_device_pkey
HINT: Use DROP ... CASCADE to drop the dependent objects too.
The error message you are receiving is due to the fact that changing the primary key type of the Device model will require changing the foreign key type of the device field in the Station model as well. This will create a dependency that requires the api_station_device_id_117642ec_fk_api_device_id constraint on the api_station table to be dropped, which is not allowed by the database without first dropping the dependent objects.
You can create a new id field of type IntegerField in the Device model and set it as the primary key. You can then populate this new field with the integer values of the old id field and remove the old id field after updating the Station model to reference the new id field. eg in your models.py file
class Device(models.Model):
id = models.AutoField(primary_key=True)
id_char = models.CharField(unique=True, max_length=255)
date_created = models.DateField()
def save(self, *args, **kwargs):
# set the integer value of the new id field to the integer value of the old id field
if self.id_char is not None:
self.id = int(self.id_char)
super().save(*args, **kwargs)
class Station(models.Model):
id = models.AutoField(primary_key=True)
device = models.ForeignKey('Device', on_delete=models.PROTECT, to_field='id')
Write a script to populate the id field with the integer values of
the id_char field for all existing Device objects.
Here are the steps to create a custom Django management command that runs the script:
Create a new Python module (e.g., update_device_ids.py) in a Django
app that has the Device model. You can put this module in the
management/commands/ directory within your app to define a new
management command.
Import the relevant Django models and the BaseCommand class from the
django.core.management.base module. Your module may look like this
from django.core.management.base import BaseCommand
from myapp.models import Device
class Command(BaseCommand):
help = 'Updates the Device id field with the integer value of the id_char field for all devices'
def handle(self, *args, **options):
devices = Device.objects.all()
for device in devices:
if device.id_char is not None:
device.id = int(device.id_char)
device.save()
Then follow the below process:
Update the id field in your Device model from CharField to
IntegerField.
Create a new migration by running python manage.py makemigrations.
Before running the migration, create a new temporary field in your
Device model that will store the integer value of the old id_char
field. For example, you could add the following line to your Device
model:
id_int = models.IntegerField(null=True, blank=True)
Run the python manage.py update_device_ids command to populate the
id_int field with the integer values of the old id_char field.
Create a data migration by running python manage.py makemigrations --empty yourappname.
Edit the data migration file (in the migrations/ directory) and add
the following code to the forwards() method:
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('yourappname', 'the_previous_migration'),
]
operations = [
migrations.AddField(
model_name='device',
name='id_int',
field=models.IntegerField(null=True, blank=True),
),
migrations.RunSQL('UPDATE yourappname_device SET id_int = CAST(id as integer)'),
migrations.RemoveField(
model_name='device',
name='id',
),
migrations.RenameField(
model_name='device',
old_name='id_int',
new_name='id',
),
]
This code adds a new IntegerField field called id_int, populates it with the integer values of the old id_char field using a SQL command, removes the old id_char field, and renames the id_int field to id.
Finally, run the data migration by running python manage.py migrate.
I have added a new field in my model but after that I have deleted db.sqlite3 (to ensure I don't get error below)
agrawalo#:~/myapp> ls
README.md config core manage.py requirements.txt
But still I get this error when I run makemigrations
agrawalo#:~/myapp> ./manage.py makemigrations
You are trying to add a non-nullable field 'high52' to stock without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows with a null value for this column)
2) Quit, and let me add a default in models.py
class Stock(models.Model):
name = models.CharField(max_length=255)
code = models.CharField(max_length=20, db_index=True)
price = models.DecimalField(max_digits=19, decimal_places=2)
diff = models.DecimalField(max_digits=19, decimal_places=2)
open_price = models.DecimalField(max_digits=19, decimal_places=2)
previous_close = models.DecimalField(max_digits=19, decimal_places=2)
low52 = models.DecimalField(max_digits=19, decimal_places=2)
high52 = models.DecimalField(max_digits=19, decimal_places=2)
last_updated = models.DateTimeField()
objects = DataFrameManager()
def save(self, *args, **kwargs):
''' On save, update timestamps '''
self.last_updated = timezone.now()
return super(Stock, self).save(*args, **kwargs)
def __str__(self):
return str(self.code)
low52 and high52 are the newly added fields. Please note that none of the other existing field throw this error.
You can either provide a default value to the field
high52 = models.DecimalField(max_digits=19, decimal_places=2, default=0.0)
or you can make it optional
high52 = models.DecimalField(max_digits=19, decimal_places=2, null=True, blank=True)
You can make a decision based on your choice.
To answer your question about the error, the previously existing fields might have been created in the initial migration itself and they don't need a default value. But for the newly added field, you need a default value for mandatory fields and this default value will be populated in the existing records. This doesn't depend on whether you have deleted the existing database or not. This depends on the current state of migrations for that model. Since this is not the initial migration, you will need to provide a default value or make it optional.
It doesn't matter if you deleted the database file or not. makemigrations does not check the database.
You can only add a non-nullable field to a model if you add it to a new model and make an initial migration. This is because, after you make that initial migration, Django has no way of knowing whether you deployed your application somewhere else, so it has no way of knowing if there are instances of a model out there. A situation where this would go wrong:
Create a model X and makemigrations on your local machine.
Deploy your Django application to a server, where the database is populated with instances of model X.
Delete your local database, add non-nullable field Y to model X, makemigrations.
Deploy you Django application to the server.
Problems occur.
The solution here is to either:
Set the Field to null=True
Add a default to the model.
Provide a default when making the migrations.
In your situation, I would say it is ok to provide a one-off default, because it sounds like you have no populated database yet.
You need to provide blank and null True for high52 field .
high52 = models.SomeField(blank=True,null=True)
If you don't want so then you can select any of these two options.
For example If high52 is CharField then you can choose the 1 option and provide some value like '..' or you can set defaults in your models.py
I am importing an existing database into it's own Django project. I have generated the initial models from the database, via inspectdb, and am enabling Django to control each table one at a time by commenting the managed=False lines in the table meta settings. I've started with the simple models and am hitting a snag when enabling tables with foreign keys. Django keeps generating the same migration for the foreign key DocTagID and I'm not sure why it is doing so ?
The table in question is shown below, everything is as generated by inspectdb with the exception of the commented line which is where I pass control of the table over to Django.
class Doctagversion(models.Model):
id = models.IntegerField(db_column='Id', primary_key=True, blank=True) # Field name made lowercase.
doctagid = models.ForeignKey(DocTag, models.DO_NOTHING, db_column='DocTagId') # Field name made lowercase.
groupname = models.TextField(db_column='GroupName') # Field name made lowercase.
name = models.TextField(db_column='Name') # Field name made lowercase.
creationdate = models.DateTimeField(db_column='CreationDate') # Field name made lowercase.
lasteditdate = models.DateTimeField(db_column='LastEditDate', blank=True, null=True) # Field name made lowercase.
lastedituserid = models.IntegerField(db_column='LastEditUserId') # Field name made lowercase.
lastedituserdisplayname = models.TextField(db_column='LastEditUserDisplayName') # Field name made lowercase.
releasedate = models.DateTimeField(db_column='ReleaseDate', blank=True, null=True) # Field name made lowercase.
class Meta:
# managed = False
db_table = 'DocTagVersion'
Before passing on this control an initial migration for the schema in question is generated using python -m manage.py makemigrations, and applied with python -m manage.py migrate. This initial migration for the table is as follows, managed is set to False initially and the commented line is an entry I believe I should add to inform Django of the foreign key (inspectdb states as much in the generated models.py).
migrations.CreateModel(
name='Doctagversion',
fields=[
('id', models.IntegerField(blank=True, db_column='Id', primary_key=True, serialize=False)),
# ('doctagid',models.ForeignKey(db_column='DocTagId', default=-1, on_delete=models.deletion.DO_NOTHING, to='DocTag')),
('groupname', models.TextField(db_column='GroupName')),
('name', models.TextField(db_column='Name')),
('creationdate', models.DateTimeField(db_column='CreationDate')),
('lasteditdate', models.DateTimeField(blank=True, db_column='LastEditDate', null=True)),
('lastedituserid', models.IntegerField(db_column='LastEditUserId')),
('lastedituserdisplayname', models.TextField(db_column='LastEditUserDisplayName')),
('releasedate', models.DateTimeField(blank=True, db_column='ReleaseDate', null=True)),
],
options={
'db_table': 'DocTagVersion',
'managed': False,
},
),
When I enable control over the table the first migration simply changes the table options.
migrations.AlterModelOptions(
name='doctagversion',
options={},
),
Django adds the foreign key in question if it's not present in the initial migration as follows.
migrations.AddField(
model_name='doctagversion',
name='doctagid',
field=models.ForeignKey(db_column='DocTagId', default=-1, on_delete=django.db.models.deletion.DO_NOTHING, to='docutoo.DocTag'),
preserve_default=False,
),
Thereafter it repeatedly generates the following migration as one cycles between python m manage.py makemigrations and python -m manage.py migrate.
migrations.AlterField(
model_name='doctagversion',
name='doctagid',
field=models.ForeignKey(db_column='DocTagId', on_delete=django.db.models.deletion.DO_NOTHING, to='docutoo.DocTag'),
),
Perhaps my strategy is wrong and I should simply enable all tables in one migration ?
As far as I can tell the following related question(s) do not account for my situation :
Assigning function result instead of function to field attribute
From this Bug Report it seems Django migrations are quite sensitive to naming, I had commented out the db_table in all of my models. I believe I did this after creating the initial migration unwittingly breaking later ones.
class Meta :
...
# db_table = "TableName"
Because I broke the naming makemigrations could not see that the table name had changed and was doing it's best to resolve it by repeatedly declaring the field.
Aside
Coincidentally when one creates the initial migration for an existing database as described in my question (inspectdb -> makemigrations -> migrate), Django traverses the models alphabetically creating tables in the same order and ignoring ones foreign keys. It creates the tables alphabetically and then modifies them later to include foreign keys, clobbering any data that might be present in a existing database. It seems one must define the foreign keys as integer fields initially and change them back as one lets Django manage the tables. Alternatively one can comment out all of their models and generate migrations as one uncomments them in a fashion that resolves the foreign keys; Django will then link them accordingly. One may then squash the migrations into a single migration and set this as the initial migration.
I have a model that looks like so:
class FoodPreferences(models.Model):
foods = models.ManyToManyField(to='Food', db_index=True,
through='FoodToPreferenceMap')
is_vegan = models.BooleanField(default=True)
some_new_unique_field = models.CharField(max_length=64, unique=True)
and now I want to add the some_new_unique_field to the model, but because it has no default value migrations would ask me for a new value. The field is supposed to be computed based on each row's foods values. How should I do approach this? is it possible to set a default value to be computed dynamically?
EDIT:
I'm giving custom migrations a try and here's my migration:
def gen_stuff(apps, schema_editor):
MyModel = apps.get_model('myapp', 'foodpreferences')
while MyModel.objects.filter(some_new_unique_field__isnull=True).exists():
with transaction.atomic():
for row in MyModel.objects.filter(some_new_unique_field__isnull=True)[:1000]:
row.some_new_unique_field = row.generate_new_field_stuff()
row.save(update_fields=['some_new_unique_field'])
class Migration(migrations.Migration):
dependencies = [
('stars_api', '0074_foodpreferences_ some_new_unique_field'),
]
operations = [
migrations.RunPython(gen_stuff, reverse_code=migrations.RunPython.noop)
]
but I'm running into
AttributeError: 'FoodPreferences' object has no attribute 'generate_new_field_stuff'
EDIT: Fixed it, it was creating a shallow copy using get_model
Here is a suggestion given on the django doc. You can give it a try. And in short it says:
First creat a migration that allows the new_field to be nullable;
Then create another migration that populates the new_field with proper values to every row in the model;
Then another migration which adds the not-null constraint in the column.
I'm trying to modify a M2M field to a ForeignKey field. The command validate shows me no issues and when I run syncdb :
ValueError: Cannot alter field xxx into yyy they are not compatible types (you cannot alter to or from M2M fields, or add or remove through= on M2M fields)
So I can't make the migration.
class InstituteStaff(Person):
user = models.OneToOneField(User, blank=True, null=True)
investigation_area = models.ManyToManyField(InvestigationArea, blank=True,)
investigation_group = models.ManyToManyField(InvestigationGroup, blank=True)
council_group = models.ForeignKey(CouncilGroup, null=True, blank=True)
#profiles = models.ManyToManyField(Profiles, null = True, blank = True)
profiles = models.ForeignKey(Profiles, null = True, blank = True)
Any suggestions?
I stumbled upon this and although I didn't care about my data much, I still didn't want to delete the whole DB. So I opened the migration file and changed the AlterField() command to a RemoveField() and an AddField() command that worked well. I lost my data on the specific field, but nothing else.
I.e.
migrations.AlterField(
model_name='player',
name='teams',
field=models.ManyToManyField(related_name='players', through='players.TeamPlayer', to='players.Team'),
),
to
migrations.RemoveField(
model_name='player',
name='teams',
),
migrations.AddField(
model_name='player',
name='teams',
field=models.ManyToManyField(related_name='players', through='players.TeamPlayer', to='players.Team'),
),
NO DATA LOSS EXAMPLE
I would say: If machine cannot do something for us, then let's help it!
Because the problem that OP put here can have multiple mutations, I will try to explain how to struggle with that kind of problem in a simple way.
Let's assume we have a model (in the app called users) like this:
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person)
def __str__(self):
return self.name
but after some while we need to add a date of a member join. So we want this:
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through='Membership') # <-- through model
def __str__(self):
return self.name
# and through Model itself
class Membership(models.Model):
person = models.ForeignKey(Person, on_delete=models.CASCADE)
group = models.ForeignKey(Group, on_delete=models.CASCADE)
date_joined = models.DateField()
Now, normally you will hit the same problem as OP wrote. To solve it, follow these steps:
start from this point:
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person)
def __str__(self):
return self.name
create through model and run python manage.py makemigrations (but don't put through property in the Group.members field yet):
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person) # <-- no through property yet!
def __str__(self):
return self.name
class Membership(models.Model): # <--- through model
person = models.ForeignKey(Person, on_delete=models.CASCADE)
group = models.ForeignKey(Group, on_delete=models.CASCADE)
date_joined = models.DateField()
create an empty migration using python manage.py makemigrations users --empty command and create conversion script in python (more about the python migrations here) which creates new relations (Membership) for an old field (Group.members). It could look like this:
# Generated by Django A.B on YYYY-MM-DD HH:MM
import datetime
from django.db import migrations
def create_through_relations(apps, schema_editor):
Group = apps.get_model('users', 'Group')
Membership = apps.get_model('users', 'Membership')
for group in Group.objects.all():
for member in group.members.all():
Membership(
person=member,
group=group,
date_joined=datetime.date.today()
).save()
class Migration(migrations.Migration):
dependencies = [
('myapp', '0005_create_models'),
]
operations = [
migrations.RunPython(create_through_relations, reverse_code=migrations.RunPython.noop),
]
remove members field in the Group model and run python manage.py makemigrations, so our Group will look like this:
class Group(models.Model):
name = models.CharField(max_length=128)
add members field the the Group model, but now with through property and run python manage.py makemigrations:
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through='Membership')
and that's it!
Now you need to change creation of members in a new way in your code - by through model. More about here.
You can also optionally tidy it up, by squashing these migrations.
Potential workarounds:
Create a new field with the ForeignKey relationship called profiles1 and DO NOT modify profiles. Make and run the migration. You might need a related_name parameter to prevent conflicts. Do a subsequent migration that drops the original field. Then do another migration that renames profiles1 back to profiles. Obviously, you won't have data in the new ForeignKey field.
Write a custom migration: https://docs.djangoproject.com/en/1.7/ref/migration-operations/
You might want to use makemigration and migration rather than syncdb.
Does your InstituteStaff have data that you want to retain?
If you're still developing the application, and don't need to preserve your existing data, you can get around this issue by doing the following:
Delete and re-create the db.
go to your project/app/migrations folder
Delete everything in that folder with the exception of the init.py file. Make sure you also delete the pycache dir.
Run syncdb, makemigrations, and migrate.
Another approach that worked for me:
Delete the existing M2M field and run migrations.
Add the FK field and run migrations again.
FK field added in this case has no relation to the previously used M2M field and hence should not create any problems.
This link helps you resolve all problems related to this
The one which worked for me is python3 backend/manage.py migrate --fake "app_name"
I literally had the same error for days and i had tried everything i saw here but still didn'y work.
This is what worked for me:
I deleted all the files in migrations folder exceps init.py
I also deleted my database in my case it was the preinstalled db.sqlite3
After this, i wrote my models from the scratch, although i didn't change anything but i did write it again.
Apply migrations then on the models and this time it worked and no errors.
This worked for Me as well
Delete last migrations
run command python manage.py migrate --fake <application name>
run command 'python manage.py makemigrations '
run command 'python manage.py migrate'
Hope this will solve your problem with deleting database/migrations
First delete the migrations in your app (the folders/ files under 'migrations'
folder)
Showing the 'migrations' folder
Then delete the 'db.sqlite3' file
Showing the 'db.sqlite3' file
And run python manage.py makemigrations name_of_app
Finally run python manage.py migrate
I had the same problem and found this How to Migrate a ‘through’ to a many to many relation in Django article which is really really helped me to solve this problem. Please have a look. I'll summarize his answer here,
There is three model and one(CollectionProduct) is going to connect as many-to-many relationship.
This is the final output,
class Product(models.Model):
pass
class Collection(models.Model):
products = models.ManyToManyField(
Product,
blank=True,
related_name="collections",
through="CollectionProduct",
through_fields=["collection", "product"],
)
class CollectionProduct(models.Model):
collection = models.ForeignKey(Collection, on_delete=models.CASCADE)
product = models.ForeignKey(Product, on_delete=models.CASCADE)
class Meta:
db_table = "product_collection_products"
and here is the solution,
The solution
Take your app label (the package name, e.g. ‘product’) and your M2M field name, and combine them together with and underscore:
APPLABEL + _ + M2M TABLE NAME + _ + M2M FIELD NAME
For example in our case, it’s this:
product_collection_products
This is your M2M’s through database table name. Now you need to edit your M2M’s through model to this:
Also found another solution in In Django you cannot add or remove through= on M2M fields article which is going to edit migration files. I didn't try this, but have a look if you don't have any other solution.
this happens when adding 'through' attribute to an existing M2M field:
as M2M fields are by default handled by model they are defined in (if through is set).
although when through is set to new model the M2M field is handled by that new model, hence the error in alter
solutions:-
you can reset db or
remove those m2m fields and run migration as explained above then create them again
*IF YOU ARE IN THE INITIAL STAGES OF DEVELOPMENT AND CAN AFFORD TO LOOSE DATA :)
delete all the migration files except init.py
then apply the migrations.
python manage.py makemigrations
python manage.py migrate
this will create new tables.