After experimenting with setting up CustomUser and many-to-many relationships, I decided to make some (what turned out to be major) changes to the models. I had read that CustomUser/Auth needs to be set up first, or else it'll be a mess. But since I'm just learning and there was barely any data involved, I just went for it--not to mess up the CustomUser model intentionally, but I changed a model name--and (maybe this is the critical error) I updated the names everywhere it appeared. Now I'd deleted and reset the database, VENV folder, all the migration (along with __pycache__) files, and recreated test data so many times that I thought I could download the GitHub repo and set up a fresh new app easily again. But the app refuses to run and keeps asking for a missing table.
Re-creating the app isn't my main concern-I'd like to understand what happened-what is this missing table (see the snippet of the error code)? AND: Why is a perfectly intact repo having the same issue with the missing table?
Exception in thread django-main-thread:
Traceback (most recent call last):
File "/Volumes/Volume2/dev/lms6-v2/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/Volumes/Volume2/dev/lms6-v2/venv/lib/python3.9/site-packages/django/db/backends/sqlite3/base.py", line 477, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: accounts_staff
The app is called accounts and the CustomUser models are:
class CustomUser(AbstractUser):
user_type_data = ((1, "HOD"), (2, "Staff"), (3, "Student"))
user_type = models.PositiveSmallIntegerField(default=1, choices=user_type_data)
email = models.EmailField(_('email address'), unique=True)
is_staff = models.BooleanField(default=False)
is_active = models.BooleanField(default=True)
date_joined = models.DateTimeField(default=timezone.now)
USERNAME_FIELD = 'email'
REQUIRED_FIELDS = []
objects = CustomUserManager()
def __str__(self):
return self.email
class AdminHOD(models.Model):
id = models.AutoField(primary_key=True)
admin = models.OneToOneField(CustomUser, on_delete=models.CASCADE)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now_add=True)
objects = models.Manager()
class Staff(models.Model):
id = models.AutoField(primary_key=True)
admin = models.OneToOneField(CustomUser, on_delete=models.CASCADE)
user_type = "Staff"
email = models.EmailField()
address = models.TextField(blank=True)
date_joined = models.DateTimeField(auto_now_add=True)
note = models.TextField(blank=True)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now_add=True)
objects = models.Manager()
def __str__(self):
return self.admin.first_name + " " + self.admin.last_name
def get_absolute_url(self):
return reverse('staff_detail', kwargs={'pk': self.pk})
first of all u need to delete all your migrations files(sure u have ) and the (pycache) ...
After you have done that dont just run makemigration u have to be logical with it
run python manage.py makemigrations <app_name:the app that contains the AUTH_USER_MODEL>
run python manage.py makemigrations <app_name> but this time u decide what app comes next in the way u wrote ur models they makemigrations in that order'
3)run python manage.py makemigrations
4)lastly run python manage.py migrations
... this how i solved this issue anytime i run into this problems
Obviously messing with the CustomUser model in Django isn't to be attempted casually, but this was a good learning experience. So for the newly initiated accidentally stuck at the same spot, the answer by Ogechuwku Matthew is a good start.
To elaborate-this is what I did:
Deleted all the migration files, including files in __pycache__ (EXCEPT for the __init__ file(s).
Disabled the AUTH_USER_MODEL = 'myapp.MyCustomUser' line in settings.py This step is key.
Removed (or commented out) all but the most essential files.
Commented out all but the CustomUser model and models extended from it.
Ran python manage.py makemigrations <app_name> and then 'python manage.py migrate`
After step 4 worked, ran python manage.py makemigrations <app_name> and python manage.py migrate again.
Moved the rest of the files back in, starting with the simplest bits, like 'HomeView' in views.py and 'home.html'.
When the app started up with 'home.html', un-commented models, views, and urls one cluster at a time.
Additionally: If error occur when running the app, first check the embedded dynamic links--they may extend or include files that aren't there yet-same with {% load static %}
I'm working with a project with docker-compose, where I have a postgre container.
When I run:
docker-compose -f dev.yml run django python manage.py migrate
I get the error:
django.db.utils.ProgrammingError: multiple default values specified for column "id" of table "scrapy_scrapy"
That is happening before I made some changes to my models.py file. But now the file is correct and should be working. This is the content of the models.py file:
from django.db import models
import django
# Create your models here.
class Scrapy(models.Model):
user = models.CharField(max_length=50,blank=True)
password = models.CharField(max_length=50,blank=True)
projecte = models.CharField(max_length=100)
estat_last_check = models.CharField(max_length=700, default="", blank=True)
date = models.DateTimeField(default=django.utils.timezone.now, blank=True)
app_label = ''
def __str__(self): # __unicode__ on Python 2
return self.projecte + " - " + self.user
class Meta:
app_label = 'scrapy'
As you can see, no id filed is defined anymore, so, why is complaining about that field?
I've done my research and tried some possible solutions, but no luck. I've already tried deleting the full Docker container and creating it again, or trying to delete the database.
Any ideas?
To improve my code and database, I needed to change the field name in a model away from a value that was confusing and close to a reserved word.
The original model was effectively this:
class Company(models.Model):
my_old_name = models.CharField(max_length=100, db_index=True)
The new model is effectively this:
class Company(models.Model):
my_new_name = models.CharField(max_length=100, db_index=True)
I ran:
python manage.py makemigrations my_app
and the migration code was this:
class Migration(migrations.Migration):
dependencies = [
('my_app', '0004_auto_20160605_1852'),
]
operations = [
migrations.RenameField(
model_name='company',
old_name='my_old_name',
new_name='my_new_name',
),
]
And then I executed:
python manage.py migrate
The site runs fine. All the pages show up and use 'new_name' as desired. However, I cannot use the admin site to administrate the database. It throw the error:
Error during template rendering
In template /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/s... error at line 91
'Company' object has no attribute 'my_old_name'
I've been at this for a while with no luck. Any help would be appreciated.
As an addendum to the first suggestion, this is the contents of admin.py:
from django.contrib import admin
from .models import Company, Parameters
admin.site.register(Company)
admin.site.register(Parameters)
It doesn't contain 'my_old_name'...
OK, found the error. The class model definition also contains the code:
def __str__(self):
return self.my_old_name
Changing it to:
def __str__(self):
return self.my_new_name
solved the problem. I missed this bit of code which tells admin what to display. (The model is more complex and contains another few dozen lines than snippets that I posted.)
I'm trying to modify a M2M field to a ForeignKey field. The command validate shows me no issues and when I run syncdb :
ValueError: Cannot alter field xxx into yyy they are not compatible types (you cannot alter to or from M2M fields, or add or remove through= on M2M fields)
So I can't make the migration.
class InstituteStaff(Person):
user = models.OneToOneField(User, blank=True, null=True)
investigation_area = models.ManyToManyField(InvestigationArea, blank=True,)
investigation_group = models.ManyToManyField(InvestigationGroup, blank=True)
council_group = models.ForeignKey(CouncilGroup, null=True, blank=True)
#profiles = models.ManyToManyField(Profiles, null = True, blank = True)
profiles = models.ForeignKey(Profiles, null = True, blank = True)
Any suggestions?
I stumbled upon this and although I didn't care about my data much, I still didn't want to delete the whole DB. So I opened the migration file and changed the AlterField() command to a RemoveField() and an AddField() command that worked well. I lost my data on the specific field, but nothing else.
I.e.
migrations.AlterField(
model_name='player',
name='teams',
field=models.ManyToManyField(related_name='players', through='players.TeamPlayer', to='players.Team'),
),
to
migrations.RemoveField(
model_name='player',
name='teams',
),
migrations.AddField(
model_name='player',
name='teams',
field=models.ManyToManyField(related_name='players', through='players.TeamPlayer', to='players.Team'),
),
NO DATA LOSS EXAMPLE
I would say: If machine cannot do something for us, then let's help it!
Because the problem that OP put here can have multiple mutations, I will try to explain how to struggle with that kind of problem in a simple way.
Let's assume we have a model (in the app called users) like this:
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person)
def __str__(self):
return self.name
but after some while we need to add a date of a member join. So we want this:
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through='Membership') # <-- through model
def __str__(self):
return self.name
# and through Model itself
class Membership(models.Model):
person = models.ForeignKey(Person, on_delete=models.CASCADE)
group = models.ForeignKey(Group, on_delete=models.CASCADE)
date_joined = models.DateField()
Now, normally you will hit the same problem as OP wrote. To solve it, follow these steps:
start from this point:
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person)
def __str__(self):
return self.name
create through model and run python manage.py makemigrations (but don't put through property in the Group.members field yet):
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=128)
def __str__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person) # <-- no through property yet!
def __str__(self):
return self.name
class Membership(models.Model): # <--- through model
person = models.ForeignKey(Person, on_delete=models.CASCADE)
group = models.ForeignKey(Group, on_delete=models.CASCADE)
date_joined = models.DateField()
create an empty migration using python manage.py makemigrations users --empty command and create conversion script in python (more about the python migrations here) which creates new relations (Membership) for an old field (Group.members). It could look like this:
# Generated by Django A.B on YYYY-MM-DD HH:MM
import datetime
from django.db import migrations
def create_through_relations(apps, schema_editor):
Group = apps.get_model('users', 'Group')
Membership = apps.get_model('users', 'Membership')
for group in Group.objects.all():
for member in group.members.all():
Membership(
person=member,
group=group,
date_joined=datetime.date.today()
).save()
class Migration(migrations.Migration):
dependencies = [
('myapp', '0005_create_models'),
]
operations = [
migrations.RunPython(create_through_relations, reverse_code=migrations.RunPython.noop),
]
remove members field in the Group model and run python manage.py makemigrations, so our Group will look like this:
class Group(models.Model):
name = models.CharField(max_length=128)
add members field the the Group model, but now with through property and run python manage.py makemigrations:
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through='Membership')
and that's it!
Now you need to change creation of members in a new way in your code - by through model. More about here.
You can also optionally tidy it up, by squashing these migrations.
Potential workarounds:
Create a new field with the ForeignKey relationship called profiles1 and DO NOT modify profiles. Make and run the migration. You might need a related_name parameter to prevent conflicts. Do a subsequent migration that drops the original field. Then do another migration that renames profiles1 back to profiles. Obviously, you won't have data in the new ForeignKey field.
Write a custom migration: https://docs.djangoproject.com/en/1.7/ref/migration-operations/
You might want to use makemigration and migration rather than syncdb.
Does your InstituteStaff have data that you want to retain?
If you're still developing the application, and don't need to preserve your existing data, you can get around this issue by doing the following:
Delete and re-create the db.
go to your project/app/migrations folder
Delete everything in that folder with the exception of the init.py file. Make sure you also delete the pycache dir.
Run syncdb, makemigrations, and migrate.
Another approach that worked for me:
Delete the existing M2M field and run migrations.
Add the FK field and run migrations again.
FK field added in this case has no relation to the previously used M2M field and hence should not create any problems.
This link helps you resolve all problems related to this
The one which worked for me is python3 backend/manage.py migrate --fake "app_name"
I literally had the same error for days and i had tried everything i saw here but still didn'y work.
This is what worked for me:
I deleted all the files in migrations folder exceps init.py
I also deleted my database in my case it was the preinstalled db.sqlite3
After this, i wrote my models from the scratch, although i didn't change anything but i did write it again.
Apply migrations then on the models and this time it worked and no errors.
This worked for Me as well
Delete last migrations
run command python manage.py migrate --fake <application name>
run command 'python manage.py makemigrations '
run command 'python manage.py migrate'
Hope this will solve your problem with deleting database/migrations
First delete the migrations in your app (the folders/ files under 'migrations'
folder)
Showing the 'migrations' folder
Then delete the 'db.sqlite3' file
Showing the 'db.sqlite3' file
And run python manage.py makemigrations name_of_app
Finally run python manage.py migrate
I had the same problem and found this How to Migrate a ‘through’ to a many to many relation in Django article which is really really helped me to solve this problem. Please have a look. I'll summarize his answer here,
There is three model and one(CollectionProduct) is going to connect as many-to-many relationship.
This is the final output,
class Product(models.Model):
pass
class Collection(models.Model):
products = models.ManyToManyField(
Product,
blank=True,
related_name="collections",
through="CollectionProduct",
through_fields=["collection", "product"],
)
class CollectionProduct(models.Model):
collection = models.ForeignKey(Collection, on_delete=models.CASCADE)
product = models.ForeignKey(Product, on_delete=models.CASCADE)
class Meta:
db_table = "product_collection_products"
and here is the solution,
The solution
Take your app label (the package name, e.g. ‘product’) and your M2M field name, and combine them together with and underscore:
APPLABEL + _ + M2M TABLE NAME + _ + M2M FIELD NAME
For example in our case, it’s this:
product_collection_products
This is your M2M’s through database table name. Now you need to edit your M2M’s through model to this:
Also found another solution in In Django you cannot add or remove through= on M2M fields article which is going to edit migration files. I didn't try this, but have a look if you don't have any other solution.
this happens when adding 'through' attribute to an existing M2M field:
as M2M fields are by default handled by model they are defined in (if through is set).
although when through is set to new model the M2M field is handled by that new model, hence the error in alter
solutions:-
you can reset db or
remove those m2m fields and run migration as explained above then create them again
*IF YOU ARE IN THE INITIAL STAGES OF DEVELOPMENT AND CAN AFFORD TO LOOSE DATA :)
delete all the migration files except init.py
then apply the migrations.
python manage.py makemigrations
python manage.py migrate
this will create new tables.
After being inspired by django-oscar, I tried to put a cmsplugin inside its main app. However, after doing a schemamigration --initial I can't use the plugin in Django CMS.
DatabaseError: (1146, "Table 'devel_test.cmsplugin_galleries_gallerycontainer_galleries' doesn't exist")
App Tree
gallery
apps
cmsplugin_galleries
migrations
cms_plugins.py
init.py
models.py
init.py
migrations
static
templates
admin.py
init.py
models.py
views.py
Gallery models
class Gallery(models.Model):
title = models.CharField(_(u'title'), max_length=200)
description = models.TextField(_(u'description'), blank=True)
image_folder = FilerFolderField(verbose_name=_(u'image folder'))
is_video = models.BooleanField(_(u'is video content'), default=False)
snippet = models.TextField(_(u'video snippet'))
class Meta:
verbose_name = _(u'Gallery')
verbose_name_plural = _(u'Galleries')
class GalleryImage(models.Model):
gallery = models.ForeignKey(Gallery, verbose_name=_(u'gallery'))
title = models.CharField(_(u'title'), max_length=200, blank=True)
src = FilerImageField(verbose_name=_(u'image'))
Cmsplugin_galleries models
class GalleryContainer(CMSPlugin):
title = models.CharField(_(u'title'), max_length=200)
galleries = models.ManyToManyField(Gallery, verbose_name=_(u'galleries'))
def __unicode__(self):
return u'%s' % self.title
Since I can properly run it using syncdb --all, what did I do wrong?
Notes:
Using syncdb creates a table with the name cmsplugin_galleries_gallerycontainer_galleries but using south, the table name is cmsplugin_gallerycontainer_galleries
Thanks
Since there was no other alternative I followed #daigorocub tip and replaced all the shorten_name instances of cmsplugin_gallerycontainer_galleries inside the migration file
m2m_table_name = db.shorten_name(u'cmsplugin_gallerycontainer_galleries')
to
m2m_table_name = db.shorten_name(u'cmsplugin_galleries_gallerycontainer_galleries')
This issue was fixed in the development branch of django cms 3.0 (https://github.com/divio/django-cms/issues/2291#ref-pullrequest-26529456) but since my version is older I couldn't update to the latest branch
Thank you guys for your help!