An interesting issue.
Getting Unknown column exception -- Please find the stack trace
I try to get new leads list and responded leads. I merge them. When I merge them there is an exception.
After debugging its found that new_leads method has exclude of two fields collection and delivery . If we make it one exclude all is well . I mean dont check the other, if we include both the filters we have an issue.
I tried using filter/ exclude etc. but it didnt work.
Query Set contains following method
def all_leads_related_to_user(self, user):
""" User new and past leads
Use this queryset for performing lead search.
"""
new_leads = self.new_leads_for_user(user)
responded_leads = self.leads_responded_by_user(user)
all_leads = (new_leads | responded_leads).distinct() <= Issue is here.
return all_leads
def new_leads_for_user(self, user):
....
# User's location filter
if user.sub_region_excluded_list:
sub_region_exclude_list = [10, 12]
qs = qs.exclude( Q(collection_point__sub_region_id__in=sub_region_exclude_list) |
Q(delivery_point__sub_region_id__in=sub_region_exclude_list))
# <== Make it just one exclude it works.
Model
class Suburb(models.Model):
state = models.ForeignKey(State, blank=False)
sub_region = models.ForeignKey(SubRegion, blank=False)
postcode = models.CharField(_('postcode'), blank=False, max_length=10)
name = models.CharField(_('suburb name'), blank=False, max_length=200)
class Load(models.Model):
.....
collection_point = models.ForeignKey(Suburb, related_name='collection_point', on_delete=models.SET_NULL, null=True)
delivery_point = models.ForeignKey(Suburb, related_name='delivery_point', on_delete=models.SET_NULL, null=True)
Stack Trace:-
>>> Load.objects.all_leads_related_to_user(User.objects.all()[0])
Load.objects.all_leads_related_to_user(User.objects.all()[0])
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/data/fq/venv/lib/python3.4/site-packages/django/db/models/query.py", line 226, in __repr__
data = list(self[:REPR_OUTPUT_SIZE + 1])
File "/data/fq/venv/lib/python3.4/site-packages/django/db/models/query.py", line 250, in __iter__
self._fetch_all()
File "/data/fq/venv/lib/python3.4/site-packages/django/db/models/query.py", line 1103, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/data/fq/venv/lib/python3.4/site-packages/django/db/models/query.py", line 53, in __iter__
results = compiler.execute_sql(chunked_fetch=self.chunked_fetch)
File "/data/fq/venv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 886, in execute_sql
raise original_exception
File "/data/fq/venv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 876, in execute_sql
cursor.execute(sql, params)
File "/data/fq/venv/lib/python3.4/site-packages/django/db/backends/utils.py", line 80, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/data/fq/venv/lib/python3.4/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/data/fq/venv/lib/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/data/fq/venv/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/data/fq/venv/lib/python3.4/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/data/fq/venv/lib/python3.4/site-packages/django/db/backends/mysql/base.py", line 101, in execute
return self.cursor.execute(query, args)
File "/data/fq/venv/lib/python3.4/site-packages/MySQLdb/cursors.py", line 250, in execute
self.errorhandler(self, exc, value)
File "/data/fq/venv/lib/python3.4/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
raise errorvalue
File "/data/fq/venv/lib/python3.4/site-packages/MySQLdb/cursors.py", line 247, in execute
res = self._query(query)
File "/data/fq/venv/lib/python3.4/site-packages/MySQLdb/cursors.py", line 411, in _query
rowcount = self._do_query(q)
File "/data/fq/venv/lib/python3.4/site-packages/MySQLdb/cursors.py", line 374, in _do_query
db.query(q)
File "/data/fq/venv/lib/python3.4/site-packages/MySQLdb/connections.py", line 292, in query
_mysql.connection.query(self, query)
django.db.utils.OperationalError: (1054, "Unknown column 'locations_suburb.sub_region_id' in 'having clause'")
>>>
I'm using MySqlDB
Note:-
All migrations are applied and Db is in right state
Update
Issue is related to MYSQL madating columns to be available in select statements when its creating a Having clause . Refer to -
Unknown column in 'having clause'.
Django in this instance is not adding as the column to be selected. hence the error.
I need to some how find a way to add this in the select clause with other params.
This usually pops up when you haven't made or applied migrations. Specifically, when you modify the fields any model in Django (or any ORM), you need to inform the SQL server so it can reflect it in its tables. Modern Django implements this by a series of migrations, so that if you have data from any time in the life of your project, you can run it on code from any time in history by simply running the migrations forwards or backwards.
Long story short, MySQL claims the sub_region field doesn't exist. You need to sync the tables to reflect your models.
There are two steps, making the migrations and running them on your server, shown below for your locations app. Take a backup before running the second command, especially on MySQL or SQLite!
$ python manage.py makemigrations locations
$ python manage.py migrate locations
This will cause the database server to create the column, and you should no longer get an OperationalError.
It is not a migration issue.
When having a HAVING Clause MYSQL mandates those parameters to be in selected, if not selected it will throw an exception.
Therefore I had to force Django to include both the fields by:
qs = qs.annotate(collection_point__sub_region_id = F("collection_point__sub_region_id"),
delivery_point__sub_region_id = F("delivery_point__sub_region_id"))
This is the answer to this issue:
Step 1. You need to fake the migrations
python manage.py migrate --fake
Step 2. Migrate again
python manage.py migrate
Step 3. Comment out the column for which the error comes up e.g. "Can't DROP 'address'
Step 4. Make migrations and migrate again
python manage.py makemigrations
python manage.py migrate
Since my table didn't have any data, I deleted it and recreated it on MySQL and then deleted it from Django. Finally I recreated on Django thus solving the error.
Related
Just like the title of the page says, I'm Getting Started with Tastypie by following the linked tutorial. But when I try to load /api/entry/?format=json, I get HTTP 500 response, with this error message:
no such table: myapp_entry
When I look in sqlite3, indeed, there's no such table.
Here's how I followed the tutorial:
$ django-admin startproject mysite
$ cd mysite
$ django-admin startapp myapp
I created/edited myapp/models.py, myapp/api.py and mysite/urls.py as specified in the tutorial, and added 'tastypie' to my INSTALLED_APPS in mysite/settings.py.
Note: It wasn't clear to me which urls.py file to edit or create, so I edited the existing one in mysite. So now it looks like this:
from django.conf.urls import url, include
from myapp.api import EntryResource
from django.contrib import admin
entry_resource = EntryResource()
urlpatterns = [
url(r'^admin/', admin.site.urls),
# url(r'^blog/', include('myapp.urls')),
url(r'^api/', include(entry_resource.urls)),
]
I commented out the 'blog' line, because it caused the error ImportError: No module named 'myapp.urls'. I think this is the step I'm stuffing up, but when I tried putting the tutorial code in myapp/urls.py instead, I got a 404 when I tried loading the page, and when I then tried adding url(r'^blog/', include('myapp.urls')) to mysite/urls.py, I got a stack overflow. So I've gone back to the code as shown above.
To be clear, here's what my file structure looks like now:
$ find . -type f -not -name '*.pyc'
./manage.py
./myapp/__init__.py
./myapp/views.py
./myapp/models.py
./myapp/tests.py
./myapp/admin.py
./myapp/apps.py
./myapp/migrations/__init__.py
./myapp/api.py
./db.sqlite3
./mysite/__init__.py
./mysite/settings.py
./mysite/urls.py
./mysite/wsgi.py
The one other change I made was adding a Meta subclass to my Entry class, so the first dozen lines look like this:
class Entry(models.Model):
user = models.ForeignKey(User)
pub_date = models.DateTimeField(default=now)
title = models.CharField(max_length=200)
slug = models.SlugField(null=True, blank=True)
body = models.TextField()
class Meta:
app_label = 'myapp'
# __unicode__() and save() as in the tutorial
If I don't do that, I get this in the console: RuntimeError: Model class myapp.models.Entry doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
Another thing I tried was python manage.py migrate. It found things to do, but this hasn't fixed this error.
Here's what I've got installed (in requirements.txt/virtualenv):
Django (1.10)
django-tastypie (0.13.3)
PyYAML (3.12)
mysqlclient (1.3.10) (although I haven't actually set it up yet—I'm using the default sqlite3 setup for now)
I'm running Python 3.4.3, but I got exactly the same error using a quite similar setup in Python 2.7.6.
Finally, here's the full stack trace from that HTTP 500 page:
{"error_message": "no such table: myapp_entry", "traceback": "Traceback (most recent call last):
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/backends/sqlite3/base.py", line 337, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: no such table: myapp_entry
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "~/myproject/virtualenv/lib/python3.4/site-packages/tastypie/resources.py", line 219, in wrapper
response = callback(request, *args, **kwargs)
File "~/myproject/virtualenv/lib/python3.4/site-packages/tastypie/resources.py", line 450, in dispatch_list
return self.dispatch('list', request, **kwargs)
File "~/myproject/virtualenv/lib/python3.4/site-packages/tastypie/resources.py", line 482, in dispatch
response = method(request, **kwargs)
File "~/myproject/virtualenv/lib/python3.4/site-packages/tastypie/resources.py", line 1335, in get_list
to_be_serialized = paginator.page()
File "~/myproject/virtualenv/lib/python3.4/site-packages/tastypie/paginator.py", line 194, in page
count = self.get_count()
File "~/myproject/virtualenv/lib/python3.4/site-packages/tastypie/paginator.py", line 126, in get_count
return self.objects.count()
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/models/query.py", line 369, in count
return self.query.get_count(using=self.db)
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/models/sql/query.py", line 476, in get_count
number = obj.get_aggregation(using, ['__count'])['__count']
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/models/sql/query.py", line 457, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/models/sql/compiler.py", line 835, in execute_sql
cursor.execute(sql, params)
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "~/myproject/virtualenv/lib/python3.4/site-packages/django/db/backends/sqlite3/base.py", line 337, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: no such table: myapp_entry
"}
Can anybody see what I'm doing wrong? Or does the tutorial not apply to these versions?
sqlite3.OperationalError: no such table: myapp_entry
For one, your project is using sqlite, not mysql. You should update your DATABASES setting.
Add myapp to your INSTALLED_APPS in your project settings.
Then, run ./manage.py migrate. After fixing INSTALLED_APPS Django should be able to find the app's Entry model and create the table for it.
models.py
class Stop(models.Model):
idn = models.PositiveIntegerField(primary_key=True, unique=True)
label = models.CharField(null=False, blank=False, max_length=512)
coor_x = models.FloatField()
coor_y = models.FloatField()
buses = models.ManyToManyField(Bus)
latest_query_datetime = models.DateTimeField(default=datetime(2000, 1, 1, 0, 0, 0))
latest_query_data = JSONField(default={})
class Meta:
ordering = ["label"]
def __str__(self):
return self.label
When I run:
python3 manage.py makemigrations && python3 manage.py migrate
It raises a ProgrammingError saying that jsonb datatype does not exist:
Migrations for 'rest':
0007_auto_20160612_1301.py:
- Alter field latest_query_data on stop
Operations to perform:
Apply all migrations: contenttypes, rest, auth, sessions, admin
Running migrations:
Rendering model states... DONE
Applying rest.0005_auto_20160612_1237...Traceback (most recent call last):
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
psycopg2.ProgrammingError: type "jsonb" does not exist
LINE 1: ... TABLE "rest_stop" ADD COLUMN "latest_query_data" jsonb DEFA...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
utility.execute()
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/core/management/__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/core/management/commands/migrate.py", line 200, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/migrations/executor.py", line 92, in migrate
self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/migrations/executor.py", line 121, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/migrations/executor.py", line 198, in apply_migration
state = migration.apply(state, schema_editor)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/migrations/migration.py", line 123, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/migrations/operations/fields.py", line 62, in database_forwards
field,
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/backends/base/schema.py", line 396, in add_field
self.execute(sql, params)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/backends/base/schema.py", line 110, in execute
cursor.execute(sql, params)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/utils.py", line 95, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/home/erayerdin/.venv/eshot-api/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: type "jsonb" does not exist
LINE 1: ... TABLE "rest_stop" ADD COLUMN "latest_query_data" jsonb DEFA...
I use PostgreSQL to use JSONField and update it when a user requests a view. If I do not use default={}, it tells me to make one.
Further Analysis
I changed latest_query_data field to TextField so that I can store as string and convert to dict when I need. However, this also raised the same error.
Environment
django 1.9.6
psycopg 2.6.1
According to the Django docs, JSONField requires PostgreSQL ≥ 9.4 and Psycopg2 ≥ 2.5.4
What PostgreSQL version are you using?
See https://docs.djangoproject.com/en/dev/ref/contrib/postgres/fields/#django.contrib.postgres.fields.JSONField
Ubuntu 14.04 repositories contain only 9.3 version. You can review this to upgrade your version.
Based on the Anonymous comment, I found the following to work:
from django.contrib.postgres import fields
class OldJSONField(fields.JSONField):
def db_type(self, connection):
return 'json'
class Stop(models.Model):
...
latest_query_data = OldJSONField(default=dict)
...
If you are getting this error and you have installed Postgres > 9.4 then I would check that you aren't connecting to an older version of Postgres that is also installed on your instance.
To confirm what you are connecting to from Django you can use psycopg2 from the shell:
import psycopg2
conn = psycopg2.connect("dbname=<your database> user=<your user> password=<your password>")
cur = conn.cursor()
cur.execute("SELECT Version();")
cur.fetchone()
Make sure the version here is > 9.4. If not you probably have a couple of versions installed and your service configuration is pointing at the other version.
With older versions of PostgreSQL for example "PostgreSQL9.2.x",
We can use an alternative as
from jsonfield import JSONField
instead of:
from django.contrib.postgres.fields import JSONField
for example:
from django.db import models
from jsonfield import JSONField
class MyModel(models.Model):
json = JSONField()
Can install as:
pip install jsonfield
Check this out:
Add support for PostgreSQL 9.2+'s native json type. #32
This solution is especially good for some limited edition versions. For example,
on VPS & Cpanel that supports postgresql9.2 by default
more details!, see"rpkilby-jsonfield"
So, it seems a bug on psycopg2 or django, I will post an issue on both repositories. This is how I've solved (at least, ProgrammingError) the problem.
Change JSONField to TextField.
Flush your database.
Beware! This operation will erase all the data but the structure in your database.
Remove all migrations folder in all of your apps.
Run python3 manage.py makemigrations && python3 manage.py migrate in all of your apps.
Run python manage.py makemigrations <appname> && python3 manage.py migrate <appname> for each app you have.
Use built-in json module to convert between str and dict.
However, remember, this solution requires so much effort if you want to filter a QuerySet of a model. I do not recommend it, but there was no other solution to get rid of this error and all I needed to do was to save data and represent it.
! This answer will be accepted as default if there will not occur any other better solution in 48 hours.
I asked a question to try and fix an issue I was having where I didn't understand how to use ForeignKeys properly. That was very helpfully fixed, although I had a subsequent problem with django.db.utils.IntegrityError: core_team.blahblah_id may not be NULL and I decided to roll back, do something slightly simpler - in order to avoid a double lookup (Match is linked to both Team and League), I would write a management command to import the teams.
On my (clearly flawed) understanding from the previous question, I've done it right - it uses get_or_create to check for the league instance and then assigns the team based on that. I've also doublechecked that the DB is up to date (I'm running south and did the forward migration the last time I changed the scheme, nothing's changed since then. Last change was to make the names in both models the primary key (as there's only one team of each name, only one league of each name.)
Most recently, I've added code to provide the default to the team get_or_create section, but am receiving the same error. I understand the cause (I think) of the error - that the ForeignKey 'league' in Team already exists in the database, and can't be null inserting another team (from https://docs.djangoproject.com/en/1.5/ref/models/querysets/#get-or-create), just not how to fix it.
Management command:
from django.core.management.base import BaseCommand, CommandError
import csv
import csvImporter
#from core.models import Match
from time import strptime
from datetime import datetime
master_data = open ('/Users/chris/Desktop/AllDataTruncated.csv', 'r')
data = list(tuple(rec) for rec in csv.reader(master_data, delimiter=','))
from core.models import League, Team
team_list = []
for row in data:
if row[2] == "HomeTeam":
print "Continuing"
continue
elif row[2] == "":
print "Continuing"
continue
else:
league, _ = League.objects.get_or_create(name=row[0])
print league
team, _ = Team.objects.get_or_create(team_name=row[2], defaults={'league':league})
current_team = Team(league = league, team_name=team)
print current_team
And relevant bits of models.py:
class League (models.Model):
name = models.CharField(max_length=2, primary_key=True)
last_modified = models.CharField(max_length=50)
def __unicode__(self):
return unicode(self.name)
class Team(models.Model):
team_name = models.CharField(max_length=50, primary_key=True)
league = models.ForeignKey(League)
team_colour = models.CharField(max_length=6, null=True, blank=True)
def __unicode__(self):
return unicode (self.team_name)
The full traceback is:
$ python manage.py importteams
Continuing
E0
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 453, in execute_from_command_line
utility.execute()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 272, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/core/management/__init__.py", line 77, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Users/chris/Dropbox/Django/gmblnew/core/management/commands/importteams.py", line 26, in <module>
team2, _ = Team.objects.get_or_create(team_name=row[3])
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/manager.py", line 146, in get_or_create
return self.get_query_set().get_or_create(**kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 487, in get_or_create
six.reraise(*exc_info)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 477, in get_or_create
obj.save(force_insert=True, using=self.db)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/base.py", line 546, in save
force_update=force_update, update_fields=update_fields)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/base.py", line 650, in save_base
result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/manager.py", line 215, in _insert
return insert_query(self.model, objs, fields, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/query.py", line 1661, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 937, in execute_sql
cursor.execute(sql, params)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/util.py", line 41, in execute
return self.cursor.execute(sql, params)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 364, in execute
six.reraise(utils.IntegrityError, utils.IntegrityError(*tuple(e.args)), sys.exc_info()[2])
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 362, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.IntegrityError: core_team.league_id may not be NULL
Now - I think that it's odd that it's saying league_id here, as this shouldn't be relevant anymore? When I did the migration, the question came up:
? The field 'League.id' does not have a default specified, yet is NOT NULL.
? Since you are removing this field, you MUST specify a default
? value to use for existing rows. Would you like to:
? 1. Quit now.
? 2. Specify a one-off value to use for existing columns now
? 3. Disable the backwards migration by raising an exception; you can edit the migration to fix it later
? Please select a choice: 3
Is this what's perpetuating this issue?
Edit: Seems not. Dropped the DB and moved the South migrations folder, and it's still doing it. The source CSV is also fine (no blank lines or empty strings/segments), and the code above has a section to skip those segments anyway; it's not getting that far.
Ugh. The answer to this, for any other newbies who are coming to it later, is actually ludicrously simple. What I'm doing here is creating an entry in the table 'Team', which has a ForeignKey going back to 'League'.
The 'trick' (it's not a trick, just really badly explained in the documentation, imho) is that you need to explicitly pass the league back when you do the get_or_create for the Team object. It's not just about matching the team name
I thought I'd done this, but I hadn't, it appears. This code works (and quite effectively ensures there are no duplicates):
for row in data:
if row[2] == "HomeTeam":
print "Continuing"
continue
elif row[2] == "":
print "Continuing"
continue
else:
league, _ = League.objects.get_or_create(name=row[0])
print league
team, _ = Team.objects.get_or_create(team_name=row[2], league=league)
current_team = Team(league = league, team_name=team)
print current_team
I have a Django site that's working fine. It has a small amount of data in the database which I want to use for testing. I've done dumpdata to generate a few .json fixtures. But, when I try to run a test on my app ("tagger") I get a Postgresql error, and I can't work out how to solve it:
(django-projectname)$ ./manage.py test tagger
Creating test database for alias 'default'...
Problem installing fixture '/Users/phil/Projects/RIG/projectname/django-projectname/projectname/tagger/fixtures/auth_testdata.json': Traceback (most recent call last):
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/core/management/commands/loaddata.py", line 174, in handle
obj.save(using=using)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/core/serializers/base.py", line 165, in save
models.Model.save_base(self.object, using=using, raw=True)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/base.py", line 526, in save_base
rows = manager.using(using).filter(pk=pk_val)._update(values)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/query.py", line 491, in _update
return query.get_compiler(self.db).execute_sql(None)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 869, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/Users/phil/.virtualenvs/django-projectname/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
IntegrityError: duplicate key value violates unique constraint "auth_permission_content_type_id_codename_key"
DETAIL: Key (content_type_id, codename)=(3, add_htuser) already exists.
I tried doing dumpdata for the auth app using the --natural flag too, but that didn't change anything. I'm stumped, and can't work out where it's getting a duplicate key from.
The "htuser" class that it refers to is a Proxy model on the auth User class.
I'm using South for my apps' migrations and it sounds like https://stackoverflow.com/a/2888916/250962 may be the answer, but I don't understand how to "Remove the explicitly-set primary key values from the datamigrations"?
I am relatively new to django. I have defined my db schema and validated it with no errors (manage.py validate reports 0 errors found).
Yet when I run ./manage.py syncdb
I get the following stack trace:
Creating table demo_foobar_one
Creating table demo_foobar_two
<snip>...</snip>
Traceback (most recent call last):
File "manage.py", line 11, in <module>
execute_manager(settings)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/usr/local/lib/python2.6/dist-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 218, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/base.py", line 347, in handle
return self.handle_noargs(**options)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/commands/syncdb.py", line 103, in handle_noargs
emit_post_sync_signal(created_models, verbosity, interactive, db)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/sql.py", line 185, in emit_post_sync_signal
interactive=interactive, db=db)
File "/usr/local/lib/python2.6/dist-packages/django/dispatch/dispatcher.py", line 162, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/local/lib/python2.6/dist-packages/django/contrib/auth/management/__init__.py", line 28, in create_permissions
defaults={'name': name, 'content_type': ctype})
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 135, in get_or_create
return self.get_query_set().get_or_create(**kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 373, in get_or_create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 435, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py", line 528, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py", line 1479, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 783, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py", line 727, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/util.py", line 15, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/mysql/base.py", line 86, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 176, in execute
if not self._defer_warnings: self._warning_check()
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 92, in _warning_check
warn(w[-1], self.Warning, 3)
_mysql_exceptions.Warning: Data truncated for column 'name' at row 1
I have checked (and double checked) my table schema. All name field are CharField type with maximum length = 64. The backend db I am using is MySQL, so I am sure that indexes can be created for strings of length 64.
What could be causing this error (I suspect it is a misleading error message - even though its coming from the db itself)...
The traceback is happening during the creation of a django.contrib.auth.Permission: some of these get created for your models automatically as part of syncdb.
Permission.name has max_length=50, so you probably have an application and/or model class with a really long name?
Try the following query in manage.py dbshell:
SELECT * FROM auth_permission WHERE LENGTH(name) = 50;
If you cannot change your model name, then you can fix this problem by reducing the length of the generated Permission.name by specifying verbose_name in the model Meta class (see here for more details):
class MyVeryLongModelNameThatIsBreakingMyMigration(models.Model):
class Meta:
verbose_name = 'my long model'
Update
There's an open (as of 2013) Django ticket to fix this:
#8162 (Increase Permission.name max_length)
As Piet Delport noted, the problem is that your model name is too long.
You're certainly going to have to shorten your model name, and then clean up your database. How you do that depends upon where you are in the development process.
If this is a brand new application, with a dedicated database, and no actual data, the simple answer is to drop and recreate the database, and then re-run python manage.py syncdb.
If you have other tables in the database that need to be left alone, but the tables for your Django have no 'real' data, and can thus be dropped without damage, then you can use manage.py sqlclear to generate SQL DDL to drop all of the Django-generated tables, constraints, and indexes.
Do the following:
apps="auth contenttypes sessions sites messages admin <myapp1> <myapp2>"
python manage.py sqlclear ${apps} > clear.sql
You can feed the generated script to mysql or python manage.py dbshell. Once that's done, you can re-run python manage.py syncdb.
If you have actual data in your database tables that can't be dropped or deleted: Slap yourself and repeat 100 times "I will never do development against a production database again. I will always back up my databases before changing them." Now you're going to have to hand-code SQL to change the affected table name, as well as anything else that references it and any references in the auth_permissions table and any other Django system tables. Your actual steps will depend entirely upon the state of your database and tables.
I also got error like this one using postgresql django 1.2, but the problem was not the length, but using ugettext_lazy for translating the names. ('can_purge', _("Can purge")) is evidently unacceptable, since the name is stored in the database as text