Django: Why don't foreign key lookups automatically use the pk? - django

I have
class Achievement(MyBaseModel):
pass
class Alias(MyBaseModel):
achievements = models.ManyToManyField('Achievement')
>>> ach = Achievement.objects.all()[1]
This works :
>>> Alias.objects.all().filter(achievements__pk__contains=ach.pk).count()
77L
But this doesn't :
>>> Alias.objects.all().filter(achievements__contains=ach).count()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/var/home/ptarjan/django/mysite/django/db/models/query.py", line 489, in filter
return self._filter_or_exclude(False, *args, **kwargs)
File "/var/home/ptarjan/django/mysite/django/db/models/query.py", line 507, in _filter_or_exclude
clone.query.add_q(Q(*args, **kwargs))
File "/var/home/ptarjan/django/mysite/django/db/models/sql/query.py", line 1258, in add_q
can_reuse=used_aliases)
File "/var/home/ptarjan/django/mysite/django/db/models/sql/query.py", line 1201, in add_filter
self.where.add((alias, col, field, lookup_type, value), connector)
File "/var/home/ptarjan/django/mysite/django/db/models/sql/where.py", line 48, in add
params = field.get_db_prep_lookup(lookup_type, value)
File "/var/home/ptarjan/django/mysite/django/db/models/fields/related.py", line 156, in get_db_prep_lookup
raise TypeError, "Related Field has invalid lookup: %s" % lookup_type
TypeError: Related Field has invalid lookup: contains
Why? (Django 1.0.2)
Looking at the query log, it is doing something that I didn't expect! That query yielded :
>>> connection.queries[-1]
{'time': '0.027', 'sql': u'SELECT COUNT(*) FROM `yourock_alias` INNER JOIN `yourock_achiever` ON (`yourock_alias`.`id` = `yourock_achiever`.`alias_id`) WHERE `yourock_achiever`.`achievement_id` LIKE BINARY %j0xvw9% '}
But doing this
>>> Alias.objects.all().filter(achievements=ach).count()
77L
Gives this query
>>> connection.queries[-1]
{'time': '0.023', 'sql': u'SELECT COUNT(*) FROM `yourock_alias` INNER JOIN `yourock_achiever` ON (`yourock_alias`.`id` = `yourock_achiever`.`alias_id`) WHERE `yourock_achiever`.`achievement_id` = j0xvw9 '}
which is what I wanted, but the = seems to me to mean that it IS the one object. The query that django is doing actually is returning if the object is anywhere in the achievement list.
Is this correctly setup and is just very counter-intuitive or am I doing something wrong?

I can't answer why the design decision was made to implement it this way, but most likely its to follow the Python philosophy that things should be specified explicitly and not implied.
The reason it doesn't work is because __contains expects a field to reference. The bit you passed it was a reference to a whole object.

In the second case, you are comparing objects. And, from Django documentation, in order to compare an object you have to use the == operator.
On top of that: why don't you use ach.alias_set.objects.count(), as explained in the query section of the manual?

Related

Is there a workaround for Django's `ModelDoesNotExist` error in a situation like this?

So I get a ModelDoesNotExist error when I run manage.py runserver because in a file in my directory, I make a query to a table which has not been populated yet.
def __init__(self):
self.spotify_object = SocialToken.objects.get(account__provider="spotify")
The above is a class instantiated to perform some sort of authentication and the SocialToken table gets populated only after I login. Now, I was wondering if there was a way to escape the error by triggering this part of the code only after I login? I only use the class in an endpoint, and during that period, the table would have been populated but the fact that it is not populated before running the server is causing a DoesNotExist error. Is there a solution to this?
Traceback
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\Sparison\views.py", line 4, in <module>
from .authentication import SparisonCacheHandler
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\Sparison\authentication.py", line 43, in
<module>
cache_handler = SparisonCacheHandler() ,
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\Sparison\authentication.py", line 25,
in __init__
self.spotify_object = SocialToken.objects.get(account__provider="spotify")
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\venv\lib\site-
packages\django\db\models\manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "C:\Users\Kwaku Biney\Desktop\sparison-1\project\venv\lib\site-
packages\django\db\models\query.py", line 429, in get
raise self.model.DoesNotExist(
allauth.socialaccount.models.DoesNotExist: SocialToken matching query does not exist.
In my views.py, I import the class which has the query and the error comes up.
There are two ways to avoid the DoesNotExist Error.
A) Use .filter() instead of .get()
Filtering leads to an empty queryset when the search comes up empty.
def __init__(self):
self.spotify_object = SocialToken.objects.filter(account__provider="spotify")
B) Use get_object_or_404() instead of .get()
This is a built-in function by django:
Calls get() on a given model manager, but it raises Http404 instead of the model’s DoesNotExist exception.
Django Documentation
def __init__(self):
self.spotify_object = SocialToken.objects.get_object_or_404(account__provider="spotify")
Hope I could help you. Have a nice day.

Django: Annotating age from date_of_birth

I have a date of birth (dob) DateField in my Django Model (Author). I tried to annotate age parameter. I searched for many possible ways to do it and each procedure generated some kind of error.
Here I tried in python console first to make sure that the expression would be a valid one:
>>> from datetime import datetime
>>> (datetime.now() - datetime(2000,1,1)).days #output: 7506
First Try:
>>> from django.db.models import F
>>> authors = Author.objects.annotate(age = (datetime.now()-F('dob')).days) #no-error here
>>> print(authors) # Exception thrown here
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 324, in __getitem__
qs._fetch_all()
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 1305, in _fetch_all
self._result_cache = list(self._iterable_class(self))
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/query.py", line 70, in __iter__
for row in compiler.results_iter(results):
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/models/sql/compiler.py", line 1100, in apply_converters
value = converter(value, expression, connection)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/db/backends/sqlite3/operations.py", line 291, in convert_datefield_value
value = parse_date(value)
File "/home/puru/Documents/Python/django/prac1/django-master/lib/python3.7/site-packages/Django-3.2-py3.7.egg/django/utils/dateparse.py", line 75, in parse_date
match = date_re.match(value)
TypeError: expected string or bytes-like object
Second time I used ExpressionWrapper to define that output type will be DateTimeField
>>> from django.db.models import DateTimeField, ExpressionWrapper
>>> authors = Author.objects.annotate(age = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField()).days)
AttributeError: 'ExpressionWrapper' object has no attribute 'days'
I have also tried the RawSQL. I am using Sqlite database here. so date('now') would provide me current date.
>>> from django.db.models.expressions import RawSQL
>>> authors = Author.objects.annotate(age=RawSQL("date('now')-dob"))
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: __init__() missing 1 required positional argument: 'params'
So is there any way, I could solve the issue?
You were almost there, the issue was the output_field value. It should be DurationField instead of DateTimeField
age_expr = ExpressionWrapper(timezone.now() - F('dob'), output_field=DateTimeField())
queryset = Author.objects.annotate(age=age_expr)
NOTE: You can't use .days along with ExpressionWrapper since the annotate operation performs in the DB level.
To calculate the age, you may need to use the .total_seconds() method
print(queryset[0].age.total_seconds() / 60 / 60 / 24 / 365.25)
Empty string or list params worked for the RawSQL command.
authors = Author.objects.annotate(age=RawSQL("date('now')-dob", params=""))

Django ORM raw delete query not deleting records

I am using raw_sql queries for my convenience for keeping my database minimal I am deleting extra records. By this query
#d is from a loop and has values
res=MyModel.objects.raw("DELETE FROM mydb_mymodel WHERE mydb_mymodel.s_type = '%s' and mydb_mymodel.barcode = '%s' and mydb_mymodel.shopcode = '%s' and mydb_mymodel.date = '%s'" ,[d.s_type,d.barcode,d.shopcode,d.date])
It is not deleting records in database but
when I do res.query and run it from postgres console it works!
Yes I can use
MyModel.objects.filter(s_type=d.s_type,barcode=d.barcode,
shopcode=d.shopcode,date=d.date).delete()
but what I am missing in raw_sql?
A .raw(..) is not executed eagerly, it is, just like most Django ORM queries performed lazily. It thus returns a RawQuerySet object with the query in the object. For example:
>>> User.objects.raw('BLA BLA BLA', [])
<RawQuerySet: BLA BLA BLA>
A query like BLA BLA BLA does not make any sense: a database will error on it, but still we retrieve a RawQuerySet.
You can force evaluation by for example iterating over it, and then we get:
>>> list(User.objects.raw('BLA BLA BLA', []))
Traceback (most recent call last):
File "/djangotest/env/lib/python3.6/site-packages/django/db/backends/utils.py", line 85, in _execute
return self.cursor.execute(sql, params)
File "/djangotest/env/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 71, in execute
return self.cursor.execute(query, args)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 250, in execute
self.errorhandler(self, exc, value)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/connections.py", line 50, in defaulterrorhandler
raise errorvalue
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 247, in execute
res = self._query(query)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 412, in _query
rowcount = self._do_query(q)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 375, in _do_query
db.query(q)
File "/djangotest/env/lib/python3.6/site-packages/MySQLdb/connections.py", line 276, in query
_mysql.connection.query(self, query)
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'BLA BLA BLA' at line 1")
So the list(..) forces evaluation, and now the database of course produces an error. However even if it was a valid DELETE query, it would still raise an error, since such query does not return any record.
In order to make DELETE calls, the Django manual specifies that you should use a cursor [Django-doc]:
from django.db import connection
with connection.cursor() as cursor:
cursor.execute(
"DELETE FROM mydb_mymodel WHERE s_type = '%s' AND barcode = '%s' AND shopcode = '%s' AND date = '%s'" ,
[d.s_type,d.barcode,d.shopcode,d.date]
)
But I think it is probably a lot simpler to specify it like:
MyModel.objects.filter(
s_type=d.s_type,
barcode=d.barcode,
shopcode=d.shopcode,
date=d.date
).delete()
This will construct a DELETE query, and serialize the parameters properly. A .delete() query is done eagerly, so the odds of making above discussed mistakes is a lot lower: if the ORM is implemented correctly, then we do not need to worry about that.
try this:
from django.db import connection
def executeQuery(self, sql, params=[]):
with connection.cursor() as cursor:
cursor.execute(
sql,
params
)
use it like:
executeQuery("DELETE FROM mydb_mymodel WHERE mydb_mymodel.s_type = '%s' and mydb_mymodel.barcode = '%s' and mydb_mymodel.shopcode = '%s' and mydb_mymodel.date = '%s'" ,[d.s_type,d.barcode,d.shopcode,d.date])

Why am I getting TypeError: 'vc' is an invalid keyword argument for this function error?

I don't understand what I am doing wrong here. Here is my Django model:
class VMMigrationEvent(models.Model):
created = models.DateTimeField(auto_now_add=True) # DateTime because there may be multiple events in a day.
dc = models.TextField(max_length=150), # data center
destination_host = models.TextField(max_length=150), # host migrated to.
message = models.TextField(), # The message from virtual center.
updated = models.DateTimeField(auto_now=True) # DateTime because there may be multifple events in a day.
user = models.TextField(max_length=150), # The user logged into the virtual center that execute the migration.
vc = models.TextField(max_length=150), # virtual center
vm = models.ForeignKey(VirtualMachine) # The VirtualMachine record associated with this event.
And from the python console I do this:
>>> from cm.models import *
>>> dc='DCM01N-01'
>>> destination_host='auroravm2-1.example.com'
>>> message='my hovercraft is full of eels.'
>>> user='mister_gumby'
>>> vc='vca-001-s.example.com'
>>> vm='ads-108'
>>> vm_db_obj = VirtualMachine.objects.filter(name=vm).latest('create_date')
>>> vmme = VMMigrationEvent.objects.create(dc=dc,
... destination_host=destination_host,
... message=message, user=user, vc=vc, vm=vm_db_obj)
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/apps/man/man/env/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/apps/man/man/env/lib/python2.7/site-packages/django/db/models/query.py", line 392, in create
obj = self.model(**kwargs)
File "/apps/man/man/env/lib/python2.7/site-packages/django/db/models/base.py", line 573, in __init__
raise TypeError("'%s' is an invalid keyword argument for this function" % list(kwargs)[0])
TypeError: 'vc' is an invalid keyword argument for this function
Why is vc invalid? It is a field in my model?
You have commas after several of your field definitions: vc, but also dc, destination_host and user. This turns them into tuples, which can't be used as fields on the model.
Delete those commas
(Also, you probably meant CharField rather than TextField.)

Django Official Tutorial Part 1 index out of bound error

I started learning Django recently and am having a strange problem with the tutorial. Everything was going fine until I started playing with the interactive shell and then I got an error whenever I tried to call all the objects in one of the tables.
I am using Django 1.1, Python 2.5 on MacOs X.
For those unfamiliar with the tutorial you are making a website to manage Polls. You have the following code in the model:
from django.db import models
import datetime
class Poll(models.Model):
question = models.CharField(max_length=200)
pub_date = models.DateTimeField('date published')
def __unicode__(self):
return self.question
def was_published_today(self):
return self.pub_date.date() == datetime.date.today()
was_published_today.short_description = 'Published today?'
class Choice(models.Model):
poll = models.ForeignKey(Poll)
choice = models.CharField(max_length=200)
votes = models.IntegerField()
def __unicode__(self):
return self.choice
After creating the model you add a poll item and then add some choices to it.
Everything was fine until I tried to see all the objects in the choices table or tried to see all the choices in a particular poll. Then I got an error. Heres an example series of commands in the interactive shell. Please note that the count of the choices is correct (I have experimented a bit after running into the error so the count is a bit high.)
>>> from mysite.polls.models import Poll, Choice
>>> Poll.objects.all()
[<Poll: What's up>, <Poll: Yups>]
>>> Choice.objects.count()
10
>>> Choice.objects.all()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Library/Python/2.5/site-packages/django/db/models/query.py", line 68, in __repr__
data = list(self[:REPR_OUTPUT_SIZE + 1])
File "/Library/Python/2.5/site-packages/django/db/models/query.py", line 83, in __len__
self._result_cache.extend(list(self._iter))
File "/Library/Python/2.5/site-packages/django/db/models/query.py", line 238, in iterator
for row in self.query.results_iter():
File "/Library/Python/2.5/site-packages/django/db/models/sql/query.py", line 287, in results_iter
for rows in self.execute_sql(MULTI):
File "/Library/Python/2.5/site-packages/django/db/models/sql/query.py", line 2369, in execute_sql
cursor.execute(sql, params)
File "/Library/Python/2.5/site-packages/django/db/backends/util.py", line 19, in execute
return self.cursor.execute(sql, params)
File "/Library/Python/2.5/site-packages/django/db/backends/sqlite3/base.py", line 193, in execute
return Database.Cursor.execute(self, query, params)
File "/Library/Python/2.5/site-packages/django/db/backends/util.py", line 82, in typecast_timestamp
seconds = times[2]
IndexError: list index out of range
The Django tutorial(part 1) can be found here
Thanks!
The problem seemed to be that the database was not synchronized with the models. Resetting the database worked fine. Thanks to Alasdair for the suggestion.
It looks like the problem is that was_published_today() is comparing a datetime to a date. Try changing it to:
return self.pub_date.date() == datetime.date.today()
Since the problem seems to be in the code that interprets strings as timestamps, I'd be interested to see the actual data in the db. It looks like there's a timestamp in there that isn't in the proper form. Not sure how it got there without seeing it, but I bet there's a clue there.