I have these models:
def Foo(Models.model):
size = models.IntegerField()
# other fields
def is_active(self):
if check_condition:
return True
else:
return False
def Bar(Models.model):
foo = models.ForeignKey("Foo")
# other fields
Now I want to query Bars that are having active Foo's as such:
Bar.objects.filter(foo.is_active())
I am getting error such as
SyntaxError at /
('non-keyword arg after keyword arg'
How can I achieve this?
You cannot query against model methods or properties. Either use the criteria within it in the query, or filter in Python using a list comprehension or genex.
You could also use a custom manager. Then you could run something like this:
Bar.objects.foo_active()
And all you have to do is:
class BarManager(models.Manager):
def foo_active(self):
# use your method to filter results
return you_custom_queryset
Check out the docs.
I had similar problem: I am using class-based view object_list and I had to filter by model's method. (storing the information in database wasn't an option because the property was based on time and I would have to create a cronjob and/or... no way)
My answer is ineffective and I don't know how it's gonna scale on larger data; but, it works:
q = Model.objects.filter(...)...
# here is the trick
q_ids = [o.id for o in q if o.method()]
q = q.filter(id__in=q_ids)
You can't filter on methods, however if the is_active method on Foo checks an attribute on Foo, you can use the double-underscore syntax like Bar.objects.filter(foo__is_active_attribute=True)
Related
While creating a front end for a Django module I faced the following problem inside Django core:
In order to display a link to the next/previous object from a model query, we can use the extra-instance-methods of a model instance: get_next_by_FIELD() or get_previous_by_FIELD(). Where FIELD is a model field of type DateField or DateTimeField.
Lets explain it with an example
from django.db import models
class Shoe(models.Model):
created = models.DateTimeField(auto_now_add=True, null=False)
size = models.IntegerField()
A view to display a list of shoes, excluding those where size equals 4:
def list_shoes(request):
shoes = Shoe.objects.exclude(size=4)
return render_to_response(request, {
'shoes': shoes
})
And let the following be a view to display one shoe and the corresponding
link to the previous and next shoe.
def show_shoe(request, shoe_id):
shoe = Shoe.objects.get(pk=shoe_id)
prev_shoe = shoe.get_previous_by_created()
next_shoe = shoe.get_next_by_created()
return render_to_response('show_shoe.html', {
'shoe': shoe,
'prev_shoe': prev_shoe,
'next_shoe': next_shoe
})
Now I have the situation that the show_shoe view displays the link to the previous/next regardless of the shoes size. But I actually wanted just shoes whose size is not 4.
Therefore I tried to use the **kwargs argument of the get_(previous|next)_by_created() methods to filter out the unwanted shoes, as stated by the documentation:
Both of these methods will perform their queries using the default manager for the model. If you need to emulate filtering used by a custom manager, or want to perform one-off custom filtering, both methods also accept
optional keyword arguments, which should be in the format described in Field lookups.
Edit: Keep an eye on the word "should", because then also (size_ne=4) should work, but it doesn't.
The actual problem
Filtering using the lookup size__ne ...
def show_shoe(request, shoe_id):
...
prev_shoe = shoe.get_previous_by_created(size__ne=4)
next_shoe = shoe.get_next_by_created(size__ne=4)
...
... didn't work, it throws FieldError: Cannot resolve keyword 'size_ne' into field.
Then I tried to use a negated complex lookup using Q objects:
from django.db.models import Q
def show_shoe(request, shoe_id):
...
prev_shoe = shoe.get_previous_by_created(~Q(size=4))
next_shoe = shoe.get_next_by_created(~Q(size=4))
...
... didn't work either, throws TypeError: _get_next_or_previous_by_FIELD() got multiple values for argument 'field'
Because the get_(previous|next)_by_created methods only accept **kwargs.
The actual solution
Since these instance methods use the _get_next_or_previous_by_FIELD(self, field, is_next, **kwargs) I changed it to accept positional arguments using *args and passed them to the filter, like the **kwargs.
def my_get_next_or_previous_by_FIELD(self, field, is_next, *args, **kwargs):
"""
Workaround to call get_next_or_previous_by_FIELD by using complext lookup queries using
Djangos Q Class. The only difference between this version and original version is that
positional arguments are also passed to the filter function.
"""
if not self.pk:
raise ValueError("get_next/get_previous cannot be used on unsaved objects.")
op = 'gt' if is_next else 'lt'
order = '' if is_next else '-'
param = force_text(getattr(self, field.attname))
q = Q(**{'%s__%s' % (field.name, op): param})
q = q | Q(**{field.name: param, 'pk__%s' % op: self.pk})
qs = self.__class__._default_manager.using(self._state.db).filter(*args, **kwargs).filter(q).order_by('%s%s' % (order, field.name), '%spk' % order)
try:
return qs[0]
except IndexError:
raise self.DoesNotExist("%s matching query does not exist." % self.__class__._meta.object_name)
And calling it like:
...
prev_shoe = shoe.my_get_next_or_previous_by_FIELD(Shoe._meta.get_field('created'), False, ~Q(state=4))
next_shoe = shoe.my_get_next_or_previous_by_FIELD(Shoe._meta.get_field('created'), True, ~Q(state=4))
...
finally did it.
Now the question to you
Is there an easier way to handle this? Should shoe.get_previous_by_created(size__ne=4) work as expected or should I report this issue to the Django guys, in the hope they'll accept my _get_next_or_previous_by_FIELD() fix?
Environment: Django 1.7, haven't tested it on 1.9 yet, but the code for _get_next_or_previous_by_FIELD() stayed the same.
Edit: It is true that complex lookups using Q object is not part of "field lookups", it's more part of the filter() and exclude() functions instead. And I am probably wrong when I suppose that get_next_by_FIELD should accept Q objects too. But since the changes involved are minimal and the advantage to use Q object is high, I think these changes should get upstream.
tags: django, complex-lookup, query, get_next_by_FIELD, get_previous_by_FIELD
(listing tags here, because I don't have enough reputations.)
You can create custom lookup ne and use it:
.get_next_by_created(size__ne=4)
I suspect the method you've tried first only takes lookup arg for the field you're basing the get_next on. Meaning you won't be able to access the size field from the get_next_by_created() method, for example.
Edit : your method is by far more efficient, but to answer your question on the Django issue, I think everything is working the way it is supposed to. You could offer an additional method such as yours but the existing get_next_by_FIELD is working as described in the docs.
You've managed to work around this with a working method, which is OK I guess, but if you wanted to reduce the overhead, you could try a simple loop :
def get_next_by_field_filtered(obj, field=None, **kwargs):
next_obj = getattr(obj, 'get_next_by_{}'.format(field))()
for key in kwargs:
if not getattr(next_obj, str(key)) == kwargs[str(key)]:
return get_next_by_field_filtered(next_obj, field=field, **kwargs)
return next_obj
This isn't very efficient but it's one way to do what you want.
Hope this helps !
Regards,
Of course, I don't mean to do what prefetch_related does already.
I'd like to mimic what it does.
What I'd like to do is the following.
I have a list of MyModel instances.
A user can either follows or doesn't follow each instance.
my_models = MyModel.objects.filter(**kwargs)
for my_model in my_models:
my_model.is_following = Follow.objects.filter(user=user, target_id=my_model.id, target_content_type=MY_MODEL_CTYPE)
Here I have n+1 query problem, and I think I can borrow what prefetch_related does here. Description of prefetch_related says, it performs the query for all objects and when the related attribute is required, it gets from the pre-performed queryset.
That's exactly what I'm after, perform query for is_following for all objects that I'm interested in. and use the query instead of N individual query.
One additional aspect is that, I'd like to attach queryset rather than attach the actual value, so that I can defer evaluation until pagination.
If that's too ambiguous statement, I'd like to give the my_models queryset that has is_following information attached, to another function (DRF serializer for instance).
How does prefetch_related accomplish something like above?
A solution where you can get only the is_following bit is possible with a subquery via .extra.
class MyModelQuerySet(models.QuerySet):
def annotate_is_follwing(self, user):
return self.extra(
select = {'is_following': 'EXISTS( \
SELECT `id` FROM `follow` \
WHERE `follow`.`target_id` = `mymodel`.id \
AND `follow`.`user_id` = %s)' % user.id
}
)
class MyModel(models.Model):
objects = MyModelQuerySet.as_manager()
usage:
my_models = MyModel.objects.filter(**kwargs).annotate_is_follwing(request.user)
Now another solution where you can get a whole list of following objects.
Because you have a GFK in the Follow class you need to manually create a reverse relation via GenericRelation. Something like:
class MyModelQuerySet(models.QuerySet):
def with_user_following(self, user):
return self.prefetch_related(
Prefetch(
'following',
queryset=Follow.objects.filter(user=user) \
.select_related('user'),
to_attr='following_user'
)
)
class MyModel(models.Model):
following = GenericRelation(Follow,
content_type_field='target_content_type',
object_id_field='target_id'
related_query_name='mymodels'
)
objects = MyModelQuerySet.as_manager()
def get_first_following_object(self):
if hasattr(self, 'following_user') and len(self.following_user) > 0:
return self.following_user[0]
return None
usage:
my_models = MyModel.objects.filter(**kwargs).with_user_following(request.user)
Now you have access to following_user attribute - a list with all follow objects per mymodel, or you can use a method like get_first_following_object.
Not sure if this is the best approach, and I doubt this is what prefetch_related does because I'm joining here.
I found there's way to select extra columns in your query.
extra_select = """
EXISTS(SELECT * FROM follow_follow
WHERE follow_follow.target_object_id = myapp_mymodel.id AND
follow_follow.target_content_type_id = %s AND
follow_follow.user_id = %s)
"""
qs = self.extra(
select={'is_following': extra_select},
select_params=[CONTENT_TYPE_ID, user.id]
)
So you can do this with join.
prefetch_related way of doing it would be separate queryset and look it up in queryset for the attribute.
I'm trying to order by a count of a manyToMany field is there a way to do this with TastyPie?
For example
class Person(models.Model):
friends = models.ManyToMany(User, ..)
I want PersonResource to spit out json that is ordered by the number of friends a person has...
is that possible?
I know this is an old question, but I recently encountered this problem and came up with a solution.
Tastypie doesn't easily allow custom ordering, but it is easy to modify the queryset it uses.
I actually just modified the default queryset for the model using a custom manager.
for instance:
class PersonManager(models.Manager):
def get_query_set(self):
return super(PersonManager self).get_query_set().\
annotate(friend_count=models.Count('friends'))
class Person(models.Model):
objects = PersonManager()
friends = ...
You could also add the annotation in Tastypie, wither in the queryset=... in the Meta class, or overriding the get_object_list(self,request) method.
I wasn't able to get the results ordering as per coaxmetal's solution, so I solved this a different way, by overriding the get_object_list on the Resource object as per http://django-tastypie.readthedocs.org/en/latest/cookbook.html. Basically if the 'top' querystring parameter exists, then the ordered result is returned.
class MyResource(ModelResource):
class Meta:
queryset = MyObject.objects.all()
def get_object_list(self, request):
try:
most_popular = request.GET['top']
result = super(MyResource, self).get_object_list(request).annotate(num_something=Count('something')).order_by('num_something')
except:
result = super(MyResource, self).get_object_list(request)
return result
I have not used TastyPie, but your problem seems to be more general. You can't have custom ordering in a Django ORM query. You're better off storing tuples of the form (Person, friend_count). This is pretty easy:
p_list = []
for person in Person.objects.all():
friendcount = len(person.friends.all())
p_list.append((person, friendcount))
Then, you can use the built in sorted function like so:
sorted_list = [person for (person, fc) in sorted(p_list, key=lambda x: x[1])]
The last line basically extracts the Persons from a sorted list of Persons, sorted on the no of friends one has.
`
I created my own manager:
class DataManager(models.Manager):
def optfilter(self, options = dict()):
kwargs = dict()
if options.has_key('active'):
kwargs['active__id'] = options['active']
return self.filter(**kwargs)
active is ManyToMany field.
and it working exacly as I want. But what about when I want exclude object from filter?
Something like this:
kwargs['exclude_active_id'] = options['active']
Same idea as you're already implemented except use exclude instead of filter
I have a private boolean flag on my model, and a custom manager that overwrites the get_query_set method, with a filter, removing private=True:
class myManager(models.Manager):
def get_query_set(self):
qs = super(myManager, self).get_query_set()
qs = qs.filter(private=False)
return qs
class myModel(models.Model):
private = models.BooleanField(default=False)
owner = models.ForeignKey('Profile', related_name="owned")
#...etc...
objects = myManager()
I want the default queryset to exclude the private models be default as a security measure, preventing accidental usage of the model showing private models.
Sometimes, however, I will want to show the private models, so I have the following on the manager:
def for_user(self, user):
if user and not user.is_authenticated():
return self.get_query_set()
qs = super(myManager, self).get_query_set()
qs = qs.filter(Q(owner=user, private=True) | Q(private=False))
return qs
This works excellently, with the limitation that I can't chain the filter. This becomes a problem when I have a fk pointing the myModel and use otherModel.mymodel_set. otherModel.mymodel_set.for_user(user) wont work because mymodel_set returns a QuerySet object, rather than the manager.
Now the real problem starts, as I can't see a way to make the for_user() method work on a QuerySet subclass, because I can't access the full, unfiltered queryset (basically overwriting the get_query_set) form the QuerySet subclass, like I can in the manager (using super() to get the base queryset.)
What is the best way to work around this?
I'm not tied to any particular interface, but I would like it to be as djangoy/DRY as it can be. Obviously I could drop the security and just call a method to filter out private tasks on each call, but I really don't want to have to do that.
Update
manji's answer below is very close, however it fails when the queryset I want isn't a subset of the default queryset. I guess the real question here is how can I remove a particular filter from a chained query?
Define a custom QuerySet (containing your custom filter methods):
class MyQuerySet(models.query.QuerySet):
def public(self):
return self.filter(private=False)
def for_user(self, user):
if user and not user.is_authenticated():
return self.public()
return self.filter(Q(owner=user, private=True) | Q(private=False))
Define a custom manager that will use MyQuerySet (MyQuerySet custom filters will be accessible as if they were defined in the manager[by overriding __getattr__]):
# A Custom Manager accepting custom QuerySet
class MyManager(models.Manager):
use_for_related_fields = True
def __init__(self, qs_class=models.query.QuerySet):
self.queryset_class = qs_class
super(QuerySetManager, self).__init__()
def get_query_set(self):
return self.queryset_class(self.model).public()
def __getattr__(self, attr, *args):
try:
return getattr(self.__class__, attr, *args)
except AttributeError:
return getattr(self.get_query_set(), attr, *args)
Then in the model:
class MyModel(models.Model):
private = models.BooleanField(default=False)
owner = models.ForeignKey('Profile', related_name="owned")
#...etc...
objects = myManager(MyQuerySet)
Now you can:
¤ access by default only public models:
MyModel.objects.filter(..
¤ access for_user models:
MyModel.objects.for_user(user1).filter(..
Because of (use_for_related_fields = True), this same manager wil be used for related managers. So you can also:
¤ access by default only public models from related managers:
otherModel.mymodel_set.filter(..
¤ access for_user from related managers:
otherModel.mymodel_set.for_user(user).filter(..
More informations: Subclassing Django QuerySets & Custom managers with chainable filters (django snippet)
To use the chain you should override the get_query_set in your manager and place the for_user in your custom QuerySet.
I don't like this solution, but it works.
class CustomQuerySet(models.query.QuerySet):
def for_user(self):
return super(CustomQuerySet, self).filter(*args, **kwargs).filter(private=False)
class CustomManager(models.Manager):
def get_query_set(self):
return CustomQuerySet(self.model, using=self._db)
If you need to "reset" the QuerySet you can access the model of the queryset and call the original manager again (to fully reset). However that's probably not very useful for you, unless you were keeping track of the previous filter/exclude etc statements and can replay them again on the reset queryset. With a bit of planning that actually wouldn't be too hard to do, but may be a bit brute force.
Overall manji's answer is definitely the right way to go.
So amending manji's answer you need to replace the existing "model"."private" = False with ("model"."owner_id" = 2 AND "model"."private" = True ) OR "model"."private" = False ). To do that you will need to walk through the where object on the query object of the queryset to find the relevant bit to remove. The query object has a WhereNode object that represents the tree of the where clause, with each node having multiple children. You'd have to call the as_sql on the node to figure out if it's the one you are after:
from django.db import connection
qn = connection.ops.quote_name
q = myModel.objects.all()
print q.query.where.children[0].as_sql(qn, connection)
Which should give you something like:
('"model"."private" = ?', [False])
However trying to do that is probably way more effort than it's worth and it's delving into bits of Django that are probably not API-stable.
My recommendation would be to use two managers. One that can access everything (an escape hatch of sort), the other with the default filtering applied. The default manager is the first one, so you need to play around with the ordering depending on what you need to do. Then restructure your code to know which one to use - so you don't have the problem of having the extra private=False clause in there already.