I have a complicated query in a Django model and I want to do two things:
Get all objects that satisify the query
Check if one object satisfies the query
To do (1), I have a Q object encoding the query, and I just do
Model.objects.filter(THE_QUERY)
The query is something like
THE_QUERY = Q(field_1__isnull=False) & Q(field_2__gte=2) & Q(field3=0)
But I don't know how to reuse the query in THE_QUERY for (2). I want to have the predicate of the query in just one place and use that information to do (1) and (2), so that, if I ever have to change the query, both actions would do as expected.
Is there a way to put the query in just one place?
Model.objects.filter(THE_QUERY) returns an unevaluated queryset. You can extend this with extra conditions - in this case, you can add a filter to a specific ID and then an exists() call.
Model.objects.filter(THE_QUERY).filter(pk=my_object_id).exists()
Related
Let's say I need to do some work both on a set of model objects, as well as a subset of the first set:
things = Thing.objects.filter(active=True)
for thing in things: # (1)
pass # do something with each `thing`
special_things = things.filter(special=True)
for thing in special_things: # (2)
pass # do something more with these things
My understanding is that at point (1) marked in the code above, an actual SQL query something like SELECT * FROM things_table WHERE active=1 will get executed against the database. The QuerySet documentation also says:
When a QuerySet is evaluated, it typically caches its results.
Now my question is, what happens at point (2) in the example Python code above?
Will Django execute a second SQL query, something like SELECT * FROM things_table WHERE active=1 AND special=1?
Or, will it use the cached result from earlier, automatically doing for me behind the scenes something like the more optimal filter(lambda d: d.special == True, things), i.e. avoiding a needless second trip to the database?
Either way, is the current behavior guaranteed (by documentation or something) or should I not rely on it? For example, it is not only a point of optimization, but could also make a possible logic difference if the database table is modified by another thread/process between the two potential queries.
It will execute a second SQL query. filter creates a new queryset, which doesn't copy the results cache.
As for guarantees - well, the docs specify that filter returns a new queryset object. I think you can be confident that that new queryset won't have cached results yet. As further support, the "when are querysets evaluated" docs suggest using .all() to get a new queryset if you want to pick up possibly changed results:
If the data in the database might have changed since a QuerySet was
evaluated, you can get updated results for the same query by calling
all() on a previously evaluated QuerySet.
I want to do something like
query.annotate(Count('foreign_model_relation', somefield_from_foreign_model=some_value))
That means, i want to count, how many objects from another queryset are pointing to this object. The difference between using something like filter(in=other_queryset) is, that i would like to combine this in one query, to avoid generating one query per object.
Simplified Models:
Group
Object
group (Group)
Vote
object (Object)
up (Boolean)
Now i want to query the up/down count of all Objects for one Group, with one or two queries, not with one/two queries per Object.
You can do this with two queries:
YourObject.objects.filter(vote__up=True, group=some_group).annotate(total_votes_up=Count('vote'))
YourObject.objects.filter(vote__up=False, group=some_group).annotate(total_votes_down=Count('vote'))
But I think that should exist some more elegante way to do this.
I've cached a common queryset, which I would like to filter based off of different fields depending on the situation. I'm wondering if by filtering an evaluated queryset if I lose the advantage of caching it in the first place; does Django just create another queryset from scratch that's an aggregate of the querysets involved in creating the cached queryset and the filter that I apply afterwards?
Yes, the results get thrown out.
You can see this from the source: filter() calls _filter_or_exclude(), which calls _clone() and then adds to its query. _clone, you can see, doesn't set the _result_cache attribute.
In general, it's not really clear what it could possibly do to keep the common results. If it's a complicated query with a small result set, it could be replaced by just issuing SQL that checks that the primary key is one of the results you've found, but that's not always going to be more efficient, and in some situations it would confusingly mess with the semantics (if the DB changes in a way that affects the query results in the time between when it's cached and when you do the additional filter).
If you want to force this behavior of saving the IDs manually, you can do that:
pks = SomeObject.objects.filter(...).values_list('pk', flat=True)
some_of_them = SomeObject.objects.filter(pk_in=pks).filter(...)
others = SomeObject.objects.filter(pk_in=pks).filter(...)
You can also of course just do the filtering in Python, e.g. by
common = SomeObject.objects.filter(...)
some_of_them = [m for m in common if m.attribute == 'foo']
others = [m for m in common if m.other_attribute == 'bar']
(You could also use filter(lambda m: m.attribute == 'foo', common) if you preferred, or wrap the definition of common in list to be more explicit.)
Whether one of these or reissuing the query depends a lot on the size of the sets involved, the complexity of the filters, and what indices are present.
Consider a model and a query using annotations, for example the following example from the Django documentation:
http://docs.djangoproject.com/en/dev/topics/db/aggregation/
Publisher.objects.filter(book__rating__gt=3.0).annotate(num_books=Count('book'))
The result of this query will only contain objects matching the filter (i.e. has a book_rating greater than 3.0), and these objects has been annotated. But what if I want the query to contain all objects, but only annotate objects which matches a filter (or for example annotate them with 0)? Or is this even possible?
No, you can't do that - because that's not how the underlying SQL works.
The only thing I can think of is to do two queries, one with the filter/annotation and one without, then iterate through them in Python, appending the annotation to the matching objects from the non-filtered list.
Is there a difference between filter and exclude in django? If I have
self.get_query_set().filter(modelField=x)
and I want to add another criteria, is there a meaningful difference between to following two lines of code?
self.get_query_set().filter(user__isnull=False, modelField=x)
self.get_query_set().filter(modelField=x).exclude(user__isnull=True)
is one considered better practice or are they the same in both function and performance?
Both are lazily evaluated, so I would expect them to perform equivalently. The SQL is likely different, but with no real distinction.
It depends what you want to achieve. With boolean values it is easy to switch between .exclude() and .filter() but what about e.g. if you want to get all articles except those from March? You can write the query as
Posts.objects.exclude(date__month=3)
With .filter() it would be (but I not sure whether this actually works):
Posts.objects.filter(date__month__in=[1,2,4,5,6,7,8,9,10,11,12])
or you would have to use a Q object.
As the function name already suggest, .exclude() is used to exclude datasets from the resultset. For boolean values you can easily invert this and use .filter() instead, but for other values this can be more tricky.
In general exclude is opposite of filter. In this case both examples works the same.
Here:
self.get_query_set().filter(user__isnull=False, modelField=x)
You select entries that field user is not null and modelField has value x
In this case:
self.get_query_set().filter(modelField=x).exclude(user__isnull=True)
First you select entries that modelField has value x(both user in null and user is not null), then you exclude entries that have field user null.
I think that in this case it would be better use first option, it looks more cleaner. But both work the same.