How can join two django querysets in one? - django

I need order a Queryset by date in desc order, but i need put in the end the objects at the end, I do this:
qs1 = Model.objects.exclude(date=None).order_by('-date')
qs2 = Model.objects.filter(date=None).order_by('-date')
and my list is:
l = list(qs1)+list(qs2)
There is a more efficiently way for this?

Model.objects.extra(select={'nodate':'ISNULL(date)'}, order_by=['nodate', '-date'])

So, you're looking for all the objects that have a date ... and all the objects that don't have a date?
Wouldn't this work better:
Model.objects.order_by('-date)
Anyway, here's a good place to start ... read about Django filter chaining: http://docs.djangoproject.com/en/dev/topics/db/queries/#id1.
As a side note, isn't your query canceling itself out?
>>> d = MyModel.objects.filter(date=None).exclude(date=None).order_by('-date')
>>> d
[]
>>> d.query.get_compiler('default').as_sql()
('SELECT "date" FROM "table" WHERE ("table"."date" IS NULL AND NOT ("table"."date" IS NULL)) ORDER BY "table"."date" DESC', ())
I'm probably not understanding what you're asking...

Related

Django queryset union appears not to be working when combined with .annotate()

I have the following queryset:
photos = Photo.objects.all()
I filter out two queries:
a = photos.filter(gallery__name='NGL')
b = photos.filter(gallery__name='NGA')
I add them together, and they form one new, bigger queryset:
c = a | b
Indeed, the length of a + b equals c:
a.count() + b.count() == c.count()
>>> True
So far so good. Yet, if I introduce a .annotate(), the | no longer seems to work:
a = photos.annotate(c=Count('label').exclude(c__lte=4)
b = photos.filter(painting=True)
c = a | b
a.count() + b.count() == c.count()
>>> False
How do I combine querysets, even when .annotate() is being used? Note that query one and two both work as intended in isolation, only when combining them using | does it seem to go wrong.
the pipe | or ampersand & to combine querysets actually puts OR or AND to SQL query so it looks like combined.
one = Photo.objects.filter(id=1)
two = Photo.objects.filter(id=2)
combined = one | two
print(combined.query)
>>> ... WHERE ("photo_photo"."id" = 1 OR "photo_photo"."id" = 2)...
But when you combine more filters and excludes you may notice it will give you strange results due to this. So that is why it doesn't match when you compare counts.
If you use .union() you have to have same columns with same data type, so you have to annotate both querysets. Info about .union()
SELECT statement within .UNION() must have the same number of columns
The columns must also have similar data types
The columns in each SELECT statement must also be in the same order
You have to keep in mind, that pythons argument kwargs for indefinite number of arguments are dictionary, so if you want to use annotate with multiple annotations, you can't ensure correct order of columns. Fortunatelly you can solve this with chaining annotate commands.
# printed query of this won't be consistent
photo_queryset.annotate(label_count=Count('labels'), tag_count=Count('tags'))
# this will always have same order of columns
photo_queryset.annotate(label_count=Count('labels')).annotate(tag_count=Count('tags'))
Then you can use .union() and it won't mess up results of annotation. Also .union() should be last method, because after .union() you can't use filter like methods. If you want to preserve duplicates, you use .union(qs, all=True) since .union() has default all=False and calls DISTINCT on queryset
photos = Photo.objects.annotate(c=Count('labels'))
one = photos.exclude(c__lte=4)
two = photos.filter(painting=True)
all = one.union(two, all=True)
one.count() + two.count() == all.count()
>>> True
then it should work like you described in question

querying using Q objects and also using Q single object

i have 2 queries which i have to join using '|' and apply the final one to get
the result.First i have list of countries i.e eu countries.and second i have a country from which the user is logged in.I want to join both ones and get the result using Q.First one is q= Q(country=auth.country)and second one is
q2 = Q(country = settings.euCountries), in which settings.enCountries is a django list.Any help in this matter is highly appreciated
If you need an OR for all countries
q = [Q(country=auth.country)] + [Q(country=i) for i in settings.euContries]
Then
import operator
Model.objects.filter(reduce(operator.or_, q))
I don't think you need multiple Q() objects here. You can use the __in lookup.
Q(country_in=[auth.country] + settings.euCountries)
Depending on your code, you might not need the Q object at all if you can do something like the following:
queryset = MyModel.objects.filter(country_in=[auth.country] + settings.euCountries)

Remove (filter out) objects from queryset

I'd like to remove 3 objects from my queryset. This working with the help of an extra list, but im pretty sure there should be a better way to do this with the QuerySet API. However I didnt figure out how yet:
What I'm doing:
ranks = Rank.objects.all()
remove_ranks = ['Field Marshall', 'Military Attache', 'Mercenary Recruiter']
new_ranks =[]
for rank in ranks:
if not rank.name in remove_ranks:
new_ranks.append(rank)
How can I do this using the Django API ?
try
remove_ranks = ['Field Marshall', 'Military Attache', 'Mercenary Recruiter']
Rank.objects.exclude(name__in=remove_ranks)
what does it do?
.exclude is the opposite of .filter
name__in is the equivalent of a IN-Statement in SQL
This should produce a sql query something along the line
Select * from rank where name not in ('Field Marshall', 'Military Attache', 'Mercenary Recruiter')

How to change the default ordering of a query?

I don't want any ordering to be applied to a query. So, I have a QuerySet follow as:
question_obj = Question.objects.filter(pk__in=[100,50,27,35,10,42,68]).order_by()
However, when I retrieve the results, they are always ordered by questionID. I iterate the question_obj and this is the result:
for obj in question_obj:
obj.questionID
The result is displayed such as:
10L
27L
35L
42L
50L
68L
100L
If you want to display the objects in the same order as the list of primary keys, then you could use in_bulk to create a dictionary keyed by pk. You can then use a list comprehension to generate the list of questions.
>>> pks = [100,50,27,35,10,42,68]
>>> questions_dict = Question.objects.in_bulk(pks)
>>> questions = [questions_dict[pk] for pk in pks]
>>> for question in questions:
print question.pk
100
50
27
35
...
If you want an unordered collection, use Python's Set object, documented here: http://docs.python.org/tutorial/datastructures.html#sets
If you want the ordering to be the same as the list you're passing as the value for pk__in, you could try:
ids = [100,50,27,35,10,42,68]
questions = list(Question.objects.filter(pk__in=ids))
question_obj = sorted(questions, key=lambda x: ids.index(x.id))
EDIT: And because it's extremely unclear as to what you mean by 'unordered' in reference to a data structure that is by definition ordered: Random ordering can be accomplished through the following:
.order_by('?')
Luke, use the Source, er, the Docs! Yeah, that's it!
Django QuerySet API - order_by()
You could do some raw SQL (with FIELD()) a lá:
Ordering by the order of values in a SQL IN() clause
which should allow you to retrieve them in the order suggested in the list.
To run custom SQL with the ORM:
https://docs.djangoproject.com/en/dev/topics/db/sql/#executing-custom-sql-directly

Django object multiple exclude()

Is there a way to do a query and exclude a list of things, instead of calling exclude multiple times?
Based on your reply to Ned, it sounds like you just want to exclude a list of tags. So you could just use the in filter:
names_to_exclude = [o.name for o in objects_to_exclude]
Foo.objects.exclude(name__in=names_to_exclude)
Does that do what you want?
What's wrong with calling exclude multiple times? Queries are lazy, nothing happens until you try to pull data from it, so there's no downside to using .exclude() more than once.
You can try this also.
exclude_list = ['A', 'B', 'C']
qs = Foo.objects.exclude(items__in=exclude_list)
You can do it pretty easily with the Q object:
from django.db.models import Q
excludes = None
for tag in ignored_tags:
q = Q(tag=tag)
excludes = (excludes and (excludes | q)) or q # makes sure excludes is set properly
set_minus_excluded = Foo.objects.exclude(excludes)
You should also be able to do it dynamically with exclude():
qs = Foo.objects.all()
for tag in ignored_tags:
qs = qs.exclude(tag=tag)
To improve on Daniel Roseman's answer I think it would be better to get the values you need directly from the queryset instead of the for loop that could be expensive on large data sets i.e.
names_to_exclude = objects_to_exclude.values_list('name')
Foo.objects.exclude(name__in=names_to_exclude)