I have a Django model called Collection that represents a collection of items (CollectionItem). Each Collection only contains items of a specific type. (CollectionItem has a foreign key to Collection).
I want to get all of the CollectionItems that are in public-flagged lists of a specific type and return them sorted by a particular field. Here's the query code that I use:
lists = Collection.objects.filter(is_public=True, type=7)
items = CollectionItem.objects.none()
for list in lists:
items |= CollectionItem.objects.filter(collection=list)
items = items.order_by('name')
I have to imagine that this will not scale well at all when I have a large database with tons of lists and items. Is there a better way to do this in Django? Or is the inefficiency involved in the query loop negligible enough compared to other options that I shouldn't worry too much?
Sounds like you just need:
items = CollectionItem.objects.filter(
collection__is_public=True, collection__type=7
).order_by('name')
Related
Can anyone help, I want to return an ordered list based on forloop in Django using a field in the model that contains both integer and string in the format MM/1234. The loop should return the values with the least interger(1234) in ascending order in the html template.
Ideally you want to change the model to have two fields, one integer and one string, so you can code a queryset with ordering based on the integer one. You can then define a property of the model to return the self.MM+"/"+str( self.nn) composite value if you often need to use that. But if it's somebody else's database schema, this may not be an option.
In which case you'll have to convert your queryset into a list (which reads all the data rows at once) and then sort the list in Python rather than in the database. You can run out of memory or bring your server to its knees if the list contains millions of objects. count=qs.count() is a DB operation that won't.
qs = Foo.objects.filter( your_selection_criteria)
# you might want to count this before the next step, and chicken out if too many
# also this simple key function will crash if there's ever no "/" in that_field
all_obj = sorted( list( qs),
key = lambda obj: obj.that_field.split('/')[1] )
I looked at this thread, with an example of sorting dictionaries.
I have a dictionary of programme objects where the key is a programme object and the value is a lookup of the number of related Project objects.
def DepartmentDetail(request, pk):
department = Department.objects.get(pk=pk)
programmes = Programme.objects.all().filter(department=department).exclude(active=False).order_by('long_name')
combi = {}
for p in programmes:
prj = Project.objects.all().filter(programme=p)
combi[p] = str(len(prj))
return render(request, 'sysadmin/department.html',{'department': department, 'programmes': programmes, 'combi': sorted(combi.items())})
In the model, Programme returns a string 'long_name', so I believe that I am trying to sort a string key and a string value.
In the template I get to the keys and values as so,
{% for programme, n in combi %}
This gives me the error..
unorderable types: Programme() < Programme()
I don't really understand the error, in the python 3 documentation it states that the sorted() method accepts any iterable - So why does this happen?
I'm looking at collections.OrderedDict to solve the problem, but I want to know why this doesn't work.
Thanx.
Databases with index on columns are really good at sorting. There's almost never a need to sort on the client side. You can almost always do it in the server. The funny part it you apparently know how to do it too.
....exclude(active=False).order_by('long_name') # <--- this
Guess what, your data is already sorted there isn't a need to sort it again inside python!!
But there is a much bigger issue in your code. You are fetching a set of Project items and then looping through that set to retrieve them all over again one by one. So if you have 200 Project items you are doing 200 queries when one query does the job just as well. Just add select_related or prefetch_related depending on which direction you have the relationship.
Your code ideally should be something like this
department = Department.objects.get(pk=pk)
programmes = Programme.objects.all().filter(department=department).exclude(active=False).order_by('long_name')
return render(request, 'sysadmin/department.html',{'department': department, 'programmes': programmes,})
As far as I can see combi just contains duplicated data. The same thing can be accessed from programmes eg. programme.project_set.all()
(again this depends on which direction you have the relationship, your models are not shown)
Recommended reading: https://docs.djangoproject.com/en/1.10/ref/models/fields/#django.db.models.ForeignKey
The issue is that sorted expects a way to be able to order the items and by default there isn't any way to know how to order your objects. You can provide a key
sorted(combi.items(), key=lambda i: i.long_name)
Long story short, I am using polymorphic models and some queries I just cannot do as normal, so I have to convert the query result into a normal list. Then use the paginator to page results.
To put things into perspective, normally we would do it like this:
instances = Model.objects.filter(field=True)
Paginator(instances, 10).page(1)
In my case:
instances = [x for x in Model.objects.filter(field=True)]
Paginator(instances, 10).page(1)
Given the ORM is lazy, so normally when paged, the raw query will have OFFSET and LIMITS which makes the query very lightweight. But in my case, since I am looping through the query result, I am actually hitting the database for all instances, then apply pagination.
I tried to read the raw queries by printing connection.queries but the polymorphic model queries are kind of headache to read through.
So hopefully this will be a quick question for anyone knows the guts of django ORM.
EDIT
To anyone who is curious why I need to convert to a list, have a look at this link and look for my problem https://github.com/bconstantin/django_polymorphic/issues/19 my way to work around it is to loop through the tags and filter in the loop as below:
_items = []
for tag in tags:
for item in items.filter(tags__slug = tag):
# get unique items
if item not in _items:
_items.append(item)
# sadly the following doesn't work due to polymorphic query limitation
# _items = None
# for item in items.filter(tags__slug = tag):
# _items = _items | items.filter(tags__slug = tag)
I have three models, Which has some common but not exact fields, from a single view ex. home I am retrieving like this
interviews = Interviews.objects.all().order_by('pub_date')[:3]
publications = Publications.objects.all().order_by('pub_date')[:3]
published = Published.objects.all().order_by('pub_date')[:3]
From home view I want them to show in template in such order that all the latest/New entries associated with these models will be in top.
like if interview entry 10 is the most recent entry in these all models, then it will be first, then if the published one is second recent it will be second ... etc .etc
Can any one tell me how to do it?
Depending on your other requirements it may be a good idea to make them into sublcasses of a common superclass - containing the common elements.
It's hard to say if it's valid, but if you do you can query the different types of objects separately or together by calling
SuperClass.objects.all().order_by('-pub_date')[:9]
which will render the 9 first objects regerdless of what sublcass they are. Of course assuming the superclass is named SuperClass. Of course this does not insure that there are 3 of each model.
Another simple way of solving it - although admittedly not using a query would simply be to sort the lists.
entries = sorted(list(interviews) + list(publications) + list(published), key=lambda x: x.pub_date, reverse=True)
should work - basically turning them into lists and sorting them.
One possible way is using lambda to sort data, but costs are hiher since you will doing this to python, not DBMS...
lst = []
lst.extend(list(Interviews.objects.order_by('pub_date')[:10]))
lst.extend(list(Publications.objects.order_by('pub_date')[:10]))
lst.extend(list(Published.objects.order_by('pub_date')[:10]))
# take 10 records for each, since you could not know how many records will be picked from which table
# now order them...
lst.sort(lambda x, y: cmp(x.pub_date, y.pub_date))
# and reverse the order, so newset is the first...
lst.reverse()
this will give you a list of objects ordered py pub_date, so you can slice the final list to get any number of records you want...
lst = lst[:10]
I have Django models that are lists of items and I want the items on each list to have a separate sort order. Maybe list 1 will sort line items by name, list 2 by date, and list 3 by priority.
The models looks more or less like this:
class ListItem(models.Model):
priority = models.IntegerField(choices=PRIORITY_CHOICES)
name = models.CharField(max_length=240)
due_date = models.DateTimeField(null=True, blank=True)
list = models.ForeignKey(List)
Right now I'm using this query in my view:
lists = List.objects.filter(user=request.user)
return render_to_response('showlists.html', {'lists': lists })
I've read examples that imply that I could do something like this:
lists = List.objects.filter(user=request.user).select_related().order_by("items.name")
Assuming that worked, it would give me the collection of lists with an identical sort order for each batch of items. How would I go about sorting each individual list's related items differently?
The syntax for ordering across related models is the same double-underscore format as for filter, so you can do:
List.objects.filter(user=request.user).select_related().order_by("listitems__name")
You should just able to use the different fields in the same way in the order_by call.
If the lists are small, then you could do it in python, using sort and tagging
list = <...code to get sublist here ...>
list.sort(key=lambda x: x.due_date) #sort by due date, say
Otherwise kick off a query for each sub-list