I'm trying to display a chat log in django. I can get my entire chatlog in the proper order with this query.
latest_chats_list = Chat.objects.order_by('timestamp')
I want the functionality of this line (last 10 elements in order), but django doesn't allow negative indexes.
latest_chats_list = Chat.objects.order_by('timestamp')[-10:]
if I try this line, I get the messages I want, but they're in the wrong order.
latest_chats_list = Chat.objects.order_by('-timestamp')[:10]
This line gives the first 10 chats instead of the most recent.
latest_chats_list = Chat.objects.order_by('-timestamp')[:10].reverse()
last_ten = Chat.objects.all().order_by('-id')[:10]
last_ten_in_ascending_order = reversed(last_ten)
Edit (from comments)
Why not use Django's queryset.reverse() ?
Because it messes with the SQL query, as does queryset.order_by(). Slicing the queryset ([:10]) also alters the SQL query, adding LIMIT and OFFSET to it. The two can combine in not-obviously-expected ways...
On the other hand, the built-in Python function reversed(iterable) only changes the way queryset gets iterated over, not effecting the SQL at all.
Related
Can anyone help, I want to return an ordered list based on forloop in Django using a field in the model that contains both integer and string in the format MM/1234. The loop should return the values with the least interger(1234) in ascending order in the html template.
Ideally you want to change the model to have two fields, one integer and one string, so you can code a queryset with ordering based on the integer one. You can then define a property of the model to return the self.MM+"/"+str( self.nn) composite value if you often need to use that. But if it's somebody else's database schema, this may not be an option.
In which case you'll have to convert your queryset into a list (which reads all the data rows at once) and then sort the list in Python rather than in the database. You can run out of memory or bring your server to its knees if the list contains millions of objects. count=qs.count() is a DB operation that won't.
qs = Foo.objects.filter( your_selection_criteria)
# you might want to count this before the next step, and chicken out if too many
# also this simple key function will crash if there's ever no "/" in that_field
all_obj = sorted( list( qs),
key = lambda obj: obj.that_field.split('/')[1] )
When I do something like
I. objects = Model.objects.all()
and then
II. objects.filter(field_1=some_condition)
I hit db every time when on the step 2 with various conditions. Is there any way to get all data in first action and then just take care of the result?
You actually don't hit the db until you evaluate the qs, queries are lazy.
Read more here.
edit:
After re-reading your question it becomes apparent you were asking how to prevent db hits when filtering for different conditions.
qs = SomeModel.objects.all()
qs1 = qs.filter(some_field='some_value')
qs2 = qs.filter(some_field='some_other_value')
Usually you would want the database to do the filtering for you.
You could force an evaluation of the qs by converting it to a list. This would prevent further db hits, however it would likely be worse than having your db return you results.
qs_l = list(qs)
qs1_l = [element for element in qs_l if element.some_field='some_value']
qs2_l = [element for element in qs_l if element.some_field='some_other_value']
Of course you will hit db every time. filter() transforms to SQL statement which is executed by your db, you can't filter without hitting it. So you can retrieve all the objects you need with values() or list(Model.objects.all()) and, as zeekay suggested, use Python expressions (like list comprehensions) for additional filtering.
Why don't you just do objs = Model.objects.filter(field=condition)? That said, once the SQL query is executed you can use Python expressions to do further filtering/processing without incurring additional database hits.
I'm trying to order a list of items in django by the number of comments they have. However, there seems to be an issue in that the Count function doesn't take into account the fact that django comments also uses a content_type_id to discern between comments for different objects!
This gives me a slight problem in that the comment counts for all objects are wrong using the standard methods; is there a 'nice' fix or do I need to drop back to raw sql?
Code to try and ge the correct ordering:
app_list = App.objects.filter(published=True)
.annotate(num_comments=Count('comments'))
.order_by('-num_comments')
Sample output from the query (note no mention of the content type id):
SELECT "apps_app"."id", "apps_app"."name",
"apps_app"."description","apps_app"."author_name", "apps_app"."site_url",
"apps_app"."source_url", "apps_app"."date_added", "apps_app"."date_modified",
"apps_app"."published", "apps_app"."published_email_sent", "apps_app"."created_by_id",
"apps_app"."rating_votes", "apps_app"."rating_score", COUNT("django_comments"."id") AS
"num_comments" FROM "apps_app" LEFT OUTER JOIN "django_comments" ON ("apps_app"."id" =
"django_comments"."object_pk") WHERE "apps_app"."published" = 1 GROUP BY
"apps_app"."id", "apps_app"."name", "apps_app"."description", "apps_app"."author_name",
"apps_app"."site_url", "apps_app"."source_url", "apps_app"."date_added",
"apps_app"."date_modified", "apps_app"."published", "apps_app"."published_email_sent",
"apps_app"."created_by_id", "apps_app"."rating_votes", "apps_app"."rating_score" ORDER
BY num_comments DESC LIMIT 4
Think I found the answer: Django Snippet
How would one go about retrieving the last 1,000 values from a database via a Objects.filter? The one I am currently doing is bringing me the first 1,000 values to be entered into the database (i.e. 10,000 rows and it's bringing me the 1-1000, instead of 9000-1,000).
Current Code:
limit = 1000
Shop.objects.filter(ID = someArray[ID])[:limit]
Cheers
Solution:
queryset = Shop.objects.filter(id=someArray[id])
limit = 1000
count = queryset.count()
endoflist = queryset.order_by('timestamp')[count-limit:]
endoflist is the queryset you want.
Efficiency:
The following is from the django docs about the reverse() queryset method.
To retrieve the ''last'' five items in
a queryset, you could do this:
my_queryset.reverse()[:5]
Note that this is not quite the same
as slicing from the end of a sequence
in Python. The above example will
return the last item first, then the
penultimate item and so on. If we had
a Python sequence and looked at
seq[-5:], we would see the fifth-last
item first. Django doesn't support
that mode of access (slicing from the
end), because it's not possible to do
it efficiently in SQL.
So I'm not sure if my answer is merely inefficient, or extremely inefficient. I moved the order_by to the final query, but I'm not sure if this makes a difference.
reversed(Shop.objects.filter(id=someArray[id]).reverse()[:limit])
Thank to this post I'm able to easily do count and group by queries in a Django view:
Django equivalent for count and group by
What I'm doing in my app is displaying a list of coin types and face values available in my database for a country, so coins from the UK might have a face value of "1 farthing" or "6 pence". The face_value is the 6, the currency_type is the "pence", stored in a related table.
I have the following code in my view that gets me 90% of the way there:
def coins_by_country(request, country_name):
country = Country.objects.get(name=country_name)
coin_values = Collectible.objects.filter(country=country.id, type=1).extra(select={'count': 'count(1)'},
order_by=['-count']).values('count', 'face_value', 'currency_type')
coin_values.query.group_by = ['currency_type_id', 'face_value']
return render_to_response('icollectit/coins_by_country.html', {'coin_values': coin_values, 'country': country } )
The currency_type_id comes across as the number stored in the foreign key field (i.e. 4). What I want to do is retrieve the actual object that it references as part of the query (the Currency model, so I can get the Currency.name field in my template).
What's the best way to do that?
You can't do it with values(). But there's no need to use that - you can just get the actual Collectible objects, and each one will have a currency_type attribute that will be the relevant linked object.
And as justinhamade suggests, using select_related() will help to cut down the number of database queries.
Putting it together, you get:
coin_values = Collectible.objects.filter(country=country.id,
type=1).extra(
select={'count': 'count(1)'},
order_by=['-count']
).select_related()
select_related() got me pretty close, but it wanted me to add every field that I've selected to the group_by clause.
So I tried appending values() after the select_related(). No go. Then I tried various permutations of each in different positions of the query. Close, but not quite.
I ended up "wimping out" and just using raw SQL, since I already knew how to write the SQL query.
def coins_by_country(request, country_name):
country = get_object_or_404(Country, name=country_name)
cursor = connection.cursor()
cursor.execute('SELECT count(*), face_value, collection_currency.name FROM collection_collectible, collection_currency WHERE collection_collectible.currency_type_id = collection_currency.id AND country_id=%s AND type=1 group by face_value, collection_currency.name', [country.id] )
coin_values = cursor.fetchall()
return render_to_response('icollectit/coins_by_country.html', {'coin_values': coin_values, 'country': country } )
If there's a way to phrase that exact query in the Django queryset language I'd be curious to know. I imagine that an SQL join with a count and grouping by two columns isn't super-rare, so I'd be surprised if there wasn't a clean way.
Have you tried select_related() http://docs.djangoproject.com/en/dev/ref/models/querysets/#id4
I use it a lot it seems to work well then you can go coin_values.currency.name.
Also I dont think you need to do country=country.id in your filter, just country=country but I am not sure what difference that makes other than less typing.