My question is I have this code.
reset_numbers = Picture.objects.filter(author__id=author.id)
for index, pic in enumerate(reset_numbers):
pic.number = index
pic.save()
I want to reset the ording of Pictures.
How I can do this in one query or less queries to get rid of the for loop?
Is there some way to do this?
pks = Picture.objects.filter(author__id=author.id).values_list('pk', flat=True)
for index, pk in enumrate(pks):
Picture.objects.filter(pk=pk).update(number=index)
This loads only primary keys, and only update number field. But query count is same as the original code.
Related
Suppose we have a model in django defined as follows:
class DateClass:
user_id = models.IntegerField(...)
sp_date = models.DateField(...)
is_active = models.BooleanField(...)
...
I follow insert policy here, i.e, for a specific user there will be only one specific active date. That means, there will be only one active row for user=1 at date table for sp_date values 27/10/2021, 28/10/2021 and so one. There shouldn't be two active rows for 27/10/2021 for user=1, but for other users have there rows for 27/10/2021. Whenever a date has to be updated, I deactivate (is_active=False) the previous row and add a new row for specific date.
I want to find duplicate active dates for each users in one single query, and then deactivate (set is_active=False) all the duplicate values except the last row (The row which was last inserted). Two rows will be duplicate if the values of user_id and sp_date are equal and both have is_active=True. I know how to find duplicates for a specific column which is fairly easy. But I can't think of something which can do the above task elegantly. I can only think of following approach:
for user in users:
dates = DateClass(user_id=user.id, is_active=True)
for date in dates:
days = dates.filter(
sp_date=date.sp_date, is_active=True
)
if days.count() > 1:
last_day = days.last()
days.exclude(id=last_day.id).update(is_active=False)
As you can see above one is not that efficient, as I have to loop through all users. Is there any way to do this more efficiently? I am using PostgreSQL for database.
There a great answer for multiple duplicate fields queryset from this answer as i don't want to take the credit and also don't want to reinvent the wheel, so i will suggest that answer
For your case it should be:
from django.db.models import Max, Count
duplicate_date_class = DateClass.objects.values('user_id', 'sp_date') \
.annotate(records=Count('user_id')) \
.filter(records__gt=1)
# Then do operations on duplicates
for date_class in duplicate_date_class:
DateClass.objects.filter(
user_id=date_class['user_id'],
sp_date=date_class['sp_date']
)[1:].update(is_active=False)
If you want to avoid having duplicate set of multiple fields, i suggest taking a look at unique_together for model validation
I get a queryset object every time i want some data from models.
So when i say,
"items = Items.object.get(value=value)"
I get --
"<QuerySet [<Item-name>]>"
I have to iterate through the queryset object to get the data and I do that with
"items[0]"
Is there any way I can avoid this?
Edit: I meant "items = Items.object.filter(value=value)"
first of all items = Items.objects.get(value=value) does not return a queryset,
rather it returns an object of <Items: Items object (1)>
To get the first(or just one result) or last date from the object, do this Items.objects.first() or Items.objects.last()
To get the desired data without using its index position, then you can filter it like this Items.objects.filter(value=value)
You are mistaken. items = Items.object.get(value=value) will not give you a queryset, but an object. items = Items.object.filter(value=value)
would give you a queryset.
Filter method will always give you a queryset, because; in order to minimize the need of database hits, django considers you might add additional filters through your code. So if you not execute that queryset, e.g. by using list(your_queryset) django never hits the database.
# when you are using 'get' in your query, you don't need to iterate, directly get an access to the field values
try:
items = Items.object.get(value=value)
except Items.DoesNotExist:
items = None
if items:
print(items.value)
I have a list of "posts" I have to render. For each post, I must do three filter querysets, OR them together, and then count the number of objects. Is this reasonable? What factors might make this slow?
This is roughly my code:
def viewable_posts(request, post):
private_posts = post.replies.filter(permissions=Post.PRIVATE, author_profile=request.user.user_profile).order_by('-modified_date')
community_posts = post.replies.filter(permissions=Post.COMMUNITY, author_profile__in=request.user.user_profile.following.all()).order_by('-modified_date')
public_posts = post.replies.filter(permissions=Post.PUBLIC).order_by('-modified_date')
mixed_posts = private_posts | community_posts | public_posts
return mixed_posts
def viewable_posts_count(request, post):
return viewable_posts(request, post).count()
The biggest factor I can see is that you have filter actions on each post. If possible, you should query the results associated with each post in ONE query. As of the count, it's the most efficient way of getting the number of results from a query, so it's likely not a problem.
Try the following code:
def viewable_posts(request, post):
private_posts = post.replies.filter(permissions=Post.PRIVATE, author_profile=request.user.user_profile).values_list('id',flat=True)
community_posts = post.replies.filter(permissions=Post.COMMUNITY, author_profile__in=request.user.user_profile.following.values_list('id',flat=True)
public_posts = post.replies.filter(permissions=Post.PUBLIC).values_list('id',flat=True)
Lposts_id = private_posts
Lposts_id.extend(community_posts)
Lposts_id.extend(public_posts)
viewable_posts = post.filter(id__in=Lposts_id).order_by('-modified_date')
viewable_posts_count = post.filter(id__in=Lposts_id).count()
return viewable_posts,viewable_posts_count
It should improve the following things:
order_by once, instead of three times
The count method runs on a query with only the index field
django uses a faster filter with "values", both for the count and the filtering.
Depends on your database, the db own cache may pick the last queried posts for viewable_posts, and use it for viewable_posts_count
Indeed, if you can squeeze the first three filter queries into one, you will save time as well.
How would one go about retrieving the last 1,000 values from a database via a Objects.filter? The one I am currently doing is bringing me the first 1,000 values to be entered into the database (i.e. 10,000 rows and it's bringing me the 1-1000, instead of 9000-1,000).
Current Code:
limit = 1000
Shop.objects.filter(ID = someArray[ID])[:limit]
Cheers
Solution:
queryset = Shop.objects.filter(id=someArray[id])
limit = 1000
count = queryset.count()
endoflist = queryset.order_by('timestamp')[count-limit:]
endoflist is the queryset you want.
Efficiency:
The following is from the django docs about the reverse() queryset method.
To retrieve the ''last'' five items in
a queryset, you could do this:
my_queryset.reverse()[:5]
Note that this is not quite the same
as slicing from the end of a sequence
in Python. The above example will
return the last item first, then the
penultimate item and so on. If we had
a Python sequence and looked at
seq[-5:], we would see the fifth-last
item first. Django doesn't support
that mode of access (slicing from the
end), because it's not possible to do
it efficiently in SQL.
So I'm not sure if my answer is merely inefficient, or extremely inefficient. I moved the order_by to the final query, but I'm not sure if this makes a difference.
reversed(Shop.objects.filter(id=someArray[id]).reverse()[:limit])
Thank to this post I'm able to easily do count and group by queries in a Django view:
Django equivalent for count and group by
What I'm doing in my app is displaying a list of coin types and face values available in my database for a country, so coins from the UK might have a face value of "1 farthing" or "6 pence". The face_value is the 6, the currency_type is the "pence", stored in a related table.
I have the following code in my view that gets me 90% of the way there:
def coins_by_country(request, country_name):
country = Country.objects.get(name=country_name)
coin_values = Collectible.objects.filter(country=country.id, type=1).extra(select={'count': 'count(1)'},
order_by=['-count']).values('count', 'face_value', 'currency_type')
coin_values.query.group_by = ['currency_type_id', 'face_value']
return render_to_response('icollectit/coins_by_country.html', {'coin_values': coin_values, 'country': country } )
The currency_type_id comes across as the number stored in the foreign key field (i.e. 4). What I want to do is retrieve the actual object that it references as part of the query (the Currency model, so I can get the Currency.name field in my template).
What's the best way to do that?
You can't do it with values(). But there's no need to use that - you can just get the actual Collectible objects, and each one will have a currency_type attribute that will be the relevant linked object.
And as justinhamade suggests, using select_related() will help to cut down the number of database queries.
Putting it together, you get:
coin_values = Collectible.objects.filter(country=country.id,
type=1).extra(
select={'count': 'count(1)'},
order_by=['-count']
).select_related()
select_related() got me pretty close, but it wanted me to add every field that I've selected to the group_by clause.
So I tried appending values() after the select_related(). No go. Then I tried various permutations of each in different positions of the query. Close, but not quite.
I ended up "wimping out" and just using raw SQL, since I already knew how to write the SQL query.
def coins_by_country(request, country_name):
country = get_object_or_404(Country, name=country_name)
cursor = connection.cursor()
cursor.execute('SELECT count(*), face_value, collection_currency.name FROM collection_collectible, collection_currency WHERE collection_collectible.currency_type_id = collection_currency.id AND country_id=%s AND type=1 group by face_value, collection_currency.name', [country.id] )
coin_values = cursor.fetchall()
return render_to_response('icollectit/coins_by_country.html', {'coin_values': coin_values, 'country': country } )
If there's a way to phrase that exact query in the Django queryset language I'd be curious to know. I imagine that an SQL join with a count and grouping by two columns isn't super-rare, so I'd be surprised if there wasn't a clean way.
Have you tried select_related() http://docs.djangoproject.com/en/dev/ref/models/querysets/#id4
I use it a lot it seems to work well then you can go coin_values.currency.name.
Also I dont think you need to do country=country.id in your filter, just country=country but I am not sure what difference that makes other than less typing.