I would like to have my search results sorted by use in a foreign key relation when searching for one of my Django models.
Example:
Model "Tag" -> Many to Many <- Model "Post"
If I am searching for a tag, I would like to get the query matching tags returned in the order they are used in the relation. This means the most used tag that meets the search criteria first, etc.
Is that possible, if so, how?
I have big problems to adapt the proposed approach to my application, so here some code for clarification:
class Tag(models.Model):
class Meta:
ordering = #by number of relations to Post
class Post(models.Model):
tags = models.ManyToManyField('Tag')
Here's a more complete answer based on Alexandr's:
For the sake of the answer, I assume that your Tag model has a name field to search by.
from django.db.models import Count
search_term = 'this is the string you search for'
query = Tag.objects.filter(name__contains=search_term).annotate(post_count=Count('post')).order_by('-post_count')
In more detail:
Tag.objects.filter(name__contains=search_term) returns a QuerySet with Tag instances whose name contains the expression defined in the variable search_term (we assume this comes from the user, who wants to search for tags).
.annotate(post_count=Count('post') adds an extra field to the instances that contains the number of relations to Posts for that specific instance. You can refer to this in your code in the following way:
for tag in query:
print('This tag is used for', tag.post_count, 'posts')
Finally, .order_by('-post_count') sets the order to be by post count, in descending order (this is what the - denotes).
To order by most used tag, you probably should use something like
from django.db.models import Count
queryset = matching_tags.annotate(used_count=Count('posts'))
queryset = queryset.order_by('-used_count')
Related
I have following model and extracted queryset using prefetch_related as below.
queryset = Light.objects.filter(
certificate__name="A").prefetch_related('zone__namingzone'
)
From this queryset, I want to get following data set.
{"naming1":lpd1,"naming2":lpd2...}
However, when I try to extract attribute from queryset as below, I get create_reverse_many_to_one_manager
for i in queryset:
print (i.zone.namingzone)
What I want to get is naming attribute in naming table. Could anyone tell me how I can extract this?
models.py
class Certificate(models.Model):
name=models.CharField(max_length=20)
class Zone(models.Model):
zone=models.CharField(max_length=20)
class Light(models.Model):
certificate=models.ForeignKey(Certificate, on_delete=models.CASCADE,related_name='certificate')
zone=models.ForeignKey(Zone, on_delete=models.CASCADE,related_name='lightzone')
lpd=models.IntegerField()
class Meta:
unique_together = (('certificate', 'zone'),)
class Naming(models.Model):
zone=models.ForeignKey(Zone, on_delete=models.CASCADE,related_name='namingzone')
naming=models.CharField(max_length=20)
When you traverse a FK in reverse, you end up with a manager, and multiple items on the other side. So i.zone.namingzone in your for loop is a manager, not a NamingZone. If you change your print loop to:
for i in queryset:
print (i.zone.namingzone.all())
You should see all the naming zones for your item. You can extract the naming field from each NamingZone from the queryset as follows:
queryset.values('zone__namingzone__naming')
You probably want to extract a few other fields from your Light model, like the lpd for instance:
queryset.values('lpd', 'zone__namingzone__naming')
You might have a same ldp several times, as many times as it has naming zones.
I have been mulling over this for a while looking at many stackoverflow questions and going through aggregation docs
I'm needing to get a dataset of PropertyImpressions grouped by date. Here is the PropertyImpression model:
#models.py
class PropertyImpression(models.Model):
'''
Impression data for Property Items
'''
property = models.ForeignKey(Property, db_index=True)
imp_date = models.DateField(auto_now_add=True)
I have tried so many variations of the view code, but I'm posting this code because I consider to be the most logical, simple code, which according to documentation and examples should do what I'm trying to do.
#views.py
def admin_home(request):
'''
this is the home dashboard for admins, which currently just means staff.
Other users that try to access this page will be redirected to login.
'''
prop_imps = PropertyImpression.objects.values('imp_date').annotate(count=Count('id'))
return render(request, 'reportcontent/admin_home.html', {'prop_imps':prop_imps})
Then in the template when using the {{ prop_imps }} variable, it gives me a list of the PropertyImpressions, but are grouped by both imp_date and property. I need this to only group by imp_date, and by adding the .values('imp_date') according to values docs it would just be grouping by that field?
When leaving off the .annotate in the prop_imps variable, it gives me a list of all the imp_dates, which is really close, but when I group by the date field it for some reason groups by both imp_date and property.
Maybe you have defined a default ordering in your PropertyImpression model?
In this case, you should add order_by() before annotate to reset it :
prop_imps = PropertyImpression.objects.values('imp_date').order_by() \
.annotate(count=Count('id'))
It's explained in Django documentation here:
Fields that are mentioned in the order_by() part of a queryset (or which are used in the default ordering on a model) are used when selecting the output data, even if they are not otherwise specified in the values() call. These extra fields are used to group “like” results together and they can make otherwise identical result rows appear to be separate. This shows up, particularly, when counting things.
I have following models:
class Product(models.Model):
name = CharField(max_length=30)
class Store(models.Model):
name = CharField(max_length=30)
product = models.ManyToManyField(Product)
How to get Stores with product named product_name and also, get all the products (except the product with name product_name) ? Is it possible to make it in one query?
In raw SQL it would be simple JOINs. Not sure how to implement it via Django.
You can actually do these things with Django due to it's lazy queryset evaluation. Django's in field lookup accepts both lists and querysets. The following will create a nested SQL code:
products = Product.objects.filter(store_set__in=stores_qs)
stores_qs = Store.objects.filter(product__name='product_name')
Here are the Django in docs.
You should be able to filter the stores based on an attribute of Product, and then prefetch_related of the retrieved objects.
Store.objects.filter(product__name="product_name").prefetch_related('product')
This should hit the database the fewest times to achieve what you are looking for - twice.
Further documentation can be found here.
Get Stores with product named "product_name" :
Store.objects.filter(product__name='product_name')
Get all the products except the product with name "product_name":
Product.objects.exclude(name='product_name')
I'm building a basic time logging app right now and I have a todo model that uses django-taggit. My Todo model looks like this:
class Todo(models.Model):
project = models.ForeignKey(Project)
description = models.CharField(max_length=300)
is_done = models.BooleanField(default=False)
billable = models.BooleanField(default=True)
date_completed = models.DateTimeField(blank=True, null=True)
completed_by = models.ForeignKey(User, blank=True, null=True)
tags = TaggableManager()
def __unicode__(self):
return self.description
I'm trying to get a list of unique tags for all the Todos in a project and I have managed to get this to work using a set comprehension, however for every Todo in the project I have to query the database to get the tags. My set comprehension is:
unique_tags = { tag.name.lower() for todo in project.todo_set.all() for tag in todo.tags.all() }
This works just fine, however for every todo in the project it runs a separate query to grab all the tags. I was wondering if there is any way I can do something similar to prefetch_related in order to avoid these duplicate queries:
unique_tags = { tag.name.lower() for todo in project.todo_set.all().prefetch_related('tags') for tag in todo.tags.all() }
Running the previous code gives me the error:
'tags' does not resolve to a item that supports prefetching - this is an invalid parameter to prefetch_related().
I did see that someone asked a very similar question here: Optimize django query to pull foreign key and django-taggit relationship however it doesn't look like it ever got a definite answer. I was hoping someone could help me out. Thanks!
Taggit now supports prefetch_related directly on tag fields (in version 0.11.0 and later, released 2013-11-25).
This feature was introduced in this pull request. In the test case for it, notice that after prefetching tags using .prefetch_related('tags'), there are 0 additional queries for listing the tags.
Slightly hackish soution:
ct = ContentType.objects.get_for_model(Todo)
todo_pks = [each.pk for each in project.todo_set.all()]
tagged_items = TaggedItem.objects.filter(content_type=ct, object_id__in=todo_pks) #only one db query
unique_tags = set([each.tag for each in tagged_items])
Explanation
I say it is hackish because we had to use TaggedItem and ContentType which taggit uses internally.
Taggit doesn't provide any method for your particular use case. The reason is because it is generic. The intention for taggit is that any instance of any model can be tagged. So, it makes use of ContentType and GenericForeignKey for that.
The models used internally in taggit are Tag and TaggedItem. Model Tag only contains the string representation of the tag. TaggedItem is the model which is used to associate these tags with any object. Since the tags should be associatable with any object, TaggedItem uses model ContentType.
The apis provided by taggit like tags.all(), tags.add() etc internally make use of TaggedItem and filters on this model to give you the tags for a particular instance.
Since, your requirement is to get all the tags for a particular list of objects we had to make use of the internal classes used by taggit.
Use django-tagging and method usage_for_model
def usage_for_model(self, model, counts=False, min_count=None, filters=None):
"""
Obtain a list of tags associated with instances of the given
Model class.
If ``counts`` is True, a ``count`` attribute will be added to
each tag, indicating how many times it has been used against
the Model class in question.
If ``min_count`` is given, only tags which have a ``count``
greater than or equal to ``min_count`` will be returned.
Passing a value for ``min_count`` implies ``counts=True``.
To limit the tags (and counts, if specified) returned to those
used by a subset of the Model's instances, pass a dictionary
of field lookups to be applied to the given Model as the
``filters`` argument.
"""
A slightly less hackish answer than akshar's, but only slightly...
You can use prefetch_related as long as you traverse the tagged_item relations yourself, using the clause prefetch_related('tagged_items__tag'). Unfortunately, todo.tags.all() won't take advantage of that prefetch - the 'tags' manager will still end up doing its own query - so you have to step over the tagged_items relation there too. This should do the job:
unique_tags = { tagged_item.tag.name.lower()
for todo in project.todo_set.all().prefetch_related('tagged_items__tag')
for tagged_item in todo.tagged_items.all() }
I run a lab annotation website where users can annotate samples with tags relating to disease, tissue type, etc. Here is a simple example from models.py:
from django.contrib.auth.models import User
from django.db import models
class Sample(models.Model):
name = models.CharField(max_length = 255)
tags=models.ManyToManyField('Tag', through = 'Annot')
class Tag(models.Model):
name = models.CharField(max_length = 255)
class Annot(models.Model):
tag = models.ForeignKey('Tag')
sample = models.ForeignKey('Sample')
user = models.ForeignKey(User, null = True)
date = models.DateField(auto_now_add = True)
I'm looking for a query in django's ORM which will return the tags in which two users agree on the annotation of same tag. It would be helpful if I could supply a list of users to limit my query (if someone only believes User1 and User2 and wants to find the sample/tag pairs that only they agree on.)
I think I understood what you need. This one made me think, thanks! :-)
I believe the equivalent SQL query would be something like:
select t.name, s.name, count(user_id) count_of_users
from yourapp_annot a, yourapp_tag t, yourapp_sample s
where a.tag_id = t.id
and s.id = a.sample_id
group by t.name, s.name
having count_of_users > 1
While I try hard not to think in SQL when I'm coming up with django model navigation (it tends to get in the way); when it comes to aggregation queries it always helps me to visualize what the SQL would be.
In django we now have aggregations.
Here is what I came up with:
models.Annot.objects.select_related().values(
'tag__name','sample__name').annotate(
count_of_users=Count('user__id')).filter(count_of_users__gt=1)
The result set will contain the tag, the sample, and the count of users that tagged said sample with said tag.
Breaking it apart for the folks that are not used to django aggregation:
models.Annot.objects.select_related()
select_related() is forcing all tables related to Annot to be retrieved in the same query
This is what will allow me to specify tag__name and sample__name in the values() call
values('tag__name','sample__name')
values() is limiting the fields to retrieve to tag.name and sample.name
This makes sure that my aggregation on count of clients will group by just these fields
annotate(count_of_users=Count('user__id'))
annotate() adds an aggregation as an extra field to a query
filter(count_of_users__gt=1)
And finally I filter on the aggregate count.
If you want to add an additional filter on what users should be taken into account, you need to do this:
models.Annot.objects.filter(user=[... list of users...]).select_related().values(
'tag__name','sample__name').annotate(
count_of_users=Count('user__id')).filter(count_of_users__gt=1)
I think that is it.
One thing... Notice that I used tag__name and sample__name in the query above. But your models do not specify that tag names and sample names are unique.
Should they be unique? Add a unique=True to the field definitions in the models.
Shouldn't they be unique? You need to replace tag__name and sample__name with tag__id and sample__id in the query above.