I have one of the below example model
class Title(models.Model):
name = models.CharField(max_length=200)
provider_name = models.CharField(max_length=255, blank=True)
keywords = models.CharField(max_length=200, null=True, blank=True)
def __unicode__(self):
return '{0} - {1} (name/provider)'.format(name, provider_name)
so in order to order_by the Title model queryset with any model field, we can just do
titles = Title.objects.all().order_by('name')
But is it possible to order_by the queryset with particular value ? i mean i want to order_by the Title model queryset with the return value of unicode method, i.e., the combination of name and provider_name('{0} - {1} (name/provider)'.format(name, provider_name))
So overall instead of doing order_by with Model fields/database columns, i want to order by a value(Return value of unicode method in this case)
Is that possible to order_by the queryset with a value in ORM or else we need to write raw sql in order to achieve this ?
No, it's not possible to use a method in your filter because the ORM cannot translate it into SQL. In your case, if what you want is to order by name then by provider name, ordering by unicode method (even if it was possible) may give you wrong results since the names are not all the same length.
Using raw SQL for things Django can do is not a good idea either, so I think the best way to do it is:
titles = Title.objects.order_by('name', 'provider_name')
Related
Below is my post model.
class Post(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
title = models.CharField(max_length=200)
content = models.TextField()
datetime = models.DateTimeField(auto_now_add=True)
votes = models.ManyToManyField(settings.AUTH_USER_MODEL,
related_name="post_votes", default=None, blank=True)
tags = models.ManyToManyField(Tag, default=None, blank=True)
I want to filter posts which contain a certain query in their title, content or as the name of one of their tags. To do this I've tried:
query_set = Post.objects.filter(Q(content__icontains=query)|
Q(tags__name__icontains=query)|
Q(title__icontains=query))
But this often returns QuerySets with duplicate results. I have tried using the distinct method to solve this, but that results in incorrect ordering when I sort the posts later on by the number of votes they have:
query_set.annotate(vote_count=Count('votes')).order_by('-vote_count', '-datetime')
If anybody could help me I would be very grateful.
Jack
The duplicates originate from the fact that you filter on related objects. This means that Django will perform a query with a JOIN in it. You can of course perform a uniqness filter at the Django/Python level, but those are inefficient (well the ineffeciency is two-fold: first it will result in more data being transmitted from the database to the Django server, and furthermore Python does not handle large collections very well).
Furthermore the line:
query_set.annotate(vote_count=Count('votes')).order_by('-vote_count', '-datetime')
is basically a no-op, since QuerySets are immutable, here you did not sort the QuerySet on votes, you constructed a new one that will do that, but you immediately throw it away, since you do nothing with the result.
You can add the annotation and ordering and thus obtain distinct results later on:
query_set = Post.objects.filter(
Q(content__icontains=query)|
Q(tags__name__icontains=query)|
Q(title__icontains=query)
).annotate(
vote_count=Count('votes', distinct=True)
).order_by('-vote_count', '-date_time').distinct()
The distinct=True on the Count is necessary, since, as said before, the query acts like a JOIN, and JOINs can act like "multipliers" when counting things, since a row can occur multiple times.
I'm trying to query a related field to a Catalog class in which many items are related to by foreign key. I'm currently trying:
article = forms.ModelChoiceField(queryset=Catalog.objects.select_related(
'article_products'))
It seems to do the same query as:
queryset = Catalog.objects.all()
Can anyone help steer me in the right direction? Here is the model I'm working with.
class Catalog(models.Model):
products = models.CharField(max_length=200)
def __unicode__(self):
return self.products
class Article(models.Model):
catalog = models.ForeignKey(Catalog, related_name='article_products')
title = models.CharField(max_length=200)
abstract = models.TextField(max_length=1000, blank=True)
full_text = models.TextField(blank=True)
proquest_link = models.CharField(max_length=200, blank=True, null=True)
ebsco_link = models.CharField(max_length=200, blank=True, null=True)
def __unicode__(self):
return self.title
My goal is to have a form select field with all of the articles related to the Catalog. It currently just displays the name of the Catalog.
I do not think the select_related method will accomplish the goal you have set out to achieve with this ModelChoiceField. You are quite correct that the two queries below return the same resulting queryset:
Catalog.objects.all().select_related('article_products'))
Catalog.objects.all()
The select_related method of Django querysets serves a different function, specifically as a performance booster to reduce the number of database accesses required to obtain the data you want to retrieve from a model instance. The Django reference about this method contains very good documentation, with examples explaining why you would use the select_related method for performance purposes.
With that being said, your original purpose remains: The form field would display all of the articles related to a given catalog.
In order to achieve this goal, it seems best to filter the queryset of the Article objects being given to the form field. First of all, if we want to display Article objects within the ModelChoiceField, we should certainly give the ModelChoiceField a queryset containing Article objects rather than Catalog objects, like so:
article = forms.ModelChoiceField(queryset=Article.objects.all())
But this queryset argument is not quite right, either. We are still passing the queryset of all Article objects that exist in the database. Instead, we want to pass only the articles that are associated with a given Catalog object. To achieve this goal, we can filter the Article queryset to obtain only the Article objects that are related to a certain Catalog object, like so:
# cat is some catalog object
article = forms.ModelChoiceField(queryset=Article.objects.filter(catalog=cat))
In this example, the queryset filter returns only Article objects which contain a reference to the given Catalog object. This queryset will be used to populate the ModelChoiceField.
For more information about filtering by field lookup, see the Django documentation here.
Django recommends not using null on CharField, however annotate includes empty strings in the count. Is there a way to avoid that without excluding rows with empty string from the query?
My question isn't simly how to achieve my query, but fundamentally, should Annotate/Aggregate count include empty fields or not. Django consider empty as a replacement for NULL for string based fields.
My model :
class Book(models.Model):
name = models.CharField(...)
class Review(models.Model):
book = models.ForeignKey()
category = models.ForeignKey()
review = models.CharField(max_length=200, default='', blank=True)
To count non-empty reviews & group by category, I use
Review.objects.values('category').annotate(count=Count('review'))
This doesn't work because annotate counts empty values also (if the entry was NULL, it wouldn't have done so). I could filter out empty strings before the annotate call but my Query is more complex and I need all empty & non-empty objects.
Is there a smarter way to use annotate and skip empty values from count or should I change the model from
review = models.CharField(max_length=200, default='', blank=True)
to
review = models.CharField(max_length=200, default=None, blank=True, null=True)
I faced a very similar situation. I solved it using Conditional Expressions:
review_count = Case(
When(review='', then=0),
default=1,
output_field=IntegerField(),
)
Review.objects.values('category').annotate(count=review_count)
...and I need all empty & non-empty objects.
This doesn't make any sense when using values. Instead of actual objects, you'll get a list of dictionaries containing just the category and count keys. Apart from a different number in count, you'll see no difference between filtering out empty review values or not. On top of that, you filter for a single book (id=2) and somehow expect that there can be more than one review.
You need to seriously rethink what you are exactly trying to do, and how your model definition fits into that.
I have two Models in Django. The first has the hierarchy of what job functions (positions) report to which other positions, and the second is people and what job function they hold.
class PositionHierarchy(model.Model):
pcn = models.CharField(max_length=50)
title = models.CharField(max_length=100)
level = models.CharField(max_length=25)
report_to = models.ForeignKey('PositionHierachy', null=True)
class Person(model.Model):
first_name = models.CharField(max_length=50)
last_name = models.CharField(max_length=50)
...
position = models.ForeignKey(PositionHierarchy)
When I have a Person record and I want to find the person's manager, I have to do
manager = person.position.report_to.person_set.all()[0]
# Can't use .first() because we haven't upgraded to 1.6 yet
If I'm getting people with a QuerySet, I can join (and avoid a second trip to the database) with position and report_to using Person.objects.select_related('position', 'position__reports_to').filter(...), but is there any way to avoid making another trip to the database to get the person_set? I tried adding 'position__reports_to__person_set' or just position__reports_to__person to the select_related, but that doesn't seem to change the query. Is this what prefetch_related is for?
I'd like to make a custom manager so that when I do a query to get Person records, I also get their PositionHeirarchy and their manager's Person record without more round trips to the database. This is what I have so far:
class PersonWithManagerManager(models.Manager):
def get_query_set(self):
qs = super(PersonWithManagerManager, self).get_query_set()
return qs.select_related(
'position',
'position__reports_to',
).prefetch_related(
)
Yes, that is what prefetch_related() is for. It will require an additional query, but the idea is that it will get all of the related information at once, instead of once per Person.
In your case:
qs.select_related('position__report_to')
.prefetch_related('position__report_to__person_set')
should require two queries, regardless of the number of Persons in the original query set.
Compare this example from the documentation:
>>> Restaurant.objects.select_related('best_pizza')
.prefetch_related('best_pizza__toppings')
I'm making a little vocabulary-quiz app, and the basic model for a word is this:
class Word(models.Model):
id = models.AutoField(primary_key=True)
word = models.CharField(max_length=80)
id_image = models.ForeignKey(Image)
def __unicode__(self):
return self.word
class Meta:
db_table = u'word'
The model for words I'm currently quizzing myself on is this:
class WordToWorkOn(models.Model):
id = models.AutoField(primary_key=True)
id_student = models.ForeignKey(Student)
id_word = models.ForeignKey(Word)
level = models.IntegerField()
def __unicode__(self):
return u'%s %s' % (self.id_word.__unicode__(), self.id_student.__unicode__() )
class Meta:
db_table = u'word_to_work_on'
Where "level" indicates how well I've learned it. The set of words I've already learned has this model:
class WordLearned(models.Model):
id = models.AutoField(primary_key=True)
id_word = models.ForeignKey(Word, related_name='word_to_learn')
id_student = models.ForeignKey(Student, related_name='student_learning_word')
def __unicode__(self):
return u'%s %s' % (self.id_word.__unicode__(), self.id_student.__unicode__() )
class Meta:
db_table = u'word_learned'
When a queryset on WordToWorkOn comes back with too few results (because they have been learned well enough to get moved into WordLearned and deleted from WordToWorkOn), I want to find a Word to add to it. The part I don't know a good way to do is to limit it to Words which are not already in WordLearned.
So, generally speaking, I think I want to do an .exclude() of some sort on a queryset of Words, but it needs to exclude based on membership in the WordLearned table. Is there a good way to do this? I find lots of references to joining querysets, but couldn't find a good one on how to do this (probably just don't know the right term to search for).
I don't want to just use a flag on each Word to indicate learned, working on it, or not learned, because eventually this will be a multi-user app and I wouldn't want to have flags for every user. Hence, I thought multiple tables for each set would be better.
All advice is appreciated.
Firstly, a couple of notes about style.
There's no need to prefix the foreign key fields with id_. The underlying database field that Django creates for those FKs are suffixed with _id anyway, so you'll get something like id_word_id in the db. It'll make your code much clearer if you just call the fields 'word', 'student', etc.
Also, there's no need to specify the id autofields in each model. They are created automatically, and you should only specify them if you need to call them something else. Similarly, no need to specify db_table in your Meta, as this is also done automatically.
Finally, no need to call __unicode__ on the fields in your unicode method. The string interpolation will do that automatically, and again leaving it out will make your code much easier to read. (If you really want to do it explicitly, at least use the unicode(self.word) form.)
Anyway, on to your actual question. You can't 'join' querysets as such - the normal way to do a cross-model query is to have a foreignkey from one model to the other. You could do this:
words_to_work_on = Word.objects.exclude(WordLearned.objects.filter(student=user))
which under the hood will do a subquery to get all the WordLearned objects for the current user and exclude them from the list of words returned.
However, and especially bearing in mind your future requirement for a multiuser app, I think you should restructure your tables. What you want is a ManyToMany relationship between Word and Student, with an intermediary table capturing the status of a Word for a particular Student. That way you can get rid of the WordToWorkOn and WordLearned tables, which are basically duplicates.
Something like:
class Word(models.Model):
word = models.CharField(max_length=80)
image = models.ForeignKey(Image)
def __unicode__(self):
return self.word
class Student(models.Model):
... name, etc ...
words = models.ManyToManyField(Word, through='StudentWord')
class StudentWord(models.Model):
word = models.ForeignKey(Word)
student = models.ForeignKey(Student)
level = models.IntegerField()
learned = models.BooleanField()
Now you can get all the words to learn for a particular student:
words_to_learn = Word.objects.filter(studentword__student=student, studentword__learned=False)