I'm using django-haystack for a search page on my site. I'm basically done, but not quite happy with the ordering and not quite sure how haystack decides how to order everything.
I know I can over-ride the SearchQuerySet by using order_by but that over-rides it entirely. Let's say I want to force the search to order by in stock (BooleanField), so that the products that are in stock show up on top, but then do everything else as it normally would. How do I do that?
I tried doing order_by('-in_stock', 'content') figure content was what it used by default but it produces very different results from if I just leave it to do its own ordering.
Thanks for any input on this matter!
You must have a index in your search_indexes.py with in_stock:
class YourModel(indexes.SearchIndex):
in_stock = indexes.BooleanField(model_attr='model_in_stock')
and in your urls:
sqs = SearchQuerySet().models(YourModel).order_by('-in_stock', 'score') # score is a field of haystack
In this way, you show the results first if they are in stock and then by score!
To sort on a CharField, make it storable, but not indexable.
sorted_name = indexes.CharField(indexed=False, stored=True)
If you need to have it sortable and indexable, use two different fields in the SearchIndex.
Related
I need to do some custom sorting on a Member model which looks like this (simplified):
class User():
username = models.CharField()
# ...
class Member():
user = models.ForeignKey(User) # not required
invite_email = models.EmailField()
# ...
I need to sort my queryset of Member in the following way:
Members that do not have a user set come last;
Sort by user username alphabetically (case insensitive), or invite_email if Member does not have a user.
I can do the appropriate sorting with django:
queryset.order_by(Lower('user__username').asc(nulls_last=True), 'invite_email')
I thought that I could override the order_by method to apply my custom ordering.
My problem comes with DRF cursor pagination. Reading from the doc, it seems like the ordering should be a string, or an unchanging value generally speaking. But when I use the Lower expression to order my queryset as needed, the ordering received in the cursor pagination class is an OrderBy object. Doing so, the paginate_queryset method is unusable.
I find it weird that you can't apply a custom filtering with the DRF pagination classes. Am I missing something?
Thanks in advance for any tips you might have!
You may use a different kind of Pagination for your project.
Why did you chose cursor pagination?
Details from the documentation:
https://www.django-rest-framework.org/api-guide/pagination/#cursorpagination
Cursor based pagination is more complex than other schemes. It also requires that the result set presents a fixed ordering, and does not allow the client to arbitrarily index into the result set. However it does provide the following benefits:
Provides a consistent pagination view. When used properly CursorPagination ensures that the client will never see the same item twice when paging through records, even when new items are being inserted by other clients during the pagination process.
Supports usage with very large datasets. With extremely large datasets pagination using offset-based pagination styles may become inefficient or unusable. Cursor based pagination schemes instead have fixed-time properties, and do not slow down as the dataset size increases.
Further, if check the implementation of the cursor pagination you'll see:
...
assert '__' not in ordering, (
'Cursor pagination does not support double underscore lookups '
'for orderings. Orderings should be an unchanging, unique or '
'nearly-unique field on the model, such as "-created" or "pk".'
)
...
assert isinstance(ordering, (six.string_types, list, tuple)), (
'Invalid ordering. Expected string or tuple, but got {type}'.format(
type=type(ordering).__name__
)
This implementation specifically disallows other order options than simple field based ordering using a string.
If you don't have any super large datasets, you probably wanna chose another pagination implementation like Page
Basically you can always create your own paginator class and overwrite paginate_queryset and add custom sorting.
I am writing a web based music application and I want implement some feature that user can see most favor album in last week-month-year.
so this is my model :
class album(models.Model):
def get_weely_count():
...
def get_monthly_count():
...
def get_yearly_count():
...
class like(models.Model):
created_at = models.DateField()
albumID = models.ForeignKey(Album)
Now I want to receive albums that most liked in last week or last month or last year,I want done some thing like this(but I can not):
Album.objects.all().order_by('get_weekly_count')
can any one help me to fix it or give another approach to achieve that goal??
The order_by method translates into an SQL ORDER BY, therefore it works only with model fields, which correspond to table columns. It won't work if you intend to sort your elements by a model's method.
So, if you want to accomplish something like
Album.objects.all().order_by('get_weekly_count')
You'll have to do it the python way
sorted(Album.objects.all(), key=lambda x: x.get_weekly_count())
Performance-wise, this means you'll get your elements with a query and then you'll sort them with python (that's different from getting a sorted queryset in one shot).
Otherwise, if it's possible for you to turn get_weekly_count into raw SQL, you could use it with a Count() or an extra modifier, that would make order_by usable, i.e.:
Album.objects.all().extra(
select={'weekly_count': "<some SQL>"},
select_params=(<you params>,),
).order_by('weekly_count')
Have a look at https://docs.djangoproject.com/en/1.8/ref/models/querysets/#extra
According to the documentation, you should use:
from django.db.models import Count
like.objects.filter(created_at__gt=START_OF_MONTH, created_at__lt=Datetime.now()).values('albumID').annotate(count=Count('albumID')).order_by('count')
This will get results for you in single db query. For more details visit https://docs.djangoproject.com/en/dev/topics/db/aggregation/.
I'm curious if there's any way to do a query in Django that's not a "SELECT * FROM..." underneath. I'm trying to do a "SELECT DISTINCT columnName FROM ..." instead.
Specifically I have a model that looks like:
class ProductOrder(models.Model):
Product = models.CharField(max_length=20, promary_key=True)
Category = models.CharField(max_length=30)
Rank = models.IntegerField()
where the Rank is a rank within a Category. I'd like to be able to iterate over all the Categories doing some operation on each rank within that category.
I'd like to first get a list of all the categories in the system and then query for all products in that category and repeat until every category is processed.
I'd rather avoid raw SQL, but if I have to go there, that'd be fine. Though I've never coded raw SQL in Django/Python before.
One way to get the list of distinct column names from the database is to use distinct() in conjunction with values().
In your case you can do the following to get the names of distinct categories:
q = ProductOrder.objects.values('Category').distinct()
print q.query # See for yourself.
# The query would look something like
# SELECT DISTINCT "app_productorder"."category" FROM "app_productorder"
There are a couple of things to remember here. First, this will return a ValuesQuerySet which behaves differently from a QuerySet. When you access say, the first element of q (above) you'll get a dictionary, NOT an instance of ProductOrder.
Second, it would be a good idea to read the warning note in the docs about using distinct(). The above example will work but all combinations of distinct() and values() may not.
PS: it is a good idea to use lower case names for fields in a model. In your case this would mean rewriting your model as shown below:
class ProductOrder(models.Model):
product = models.CharField(max_length=20, primary_key=True)
category = models.CharField(max_length=30)
rank = models.IntegerField()
It's quite simple actually if you're using PostgreSQL, just use distinct(columns) (documentation).
Productorder.objects.all().distinct('category')
Note that this feature has been included in Django since 1.4
User order by with that field, and then do distinct.
ProductOrder.objects.order_by('category').values_list('category', flat=True).distinct()
The other answers are fine, but this is a little cleaner, in that it only gives the values like you would get from a DISTINCT query, without any cruft from Django.
>>> set(ProductOrder.objects.values_list('category', flat=True))
{u'category1', u'category2', u'category3', u'category4'}
or
>>> list(set(ProductOrder.objects.values_list('category', flat=True)))
[u'category1', u'category2', u'category3', u'category4']
And, it works without PostgreSQL.
This is less efficient than using a .distinct(), presuming that DISTINCT in your database is faster than a python set, but it's great for noodling around the shell.
Update:
This is answer is great for making queries in the Django shell during development. DO NOT use this solution in production unless you are absolutely certain that you will always have a trivially small number of results before set is applied. Otherwise, it's a terrible idea from a performance standpoint.
I have a simple model
title = models.CharField(max_length=250)
url = models.CharField(max_length=250)
title_french = models.CharField(max_length=250)
I want to order it via title_french, however when it orders in A-Z in this way, all the blank values are at the top. In the case ot blank values I display the English title.
So I get A-Z for the French titles, but at the top there is a load of English title unordered.
Any advice?
For your case, I think you should do the sorting in your python code (currently, as it is, the sorting is made in the database). It is not possible, imho, to do what you want in the db, at least without writing some sql by hand.
So the idea would be to do something like this in your view :
your_objects = list(YourObject.objects.filter(....))
your_objects.sort(key=lambda ob: ob.title_french if ob.title_french else ob.title)
As long as you sort small lists, this should not be a too problematic performance issue.
have you tried ordering by multiple fields (doc):
ordering = ('title_french', 'title')
specify both the columns, title_french and title in order_by
queryset.order_by('title_french', 'title')
title_french will be given first preference and if there are two entries with the same title_french then those two entries will be sorted by their title
Here is a way to order blank value last while only using the ORM:
from django.db.models import Case, When, Value
...
title_french_blank_last = Case(
When(title_french="", then=Value(1)),
default=Value(0)
)
...
queryset.order_by(title_french_blank_last, "title_french", "title")
Django has the option to order nulls_first and nulls_last, for details see the docs.
In your case it would be something like this (not tested):
MyModel.objects.order_by(Coalesce('title_french', 'title').asc(nulls_last=True))
You would still have to do some logic in Python to display the title when the french title is None.
I have a two models:
class Category(models.Model):
pass
class Item(models.Model):
cat = models.ForeignKey(Category)
I am trying to return all Categories for which all of that category's items belong to a given subset of item ids (fixed thanks). For example, all categories for which all of the items associated with that category have ids in the set [1,3,5].
How could this be done using Django's query syntax (as of 1.1 beta)? Ideally, all the work should be done in the database.
Category.objects.filter(item__id__in=[1, 3, 5])
Django creates the reverse relation ship on the model without the foreign key. You can filter on it by using its related name (usually just the model name lowercase but it can be manually overwritten), two underscores, and the field name you want to query on.
lets say you require all items to be in the following set:
allowable_items = set([1,3,4])
one bruteforce solution would be to check the item_set for every category as so:
categories_with_allowable_items = [
category for category in
Category.objects.all() if
set([item.id for item in category.item_set.all()]) <= allowable_items
]
but we don't really have to check all categories, as categories_with_allowable_items is always going to be a subset of the categories related to all items with ids in allowable_items... so that's all we have to check (and this should be faster):
categories_with_allowable_items = set([
item.category for item in
Item.objects.select_related('category').filter(pk__in=allowable_items) if
set([siblingitem.id for siblingitem in item.category.item_set.all()]) <= allowable_items
])
if performance isn't really an issue, then the latter of these two (if not the former) should be fine. if these are very large tables, you might have to come up with a more sophisticated solution. also if you're using a particularly old version of python remember that you'll have to import the sets module
I've played around with this a bit. If QuerySet.extra() accepted a "having" parameter I think it would be possible to do it in the ORM with a bit of raw SQL in the HAVING clause. But it doesn't, so I think you'd have to write the whole query in raw SQL if you want the database doing the work.
EDIT:
This is the query that gets you part way there:
from django.db.models import Count
Category.objects.annotate(num_items=Count('item')).filter(num_items=...)
The problem is that for the query to work, "..." needs to be a correlated subquery that looks up, for each category, the number of its items in allowed_items. If .extra had a "having" argument, you'd do it like this:
Category.objects.annotate(num_items=Count('item')).extra(having="num_items=(SELECT COUNT(*) FROM app_item WHERE app_item.id in % AND app_item.cat_id = app_category.id)", having_params=[allowed_item_ids])