In django rest framework i am using pagination of 20 on a model.
Now i want to make a queryset which takes 5 items which has the featured field as true.
The other 15 should be items which has the featured field as false.
Does anyone have an idea how to make this work?
permission_classes = [AllowAny]
serializer_class = JobSerializer
pagination_class = Pagination
order_types = {
1: 'job_order', #by randomized job order
2: '-created_date', #newest to oldest
3: 'created_date', #oldest to newest
4: 'name', #by name A -> Z,
5: '-name' #by name Z -> A,
}
def get_queryset(self):
requirements = self.request.query_params.getlist('requirements', None)
order_param = self.request.query_params.get('order_by', 1)
#make querysets
queryset_featured = Job.objects.all().filter(active=True, featured=True).order_by(self.order_types[int(order_param)])
queryset_non_featured = Job.objects.all().filter(active=True, featured=False).order_by(self.order_types[int(order_param)])
return queryset.distinct()
Use a union of two separate queries:
https://docs.djangoproject.com/en/3.1/ref/models/querysets/#union
This will allow you to make one ORM query for the featured items with [0:5], and then union to an ORM query for non-featured items with [0:15].
I haven't tested this, but here's an example with your code:
queryset_featured = Job.objects.all().filter(active=True, featured=True).order_by(self.order_types[int(order_param)])[0:5]
queryset_non_featured = Job.objects.all().filter(active=True, featured=False).order_by(self.order_types[int(order_param)])[0:15]
queryset_combined = queryset_featured.union(queryset_non_featured)
for row in queryset_combined:
print(row)
Related
I have a variable 'cptCodeTBX' which is not present as fields in django models. I need to apply filter on 'cptCodeTBX' variable. Something roughly equivalent to
cptCodeTBX = '00622'
select * from cpt where cpt.code like cptCodeTBX or cptCodeTBX is != ''
In dot net entity framework we could do it by
b = cptContext.CPTs.AsNoTracking().Where(
a =>
(String.IsNullOrEmpty(cptCodeTBX) || a.Code.StartsWith(cptCodeTBX))
This may not be the most performant solution, but I was able to get it working.
Step 1: Read the Django Filter docs.
https://django-filter.readthedocs.io/en/stable/index.html
Step 2: Add a property to your Django model named cptCodeTBX.
from django.db import models
class MyModel(models.Model):
field = models.CharField(max_length=60)
#property
def cptCodeTBX(self):
"""
Does all the code tbx cpt stuff, can do whatever you want.
"""
cptCodeTBX = 2323 #whatever value you want
return cptCodeTBX
Step 3: Add a Django Filter.
import django_filters
class CptCodeTBXFilter(django_filters.FilterSet):
"""
Filter that will do all the magic, value comes from url query params.
Replace filters.NumberFilter with any filter you want like
filters.RangeFilter.
"""
cptCodeTBX = django_filters.NumberFilter(
field_name="cptCodeTBX",
method="filter_cptCodeTBX",
label="Cpt Code TBX",
)
def filter_cptCodeTBX(self, queryset, name, value):
objects_ids = [
obj.pk for obj in MyModel.objects.all() if obj.cptCodeTBX == value
]
if objects_ids:
queryset = MyModel.objects.filter(pk__in=objects_ids)
else:
queryset = MyModel.objects.none()
return queryset
Step 4: Pass the value through the url as a query parameter.
http://example.com/?cptCodeTBX=00622
I'm trying to build a page that renders searched Oscar products and filters them by their Category using GET attributes. I'm overriding get_queryset and building my object list from there
class ProductSearchView(ListView):
model = Product
template_name = "productsearch/product_list.html"
queryset = Product.objects.none()
def get_queryset(self):
word_query_attr = self.request.GET.get('q', None) # word query
sqs = SearchQuerySet().models(Product).filter(Q(title__icontains=word_query_attr) |
Q(category__name__icontains=word_query_attr) |
Q(upc__icontains=word_query_attr) |
Q(description__icontains=word_query_attr))
qs = Product.objects.all()
if self.request.GET.get('cat', None):
cat_attr = self.request.GET.get('cat', None)
category = Category.objects.get(name=cat_attr)
qs = qs.filter(categories__in=category.get_children())
My question is, can I use SearchQuerySet() to filter through fields from result objects? (in this case, categories from Product objects)
If not, is there an effective way I can create a Product queryset using SearchQuerySet() results?
I've tried filtering through IDs
object_ids = [result.object.id for result in sqs]
qs = qs.filter(id__in=object_ids).distinct()
But there are two problems: it isn't scalable (as noted here) and some queries are very slow when I'm dealing with ~900 results.
I might be missing something, but the SearchQuerySet filter in your example is already restricting by category name? Could you just change that filter to exact instead of icontains and pass your cat GET param?
sqs = SearchQuerySet().models(Product).filter(
Q(title__icontains=word_query_attr) |
Q(upc__icontains=word_query_attr) |
Q(description__icontains=word_query_attr)
)
cat_attr = self.request.GET.get('cat')
if cat_attr:
sqs.filter(category__name__exact=cat_attr)
I think Haystack should let you chain filters like that.
I'm working on a django project with the following models.
class User(models.Model):
pass
class Item(models.Model):
user = models.ForeignKey(User)
item_id = models.IntegerField()
There are about 10 million items and 100 thousand users.
My goal is to override the default admin search that takes forever and
return all the matching users that own "all" of the specified item ids within a reasonable timeframe.
These are a couple of the tests I use to better illustrate my criteria.
class TestSearch(TestCase):
def search(self, searchterm):
"""A tuple is returned with the first element as the queryset"""
return do_admin_search(User.objects.all())
def test_return_matching_users(self):
user = User.objects.create()
Item.objects.create(item_id=12345, user=user)
Item.objects.create(item_id=67890, user=user)
result = self.search('12345 67890')
assert_equal(1, result[0].count())
assert_equal(user, result[0][0])
def test_exclude_users_that_do_not_match_1(self):
user = User.objects.create()
Item.objects.create(item_id=12345, user=user)
result = self.search('12345 67890')
assert_false(result[0].exists())
def test_exclude_users_that_do_not_match_2(self):
user = User.objects.create()
result = self.search('12345 67890')
assert_false(result[0].exists())
The following snippet is my best attempt using annotate that takes over 50 seconds.
def search_by_item_ids(queryset, item_ids):
params = {}
for i in item_ids:
cond = Case(When(item__item_id=i, then=True), output_field=BooleanField())
params['has_' + str(i)] = cond
queryset = queryset.annotate(**params)
params = {}
for i in item_ids:
params['has_' + str(i)] = True
queryset = queryset.filter(**params)
return queryset
Is there anything I can do to speed it up?
Here's some quick suggestions that should improve performance drastically.
Use prefetch_related` on the initial queryset to get related items
queryset = User.objects.filter(...).prefetch_related('user_set')
Filter with the __in operator instead of looping through a list of IDs
def search_by_item_ids(queryset, item_ids):
return queryset.filter(item__item_id__in=item_ids)
Don't annotate if it's already a condition of the query
Since you know that this queryset only consists of records with ids in the item_ids list, no need to write that per object.
Putting it all together
You can speed up what you are doing drastically just by calling -
queryset = User.objects.filter(
item__item_id__in=item_ids
).prefetch_related('user_set')
with only 2 db hits for the full query.
I want to filter my model object using two filters.
So, it can be only one filter or both or none.
My solution is to use a lot of 'if':
if _topic or _curator:
if _topic and _curator:
queryset = Article.objects.filter(topic=_topic,curator=_curator)
elif _curator:
queryset = Article.objects.filter(curator=_curator)
else # so topic is the last choice
queryset = Article.objects.filter(topic=_topic)
else
queryset = Article.objects.all()
Can someone suggest an easier way to filter?
Queryset filters are cumulative.
queryset = Article.objects.all()
if _topic:
queryset = queryset.filter(topic=_topic)
if _curator:
queryset = queryset.filter(curator=_curator)
kwargs = {}
if _topic:
kwargs[topic] = _topic
if _curator:
kwargs[curator] = _curator
queryset = Article.objects.filter(**kwargs)
EDIT:
It turns out the real question is - how do I get select_related to follow the m2m relationships I have defined? Those are the ones that are taxing my system. Any ideas?
I have two classes for my django app. The first (Item class) describes an item along with some functions that return information about the item. The second class (Itemlist class) takes a list of these items and then does some processing on them to return different values. The problem I'm having is that returning a list of items from Itemlist is taking a ton of queries, and I'm not sure where they're coming from.
class Item(models.Model):
# for archiving purposes
archive_id = models.IntegerField()
users = models.ManyToManyField(User, through='User_item_rel',
related_name='users_set')
# for many to one relationship (tags)
tag = models.ForeignKey(Tag)
sub_tag = models.CharField(default='',max_length=40)
name = models.CharField(max_length=40)
purch_date = models.DateField(default=datetime.datetime.now())
date_edited = models.DateTimeField(auto_now_add=True)
price = models.DecimalField(max_digits=6, decimal_places=2)
buyer = models.ManyToManyField(User, through='Buyer_item_rel',
related_name='buyers_set')
comments = models.CharField(default='',max_length=400)
house_id = models.IntegerField()
class Meta:
ordering = ['-purch_date']
def shortDisplayBuyers(self):
if len(self.buyer_item_rel_set.all()) != 1:
return "multiple buyers"
else:
return self.buyer_item_rel_set.all()[0].buyer.name
def listBuyers(self):
return self.buyer_item_rel_set.all()
def listUsers(self):
return self.user_item_rel_set.all()
def tag_name(self):
return self.tag
def sub_tag_name(self):
return self.sub_tag
def __unicode__(self):
return self.name
and the second class:
class Item_list:
def __init__(self, list = None, house_id = None, user_id = None,
archive_id = None, houseMode = 0):
self.list = list
self.house_id = house_id
self.uid = int(user_id)
self.archive_id = archive_id
self.gen_balancing_transactions()
self.houseMode = houseMode
def ret_list(self):
return self.list
So after I construct Itemlist with a large list of items, Itemlist.ret_list() takes up to 800 queries for 25 items. What can I do to fix this?
Try using select_related
As per a question I asked here
Dan is right in telling you to use select_related.
select_related can be read about here.
What it does is return in the same query data for the main object in your queryset and the model or fields specified in the select_related clause.
So, instead of a query like:
select * from item
followed by several queries like this every time you access one of the item_list objects:
select * from item_list where item_id = <one of the items for the query above>
the ORM will generate a query like:
select item.*, item_list.*
from item a join item_list b
where item a.id = b.item_id
In other words: it will hit the database once for all the data.
You probably want to use prefetch_related
Works similarly to select_related, but can deal with relations selected_related cannot. The join happens in python, but I've found it to be more efficient for this kind of work than the large # of queries.
Related reading on the subject