I have 3 models:
class Event(models.Model):
cur_datetime = models.DateTimeField(default=datetime.datetime(1970, 1, 1, 0, 0, 0, 0, pytz.UTC), blank=True, null=True)
week_no = models.IntegerField()
day_no = models.IntegerField()
class SubEvent(models.Model):
event = models.ForeignKey(Event, on_delete=models.CASCADE)
version = models.IntegerField()
class SubSubEvent(models.Model):
sub_event = models.ForeignKey(SubEvent, on_delete=models.CASCADE)
length = models.IntegerField()
I want to get a Queryset from SubSubEvent model, which includes all the Foreign keys as one single object. What I have now is:
querySet = SubSubEvent.objects.filter(sub_event__event__cur_datetime__range=[from_date, to_date])
This will return a queryset, and using a for loop to get __dict__ on each of objects, I get something like this:
{'event_id': 1, '_state': <django.db.models.base.ModelState object at 0x7fd7d9cefeb8>, 'id': 10, 'length': '1'}
This is just a part of the query I want to achieve. What I really want, is all the fields in event_id instead of just the id number. In other word, all the fields (including data) from Event plus SubEvent plus SubSubEvent in one queryset. This queryset should contains objects with cur_datetime, week_no, day_no, version and length.
It sounds like you're looking for select_related().
qs = SubSubEvent.objects \
.select_related('sub_event__event') \
.filter(sub_event__event__cur_datetime__range=[from_date, to_date])
You can then access the related SubEvent and Event resources without hitting the database.
sub_sub_event = qs[0]
sub_event = sub_sub_event.sub_event # doesn't hit the database
event = sub_sub_event.sub_event.event # doesn't hit the database
Related
i'm trying to call back unique constraints field , in my project i have to count number of M2M selected
class Booking(models.Model):
room_no = models.ForeignKey(Room,on_delete=models.CASCADE,blank=True,related_name='rooms')
takes_by = models.ManyToManyField(Vistor)
#property
def no_persons(self):
qnt = Booking.objects.filter(takes_by__full_information=self).count()#but this doesnt work
return qnt
Cannot query "some room information": Must be "Vistor" instance.
class Vistor(models.Model):
full_name = models.CharField(max_length=150)
dob = models.DateField(max_length=14)
city = models.ForeignKey(City,on_delete=models.CASCADE)
class Meta:
constraints = [
models.UniqueConstraint(fields=['full_name','dob','city'],name='full_information')
]
def __str__(self):
return f'{self.full_name} - {self.city} - {self.dob}'
it it possible to access full_information through Booking model ? thank you ..
If you want to count the number of Visitors related to that booking, you can count these with:
#property
def no_persons(self):
self.taken_by.count()
This will make an extra query to the database, therefore it is often better to let the database count these in the query. You can thus remove the property, and query with:
from django.db.models import Count
Booking.objects.annotate(
no_persons=Count('takes_by')
)
The Bookings that arise from this QuerySet will have an extra attribute no_persons with the number of related Visitors.
I have a History model like below
class History(models.Model):
class Meta:
app_label = 'subscription'
ordering = ['-start_datetime']
subscription = models.ForeignKey(Subscription, related_name='history')
FREE = 'free'
Premium = 'premium'
SUBSCRIPTION_TYPE_CHOICES = ((FREE, 'Free'), (Premium, 'Premium'),)
name = models.CharField(max_length=32, choices=SUBSCRIPTION_TYPE_CHOICES, default=FREE)
start_datetime = models.DateTimeField(db_index=True)
end_datetime = models.DateTimeField(db_index=True, blank=True, null=True)
cancelled_datetime = models.DateTimeField(blank=True, null=True)
Now i have a queryset filtering like below
users = get_user_model().objects.all()
queryset = users.exclude(subscription__history__end_datetime__lt=timezone.now())
The issue is that in the exclude above it is checking end_datetime for all the rows for a particular history object. But i only want to compare it with first row of history object.
Below is how a particular history object looks like. So i want to write a queryset filter which can do datetime comparison on first row only.
You could use a Model Manager method for this. The documentation isn't all that descriptive, but you could do something along the lines of:
class SubscriptionManager(models.Manager):
def my_filter(self):
# You'd want to make this a smaller query most likely
subscriptions = Subscription.objects.all()
results = []
for subscription in subscriptions:
sub_history = subscription.history_set.first()
if sub_history.end_datetime > timezone.now:
results.append(subscription)
return results
class History(models.Model):
subscription = models.ForeignKey(Subscription)
end_datetime = models.DateTimeField(db_index=True, blank=True, null=True)
objects = SubscriptionManager()
Then: queryset = Subscription.objects().my_filter()
Not a copy-pastable answer, but shows the use of Managers. Given the specificity of what you're looking for, I don't think there's a way to get it just via the plain filter() and exclude().
Without knowing what your end goal here is, it's hard to say whether this is feasible, but have you considered adding a property to the subscription model that indicates whatever you're looking for? For example, if you're trying to get everyone who has a subscription that's ending:
class Subscription(models.Model):
#property
def ending(self):
if self.end_datetime > timezone.now:
return True
else:
return False
Then in your code: queryset = users.filter(subscription_ending=True)
I have tried django's all king of expressions(aggregate, query, conditional) but was unable to solve the problem so i went with RawSQL and it solved the problem.
I have used the below SQL to select the first row and then compare the end_datetime
SELECT (end_datetime > %s OR end_datetime IS NULL) AS result
FROM subscription_history
ORDER BY start_datetime DESC
LIMIT 1;
I will select my answer as accepted if not found a solution with queryset filter chaining in next 2 days.
I am trying to filter in view my queryset based on relation between 2 fields .
however always getting the error that my field is not defined .
My Model has several calculated columns and I want to get only the records where values of field A are greater than field B.
So this is my model
class Material(models.Model):
version = IntegerVersionField( )
code = models.CharField(max_length=30)
name = models.CharField(max_length=30)
min_quantity = models.DecimalField(max_digits=19, decimal_places=10)
max_quantity = models.DecimalField(max_digits=19, decimal_places=10)
is_active = models.BooleanField(default=True)
def _get_totalinventory(self):
from inventory.models import Inventory
return Inventory.objects.filter(warehouse_Bin__material_UOM__UOM__material=self.id, is_active = true ).aggregate(Sum('quantity'))
totalinventory = property(_get_totalinventory)
def _get_totalpo(self):
from purchase.models import POmaterial
return POmaterial.objects.filter(material=self.id, is_active = true).aggregate(Sum('quantity'))
totalpo = property(_get_totalpo)
def _get_totalso(self):
from sales.models import SOproduct
return SOproduct.objects.filter(product__material=self.id , is_active=true ).aggregate(Sum('quantity'))
totalso = property(_get_totalpo)
#property
def _get_total(self):
return (totalinventory + totalpo - totalso)
total = property(_get_total)
And this is line in my view where I try to get the conditional queryset
po_list = MaterialFilter(request.GET, queryset=Material.objects.filter( total__lte = min_quantity ))
But I am getting the error that min_quantity is not defined
What could be the problem ?
EDIT:
My problem got solved thank you #Moses Koledoye but in the same code I have different issue now
Cannot resolve keyword 'total' into field.Choices are: am, author, author_id, bom, bomversion, code, creation_time, description, id, inventory, is_active, is_production, itemgroup, itemgroup_id, keywords, materialuom, max_quantity, min_quantity, name, pomaterial, produce, product, slug, trigger_quantity, uom, updated_by, updated_by_id, valid_from, valid_to, version, warehousebin
Basically it doesn't show any of my calculated fields I have in my model.
Django cannot write a query which is conditioned on a field whose value is unknown. You need to use a F expression for this:
from django.db.models import F
queryset = Material.objects.filter(total__lte = F('min_quantity'))
And your FilterSet becomes:
po_list = MaterialFilter(request.GET, queryset = Material.objects.filter(total__lte=F('min_quantity')))
From the docs:
An F() object represents the value of a model field or annotated
column. It makes it possible to refer to model field values and
perform database operations using them without actually having to pull
them out of the database into Python memory
I have this model in my code:
class Conversation(models.Model):
participants = models.ManyToManyField(User, related_name="message_participants")
and I need to filter this "Conversation" model objects by the "participants" many-to-many field.
meaning: I have for example 3 User objects, so I want to retrieve the only "Conversation" objects that has this 3 Users in it's "participants" field.
I tried doing this:
def get_exist_conv_or_none(sender,recipients):
conv = Conversation.objects.filter(participants=sender)
for rec in recipients:
conv = conv.filter(participants=rec)
where sender is a User object and "recipients" is a list of User objects.
it won't raise error but it gives me the wrong Object of Conversation.
Thanks.
edit:
A more recent try lead me to this:
def get_exist_conv_or_none(sender,recipients):
participants=recipients
participants.append(sender)
conv = Conversation.objects.filter(participants__in=participants)
return conv
which basically have the same problem. It yields Objects which has one or more of the "participants" on the list. but what Im looking for is exact match of the many-to-many object.
Meaning, an Object with the exact "Users" on it's many-to-many relation.
edit 2: My last attempt. still, won't work.
def get_exist_conv_or_none(sender,recipients):
recipients.append(sender)
recipients = list(set(recipients))
conv = Conversation.objects.annotate(count=Count('participants')).filter(participants=recipients[0])
for participant in recipients[1:]:
conv.filter(participants=participant)
conv.filter(count=len(recipients))
return conv
Ok so I found the answer:
In order to make an exact match I have to chain-filter the model and then make sure it has the exact number of arguments it needs to have, so that the many-to-many field will have in it all the objects needed and no more.
I will check for the objects number using annotation: ( https://docs.djangoproject.com/en/dev/topics/db/aggregation/ )
ended up with this code:
def get_exist_conv_or_none(recipients):
conv = Conversation.objects.annotate(count=Count('participants')).filter(participants=recipients[0])
for participant in recipients[1:]:
conv = conv.filter(participants=participant)
conv = conv.filter(count=len(recipients))
return conv
For fast search using database index, I use this code:
class YandexWordstatQueue(models.Model):
regions = models.ManyToManyField(YandexRegion)
regions_cached = models.CharField(max_length=10000, editable=False, db_index=True)
phrase = models.ForeignKey(SearchPhrase, db_index=True)
tstamp = models.DateTimeField(auto_now_add=True)
class YandexWordstatRecord(models.Model):
regions = models.ManyToManyField(YandexRegion)
regions_cached = models.CharField(max_length=10000, editable=False, db_index=True)
phrase = models.ForeignKey(SearchPhrase, db_index=True)
Shows = models.IntegerField()
date = models.DateField(auto_now_add=True)
#receiver(m2m_changed, sender=YandexWordstatRecord.regions.through)
#receiver(m2m_changed, sender=YandexWordstatQueue.regions.through)
def yandexwordstat_regions_changed(sender, **kwargs):
if kwargs.get('action') in ['post_add', 'post_remove']:
instance = kwargs.get('instance')
l = list(instance.regions.values_list('RegionID', flat=True))
l.sort()
instance.regions_cached = json.dumps(l)
instance.save()
This adds overhead when saving, but now I can perform fast filter with this snippet:
region_ids = [1, 2, 3] # or list(some_queryset.values_list(...))
region_ids.sort()
regions_cahed = json.dumps(region_ids)
YandexWordstatQueue.objects.filter(regions_cached=regions_cached)
EDIT:
It turns out the real question is - how do I get select_related to follow the m2m relationships I have defined? Those are the ones that are taxing my system. Any ideas?
I have two classes for my django app. The first (Item class) describes an item along with some functions that return information about the item. The second class (Itemlist class) takes a list of these items and then does some processing on them to return different values. The problem I'm having is that returning a list of items from Itemlist is taking a ton of queries, and I'm not sure where they're coming from.
class Item(models.Model):
# for archiving purposes
archive_id = models.IntegerField()
users = models.ManyToManyField(User, through='User_item_rel',
related_name='users_set')
# for many to one relationship (tags)
tag = models.ForeignKey(Tag)
sub_tag = models.CharField(default='',max_length=40)
name = models.CharField(max_length=40)
purch_date = models.DateField(default=datetime.datetime.now())
date_edited = models.DateTimeField(auto_now_add=True)
price = models.DecimalField(max_digits=6, decimal_places=2)
buyer = models.ManyToManyField(User, through='Buyer_item_rel',
related_name='buyers_set')
comments = models.CharField(default='',max_length=400)
house_id = models.IntegerField()
class Meta:
ordering = ['-purch_date']
def shortDisplayBuyers(self):
if len(self.buyer_item_rel_set.all()) != 1:
return "multiple buyers"
else:
return self.buyer_item_rel_set.all()[0].buyer.name
def listBuyers(self):
return self.buyer_item_rel_set.all()
def listUsers(self):
return self.user_item_rel_set.all()
def tag_name(self):
return self.tag
def sub_tag_name(self):
return self.sub_tag
def __unicode__(self):
return self.name
and the second class:
class Item_list:
def __init__(self, list = None, house_id = None, user_id = None,
archive_id = None, houseMode = 0):
self.list = list
self.house_id = house_id
self.uid = int(user_id)
self.archive_id = archive_id
self.gen_balancing_transactions()
self.houseMode = houseMode
def ret_list(self):
return self.list
So after I construct Itemlist with a large list of items, Itemlist.ret_list() takes up to 800 queries for 25 items. What can I do to fix this?
Try using select_related
As per a question I asked here
Dan is right in telling you to use select_related.
select_related can be read about here.
What it does is return in the same query data for the main object in your queryset and the model or fields specified in the select_related clause.
So, instead of a query like:
select * from item
followed by several queries like this every time you access one of the item_list objects:
select * from item_list where item_id = <one of the items for the query above>
the ORM will generate a query like:
select item.*, item_list.*
from item a join item_list b
where item a.id = b.item_id
In other words: it will hit the database once for all the data.
You probably want to use prefetch_related
Works similarly to select_related, but can deal with relations selected_related cannot. The join happens in python, but I've found it to be more efficient for this kind of work than the large # of queries.
Related reading on the subject