My example:
class Product(models.Model):
name = models.CharField(max_length=50)
category = models.ManyToManyField("wms.ProductCategory", blank=True)
#property
def quantity_in(self):
return self.intodocumentproduct_set.aggregate(total=Sum('quantity_in', default=0))['total']
class IntoDocumentProduct(models.Model):
product = models.ForeignKey("wms.Product", on_delete=models.CASCADE)
quantity_in = models.FloatField(blank=True, null=True)
class ProductListAPIView(ListAPIView):
# queryset = Product.objects.prefetch_related('category').annotate(sum_quantity_in=Sum('intodocumentproduct__quantity_in', default=0)).all()
queryset = Product.objects.prefetch_related('category').all()
serializer_class = ProductModelSerializer
Commented queryset results in 4 queries while other query (using property) results in 6 queries. Probably n+ problem.
I have to use properties like: quantity_ordered,
quantity_reserved,
quantity_out,
quantity_in,
quantity_stock,
quantity_available,
quantity_pending_release and more in many places in web app. Calculating them in every view will be time comsuming and error susceptible.
This solution is not very handy when property is used in many views with many properties.
Is there any other solution to remove extra queries?
Related
I really don't understand all the ways to build the right query.
I have the following models in the code i'm working on. I can't change models.
models/FollowUp:
class FollowUp(BaseModel):
name = models.CharField(max_length=256)
questions = models.ManyToManyField(Question, blank=True, )
models/Survey:
class Survey(BaseModel):
name = models.CharField(max_length=256)
followup = models.ManyToManyField(
FollowUp, blank=True, help_text='questionnaires')
user = models.ManyToManyField(User, blank=True, through='SurveyStatus')
models/SurveyStatus:
class SurveyStatus(models.Model):
user = models.ForeignKey(User, on_delete=models.CASCADE)
survey = models.ForeignKey(Survey, on_delete=models.CASCADE)
survey_status = models.CharField(max_length=10,
blank=True,
null=True,
choices=STATUS_SURVEY_CHOICES,
)
models/UserSurvey:
class UserSurvey(BaseModel):
user = models.ForeignKey(User, null=True, blank=True,
on_delete=models.DO_NOTHING)
followups = models.ManyToManyField(FollowUp, blank=True)
surveys = models.ManyToManyField(Survey, blank=True)
questions = models.ManyToManyField(Question, blank=True)
#classmethod
def create(cls, user_id):
user = User.objects.filter(pk=user_id).first()
cu_quest = cls(user=user)
cu_quest.save()
cu_quest._get_all_active_surveys
cu_quest._get_all_followups()
cu_quest._get_all_questions()
return cu_quest
def _get_all_questions(self):
[[self.questions.add(ques) for ques in qstnr.questions.all()]
for qstnr in self.followups.all()]
return
def _get_all_followups(self):
queryset = FollowUp.objects.filter(survey__user=self.user).filter(survey__user__surveystatus_survey_status='active')
# queryset = self._get_all_active_surveys()
[self.followups.add(quest) for quest in queryset]
return
#property
def _get_all_active_surveys(self):
queryset = Survey.objects.filter(user=self.user,
surveystatus__survey_status='active')
[self.surveys.add(quest) for quest in queryset]
return
Now my questions:
my view sends to the create of the UserSurvey model in order to create a questionary.
I need to get all the questions of the followup of the surveys with a survey_status = 'active' for the user (the one who clicks on a button)...
I tried several things:
I wrote the _get_all_active_surveys() function and there I get all the surveys that are with a survey_status = 'active' and then the _get_all_followups() function needs to call it to use the result to build its own one. I have an issue telling me that
a list is not a callable object.
I tried to write directly the right query in _get_all_followups() with
queryset = FollowUp.objects.filter(survey__user=self.user).filter(survey__user__surveystatus_survey_status='active')
but I don't succeed to manage all the M2M relationships. I wrote the query above but issue also
Related Field got invalid lookup: surveystatus_survey_status
i read that a related_name can help to build reverse query but i don't understand why?
it's the first time i see return empty and what it needs to return above. Why this notation?
If you have clear explanations (more than the doc) I will very appreciate.
thanks
Quite a few things to answer here, I've put them into a list:
Your _get_all_active_surveys has the #property decorator but neither of the other two methods do? It isn't actually a property so I would remove it.
You are using a list comprehension to add your queryset objects to the m2m field, this is unnecessary as you don't actually want a list object and can be rewritten as e.g. self.surveys.add(*queryset)
You can comma-separate filter expressions as .filter(expression1, expression2) rather than .filter(expression1).filter(expression2).
You are missing an underscore in surveystatus_survey_status it should be surveystatus__survey_status.
Related name is just another way of reverse-accessing relationships, it doesn't actually change how the relationship exists - by default Django will do something like ModelA.modelb_set.all() - you can do reverse_name="my_model_bs" and then ModelA.my_model_bs.all()
I am struggling with a queryset in a Django view. Basically, I have three models: User, ActivityLog, & ActivityCategory.
User is the built-in.
ActivityLog looks like this:
class ActivityLog(models.Model):
activity_datetime = models.DateTimeField(default=timezone.now)
user = models.ForeignKey(User, on_delete=models.CASCADE, null=True, related_name='activity_user')
activity_category = models.ForeignKey(ActivityCategory, on_delete=models.CASCADE, null=True, related_name='activity_cat')
activity_description = models.CharField(max_length=100, default="Misc Activity")
Activity Category:
class ActivityCategory(models.Model):
activity_name = models.CharField(max_length=40)
activity_description = models.CharField(max_length=150)
pts = models.IntegerField()
My goal is to return an aggregate by user of all the points they have accumulated by participating in activities in the log. Each log references an ActivityType, different types are worth different points.
I accomplished this with the following query in the view:
class UserScoresAPIView(generics.ListAPIView):
queryset = User.objects.all()
serializer_class = UserScoresSerializer
def get_queryset(self):
queryset = User.objects.annotate(total_pts=Coalesce(Sum('activity_user__activity_category__pts'), 0)).order_by('-total_pts')
return queryset
So now I need to add to this query and restrict it based on date of the activity. I want to basically add a filter:
.filter('activity_user__activity_datetime__gte=datetime.date(2020,10,1)')
How can I add this into my current query to accomplish this? I tried to do so here:
queryset = User.objects.annotate(total_pts=Coalesce(Sum('activity_user__activity_category__pts').filter('activity_user__activity_datetime__gte=datetime.date(2020,10,1)') , 0)).order_by('-total_pts')
But that would happen after the Sum and wouldn't be helpful (or work...) so I tried to chain it where it is pulling the User objects
User.objects.filter('activity_user__activity_datetime__gte=datetime.date(2020,10,1)').annotate(total_pts=Coalesce(Sum('activity_user__activity_category__pts'), 0)).order_by('-total_pts')
But now I am receiving an error when trying to parse my query:
ValueError: too many values to unpack (expected 2)
I am confused at where to go next and appreciate any guidance.
Thank you.
BCBB
Aaaaaaand I got so focused on how to chain these together that I thought I was doing it wrong but in reality, I am just not sure what possessed me to enclose the filter in quotes...
Arrrrg. It's working now as listed last without the quotes...
User.objects.filter(activity_user__activity_datetime__gte=datetime.date(2020,10,1)).annotate(total_pts=Coalesce(Sum('activity_user__activity_category__pts'), 0)).order_by('-total_pts')
This question already has an answer here:
How to perform a join and aggregate count in Django
(1 answer)
Closed 4 years ago.
I am trying to send sub model count information with the main model to HTML template. I have PROJECT and Companies models:
class Projects(models.Model):
name = models.CharField(max_length=255)
note = models.CharField(max_length=255, default='')
def __str__(self):
return self.name
class Companies(models.Model):
project = models.ForeignKey(Projects, on_delete=models.CASCADE)
name = models.CharField(max_length=255)
note = models.CharField(max_length=255, default='')
def __str__(self):
return self.name
I need to show, projects have how many companies(company count):
Project name | company.no
project1 | 3
project2 | 5
Method-1 : Using a GROUP BY statement using Django ORM
from django.db.models import Count
Projects.objects.values('name').annotate(count=Count('companies'))
This is equivalant to the SQL query, SELECT name,Count('companies') as count from Projects. This will return a QuerySet as,
<QuerySet [{'name': 'project_name_1', 'count': 10}, {'name': 'project_name_2', 'count': 6}]>
Method-2 : Use #property decorator (as #Kirollos Morkos mentioned )
Hence your model became
class Projects(models.Model):
name = models.CharField(max_length=255)
note = models.CharField(max_length=255, default='')
#property
def company_count(self):
return self.companies_set.count()
def __str__(self):
return self.name
Then you'll get the comapny count in template by {{ project_obj.company_count}}
NOTE: The Method-1 is more efficent and fast, because it's being done in database level
You can use the property decorator to provider an arbitrary getter on your Project model. In your Company model, add a related_name to the project foreign key so you can get the reverse relationship:
project = models.ForeignKey(Projects, on_delete=models.CASCADE, related_name='companies')
Then, you can add a property on your Project model to get the company count:
class Projects(models.Model):
#property
def company_count(self):
return self.companies.count()
...
# Following the sample counts you gave
print(project1.company_count) # 3
print(project2.company_count) # 5
As JPG says you can use annotate. you dont need to add values queryset. see below:
from django.db.models import Count
Projects.objects.filter(<any filter apply to queryset>).annotate(company_count=Count('companies'))
you can add any field to annotation. then in your template or any code you can use this count by calling:
project.company_count
you also can add in your template
{{ project.companies_set.count }}
to get count of companies. but this method is too slow. because for each project record you have a database hit.
My Django models look like this:
class User(models.Model):
userid = models.CharField(max_length=26,unique=True)
#some more fields that are currently not relevant
class Followers(models.Model):
user = models.ForeignKey('User',related_name='usr')
coins = models.IntegerField()
followers = models.CharField(max_length=26, null=True, blank=True)
I would now like to make a filter query in my Followers table selecting every entry where users have ID x and followers have ID y (I expect to get one result from the query).
To visualize what I have tried and know won't work is this:
queryfilter = Followers.object.filter(followers=fid, user=uid)
and this:
queryfilter = Followers.object.filter(followers=fid, user__userid=uid)
In the end I would like to access the coins:
c = queryfilter.coins
It may be possible that I cannot do it with one single query and need two, since I am trying to do a filter query with two tables involved.
Firstly, I have modified your 'Followers' model (for naming convention).
models.py
class Follower(models.Model):
user = models.ForeignKey('User', related_name='followers')
coins = models.IntegerField()
key = models.CharField(max_length=26, null=True, blank=True)
Your queryset should be ..
views.py
#coins
coins = Follower.objects.filter(key=fid, user__userid=uid).get().coins
I've the below inefficient 'destroy' method for deleting Ratings that are held in Stimulus which itself is held within Experiment (I have simplified my models, for reasons of clarity).
Could you advise on a more efficient way of achieving this?
class Rating(models.Model):
rater = TextField(null=True)
rating = FloatField(null=True)
created = models.DateTimeField(null=True)
class Stimulus(TimeStampedModel):
genes = TextField()
weights = ListField()
ratings = ManyToManyField(Rating, null=True)
evaluation = FloatField(null=True)
complete = BooleanField(default=False)
Class Experiment(models.Model):
all_individuals = ManyToManyField(Stimulus, null=True)
def destroy(self):
all_ratings = Rating.objects.all()
for ind in self.all_individuals.all():
ratings = ind.ratings.all()
for rating in ratings:
if rating in all_ratings:
Rating.objects.filter(id = rating.id).delete()
Background: I am using Django to run an experiment (Experiment) which shows Users many Stimuli (Stimulus). Each Stimulus gets rated many times. Thus, I need to save multiple ratings per stimulus (and multiple stimuli per experiment).
Some simple improvements
Remove the if rating in all_ratings, every rating will be in the list of all ratings
Do the delete on the database side
ind.ratings.all().delete()
Use prefetch_related to get the foreign key objects
self.all_individuals.prefetch_related('ratings'):
Combined would be:
def destroy(self):
for ind in self.all_individuals.prefetch_related('ratings'):
ratings = ind.ratings.all().delete()
I think that in this case using ManyToManyField isn't the best choice.
You'll have less problems using common ForeignKey's changing a little the structure of this models.
Eg.
class Rating(models.Model):
rater = TextField(null=True)
rating = FloatField(null=True)
created = models.DateTimeField(null=True)
stimulus = models.ForeignKey('Stimulus', related_name='ratings')
class Stimulus(TimeStampedModel):
genes = TextField()
weights = ListField()
#ratings = ManyToManyField(Rating, null=True)
evaluation = FloatField(null=True)
complete = BooleanField(default=False)
experiment = models.ForeignKey('Experiment', related_name='stimulus')
class Experiment(models.Model):
#all_individuals = ManyToManyField(Stimulus, null=True)
name = models.CharField(max_length=100)
This is a more clear structure and when you delete Experiment by, experiment_instance.delete() a delete cascade will delete all other related models.
Hope it helps.