Query for a ManytoMany Field with Through in Django - django

I have a models in Django that are something like this:
class Classification(models.Model):
name = models.CharField(choices=class_choices)
...
class Activity(models.Model):
name = models.CharField(max_length=300)
fee = models.ManyToManyField(Classification, through='Fee')
...
class Fee(models.Model):
activity = models.ForeignKey(Activity)
class = models.ForeignKey(Classification)
early_fee = models.IntegerField(decimal_places=2, max_digits=10)
regular_fee = models.IntegerField(decimal_places=2, max_digits=10)
The idea being that there will be a set of fees associated with each Activity and Classification pair. Classification is like Student, Staff, etc.
I know that part works right.
Then in my application, I query for a set of Activities with:
activities = Activity.objects.filter(...)
Which returns a list of activities. I need to display in my template that list of Activities with their Fees. Something like this:
Activity Name
Student Early Price - $4
Student Regular Price - $5
Staff Early Price - $6
Staff Regular Price - $8
But I don't know of an easy way to get this info without a specific get query of the Fees object for each activity/class pair.
I hoped this would work:
activity.fee.all()
But that just returns the Classification Object. Is there a way to get the Fee Object Data for the Pair via the Activities I already queried?
Or am I doing this completely wrong?

Considering michuk's tip to rename "fee" to "classification":
Default name for Fee objects on Activity model will be fee_set. So in order to get your prices, do this:
for a in Activity.objects.all():
a.fee_set.all() #gets you all fees for activity
There's one thing though, as you can see you'll end up doing 1 SELECT on each activity object for fees, there are some apps that can help with that, for example, django-batch-select does only 2 queries in this case.

First of all I think you named your field wrong. This:
fee = models.ManyToManyField(Classification, through='Fee')
should be rather that:
classifications = models.ManyToManyField(Classification, through='Fee')
as ManyToManyField refers to a list of related objects.
In general ManyToManyField, AFAIK, is only a django shortcut to enable easy fetching of all related objects (Classification in your case), with the association table being transparent to the model. What you want is the association table (Fee in your case) not being transparent.
So what I would do is to remove the ManyToManyField field from Activity and simply get all the fees related with the activity. And thenm if you need a Classification for each fee, get the Classification separately.

Related

how to build a model on top of the other on django

I have two models
Job Position: It's the first object I create and represent a position in a company. `
Job offer: It is related to Job Position through a foreign key and represent a job offer sent out for a specific Job Position
Each job position can have multiple job offers associated as you can send multiple job offers out.
Now, the Job Position model has some data such as "Salary" that I would like each Job Offer instance to inherit automatically BUT I would also like to be able to change at the Job Offer level (after negotiations).
For example:
A jobposition is open with salary=50k
10 joboffers are sent out, each with salary 50k
After some negotiations, some joboffers are changed to salary 80k
Should joboffers have a salary field that I simply set equal to the salary jobposition when I create the object, or should I find a way to tie it initially within the model definition through some sort of foreign key but then make it flexibile?
class JobPosition(models.Model):
..
salary=models.Integer()
class JobOffer(models.Model):
position=models.ForeignKey(JobPosition, on_delete=models.CASCADE)
...
Your JobOffer models should have another field named salary as you said sometimes salary value can be different from JobPosition`
or you can some something like this:
class JobPosition(models.Model):
..
salary=models.Integer()
class JobOffer(models.Model):
position=models.ForeignKey(JobPosition, on_delete=models.CASCADE)
salary=models.Integer()
def job_position_salary(self):
return self.position.salary
...

Django: Joining on fields other than IDs (Using a date field in one model to pull data from a second model)

I'm attempting to use Django to build a simple website. I have a set of blog posts that have a date field attached to indicate the day they were published. I have a table that contains a list of dates and temperatures. On each post, I would like to display the temperature on the day it was published.
The two models are as follows:
class Post(models.Model):
title = models.CharField(max_length=200)
text = models.TextField()
date = models.DateField()
class Temperature(models.Model):
date = models.DateField()
temperature = models.IntegerField()
I would like to be able to reference the temperature field from the second table using the date field from the first. Is this possible?
In SQL, this is a simple query. I would do the following:
Select temperature from Temperature t join Post p on t.date = p.date
I think I really have two questions:
Is it possible to brute force this, even if it's not best practice? I've googled a lot and tried using raw sql and objects.extra, but can't get them to do what I want. I'm also wary of relying on them for the long haul.
Since this seems to be a simple task, it seems likely that I'm overcomplicating it by having my models set up sub-optimally. Is there something I'm missing about how I should design my models? That is, what's the best practice for doing something like this? (I've successfully pulled the temperature into my blog post by using a foreign key in the Temperature model. But if I go that route, I don't see how I could easily make sure that my temperature dates get the correct foreign key assigned to them so that the temperature date maps to the correct post date.)
There will likely be better answers than this one, but I'll throw in my 2ยข anyway.
You could try a property inside the Post model that returns the temperature:
#property
def temperature(self):
try:
return Temperature.objects.values_list('temperature',flat=True).get(date=self.date)
except:
return None
(code not tested)
About your Models:
If you will be displaying the temperature in a Post list (a list of Posts with their temperatures), then maybe it will be simpler to code and a faster query to just add a temperature field to your Post model.
You can keep the Temperature model. Then:
Assuming you have the temperature data already present in you Temperature model at the time of Post instance creation, you can fill that new field in a custom save method.
If you get temperature data after Post creation, you cann fill in that new temperature field through a background job (maybe triggered by crontab or similar).
Sometimes database orthogonality (not repeating info in many tables) is not the best strategy. Just something to think about, depending on how often you will be querying the Post models and how simple you want to keep that query code.
I think this might be a basic approach to solve the problem
post_dates = Post.objects.all().values('date')
result_temprature = Temperature.objects.filter(date__in = post_dates).values('temperature')
Subqueries could be your friend here. Something like the following should work:
from django.db.models import OuterRef, Subquery
temps = Temperature.objects.filter(date=OuterRef('date'))
posts = Post.objects.annotate(temperature=Subquery(temps.values('temperature')[:1]))
for post in posts:
temperature = post.temperature
Then you can just iterate through posts and access the temperature off each post instance

Grouping Django model entries by day using its datetime field

I'm working with an Article like model that has a DateTimeField(auto_now_add=True) to capture the publication date (pub_date). This looks something like the following:
class Article(models.Model):
text = models.TextField()
pub_date = models.DateTimeField(auto_now_add=True)
I want to do a query that counts how many article posts or entries have been added per day. In other words, I want to query the entries and group them by day (and eventually month, hour, second, etc.). This would look something like the following in the SQLite shell:
select pub_date, count(id) from "myapp_article"
where id = 1
group by strftime("%d", pub_date)
;
Which returns something like:
2012-03-07 18:08:57.456761|5
2012-03-08 18:08:57.456761|9
2012-03-09 18:08:57.456761|1
I can't seem to figure out how to get that result from a Django QuerySet. I am aware of how to get a similar result using itertools.groupby, but that isn't possible in this situation (explanation to follow).
The end result of this query will be used in a graph showing the number of posts per day. I'm attempting to use the Django Chartit package to achieve this goal. Chartit puts a constraint on the data source (DataPool). The source must be a Model, Manager, or QuerySet, so using itertools.groupby is not an option as far as I can tell.
So the question is... How do I group or aggregate the entries by day and end up with a QuerySet object?
Create an extra field that only store date data(not time) and annotate with Count:
Article.objects.extra({'published':"date(pub_date)"}).values('published').annotate(count=Count('id'))
Result will be:
published,count
2012-03-07,5
2012-03-08,9
2012-03-09,1

Find the most recent rating for a user in a django queryset

I'm looking for a method to get the most recent rating for a specific Person for all Resources.
Currently I'm using a query like Rating.objects.filter(Person = person).order_by('-timestamp')
then passing that through a unique_everseen with a key on the resource and user attributes and then re-looking up with a Rating.objects.filter(id__in = uniquelist). Is there a more elegant way to do this with the django queryset functions?
These are the relevant models.
class Person(models.Model):
pass
class Resource(models.Model):
pass
class Rating(models.Model):
rating = models.IntegerField()
timestamp = models.DateField()
resource = models.ForeignKey('Resource')
user = models.ForeignKey('Person')
I need to keep all of the old Ratings around since other functions need to be able to keep a history of how things are 'changing'.
I am not 100% clear on what you are looking for here, do you want to find the most recent rating by a user for all the resources they have rated? If you can provide detail on what unique_everseen actually does it would help to clarify what you are looking for.
You could rather look from a resource perspective:
resources = Resource.objects.filter(rating__user=person).order_by('-rating__timestamp')
resource_rating = [(resource, resource.rating_set.filter(person=person).get_latest('timestamp')) for resource in resources]
You might be able to use Aggregate functions to get to the most recent record per resource, or some clever use of the Q object to limit the SQL requests (my example may save you some requests, and be more elegant but it is not as simple as what you could produce with a raw SQL request). In raw SQL you would be using an inner SELECT or a well executed GROUP BY to get the most recent rating, so mimicking that would be ideal.
You could also create a post_save signal hook and an 'active' or 'current' boolean field on your Rating model, which would iterate other ratings matching user/resource and set their 'active' field to False. i.e. the post_save hook would mark all other ratings as inactive for a user/resource using something like:
if instance.active:
for rating in Rating.objects.filter(user=instance.user,resource=instance.resource).exclude(id=instance.id):
rating.active=False
rating.save()
You could then do a simple query for:
Rating.objects.filter(user=person,active=True).order_by('-timestamp')
This would be the most economical of queries (even if you make the complicated group by/inner select in raw SQL you are doing a more complicated query than necessary). Using the boolean field also means you can provide 'step forward/step backwards'/'undo/redo' behavior for a user's ratings if that is relevant.

How to sort by annotated Count() in a related model in Django

I'm building a food logging database in Django and I've got a query related problem.
I've set up my models to include (among other things) a Food model connected to the User model through an M2M-field "consumer" via the Consumption model. The Food model describes food dishes and the Consumption model describes a user's consumption of Food (date, amount, etc).
class Food(models.Model):
food_name = models.CharField(max_length=30)
consumer = models.ManyToManyField("User", through=Consumption)
class Consumption(models.Model):
food = models.ForeignKey("Food")
user = models.ForeignKey("User")
I want to create a query that returns all Food objects ordered by the number of times that Food object appears in the Consumption table for that user (the number of times the user has consumed the food).
I'm trying something in the line of:
Food.objects.all().annotate(consumption_times = Count(consumer)).order_by('consumption_times')`
But this will of course count all Consumption objects related to the Food object, not just the ones associated with the user. Do I need to change my models or am I just missing something obvious in the queries?
This is a pretty time-critical operation (among other things, it's used to fill an Autocomplete field in the Frontend) and the Food table has a couple of thousand entries, so I'd rather do the sorting in the database end, rather than doing the brute force method and iterate over the results doing:
Consumption.objects.filter(food=food, user=user).count()
and then using python sort to sort them. I don't think that method would scale very well as the user base increases and I want to design the database as future proof as I can from the start.
Any ideas?
Perhaps something like this?
Food.objects.filter(consumer__user=user)\
.annotate(consumption_times=Count('consumer'))\
.order_by('consumption_times')
I am having a very similar issue. Basically, I know that the SQL query you want is:
SELECT food.*, COUNT(IF(consumption.user_id=123,TRUE,NULL)) AS consumption_times
FROM food LEFT JOIN consumption ON (food.id=consumption.food_id)
ORDER BY consumption_times;
What I wish is that you could mix aggregate functions and F expression, annotate F expressions without an aggregate function, have a richer set of operations/functions for F expressions, and have virtual fields that are basically an automatic F expression annotation. So that you could do:
Food.objects.annotate(consumption_times=Count(If(F('consumer')==user,True,None)))\
.order_by('consumtion_times')
Also, just being able more easily able to add your own complex aggregate functions would be nice, but in the meantime, here's a hack that adds an aggregate function to do this.
from django.db.models import aggregates,sql
class CountIf(sql.aggregates.Count):
sql_template = '%(function)s(IF(%(field)s=%(equals)s,TRUE,NULL))'
sql.aggregates.CountIf = CountIf
consumption_times = aggregates.Count('consumer',equals=user.id)
consumption_times.name = 'CountIf'
rows = Food.objects.annotate(consumption_times=consumption_times)\
.order_by('consumption_times')