we have Project as main model, which contains 2 fields of M2M relationship.
class First(models.Model):
first_results_M2M = models.ManyToManyField(First_Results)
class Second(models.Model):
second_results_M2M = models.ManyToManyField(Second_Results)
class Project(models.Model):
project_first_M2M = models.ManyToManyField(First)
project_second_M2M = models.ManyToManyField(Second)
I m trying to count all the objects present in first_results_M2M of all the project_first_M2M objects within each Project object.
Here's the below example to count all the objects of first_results_M2M for Project object 1.
total_first_all = First_Results.objects.filter(first__project__id=1).count()
I want to render the total count of total_first_all and total_second_all in the template.
Project_Query = Project.objects.all()
for each_proj in Project_Query:
print(each_proj.total_first_all) ## should print the count the `first_resuls_M2M` for each project obj.
Please let me know how to do achieve it in more effecient/fast way besides annotate.
annotate.total_first_all=Count('project_first_M2M__first_results_M2M')
You .annotate(..) [Django-doc] your queryset, like:
from django.db.models import Count
project_query = Project.objects.annotate(
total_first_all=Count('project_first_M2M__first_results_M2M')
)
for project in project_query:
print(project.total_first_all)
This will not make a query per Project object, but calculate the counts for all Projects in "bulk".
For multiple ones, you can make use of subqueries to reduce the amount of nested JOIN:
from django.db.models import Count, OuterRef, Subquery
project_query = Project.objects.annotate(
total_first_all=Subquery(
First_Results.objects.filter(first__project=OuterRef('pk')).values('first__project').values(cnt=Count('*')).order_by('first__project')
),
total_second_all=Subquery(
Second_Results.objects.filter(second__project=OuterRef('pk')).values('second__project').values(cnt=Count('*')).order_by('second__project')
)
)
Related
How can I check whether a many-to-many relationship exists or another property is fulfilled? When I try the query, it returns some rows twice!
Given a model
from django.db import models
class Plug(models.Model):
name = models.CharField(primary_key=True, max_length=99)
class Widget(models.Model):
name = models.CharField(primary_key=True, max_length=99)
shiny = models.BooleanField()
compatible = models.ManyToManyField(Plug)
I have the following items in my database:
from django.db.models import Q
schuko = Plug.objects.create(name='F')
uk = Plug.objects.create(name='G')
Widget.objects.create(name='microwave', shiny=True).compatible.set([uk])
Widget.objects.create(name='oven', shiny=False).compatible.set([uk])
Widget.objects.create(name='pc', shiny=True).compatible.set([uk, schuko])
Now I want all names of widgets that are shiny and/or compatible with Schuko:
shiny_or_schuko = sorted(
Widget.objects.filter(Q(shiny=True) | Q(compatible=schuko))
.values_list('name', flat=True))
But to my surprise, this does not return ['microwave', 'pc']. Instead, 'pc' is listed twice, i.e. shiny_or_schuko is ['microwave', 'pc', 'pc'].
Is this a Django bug? If not, how can I set up the query that I get 'pc' just once?
Is this a Django bug?
No. You simply perform a LEFT OUTER JOIN with the many-to-many table. If two or more related objects match, it will be included multiple times. This can be wanted behavior, for example if you add extra annotations to the elements that takes values from these related objects.
You can make use of .distinct() [Django-doc] to return only distinct elements:
Widget.objects.filter(
Q(shiny=True) | Q(compatible=schuko)
).values_list('name', flat=True).distinct()
I have the following model, which has 3 classes Project,CrawlerProject,CrawlerResults
class CrawlerProject(models.Model):
user = models.ForeignKey(User,on_delete=models.SET_NULL,null=True)
cralwer_results_M2M = models.ManyToManyField(CrawlerResults,blank=True)
class Project(models.Model):
user = models.ForeignKey(User,on_delete=models.SET_NULL,null=True)
crawler_project_M2M = models.ManyToManyField(CrawlerProject,blank=True)
Here, i want to count the total number of CrawlerResults objects are present for all CrawlerProjects within individual Project object.
projects = Project.objects.all().prefetch_related('crawler_project_M2M')
for each_proj in projects:
total_num_of_crawler_results = each_proj.crawler_project_M2M__cralwer_results_M2M.count() ## count all the crawler_results objects of all crawler_project present in current `project` object.
How do I achieve in an efficient way(single query) to get the total count of nested ManyToMany class?
Try this:
from django.db.models import Count
CrawlerProjects =CrawlerProject.objects.all().annotate(CrawlerResults_count=Count('project'))
I have 4 models:
class App(models.Model):
...
class AppVersion(models.Model):
app = models.ForeignKey(App)
version_code = models.IntegerField()
class Meta:
ordering = ('-version_code',)
...
class Apk(models.Model):
version = models.OneToOneField(AppVersion)
size = models.IntegerField()
class Obb(models.Model):
version = models.ForeignKey(AppVersion)
size = models.IntegerField()
AppVersion version always has one Apk, but may have 0, 1 or 2 Obb's
I want to annotate QuerySet by total size of the App (which is Apk.size + sum of all Obb.size for given AppVersion).
My App QuerySet looks like this:
qs = App.objects.filter(is_visible=True)
and versions subquery is:
latest_versions = Subquery(AppVersion.objects.filter(application=OuterRef(OuterRef('pk'))).values('pk')[:1])
This subquery always gives the latest AppVersion of the App.
So what subquery should I use to annotate qs with size attribute calculated as shown above?
How about something like this - from my understanding you want an Apps apk, and obb sizes summed. apk and obb_set can be replaced by the fields related name if you added one. What I chose should be the defaults for a django OneToOne and Fk related name.
from django.db.models import F, Value, Sum, IntegerField
qs = App.objects.filter(
is_visible=True
).annotate(
apk_size=Sum('apk__size'),
obb_size=Sum('obb_set__size')
).annotate(
total_size=Value(
F('apk_size') + F('obb_size'),
output_field=IntegerField()
)
I'm trying to add an annotation to a QuerySet that is True/False when the value of a field on one related object is less than the value of a field on a different related object.
Here are some models for an example:
class RobotManager(models.Manager):
queryset = super(RobotManager, self).get_queryset()
queryset = queryset.annotate(canteen_empty=UNKNOWN CODE)
return queryset
class Robot(models.Model):
# Has some other unrelated stuff
objects = RobotManager()
class CanteenLevel(models.Model):
time = models.DateTimeField()
robot = models.ForeignKey("SomeApp.Robot")
gallons = models.IntegerField()
class RobotConfiguration(models.Model):
time = models.DateTimeField()
robot = models.ForeignKey("SomeApp.Robot")
canteen_empty_level = models.IntegerField()
With the above models, as the Robot's Configuration or CanteenLevel change, we create new records and save the historicals.
What I would like to do is add an annotation to a Robot QuerySet that states if the Robot's Canteen is considered empty (Robot's latest CanteenLevel.gallons is less than the Robot's latest Configuration.canteen_empty_level).
The aim is to allow for a statement like this using the annotation in the QuerySet:
bad_robots = Robot.objects.filter(canteen_empty=True)
I had tried something like this in the annotation:
canteen_empty=ExpressionWrapper(CanteenLevel.objects.filter(robot=OuterRef('pk')).order_by('-time').values('gallons')[:1] <= RobotConfiguration.objects.filter(robot=OuterRef('robot')).order_by('-time').values('canteen_empty_level')[:1], output_field=models.BooleanField))
But obviously the "<=" operator isn't allowed.
I also tried this:
canteen_empty=Exists(CanteenLevel.objects.filter(robot=OuterRef('pk')).order_by('-time').values('gallons')[:1].filter(gallons__lte=Subquery(RobotConfiguration.objects.filter(robot=OuterRef('robot')).order_by('-time').values('canteen_empty_level')[:1]))))
But you can't filter after taking a slice of a QuerySet.
Any help would be appreciated!
We can make two annotations here:
from django.db.models import Subquery, OuterRef
latest_gallons = Subquery(CanteenLevel.objects.filter(
robot=OuterRef('pk')
).order_by('-time').values('gallons')[:1])
latest_canteen = Subquery(RobotConfiguration.objects.filter(
robot=OuterRef('pk')
).order_by('-time').values('canteen_empty_level')[:1])
then we can first annotate the Robot objects with these, and filter:
from django.db.models import F
Robot.objects.annotate(
latest_gallons=latest_gallons,
latest_canteen=latest_canteen
).filter(latest_gallons__lte=F('latest_canteen'))
This will construct a query that looks like:
SELECT robot.*,
(SELECT U0.gallons
FROM canteenlevel U0
WHERE U0.robot_id = robot.id
ORDER BY U0.time DESC
LIMIT 1) AS latest_gallons,
(SELECT U0.canteen_empty_level
FROM robotconfiguration U0
WHERE U0.robot_id = robot.id
ORDER BY U0.time DESC
LIMIT 1) AS latest_canteen
FROM robot
WHERE
(SELECT U0.gallons
FROM canteenlevel U0
WHERE U0.robot_id = robot.id
ORDER BY U0.time DESC
LIMIT 1
) <= (
SELECT U0.canteen_empty_level
FROM robotconfiguration U0
WHERE U0.robot_id = robot.id
ORDER BY U0.time DESC
LIMIT 1
)
Note however that if a Robot has no related CanteenLevel, or RobotConfiguration (one of them, or both), then that Robot will not be included in the queryset.
I have two tables (similar to the ones below):
class Piece(models.Model):
cost = models.IntegerField(default=50)
piece = models.CharField(max_length=256)
class User_Piece (models.Model):
user = models.ForeignKey(User)
piece = models.ForeignKey(Piece)
I want to do a query that returns all items in Piece, but annotates each row with whether or not the logged in user owns that piece (so there exists a row in User_Piece where user is the logged in user).
I tried:
pieces = Piece.objects.annotate(owned=Count('user_piece__id'))
But it puts a count > 0 for any piece that is owned by any user. I'm not sure where/how I put in the condition that the user_piece must have the specified user I want. If I filter on user__piece__user=user, then I don't get all the rows from Piece, only those that are owned.
You could use Exist subquery wrapper:
from django.db.models import Exists, OuterRef
subquery = User_Piece.objects.filter(user=user, piece=OuterRef('pk'))
Piece.objects.annotate(owned=Exists(subquery))
https://docs.djangoproject.com/en/dev/ref/models/expressions/#exists-subqueries
In newer versions of Django, you can do:
from django.db.models import Exists, OuterRef
pieces = Piece.objects.annotate(
owned=Exists(UserPiece.objects.filter(piece=OuterRef('id'), user=request.user))
)
for piece in pieces:
print(piece.owned) # prints True or False
Of course, you can replace the name owned with any name you want.
Easy approach, be careful with performance:
pk_pices = ( User_Piece
.objects
.filter(user=user)
.distinct()
.values_list( 'id', flat=True)
)
pieces = pieces.objects.filter( id__in = pk_pieces )
Also, notice that you have a n:m relation ship, you can rewrite models as:
class Piece(models.Model):
cost = models.IntegerField(default=50)
piece = models.CharField(max_length=256)
users = models.ManyToManyField(User, through='User_Piece', #<- HERE!
related_name='Pieces') #<- HERE!
And get user pieces as:
pieces = currentLoggedUser.pieces.all()