How to specify GROUP BY field in Dajngo ORM? - django

Question

I have the following working SQL statement:
SELECT id FROM ops_kpitarget WHERE (site_id = 1 AND validFrom <= "2019-08-28") GROUP BY kpi_id HAVING validFrom = (MAX(validFrom))
But I cannot get this to work inside Django ORM.
The best I got was the code below, but then the database is complaining that it is missing a GROUP BY clause to make HAVING work.
How can I get the same query with specifying "kpi_id" as the GROUP BY clause using Djangos ORM? Any ideas?
KpiTarget.objects
.filter(validFrom__lte=fromDate)
.values("id", "kpi")
.filter(validFrom=Max("validFrom"))
... which translates to:
SELECT "ops_kpitarget"."id", "ops_kpitarget"."kpi_id" FROM "ops_kpitarget" WHERE "ops_kpitarget"."validFrom" <= 2019-08-14 HAVING "ops_kpitarget"."validFrom" = (MAX("ops_kpitarget"."validFrom"))
I played around with annotate but this is not really giving me what I want...
Update:
Some background: I have 3 tables: Kpi, KpiTarget, and KpiTargetObservation.
Kpi holds all general information regarding the KPI like name, typeetc.
KpiTarget stores target values defined for several different sites. These target values can change over time. Hence, I have included the combination of MAX() and validFrom <= (some date) to determine the latest valid target for any given KPI.
KpiTargetObservation stores the individual observations per defined KPI target. It just holds the link to KpiTarget, the date of the observation, and the observation value.
The final queries I need to build will have to give me something like the following:
give me all known KPIs per given site
tell me the most recent target value for the KPIs you found
get me any known observation that is related to the identified kpi targets
I am struggling with the 2nd query, and specifically how to get this working using Djangos ORM. I could just escape to RAW SQL, but I would prefer to not to, if possible.
The models:
class KpiCategory(models.Model):
name = models.CharField(max_length=255)
def __str__(self):
return self.name
class Kpi(models.Model):
KPI_KIND_CHOICES = [("BOOL", "Boolean"), ("FLOAT", "Float"), ("STRING", "String")]
# firstCreated = models.DateField(auto_now_add=True)
# firstCreatedBy = models.OneToOneField(User, on_delete=models.CASCADE)
# lastEdited = models.DateField(auto_now=True)
# lastEditedBy = models.OneToOneField(User, on_delete=models.CASCADE)
name = models.CharField(max_length=255)
category = models.ForeignKey(KpiCategory, on_delete=models.CASCADE)
kind = models.CharField(max_length=150, choices=KPI_KIND_CHOICES)
def __str__(self):
return self.name
class KpiTarget(models.Model):
# firstCreated = models.DateField(auto_now_add=True)
# firstCreatedBy = models.OneToOneField(User, on_delete=models.CASCADE)
# lastEdited = models.DateField(auto_now=True)
# lastEditedBy = models.OneToOneField(User, on_delete=models.CASCADE)
kpi = models.ForeignKey(Kpi, on_delete=models.CASCADE, related_name="kpiTargetSet")
targetDouble = models.DecimalField(
max_digits=20, decimal_places=15, blank=True, null=True
)
targetBool = models.BooleanField(blank=True, null=True)
targetStr = models.CharField(max_length=255, blank=True)
site = models.ForeignKey(Site, on_delete=models.CASCADE)
validFrom = models.DateField()
def __str__(self):
return str(self.kpi)
class KpiObservation(models.Model):
# firstCreated = models.DateField(auto_now_add=True)
# firstCreatedBy = models.OneToOneField(User, on_delete=models.CASCADE)
# lastEdited = models.DateField(auto_now=True)
# lastEditedBy = models.OneToOneField(User, on_delete=models.CASCADE)
kpiTarget = models.ForeignKey(
KpiTarget, on_delete=models.CASCADE, related_name="kpiObservationSet"
)
observed = models.DateField()
observationDouble = models.DecimalField(
max_digits=20, decimal_places=15, blank=True, null=True
)
observationBool = models.BooleanField(blank=True, null=True)
observationStr = models.CharField(max_length=255, blank=True)
def __str__(self):
return str(self.observed)

Answer 1

KpiTarget.objects.filter(validFrom__lte=fromDate).annotate(validFrom=Max("validFrom")).order_by('kpi__id').values("id", "kpi")