What is the difference between these two Django Q-based queries: - django

Assuming a simple model like this:
class Item(models.Model):
name = models.CharField(max_length=10)
class Relation(models.Model):
item = models.ForeignKey(Item)
weight = models.IntegerField()
And a couple of Q objects like this:
some = Q(relation__x__gt=3)
others = Q(relation__x=7)
What is the semantic difference between:
first = Item.objects.filter(some, ~others)
and
second = Item.objects.filter(some).exclude(others)
NOTE: Querying through the relation appears to be different than querying a single simple object. The SQL generated for the two above queries is different.
Here's the generated SQL for first:
SELECT "item_item"."id", "item_item"."name"
FROM "item_item"
INNER JOIN "item_relation"
ON ("item_item"."id" = "item_relation"."item_id")
WHERE ("item_relation"."weight" > 3
AND NOT ("item_item"."id" IN
(SELECT U1."item_id"
FROM "item_relation" U1
WHERE (U1."weight" = 7 AND U1."item_id" IS NOT NULL))))
And the SQL for second:
SELECT "item_item"."id", "item_item"."name"
FROM "item_item"
INNER JOIN "item_relation"
ON ("item_item"."id" = "item_relation"."item_id")
INNER JOIN "item_relation" T3
ON ("item_item"."id" = T3."item_id")
WHERE ("item_relation"."weight" > 3 AND NOT (T3."weight" = 7 ))

Related

Django ORM. Joining subquery on condition

I have a table TickerStatement, which contains financial statements about companies
class Statements(models.TextChoices):
"""
Supported statements
"""
capital_lease_obligations = 'capital_lease_obligations'
net_income = 'net_income'
price = 'price'
total_assets = 'total_assets'
short_term_debt = 'short_term_debt'
total_long_term_debt = 'total_long_term_debt'
total_revenue = 'total_revenue'
total_shareholder_equity = 'total_shareholder_equity'
class TickerStatement(TimeStampMixin):
"""
Model that represents ticker financial statements
"""
name = models.CharField(choices=Statements.choices, max_length=50)
fiscal_date_ending = models.DateField()
value = models.DecimalField(max_digits=MAX_DIGITS, decimal_places=DECIMAL_PLACES)
ticker = models.ForeignKey(Ticker, on_delete=models.CASCADE, null=False,
related_name='ticker_statements')
And now I'm trying to calculate a multiplier. The formula looks like:
(short_term_debt + total_long_term_debt) / total_shareholder_equity
I wrote a raw SQL query
SELECT "fin_tickerstatement"."fiscal_date_ending",
t2.equity AS "equity",
value AS "debt",
short_term_debt AS "short_term_debt",
(value + short_term_debt) / t2.equity AS "result"
FROM "fin_tickerstatement"
JOIN
(SELECT "fin_tickerstatement"."fiscal_date_ending",
fin_tickerstatement.value AS "equity"
FROM "fin_tickerstatement"
WHERE ("fin_tickerstatement"."ticker_id" = 12
AND "fin_tickerstatement"."fiscal_date_ending" >= date'2015-09-03'
AND "fin_tickerstatement"."name" = 'total_shareholder_equity')
GROUP BY "fin_tickerstatement"."fiscal_date_ending",
fin_tickerstatement.value
ORDER BY "fin_tickerstatement"."fiscal_date_ending" DESC) t2
ON fin_tickerstatement.fiscal_date_ending = t2.fiscal_date_ending
JOIN
(SELECT "fin_tickerstatement"."fiscal_date_ending",
fin_tickerstatement.value AS "short_term_debt"
FROM "fin_tickerstatement"
WHERE ("fin_tickerstatement"."ticker_id" = 12
AND "fin_tickerstatement"."fiscal_date_ending" >= date'2015-09-03'
AND "fin_tickerstatement"."name" = 'short_term_debt')
GROUP BY "fin_tickerstatement"."fiscal_date_ending",
fin_tickerstatement.value
ORDER BY "fin_tickerstatement"."fiscal_date_ending" DESC) t3
ON fin_tickerstatement.fiscal_date_ending = t3.fiscal_date_ending
WHERE ("fin_tickerstatement"."ticker_id" = 12
AND "fin_tickerstatement"."fiscal_date_ending" >= date'2015-09-03'
AND "fin_tickerstatement"."name" = 'total_long_term_debt')
GROUP BY "fin_tickerstatement"."fiscal_date_ending",
equity,
debt,
short_term_debt
ORDER BY "fin_tickerstatement"."fiscal_date_ending" DESC;
and have no idea how to translate it into Django ORM. Maybe you have some ideas or know some Django plugins that can help me.
The only way to solve this problem is to install django-query-builder.

Django Select across three models with filter criteria

I have three models:
class Vehicle(models.Model):
Vehicle_ID = models.AutoField('ID', primary_key= True)
Vehicle_VIN = models.CharField('FIN', max_length=30)
Vehicle_DELETED = models.BooleanField('Gelöscht',default=False)
class Recall(models.Model):
Recall_ID = models.AutoField('ID', primary_key= True)
Recall_CODE = models.CharField('Rückruf-Code',max_length=500, unique= True)
class Vehicle_Recall(models.Model):
Vehicle_Recall_ID = models.AutoField('ID', primary_key=True)
Vehicle = models.ForeignKey(Vehicle, on_delete=models.CASCADE)
Recall = models.ForeignKey(Recall, on_delete=models.CASCADE)
I want to make a Select Statement like this:
SELECT * FROM Vehicle INNER JOIN(Recall INNER JOIN Vehicle_Recall ON Recall.ID = Vehicle_Recall.Recall) ON Vehicle.ID = Vehicle_Recall.Vehicle WHERE Recall.ID=1 AND Vehicle.DELETED)=FALSE;
Is there a way to make such query in django?
You canuse the Django's ORM as follows (it is an example):
vehicles = Vehicle.objects.filter(Vehicle_DELETED=False).filter(
Q(vehicle_recall__Recall__Recall_ID=1)
)
which generates the following SQL query:
SQL SELECT "vehicle"."Vehicle_ID", "vehicle"."Vehicle_VIN", "vehicle"."Vehicle_DELETED" FROM "vehicle" INNER JOIN "vehicle_recall" ON ("vehicle"."Vehicle_ID" = "vehicle_recall"."Vehicle_id") WHERE ("vehicle"."Vehicle_DELETED" = False AND "vehicle_recall"."Recall_id" = 1)
You can use the raw query as follows:
vehicles = Vehicle.objects.raw(f"""
SELECT * FROM Vehicle
INNER JOIN(Recall INNER JOIN Vehicle_Recall ON Recall.ID = Vehicle_Recall.Recall)
ON Vehicle.ID = Vehicle_Recall.Vehicle
WHERE Recall.ID=1 AND Vehicle.DELETED=FALSE;
""")
and remember to adjust name of tables.
I propose the first solution, but the appropriate code depends on needs. I prepared only some example to present its simplicity.

Django m2m produce Unnecessary inner join, that change SQL result

Django 2.1.4 (the like behavior exist on 2.0.4 too)
Models:
class Application(models.Model):
# many fileds
name = models.CharField(max_length=255)
seers = models.ManyToManyField('Agency', through='ApplicationAgencySeer')
parent = models.ForeignKey("self", null=True, blank=True, on_delete=SET_NULL)
class ApplicationAgencySeer(models.Model):
application = models.ForeignKey(Application, on_delete=models.CASCADE)
agency = models.ForeignKey('Agency', on_delete=models.CASCADE)
created = models.DateTimeField(auto_now_add=True)
now I wanna filter
# count 0
Application.objects.filter(seers__agency__id='c3e5ed58-a4d9-4ca6-a8f7-6793eb8e3e24').count() # 0
# but count 1
ApplicationAgencySeer.objects.filter(agency__id='c3e5ed58-a4d9-4ca6-a8f7-6793eb8e3e24').count() # 1
SELECT *
FROM "app_application"
INNER JOIN "myapp_applicationagencyseer"
ON ("app_application"."id" = "app_applicationagencyseer"."application_id")
INNER JOIN "myapp_agency"
ON ("app_applicationagencyseer"."agency_id" = "app_agency"."organization_ptr_id")
INNER JOIN "myapp_agency" T4 ON ("app_agency"."organization_ptr_id" = T4."parent_id")
WHERE T4."organization_ptr_id" = 'c3e5ed58-a4d9-4ca6-a8f7-6793eb8e3e24'
if remove INNER JOIN "myapp_agency" T4 ON ("app_agency"."organization_ptr_id" = T4."parent_id") all be right.
WHY parent_id why-why
I found the bug, maybe it some related <djangoproject> but 6 years ago. I think it already fixes before 2.1.4 release.
How to me compose right filter query, or avoid this situation. help me I am stuck.
SELECT *
FROM "myapp_application"
WHERE NOT ("myapp_application"."id" IN (SELECT U1."application_id"
FROM "myapp_applicationagencyseer" U1
INNER JOIN "myapp_agency" U2 ON (U1."agency_id" = U2."organization_ptr_id")
INNER JOIN "myapp_agency" U3 ON (U2."organization_ptr_id" = U3."parent_id")
WHERE U3."organization_ptr_id" = '9e71cff4-443d-4c60-ac2d-9dcca2a9c147'))
ORDER BY "myapp_application"."created_date" DESC;
result
application_id
7d83d056-5a7d-4095-9037-98bde29a3d78 otherfields..
7cb60afc-109d-4570-ad24-6cad6b7ddd9a otherfields.. <-- this row error
--return 0
(SELECT U1."application_id"
FROM "myapp_applicationagencyseer" U1
INNER JOIN "myapp_agency" U2 ON (U1."agency_id" = U2."organization_ptr_id")
INNER JOIN "myapp_agency" U3 ON (U2."organization_ptr_id" = U3."parent_id")
WHERE U3."organization_ptr_id" = '9e71cff4-443d-4c60-ac2d-9dcca2a9c147')
--althouth I have myapp_applicationagencyseer
id created agency_id application_id status
1 2018-12-10 17:41:14.272684 9e71cff4-443d-4c60-ac2d-9dcca2a9c147 7cb60afc-109d-4570-ad24-6cad6b7ddd9a 1
2 2018-12-11 19:25:58.818000 9e71cff4-443d-4c60-ac2d-9dcca2a9c147 7cb60afc-109d-4570-ad24-6cad6b7ddd9a 0
-- myapp_agency
organization_ptr_id accreditation parent
aff44d42-ce81-4c3e-b6e1-056ed9351adb Null Null
9e71cff4-443d-4c60-ac2d-9dcca2a9c147 10АА71 Null <-- It have Null parent

Django/Postgres: Choose from DISTINCT ON sets

I have these models:
class Product(Model):
...
class Scanning(Model):
product = ForeignKey(..)
datetime = DateTimeField(...)
...
I'm trying to get one scanning for each product where the scanning is a latest one from product.scanning.all() set.
s1 = (product1,01.01.1000)
s2 = (product2,01.01.1200)
s3 = (product1,01.01.1900)
s4 = (product2,01.01.1988)
s5 = (product3,01.01.2015)
s6 = (product3,01.01.1970)
would return <s4,s3,s5>
Scanning.objects.filter(product__user=u,product__active=True).distinct('product_id').order_by('datetime')
Raises exception:
ProgrammingError: SELECT DISTINCT ON expressions must match initial
ORDER BY expressions LINE 1: SELECT DISTINCT ON
("productapp_scanning"."product_id") "pro...
^
How to make it work?
With postgres, you cannot do distinct on a field, unless it's also what you sort by:
Scanning.objects.filter(product__user=u,product__active=True).distinct('product_id').order_by('product_id', 'datetime')
If that's not good enough, one solution is to make a double query like this:
q1 = Scanning.objects.filter(product__user=u,product__active=True).values('product_id').distinct().annotate(x=Max('id'))
q2 = Scanning.objects.filter(id__in=[i["x"] for i in q1])

Join with more than one, specific one-to-many rows

I have to join one row from table Type1 with two rows from table Type2. There are example models of database:
class Type1(models.Model):
name = models.CharField(max_length=300)
class Type2(models.Model):
foreign = models.ForeignKey(Type1)
name = models.CharField(max_length=50)
value = models.IntegerField()
In SQL, query looks like this:
SELECT A.name, B.value, C.value
FROM Type1 A, Type2 B, Type2 C
WHERE B.foreign = C.foreign = A.id
AND B.name = _VAR1_ AND C.name = _VAR2_
How to do that query using Django methods (not raw SQL!)?
Not sure if I understand your question correctly, and know very little SQL to understand the SQL code.
Interpretation 1 is you want a reverse relationship query and a Q object for the OR part. See 'lookups that span relationship' and 'Complex lookups with Q objects' in this page.
Ex.
Type1.objects.filter(Q(type2__name = '_VAR1_' | type2__name = '_VAR2_'))
This return all rows of model Type1 that has an associated Type2 row with value VAR1 or VAR2 for name.
Interpretation 2: you want to define the relationship.
Well, that's easier. You have a Type1 row and want to associate two other rows with this one.
Query the Type1 row and save it to a variable.
t1 = Type1.object.get(id = X)
Then create type2 objects using t1 object in the foreign key:
t2 = Type2(foreign = t1, name = 'whatever', value = 'value')
t2.save()
and the same for the other object
t3 = Type2(foreign = t1, name = 'whatever2', value = 'value2')
t3.save()
Let me know if I misinterpreted the question.
Cheers.