Performing a Subquery, Sum and Join in django ORM - django

I have 2 django models which aren't linked by ForeignKey due to legacy system.
class Parent(model):
name -> CharField()
class Child(model)
parent_name -> CharField()
cost -> IntegerField()
I want to achieve a left join which gives me all parent columns along with sum of cost column from children.
SQL in postgres translates to
select parent.name, sum(child.cost) as cost from parent join child on parent.name = child.parent_name group by parent.name;
Is there any way to achieve this with django ORM
I have tried a lot of things but https://code.djangoproject.com/ticket/28296 might be what is blocking.

Please use a ForeignKey to refer to a parent, not a CharField that joins on the name. This will guarantee referential integrity, avoid data duplication and makes it more convenient to query in Django.
You thus define the models as:
class Parent(models.Model):
name = models.CharField(max_length=128)
class Child(models.Model):
parent = models.ForeignKey(
Parent,
on_delete=models.CASCADE
)
cost = models.IntegerField()
or if the name of the Parent is unique, you can let it refer to the name:
class Parent(models.Model):
name = models.CharField(max_length=128, unique=True)
class Child(models.Model):
parent = models.ForeignKey(
Parent,
to_field='name',
db_column='parent_name',
on_delete=models.CASCADE
)
cost = models.IntegerField()
then you can .annotate(…) [Django-doc] the Parent model with:
from django.db.models import Sum
Parent.objects.annotate(
cost=Sum('child__cost')
)
the Parent objects that arise from this queryset will have an extra attribute .cost that will contain the sum of the costs of the Childs that point to that Parent.

Related

Update column in Django with child columns

I have 2 models Parent, Child
class Parent(models.Model):
id = Base64UUIDField(primary_key=True, editable=False)
cost = models.DateTimeField(default=None, blank=True, null=True)
class Child(models.Model):
id = Base64UUIDField(primary_key=True, editable=False)
cost = models.DateTimeField(default=None, blank=True, null=True)
parent = models.ForeignKey(Parent, related_name= "children", related_query_name= "child")
I need to populate cost column of Parent objects to maximum cost of all children of that parent
I have tried to annotate to a new column new_cost, and its works.
parents.annotate(new_cost=Max('child__cost'))
But I need to populate values to existing column cost. Tried something like this, but not working.
parents.update(cost=Max('child__cost'))
Probably that can be achieved with a Subquery expression:
from django.db.models import OuterRef, Subquery
Parent.objects.update(
cost=Subquery(Child.objects.filter(
parent_id=OuterRef('pk')
).values('cost').order_by('-cost')[:1])
)
I would however advise to simply use .annotate(…) when you need the largest cost of the related Childs.
Use aggregate againist to annotate
test = parents.aggregate(new_cost=Max('child__cost'))
parents.update(cost=test["new_cost"])

How to retrieve related instances without FK using one query in django

Imagine there are three models named Movie, Actor, and Participation.
class Movie(models.Model):
identifier = models.CharField()
class Actor(models.Model):
name = models.CharField()
class Participation(models.Model):
movie_identifier = models.CharField()
actor = models.ForgeinKey(Actor, on_delete=models.CASCADE)
Let's assume that I can't use ForgeinKey for the movie in the Participation model.
how can I retrieve all the participation records of a movie with only one query?
Here is the solution if I had a foreign key for the movie in the participation table:
qs = Movie.objects.filter(identifier="an_identiier").prefetch_related("participations_set")
How can I do this without having a Movie foreign key in the Participation model?
Thanks!
One of the most important things when designing a database (hence when designing your models) is database normalization [Wikipedia].
You talk about Participation being related to multiple models like Movie, Series, Episode, etc. this means that Movie, Series, Episode all can be said to have something in common or they can be said to be a specialization of another entity let us say Participatable for the lack of a better word, or we can say Participatable is a generalization of Movie, Series, Episode, etc.
How do we model these? Well we will just have an extra model that our other models will have a OneToOneField with:
class Participatable(models.Model):
# Any common fields here
MOVIE = 'M'
SERIES = 'S'
TYPE_CHOICES = [
(MOVIE, 'Movie'),
(SERIES, 'Series'),
]
subject = models.CharField(max_length=1, choices=TYPE_CHOICES)
class Movie(models.Model):
# uncommon fields
participatable = models.OneToOneField(
Participatable,
on_delete=models.CASCADE,
related_name='movie',
)
class Series(models.Model):
# uncommon fields
participatable = models.OneToOneField(
Participatable,
on_delete=models.CASCADE,
related_name='series',
)
class Participation(models.Model):
participatable = models.ForgeinKey(Participatable, on_delete=models.CASCADE)
actor = models.ForgeinKey(Actor, on_delete=models.CASCADE)
Other than this solution which I find is the best for such modelling you can go with using the content-types framework which will essentially do what you do currently. That is it will use a field that stores the related id and also a foreign key that points to an entry in a table that will simply describe which table this id is for.

how to create model instance with circular foreign key that cannot be null

In my django app I have two Model in a one-to-many relationship: Term and TermName (Term has many TermName). I want to keep track in Term of a particular TermName instance, say TermName models are name alias or the related Term model, but one of them is the "reference" name alias. For this I have added a one-to-one relation between the two Models. Here is the code:
class TermName(models.Model):
name = models.CharField(max_length=255)
term = models.ForeignKey(
'Term',
on_delete=models.CASCADE
)
class Term(models.Model):
ref_termname = models.OneToOneField(
TermName,
on_delete=models.DO_NOTHING,
related_name = 'reference_of_term',
)
Problem is if I create a new Term instance, django complains that ref_termname cannot be null. However sames goes if I want to create the "reference" TermName beforehands; it now complains about term being null ...
My workaround is to let ref_termname be null (i.e. use null=True field option). But for my model design I would like to make it not nullable. Can this be possible?
thanks for your help!
What if you move your OneToOneField to your TermName model? I'm thinking you could model this as follows:
class TermName(models.Model):
name = models.CharField(max_length=255)
term = models.ForeignKey(
'Term',
on_delete=models.CASCADE
)
reference_of_term = models.OneToOneField(
TermName,
on_delete=models.PROTECT,
related_name = 'ref_termname',
)
class Term(models.Model):
pass
Now you should be able to create Term's instances first and then create TermNames that point to a Term. The only disadvantage is that Terms are not forced to have a TermName.

Why Django ORM failing on simple foreign key relation?

I'm trying to simply create one to many relation model of categories using Django amazing ORM.
SQL:
create table categories(
id serial primary key not null,
parent_id int
);
insert into categories values(default,default,default);
update categories set parent_id = 1 where id > 1;
select * from categories;
id | parent_id
----+-----------
2 | 1
3 | 1
1 |
(3 rows)
Django amazing orm model:
class Categories(models.Model):
id = models.AutoField(primary_key=True)
parent_id = models.ForeignKey('self')
class Meta:
managed = False
db_table = 'categories'
Django Query:
Categories.objects.get(id=1)
OUTPUT:
django.db.utils.ProgrammingError: column categories.parent_id_id does not exist
LINE 1: SELECT "categories"."id", "categories"."parent_id_id" FROM "...
^
HINT: Perhaps you meant to reference the column "categories.parent_id".
Why it uses parent_id_id column instead of parent_id and how I can force it to use parent_id?
EDIT
I just changed parent_id field to parent.
EDIT 2
tatlar answer is not in my case becouse i already have database schema.
So after digging more deeper in documentation and other questions on stackoverflow there is what i have in result. This model contains reference to parent and children categories for each row. It could be inherited for all graph alike data models (comments, categories etc).
class Categories(models.Model):
id = models.AutoField(primary_key=True)
parent = models.ForeignKey('self', on_delete=None, parent_link=True, related_name='children')
class Meta:
managed = False
db_table = 'categories'
Get all children for category 1:
from myapp.models import Categories
ch = Categories.objects.get(id=1).children print (ch)
# <QuerySet [<Categories: Categories object (2)>, <Categories: Categories object (3)>]>
Get parent for category 2:
from myapp.models import Categories
ch = Categories.objects.get(id=1).parent
print (ch)
# <Categories: Categories object (1)>
Sorry to hear that you are having trouble with Django. In time you may grow to love the Django ORM and how it abstracts all the SQL code for you :)
You need to dig a little deeper into how the ORM works -- it's not a 1:1 replacement for SQL code. Check out the Model docs.
In your specific case, you need to create a new class called Parent and reference that class (via a ForeignKey) from your Categories class (you might also like to rename your Categories class to Category -- the ORM also handles plurals).
Try the code below (where I have already renamed Categories to Category for you):
from django.db import models
class Category(models.Model):
id = models.AutoField(primary_key=True)
parent = models.ForeignKey(Parent)
# ... Extra model attributes
class Meta:
verbose_name_plural = "categories"
class Parent(models.Model):
id = models.AutoField(primary_key=True)
# ... Extra model attributes
Then add all the extra attributes you need. This will create all the database tables, and their relationships, without you ever writing any SQL. If you are used to writing SQL it is a change, but it makes sense as you work more with the ORM and understand how good it is actually architected.
Good luck!
tatlar answer is not in my case becouse i already have database schema.
So after digging more deeper in documentation and other questions on stackoverflow there is what i have in result. This model contains reference to parent and children categories for each row. It could be inherited for all graph alike data models (comments, categories etc).
class Categories(models.Model):
id = models.AutoField(primary_key=True)
parent = models.ForeignKey('self', on_delete=None, parent_link=True, related_name='children')
class Meta:
managed = False
db_table = 'categories'
Get all children for category 1:
from myapp.models import Categories
ch = Categories.objects.get(id=1).children
print (ch)
# <QuerySet [<Categories: Categories object (2)>, <Categories: Categories object (3)>]>
Get parent for category 2:
from myapp.models import Categories
ch = Categories.objects.get(id=1).parent
print (ch)
# <Categories: Categories object (1)>

"already exists" error when using ManyToMany relationship between the same two models

class Product( models.Model ):
name = models.CharField(verbose_name="Name", max_length=255, null=True, blank=True)
the_products_inside_combo = models.ManyToManyField('self', verbose_name="Products Inside Combo", help_text="Only for Combo Products", blank=True)
However, I got this error when I tried to put the duplicate values:
From_product-to_product relationship with this From product and To
product already exists.
Screencap of the error.
Each pair (Product, Product) must be unique. This is why you get already exists error.
Behind the scenes, Django creates an intermediary join table to
represent the many-to-many relationship.
What do you want to do is to have many-to-many relationship between two models (nevermind that they are the same) with additional information stored - quantity (so you would have ProductA = 2x ProductB + ....
In order to model this relationship you will have to create intermediary model and use through option. Documentation explains it very well, so have a look:
https://docs.djangoproject.com/en/dev/topics/db/models/#intermediary-manytomany
Update
Here is minimal working example:
class Product(models.Model):
name = models.CharField(verbose_name='Name', max_length=255, null=True, blank=True)
products = models.ManyToManyField('self', through='ProductGroup', symmetrical=False)
def __str__(self):
return self.name
class ProductGroup(models.Model):
parent = models.ForeignKey('Product', related_name='+')
child = models.ForeignKey('Product', related_name='+')
and admin models:
class ProductGroupInline(admin.TabularInline):
model = Product.products.through
fk_name = 'parent'
class ProductAdmin(admin.ModelAdmin):
inlines = [
ProductGroupInline,
]
admin.site.register(Product, ProductAdmin)
admin.site.register(ProductGroup)
As you can see recursive Product-Product relation is modeled with ProductGroup (through parameter). Couple of notes:
Many-to-many fields with intermediate tables must not be symmetrical, hence symmetrical=False. Details.
Reverse accessors for ProductGroup are disabled ('+') (in general you can just rename them, however, you don't want to work with ProductGroup directly). Otherwise we would get Reverse accessor for 'ProductGroup.child' clashes with reverse accessor for 'ProductGroup.parent'..
In order to have a nice display of ManyToMany in admin we have to use inline models (ProductGroupInline). Read about them in documentation. Please note, however, fk_name field. We have to specify this because ProductGroup itself is ambiguous - both fields are foreign keys to the same model.
Be cautious with recurrency. If you would define, for example, __str__ on Product as: return self.products having ProductGroup with the same parent as the child you would loop infinitely.
As you can see in the screencap pairs can be duplicated now. Alternatively you would just add quantity field to ProductGroup and check for duplication when creating objects.