I have following models:
class Product(models.Model):
name = CharField(max_length=30)
class Store(models.Model):
name = CharField(max_length=30)
product = models.ManyToManyField(Product)
How to get Stores with product named product_name and also, get all the products (except the product with name product_name) ? Is it possible to make it in one query?
In raw SQL it would be simple JOINs. Not sure how to implement it via Django.
You can actually do these things with Django due to it's lazy queryset evaluation. Django's in field lookup accepts both lists and querysets. The following will create a nested SQL code:
products = Product.objects.filter(store_set__in=stores_qs)
stores_qs = Store.objects.filter(product__name='product_name')
Here are the Django in docs.
You should be able to filter the stores based on an attribute of Product, and then prefetch_related of the retrieved objects.
Store.objects.filter(product__name="product_name").prefetch_related('product')
This should hit the database the fewest times to achieve what you are looking for - twice.
Further documentation can be found here.
Get Stores with product named "product_name" :
Store.objects.filter(product__name='product_name')
Get all the products except the product with name "product_name":
Product.objects.exclude(name='product_name')
Related
few years ego I worked with Odoo framework. and Odoo has very nice feature like this:
partner_id = field.Many2one(Partner)
partner_name = fields.Char(string='Partner name', related='partner_id.name')
basically whenever you would assign different partner_id from Partner table, partner_name would be assigned automatically. Now I started to work with django (absolute newbie), and I can't seem to find a similar functionality.
My question is what could be possible solution for this problem. Maybe there are already established external libraries that has this sort of functionality?
Expected result:
product = models.ForeignKey(Product)
product_color = models.CharField(string='Partner name', related='product.color')
having in mind that product object would have color field and it would be assigned to product_color whenever product field value Product object color value changes. Also what about storing it to database? Would be nice if there was an option to chose between storing it in database or getting it on the fly.
Cheers!
Creating a getter is pretty easy, because you can simply have functions in a Python object behave as a property:
class SampleModel(models.Model):
product = models.ForeignKey(Product)
#property
def product_color(self):
return self.product.color
This does retrieve the property on the fly, which will cause a call to the database.
Duplicating data, is usually a (more severe) antipattern. Synchronizing data, even in two tables in the same database, often turns out harder than one might expect. Even if you would use Django's signal framework for example, then some Django ORM calls can circumvent that (for example .update(..) [Django-doc]). But even if you somehow would cover those cases, then another program that talks to the database could update one of the two fields.
Most databases have triggers that can help. But again, the number of cases to cover are often larger than expected. For example, if the Product that we refer to is removed, then or the foreign key now points to a different Product, then we will need to update that field.
Therefore it is often better, to fetch the name of the related product when we need it. We can do so by (a) defining a property; or (b) make an annotation, for example in the manager.
Defining a property
We can define a property that will load the related product, and fetch the related name, like:
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
#property
def product_name(self):
return self.product.name
Then we can fetch the product name with some_order.product_name. This might not be very efficient if we need to fetch it often, since the relations are, by default, loaded lazily in Django, and thus can result in an N+1 problem.
Annotate the queryset
We can make an annotation that will fetch the name of the product in the same query when we fetch the Order, for example:
from django.db.models import F
class OrderManager(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
product_name=F('product__name')
)
class Order(models.Model):
product = models.ForeignKey(Product, on_delete=models.PROTECT)
objects = OrderManager()
Then if we fetch an order. For example with Order.objects.get(pk=1), then that Order object will have an attribute product_name with the name of the product.
I need to get list of all companies and join the company user with minimal companyuser id.
There are two models:
class Company(models.Model):
name = models.CharField(max_length=255)
kind = models.CharField(max_length=255)
class CompanyUser(models.Model):
company = models.ForeignKey('Company')
email = models.EmailField(max_length=40, unique=True)
#other fields
I've tried something like this:
companies = Company.objects.all().select_related(Min('companyuser__email'))
but It doesn't work. How can I do this with Django ORM? Is there any way to do it without raw SQL?
from django.db.models import Min
Company.objects.annotate(lowest_companyuser_id=Min("companyuser__id"))
Explanation
select_related() can be used for telling Django which related tables should be joined to the resulting queryset for reducing the number of queries, namely solving the dreaded "N+1 problem" when looping over a queryset and accessing related objects in iteration. (see docs)
With using Min() you were on the right track, but it ought to be used in conjunction with the annotate() queryset method. Using annotate() with aggregate expressions like Min(), Max(), Count(), etc. translates in an SQL query using one of the aforementioned aggregate expressions with GROUP BY. (see docs about annotate() in Django, about GROUP BY in Postgres docs)
As Burhan said - do not rely on the pk, but if u must...
companies = Company.objects.all().order_by('pk')[0]
I have the following situation. I have three models, Post, User and Friends.
class User(models.Model):
name = models.CharField(max_length=100)
class Friend(models.Model):
user1 = models.ForeignKey(User,related_name='my_friends1')
user2 = models.ForeignKey(User,related_name='my_friends2')
class Post(models.Model):
subject = models.CharField(max_length=100)
user = models.ForeignKey(User)
Every time I bring users, I want to bring the number of his friends:
User.objects.filter(name__startswith='Joe').annotate(fc=Count('my_friends1'))
This works fine.
However, I want to make this work when I bring the users as nested objects of Post. I'm using there select_related to minimized DB calls, so I want to do something like:
Post.objects.filter(subject='sport').select_related('user').annotate(user__fc=Count('user__my_friends1'))
However, this creates field user__fc under post, and not field fc under post.user.
Is there a way to achieve this functionality?
You can make use of Prefetch class:
from django.db.models import Count, Prefetch
posts = Post.objects.all().prefetch_related(Prefetch('user', User.objects.annotate(fc=Count('my_friends1'))))
for post in posts:
print(post.subject)
print(post.user.fc)
NB : this does two database queries (Django does the join between Post and User in this case) :
'SELECT "myapp_post"."id", "myapp_post"."subject", "myapp_post"."user_id" FROM "myapp_post"
'SELECT "myapp_user"."id", "myapp_user"."password", "myapp_user"."last_login", "myapp_user"."is_superuser", "myapp_user"."username", "myapp_user"."first_name", "myapp_user"."last_name", "myapp_user"."email", "myapp_user"."is_staff", "myapp_user"."is_active", "myapp_user"."date_joined", COUNT("myapp_friend"."id") AS "fc" FROM "myapp_user" LEFT OUTER JOIN "myapp_friend" ON ("myapp_user"."id" = "myapp_friend"."user1_id") WHERE "myapp_user"."id" IN (3, 4) GROUP BY "myapp_user"."id", "myapp_user"."password", "myapp_user"."last_login", "myapp_user"."is_superuser", "myapp_user"."username", "myapp_user"."first_name", "myapp_user"."last_name", "myapp_user"."email", "myapp_user"."is_staff", "myapp_user"."is_active", "myapp_user"."date_joined"
You can define a custom manger for your models, as described here and then override its get_queryset() method to add the custom column to your model upon query.
In order to use this manager for a reverse relation, you should set the base manager as described in the docs.
Another approach would be something like this, which you specify the manager of the related model with a hard-coded attribute.
How to get model instances of Filter given they should be m2m related to a given list of Tags?
class Tag(models.Model):
name=models.CharField(max_length=20)
details=models.TextField(blank=True)
parenttag=models.ForeignKey('self',null=True,related_name="childtags")
class Filter(models.Model):
tags=models.ManyToManyField(Tag, related_name='infilters')
parenttag = models.ForeignKey(Tag, related_name = 'child_filters')
This must be a relatively simple question but I didnot get answer on google.
If you have a queryset of Tags and you want to get the Filter objects that relate to them you could use the in query lookup:
Filter.objects.filter(tags__id__in=tags.values_list('id'))
I run a lab annotation website where users can annotate samples with tags relating to disease, tissue type, etc. Here is a simple example from models.py:
from django.contrib.auth.models import User
from django.db import models
class Sample(models.Model):
name = models.CharField(max_length = 255)
tags=models.ManyToManyField('Tag', through = 'Annot')
class Tag(models.Model):
name = models.CharField(max_length = 255)
class Annot(models.Model):
tag = models.ForeignKey('Tag')
sample = models.ForeignKey('Sample')
user = models.ForeignKey(User, null = True)
date = models.DateField(auto_now_add = True)
I'm looking for a query in django's ORM which will return the tags in which two users agree on the annotation of same tag. It would be helpful if I could supply a list of users to limit my query (if someone only believes User1 and User2 and wants to find the sample/tag pairs that only they agree on.)
I think I understood what you need. This one made me think, thanks! :-)
I believe the equivalent SQL query would be something like:
select t.name, s.name, count(user_id) count_of_users
from yourapp_annot a, yourapp_tag t, yourapp_sample s
where a.tag_id = t.id
and s.id = a.sample_id
group by t.name, s.name
having count_of_users > 1
While I try hard not to think in SQL when I'm coming up with django model navigation (it tends to get in the way); when it comes to aggregation queries it always helps me to visualize what the SQL would be.
In django we now have aggregations.
Here is what I came up with:
models.Annot.objects.select_related().values(
'tag__name','sample__name').annotate(
count_of_users=Count('user__id')).filter(count_of_users__gt=1)
The result set will contain the tag, the sample, and the count of users that tagged said sample with said tag.
Breaking it apart for the folks that are not used to django aggregation:
models.Annot.objects.select_related()
select_related() is forcing all tables related to Annot to be retrieved in the same query
This is what will allow me to specify tag__name and sample__name in the values() call
values('tag__name','sample__name')
values() is limiting the fields to retrieve to tag.name and sample.name
This makes sure that my aggregation on count of clients will group by just these fields
annotate(count_of_users=Count('user__id'))
annotate() adds an aggregation as an extra field to a query
filter(count_of_users__gt=1)
And finally I filter on the aggregate count.
If you want to add an additional filter on what users should be taken into account, you need to do this:
models.Annot.objects.filter(user=[... list of users...]).select_related().values(
'tag__name','sample__name').annotate(
count_of_users=Count('user__id')).filter(count_of_users__gt=1)
I think that is it.
One thing... Notice that I used tag__name and sample__name in the query above. But your models do not specify that tag names and sample names are unique.
Should they be unique? Add a unique=True to the field definitions in the models.
Shouldn't they be unique? You need to replace tag__name and sample__name with tag__id and sample__id in the query above.