I need to get a list of users who have been referenced as a ForeignKey in two other models. I'm still a little unclear on queries when they reach this complexity.
Models:
class Page(models.Model):
user = models.ForeignKey(User)
class EmailSent(models.Model):
user = models.ForeignKey(User)
name = models.CharField(max_length=200)
In English, what I want to get is: 10 active users who have 0 pages and have never had an email with the name check_in sent to them.
Here's where I am:
users = User.objects.filter(is_active=1).annotate(page_count=Count('pages')).filter(page_count=0)[10]
but not sure how to do what is essentially:
email_sent = EmailSent.objects.filter(user=user, name='check_in')
Any ideas?
One possible way to get what you want is:
users = User.objects.filter(is_active=1).annotate(page_count=Count('pages')).filter(page_count=0)
EmailSent.objects.filter(user__in=users, name='check_in')[10]
Another way is,
users = User.objects.filter(is_active=1).annotate(page_count=Count('pages')).filter(page_count=0)
users.emailsent_set.all()[10]
Try the following:
users = User.objects.filter(is_active=1)\
.exclude(emailsent__name='check_in')\
.extra(select={'page_count': "select count(*) from YOURAPP_page where user_id = auth_user.id"}, where=['page_count 0'])[:10]
Related
I have a simple question but multiple google searches left me without a nice solution. Currently I am doing the following:
allowed_categories = self.allowed_view.all().difference(self.not_allowed_view.all())
users = []
for cat in allowed_categories:
for member in cat.members.all():
users.append(member)
return users
I have a ManyToManyField to Objects that also have a ManyToManyField for instances of Users. In the code above, I am trying to get all the users from all those categories and get a list of all Users.
Later I would like the same in a method allowed_to_view(self, user_instance) but that's for later.
How would I achieve this using Django ORM without using nested for-loops?
[edit]
My models are as follows:
class RestrictedView(models.Model):
allowed_view = models.ManyToManyField(Category)
not_allowed_view = models.ManyToManyField(Category)
class Category(models.Model):
name = models.CharField(max_length=30)
members = models.ManyToManyField(User)
So, I've made the following one-liner with only one query towards the database. It took me some time...
users = User.objects.filter(pk__in=self.allowed_view.all().values("users").difference(self.not_allowed_view.all().values("users")))
This gives me a nice queryset with only the users that are in the allowed_view and explicitly not in the not_allowed_view.
Without seeing you database structure / models.py file its hard to say, but you can do a search on member objects like so:
members_queryset = Member.objects.filter(
category = <allowed categories>,
...
)
users += list(members.all())
I have 2 models with a one-to-many relation on a MySQL DB:
class Domains(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=50, unique=True)
description = models.TextField(blank=True, null=True)
class Kpis(models.Model):
id = models.AutoField(primary_key=True)
name = models.CharField(max_length=50, unique=True)
description = models.TextField(blank=True, null=True)
domain_id = models.ForeignKey(Domains, on_delete=models.CASCADE, db_column='domain_id')
In order to bring ALL the domains with all their kpis objects, i use this code with a for loop:
final_list = []
domains_list = Domains.objects.all()
for domain in domains_list:
# For each domain, get all related KPIs
domain_kpis = domain.kpis_set.values()
final_list.append({domain:domains_kpis})
The total number of queries i run is: 1 + the number of total domains i have, which is quite a lot.
I'm looking for a way to optimize this, preferably to execute it within only one query on the database. Is this possible?
You use .prefetch_related(…) [Django-doc] for this:
final_list = []
domains_list = Domains.objects.prefetch_related('kpis_set')
for domain in domains_list:
# For each domain, get all related KPIs
domain_kpis = domain.kpis_set.all()
final_list.append({domain:domains_kpis})
This will make two queries: one to query the domains, and a second to query all the related Kpis with a single query into memory.
Furthermore please do not use .values(). You can serialze data to JSON with Django's serializer framework, by making use of .values() you "erode" the model layer. See the Serializing Django objects section of the documentation for more information.
Just wanted to add that you are asking a solution for "classic" N +1 queries problem. Here you can read a something about it and aslo find the examples for prefetch_related method adviced in Willem's answer.
Another thing worth mentioning is that probably you aren't suppose to use this dict final_list.append({domain:domains_kpis}), but instead you may want to map some field(s) from Domain to some field(s) from Kapis models and, if this is true, you can specify exact fields you'd like to have prefetched using Prefetch:
domains_list = Domains.objects.prefetch_related(Prefetch('kpis_set'), queryset=Kapis.objects.all().only('some_field_you_want_to_have'))
final_list = []
for domain in domains_list:
domain_kpis = domain.kpis_set.all()
final_list.append({domain.some_field:domains_kpis.prefetched_field})
This should give another boost to performance on big-volume table's.
I have two models that I'm relating using Django's OneToOneField, following this documentation: https://docs.djangoproject.com/en/2.0/topics/db/examples/one_to_one/
class Seats(models.Model):
north = models.OneToOneField('User',on_delete=models.CASCADE,related_name='north', default=None, null=True)
bridgetable = models.OneToOneField('BridgeTable',on_delete=models.CASCADE, default=None, null=True)
class BridgeTableManager(models.Manager):
def create_deal(self):
deal = construct_deal()
table = self.create(deal=deal)
s = Seats(bridgetable=table)
s.save()
return table
class BridgeTable(models.Model):
deal = DealField(default=None,null=True)
When I run this code I can successfully get the relationship working
table = BridgeTable.objects.get(pk='1')
user = User.objects.get(username=username)
table.seats.north = user
table.seats.north.save()
print(table.seats.north)
The print statement prints out the name of the player sitting north. But if I try to access the table again like this:
table = BridgeTable.objects.get(pk='1')
print(table.seats.north)
I get "None" instead of the user's name. Is there something I'm missing, like a save that I missed or some concept I'm not understanding? Thanks.
You should save Seats model object that is table.seats.save()
Try print table.seats.north
While table.seats.north.save() runs save on User object
Here are correct steps:
table = BridgeTable.objects.get(pk='1')
user = User.objects.get(username=username)
table.seats.north = user
table.seats.save()
print(table.seats.north)
I am implementing a web interface for email lists. When a list administrator logs in, the site will visually display which lists they are an owner of and corresponding information about the lists. For this I have decided to have two tables:
1) An owner table which contains entries for information about list administrators. Each of these entries contains a 'ManyToManyField' which holds the information about which lists the owner in any given entry is an administrator for.
2) A list table which contains entries with information about each email list. Each entry contains the name of the list a 'ManyToManyField' holding information about which owners are administrators the list.
Here is the code in models.py:
from django.db import models
class ListEntry(models.Model):
name = models.CharField(max_length=64)
owners = models.ManyToManyField('OwnerEntry')
date = models.DateTimeField('date created')
class Meta:
ordering = ('name',)
class OwnerEntry(models.Model):
name = models.CharField(max_length=32)
lists = models.ManyToManyField('ListEntry')
class Meta:
ordering = ('name',)
I have already set up a simple local database to create a basic working website with. I have populated it with test entries using this code:
from list_app.models import *
from datetime import *
le1 = ListEntry(
name = "Physics 211 email list",
date = datetime.now(),
)
le1.save()
le2 = ListEntry(
name = "Physics 265 email list",
date = datetime(2014,1,1),
)
le2.save()
oe1 = OwnerEntry(
name = 'wasingej',
)
oe1.save()
oe1.lists.add(le1,le2)
le1.owners.add(oe1)
le2.owners.add(oe1)
oe2 = OwnerEntry(
name = 'doej',
)
oe2.save()
oe2.lists.add(le1)
le1.owners.add(oe2)
Here is where my error occurs: When the user has logged in via CAS, I have them redirected to this page in views.py:
def login_success(request):
u = OwnerEntry(name=request.user)
print(u.name)
print(u.lists)
return HttpResponse("login success!")
At the line 'print(u.lists)', I get the error "" needs to have a value for field "ownerentry" before this many-to-many relationship can be used.
What am I doing wrong here?
Your model structure is broken, for a start. You don't need ManyToManyFields on both sides of the relationship, only one - Django will provide the accessor for the reverse relationship.
Your issue is happening because you are not querying an existing instance from the database, you are instantiating an unsaved one. To query, you use model.objects.get():
u = OwnerEntry.objects.get(name=request.user.username)
You need to provide the actual class to the ManyToManyField constructor, not a string.
https://docs.djangoproject.com/en/dev/topics/db/examples/many_to_many/
I have a Q&A app that allows users to block/hide other users they find annoying or offensive. When a current_user views all questions – Question.objects.all() – I would like to exclude from the queryset all questions asked by users that have been blocked by current_user – Block.objects.filter(user_is_blocking=current_user)
What’s the most efficient way to do this? If it makes a difference to your answer, I’ll be applying the same exclusion list to other models, like Answers, that also include User as a FK.
Models:
class Question(models.Model):
user = models.ForeignKey(User)
question = models.CharField()
class Block(models.Model):
user_is_blocking = models.ForeignKey(User, related_name="user_blocking")
user_is_blocked = models.ForeignKey(User, related_name="user_blocked")
UPDATE:
I got this working thanks to sk1p. I then wanted to make the exclude bi-directional - so I only see questions from users I have not blocked AND who have not blocked me. I did that as follows for anyone interested in combining subqueries:
from itertools import chain
i_am_blocking = User.objects.filter(user_blocked__user_is_blocking=request.user)
is_blocking_me = User.objects.filter(user_blocking__user_is_blocked=request.user)
blocked_users = list(chain(i_am_blocking, is_blocking_me))
questions = Question.objects.exclude(user__in=blocked_users)
...
You can use the __in field lookup:
class Question(models.Model):
user = models.ForeignKey(User)
question = models.CharField()
class Block(models.Model):
user_blocking = models.ForeignKey(User, related_name="blocks_set")
user_blocked = models.ForeignKey(User, related_name="blocks_received")
blocked_users = User.objects.filter(blocks_received__user_blocking=current_user)
questions = Question.objects.exclude(user__in=blocked_users)
(I took the liberty of renaming your related names and fields to understand the problem better, you don't have to change your names of course...)
The __in lookup will be translated into a subquery, so the database will do most of the work.
The actual performance then depends on your database. The Django documentation warns about bad optimization by MySQL, in particular, so I'd recommend running the resulting query against a copy of your production database, and have a look at the query plan with EXPLAIN or EXPLAIN ANALYZE (on PostgreSQL).
If the subquery-based approach doesn't work well with your database, you can rewrite it as a raw SQL query and compare performance:
SELECT q.*
FROM app_question q,
auth_user author
LEFT JOIN app_block b ON (b.user_blocked_id = author.id)
LEFT JOIN auth_user ON (b.user_blocking_id = auth_user.id AND auth_user.id = %s)
WHERE author.id = q.user_id AND auth_user.id IS NULL;
Proof-of-concept SQLFiddle
The idea is to join the block table to the question's author, and if one exists, discard the question. The %s is the parameter for the current user's id; be sure to not use string formatting!