If I have queries on multiple tables like:
d = Relations.objects.filter(follow = request.user).filter(date_follow__lt = last_checked)
r = Reply.objects.filter(reply_to = request.user).filter(date_reply__lt = last_checked)
article = New.objects.filter(created_by = request.user)
vote = Vote.objects.filter(voted = article).filter(date__lt = last_checked)
and I want to display the results from all of them ordered by date (I mean not listing all the replies, then all the votes, etc ).
Somehow, I want to 'join all these results', in a single queryset.
Is there possible?
It seems like you need different objects to have common operations ...
1) In this case it might be better to abstract these properties in a super class... I mean that you could have an Event class that defines a user field, and all your other event classes would subclass this.
class Event(model.Model):
user = models.ForeignKey(User)
date = ...
class Reply(Event):
#additional fields
class Vote(Event):
#additional fields
Then you would be able to do the following
Event.objects.order_by("date") #returns both Reply, Vote and Event
Check-out http://docs.djangoproject.com/en/1.2/topics/db/models/#id5 for info on model inheritance.
2) You could also have an Event model with a generic relation to another object. This sounds cleaner to me as a Vote is conceptually not an "event". Check-out : http://docs.djangoproject.com/en/dev/ref/contrib/contenttypes/#id1
Anyway, I think your problem is a matter of design
In addition to to Sebastien's proposal number 2: Django actually has some built-in functionality that you could "abuse" for this; for the admin it has already a model that logs the user's actions and references the objects through a generic foreign key relation, I think you could just sub-class this model and use it for your purposes:
from django.contrib.admin.models import LogEntry, ADDITION
from django.utils.encoding import force_unicode
from django.contrib.contenttypes.models import ContentType
class MyLog(LogEntry):
class Meta(LogEntry.Meta):
db_table_name = 'my_log_table' #use another name here
def log_addition(request, object):
LogEntry.objects.log_action(
user_id = request.user.pk,
content_type_id = ContentType.objects.get_for_model(object).pk,
object_id = object.pk,
object_repr = force_unicode(object),
action_flag = ADDITION
)
You can now log all your notifications etc. where they happen with with log_addition(request, object) and filter the Log table than for your purposes! If you want to log also changes / deletions etc. you can make yourself some helper functions for that!
Related
I have users who create (or receive) transactions. The transaction hierarchy I have is a multi-table inheritance, with Transaction as the base model containing the common fields between all transaction types, such as User (FK), amount, etc. I have several transaction types, which extend the Transaction model with type specific data.
For the sake of this example, a simplified structure illustrating my problem can be found below.
from model_utils.managers import InheritanceManager
class User(models.Model):
pass
class Transaction(models.Model):
DEPOSIT = 'deposit'
WITHDRAWAL = 'withdrawal'
TRANSFER = 'transfer'
TYPES = (
(DEPOSIT, DEPOSIT),
(WITHDRAWAL, WITHDRAWAL),
(TRANSFER, TRANSFER),
)
type = models.CharField(max_length=24, choices=TYPES)
user = models.ForeignKey(User)
amount = models.PositiveIntegerField()
objects = InheritanceManager()
class Meta:
indexes = [
models.Index(fields=['user']),
models.Index(fields=['type'])
]
class Withdrawal(Transaction):
TYPE = Transaction.WITHDRAWAL
bank_account = models.ForeignKey(BankAccount)
class Deposit(Transaction):
TYPE = Transaction.DEPOSIT
card = models.ForeignKey(Card)
class Transfer(Transaction):
TYPE = Transaction.Transfer
recipient = models.ForeignKey(User)
class Meta:
indexes = [
models.Index(fields=['recipient'])
]
I then set each transaction's type in the inherited model's .save() method. This is all fine and well.
The problem comes in when I would like to fetch the a user's transactions. Specifically, I require the sub-model instances (deposits, transfers and withdrawals), rather than the base model (transactions). I also require transactions that the user both created themselves AND transfers they have received. For the former I use django-model-utils's fantastic IneritanceManager, which works great. Except that when I include the filtering on the transfer submodel's recipient FK field the DB query increases by an order of magnitude.
As illustrated above I have placed indexes on the Transaction user column and the Transfer recipient column. But it appeared to me that what I may need is an index on the Transaction subtype, if that is at all possible. I have attempted to achieve this effect by putting an index on the Transaction type field and including it in the query, as you will see below, but this appears to have no effect. Furthermore, I use .select_related() for the user objects since they are required in the serializations.
The query is structured as such:
from django.db.models import Q
queryset = Transaction.objects.select_related(
'user',
'transfer__recipient'
).select_subclasses().filter(
Q(user=request.user) |
Q(type=Transaction.TRANSFER, transfer__recipient=request.user)
).order_by('-id')
So my question is, why is there an order of magnitude difference on the DB query when including the Transfer.recipient in the query? Have I missed something? Am I doing something silly? Or is there a way I can optimise this further?
I want to build an webapp like Quora or Medium, where a user can follow users or some topics.
eg: userA is following (userB, userC, tag-Health, tag-Finance).
These are the models:
class Relationship(models.Model):
user = AutoOneToOneField('auth.user')
follows_user = models.ManyToManyField('Relationship', related_name='followed_by')
follows_tag = models.ManyToManyField(Tag)
class Activity(models.Model):
actor_type = models.ForeignKey(ContentType, related_name='actor_type_activities')
actor_id = models.PositiveIntegerField()
actor = GenericForeignKey('actor_type', 'actor_id')
verb = models.CharField(max_length=10)
target_type = models.ForeignKey(ContentType, related_name='target_type_activities')
target_id = models.PositiveIntegerField()
target = GenericForeignKey('target_type', 'target_id')
tags = models.ManyToManyField(Tag)
Now, this would give the following list:
following_user = userA.relationship.follows_user.all()
following_user
[<Relationship: userB>, <Relationship: userC>]
following_tag = userA.relationship.follows_tag.all()
following_tag
[<Tag: tag-job>, <Tag: tag-finance>]
To filter I tried this way:
Activity.objects.filter(Q(actor__in=following_user) | Q(tags__in=following_tag))
But since actor is a GenericForeignKey I am getting an error:
FieldError: Field 'actor' does not generate an automatic reverse relation and therefore cannot be used for reverse querying. If it is a GenericForeignKey, consider adding a GenericRelation.
How can I filter the activities that will be unique, with the list of users and list of tags that the user is following? To be specific, how will I filter GenericForeignKey with the list of the objects to get the activities of the following users.
You should just filter by ids.
First get ids of objects you want to filter on
following_user = userA.relationship.follows_user.all().values_list('id', flat=True)
following_tag = userA.relationship.follows_tag.all()
Also you will need to filter on actor_type. It can be done like this for example.
actor_type = ContentType.objects.get_for_model(userA.__class__)
Or as #Todor suggested in comments. Because get_for_model accepts both model class and model instance
actor_type = ContentType.objects.get_for_model(userA)
And than you can just filter like this.
Activity.objects.filter(Q(actor_id__in=following_user, actor_type=actor_type) | Q(tags__in=following_tag))
What the docs are suggesting is not a bad thing.
The problem is that when you are creating Activities you are using auth.User as an actor, therefore you can't add GenericRelation to auth.User (well maybe you can by monkey-patching it, but that's not a good idea).
So what you can do?
#Sardorbek Imomaliev solution is very good, and you can make it even better if you put all this logic into a custom QuerySet class. (the idea is to achieve DRY-ness and reausability)
class ActivityQuerySet(models.QuerySet):
def for_user(self, user):
return self.filter(
models.Q(
actor_type=ContentType.objects.get_for_model(user),
actor_id__in=user.relationship.follows_user.values_list('id', flat=True)
)|models.Q(
tags__in=user.relationship.follows_tag.all()
)
)
class Activity(models.Model):
#..
objects = ActivityQuerySet.as_manager()
#usage
user_feed = Activity.objects.for_user(request.user)
but is there anything else?
1. Do you really need GenericForeignKey for actor? I don't know your business logic, so probably you do, but using just a regular FK for actor (just like for the tags) will make it possible to do staff like actor__in=users_following.
2. Did you check if there isn't an app for that? One example for a package already solving your problem is django-activity-steam check on it.
3. IF you don't use auth.User as an actor you can do exactly what the docs suggest -> adding a GenericRelation field. In fact, your Relationship class is suitable for this purpose, but I would really rename it to something like UserProfile or at least UserRelation. Consider we have renamed Relation to UserProfile and we create new Activities using userprofile instead. The idea is:
class UserProfile(models.Model):
user = AutoOneToOneField('auth.user')
follows_user = models.ManyToManyField('UserProfile', related_name='followed_by')
follows_tag = models.ManyToManyField(Tag)
activies_as_actor = GenericRelation('Activity',
content_type_field='actor_type',
object_id_field='actor_id',
related_query_name='userprofile'
)
class ActivityQuerySet(models.QuerySet):
def for_userprofile(self, userprofile):
return self.filter(
models.Q(
userprofile__in=userprofile.follows_user.all()
)|models.Q(
tags__in=userprofile.relationship.follows_tag.all()
)
)
class Activity(models.Model):
#..
objects = ActivityQuerySet.as_manager()
#usage
#1st when you create activity use UserProfile
Activity.objects.create(actor=request.user.userprofile, ...)
#2nd when you fetch.
#Check how `for_userprofile` is implemented this time
Activity.objects.for_userprofile(request.user.userprofile)
As stated in the documentation:
Due to the way GenericForeignKey is implemented, you cannot use such fields directly with filters (filter() and exclude(), for example) via the database API. Because a GenericForeignKey isn’t a normal field object, these examples will not work:
You could follow what the error message is telling you, I think you'll have to add a GenericRelation relation to do that. I do not have experience doing that, and I'd have to study it but...
Personally I think this solution is too complex to what you're trying to achieve. If only the user model can follow a tag or authors, why not include a ManyToManyField on it. It would be something like this:
class Person(models.Model):
user = models.ForeignKey(User)
follow_tag = models.ManyToManyField('Tag')
follow_author = models.ManyToManyField('Author')
You could query all followed tag activities per Person like this:
Activity.objects.filter(tags__in=person.follow_tag.all())
And you could search 'persons' following a tag like this:
Person.objects.filter(follow_tag__in=[<tag_ids>])
The same would apply to authors and you could use querysets to do OR, AND, etc.. on your queries.
If you want more models to be able to follow a tag or author, say a System, maybe you could create a Following model that does the same thing Person is doing and then you could add a ForeignKey to Follow both in Person and System
Note that I'm using this Person to meet this recomendation.
You can query seperately for both usrs and tags and then combine them both to get what you are looking for. Please do something like below and let me know if this works..
usrs = Activity.objects.filter(actor__in=following_user)
tags = Activity.objects.filter(tags__in=following_tag)
result = usrs | tags
You can use annotate to join the two primary keys as a single string then use that to filter your queryset.
from django.db.models import Value, TextField
from django.db.models.functions import Concat
following_actor = [
# actor_type, actor
(1, 100),
(2, 102),
]
searchable_keys = [str(at) + "__" + str(actor) for at, actor in following_actor]
result = MultiKey.objects.annotate(key=Concat('actor_type', Value('__'), 'actor_id',
output_field=TextField()))\
.filter(Q(key__in=searchable_keys) | Q(tags__in=following_tag))
I have room booking application with a Booking model that has a simple ForeignKey relationship to my Room model. Here's a super-brief idea of what I'm dealing with:
class Room(..):
floor = models.IntegerField()
number = models.IntegerField()
...
class Booking(..):
room = models.ForeignKey('Room')
...
I've been using django-import-export on the Booking model to let the admin take backups of this data in a way they can import into Excel. It's been really useful for quickly giving them data on demand without costing them my time.
My problem is that the room relationship is ignored; it pumps out the room_id instead of following the relationship. I understand how this makes sense for the sake of importing data but for all practical external purposes, the Room's primary_key is completely useless. People looking at Booking data need to know which floor and number Room it's for.
Is there a simple way to add extra data to the django-import-export's data to essentially denormalise those fields through?
See Getting started in documentation.
When defining ModelResource fields it is possible to follow model relationships:
class BookResource(resources.ModelResource):
class Meta:
model = Book
fields = ('author__name',)
In your case fields would be ('room__floor', 'room__number',)
It's actually not that painful! We need to override a couple of the ImportExportModelAdmin functions on our ModelAdmin to add the extra data to the query, and then pull it through.
class BookingAdmin(ImportExportModelAdmin):
...
def get_resource_class(self):
from import_export.fields import Field
rc = super(BookingAdmin, self).get_resource_class()
for f in ('room_floor', 'room_number'):
rc.fields[f] = Field(f, f, readonly=True)
return rc
def get_export_queryset(self, request):
rs = super(BookingAdmin, self).get_export_queryset(request)
return rs.annotate(
room_floor=F('room__floor'),
room_number=F('room__number')
)
No idea what happens on import though. I've set the fields to readonly so they should be ignored. Thankfully we're past the point where importing matters.
I have models:
class Z(models.Model):
name = ...
class B(model.Model):
something = model...
other = models.ForeignKey(Z)
class A(models.Model):
date = model.DateTimeField()
objs_b = models.ManyToManyField(B)
def get_obj_b(self,z_id):
self.obj_b = self.objs_b.get(other=z_id)
and query:
qs = A.objects.filter(...)
but if I want get object B related to A I must call get_obj_b:
for item in gs:
item.get_obj_b(my_known_z_id)
It was generate many queries. How to do it simple? I can not change models, and generally I must use filter (not my own manager) function.
If you are using Django 1.4, I would suggest that you use prefetch_related like this:
A.objects.all().prefetch_related('objs_b__other')
This would minimize the number of queries to 2: one for model A, and one for 'objs_b' joined with 'other'
And you can combine it with a filter suggested by pastylegs:
A.objects.filter(objs_b__other__id=z_id).prefetch_related('objs_b__other')
For details see: https://docs.djangoproject.com/en/1.4/ref/models/querysets/#prefetch-related
If there are models as below:
class Donation(models.Model):
user = FKey(User)
project = FKey(Project)
...
class Campaign(models.Model):
user = Fkey(User)
project = FKey(Project)
...
class Project(models.Model)
...
What is the simplest django ORM query to find a list of all projects that a given user is associated with.
One direct solution is to obtain all the ids of the projects from both models and query the Projects model for the given ids.
But there has to be better solutions.
The simplest direct ORM query to do this, can be,
from django.db.models import Q
Project.objects.filter(
Q(id__in=Campaign.objects.filter(user=SomeUser).values('project')) |
Q(id__in=Donation.objects.filter(user=SomeUser).values('project'))
)
I think we can't merge Campaign QuerySet and Donation so we have to do it in 2 steps:
Project.objects.filter(id = Campaign.objects.filter(user = searchedUser))
Project.objects.filter(id = Donation.objects.filter(user = searchedUser))
and now u can divorce in your page Campaign and Donation in what user have part so there is advantage :)