I have several models connected to each other with ForeignKeys relationships.
The main one in this sort of hierarchy contains a owner field.
I would like to create a single custom manager for all these models that changes the returned queryset depending on the models that is calling it.
I know that manager can access self.model to get the model that it is attached to.
Class Main(models.Model)
owner=models.ForeignKey (User)
owned = OwnedManager()
Class Second(models.Model)
main=models.ForeignKey('Main')
owned = OwnedManager()
Class Third(models.Model)
second=models.ForeignKey('Second')
owned = OwnedManager()
I would like my Custom Manager to have this sort of behavior:
class OwnedManager(models.Manager):
def get_owned_objs(self, owner):
if self.model == 'Main': # WRONG: How do I get the model name?
owned_main = self.filter(owner=owner)
return owned_main
elif self.model == 'Second':
owned_second = self.filter(main__owner=owner)
return owned_second
else:
owned_third = self.filter(second__main__owner=owner)
return owned_third
In order to have a consistent way to call it across different models, like so:
main_object.owned.get_owned_objs(owner=user1) # of the Model Main
second_object.owned.get_owned_objs(owner=user1) # of the Model Second
third_object.owned.get_owned_objs(owner=user1) # of the Model Third
QUESTION:
self.model == 'Main' is wrong. I don't get the model name like this. Is there a way to get it?
Is this efficient? Do you know a better way to implement this? Maybe Custom Managers Inheritance?
EDIT - MY SOLUTION:
The accepted answer below is a good solution but I also found a way to get the model name of the particular model calling the custom manager, that is:
if self.model.__name__ == 'Main':
The key here is the attribute __name__
1) Make abstract model
class AbstractModel(models.Model):
class Meta(models.Meta):
abstract = True
objects = OwnedManager()
2) Inherit your models from AbstractModel, put some key in meta
class Model(AbstractModel)
class Meta(AbstractModel.Meta):
filter_key = 'some_key'
3) Redesign your OwnedManager
class OwnedManager(models.Manager):
def get_owned_objs(self, owner):
if hasattr(self.model._meta, 'filter_key'):
return self.filter(**{self.model._meta.filter_key: owner})
Now you can use SomeModel.objects.get_owned_objs(owner=user1) in any inherited models, where filter_key is setted without getting models's name.
Related
Using Django 3.2 -- I will simplify the problem as much as I can.
I have three model classes:
# abstract base class
MyAbstractModel(models.Model)
# derived model classes
Person(MyAbstractModel)
LogoImage(MyAbstractModel)
Each Person has:
image = ForeignKey(LogoImage, db_index=True, related_name="person", null=True,
on_delete=models.PROTECT)
The MyAbstractModel defines a few model managers:
objects = CustomModelManager()
objects_all_states = models.Manager()
as well as a state field, that can be either active or inactive
CustomModelManager is defined as something that'll bring only records that have state == 'active':
class CustomModelManager(models.Manager):
def get_queryset(self):
return super().get_query().filter(self.model, using=self._db).filter(state='active')
In my database I have two objects in two tables:
Person ID 1 state = 'active'
Image ID 1 state = 'inactive'
Person ID 1 has a foreign key connection to Image ID 1 via the Person.image field.
------ NOW for the issue ----------------
# CORRECT: gives me the person object
person = Person.objects.get(id=1)
# INCORRECT: I get the image, but it should not work...
image = person.image
Why is that incorrect? because I queried for the person object using the objects model manager which is supposed to bring only those items with active status. It brought the Person which is fine, because Person (ID=1) is state==active -- but the object under person.image is state==inactive. Why am I getting it?
WORKAROND ATTEMPT:
added base_manager_name = "objects" to the MyAbstractModel class Meta: section
ATTEMPTING AGAIN:
# CORRECT: gives me the person object
person = Person.objects.get(id=1)
# CORRECT: gives me a "Does not Exist" exception.
image = person.image
However..... Now I try this:
# CORRECT: getting the person
person.objects_all_states.get(id=1)
# INCORRECT: throws a DoesNotExist, as it's trying to use the `objects` model manager I hard coded in the `MyAbstractModel` class meta.
image = person.image
Since I got the Person under the objects_all_states which does not care about state==active -- I expect I would also get the person.image in a similar way. But that doesn't work as expected.
THE ROOT ISSUE
How do I force the same model manager used to fetch the parent object (Person) -- in the fetching of every single ForeignKey object a Person has? I can't find the answer. I've been going in circles for days. There is simply no clear answer anywhere. Either I am missing something very fundamental, or Django has a design flaw (which of course I don't really believe) -- so, what am I missing here?
Why they don't play well together
Foreign key classes use separate instances of managers, so there's no shared state.
There's no information about the manager used on the parent instance either.
As per django.db.models.Model._base_manager, Django simply uses _base_manager:
return self.field.remote_field.model._base_manager.db_manager(hints=hints).all()
...where hints would be {'instance': <Person: Person object (1)>}.
Since we have a reference to the parent, in some scenarios, we could support this inference.
Fair warning
Django specifically mentions not to do this.
From django.db.models.Model._base_manager:
Don’t filter away any results in this type of manager subclass
This manager is used to access objects that are related to from some other model. In those situations, Django has to be able to see all the objects for the model it is fetching, so that anything which is referred to can be retrieved.
Therefore, you should not override get_queryset() to filter out any rows. If you do so, Django will return incomplete results.
1. How you could implement this inference
You could:
override get() to actively store some information on the instance (that will be passed as hint) about whether an instance of CustomModelManager was used to get it, and then
in get_queryset, check that and try to fallback on objects_all_states.
class CustomModelManager(models.Manager):
def get(self, *args, **kwargs):
instance = super().get(*args, **kwargs)
instance.hint_manager = self
return instance
def get_queryset(self):
hint = self._hints.get('instance')
if hint and isinstance(hint.__class__.objects, self.__class__):
hint_manager = getattr(hint, 'hint_manager', None)
if not hint_manager or not isinstance(hint_manager, self.__class__):
manager = getattr(self.model, 'objects_all_states', None)
if manager:
return manager.db_manager(hints=self._hints).get_queryset()
return super().get_queryset().filter(state='active')
Limitations
One of possibly many edge cases where this wouldn't work is if you queried person via Person.objects.filter(id=1).first().
2. Using explicit instance context
Usage:
person = Person.objects_all_states.get(id=1)
# image = person.image
with CustomModelManager.disable_for_instance(person):
image = person.image
Implementation:
class CustomModelManager(models.Manager):
_disabled_for_instances = set()
#classmethod
#contextmanager
def disable_for_instance(cls, instance):
is_already_in = instance in cls._disabled_for_instances
if not is_already_in:
cls._disabled_for_instances.add(instance)
yield
if not is_already_in:
cls._disabled_for_instances.remove(instance)
def get_queryset(self):
if self._hints.get('instance') in self._disabled_for_instances:
return super().get_queryset()
return super().get_queryset().filter(state='active')
3. Using explicit thread-local context
Usage:
# person = Person.objects_all_states.get(id=1)
# image = person.image
with CustomModelManager.disable():
person = Person.objects.get(id=1)
image = person.image
Implementation:
import threading
from contextlib import contextmanager
from django.db import models
from django.utils.functional import classproperty
class CustomModelManager(models.Manager):
_data = threading.local()
#classmethod
#contextmanager
def disable(cls):
is_disabled = cls._is_disabled
cls._data.is_disabled = True
yield
cls._data.is_disabled = is_disabled
#classproperty
def _is_disabled(cls):
return getattr(cls._data, 'is_disabled', None)
def get_queryset(self):
if self._is_disabled:
return super().get_queryset()
return super().get_queryset().filter(state='active')
Well, i must point out a few design flaws in your approach.
First - you should not override get_queryset method for manager. Instead, make a separate method to filter specific cases. Even better if you make a custom QuerySet class with those methods, since then you will able to chain them
class ActiveQuerySet(QuerySet):
def active(self):
return self.filter(state="active")
# in your model
objects = ActiveQueryset.as_manager()
Also, you should not place field state in every model and expect, that Django will handle this for you. It will be much easier to handle for you if you decide from domain perspective, which model is your root model and have state there. For example, if Person can be inactive, then probably all of his images are also inactive, so you may safely assume, that Persons status is shared by all related models.
I would specifically look for a way to avoid such issue from design perspective, instead of trying to brutforce Django to process such filtration cases
I'm writing a decorator intended to add functionality to decorated Django Model classes. Something like this:
class NewFunctionality:
#classmethod
def fun1(cls):
...
#property
def prop1(self):
...
def add_functionality(Decorated):
class NewClass(Decorated, NewFunctionality):
class Meta(getattr(Decorated, 'Meta', object)):
app_label = Decorated._meta.app_label
NewClass.__name__ = Decorated.__name__
NewClass.__doc__ = Decorated.__doc__
return NewClass
#add_functionality
class MyModel(models.Model):
...
This seems to work until there are two decorated model classes, when I get an error Conflicting 'modelclass' models in application 'my_app'.
This is apparently due to the registry of models that Django keeps, which clearly has some automagic that doesn't appreciate new model classes being made, even if they are direct replacements of the existing one.
Is there anything I can do to accomplish this, other than by monkeypatching the decorated class, adding each needed method?
Edit
I've avoided the error by making the wrapper class into a proxy class of the decorated class:
def add_functionality(Decorated):
class NewClass(Decorated, NewFunctionality):
class Meta(getattr(Decorated, 'Meta', object)):
app_label = Decorated._meta.app_label
proxy = True
NewClass.__name__ = Decorated.__name__
NewClass.__doc__ = Decorated.__doc__
return NewClass
I also replace the use of the class keyword, using builtins.type, as in:
def add_functionality(Decorated):
return type(
Decorated.__name__,
(NewFunctionality, Decorated,),
{
'Meta': type('Meta', (object,), {
'app_label': Decorated._meta.app_label,
'proxy': True,
'__module__': Decorated.__module__,
}
)
I really like this as it seemingly completely replaces the decorated class. Now, however, I'm getting the following warning from Django:
.venv/lib/python3.9/site-packages/django/db/models/base.py:321
/home/tyler/orm-cache/django-ormsgpack/.venv/lib/python3.9/site-packages/django/db/models/base.py:321: RuntimeWarning: Model 'my_app.atestmodel' was already registered. Reloading models is not advised as it can lead to inconsistencies, most notably with related models.
new_class._meta.apps.register_model(new_class._meta.app_label, new_class)
If anyone can shed light onto how I might utilize the Django api to properly overwrite the registered model, I'd love to hear it!
thanks
I have the following structures
class State(models.Model):
label = models.CharField(max_length=128)
....
class ReviewState(models.Model):
state = models.ForeignKey(State, on_delete=models.CASCADE)
...
class MySerializer(serializers.HyperlinkedModelSerializer):
state = serializers.SlugRelatedField(queryset=ReviewState.objects.all(), slug_field='state__label', required=False)
class Meta:
model = MyModel
fields = [
'id',
'state', # this points to a ReviewState object
....
]
What I'm trying to do is using the State object's label as the field instead. But it doesn't seem like djangorestframework likes the idea of using __ to lookup slug fields. Would it be possible to do this? If it was:
class MySerializer(serializers.HyperlinkedModelSerializer):
state = serializers.SlugRelatedField(queryset=State.objects.all(), slug_field='label', required=False)
that would be no problem, but I'm trying to use the ReviewState instead. I'm also trying to avoid having a ReviewStateSerializer as the resulting json would look like this
{...
'state': {'state': 'Pending'}}
}
Interesting question, and well put.
Using SlugRelatedField('state__label', queryset=...) works fine, with 1 caveat: its just calling queryset.get(state__label="x") which errors if there isn't exactly 1 match.
1) Write a custom field?
Inherit from SlugRelatedField and override to_internal_value(), maybe by calling .first() instead of .get(), or whatever other logic you need.
2) Re-evaluate this relationship, maybe its 1-to-1? a choice field?
I'm a bit confused on how this would all work, since you can have a "1 to many" with State => ReviewState. The default lookup (if you don't do #1) will throw an error when multiple matches occur.
Maybe this is a 1-to-1 situation with the model? Perhaps the ReviewState can use a ChoiceField instead of a table of states?
Perhaps the 'label' can be the PK of the State table, and also a SlugField rather than a non-unique CharField?
3) Write different serializers for the List and Create cases
DRF doesn't give us a built-in way to do this, but this reliance on "one serializer to do it all" is the cause of a lot of problems I see on SO. Its just really hard to get what you want without having different serializers for different cases. It's not hard to roll-your-own mixin to do it, but here's an example which uses an override:
from rest_framework import serializers as s
class MyCreateSerializer(s.ModelSerializer):
state = s.SlugRelatedField(...)
...
class MyListSerializer(s.ModelSerializer):
# use dotted notation, serializers read *object* attributes
state = s.CharField(source="state.state.label")
...
class MyViewSet(ModelViewSet):
queryset = MyModel.objects.select_related('state__state')
...
def get_serializer_class(self):
if self.action == "create":
return MyCreateSerializer
else:
return MyListSerializer
I have a private boolean flag on my model, and a custom manager that overwrites the get_query_set method, with a filter, removing private=True:
class myManager(models.Manager):
def get_query_set(self):
qs = super(myManager, self).get_query_set()
qs = qs.filter(private=False)
return qs
class myModel(models.Model):
private = models.BooleanField(default=False)
owner = models.ForeignKey('Profile', related_name="owned")
#...etc...
objects = myManager()
I want the default queryset to exclude the private models be default as a security measure, preventing accidental usage of the model showing private models.
Sometimes, however, I will want to show the private models, so I have the following on the manager:
def for_user(self, user):
if user and not user.is_authenticated():
return self.get_query_set()
qs = super(myManager, self).get_query_set()
qs = qs.filter(Q(owner=user, private=True) | Q(private=False))
return qs
This works excellently, with the limitation that I can't chain the filter. This becomes a problem when I have a fk pointing the myModel and use otherModel.mymodel_set. otherModel.mymodel_set.for_user(user) wont work because mymodel_set returns a QuerySet object, rather than the manager.
Now the real problem starts, as I can't see a way to make the for_user() method work on a QuerySet subclass, because I can't access the full, unfiltered queryset (basically overwriting the get_query_set) form the QuerySet subclass, like I can in the manager (using super() to get the base queryset.)
What is the best way to work around this?
I'm not tied to any particular interface, but I would like it to be as djangoy/DRY as it can be. Obviously I could drop the security and just call a method to filter out private tasks on each call, but I really don't want to have to do that.
Update
manji's answer below is very close, however it fails when the queryset I want isn't a subset of the default queryset. I guess the real question here is how can I remove a particular filter from a chained query?
Define a custom QuerySet (containing your custom filter methods):
class MyQuerySet(models.query.QuerySet):
def public(self):
return self.filter(private=False)
def for_user(self, user):
if user and not user.is_authenticated():
return self.public()
return self.filter(Q(owner=user, private=True) | Q(private=False))
Define a custom manager that will use MyQuerySet (MyQuerySet custom filters will be accessible as if they were defined in the manager[by overriding __getattr__]):
# A Custom Manager accepting custom QuerySet
class MyManager(models.Manager):
use_for_related_fields = True
def __init__(self, qs_class=models.query.QuerySet):
self.queryset_class = qs_class
super(QuerySetManager, self).__init__()
def get_query_set(self):
return self.queryset_class(self.model).public()
def __getattr__(self, attr, *args):
try:
return getattr(self.__class__, attr, *args)
except AttributeError:
return getattr(self.get_query_set(), attr, *args)
Then in the model:
class MyModel(models.Model):
private = models.BooleanField(default=False)
owner = models.ForeignKey('Profile', related_name="owned")
#...etc...
objects = myManager(MyQuerySet)
Now you can:
¤ access by default only public models:
MyModel.objects.filter(..
¤ access for_user models:
MyModel.objects.for_user(user1).filter(..
Because of (use_for_related_fields = True), this same manager wil be used for related managers. So you can also:
¤ access by default only public models from related managers:
otherModel.mymodel_set.filter(..
¤ access for_user from related managers:
otherModel.mymodel_set.for_user(user).filter(..
More informations: Subclassing Django QuerySets & Custom managers with chainable filters (django snippet)
To use the chain you should override the get_query_set in your manager and place the for_user in your custom QuerySet.
I don't like this solution, but it works.
class CustomQuerySet(models.query.QuerySet):
def for_user(self):
return super(CustomQuerySet, self).filter(*args, **kwargs).filter(private=False)
class CustomManager(models.Manager):
def get_query_set(self):
return CustomQuerySet(self.model, using=self._db)
If you need to "reset" the QuerySet you can access the model of the queryset and call the original manager again (to fully reset). However that's probably not very useful for you, unless you were keeping track of the previous filter/exclude etc statements and can replay them again on the reset queryset. With a bit of planning that actually wouldn't be too hard to do, but may be a bit brute force.
Overall manji's answer is definitely the right way to go.
So amending manji's answer you need to replace the existing "model"."private" = False with ("model"."owner_id" = 2 AND "model"."private" = True ) OR "model"."private" = False ). To do that you will need to walk through the where object on the query object of the queryset to find the relevant bit to remove. The query object has a WhereNode object that represents the tree of the where clause, with each node having multiple children. You'd have to call the as_sql on the node to figure out if it's the one you are after:
from django.db import connection
qn = connection.ops.quote_name
q = myModel.objects.all()
print q.query.where.children[0].as_sql(qn, connection)
Which should give you something like:
('"model"."private" = ?', [False])
However trying to do that is probably way more effort than it's worth and it's delving into bits of Django that are probably not API-stable.
My recommendation would be to use two managers. One that can access everything (an escape hatch of sort), the other with the default filtering applied. The default manager is the first one, so you need to play around with the ordering depending on what you need to do. Then restructure your code to know which one to use - so you don't have the problem of having the extra private=False clause in there already.
If I have queries on multiple tables like:
d = Relations.objects.filter(follow = request.user).filter(date_follow__lt = last_checked)
r = Reply.objects.filter(reply_to = request.user).filter(date_reply__lt = last_checked)
article = New.objects.filter(created_by = request.user)
vote = Vote.objects.filter(voted = article).filter(date__lt = last_checked)
and I want to display the results from all of them ordered by date (I mean not listing all the replies, then all the votes, etc ).
Somehow, I want to 'join all these results', in a single queryset.
Is there possible?
It seems like you need different objects to have common operations ...
1) In this case it might be better to abstract these properties in a super class... I mean that you could have an Event class that defines a user field, and all your other event classes would subclass this.
class Event(model.Model):
user = models.ForeignKey(User)
date = ...
class Reply(Event):
#additional fields
class Vote(Event):
#additional fields
Then you would be able to do the following
Event.objects.order_by("date") #returns both Reply, Vote and Event
Check-out http://docs.djangoproject.com/en/1.2/topics/db/models/#id5 for info on model inheritance.
2) You could also have an Event model with a generic relation to another object. This sounds cleaner to me as a Vote is conceptually not an "event". Check-out : http://docs.djangoproject.com/en/dev/ref/contrib/contenttypes/#id1
Anyway, I think your problem is a matter of design
In addition to to Sebastien's proposal number 2: Django actually has some built-in functionality that you could "abuse" for this; for the admin it has already a model that logs the user's actions and references the objects through a generic foreign key relation, I think you could just sub-class this model and use it for your purposes:
from django.contrib.admin.models import LogEntry, ADDITION
from django.utils.encoding import force_unicode
from django.contrib.contenttypes.models import ContentType
class MyLog(LogEntry):
class Meta(LogEntry.Meta):
db_table_name = 'my_log_table' #use another name here
def log_addition(request, object):
LogEntry.objects.log_action(
user_id = request.user.pk,
content_type_id = ContentType.objects.get_for_model(object).pk,
object_id = object.pk,
object_repr = force_unicode(object),
action_flag = ADDITION
)
You can now log all your notifications etc. where they happen with with log_addition(request, object) and filter the Log table than for your purposes! If you want to log also changes / deletions etc. you can make yourself some helper functions for that!