there are couple of queries which are important and used several times in different part of application. I wounder it would be great to have a method in the Model itself to retrieve requested data as needed. tried Classmethod and instance method but as the Manager is called in the query I got below error:
AttributeError: Manager isn't accessible via ... instances
code is look like below:
class XDL(models.Model):
name = models.CharField(max_length = 20)
Order_Code_Description = models.CharField(max_length = 355)
Status = models.CharField(max_length = 20)
Supplier = models.CharField(max_length = 100)
#classmethod
def get_count_xdls(cls):
return cls.objects.all() #query example
def get_all_supps(self):
return self.objects.all() #query example
explained in Django documentation below:
Managers are accessible only via model classes, rather than from model instances, to enforce a separation between “table-level” operations and “record-level” operations
Is there a best practice to store important queries in a method somewhere in Model (or other parts)? as these queries run several time and also writing it as a method help the documentation as well.
A customer manager is a powerful tool for code factorization and labeling complex query.
class XDLManager(models.Manager):
pass
class XDLQuerySet(models.QuerySet):
def supplied_by_foo(self):
return self.filter(supplier='Foo')
class XDL(models.Model):
objects = XDLManager.from_queryset(XDLQuerySet)()
Any methods defined in XDLManager will only be accessible behind XDL.objects, methods defined in XDLQueryset will be accesible both behind XDL.objects and any function that returns a queryset (XDL.objects.all() for instance).
XDL.objects.filter(supplier='Foo') and XDL.objects.supplied_by_foo() are equivalent
Related
Using Django 3.2 -- I will simplify the problem as much as I can.
I have three model classes:
# abstract base class
MyAbstractModel(models.Model)
# derived model classes
Person(MyAbstractModel)
LogoImage(MyAbstractModel)
Each Person has:
image = ForeignKey(LogoImage, db_index=True, related_name="person", null=True,
on_delete=models.PROTECT)
The MyAbstractModel defines a few model managers:
objects = CustomModelManager()
objects_all_states = models.Manager()
as well as a state field, that can be either active or inactive
CustomModelManager is defined as something that'll bring only records that have state == 'active':
class CustomModelManager(models.Manager):
def get_queryset(self):
return super().get_query().filter(self.model, using=self._db).filter(state='active')
In my database I have two objects in two tables:
Person ID 1 state = 'active'
Image ID 1 state = 'inactive'
Person ID 1 has a foreign key connection to Image ID 1 via the Person.image field.
------ NOW for the issue ----------------
# CORRECT: gives me the person object
person = Person.objects.get(id=1)
# INCORRECT: I get the image, but it should not work...
image = person.image
Why is that incorrect? because I queried for the person object using the objects model manager which is supposed to bring only those items with active status. It brought the Person which is fine, because Person (ID=1) is state==active -- but the object under person.image is state==inactive. Why am I getting it?
WORKAROND ATTEMPT:
added base_manager_name = "objects" to the MyAbstractModel class Meta: section
ATTEMPTING AGAIN:
# CORRECT: gives me the person object
person = Person.objects.get(id=1)
# CORRECT: gives me a "Does not Exist" exception.
image = person.image
However..... Now I try this:
# CORRECT: getting the person
person.objects_all_states.get(id=1)
# INCORRECT: throws a DoesNotExist, as it's trying to use the `objects` model manager I hard coded in the `MyAbstractModel` class meta.
image = person.image
Since I got the Person under the objects_all_states which does not care about state==active -- I expect I would also get the person.image in a similar way. But that doesn't work as expected.
THE ROOT ISSUE
How do I force the same model manager used to fetch the parent object (Person) -- in the fetching of every single ForeignKey object a Person has? I can't find the answer. I've been going in circles for days. There is simply no clear answer anywhere. Either I am missing something very fundamental, or Django has a design flaw (which of course I don't really believe) -- so, what am I missing here?
Why they don't play well together
Foreign key classes use separate instances of managers, so there's no shared state.
There's no information about the manager used on the parent instance either.
As per django.db.models.Model._base_manager, Django simply uses _base_manager:
return self.field.remote_field.model._base_manager.db_manager(hints=hints).all()
...where hints would be {'instance': <Person: Person object (1)>}.
Since we have a reference to the parent, in some scenarios, we could support this inference.
Fair warning
Django specifically mentions not to do this.
From django.db.models.Model._base_manager:
Don’t filter away any results in this type of manager subclass
This manager is used to access objects that are related to from some other model. In those situations, Django has to be able to see all the objects for the model it is fetching, so that anything which is referred to can be retrieved.
Therefore, you should not override get_queryset() to filter out any rows. If you do so, Django will return incomplete results.
1. How you could implement this inference
You could:
override get() to actively store some information on the instance (that will be passed as hint) about whether an instance of CustomModelManager was used to get it, and then
in get_queryset, check that and try to fallback on objects_all_states.
class CustomModelManager(models.Manager):
def get(self, *args, **kwargs):
instance = super().get(*args, **kwargs)
instance.hint_manager = self
return instance
def get_queryset(self):
hint = self._hints.get('instance')
if hint and isinstance(hint.__class__.objects, self.__class__):
hint_manager = getattr(hint, 'hint_manager', None)
if not hint_manager or not isinstance(hint_manager, self.__class__):
manager = getattr(self.model, 'objects_all_states', None)
if manager:
return manager.db_manager(hints=self._hints).get_queryset()
return super().get_queryset().filter(state='active')
Limitations
One of possibly many edge cases where this wouldn't work is if you queried person via Person.objects.filter(id=1).first().
2. Using explicit instance context
Usage:
person = Person.objects_all_states.get(id=1)
# image = person.image
with CustomModelManager.disable_for_instance(person):
image = person.image
Implementation:
class CustomModelManager(models.Manager):
_disabled_for_instances = set()
#classmethod
#contextmanager
def disable_for_instance(cls, instance):
is_already_in = instance in cls._disabled_for_instances
if not is_already_in:
cls._disabled_for_instances.add(instance)
yield
if not is_already_in:
cls._disabled_for_instances.remove(instance)
def get_queryset(self):
if self._hints.get('instance') in self._disabled_for_instances:
return super().get_queryset()
return super().get_queryset().filter(state='active')
3. Using explicit thread-local context
Usage:
# person = Person.objects_all_states.get(id=1)
# image = person.image
with CustomModelManager.disable():
person = Person.objects.get(id=1)
image = person.image
Implementation:
import threading
from contextlib import contextmanager
from django.db import models
from django.utils.functional import classproperty
class CustomModelManager(models.Manager):
_data = threading.local()
#classmethod
#contextmanager
def disable(cls):
is_disabled = cls._is_disabled
cls._data.is_disabled = True
yield
cls._data.is_disabled = is_disabled
#classproperty
def _is_disabled(cls):
return getattr(cls._data, 'is_disabled', None)
def get_queryset(self):
if self._is_disabled:
return super().get_queryset()
return super().get_queryset().filter(state='active')
Well, i must point out a few design flaws in your approach.
First - you should not override get_queryset method for manager. Instead, make a separate method to filter specific cases. Even better if you make a custom QuerySet class with those methods, since then you will able to chain them
class ActiveQuerySet(QuerySet):
def active(self):
return self.filter(state="active")
# in your model
objects = ActiveQueryset.as_manager()
Also, you should not place field state in every model and expect, that Django will handle this for you. It will be much easier to handle for you if you decide from domain perspective, which model is your root model and have state there. For example, if Person can be inactive, then probably all of his images are also inactive, so you may safely assume, that Persons status is shared by all related models.
I would specifically look for a way to avoid such issue from design perspective, instead of trying to brutforce Django to process such filtration cases
I have several models connected to each other with ForeignKeys relationships.
The main one in this sort of hierarchy contains a owner field.
I would like to create a single custom manager for all these models that changes the returned queryset depending on the models that is calling it.
I know that manager can access self.model to get the model that it is attached to.
Class Main(models.Model)
owner=models.ForeignKey (User)
owned = OwnedManager()
Class Second(models.Model)
main=models.ForeignKey('Main')
owned = OwnedManager()
Class Third(models.Model)
second=models.ForeignKey('Second')
owned = OwnedManager()
I would like my Custom Manager to have this sort of behavior:
class OwnedManager(models.Manager):
def get_owned_objs(self, owner):
if self.model == 'Main': # WRONG: How do I get the model name?
owned_main = self.filter(owner=owner)
return owned_main
elif self.model == 'Second':
owned_second = self.filter(main__owner=owner)
return owned_second
else:
owned_third = self.filter(second__main__owner=owner)
return owned_third
In order to have a consistent way to call it across different models, like so:
main_object.owned.get_owned_objs(owner=user1) # of the Model Main
second_object.owned.get_owned_objs(owner=user1) # of the Model Second
third_object.owned.get_owned_objs(owner=user1) # of the Model Third
QUESTION:
self.model == 'Main' is wrong. I don't get the model name like this. Is there a way to get it?
Is this efficient? Do you know a better way to implement this? Maybe Custom Managers Inheritance?
EDIT - MY SOLUTION:
The accepted answer below is a good solution but I also found a way to get the model name of the particular model calling the custom manager, that is:
if self.model.__name__ == 'Main':
The key here is the attribute __name__
1) Make abstract model
class AbstractModel(models.Model):
class Meta(models.Meta):
abstract = True
objects = OwnedManager()
2) Inherit your models from AbstractModel, put some key in meta
class Model(AbstractModel)
class Meta(AbstractModel.Meta):
filter_key = 'some_key'
3) Redesign your OwnedManager
class OwnedManager(models.Manager):
def get_owned_objs(self, owner):
if hasattr(self.model._meta, 'filter_key'):
return self.filter(**{self.model._meta.filter_key: owner})
Now you can use SomeModel.objects.get_owned_objs(owner=user1) in any inherited models, where filter_key is setted without getting models's name.
I'm trying to figure out how to design my model. I've been going over the documentation, and it ultimately seems like I should be using the "through" attribute, but I just can't figure out how to get it to work how I want.
If someone could take a look and point out what I'm missing, that would be really helpful. I have pasted my model below.
This is what I am trying to do:
1) Have a list of server types
2) Each server type will need to have different parts available to that specific server type
3) The asset has a FK to the servermodel, which has a M2M to the parts specific to that server type.
My question is, how can each "Asset" store meta data for each "Part" specific to that "Asset"? For example, each "Asset" should have it's own last_used data for the part that's assigned to it.
Thanks! :)
class Part(models.Model):
part_description = models.CharField(max_length=30,unique=1)
last_used = models.CharField(max_length=30)
def __unicode__(self):
return self.part_description
class ServerModel(models.Model):
server_model = models.CharField(max_length=30,unique=1)
parts = models.ManyToManyField(Part)
def __unicode__(self):
return self.server_model
class Asset(models.Model):
server_model = models.ForeignKey(ServerModel)
serial_number = models.CharField(max_length=10,unique=1)
def __unicode__(self):
return self.server_model.server_model
EDIT:
Thank you for the help!
I may have not explained myself clearly, though. It's probably my confusing model names.
Example:
ServerModel stores the type of server being used, say "Dell Server 2000".
The "Dell Server 2000" should be assigned specific parts:
"RAM"
"HARD DISK"
"CDROM"
Then, I should be able to create 10x Assets with a FK to the ServerModel. Now, each of these assets should be able to mark when the "RAM" part was last used for this specific asset.
I'm not sure I exactly understand what you want to do, but basically you can solve that with a "through" model, as you expected:
import datetime
class Part(models.Model):
name = models.CharField(max_length=30,unique=1)
class ServerModel(models.Model):
server_model = models.CharField(max_length=30,unique=1)
parts = models.ManyToManyField(Part,through='Asset')
class Asset(models.Model):
server_model = models.ForeignKey(ServerModel)
part = models.ForeignKey(Part)
serial_number = models.CharField(max_length=10,unique=1)
used = models.DateTimeField(default=datetime.datetime.now())
First thing to notice is the relation of the parts to the servermodel using the "through"-model: that way for each Part instance assigned to the "parts"-property of a ServerModel instance a new Asset instance is created (Phew - hope that doesn't sound too complicated). At the time of creation the "used"-property of the Asset instance is set to the current date and time (thats what default=datetime.datetime.now() does).
If you do that, you can then just query the database for the last asset containing your part. That queryset can then be sorted by the "used" property of the Asset model, which is the date when the Asset instance has been created.
ServerModel.objects.filter(parts__name='ThePartYouAreLookingFor').order_by('asset__used')
I'm not absolutely sure if the queryset is correct, so if someone finds huge nonsense in it, feel free to edit ;)
edit:
The models above do not exactly that. But you do not even need a through model for what you want:
class ServerModel(models.Model):
server_model = models.CharField(max_length=30,unique=1)
parts = models.ManyToManyField(Part)
class Asset(models.Model):
server_model = models.ForeignKey(ServerModel)
parts = models.ForeignKey(Part)
serial_number = models.CharField(max_length=10,unique=1)
used = models.DateTimeField(default=datetime.datetime.now())
Basically you can just add assets and then query all assets that have a RAM in parts.
Asset.objects.filter(parts__contains='RAM').order_by('used')
Get the date of the first (or last) result of that queryset and you have the date of the last usage of your 'RAM'-part.
My problem is that I don't really know how to put my design in terms of a relational db.
I have a class 'Feat', which should contain several other classes, namely lists of 'Stat's, 'Skill's and 'Attribute's. The exact number of those depends on the instance of 'Feat', so I can't just define a static number of fields.
In normal Python, I'd simply use lists:
class Feat():
name
desc
att_effects = []
skill_effects = []
stat_effects = []
I tried to replicate this in Django using the Many-to-Many relationship, but that didn't quite work out.
class Attribute_Name(models.Model):
name = models.CharField(max_length=100)
desc = models.TextField()
def __unicode__(self):
return str(self.name)
...
class Attribute(models.Model):
name = models.ForeignKey(Attribute_Name)
value = models.IntegerField()
def __unicode__(self):
return "%s: %s" % (self.name, self.value)
...
class Feat(models.Model):
name = models.CharField(max_length=100)
desc = models.TextField()
att_effects = models.ManyToManyField(Attribute)
skill_effects = models.ManyToManyField(Skill)
stat_effects = models.ManyToManyField(Stat)
With these models, I could create new 'Attribute's when creating a new 'Feat' in the admin interface. However, I would always see all existing 'Attribute's, not just those pertaining to my new 'Feat'. Is there a way to restrict the view to only those created along with the new 'Feat'? Something like a back-link to the 'Feat' which created the 'Attribute'?
Basically, when I create or edit a 'Feat', I should only be able to use 'Attribute's created from the page of this instance or add a new 'Attribute' instance. I don't want 'Attribute's belonging to another 'Feat' (or another unrelated class) to show up.
Note that I don't want to add a field to the 'Attribute' class, since it should also be used along with other classes (eg 'Character')
I'd also like to do this within the admin interface, since it saves me quite a bit of work.
On a related note: Is there a way to leave a Many-to-Many field empty? Or should I just create a magic 'empty' value and set it as the default?
"there a way to restrict the view to only those created along with the new 'Feat'? Something like a back-link to the 'Feat' which created the 'Attribute'?"
Yes: use the queryset feature of ModelAdmin as described in this question
I have a base LoggedEvent model and a number of subclass models like follows:
class LoggedEvent(models.Model):
user = models.ForeignKey(User, blank=True, null=True)
timestamp = models.DateTimeField(auto_now_add=True)
class AuthEvent(LoggedEvent):
good = models.BooleanField()
username = models.CharField(max_length=12)
class LDAPSearchEvent(LoggedEvent):
type = models.CharField(max_length=12)
query = models.CharField(max_length=24)
class PRISearchEvent(LoggedEvent):
type = models.CharField(max_length=12)
query = models.CharField(max_length=24)
Users generate these events as they do the related actions. I am attempting to generate a usage-report of how many of each event-type each user has caused in the last month. I am struggling with Django's ORM and while I am close I am running into a problem. Here is the query code:
def usage(request):
# Calculate date range
today = datetime.date.today()
month_start = datetime.date(year=today.year, month=today.month - 1, day=1)
month_end = datetime.date(year=today.year, month=today.month, day=1) - datetime.timedelta(days=1)
# Search for how many LDAP events were generated per user, last month
baseusage = User.objects.filter(loggedevent__timestamp__gte=month_start, loggedevent__timestamp__lte=month_end)
ldapusage = baseusage.exclude(loggedevent__ldapsearchevent__id__lt=1).annotate(count=Count('loggedevent__pk'))
authusage = baseusage.exclude(loggedevent__authevent__id__lt=1).annotate(count=Count('loggedevent__pk'))
return render_to_response('usage.html', {
'ldapusage' : ldapusage,
'authusage' : authusage,
}, context_instance=RequestContext(request))
Both ldapusage and authusage are both a list of users, each user annotated with a .count attribute which is supposed to represent how many particular events that user generated. However in both lists, the .count attributes are the same value. Infact the annotated 'count' is equal to how many events that user generated, regardless of type. So it would seem that my specific
authusage = baseusage.exclude(loggedevent__authevent__id__lt=1)
isn't excluding by subclass. I have tried id__lt=1, id__isnull=True, and others. Halp.
The key to Django model inheritance is remembering that with a non-abstract base class everything is really an instance of the base class which might happen to have some extra data strapped on the side from a separate table. This means that when you do searches on the base table you get back instances of the base class and there's no way to tell which subclass it is without doing repeated database queries on the subclass tables to see if they contain a record with a matching key ("I have an event. Does it have a record in AuthEvent? No. What about LDAP Event?…"). Among other things this means that you can't easily filter on them in normal queries on the base class without doing a join on every subclass table.
You have a couple of choices: one would simply be to do your queries on the subclass and tally the results (ldap_event_count = LDAPEvent.objects.filter(user=foo).count(), …), which might be sufficient for a single report. I usually recommend adding a content type field to the base class so you can efficiently tell which particular subclass an instance is without having to do another query:
content_type = models.ForeignKey("contenttypes.ContentType")
That allows two major improvements: the most common one is that you can deal with many Events generically without having to do something like hit the subclass-specific accessors (e.g. event.authevent or event.ldapevent) and handling DoesNotExist. In this case it would also make it trivial to rewrite your query since you could just do something like Event.objects.aggregate(Count("content_type")) to get the report values, which becomes particularly handy if your logic gets more complicated ("Event is Auth or LDAP and …").