Right now i'm reading about custom managers that you can use to add additional logic when doing a CRUD action like Create. You make a custom manager class and then initialize the objects attribute of the table class with an instance of the custom manager class.
I've also read that you can also use the pre_save and post_save signals to add additional save logic to the save action. My question is: When should i use a custom manager class over signals and is the use of a custom manager slower then the default manager of the Model class?
Thank you
OK, here's one I wrote earlier. I've cut out a lot of irrelevant stufff, hoping all the essentials remain. You need to know there's a Foreign Key chain PRSBlock -> PRS2 -> Jobline -> Description. Using this, all queries for PRSBlock.objects.filter(... will return PRSBlock objects with extra fields membrane_t, substrate_t and material from the Description found by following this chain. They're almost always needed in this context, unlike most of the other things along that chain. select_related would be serious overkill.
class PRSBlock_Manager(models.Manager):
def get_queryset(self):
return super().get_queryset().annotate(
membrane_t = F('PRS__jobline__description__mt'),
substrate_t= F('PRS__jobline__description__ft'),
material = F('PRS__jobline__description__material')
)
class PRSblock( models.Model):
# override the default manager
objects=PRSBlock_Manager() # annotate always with description parametrs relevant to block LUTs
PRS = models.ForeignKey( PRS2, models.CASCADE, related_name='prs_blocks')
# and lots of other fields that aren't relevant
Related
I have a medium size Django REST app that I'm looking to add gamification features to.
The application in question is a school webapp where students can create mockup quizzes, participate in exams that teachers publish, and write didactical content that gets voted by other students and teachers.
I want to add some gamification features to make the app more interesting and to incentivize participation and usage of the various features: for example, each student will have a personal "reputation" score, and gain points upon completing certain actions--a student may gain points when completing a quiz with a high score, when submitting some content, or when receiving upvotes to such content.
The tricky part is I want to be able to have this logic be as separate as possible from the existing codebase, for various reasons: separation of concerns, ability to plug the engine in/out if needed, ability to easily deactivate features for certain groups of users, etc.
What I'm looking for here is some software engineering advice that's also Django-specific. Here's a high level description of what I'm thinking of doing--I'd like some advice on the approach.
create a new gamification app. Here I will have models that describe a change in reputation for a user and possibly other related events. The app should also send notifications when gamification-related events occur
from the gamification app, expose a callback-based interface, which the other primary app can call into to dispatch events
use the django-lifecycle package to call the callbacks from gamification when triggers occur.
This way, my existing models would only get touched to register the triggers from django-lifecycle (similar to signals). For example, let's say I want to give students points when they turn in an assignment. Let's say I have an AssignmentSubmission model to handle assignment submissions. With the added lifecycle hook, it'd look like this:
class AssignmentSubmission(models.Model):
NOT_TURNED_IN = 0
TURNED_IN = 1
STATES = ((NOT_TURNED_IN, 'NOT_TURNED_IN'), (TURNED_IN, 'TURNED_IN'))
user = models.ForeignKey(user)
assignment = models.ForeignKey(assignment)
state = models.PositiveSmallIntegerField(choices=STATES, default=NOT_TURNED_IN)
#hook(AFTER_UPDATE, when="state", was=NOT_TURNED_IN, is_now=TURNED_IN)
def on_turn_in(self):
get_gamification_interface().on_assignment_turn_in(self.user)
The on_assignment_turn_in method might look something like:
def on_assignment_turn_in(user):
ReputationIncrease.objects.create(user, points=50)
notifications.notify(user, "You gained 50 points")
This is pretty much just a sketch to give an idea.
I am unsure how get_gamification_interface() would work. Should it return a singleton? Maybe instantiate an object? Or return a class with static methods? I think it'd be best to have a getter like this as opposed to manually importing methods from the gamification app, but maybe it could also create too much overhead.
What's a good way to handle adding "pluggable" features to a project that are inherently entangled with existing models and business logic while also touching those as little as possible?
The foreign key approach is fine. You can easily chain and link queries using information from existing tables and you could even avoid touching the original code by importing your models to the new app. You can use Django signals in your new app and ditch the django-lifecycle extension to avoid adding lines to your core models. I used the following approach to keep track of modified records in a table; take a TrackedModel with fields field_one, field_two, field_n... which will be tracked by one of your new app's model, namely RecordTrackingModel:
from parent_app.models import TrackedModel # The model you want to track from a parent app.
from django.db.models.signals import post_save # I'm choosing post_save just to illustrate.
from django.dispatch import receiver
from datetime import datetime
class RecordTrackingModel(models.Model):
record = models.ForeignKey(TrackedModel, verbose_name=("Tracked Model"), on_delete=models.CASCADE)
field_one = models.TextField(verbose_name=("Tracked Field One"), null=True, blank=True) # Use same field type as original
field_two = models.TextField(("Tracked Field Two"))
field_n = ...
notes = models.TextField(verbose_name=("notes"), null=True, blank=True)
created = models.DateTimeField(verbose_name=("Record creation date"), auto_now=False, auto_now_add=True)
#receiver(post_save, sender=TrackedModel) # Here, listen for the save signal coming from a saved or updated TrackedModel record.
def modified_records(instance, **kwargs):
record = instance
tracked_field_one = instance.field_one
tracked_field_two = instance.field_two
tracked_field_n = another_function(instance.field_n) #an external function that could take a field content as input.
...
note = 'Timestamp: ' + str(datetime.now().isoformat(timespec='seconds'))
track_record = RecordTrackingModel.objects.create(record=record, field_one=tracked_field_one, field_two=tracked_field_two, field_n=tracked_field_n, ..., notes=note)
return track_record
There's no need to add functions to your pre-existing models as the signal dispatcher triggers whenever a save or delete signal appears at TrackedModel. Then you could place "if" statements for wether or not to perform actions based on field values, i.e.: just pass if an AssignmentSubmission record has a "Not Turned In" status.
Check Django signals reference for more information about when they trigger.
Additionally, I would suggest to change the "state" field to boolean type and rename it to something more explicit like "is_turned_in" for ease of use. It will simplify your forms and code. Edit: For more than 2 choices (non-boolean), I prefer using ForeignKey instead. It will let you modify choices over time easily from the admin site.
Edit:
Another approach could be mirroring the original models in your gamification app and call for a mirror record update when a save method is used in the original model.
gamification_app/models.py:
from parent_app.models import OriginalModel # The model you want to track from a parent app.
from django.db.models.signals import post_save # I'm choosing post_save just to illustrate.
from django.dispatch import receiver
from datetime import datetime
def gamification_function(input, parameters):
output = *your gamification logic*
return output
class MirrorModel(models.Model):
original_model = (OriginalModel, verbose_name=("Original Model"), on_delete=models.CASCADE)
field_one = ... #Same type as the original
field_two = ...
field_n = ...
#hook(AFTER_UPDATE, when="field_n", was=OPTION_1, is_now=OPTION_2)
def on_turn_in(self):
gamification_function(self.field, self.other_field)
#receiver(post_save, sender=OriginalModel) # Here, listen for the save signal coming from the original app change record.
def mirror_model_update(instance, **kwargs):
pk = instance.pk
mirror_model = []
if MirrorModel.objects.get(original_model.pk=pk).exists():
mirror_model = MirrorModel.objects.get(original_model.pk=pk)
mirror_model.field_one = instance.field_one # Map field values
*other logic ...*
mirror_model.save() # This should trigger your hook
else:
mirror_model = MirrorModel(original_model = instance, field_one = instance.field_one, ...)
mirror_model.save() #This should trigger your hooks as well, but for a new record
This way you can decouple the gamification app and even choose not to keep a record of all or the same fields as the original model, but fields specific to your functionality needs.
Your idea was good.
In the gamification app add your views, protect it with LoginRequiredMixin, and extend it with a check if a related record in the AssignmentSubmission table exists and is turned on.
In this way you have a 100% separated gamification views, models, logic, ecc...
In the existing views you can add a link to a gamification view.
While going through the official Django documentation I came across the Model Instance reference section in which it is mentioned that you can create an instance of the model using the custom Model using self.create. I wanted to know what's the difference between using the create method and the custom create_method when both are using the same fields and in both the cases the data is being saved in the DB.
Documentation:
https://docs.djangoproject.com/en/2.2/ref/models/instances/#creating-objects
class BookManager(models.Manager):
def create_book(self, title):
book = self.create(title=title)
return book
class Book(models.Model):
title = models.CharField(max_length=100)
objects = BookManager()
book = Book.objects.create_book("Pride and Prejudice")
Difference between these two
book2 = Book.objects.create(title="Pride and Prejudice")
Well in this simplest case, there is no difference. The reason of describing this technique in docs is obvious there
You may be tempted to customize the model by overriding the __init__
method. If you do so, however, take care not to change the calling
signature as any change may prevent the model instance from being
saved. Rather than overriding __init__, try using one of these
approaches:
It means you may be want to set some extra/default values to model instance. If you override constructor for this purpose, it is a little unsafe (and not usually a practice in django). That's why two other techniques for doing this are described. You are mentioning one of them. You can do some extra stuff in custom manager method if you want
class BookManager(models.Manager):
def create_book(self, title):
# you can do some extra stuff here for instance creation
book = self.create(title=title)
# or here when it is saved to db
return book
Otherwise there is no difference.
I have a custom account class in a Django app using PermissionsMixin:
class Account(AbstractBaseUser, PermissionsMixin):
Our CMS calls various .groups methods on this class in order to ascertain permissions.
We essentially want to override the queryset that is returned from .groups in this custom Account class and to inject an additional group under specific conditions. (I.e. the user has an active subscription and we then want to return "member" as one of the groups for that user, despite them not actually being in the group.)
How should we handle this override? We need to get the original groups, so that basic group functionality isn't broken, then inject our "virtual" group into the queryset.
Override the get_queryset method ManyRelatedManager. An object of ManyRelatedManager class has access to the parent instance.
Code Sample:
def add_custom_queryset_to_many_related_manager(many_related_manage_cls):
class ExtendedManyRelatedManager(many_related_manage_cls):
def get_queryset(self):
qs = super(ExtendedManyRelatedManager, self).get_queryset()
# some condition based on the instance
if self.instance.is_staff:
return qs.union(Group.objects.filter(name='Gold Subscription'))
return qs
return ExtendedManyRelatedManager
ManyRelatedManager class is obtained from the
ManyToManyDescriptor.
class ExtendedManyToManyDescriptor(ManyToManyDescriptor):
#cached_property
def related_manager_cls(self):
model = self.rel.related_model if self.reverse else self.rel.model
return add_custom_queryset_to_many_related_manager(create_forward_many_to_many_manager(
model._default_manager.__class__,
self.rel,
reverse=self.reverse,
))
Associated the ExtendedManyToManyDescriptor with groups field when
the Account class is initialized.
class ExtendedManyToManyField(ManyToManyField):
def contribute_to_class(self, cls, name, **kwargs):
super(ExtendedManyToManyField, self).contribute_to_class(cls, name, **kwargs)
setattr(cls, self.name, ExtendedManyToManyDescriptor(self.remote_field, reverse=False))
Override PermissionsMixin to use ExtendedManyToManyField for
groups field instead of ManyToManyField.
class ExtendedPermissionsMixin(PermissionsMixin):
groups = ExtendedManyToManyField(
Group,
verbose_name=_('groups'),
blank=True,
help_text=_(
'The groups this user belongs to. A user will get all permissions '
'granted to each of their groups.'
),
related_name="user_set",
related_query_name="user",
)
class Meta:
abstract = True
Reference:
django.db.models.fields.related_descriptors.create_forward_many_to_many_manager
Testing:
account = Account.objects.get(id=1)
account.is_staff = True
account.save()
account.groups.all()
# output
[<Group: Gold Subscription>]
The groups related manager is added by the PermissionMixin, you could actually remove the mixin and add only the parts of it that you need and redefine groups:
class Account(AbstractBaseUser):
# add the fields like is_superuser etc...
# as defined in https://github.com/django/django/blob/master/django/contrib/auth/models.py#L200
default_groups = models.ManyToManyField(Group)
#property
def groups(self):
if self.is_subscribed:
return Group.objects.filter(name="subscribers")
return default_groups.all()
Then you can add your custom groups using the Group model. This approach should work fine as long it is ok for all parts that groups returns a queryset instead of a manager (which probably mostly should be fine as managers mostly offer the same methods - but you probably need to find out yourself).
Update
After reading carefully the docs related to Managers and think about your requirement, I've to say there is no way to achieve the magic you want (I need to override the original, not to add a new ext_groups set - I need to alter third party library behavior that is calling groups.) Without touch the Django core itself (monkey patching would mess up admin, the same with properties).
In the solution I'm proposing, you have the necessary to add a new manager to Group, perhaps, you should start thinking in override that third-party library you're using, and make it use the Group's Manager you're interested in.
If the third-party library is at least medium quality it will have implemented tests that will help you to keep it working after the changes.
Proposed solution
Well, the good news is you can fulfill your business requirements, the bad news is you will have code a little more than you surely expect.
How should we handle this override?
You could use a proxy model to the Group class in order to add a custom manager that returns the desired QuerySet.
A proxy manager won't add an extra table for groups and will keep all the Group functionality besides, you can set custom managers on proxy models too, so, its perfect for this case use.
class ExtendedGroupManager(models.Manager):
def get_queryset(self):
qs = super().get_queryset()
# Do work with qs.
return qs
class ExtendedGroup(Group):
objects = ExtendedGroupManager()
class Meta:
proxy = True
Then your Account class should then have a ManyToMany relationship to ExtendedGroup that can be called ... ext_groups?
Till now you can:
acc = Account(...)
acc.groups.all() # All groups for this account (Django default).
acc.ext_groups.all() # Same as above, plus any modification you have done in get_queryset method.
Then, in views, you can decide if you call one or another depending on a condition of your own selection (Eg. user is subscribed).
Is worth mention you can add a custom manager to an existeing model using the method contribute_to_class
em = ExtendGroupManager()
em.contribute_to_class(Group, 'ext_group')
# Group.ext_group is now available.
If your CMS calls Account.groups directly I would investigate how to override a ManyRelatedManager since the account.groups is actually a django ManyRelatedManager object.
It can probably be achieved using django Proxy model for the Group.
One strategy to investigate would then be:
Add the virtual group to the groups in the database.
Override the ManyRelatedManager by changing get_queryset() (The base queryset)
Something like (pseudo code):
def get_queryset():
account = self.instance # Guess this exist in a RelatedManager
if account.has_active_subscribtion():
return self.get_all_relevant_groups_including_the_virtual_one()
return self.get_all_relevant_groups_excluding_the_virtual_one()
The key here is to get access to the current instance in the custom RelatedManager
This may be a useful discussion around custom related managers.
Subclass a Django ManyRelatedManager a.k.a. ManyToManyField
I do not recommend to try fiddle/monkey patch with the queryset itself since it very easy to break the django admin etc...
I have two models, Invoice and InvoiceItems, which have a one-to-many relationship.
Throughout the code base we're creating InvoiceItems for a given Invoice using the Manager object as:
invoice.invoice_items.create(...)
The thing is, now we have a validation that has to take place before trying to create an InvoiceItem, and going through the codebase, refactoring all the creation pieces would be a headache.
I wonder if there's a way to override the create method itself or should we go for the model's save()?
To modify a Manager's method you need to create your own. Given the following case:
# models
class MyModel(models.Model):
# ... fields
objects = MyManager()
class MyManager(models.Manager):
def create(self):
# write your own code here
pass
Do not worry about the others methods (filter, delete, etc.) all of them will work as usual.
You can find more about custom managers here
I'm developing a legacy Django 1.7 system for a client. The programmers before me overrode the Member model (basically the User model) "objects" property with a filter query that removes anything with "is_deleted" set to "True". I've listed the snippets below:
Member class snippet:
class Member(AbstractUser):
objects = MemberManager()
all_objects = models.Manager()
MemberManager class snippet:
class MemberManager(BaseUserManager):
def get_queryset(self):
return super(MemberManager, self).get_queryset().filter(is_deleted=False)
Now when I try to update a user that has the is_deleted flag set to "True" it fails. Below is an example code snippet. Notice how I use "all_objects" which is the default models.Manager() that returns all records.
user = Member.all_objects.get(pk=id) # id of an is_deleted = True record
user.is_deleted = False
user.save()
This code causes this Django query to run which unfortunately has "is_deleted = 0" included in the WHERE clause, which causes it to not find the record. Below is what shows up in the logs:
UPDATE Member [[snip...]] WHERE (Member.is_deleted = 0 AND Member.id = 6)
Is there any way to call "save()" that will not use the MemberManager.objects get_queryset filter?
I think the problem stems from having MemberManager listed first. As the documentation says:
Take note that the first Manager Django encounters (in the order in which they’re defined in the model) has a special status. Django interprets the first Manager defined in a class as the "default" Manager, and several parts of Django will use that Manager exclusively for that model. As a result, it’s a good idea to be careful in your choice of default manager in order to avoid a situation where overriding get_queryset() results in an inability to retrieve objects you’d like to work with.
Reversing the order of objects and all_objects should fix the problem.
I tried Kevin Christopher Henry's answer and unfortunately changing the default manager messed up the authentication code, which needed to inherit from the "BaseUserManager" class. The solution was to create an "undelete" function in the Member model class that uses the "all_objects" property to undelete the user before it's saved.
class Member(AbstractUser):
objects = MemberManager() # default manager
all_objects = models.Manager()
def undelete(self):
if self.is_deleted:
Member.all_objects.filter(id=self.id).update(is_deleted=False)
Then in my code I did this:
user = Member.all_objects.get(pk=id)
user.undelete()