I have a medium size Django REST app that I'm looking to add gamification features to.
The application in question is a school webapp where students can create mockup quizzes, participate in exams that teachers publish, and write didactical content that gets voted by other students and teachers.
I want to add some gamification features to make the app more interesting and to incentivize participation and usage of the various features: for example, each student will have a personal "reputation" score, and gain points upon completing certain actions--a student may gain points when completing a quiz with a high score, when submitting some content, or when receiving upvotes to such content.
The tricky part is I want to be able to have this logic be as separate as possible from the existing codebase, for various reasons: separation of concerns, ability to plug the engine in/out if needed, ability to easily deactivate features for certain groups of users, etc.
What I'm looking for here is some software engineering advice that's also Django-specific. Here's a high level description of what I'm thinking of doing--I'd like some advice on the approach.
create a new gamification app. Here I will have models that describe a change in reputation for a user and possibly other related events. The app should also send notifications when gamification-related events occur
from the gamification app, expose a callback-based interface, which the other primary app can call into to dispatch events
use the django-lifecycle package to call the callbacks from gamification when triggers occur.
This way, my existing models would only get touched to register the triggers from django-lifecycle (similar to signals). For example, let's say I want to give students points when they turn in an assignment. Let's say I have an AssignmentSubmission model to handle assignment submissions. With the added lifecycle hook, it'd look like this:
class AssignmentSubmission(models.Model):
NOT_TURNED_IN = 0
TURNED_IN = 1
STATES = ((NOT_TURNED_IN, 'NOT_TURNED_IN'), (TURNED_IN, 'TURNED_IN'))
user = models.ForeignKey(user)
assignment = models.ForeignKey(assignment)
state = models.PositiveSmallIntegerField(choices=STATES, default=NOT_TURNED_IN)
#hook(AFTER_UPDATE, when="state", was=NOT_TURNED_IN, is_now=TURNED_IN)
def on_turn_in(self):
get_gamification_interface().on_assignment_turn_in(self.user)
The on_assignment_turn_in method might look something like:
def on_assignment_turn_in(user):
ReputationIncrease.objects.create(user, points=50)
notifications.notify(user, "You gained 50 points")
This is pretty much just a sketch to give an idea.
I am unsure how get_gamification_interface() would work. Should it return a singleton? Maybe instantiate an object? Or return a class with static methods? I think it'd be best to have a getter like this as opposed to manually importing methods from the gamification app, but maybe it could also create too much overhead.
What's a good way to handle adding "pluggable" features to a project that are inherently entangled with existing models and business logic while also touching those as little as possible?
The foreign key approach is fine. You can easily chain and link queries using information from existing tables and you could even avoid touching the original code by importing your models to the new app. You can use Django signals in your new app and ditch the django-lifecycle extension to avoid adding lines to your core models. I used the following approach to keep track of modified records in a table; take a TrackedModel with fields field_one, field_two, field_n... which will be tracked by one of your new app's model, namely RecordTrackingModel:
from parent_app.models import TrackedModel # The model you want to track from a parent app.
from django.db.models.signals import post_save # I'm choosing post_save just to illustrate.
from django.dispatch import receiver
from datetime import datetime
class RecordTrackingModel(models.Model):
record = models.ForeignKey(TrackedModel, verbose_name=("Tracked Model"), on_delete=models.CASCADE)
field_one = models.TextField(verbose_name=("Tracked Field One"), null=True, blank=True) # Use same field type as original
field_two = models.TextField(("Tracked Field Two"))
field_n = ...
notes = models.TextField(verbose_name=("notes"), null=True, blank=True)
created = models.DateTimeField(verbose_name=("Record creation date"), auto_now=False, auto_now_add=True)
#receiver(post_save, sender=TrackedModel) # Here, listen for the save signal coming from a saved or updated TrackedModel record.
def modified_records(instance, **kwargs):
record = instance
tracked_field_one = instance.field_one
tracked_field_two = instance.field_two
tracked_field_n = another_function(instance.field_n) #an external function that could take a field content as input.
...
note = 'Timestamp: ' + str(datetime.now().isoformat(timespec='seconds'))
track_record = RecordTrackingModel.objects.create(record=record, field_one=tracked_field_one, field_two=tracked_field_two, field_n=tracked_field_n, ..., notes=note)
return track_record
There's no need to add functions to your pre-existing models as the signal dispatcher triggers whenever a save or delete signal appears at TrackedModel. Then you could place "if" statements for wether or not to perform actions based on field values, i.e.: just pass if an AssignmentSubmission record has a "Not Turned In" status.
Check Django signals reference for more information about when they trigger.
Additionally, I would suggest to change the "state" field to boolean type and rename it to something more explicit like "is_turned_in" for ease of use. It will simplify your forms and code. Edit: For more than 2 choices (non-boolean), I prefer using ForeignKey instead. It will let you modify choices over time easily from the admin site.
Edit:
Another approach could be mirroring the original models in your gamification app and call for a mirror record update when a save method is used in the original model.
gamification_app/models.py:
from parent_app.models import OriginalModel # The model you want to track from a parent app.
from django.db.models.signals import post_save # I'm choosing post_save just to illustrate.
from django.dispatch import receiver
from datetime import datetime
def gamification_function(input, parameters):
output = *your gamification logic*
return output
class MirrorModel(models.Model):
original_model = (OriginalModel, verbose_name=("Original Model"), on_delete=models.CASCADE)
field_one = ... #Same type as the original
field_two = ...
field_n = ...
#hook(AFTER_UPDATE, when="field_n", was=OPTION_1, is_now=OPTION_2)
def on_turn_in(self):
gamification_function(self.field, self.other_field)
#receiver(post_save, sender=OriginalModel) # Here, listen for the save signal coming from the original app change record.
def mirror_model_update(instance, **kwargs):
pk = instance.pk
mirror_model = []
if MirrorModel.objects.get(original_model.pk=pk).exists():
mirror_model = MirrorModel.objects.get(original_model.pk=pk)
mirror_model.field_one = instance.field_one # Map field values
*other logic ...*
mirror_model.save() # This should trigger your hook
else:
mirror_model = MirrorModel(original_model = instance, field_one = instance.field_one, ...)
mirror_model.save() #This should trigger your hooks as well, but for a new record
This way you can decouple the gamification app and even choose not to keep a record of all or the same fields as the original model, but fields specific to your functionality needs.
Your idea was good.
In the gamification app add your views, protect it with LoginRequiredMixin, and extend it with a check if a related record in the AssignmentSubmission table exists and is turned on.
In this way you have a 100% separated gamification views, models, logic, ecc...
In the existing views you can add a link to a gamification view.
Related
In Django I'm trying to implement some kind of a "security middleware", which gives access to certain db information only, if the logged in user matches.
Up to now I found two approaches: middleware or custom function in manager (Both explained here Django custom managers - how do I return only objects created by the logged-in user?).
Example for cleanest solution: custom function
class UserContactManager(models.Manager):
def for_user(self, user):
return self.get_query_set().filter(creator=user)
class MyUser(models.Model):
# Managers
objects = UserContactManager()
# then use it like this in views
data = MyUser.objects.for_user(request.user)
However, this solution only works, if you have control over the code which invokes this custom function (here: for_user()).
But if you are using third parties apps like Django-REST, Graphene or AdminViews, they don't have the possibility to configure a specific query-func to use.
My goal:
Replace the default Manager
Add user-based filters to query_set
Use the model as normal in all app configurations
Idea (pseudo code!)
from django.x import current_user
class UserBasedManager(models.Manager):
def get_queryset(self):
return super().get_queryset().filter(author=current_user.name)
class Book(models.Model):
title = models.CharField(max_length=100)
author = models.CharField(max_length=50)
objects = UserBasedManager() # The default manager.
Now I could use the model Book as normal in all extensions.
I know this solution would have some drawbacks, which are okay for me:
Can be only used when user is available (e.g. no direct script support)
Handling no or anonymous user is missing (left it away to keep example short)
My use case
I have a project, which shall give access to data by different interfaces (admin pages, custom views, rest, graphql).
As I have so many interfaces, I don't want to implement the access-rights in the views. This would cost too much time and is hard to maintain and the risk for security problems in one specific view/interface is too high.
Let's say I'm storing commits of git repositories in a database.
And the user shall get access to all commits, which are part of repos the user has READ access.
The question is: How can I implement this as a generic, view/interface independent solution?
Install django-threadlocals package and call get_current_user function to get the current user
from threadlocals.threadlocals import get_current_user
class UserBasedManager(models.Manager):
def get_queryset(self):
current_user = get_current_user()
return super().get_queryset().filter(author=current_user.name)
I have nested data in my Django Rest Framework app, something like this:
class Student(models.Model):
studyGroup = models.ForeignKey(StudyGroup, on_delete=models.SET_NULL, blank=True, null=True, related_name='student')
Each student may have a study group; a student may have no study group.
Many students can have the same study group.
I would like to automatically delete any StudyGroup that is not referenced by any students, either because the student was deleted or because it was updated.
I think this can be done by customising the 'save' and 'delete' methods for Student, checking whether their StudyGroup is referenced by any other Student, and deleting it if is not referenced. Or perhaps more elegantly by using signals. But it feels like there should be a simpler way to do this - like an inverse of on_delete=models.CASCADE.
Is there a way to tell the database to do this automatically? Or do I need to write the custom code?
You can remove the StudyGroup objects that are no longer referenced by a Student with the following query:
StudyGroup.objects.filter(students__isnull=True).delete()
(this given the related_name= parameter [Django-doc] of your ForeignKey [Django-doc] is set to 'students', since this is the name of the relation in reverse).
Depending on the database backend, you could implement a trigger that can perform certain actions, for example when you remove/update an Student record. But that is backend-specific.
We can add a trigger to the Student model to remove the StudyGroups without a Student when we delete or save Students:
# app/signals.py
from app.models import Student
from django.db.models.signals import post_delete, post_save
from django.dispatch import receiver
#receiver([post_delete, post_save], sender=Student)
def update_delete_student(sender, instance, **kwargs):
StudyGroup.objects.filter(students__isnull=True).delete()
You will need to import the signals module in your application config:
# app/app.py
from django.apps import AppConfig
class MyAppConfig(AppConfig):
# ...
def ready(self):
import app.signals
But there are ways to bypass the Django signals through the ORM. For examply by using QuerySet.update [Django-doc].
Therefore it might be useful to run the method periodically, for example each day/hour. We can use celery for that [realpython] or django-periodically [GitHub].
That being said, it might not be per se the most necessary to remove the StudyGroups anyway. If you for example want to retrieve a QuerySet of StudyGroups that have at least one student, we can write this like:
# StudyGroups with at least one Student
StudyGroup.objects.filter(student__isnull=False).distinct()
So instead of removing the StudyGroups, you might decide not to show these StudyGroups, like a soft delete [wiktionary]. Then you can still recover the data later on, this of course depends on the use case.
Note: the related_name of a ForeignKey is the name of the relation in reverse, so the name of the attribute of a StudyGroup to retrieve the QuerySet of Students. Therefore naming this 'studyGroup' is a bit "weird". It would also easily result in collisions if there are two or more ForeignKeys that point to StudyGroup with the same name.
I want to present general statistics about the instances of the class "ModelClass" on a web page for my users. Lets say there are some thousands ModelClass-objects, and that there are a lot of statistics that I need to calculate. I have figured out that I can do this with model managers, and here is a (very) simplified example:
class ModelClassCustomManager(models.Manager):
def get_query_set(self):
return super(ModelClassCustomManager, self).get_query_set().filter(is_complete = True)
class ModelClass(models.Model):
is_complete = models.BooleanField(default = False)
...
objects = models.Manager()
complete_objects = ModelClassCustomManager()
My concern is that this drains a lot of resources if this is calculated when users view the page. I would therefore like to calculate it only when I change or create new ModelClass objects since this is the only time the statistics really change. I guess this can be done by overriding the ModelClass save()-method.
What is the best way to save these results? Should I create another django-model for holding the calculated statistics, or is there another way of storing this information?
EDIT: Thanks to pastylegs for a good answer. Doing it this way, however causes some minor bugs, and I figured I'll explain them here, in case anyone else runs into this issue.
First of all, I incorrectly put the imports in the ModelClass at the top, so I got a circular dependency, which gave some weird results. To save yourself some frustration, put them where pastylegs did. Secondly, in order to overwrite the previous calculations of statistics (and not create a new one every time), just replace
if sender is ModelClass and instance is not Null:
count = ModelClass.objects.all().count()
stat = Stat(name='some_name', value=count).save()
with this:
if sender is ModelClass:
count = Match.objects.all().count()
try:
stat = Stat.objects.get(key="Total")
stat.update(key="Total", value=count) #Update statistic
except:
Stat(key="Total", value=count).save() #Create new
I'd create an app called statistics (or similar) that has a simple model to hold key,value pairs:
class Stat(models.Model):
key = models.CharField(..)
value = models.CharField(...)
then use a signal on your own model to run a function every time a user saves/updates a instance of your model. In models.py:
class ModelClass(models.Model):
...
from django.db.models.signals import post_save
from myapp.signal import updates_stats
post_save.connect(updates_stats, sender=ModelClass)
and then create the receiver function to calculate what ever statistics you need in signals.py
from myapp.models import ModelClass
from stats.models import Stat
def update_stats(sender, instance, signal, *args, **kwargs):
if sender is ModelClass and instance is not Null:
count = ModelClass.objects.all().count()
stat = Stat(name='some_name', value=count).save()
This is a very simple and basic approach and only outlines how you can make use of signals to perform calculations but it is a good start. Ideally you should try to calculate these values outside of the request/response cycle (especially if they are big caculations) as it will hang your server and potentially time out your requests, so you should consider using a queuing/task system like Celery to perform the calculation in the background
So I've got a UserProfile in Django that has certain fields that are required by the entire project - birthday, residence, etc. - and it also contains a lot of information that doesn't actually have any importance as far as logic goes - hometown, about me, etc. I'm trying to make my project a bit more flexible and applicable to more situations than my own, and I'd like to make it so that administrators of a project instance can add any fields they like to a UserProfile without having to directly modify the model. That is, I'd like an administrator of a new instance to be able to create new attributes of a user on the fly based on their specific needs. Due to the nature of the ORM, is this possible?
Well a simple solution is to create a new model called UserAttribute that has a key and a value, and link it to the UserProfile. Then you can use it as an inline in the django-admin. This would allow you to add as many new attributes to a UserProfile as you like, all through the admin:
models.py
class UserAttribute(models.Model):
key = models.CharField(max_length=100, help_text="i.e. Age, Name etc")
value = models.TextField(max_length=1000)
profile = models.ForeignKey(UserProfile)
admin.py
class UserAttributeInline(admin.StackedInline):
model = UserAttribute
class UserProfile(admin.ModelAdmin):
inlines = [UserAttibuteInline,]
This would allow an administrator to add a long list of attributes. The limitations are that you cant's do any validation on the input(outside of making sure that it's valid text), you are also limited to attributes that can be described in plain english (i.e. you won't be able to perform much login on them) and you won't really be able to compare attributes between UserProfiles (without a lot of Database hits anyway)
You can store additional data in serialized state. This can save you some DB hits and simplify your database structure a bit. May be the best option if you plan to use the data just for display purposes.
Example implementation (not tested)::
import yaml
from django.db import models
class UserProfile(models.Model):
user = models.OneToOneField('auth.User', related_name='profile')
_additional_info = models.TextField(default="", blank=True)
#property
def additional_info(self):
return yaml.load(self._additional_info)
#additional_info.setter
def additional_info(self, user_info_dict):
self._additional_info = yaml.dump(user_info_dict)
When you assign to profile.additional_info, say, a dictionary, it gets serialized and stored in _additional_info instead (don't forget to save the instance later). And then, when you access additional_info, you get that python dictionary.
I guess, you can also write a custom field to deal with this.
UPDATE (based on your comment):
So it appears that the actual problem here is how to automatically create and validate forms for user profiles. (It remains regardless on whether you go with serialized options or complex data structure.)
And since you can create dynamic forms without much trouble[1], then the main question is how to validate them.
Thinking about it... Administrator will have to specify validators (or field type) for each custom field anyway, right? So you'll need some kind of a configuration option—say,
CUSTOM_PROFILE_FIELDS = (
{
'name': 'user_ip',
'validators': ['django.core.validators.validate_ipv4_address'],
},
)
And then, when you're initializing the form, you define fields with their validators according to this setting.
[1] See also this post by Jacob Kaplan-Moss on dynamic form generation. It doesn't deal with validation, though.
I am writing a Django application that will track changes to the models, in a similar way to the admin interface. For example, I will be able to display a list of changes to a model, that look something like Changed Status from Open to Closed.
I am using the pre_save signal to do this, comparing the relevant fields between the existing item in the database, and the "instance" which is being saved. To get the existing item, I have to do sender._default_manager.get(pk=sender.pk) which seems a bit messy, but that part works.
The problem is, the view for changing this model calls the save() method on the form twice (first with commit=False) - this means that 2 changes get logged in the database, as the pre_save signal is emitted twice.
Is there any way I can accomplish this? Maybe in a different way altogether, though I remember reading that the Django admin app uses signals to track changes that users make.
Looking through the Django source, it seems that pre_save signals are sent on every call to save, even if commit is false. I would suggest inserting on the first pre_save, but add a flag column to the changes table, e.g.
class FooChanges(models.Model):
foo = models.ForeignKey(Foo)
dt = models.DateTimeField(default=datetime.now)
field = models.CharField(max_length=50)
value = models.CharField(max_length=50) # Or whatever is appropriate here
finished = models.BooleanField(default=False)
Then, your presave can be:
def pre_save_handler(sender, instance):
foo_changes, created = FooChanges.objects.get_or_create(foo=instance, finished=False, field='Status', value=instance.status)
if not created:
foo_changes.finished = True
foo_changes.save()
So on the first pre_save, you actually insert the change. On the second pass, you retrieve it from the database, and set the flag to false to make sure you don't pick it up the next time Foo's status changes.
use dispatch_uid:
http://docs.djangoproject.com/en/1.2/topics/signals/#preventing-duplicate-signals
Django Audit Log
django-audit-log is a pluggable app that does exactly what you want with little effort. I've used it in a project and I'll surely use it in many more now that I know it.