I have a view that takes, in many parts, data from a, let's say, untrusted source, and I want to save the data in a Django model's TextField. The input may have null characters, and, if I understand correctly, Postgres (the backend of my project) prohibits saving null characters in the database. I could use str.replace to replace the null characters, but I would have to riddle my codebase with str.replace, since I have many points where I create models out of the untrusted data. Is there a way to force the cleaning of the data in the model level? I mean, so that I could do my_model.save() and in a unique place in my code have str.replace, instead of having it in all the places I have a TextField.
You can do that by writing a post save signal, in the following way:
https://docs.djangoproject.com/en/1.11/ref/signals/#post-save
This is an example implementation:
from django.db.models.signals import post_save
from django.dispatch import receiver
class SampleModel(models.Model):
# ... fields here
# method for your validations
#receiver(post_save, sender=SampleModel)
def validate_input(sender, instance, **kwargs):
# your custom validations
instance.save()
Related
I have a medium size Django REST app that I'm looking to add gamification features to.
The application in question is a school webapp where students can create mockup quizzes, participate in exams that teachers publish, and write didactical content that gets voted by other students and teachers.
I want to add some gamification features to make the app more interesting and to incentivize participation and usage of the various features: for example, each student will have a personal "reputation" score, and gain points upon completing certain actions--a student may gain points when completing a quiz with a high score, when submitting some content, or when receiving upvotes to such content.
The tricky part is I want to be able to have this logic be as separate as possible from the existing codebase, for various reasons: separation of concerns, ability to plug the engine in/out if needed, ability to easily deactivate features for certain groups of users, etc.
What I'm looking for here is some software engineering advice that's also Django-specific. Here's a high level description of what I'm thinking of doing--I'd like some advice on the approach.
create a new gamification app. Here I will have models that describe a change in reputation for a user and possibly other related events. The app should also send notifications when gamification-related events occur
from the gamification app, expose a callback-based interface, which the other primary app can call into to dispatch events
use the django-lifecycle package to call the callbacks from gamification when triggers occur.
This way, my existing models would only get touched to register the triggers from django-lifecycle (similar to signals). For example, let's say I want to give students points when they turn in an assignment. Let's say I have an AssignmentSubmission model to handle assignment submissions. With the added lifecycle hook, it'd look like this:
class AssignmentSubmission(models.Model):
NOT_TURNED_IN = 0
TURNED_IN = 1
STATES = ((NOT_TURNED_IN, 'NOT_TURNED_IN'), (TURNED_IN, 'TURNED_IN'))
user = models.ForeignKey(user)
assignment = models.ForeignKey(assignment)
state = models.PositiveSmallIntegerField(choices=STATES, default=NOT_TURNED_IN)
#hook(AFTER_UPDATE, when="state", was=NOT_TURNED_IN, is_now=TURNED_IN)
def on_turn_in(self):
get_gamification_interface().on_assignment_turn_in(self.user)
The on_assignment_turn_in method might look something like:
def on_assignment_turn_in(user):
ReputationIncrease.objects.create(user, points=50)
notifications.notify(user, "You gained 50 points")
This is pretty much just a sketch to give an idea.
I am unsure how get_gamification_interface() would work. Should it return a singleton? Maybe instantiate an object? Or return a class with static methods? I think it'd be best to have a getter like this as opposed to manually importing methods from the gamification app, but maybe it could also create too much overhead.
What's a good way to handle adding "pluggable" features to a project that are inherently entangled with existing models and business logic while also touching those as little as possible?
The foreign key approach is fine. You can easily chain and link queries using information from existing tables and you could even avoid touching the original code by importing your models to the new app. You can use Django signals in your new app and ditch the django-lifecycle extension to avoid adding lines to your core models. I used the following approach to keep track of modified records in a table; take a TrackedModel with fields field_one, field_two, field_n... which will be tracked by one of your new app's model, namely RecordTrackingModel:
from parent_app.models import TrackedModel # The model you want to track from a parent app.
from django.db.models.signals import post_save # I'm choosing post_save just to illustrate.
from django.dispatch import receiver
from datetime import datetime
class RecordTrackingModel(models.Model):
record = models.ForeignKey(TrackedModel, verbose_name=("Tracked Model"), on_delete=models.CASCADE)
field_one = models.TextField(verbose_name=("Tracked Field One"), null=True, blank=True) # Use same field type as original
field_two = models.TextField(("Tracked Field Two"))
field_n = ...
notes = models.TextField(verbose_name=("notes"), null=True, blank=True)
created = models.DateTimeField(verbose_name=("Record creation date"), auto_now=False, auto_now_add=True)
#receiver(post_save, sender=TrackedModel) # Here, listen for the save signal coming from a saved or updated TrackedModel record.
def modified_records(instance, **kwargs):
record = instance
tracked_field_one = instance.field_one
tracked_field_two = instance.field_two
tracked_field_n = another_function(instance.field_n) #an external function that could take a field content as input.
...
note = 'Timestamp: ' + str(datetime.now().isoformat(timespec='seconds'))
track_record = RecordTrackingModel.objects.create(record=record, field_one=tracked_field_one, field_two=tracked_field_two, field_n=tracked_field_n, ..., notes=note)
return track_record
There's no need to add functions to your pre-existing models as the signal dispatcher triggers whenever a save or delete signal appears at TrackedModel. Then you could place "if" statements for wether or not to perform actions based on field values, i.e.: just pass if an AssignmentSubmission record has a "Not Turned In" status.
Check Django signals reference for more information about when they trigger.
Additionally, I would suggest to change the "state" field to boolean type and rename it to something more explicit like "is_turned_in" for ease of use. It will simplify your forms and code. Edit: For more than 2 choices (non-boolean), I prefer using ForeignKey instead. It will let you modify choices over time easily from the admin site.
Edit:
Another approach could be mirroring the original models in your gamification app and call for a mirror record update when a save method is used in the original model.
gamification_app/models.py:
from parent_app.models import OriginalModel # The model you want to track from a parent app.
from django.db.models.signals import post_save # I'm choosing post_save just to illustrate.
from django.dispatch import receiver
from datetime import datetime
def gamification_function(input, parameters):
output = *your gamification logic*
return output
class MirrorModel(models.Model):
original_model = (OriginalModel, verbose_name=("Original Model"), on_delete=models.CASCADE)
field_one = ... #Same type as the original
field_two = ...
field_n = ...
#hook(AFTER_UPDATE, when="field_n", was=OPTION_1, is_now=OPTION_2)
def on_turn_in(self):
gamification_function(self.field, self.other_field)
#receiver(post_save, sender=OriginalModel) # Here, listen for the save signal coming from the original app change record.
def mirror_model_update(instance, **kwargs):
pk = instance.pk
mirror_model = []
if MirrorModel.objects.get(original_model.pk=pk).exists():
mirror_model = MirrorModel.objects.get(original_model.pk=pk)
mirror_model.field_one = instance.field_one # Map field values
*other logic ...*
mirror_model.save() # This should trigger your hook
else:
mirror_model = MirrorModel(original_model = instance, field_one = instance.field_one, ...)
mirror_model.save() #This should trigger your hooks as well, but for a new record
This way you can decouple the gamification app and even choose not to keep a record of all or the same fields as the original model, but fields specific to your functionality needs.
Your idea was good.
In the gamification app add your views, protect it with LoginRequiredMixin, and extend it with a check if a related record in the AssignmentSubmission table exists and is turned on.
In this way you have a 100% separated gamification views, models, logic, ecc...
In the existing views you can add a link to a gamification view.
Just to note: I know about overwriting Model's .save() method but it won't suit me as I explained it at the end of my question.
In one of my projects, I've got more than of 30 database Models and each one of these models accept multiple CharField to store store Persian characters; However there might be some case where users are using Arabic layouts for their keyboard where some chars are differ from Persian.
For example:
The name Ali in Arabic: علي
and in Persian: علی
Or even in numbers:
345 in Arabic: ٣٤٥
And in Persian: ۳۴۵
I need to selectively, choose a set of these fields and run a function on their value before saving them (On create or update) to map these characters so I won't end up with two different form of a single word in my database.
One way to do this is overwriting the .save() method on the database Model. Is there any other way to do this so I don't have to change all my models?
Sounds like a good use of Django's Signals. With Signals you can call a function before or after save on one or more Models.
https://docs.djangoproject.com/en/3.2/topics/signals/
Django includes a “signal dispatcher” which helps decoupled applications get notified when actions occur elsewhere in the framework. In a nutshell, signals allow certain senders to notify a set of receivers that some action has taken place. They’re especially useful when many pieces of code may be interested in the same events.
django.db.models.signals.pre_save & django.db.models.signals.post_save
Sent before or after a model’s save() method is called.
https://docs.djangoproject.com/en/3.2/ref/signals/#pre-save
pre_save
django.db.models.signals.pre_save
This is sent at the beginning of a model’s save() method.
Arguments sent with this signal:
sender
The model class.
instance
The actual instance being saved.
raw
A boolean; True if the model is saved exactly as presented (i.e. when loading a fixture). One should not query/modify other records in the database as the database might not be in a consistent state yet.
using
The database alias being used.
update_fields
The set of fields to update as passed to Model.save(), or None if update_fields wasn’t passed to save().
You can connect signals to a Single, Multiple, or All models in your application.
from django.db.models.signals import pre_save
from django.dispatch import receiver
from myapp.models import MyModel
#receiver(pre_save)
def my_handler(sender, instance, **kwargs):
# logic here
In this case sender is the Model that is about to be saved, instance will be the object being saved.
You can create a completely custom model field however in your case it's really easier to just customize Django's CharField:
class MyCharField(models.CharField):
def to_python(self, value):
if value is None or value == "":
return value # Prevent None values being converted to "None"
if isinstance(value, str):
return self.func_to_call(value)
return self.func_to_call(str(value))
Replace your models CharField with MyCharField.
And create the .func_to_call() so it does whatever you need to map the values:
def func_to_call(self, value):
# Do whatever you want to map the values
return value
This is an extension from my post here preventing crud operations on django model
A short into to the problem , im currently using a package called django-river to implement a workflow system in my application. The issue is that they do not have a predefined 'start' , 'dropped' , 'completed' state. Their states are stored as a django model instance. This would mean that my application is unable to programmatically differentiate between the states. Therefore , the labels of these states has to be hardcoded into my program (Or does it? Maybe someone has a solution to this?)
Suppose that there is no solution to the issue other than hardcoding the states into my application , this would mean that i would have to prevent users from updating , or deleting these states that i have pre created initially.
My idea is to have a form of validation check within the django model's save method . This check would check that the first 3 instances of the State model is always start , deactivated and completed and in the same order. This would prevent the check from passing through whenever a user trys to change items at the ORM level.
However , it would seem that there is 2 issues with this:
I believe django admin doesn't run the model class save method
Someone is still able to change the states as long as the way they changed it does not pass through the save() method. AKA from the DB SQL commands
Although it is unlikely to happen , changing the name would 'break' my application and therefore i wish to be very sure that no one can edit and change these 3 predefined states.
Is there a fool proof way to do this?
My idea is to have a form of validation check within the django model's save method.
if i understand your description, maybe you can just override the save() function of your model like so:
class MyModel(models.Model):
[..]
def save(self, *args, **kwargs):
# Put your logic here ..
super(MyModel, self).save(*args, **kwargs)
I got the answer from django documentation
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _
def validate_even(value):
if value % 2 != 0:
raise ValidationError(
_('%(value)s is not an even number'),
params={'value': value},
)
You can add this to a model field via the field’s validators argument:
from django.db import models
class MyModel(models.Model):
even_field = models.IntegerField(validators=[validate_even])
FYI: It is not really mandatory to use gettext_lazy and you can use just message as follows
from django.core.exceptions import ValidationError
def validate_even(value):
if value % 2 != 0:
raise ValidationError(
('%(value)s is not an even number'),
params={'value': value},
)
I want to use the username of the account in which my django is running as a string to load the model fields specific to that username. I have created a file 'survey.py' which returns a dictionary and I want the keys as the fields.
How can I get the username as string?
from django.db import models
from django.contrib.auth.models import User
from multiselectfield import MultiSelectField
from survey_a0_duplicate import details, analysis
import ast
class HomeForm1(models.Model):
user= models.OneToOneField(User, on_delete=models.CASCADE,)
details.loadData(survey_name = user)#<=====This loads the data for specific user<======
global f1
f1=analysis.getQuestion(in_json=False)#<====We get the dictionary here<========
d=list(f1.keys())
###################assign the filters#######################################################
for k in d:
q=list(f1[k].keys())
q.sort()
choices=tuple(map(lambda f: (f,f),q))
locals()[k]=MultiSelectField(max_length=1000,choices=choices,blank=True)
def save(self, *args, **kwargs):
if self.pk is None:
self.user= self.user.username
super(HomeForm1,self).save(*args,**kwargs)
def __str__(self):
return self.title
This is not how you write Django code. Global variables are a bad idea anyway, but you must not use them in a multi-user, multi-process environment like Django. You will immediately have thread-safety issues; you must not do it.
Not only is there an explicit global in the code you have shown, there is clearly one inside survey_a0_duplicate - since details.loadData() does not actually return anything but you then "get the dictionary" from analysis.getQuestion. You must remove the globals from both locations.
Also, your save method is totally wrong. You have the user relationship; why would you overwrite it with the username? That not only makes no sense, it specifically destroys the type of the field that you have set. Just don't do it. Remove the entire save method.
But you need to stop messing about with choices at class level. That is never going to work. If you need to dynamically set choices, do in in a form, where you can customise the __init__ method to accept the current user and build up the choices based on that.
I have nested data in my Django Rest Framework app, something like this:
class Student(models.Model):
studyGroup = models.ForeignKey(StudyGroup, on_delete=models.SET_NULL, blank=True, null=True, related_name='student')
Each student may have a study group; a student may have no study group.
Many students can have the same study group.
I would like to automatically delete any StudyGroup that is not referenced by any students, either because the student was deleted or because it was updated.
I think this can be done by customising the 'save' and 'delete' methods for Student, checking whether their StudyGroup is referenced by any other Student, and deleting it if is not referenced. Or perhaps more elegantly by using signals. But it feels like there should be a simpler way to do this - like an inverse of on_delete=models.CASCADE.
Is there a way to tell the database to do this automatically? Or do I need to write the custom code?
You can remove the StudyGroup objects that are no longer referenced by a Student with the following query:
StudyGroup.objects.filter(students__isnull=True).delete()
(this given the related_name= parameter [Django-doc] of your ForeignKey [Django-doc] is set to 'students', since this is the name of the relation in reverse).
Depending on the database backend, you could implement a trigger that can perform certain actions, for example when you remove/update an Student record. But that is backend-specific.
We can add a trigger to the Student model to remove the StudyGroups without a Student when we delete or save Students:
# app/signals.py
from app.models import Student
from django.db.models.signals import post_delete, post_save
from django.dispatch import receiver
#receiver([post_delete, post_save], sender=Student)
def update_delete_student(sender, instance, **kwargs):
StudyGroup.objects.filter(students__isnull=True).delete()
You will need to import the signals module in your application config:
# app/app.py
from django.apps import AppConfig
class MyAppConfig(AppConfig):
# ...
def ready(self):
import app.signals
But there are ways to bypass the Django signals through the ORM. For examply by using QuerySet.update [Django-doc].
Therefore it might be useful to run the method periodically, for example each day/hour. We can use celery for that [realpython] or django-periodically [GitHub].
That being said, it might not be per se the most necessary to remove the StudyGroups anyway. If you for example want to retrieve a QuerySet of StudyGroups that have at least one student, we can write this like:
# StudyGroups with at least one Student
StudyGroup.objects.filter(student__isnull=False).distinct()
So instead of removing the StudyGroups, you might decide not to show these StudyGroups, like a soft delete [wiktionary]. Then you can still recover the data later on, this of course depends on the use case.
Note: the related_name of a ForeignKey is the name of the relation in reverse, so the name of the attribute of a StudyGroup to retrieve the QuerySet of Students. Therefore naming this 'studyGroup' is a bit "weird". It would also easily result in collisions if there are two or more ForeignKeys that point to StudyGroup with the same name.