We have a system consisting of a Django server connected to a PostgreSQL database, and some AWS Lambda functions which need to access the database. When an admin saves a model (PremiumUser - contains premium plan information that needs to be read by the Lambdas), we want to set up a schedule of CloudWatch events based on the saved information. Those events then trigger other Lambdas which also need to directly access the database, as the database state may change at any time and they should work off the most recent state of the database.
The issue is that Django seems to think it has saved the values, but when the Lambdas read the database the expected values aren't there. We have tried using Django's post_save signal, calling the Lambdas inside the triggered function; we have tried overriding Django's default PremiumUser.save method to perform super(PremiumUser, self).save(*args, **kwargs), and only then call Lambdas (in case the post_save signal was getting triggered too early); and we have tried overriding the PremiumUser.save method and calling super(PremiumUser, self).save(*args, **kwargs) in the context of an atomic transaction (ie with transactions.atomic():).
When we call the Lambdas a few seconds after the admin dashboard has updated, they can find the values as expected and work properly, which suggests that somehow Django considers the model as having been 'saved' to the database, while the database has not yet been updated.
Is there a way to force Django to write to the database immediately? This would be the preferred solution, as it would keep Django's model and the database consistent.
An alternative solution we have considered but would prefer not to resort to would be to put a sleep in the Lambdas and calling them asynchronously so that Django's save is able to complete before the Lambda functions access the database. Obviously this would be a race condition, so we don't want to do this if it can be at all avoided.
Alright, after spending more time poring over Django's documentation we found a solution: using on_commit. It seems that post_save is triggered after Django's model is updated, but before the database has been written to. on_commit, on that other hand is triggered after the current database transaction has been committed (completed).
For us, the solution involved setting up some code like this:
from django.db import models, transaction
class PremiumUser(models.Model):
# Normal setup code...
def save(self, *args, **kwargs):
# Do some necessary things before the actual save occurs
super(PremiumUser, self).save(*args, **kwargs)
# Set up a callback for the on_commit
def create_new_schedule():
# Call lambda to create new schedule...
# Register the callback with the current transaction's on_commit
transaction.on_commit(create_new_schedule)
Related
I have a field in one of my models in django which I want to be reset every hour.
(i.e. at each o'clock its value becomes zero)
How can I do this task? Can I schedule a function in django?
As you know we can define EVENTs and TRIGGERs in mysql and other database backend. Also I am familiar with signals in django but those can not fit in my needs. (because database event is somewhat outside of django and have problems; with signals although it seems this is impossible!)
You could use schedule, it's very easy to apply for your problem.
import schedule
import time
def job():
print("I'm working...")
schedule.every().hour.do(job)
while True:
schedule.run_pending()
time.sleep(1)
Here there is a thread where it is shown how to execute a task periodically. Then you could add some conditions to fit your scenario.
Why Django-Simple history records get created on calling save method if I call update then it doesn't create history record ?
Django : 1.11.15
Django-simple-history : 1.9.0
Python : 3.6
As is written in the documentation this is a known issue:
Django Simple History functions by saving history using a post_save
signal every time that an object with history is saved. However, for
certain bulk operations, such as bulk_create and queryset updates, signals are not sent, and the history is not saved
automatically. However, Django Simple History provides utility
functions to work around this.
So basically the app makes use of the fact that you .save() the model, and this is circumvented by some ORM calls (because then you can not perform the actions in "bulk" at the database level anymore).
Instead of using
Entry.objects.filter(pub_date__year=2010).update(comments_on=False)
you thus need to perform:
for e in Entry.objects.filter(pub_date__year=2010):
e.comments_on = False
e.save()
For a bulk_create there is a variant: bulk_create_with_history, since then it simply makes two bulk creates: one for the objects, and one for the "histories".
According to http://www.django-rest-framework.org/api-guide/serializers/#writable-nested-representations, in order to accept a nested serializer I need to create an update method. None of the examples use transactions although they do modify multiple rows/tables. Does the DRF somehow wrap things in transactions already, or should I explicitly put transaction.atomic() all over it?
Related PR:
https://github.com/tomchristie/django-rest-framework/pull/1787
You can also use Django's ATOMIC_REQUESTS database setting which will apply a transaction before the execution of each request and commit it if the request finishes successfully. More information here:
Database transactions - Tying transactions to HTTP requests
The related PR is unrelated to your question. PR is linked to the DRF specific exception handler that bypassed the default Django transaction scheme (https://github.com/tomchristie/django-rest-framework/pull/1204#issuecomment-52712621).
DRF doesn't specifically wrap things in a transaction to leave the users free to choose whatever they want to.
first import transaction module from db, and then use the following
with transtaction.atomic():
pass
This will ensure the atomicity and consistency of your data into database.
I have an authentication backend based off a legacy database. When someone logs in using that database and there isn't a corresponding User record, I create one. What I'm wondering is if there is some way to alert the Django system to this fact, so that for example I can redirect the brand-new user to a different page.
The only thing I can think of is adding a flag to the users' profile record called something like is_new which is tested once and then set to False as soon as they're redirected.
Basically, I'm wondering if someone else has figured this out so I don't have to reinvent the wheel.
I found the easiest way to accomplish this is to do exactly as you've said. I had a similar requirement on one of my projects. We needed to show a "Don't forget to update your profile" message to any new member until they had visit their profile for the first time. The client wanted it quickly so we added a 'visited_profile' field to the User's profile and defaulted that to False.
We settled on this because it was super fast to implement, didn't require tinkering with the registration process, worked with existing users, and didn't require extra queries every page load (since the user and user profile is retrieved on every page already). Took us all of 10 minutes to add the field, run the South migration and put an if tag into the template.
There's two methods that I know of to determine if an object has been created:
1) When using get_or_create a tuple is returned of the form (obj, created) where created is a boolean indicating obviously enough whether the object was created or not
2) The post_save signal passes a created paramater, also a boolean, also indicating whether the object was created or not.
At the simplest level, you can use either of these two hooks to set a session var, that you can then check and redirect accordingly.
If you can get by with it, you could also directly redirect either after calling get_or_create or in the post_save signal.
You can use a file-based cache to store the users that aren't yet saved to the database. When the user logs in for the second time, you can look in the cache, find the user object, and save it to the database for good.
Here's some info on django caching: http://docs.djangoproject.com/en/dev/topics/cache/?from=olddocs
PS: don't use Memcached because it will delete all information in the situation of a computer crash or shut down.
I have a Django app that works well for me, but currently has no notion of user: I am the only one using it, and I would like to change this...
Except for the admin views, the logged-in user should not have access to the data created by other users. There is no shared data between users.
I suppose I have to add a user foreign key to all the models I created. Correct?
Is there a simple way to implement the filtering based on request.user? Can this be done more or less automatically, or do I have to go through all the code to check each and every query done on the database?
I have written the code using TDD, and I intend to follow up... What are the best strategies to ensure that user-filtering is implemented correctly, e.g. that I did not forget to filter an existing query? I suppose I can write tests that show that a particular query is not yet filtered, and implement the filter. But what about the queries that I will write later? Is there a way I can assert that all existing and future queries return objects that only belong to the current user?
Thanks.
Yes, you'll need to add a User FK. Don't forget you'll have to migrate your database tables - either manually, or via a tool like South.
One way of implementing the filter would be to define custom Managers for your models, with a for_user method that takes the User as an argument: something like:
class ForUserManager(models.Manager):
def for_user(self, user):
return self.filter(user=user)
Now you can use this manager - subclassed and/or with a mixin as necessary - on all your models, and remember to use objects.for_user(request.user) everywhere.
This will make testing easier too - your test could monkeypatch that for_user method so that it sets a flag or a counter in a global variable somewhere, and then test that it has incremented as expected.
Edit in response to comment No, as you suspect, that won't work. It's not even that everyone will necessarily get the last-logged-in user: it's that Managers are class-level attributes, and as such are reused throughout a process, so any request served by that server process will use the same one.