I would like to execute a transaction to delete some values, after that, count in db and if the result is < 1, rollback, i tried the following code:
#login_required
#csrf_exempt
#transaction.atomic
def update_user_groups(request):
if request.POST:
userId = request.POST['userId']
groups = request.POST.getlist('groups[]')
result = None
with transaction.atomic():
try:
GroupsUsers.objects.filter(user_id=int(userId)).delete()
for group in groups:
group_user = GroupsUsers()
group_user.user_id = userId
group_user.group_id = group
group_user.save()
count = UsersInAdmin.objects.all().count()
if count < 1:
transaction.commit()
else:
transaction.rollback()
except Exception, e:
result = e
return JsonResponse(result, safe=False)
Thanks,
It's not possible to manually commit or rollback the transaction inside an atomic block.
Instead, you can raise an exception inside the atomic block. If the block completes, the transaction will be committed. If you raise an exception, the transaction will be rolled back. Outside the atomic block, you can catch the exception and carry on your view.
try:
with transaction.atomic():
do_stuff()
if ok_to_commit():
pass
else:
raise ValueError()
except ValueError:
pass
I think you should give the Django documentation about db transactions another look.
The most relevant remark is the following:
Avoid catching exceptions inside atomic!
When exiting an atomic block, Django looks at whether it’s exited normally or with an exception to determine whether to commit or roll back. If you catch and handle exceptions inside an atomic block, you may hide from Django the fact that a problem has happened. This can result in unexpected behavior.
Adapting the example from the docs, it looks like you should nest your try and with blocks like this:
#transaction.atomic
def update_user_groups(request):
# set your variables
try:
with transaction.atomic():
# do your for loop
if count < 1:
# You should probably make a more descriptive, non-generic exception
# see http://stackoverflow.com/questions/1319615/proper-way-to-declare-custom-exceptions-in-modern-python for more info
raise Exception('Count is less than one')
except:
handle_exception()
Related
Out system accesses to a middle platform(I do not know how to call it in English, we call it 中台 in Chinese) which is for authentication like logging in, JWT verifying, etc.
Then, We have encoutered questions when it is neccessary to rollback actions because of unexpected program errors. Like code below, the program will crash when running at 1 / 0, then AdminPermission.objects.create can be rolled back, but do_user_actions can not, cause it is a RPC function. So, we need to override transaction.atomic or something like this to implement our requirements. But, I do not know how to implement it. Pls give me some suggestions or sample code. Thx a lot.
#transaction.atomic # can not rollback remote func call `do_user_actions`
def create(self, request, *args, **kwargs):
# call a remote func here to create a admin account.
user_dict = do_user_actions(query_kwargs=query_kwargs, action='create-admin')
user_id = user_dict.get('id')
permission_code = 'base_permission'
AdminPermission.objects.create(user_id=user_id, permission_code=permission_code)
# some unexpected errors, like:
1 / 0
return Response('success')
Instead of using atomic with the decorator you could use it as a context manager, something like this,
with transaction.atomic():
try:
# do your stuff
user_dict = do_user_actions(query_kwargs=query_kwargs, action='create-admin')
user_id = user_dict.get('id')
permission_code = 'base_permission'
AdminPermission.objects.create(user_id=user_id, permission_code=permission_code)
1/0 # error
except SomeError: # capture error
revert_user_actions() # revert do_user_actions
transaction.set_rollback(True) # rollback
return Response(status=status.HTTP_424_FAILED_DEPENDENCY) # failure
return Response(serializer.data, status=status.HTTP_201_CREATED) # success
Django Docs - Transactions
UPDATE
As #Gorgine mention in the comment, the documentation don't recommend to handle errors inside atomic. Because is always a good idea to follow recomendations, you can put atomic inside try block. In this case atomic block will handle the rollback if an error occur, so you will need to handle the actions that are not rollback by atomic in the except block. Something like this:
try:
with transaction.atomic():
# do your stuff
user_dict = do_user_actions(query_kwargs=query_kwargs, action='create-admin')
user_id = user_dict.get('id')
permission_code = 'base_permission'
AdminPermission.objects.create(user_id=user_id, permission_code=permission_code)
1/0 # error
except SomeError: # capture error
revert_user_actions() # revert do_user_actions
return Response(status=status.HTTP_424_FAILED_DEPENDENCY) # failure
return Response(serializer.data, status=status.HTTP_201_CREATED) # success
I'm using Django and Python 3.7. I have this code
article = get_article(id)
...
article.label = label
article.save(update_fields=["label"])
Sometimes I get the following error on my "save" line ...
raise DatabaseError("Save with update_fields did not affect any rows.")
django.db.utils.DatabaseError: Save with update_fields did not affect any rows.
Evidently, in the "..." another thread may be deleting my article. Is there another way to rewrite my "article.save(...)" statement such that if the object no longer exists I can ignore any error being thrown?
A comment by gachdavit suggested using select_for_update. You could modify your get_article function to call select_for_update prior to fetching the article. By doing this, the database row holding the article will be locked as long as the current transaction does not commit or roll back. If another thread tries to delete the article at the same time, that thread will block until the lock is released. Effectively, the article won't be deleted until after you have called the save function.
Unless you have special requirements, this is the approach I'd take.
I'm not aware of any special way to handle it other than to check to see if the values have changed.
article = update_model(article, {'label': label})
def update_model(instance, updates):
update_fields = {
field: value
for field, value in updates.items()
if getattr(instance, field) != value
}
if update_fields:
for field, value in update_fields.items():
setattr(instance, field, value)
instance.save(update_fields=update_fields.keys())
return instance
Edit:
Another alternative would be to catch and handle the exception.
This is hacky, but you could override _do_update in your model and simply return True. Django itself does something kind of hacky on line 893 of _do_update to suppress the same exception when update_fields contains column names that do not appear in the model.
The return value from _do_update triggers the exception you are seeing from this block
I tested the override below and it seemed to work. I feel somewhat dirty for overriding a private-ish method, but I think I will get over it.
def _do_update(self, base_qs, using, pk_val, values, update_fields, forced_update):
updated = super(Article, self)._do_update(base_qs, using, pk_val, values, update_fields, forced_update)
if not updated and Article.objects.filter(id=pk_val).count() == 0:
return True
return updated
This solution could be genericized and moved to a mixin base class if you need to handle this for more than one model.
I used this django management command to test
from django.core.management.base import BaseCommand
from foo.models import Article
class Command(BaseCommand):
def handle(self, *args, **kwargs):
Article.objects.update_or_create(id=1, defaults=dict(label='zulu'))
print('Testing _do_update hack')
article1 = Article.objects.get(id=1)
article1.label = 'yankee'
article2 = Article.objects.get(id=1)
article2.delete()
article1.save(update_fields=['label'])
print('Done. No exception raised')
I'm making a video page can track played time and comment time, every time I reload the page, using {{ movie.play_num }} to display the played time, it's always working but it won't change the database, So I don't know how can it be right? Seems like the session.commit() doesn't work.
Besides, if I put some words in the comment field and submit, a waiting for response... shows up and looks like the server keep waiting until timeout.
Here's my code:
def play(id=None):
movie = Movie.query.join(Tag).filter(
Tag.id == Movie.tag_id,
Movie.id == int(id)
).first_or_404()
form = CommentForm()
if "user" in session and form.validate_on_submit():
data = form.data
comment = Comment(
content=data["content"],
movie_id=movie.id,
user_id=session["user_id"]
)
db.session.add(comment)
db.session.commit() # problem here
movie.comment_num = movie.comment_num + 1
flash("Success", "ok")
return redirect(url_for("home.play", id=movie.id))
movie.play_num = movie.play_num + 1
try:
db.session.commit() # and problem here
except:
db.session.rollback()
return render_template("home/play.html", movie=movie, form=form)
see the play_num has changed to 1 from 0, this is the first time I reload the page, but at the second time, the page can't be open, and the console can't collect any data.An error occurred:
This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (pymysql.err.InternalError)
How to fix this
you try catch the exception msg:
from sqlalchemy.exc import SQLAlchemyError
try:
db.session.commit()
except SQLAlchemyError as e:
print(str(e))
db.session.rollback()
check what the error msg first
PostgreSQL is able to have .distint('field name') database queries, however Sqlite isn't, so I created a try/except block which should run a more simple query if the user is using sqlite3.
try:
qs = qs.filter(tag__istartswith=self.q).order_by('tag').distinct('tag')
except NotImplementedError:
qs = qs.filter(tag__istartswith=self.q)
So if the user is using Sqlite I would expect the simple query in the except block to get executed, however the exception is thrown and the simple query never gets executed:
raise NotImplementedError('DISTINCT ON fields is not supported by this database backend')
NotImplementedError: DISTINCT ON fields is not supported by this database backend
Do you have any idea why this isn't working as expected?
Thanks
Because querysets are lazy and the error is not raised while constructing the query, but when you actually evaluate it.
You can try forcing your query to be evaluated to catch the exception:
try:
qs = qs.filter(tag__istartswith=self.q).order_by('tag').distinct('tag')
dummy_boolean_var = qs.exists()
except NotImplementedError:
qs = qs.filter(tag__istartswith=self.q)
EDIT: Apparently my untested version did not work as the assignment in the except clause tries to modify the original query. This is the working version as tested by OP:
try:
qa = qs
qs = qs.filter(tag__istartswith=self.q).order_by('tag').distinct('tag')
dummy_boolean_var = qs.exists()
except NotImplementedError:
qs = qa
qs = qs.filter(tag__istartswith=self.q)
Consider this simple example :
# a bank account class
class Account:
#transaction.commit_on_success
def withdraw(self, amount):
# code to withdraw money from the account
#transaction.commit_on_success
def add(self, amount):
# code to add money to the account
# somewhere else
#transaction.commit_on_success
def makeMoneyTransaction(src_account, dst_account, amount):
src_account.withdraw(amount)
dst_account.add(amount)
(taken from https://code.djangoproject.com/ticket/2227)
If an exception raises in Account.add(), the transaction in Account.withdraw() will still be committed and money will be lost, because Django doesn't currently handle nested transactions.
Without applying patchs to Django, how can we make sure that the commit is sent to the database, but only when the main function under the #transaction.commit_on_success decorator finishes without raising an exception?
I came across this snippet: http://djangosnippets.org/snippets/1343/ and it seems like it could do the job. Is there any drawbacks I should be aware of if I use it?
Huge thanks in advance if you can help.
P.S. I am copying the previously cited code snippet for purposes of visibility:
def nested_commit_on_success(func):
"""Like commit_on_success, but doesn't commit existing transactions.
This decorator is used to run a function within the scope of a
database transaction, committing the transaction on success and
rolling it back if an exception occurs.
Unlike the standard transaction.commit_on_success decorator, this
version first checks whether a transaction is already active. If so
then it doesn't perform any commits or rollbacks, leaving that up to
whoever is managing the active transaction.
"""
commit_on_success = transaction.commit_on_success(func)
def _nested_commit_on_success(*args, **kwds):
if transaction.is_managed():
return func(*args,**kwds)
else:
return commit_on_success(*args,**kwds)
return transaction.wraps(func)(_nested_commit_on_success)
The problem with this snippet is that it doesn't give you the ability to roll back an inner transaction without rolling back the outer transaction as well. For example:
#nested_commit_on_success
def inner():
# [do stuff in the DB]
#nested_commit_on_success
def outer():
# [do stuff in the DB]
try:
inner()
except:
# this did not work, but we want to handle the error and
# do something else instead:
# [do stuff in the DB]
outer()
In the example above, even if inner() raises an exception, its content won't be rolled back.
What you need is a savepoint for the inner "transactions". For your code, it might look like this:
# a bank account class
class Account:
def withdraw(self, amount):
sid = transaction.savepoint()
try:
# code to withdraw money from the account
except:
transaction.savepoint_rollback(sid)
raise
def add(self, amount):
sid = transaction.savepoint()
try:
# code to add money to the account
except:
transaction.savepoint_rollback(sid)
raise
# somewhere else
#transaction.commit_on_success
def makeMoneyTransaction(src_account, dst_account, amount):
src_account.withdraw(amount)
dst_account.add(amount)
As of Django 1.6, the atomic() decorator does exactly that: it uses a transaction for the outer use of the decorator, and any inner use uses a savepoint.
Django 1.6 introduces #atomic, which does exactly what I was looking for!
Not only it supports "nested" transactions, but it also replaces the old, less powerful, decorators. And it is good to have a unique and consistent behavior for transactions management across different Django apps.