I would like to know if I have something like this:
def functionA():
with transaction.atomic():
#save something
functionB()
def functionB():
with transaction.atomic():
#save another thing
Someone knows what will happen? If functionB fails, functionA will rollback too?
Thank you!
Yes, it will. Regardless of nesting, if an atomic block is exited by an exception it will roll back:
If the block of code is successfully completed, the changes are committed to the database. If there is an exception, the changes are rolled back.
Note also that an exception in an outer block will cause the inner block to roll back, and that an exception in an inner block can be caught to prevent the outer block from rolling back. The documentation addresses these issues. (Or see here for a more comprehensive follow-up question on nested transactions).
with transaction.atomic(): # Outer atomic, start a new
transaction.on_commit(foo)
try:
with transaction.atomic(): # Inner atomic block, create a savepoint
transaction.on_commit(bar)
raise SomeError() # Raising an exception - abort the savepoint
except SomeError:
pass
# foo() will be called, but not bar()
Related
How can I always commit a insert even inside an atomic transaction? In this case I need just one point to be committed and everything else rolled back.
For example, my view, decorator contains with transaction.atomic() and other stuffs:
#my_custom_decorator_with_transaction_atomic
def my_view(request):
my_core_function()
return ...
def my_core_function():
# many sql operations that need to rollback in case of error
try:
another_operation()
except MyException:
insert_activity_register_on_db() # This needs to be in DB, and not rolled back
raise MyException()
I would not like to make another decorator for my view without transaction atomic and do it manually on core. Is there a way?
I'm trying to utilize django's row-level-locking by using the select_for_update utility. As per the documentation, this can only be used when inside of a transaction.atomic block. The side-effect of using a transaction.atomic block is that if my code throws an exception, all the database changes get rolled-back. My use case is such that I'd actually like to keep the database changes, and allow the exception to propagate. This leaves me with code looking like this:
with transaction.atomic():
user = User.objects.select_for_update.get(id=1234)
try:
user.do_something()
except Exception as e:
exception = e
else:
exception = None
if exception is not None:
raise exception
This feels like a total anti-pattern and I'm sure I must be missing something. I'm aware I could probably roll-my-own solution by manually using transaction.set_autocommit to manage the transaction, but I'd have thought that there would be a simpler way to get this functionality. Is there a built in way to achieve what I want?
I ended up going with something that looks like this:
from django.db import transaction
class ErrorTolerantTransaction(transaction.Atomic):
def __exit__(self, exc_type, exc_value, traceback):
return super().__exit__(None, None, None)
def error_tolerant_transaction(using=None, savepoint=True):
"""
Wraps a code block in an 'error tolerant' transaction block to allow the use of
select_for_update but without the effect of automatic rollback on exception.
Can be invoked as either a decorator or context manager.
"""
if callable(using):
return ErrorTolerantTransaction('default', savepoint)(using)
return ErrorTolerantTransaction(using, savepoint)
I can now put an error_tolerant_transaction in place of transaction.atomic and exceptions can be raised without a forced rollback. Of course database-related exceptions (i.e. IntegrityError) will still cause a rollback, but that's expected behavior given that we're using a transaction. As a bonus, this solution is compatible with transaction.atomic, meaning it can be nested inside an atomic block and vice-versa.
I'm attempting to update my code that uses commit_manually to atomic to go to Django 1.7 then 1.8 but I'm having issues not being able to rollback while in an atomic block. The issue stems from a flag we added to out import routines, dry_run. In the prior Django versions #commit_manually allowed us to do the following:
if self.dry_run:
transaction.rollback()
else:
transaction.commit()
If I attempt to preform a roll back in an atomic block it throws an error:
TransactionManagementError : "This is forbidden when an 'atomic' block is active."
To get this to work I tried to make use of set_autocommit
Example:
def do_some_import(self)
transaction.set_autocommit(False)
#import routine
if self.dry_run:
transaction.rollback()
transaction.set_autocommit(True)
else:
transaction.commit()
transaction.set_autocommit(True)
But this feels wrong any suggestions or insights?
Have you tried set_rollback()? I think that will solve your problem.
#transaction.atomic
def do_some_import(self):
#import routine
if self.dry_run:
transaction.set_rollback(True)
I have written a test which checks whether IntegrityError was raised in case of duplicate records in the database. To create that scenario I am issue a REST API twice. The code looks like this:
class TestPost(APITestCase):
#classmethod
def setUpClass(cls):
super().setUpClass()
common.add_users()
def tearDown(self):
super().tearDown()
self.client.logout()
def test_duplicate_record(self):
# first time
response = self.client.post('/api/v1/trees/', dict(alias="some name", path="some path"))
# same request second time
response = self.client.post('/api/v1/trees/', dict(alias="some name", path="some path"))
self.assertEqual(response.status_code, status.HTTP_400_BAD_RREQUEST)
But I get an error stack like this
"An error occurred in the current transaction. You can't "
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
How can I avoid this error this is certainly undesirable.
I've come across this issue today, and it took me a while to see the bigger picture, and fix it properly. It's true that removing self.client.logout() fixes the issue, and is probably not needed there, but the problem lies in your view.
Tests which are subclasses of TestCase wrap your test cases and tests in db transactions (atomic blocks), and your view somehow breaks that transaction. For me it was swallowing an IntegrityError exception. At that point, the transaction was already broken, but Django didn't know about it, so it couldn't correctly perform a rollback. Any query that would then be executed would cause a TransactionManagementError.
The fix for me was to correctly wrap the view code in yet another atomic block:
try:
with transaction.atomic(): # savepoint which will be rolled back
counter.save() # the operation which is expected to throw an IntegrityError
except IntegrityError:
pass # swallow the exception without breaking the transaction
This may not be a problem for you in production, if you're not using ATOMIC_REQUESTS, but I still think it's the correct solution.
Try removing self.client.logout() from the tearDown method. Django rolls back the transaction at the end of each test. You shouldn't have to log out manually.
This code is supposed to get or create an object and update it if necessary. The code is in production use on a website.
In some cases - when the database is busy - it will throw the exception "DoesNotExist: MyObj matching query does not exist".
# Model:
class MyObj(models.Model):
thing = models.ForeignKey(Thing)
owner = models.ForeignKey(User)
state = models.BooleanField()
class Meta:
unique_together = (('thing', 'owner'),)
# Update or create myobj
#transaction.commit_on_success
def create_or_update_myobj(owner, thing, state)
try:
myobj, created = MyObj.objects.get_or_create(owner=user,thing=thing)
except IntegrityError:
myobj = MyObj.objects.get(owner=user,thing=thing)
# Will sometimes throw "DoesNotExist: MyObj matching query does not exist"
myobj.state = state
myobj.save()
I use an innodb mysql database on ubuntu.
How do I safely deal with this problem?
This could be an off-shoot of the same problem as here:
Why doesn't this loop display an updated object count every five seconds?
Basically get_or_create can fail - if you take a look at its source, there you'll see that it's: get, if-problem: save+some_trickery, if-still-problem: get again, if-still-problem: surrender and raise.
This means that if there are two simultaneous threads (or processes) running create_or_update_myobj, both trying to get_or_create the same object, then:
first thread tries to get it - but it doesn't yet exist,
so, the thread tries to create it, but before the object is created...
...second thread tries to get it - and this obviously fails
now, because of the default AUTOCOMMIT=OFF for MySQLdb database connection, and REPEATABLE READ serializable level, both threads have frozen their views of MyObj table.
subsequently, first thread creates its object and returns it gracefully, but...
...second thread cannot create anything as it would violate unique constraint
what's funny, subsequent get on the second thread doesn't see the object created in the first thread, due to the frozen view of MyObj table
So, if you want to safely get_or_create anything, try something like this:
#transaction.commit_on_success
def my_get_or_create(...):
try:
obj = MyObj.objects.create(...)
except IntegrityError:
transaction.commit()
obj = MyObj.objects.get(...)
return obj
Edited on 27/05/2010
There is also a second solution to the problem - using READ COMMITED isolation level, instead of REPEATABLE READ. But it's less tested (at least in MySQL), so there might be more bugs/problems with it - but at least it allows tying views to transactions, without committing in the middle.
Edited on 22/01/2012
Here are some good blog posts (not mine) about MySQL and Django, related to this question:
http://www.no-ack.org/2010/07/mysql-transactions-and-django.html
http://www.no-ack.org/2011/05/broken-transaction-management-in-mysql.html
Your exception handling is masking the error. You should pass a value for state in get_or_create(), or set a default in the model and database.
One (dumb) way might be to catch the error and simply retry once or twice after waiting a small amount of time. I'm not a DB expert, so there might be a signaling solution.
Since 2012 in Django we have select_for_update which lock rows until the end of the transaction.
To avoid race conditions in Django + MySQL
under default circumstances:
REPEATABLE_READ in the Mysql
READ_COMMITTED in the Django
you can use this:
with transaction.atomic():
instance = YourModel.objects.select_for_update().get(id=42)
instance.evolve()
instance.save()
The second thread will wait for the first thread (lock), and only if first is done, the second will read data saved by first, so it will work on updated data.
Then together with get_or_create:
def select_for_update_or_create(...):
instance = YourModel.objects.filter(
...
).select_for_update().first()
if order is None:
instnace = YouModel.objects.create(...)
return instance
The function must be inside transaction block, otherwise, you will get from Django:
TransactionManagementError: select_for_update cannot be used outside of a transaction
Also sometimes it's good to use refresh_from_db()
In case like:
instance = YourModel.objects.create(**kwargs)
response = do_request_which_lasts_few_seconds(instance)
instance.attr = response.something
you'd like to see:
instance = MyModel.objects.create(**kwargs)
response = do_request_which_lasts_few_seconds(instance)
instance.refresh_from_db() # 3
instance.attr = response.something
and that # 3 will reduce a lot a time window of possible race conditions, thus chance for that.