From the Django documentation I see that a Django TestCase wraps the tests within two nested atomic() blocks: one for the whole class and one for each test.
Then I see that setUpTestData method "allows the creation of initial data at the class level, once for the whole TestCase": so we're talking about the whole class atomic block.
This means that the --keepdb flag should not affect this behaviour, because what this flag do is just skip the database create/destroy.
But I noticed that if I run the test with the --keepdb flag the data I create in the setUpTestData method are preserved.
I'm fine with this but I want to understand what --keepdb exactly does, because I can't understand why this happens. I tried to look directly at the Django TestCase class source code but I don't see any check about the --keepdb option.
Why - if I run my test with the --keepdb option - the data I create in setUpTestData method are preserved?
Ok, I was right: the --keepdb option doesn't affect the setUpTestData behaviour.
I deleted my test database, so now I'm starting from scratch.
Now I see that all the data I create in setUpTestData are "rolled back" after the TestCase ends, also with the --keepdb option.
Probably I just messed up my test database before.
Related
Recently I've changed db engine from SQLite to PostgreSQL. I have successfully migrated whole db design to PostgreSQL (just simple makemigaretions, migrate). Once I ran tests some have failed for some unknown reason (errors suggest that some objects were not created). Failure doesn't apply to all tests, just selected few. Everything has been working before.
I'd start investigating what's going on on test-by-test basis, but some bizarre behavior has appeared. Let's say my test is in class MyTestClass and test is called test_do_something and in MyTestClass there are other tests as well.
When I'm running python manage.py test MyTestClass I'm getting info that test_do_something has failed.
When I'm running python manage.py test MyTestClass.test_do_something everything passes.
On SQLite both ways pass.
I'm assuming that setUpTestData() and setUp() methods work same way in SQLite and PostgreSQL. Or they don't?
Any clue why such discrepancy might be happening?
EDIT
I think I've noticed what might be wrong, but I don't understand why. The problem is because my function I've used to call to create object which is later used only once. Which differs from SQLite execution.
What I mean in my test I have something like this:
def create_object(self):
self.client.post(reverse('myurl'), kwargs={'myargs':arg})
def test_mytest1(self):
# Do something
self.create_object()
# Do something
def test_mytest2(self):
# Do something
self.create_object()
# Do something
def test_mytest3(self):
# Do something
self.create_object()
# Do something
Only for one test create_object() will be executed.
I believe I've fond cause of those failures. As a matter of fact it hasn't been an issue with one-time execution of support function as I expected. The problem has been with hardcoded ids I've used for various reasons. It appears that object I've been hoping to see didn't exist.
Let me explain a bit more what I've experienced. E.g. I had test where I've been referring to particular object passing this object id in URL kwargs. Before this operation I've created object and passed id=1 as kwargs, because I assumed that if this is only place within this test and setUp() it will be 1. It appears that with PostgreSQL it's not so clear. It seems that ids are incremented despite DB flush. Which is completely different behavior then SQLite has been providing.
I'd very much appreciate if someone could provide some more detailed answer why is this happening. Is ID counter not zeroed in PostgreSQL on flush? It would look so.
I'm having trouble moving away from django_nose.FastFixtureTestCase to django.test.TestCase (or even the more conservative django.test.TransactionTestCase). I'm using Django 1.7.11 and I'm testing against Postgres 9.2.
I have a Testcase class that loads three fixtures files. The class contains two tests. If I run each test individually as a single run (manage test test_file:TestClass.test_name), they each work. If I run them together, (manage test test_file:TestClass), I get
IntegrityError: Problem installing fixture '<path>/data.json': Could not load <app>.<Model>(pk=1): duplicate key value violates unique constraint "<app_model_field>_49810fc21046d2e2_uniq"
To me it looks like the db isn't actually getting flushed or rolled back between tests since it only happens when I run the tests in a single run.
I've stepped through the Django code and it looks like they are getting flushed or rolled back -- depending on whether I'm trying TestCase or TransactionTestCase.
(I'm moving away from FastFixtureTestCase because of https://github.com/django-nose/django-nose/issues/220)
What else should I be looking at? This seems like it should be a simple matter and is right within what django.test.TestCase and Django.test.TransactionTestCase are designed for.
Edit:
The test class more or less looks like this:
class MyTest(django.test.TransactionTestCase): # or django.test.TestCase
fixtures = ['data1.json', 'data2.json', 'data3.json']
def test1(self):
return # I simplified it to just this for now.
def test2(self):
return # I simplified it to just this for now.
Update:
I've managed to reproduce this a couple of times with a single test, so I suspect something in the fixture loading code.
One of my basic assumptions was that my db was clean for every TestCase. Tracing into the django core code I found instances where an object (in one case django.contrib.auth.User) already existed.
I temporarily overrode _fixture_setup() to assert the db was clean prior to loading fixtures. The assertion failed.
I was able to narrow the problem down to code that was in a TestCase.setUpClass() instead of TestCase.setUp(), and so the object was leaking out of the test and conflicting with other TestCase fixtures.
What I don't understand completely is I thought that the db was dropped and recreated between TestCases -- but perhaps that is not correct.
Update: Recent version of Django includes setUpTestData() that should be used instead of setUpClass()
I have a series of unit tests (all subclasses of TransactionTestCase) spread out through multiple apps in a single Django project. When I run all of them in one go using ./manage.py test an error occurs in one of the tests. But when I run each app's tests individually, one at a time, using ./manage.py test my_project.app_name I get no errors.
The specific error I get is a FieldError in modelform_factory, but my question isn't so much about the specific solution to this error. I'm just curious what possible data/processes/whatever could bleed over between the supposedly-self-contained test cases in Django. Any thoughts?
(For the curious, if I make all my tests subclasses of TestCase (rather than TransactionTestCase) I get a bunch of different errors, but I've chalked those up to some separate issue relating to problems with rolling back the transactions within which Django encapsulates each test case. But who knows, maybe there's a connection?)
Found the answer:
Regardless of how many different test cases are run during one ./manage.py test call, and regardless of how much the database is rolled back/truncated (for TestCase and TransactionTestCase, respectively) between tests, all tests are run under the same python thread and with the same instantiations of all model base classes. Thus any variables in the thread or any modifications to class definitions persist across test instances.
In my case, it was both. A view got a user instance out of the currently running thread, but that user had been deleted when a previous test had been rolled back. Later on a view modified a list that was declared (as a blank list) in its parent class, thus altering the parent classes list to not be a blank list, and causing problems in a later test.
I have some tables containing a lot of data (imported from geonames using django-cities: https://github.com/coderholic/django-cities) that I want to preserve in tests (since loading them via fixtures will be very slow)… how can I keep those tables and their data during tests?
I think I have to write a custom TestRunner, but I have no idea about where to start :P
It's possible, here is a way :
1) Define your own test runner look here to see how.
2) For your custom test runner look in the default test runner, you can just copy and past the code and just comment this line : connection.creation.destroy_test_db(old_name, verbosity) which is responsible for destroying the test database, and i think you should put the connection.creation.create_test_db(..) line in a try except something like this maybe:
try:
# Create the database the first time.
connection.creation.create_test_db(verbosity, autoclobber=not interactive)
except ..: # Look at the error that this will raise when create a database that already exist
# Test database already created.
pass
3) Bound TEST_RUNNER in setting.py to your test runner.
4) Now run your test like this: ./manage.py test
In situations where a test executed and changed test database tables, would database tables return to original state after each test? If not, how should I know in what order the tests are executed so that I will predict the state of database tables. For example,
class SimpleTest(Testcase):
def test_insert(self):
# testing to see if data correctly added to database
def test_other_thing(self):
# does insered data available here?
The database is rolled back at the end of every test.
For proper test isolation, when tests touch the database, you need to inherit from django.test.TestCase which handles database state isolation between one test execution and another.
Never, ever, depend on test execution order: if you need to, you are doing it wrong, because you are violating test isolation.
Remember that you don't need to use only unittest.TestCase or only django.test.TestCase: you can mix them as needed (you don't need the latter if your test does not touch the database).
Note that django.test.TestCase use transactions to speed up database state cleanup after each test, so if you need to actually test a database transaction you need to use django.test.TransactionTestCase (see https://docs.djangoproject.com/en/dev/topics/testing/#testcase)