Is there a way to see errors when django unit tests loads fixtures?
It's annoying that something like missing file, duplicate keys or badly formatted fixtures files do not show up as error when running django unit test.
If duplicate primary keys generated an error when a fixture was loaded, many existing tests would be broken. Sometimes a fixture needs to contain that error, so it can be tested against.
You can write generic tests to check for the problems you've mentioned:
If your fixture fails to load, any query relying on that fixture will fail. Write a test that attempts to retrieve an object in that fixture (eg, YourObject.objects.get(pk=1) or YourObject.objects.get(name='My Test Entry').
To test if you have duplicate primary keys, compare YourObject.objects.all().aggregate(Count('id', distinct=True)) to YourObject.objects.all().aggregate(Count('id')). If those are not equal, you have duplicate primary keys.
To avoid badly formatted fixture files, generate them from validated data. Use manage.py dumpdata, and avoid manually creating or editing fixtures.
The first test mentioned will indicate if you have an invalid path. The last recommendation removes any issues with bad formatting.
The silent failure is a result of how loaddata works. It's looking for the filenames given in the fixtures = [] list in several locations, and may fail to find the files in any one of them. In order to report an error, loaddata must track if a file has been found yet, and that requires a patch. There's a trac ticket for the silent fail, but it has been triaged at 'Design Decision Needed' for 2 years now.
You have the option to make some noise about this on the Django mailing list, and use the unapproved patch in your development copy of Django.
You can try to define your own validation method in setUp(). Testing if your fixtures are loaded would be quite easy, however, validating your fixtures would require quite a bit work depending on what format is your fixture.
Related
I am attempting to write test cases in Django using Selenium. I want to use existent fixtures so my test database (SQLite3) has test data for every test.
I have some model test cases (just using the TestCase class) as follows;
from django.test import TestCase
from django.test import LiveServerTestCases
from missions.models import Mission, MissionDataRecord
class MissionModelTests(TestCase):
fixtures = ['myproject_common/fixtures/auth_initial_load.json', 'worklog/fixtures/worklogs_initial_load',
'missions_initial_load.json']
def test_object_name_is_mission_title(self):
mission = Mission.objects.get(id=1)
self.assertEqual(mission.title, str(mission))
def test_object_name_is_mission_title_again(self):
mission = Mission.objects.get(id=1)
self.assertEqual(mission.title, str(mission))
This works as expected when run like this (I get two test passes). However, for Selenium testing I need to use LiveServerTestCase instead of TestCase.
The simple example above is a model test, but for illustrative purposes of the issue I'm facing with Selenium, if I simply replace "TestCase" with "LiveServerTestCase" the first test passes, but the second test fails, with the error
django.db.utils.IntegrityError: Problem installing fixture
'[...]/fixtures/auth_initial_load.json': Could not load
auth.User(pk=1): UNIQUE constraint failed: auth_user.username
This error occurs in the _fixture_setup of /django/test/testcases.py. This seems to suggests that my fixtures (specifically the auth_initial_load fixture) is attempting to load again ON TOP of existing data. However, from django docs reading this should not be taking place, because each test should run in its own transaction (which I believe means the fixtures are loaded for each transaction).
What is going on here, and more importantly, how can I use LiveServerTestCase with my existing fixtures (similar to how I am using TestCase currently)? In reality I need to use StaticLiveServerTestCase, but Im guessing the code will be the same.
It turns out the way I was loading fixtures correctly after all. The issue was with my fixtures themselves, using hard coded primary (and foreign) keys. In my case two users were being created before the fixtures were loaded, so when fixtures tried to load with the same primary key, a UNIQUE constraint violation occurred. The solution was to re-generate my fixtures using the --natural-primary and --natural-foreign flags as suggested in this SO answer.
Say I have this model definition:
class Foo(Model):
...
class Bar(Model):
some_m2m_field = ManyToManyField(Foo)
and this code:
bar = Bar.objects.create()
bar.some_m2m_field.set(an_id_array_with_some_invalid_pks)
When I run that normally, the last line will, as it should, throw an IntegrityError. However, if I run the same code from a django.test.TestCase, the last line will NOT throw an error. It instead will wait until the _post_teardown() phase of the test to throw the IntegrityError.
Here's a small project that demonstrates the issue: https://github.com/t-evans/m2mtest
How do I fix that? I suppose that's configurable, but I haven't been able to find it...
Follow-up question:
Ultimately, I need to handle the case when there are bad IDs being passed to the m2m_field.set() method (and I need unit tests that verify that bad IDs are being handled correctly, which is why the delayed IntegrityError in the unit test won't work).
I know I can find the bad IDs by looping over the array and hitting the DB one for each ID. Is there a more efficient way to find the bad IDs or (better) simply tell the set() method to ignore/drop the bad IDs?
TestCase wraps tests in additional atomic() blocks, compared to TransactionTestCase, so to test specific database transaction behaviour, you should use TransactionTestCase.
I believe an IntegrityError is thrown only when the transaction is committed, as that's the moment the db would find out about missing ids.
In general if you want to test for db exceptions raised during a test, you should use a TransactionTestCase and test your code using:
with self.assertRaises(IntegrityError):
# do something that gets committed to db
See the answer from #dirkgroten for how to fix the unit test issue.
As for the followup question on how to more-efficiently eliminate the bad IDs, one way is as follows:
good_ids = Foo.objects.filter(id__in=an_id_array_with_some_invalid_ids).values_list('id', flat=True)
I'm having trouble moving away from django_nose.FastFixtureTestCase to django.test.TestCase (or even the more conservative django.test.TransactionTestCase). I'm using Django 1.7.11 and I'm testing against Postgres 9.2.
I have a Testcase class that loads three fixtures files. The class contains two tests. If I run each test individually as a single run (manage test test_file:TestClass.test_name), they each work. If I run them together, (manage test test_file:TestClass), I get
IntegrityError: Problem installing fixture '<path>/data.json': Could not load <app>.<Model>(pk=1): duplicate key value violates unique constraint "<app_model_field>_49810fc21046d2e2_uniq"
To me it looks like the db isn't actually getting flushed or rolled back between tests since it only happens when I run the tests in a single run.
I've stepped through the Django code and it looks like they are getting flushed or rolled back -- depending on whether I'm trying TestCase or TransactionTestCase.
(I'm moving away from FastFixtureTestCase because of https://github.com/django-nose/django-nose/issues/220)
What else should I be looking at? This seems like it should be a simple matter and is right within what django.test.TestCase and Django.test.TransactionTestCase are designed for.
Edit:
The test class more or less looks like this:
class MyTest(django.test.TransactionTestCase): # or django.test.TestCase
fixtures = ['data1.json', 'data2.json', 'data3.json']
def test1(self):
return # I simplified it to just this for now.
def test2(self):
return # I simplified it to just this for now.
Update:
I've managed to reproduce this a couple of times with a single test, so I suspect something in the fixture loading code.
One of my basic assumptions was that my db was clean for every TestCase. Tracing into the django core code I found instances where an object (in one case django.contrib.auth.User) already existed.
I temporarily overrode _fixture_setup() to assert the db was clean prior to loading fixtures. The assertion failed.
I was able to narrow the problem down to code that was in a TestCase.setUpClass() instead of TestCase.setUp(), and so the object was leaking out of the test and conflicting with other TestCase fixtures.
What I don't understand completely is I thought that the db was dropped and recreated between TestCases -- but perhaps that is not correct.
Update: Recent version of Django includes setUpTestData() that should be used instead of setUpClass()
I'm using the entity framework.
In one of my unit tests I have a line like:
this.Set<T>().Add(entity);
On executing that line I get:
System.InvalidOperationException : The model backing the
'InvoiceNewDataContext' context has changed since the database was
created. Either manually delete/update the database, or call
Database.SetInitializer with an IDatabaseInitializer instance. For
example, the DropCreateDatabaseIfModelChanges strategy will
automatically delete and recreate the database, and optionally seed it
with new data.
Well I've actually deleted the database and removed the connection string.
I'm surprised this error is happening on adding as I wouldn't expect it to happen until I saved the data and it discovered there was no database.
In previous projects/solutions I created during unit tests I have been able to add to the context for test purposes without actually calling SaveChanges.
Would anyone know why this would be happening in my latest projects/solutions?
Are you sure it really didn't use database in your previous projects? If you do not specify any connection string it will silently use a default one to SQLExpress database with local .mdf file so make sure that isn't happening now.
Is it possible to have a fixture change between test methods? If so, how can I do this?
My syntax for this problem :
In the cakephp framework i am building tests for a behavior that is configured by adding fields to the table. This is intended to work in the same way that adding the "created"
and "modified" fields will auto-populate these fields on save.
To test this I could create dozens of fixtures/model combos to test the different setups, but it would be a hundred times better, faster and easier to just have the fixture change "shape" between test methods.
If you are not familiar with the CakePHP framework, you can maybe still help me as it uses SimpleTest
Edit: rephrased question to be more general
I'm not familiar specifically with CakePHP, but this kind of thing seems to happen anywhere with fixtures.
There is no built in way in rails at least for this to happen, and I imagine not in cakePHP or anywhere else either because the whole idea of a fixture, is that it is fixed
There are 2 'decent' workarounds I'm aware of
Write a changefixture method, and just before you do your asserts/etc, run it with the parameters of what to change. It should go and update the database or whatever needs to be done.
Don't use fixtures at all, and use some kind of object factory or object generator to create your objects each time
This is not an answer to my quetion, but a solution to my issue example.
Instead of using multiple fixtures or changing the fixtures, I edit the Model::_schema arrays by removing the fields that I wanted to test without. This has the effect that the model acts as if the fields was not there, but I am unsure if this is a 100% test. I do not think it is for all cases, but it works for my example.