I have some tables containing a lot of data (imported from geonames using django-cities: https://github.com/coderholic/django-cities) that I want to preserve in tests (since loading them via fixtures will be very slow)… how can I keep those tables and their data during tests?
I think I have to write a custom TestRunner, but I have no idea about where to start :P
It's possible, here is a way :
1) Define your own test runner look here to see how.
2) For your custom test runner look in the default test runner, you can just copy and past the code and just comment this line : connection.creation.destroy_test_db(old_name, verbosity) which is responsible for destroying the test database, and i think you should put the connection.creation.create_test_db(..) line in a try except something like this maybe:
try:
# Create the database the first time.
connection.creation.create_test_db(verbosity, autoclobber=not interactive)
except ..: # Look at the error that this will raise when create a database that already exist
# Test database already created.
pass
3) Bound TEST_RUNNER in setting.py to your test runner.
4) Now run your test like this: ./manage.py test
Related
I am struggling to create tests with django with objects that have foreign keys. The tests run with no errors when run individually, but when running all, i get ForeignKeyViolation in the teardown phase (which I don't understand what it does and can't find any info on it).
When starting the testsuite I am creating a bunch of dummy data like this:
def setUp(self) -> None:
create_bulk_users(5)
create_categories()
create_tags()
create_technologies()
self.tags = Tag.objects.all()
self.categories = Category.objects.all()
self.technologies = Technology.objects.all()
The problems I need help figuring out are:
What exactly the teardown phase does? Are there any detailed docs on it?
How should I structure my tests so to avoid the ForeignKeyViolation issue?
What exactly the teardown phase does? Are there any detailed docs on it?
I couldn't find any detailed docs about it. All I find out was by reading the actual code. Teardown cancels any operations done to the DB during the tests.
The way TestCases work (broadly) is this:
1. SetUp method runs **before** every test
2. individual test method is run
3. TearDown method runs **after** every test
4. Test database is flushed
How should I structure my tests so to avoid the ForeignKeyViolation issue?
My mistake was that I was importing a function for creating dummy data.
The imported module was running a function to populate some primary data. The problem was that was not rerun each within the setUp method because it would run just one, when importing the file.
# imported module
# will delete all tags after running first test but
# the reference to the old tags will persist
tags = FakeTags.create_bulk();
def create_bulk_articles():
article = FakeArticle.create()
article.tags.add(tags)
I fixed it by just using the trouble function inside the imported function.
# imported module
# tags are created each time before running the tests
def create_bulk_articles():
tags = FakeTags.create_bulk();
article = FakeArticle.create()
article.tags.add(tags)
I start to test my Django project recently. At first, I test my code by accessing my local full data DB, but it seems like not a proper way.
So , I try to create fake data in local migrated database without real data.
And pytest provide xunit style , so setup_module or setup_class .... become my insert fake data entry point.
Also , I add below code to conftest.py avoid insert #pytest.mark.django_db everywhere
#pytest.fixture(autouse=True)
def enable_db_access_for_all_tests(db):
pass
But, I still get error message, when I create fake data in setup_module or
setup_class.
def setup_module():
print('setup_module....')
HostSite.objects.create(prefix='us', vhost='www', enable_site=True, locale='en_us')
Error Message:
Failed: Database access not allowed, use the "django_db" mark, or the "db" or "transactional_db" fixtures to enable it.
Could anyone help me to figure out what I miss? please.
I'm having trouble moving away from django_nose.FastFixtureTestCase to django.test.TestCase (or even the more conservative django.test.TransactionTestCase). I'm using Django 1.7.11 and I'm testing against Postgres 9.2.
I have a Testcase class that loads three fixtures files. The class contains two tests. If I run each test individually as a single run (manage test test_file:TestClass.test_name), they each work. If I run them together, (manage test test_file:TestClass), I get
IntegrityError: Problem installing fixture '<path>/data.json': Could not load <app>.<Model>(pk=1): duplicate key value violates unique constraint "<app_model_field>_49810fc21046d2e2_uniq"
To me it looks like the db isn't actually getting flushed or rolled back between tests since it only happens when I run the tests in a single run.
I've stepped through the Django code and it looks like they are getting flushed or rolled back -- depending on whether I'm trying TestCase or TransactionTestCase.
(I'm moving away from FastFixtureTestCase because of https://github.com/django-nose/django-nose/issues/220)
What else should I be looking at? This seems like it should be a simple matter and is right within what django.test.TestCase and Django.test.TransactionTestCase are designed for.
Edit:
The test class more or less looks like this:
class MyTest(django.test.TransactionTestCase): # or django.test.TestCase
fixtures = ['data1.json', 'data2.json', 'data3.json']
def test1(self):
return # I simplified it to just this for now.
def test2(self):
return # I simplified it to just this for now.
Update:
I've managed to reproduce this a couple of times with a single test, so I suspect something in the fixture loading code.
One of my basic assumptions was that my db was clean for every TestCase. Tracing into the django core code I found instances where an object (in one case django.contrib.auth.User) already existed.
I temporarily overrode _fixture_setup() to assert the db was clean prior to loading fixtures. The assertion failed.
I was able to narrow the problem down to code that was in a TestCase.setUpClass() instead of TestCase.setUp(), and so the object was leaking out of the test and conflicting with other TestCase fixtures.
What I don't understand completely is I thought that the db was dropped and recreated between TestCases -- but perhaps that is not correct.
Update: Recent version of Django includes setUpTestData() that should be used instead of setUpClass()
Is there a way to see errors when django unit tests loads fixtures?
It's annoying that something like missing file, duplicate keys or badly formatted fixtures files do not show up as error when running django unit test.
If duplicate primary keys generated an error when a fixture was loaded, many existing tests would be broken. Sometimes a fixture needs to contain that error, so it can be tested against.
You can write generic tests to check for the problems you've mentioned:
If your fixture fails to load, any query relying on that fixture will fail. Write a test that attempts to retrieve an object in that fixture (eg, YourObject.objects.get(pk=1) or YourObject.objects.get(name='My Test Entry').
To test if you have duplicate primary keys, compare YourObject.objects.all().aggregate(Count('id', distinct=True)) to YourObject.objects.all().aggregate(Count('id')). If those are not equal, you have duplicate primary keys.
To avoid badly formatted fixture files, generate them from validated data. Use manage.py dumpdata, and avoid manually creating or editing fixtures.
The first test mentioned will indicate if you have an invalid path. The last recommendation removes any issues with bad formatting.
The silent failure is a result of how loaddata works. It's looking for the filenames given in the fixtures = [] list in several locations, and may fail to find the files in any one of them. In order to report an error, loaddata must track if a file has been found yet, and that requires a patch. There's a trac ticket for the silent fail, but it has been triaged at 'Design Decision Needed' for 2 years now.
You have the option to make some noise about this on the Django mailing list, and use the unapproved patch in your development copy of Django.
You can try to define your own validation method in setUp(). Testing if your fixtures are loaded would be quite easy, however, validating your fixtures would require quite a bit work depending on what format is your fixture.
I have ZF 1.11 integrated with Doctrine 1.2 + MySQL 5. I created some phpunit's tests in few files. Every test create db and populate it - using Zend_Db - then I make some actions using Doctrine's models and then I drop db using Zend_Db. I put them all in directory called "tests". And when I go to directory "tests" and write phpunit command then some of them return errors like "SQLSTATE[42S02]: Base table or view not found: 1146 Table 'here_db_name.here_table_name' doesn't exist". - but it exists, I checked! What is funny when I run every test separately then absolutly everything is ok. So, my question is: What's going on? Sorry, I can't provide code.
This is tricky to say without any code but I am making a wild guess here that if you say that every test creates and populates the db it might be that what you are experiencing is some sort of a "race-condition" because each test starts with cleaning up the database and then setting it up again.