Since upgrading my Django project to Django 1.8 using the default Django test runner, I'm finding that my tests take between 10 and 15 seconds to start up because of the new default behavior of running migrations before tests. It now takes longer for my test suite to start up then it does to run the actual tests themselves. I only have seven models, none of which are that complex, and over 180 tests. I'm running my tests against a PostgreSQL 9.4.4. database on a 1.86 GHz Mac laptop with 4 GB of RAM. I found this question which said to create a settings_test.py file that contains a MIGRATION_MODULE setting and then specify that settings_test file when running your tests. Could someone explain further how to implement this solution? Is the settings_test file supposed to look like this if you have two apps?
MIGRATION_MODULES={'app1': 'app1.migrations_not_used_in_tests',
'app2': 'app2.migrations_not_used_in_tests'}
If that's true, how would you run your tests as described if you have multiple apps?
DJANGO_SETTINGS_MODULE='???.settings_test' python manage.py test
I tried creating the exact settings_test.py file that specified 'myapp' even though I don't have an app named 'myapp' and while my tests ran, the migrations also ran prior to the tests as I expected they would. I'm sorry if this seems like a dumb question but I'm relatively new to Django.
Thanks.
Related
I'm a hobby web developer looking to build my first product with the django web framework which I will host on Heroku. Cypress looks like a great modern tool for UI testing which I certainly want to use. In local development I am using it within django testing so that the django staticliveservertestcase runs first - which creates a separate development server process, and a test database, if one hasn't been created already. Within this test I then call a bash script which executes a cypress UI test. See the example below.
Bash Script -
#!/bin/bash
echo $1 # the 1st command line argument
echo $2 # the 2nd command line argument
./node_modules/.bin/cypress run --env baseUrl=$1 --spec $2
Django Test Code -
class ExampleTest(StaticLiveServerTestCase):
def test_view_url(self):
from subprocess import run
exit_code = run([
'./cypress/run.sh',
f'http://{self.server_thread.host}:{self.server_thread.port}',
'cypress/integration/sample_spec.js'
])
self.assertEqual(0, exit_code.returncode)
From my understanding this is the opposite to the standard cypress approach. Normally cypress runs outside of the app independently and simply runs through all the tests which interact with the app. Here the cypress tests are called by the django process. Unless I am mistaken there are obvious advantages to this approach. I can test the UI and whether objects where created in the database after the UI test has run.
Is it possible, and if this how, to do this within Heroku? So Heroku would simply call the test script -
python manage.py test
And I get UI testing and unit testing all done in one go. It would also be nice to see the tests running in interactive mode. Which may be a challenge because Heroku obviously runs everything in a linux container - I don't plan on using Docker incidentally because I think that limits a lot of the features Heroku offers.
I have a django project and I am trying to write some tests for it. However, my initial_data fixtures cause an error when running the test.
The error that I am getting is:
django.db.utils.ProgrammingError: Problem installing fixture 'accounts/fixtures/initial_data.json': Could not load auth.Group(pk=1): relation "auth_group" does not exist
LINE 1: UPDATE "auth_group" SET "name" = '...
If I rename my fixture to something other than initial_data so that it doesn't get loaded by default, it works, but I don't want to rename my fixtures, because that would mean that I can no longer run loaddata without arguments.
I have found this bug, but my project does not have any initial migrations. Also, I have other fixtures which are loaded just fine.
So far, I have tried:
flushing my development database, as well as deleting any possible migration files
deleting and re-creating my virtual env
changing the order of my apps in INSTALLED_APPS
calling the flush command in the .setUp() method.
I should mention that I am using the APITestCase from django-rest-framework.
Any suggestions are welcomed. Thanks.
Ok, so finally, it seems that the problem wasn't just when I was testing. When I changed back to running my server, I noticed I was getting the same error.
Every single similar problem I found had something to do with migrations, but I didn't even had those, because running ./manage.py makemigrations was not generating them.
So I ended up doing ./manage.py makemigrations *app_name* for each of my apps, and everything started working again ...
I have a app that is deployed to Heroku, and I'd like to be able to run the test suite post-deployment on the target environment. I am using the Heroku Postgres add-on, which means that I have access to a single database only. I have no rights to create new databases, which in turn means that the standard Django test command fails, as it can't create the test_* database.
$ heroku run python manage.py test
Running `python manage.py test` attached to terminal... up, run.9362
Creating test database for alias 'default'...
Got an error creating the test database: permission denied to create database
Is there any way around this?
Turns out I was in the wrong. I was not testing what I thought was being tested... Since Heroku's Routing Mesh was sending requests to different servers, the LiveServerTestCase was starting a web server on one machine and Selenium was connecting to other machines altogether.
By updating the Heroku Procfile to:
web: python src/manage.py test --liveserver=0.0.0.0:$PORT
overriding the DATABASES setting to point to the test database, and customizing the test suite runner linked to below (the same idea still holds: override setup_databases so that it only drops/re-creates tables, not the entire database), I was able to run remote tests. But this is even more hacky/painful/inelegant. Still looking for something better! Sorry about the confusion.
(updated answer below)
Here are the steps that worked for me:
Create an additional, free Postgres database using the Heroku toolbelt
heroku addons:add heroku-postgresql:dev
Use the HerokuTestSuiteRunner class which you'll find here.
This custom test runner requires that you define a TEST_DATABASES setting which follows the typical DATABASES format. For instance:
TEST_DATABASES = {
'default': dj_database_url.config(env='TEST_DATABASE_URL')
}
Then, have the TEST_RUNNER setting be a Python path to wherever HerokuTestSuiteRunner can be found.
You should now be able to run Django tests on Heroku using the given database. This is very much a quick hack... Let me know how it could be improved / made less hackish. Enjoy!
(original answer below)
A few relevant solutions have been discussed here. As you can read in the Django docs, "[w]hen using the SQLite database engine, the tests will by default use an in-memory database".
Although this doesn't thoroughly test the database engine you're using on Heroku (I'm still on the lookout for a solution that does that), setting the database engine to SQLite will at least allow you to run your tests.
See the above-linked StackOverflow question for some pointers. There are at least two ways out: testing if 'test' in sys.argv before forcing SQLite as the database engine, or having a dedicated settings file used in testing, which you can then pass to django manage.py test using the --settings option.
Starting with version 1.8, Django now has an option called keepdb, which allows for the same database to be reused during tests.
The --keepdb option can be used to preserve the test database between test runs.
This has the advantage of skipping both the create and destroy actions which can greatly decrease the time to run tests, especially those in a large test suite.
If the test database does not exist, it will be created on the first run and then preserved for each subsequent run.
Any unapplied migrations will also be applied to the test database before running the test suite.
Since it also allows for the test database to exist prior to running tests, you can simply add a new Postgres Heroku instance to your dyno and configure the tests to use that particular database.
Bonus : you can also use the failfast option, which exits as soon as your first test crashes, so that you don't have to wait for all tests to complete.
However, if you are deploying things to Heroku and you are using Heroku Pipelines, an even better option is available : Heroku CI.
I was having a problem that the django test runner wasn't finding the tests for my app, like this one:
Django test runner not finding tests
One of the comments on that thread suggested creating a new app with django-admin.py and seeing if the tests ran there. e.g.
django-admin.py startapp delme
then
adding "delme" to my INSTALLED_APPS
then
copying my tests.py from the app where it wasn't getting found into delme/
and viola! the tests did run. So, OK, I have a work arround, but I don't understand. I re-read what I think should be the relevant parts of the django documentation but the penny refuses to drop.
BTW, the app works via runserver and wsgi, so there doesn't appear to be any gross configuration problem. And my tests all pass from their new home, so I obviously need more tests :)
Specifically, I'm running django in a virtualenv, so I had run "django-admin.py startapp" in the (activated) virtualenv where I wanted the tests to run. This doesn't make the tests run in my other virtualenvs, I still have the old symptoms there (Ran 0 tests). I have a multitude of virtualenvs, managed by non-trivial paver scripts. One uses "path.copytree" for deploying projects, rewrites apache config files, restarts apache, writes wsgi files using the appropriate virtualenv, etc. The other uses PIP/GCC/aptitude/etc for bootstrapping/tearing down the different environments, updating packages as per configuration, etc. So I want to understand the difference between django-startapp and simply copying files, so I can fix these paver scripts so the tests can run in any environment I want them to.
The only thing that makes sense to me, after reading your description, is the location of paths for your existing apps. Can you confirm the following things:
Your app is at the same folder level as the delme app
Your app folder contains an __init__.py file
Your app is listed in the INSTALLED_APPS setting
I'm going to guess that it's a missing __init__.py file, as that trips some people up. To answer your specific question, django-admin startapp doesn't do anything magical. It just creates the right folders and files in the correct place.
Your folder structure should be...
my_project/
__init__.py
manage.py
settings.py
my_app/
__init__.py
tests.py
models.py
delme/
__init__.py
tests.py
models.py
Also note this comment
You can't easily name the TestCase class directly. You name the app, the Django runner does it's discovery thing by looking in models.py and tests.py
solved (still with a little whiff of magic).
diff showed that my old app didn't have a models.py but the new app ("delme", working) did. I didn't think the old app needed one, it was importing all it's domain classes from other places.
Touching an empty models.py in my old app fixed it, now the test runner finds the tests.py and everything works as expected. Condlusion - if an app has no models.py, the django test runner won't find the app's tests.py.
What I said about not working in different virtualenvs was bogus (red herring), I was a bit confused about what my deploy scripts were doing.
In my Django projects I use sqlite database to run tests. Since it uses only memory, it's much faster than MySQL, but it's still not fast enough. During tests, only one of 4 processors is used, and not much memory is consumed. So, I'd like to have 4 sqlite databases in memory to run 4 tests in parallel.
Has anyone tried this?
Since Django 1.9 it's possible to run the tests in parallel by Django with its built-in unit-test features.
Django Docs: https://docs.djangoproject.com/en/3.0/ref/django-admin/#cmdoption-test-parallel
According to the Django 3.0 documentation there is a --parallel option that you can use.
--parallel [N]
Runs tests in separate parallel processes. Since modern processors have multiple cores, this allows running tests significantly faster.
So you can use the following command to execute the tests in parallel.
python manage.py test --parallel
You can adjust the number of processes either by providing it as the option’s value, e.g. --parallel=4, or by setting the DJANGO_TEST_PROCESSES environment variable.
This can help considerably reduce your test execution time if you have a large Django project with quite a few test unit cases.
You can use a parallel test runner for Django and Twisted described here: http://www.tomaz.me/2011/04/03/making-django-and-twisted-tests-faster.html (the source lives here https://github.com/Kami/parallel-django-and-twisted-test-runner - link at the end of the post). You can use it as described in Django docs on testing.
There is also a nose parallel test runner.
You can easily split the testing for apps on parrallalel on linux by:
$ python manage.py test cms & \
python manage.py test blog & \
python manage.py test store & \
python manage.py test registration
$
Could be helpfull for big projects with a lot of apps, the best would be a bash scripts that runs tests every four apps.