Problems with Django test runner and test client login with authentication backend - django

Using the shell, I can do this:
>>> from django.test.client import Client
>>> from django.test.utils import setup_test_environment
>>> setup_test_environment()
>>> c = Client()
>>> c.login(username="dev", password="password")
True
>>> r = c.get('/')
>>> r.status_code
200
Now with this in the test.py file:
from django.test.client import Client
__test__ = {"doctest": """
>>> c = Client()
>>> c.login(username='dev', password='password')
True
>>> r = c.get('/')
>>> r.status_code
200
"""}
I get this output:
Failed example:
c.login(username="dev", password="password")
Expected:
True
Got:
False
------------------------------------------------------
Failed example:
r.status_code
Expected:
200
Got:
302
I've looked all over the internet and I can't find anything that helps with this situation. Any ideas?
On a similar note, I've commented out: from django.views.decorators.debug import sensitive_post_parameters and all #sensitive_post_parameters() decorators from my code because each time I run ./manage.py test app django complains:
Could not import app.views. Error was: No module named debug
Removing this decorator and import statement allows it to move forward.
Im very much lost and I need StackOverflow! Thanks everyone.

sensitive_post_parameters is a new feature in Django 1.4, so if you're running Django 1.3 or earlier then the import will fail.
I believe that the commands you tried in the shell were run on the normal database. When you run your doc tests, Django will set up a test database. It looks like your user dev isn't in the test database when you run the doc tests, so the login attempt fails. One option is to create the User with User.objects.create_user before you attempt the login. Another option is to use fixtures.
With Django, I would recommend writing unit tests instead of doc tests. One big advantage is that it's easy to include fixtures to load initial data (e.g. users) into the test database. Another is that Django takes care of refreshing the database between unit tests.

Related

Cannot Authenticate Test Requests with Both User Token and API Key in Django Rest Framework

My project requires two authentication methods for some endpoints:
A global API key using this app.
A user authentication token using the default one provided by DRF
All works fine when working with the API using Postman or an iOS app, but I couldn't make the authentication work in my tests. The user auth works fine, but the global key fails.
This is how the HTTP headers look in Postman:
X-Api-Key: KEY_VALUE
Authorization: Token TOKEN_VALUE
I tried to experiment with the code in the shell and authentication worked fine using the exact same code used in the test! Only in the tests it fails so I'm not really sure how to debug this further.
Edit:
You can see a complete project on github.
Test output:
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
F
======================================================================
FAIL: test (app.tests.MyTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File ".../authtest/app/tests.py", line 21, in test
self.assertEqual(response.status_code, status.HTTP_200_OK)
AssertionError: 403 != 200
----------------------------------------------------------------------
Ran 1 test in 0.112s
FAILED (failures=1)
Destroying test database for alias 'default'...
When I open the shell with python manage.py shell and copy and paste the test code:
>>> from rest_framework.test import (APITestCase, APIRequestFactory, force_authenticate)
>>> from rest_framework import status
>>> from accounts.models import User
>>> from app.views import MyView
>>> user = User.objects.create(email="test#test.com")
>>> user.set_password('1234')
>>> factory = APIRequestFactory()
>>> API_KEY = "KIkKSSz7.ziURxOZv8e66f28eMLYwPNs7eEhrNtYl"
>>> headers = {"HTTP_X_API_KEY": API_KEY}
>>> request = factory.get('/myview', **headers)
>>> force_authenticate(request, user=user)
>>> response = MyView.as_view()(request)
>>> response.status_code
200
Also making the request with postman works. Any idea what's going on here?
The problem is the hardcoded API_KEY you are passing in your test which is not a valid key resulting in status 403. I think the reason might be django uses a separate test database when running tests and djangorestframework-api-key uses APIKey model to generate api_key.
You can confirm this by:
Removing HasAPIKey from permission classes in MyView and running test again.
or by passing empty string as API_KEY in your test resulting in same
error you are getting now.
so what i might suggest is generating a new valid api_key and passing it instead of hardcoded key.
Here i am sharing an updated test file of how you can carry out your test
from rest_framework.test import (APITestCase, APIRequestFactory, force_authenticate)
from rest_framework import status
from accounts.models import User
from .views import MyView
from rest_framework_api_key.models import APIKey
class MyTests(APITestCase):
def test(self):
user = User.objects.create(email="test#test.com")
user.set_password('1234')
user.save()
factory = APIRequestFactory()
api_key, key = APIKey.objects.create_key(name="my-remote-service")
headers = {"HTTP_X_API_KEY": key}
request = factory.get('/myview', **headers)
force_authenticate(request, user=user)
response = MyView.as_view()(request)
self.assertEqual(response.status_code, status.HTTP_200_OK)

Pytest use django_db with rest framework

I am trying to get a simple test to work against the real django_db not the test database using the django rest framework.
Basic test setup:
import pytest
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APIClient
#pytest.mark.django_db
def test_airport_list_real():
client = APIClient()
response = client.get(reverse('query_flight:airports-list'))
assert response.status_code == 200
assert len(response.json()) > 0
Running this test I get:
___________________________ test_airport_list_real ____________________________
#pytest.mark.django_db
def test_airport_list_real():
client = APIClient()
response = client.get(reverse('query_flight:airports-list'))
assert response.status_code == 200
> assert len(response.json()) > 0
E assert 0 > 0
E + where 0 = len([])
E + where [] = functools.partial(<bound method Client._parse_json of <rest_framework.test.APIClient object at 0x000001A0AB793908>>, <Response status_code=200, "application/json">)()
E + where functools.partial(<bound method Client._parse_json of <rest_framework.test.APIClient object at 0x000001A0AB793908>>, <Response status_code=200, "application/json">) = <Response status_code=200, "application/json">.json
query_flight\tests\query_flight\test_api.py:60: AssertionError
When just running in the shell using pipenv run python manage.py shell I get the expected results:
In [1]: from django.urls import reverse
In [2]: from rest_framework.test import APIClient
In [3]: client = APIClient()
In [4]: response = client.get(reverse('query_flight:airports-list'))
In [5]: len(response.json())
Out[5]: 100
Using the following packages:
pytest-django==3.2.1
pytest [required: >=2.9, installed: 3.5.1]
djangorestframework==3.8.2
django [required: >=1.8, installed: 2.0.5]
Is there anyway to get pytest to access the real database in this way?
The django_db marker is only responsible to provide a connection to the test database for the marked test. The django settings passed to pytest-django are solely responsible for the selection of database used in the test run.
You can override the database usage in pytest-django by defining the django_db_setup fixture. Create a conftest.py file in the project root if you don't have it yet and override the db configuration:
# conftest.py
import pytest
#pytest.fixture(scope='session')
def django_db_setup():
settings.DATABASES['default'] = {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': 'path/to/dbfile.sqlite3',
}
However, you shouldn't use the real database in tests. Make a dump of your current db to get a snapshot of test data (python manage.py dumpdata > testdata.json) and load it into an empty test database to populate it before the test run:
# conftest.py
import pytest
from django.core.management import call_command
#pytest.fixture(scope='session')
def django_db_setup(django_db_setup, django_db_blocker):
with django_db_blocker.unblock():
call_command('loaddata', 'testdata.json')
Now, you can't possibly corrupt your real db when running tests; any future changes in real db will not cause the tests to fail (for example, when some data was deleted) and you always have a deterministic state on each test run. If you need some additional test data, add it in JSON format to testdata.json and your tests are good to go.
Source: Examples in pytest-django docs.
You've got a couple options. Using Django's TestClient or DRF's APIClient will use the test database and local version of your app by default. To connect to your live API, you could use a library like Requests to perform HTTP requests, then use those responses in your tests:
import requests
#pytest.mark.django_db
def test_airport_list_real():
response = requests.get('https://yourliveapi.biz')
assert response.status_code == 200
assert len(response.json()) > 0
Just be extra careful to perform exclusively read-only tests on that live database.

Running django unit tests from shell with ipython notebook has strange behavior

I'm using an ipyhton notebook connected to Django shell to run some tests. I am on django 1.4.
First, if I run as configured below sometimes it works perfectly and other times, it just hangs with no output and no errors. I have to completely kill the ipyhton kernel and close all notebooks and try again (when the hang event occurs, all open notebooks stop working)
If i inherit from unittest.TestCase instead of django.test.TestCase it works perfect every time. However, I need the latter so i can use the django's TestCase.client in my actual tests.
NOTE: In both cases I am skipping the test database because I'm getting a failure on a missing celery database. I will cross that bridge at another time.
The notebook:
from django.utils import unittest
from django.test import TestCase
from django.test.utils import setup_test_environment
from django.test.simple import DjangoTestSuiteRunner
class MyTestCase(TestCase):
def test_001(self):
print "ok"
def test_002(self):
self.assertEqual(True , True)
if __name__ == '__main__':
setup_test_environment()
runner = DjangoTestSuiteRunner(verbosity=1, interactive=True, failfast=True)
suite = unittest.TestLoader().loadTestsFromTestCase(MyTestCase)
#old_config = runner.setup_databases()
result = runner.run_suite(suite)
#runner.teardown_databases(old_config)
runner.suite_result(suite, result)
In my case, I just created a test_runner function that accepts a test_class parameter, like this:
def test_runner(test_class):
from django.utils import unittest
from django.test.utils import setup_test_environment
from django.test.simple import DjangoTestSuiteRunner
setup_test_environment()
runner = DjangoTestSuiteRunner(verbosity=1, interactive=True, failfast=True)
suite = unittest.TestLoader().loadTestsFromTestCase(test_class)
result = runner.run_suite(suite)
runner.suite_result(suite, result)
After that, you could just run:
test_runner(MyTestCase)
in ipython notebook.
Make sure to use the one that's provided by django-extensions, by running:
manage.py shell_plus --notebook
Hope that helps.

Start django shell with a temporary database

I want to fire up the django shell with a temporary database (like what's done when doing django tests)
Is there any command like:
python manage.py testshell
where I can create a bunch of bogus models without polluting my database?
Nevermind, this blog post explains it
>>> from django import test
>>> test.utils.setup_test_environment() # Setup the environment
>>> from django.db import connection
>>> db = connection.creation.create_test_db() # Create the test db
You could just turn autocommit off:
from django.db import transaction
transaction.set_autocommit(False)

Django Lettuce built-in server 500 response

I am running the Lettuce built-in server to test that it returns a given reponse however, it shows a 500 response.
My features file:
Feature: home page loads
Scenario: Check that home page loads with header
Given I access the home url
then the home page should load with the title "Movies currently showing"
My steps file:
#step(u'Given I access the home url')
def given_i_access_the_home_url(step):
world.response = world.browser.get(django_url('/'))
sleep(10)
#step(u'then the home page should load with the title "([^"]*)"')
def then_the_home_page_should_load_with_the_title_group1(step, group1):
assert group1 in world.response
My Terrains file:
from django.core.management import call_command
from django.test.simple import DjangoTestSuiteRunner
from lettuce import before, after, world
from logging import getLogger
from selenium import webdriver
try:
from south.management.commands import patch_for_test_db_setup
except:
pass
logger = getLogger(__name__)
logger.info("Loading the terrain file...")
#before.runserver
def setup_database(actual_server):
'''
This will setup your database, sync it, and run migrations if you are using South.
It does this before the Test Django server is set up.
'''
logger.info("Setting up a test database...")
# Uncomment if you are using South
# patch_for_test_db_setup()
world.test_runner = DjangoTestSuiteRunner(interactive=False)
DjangoTestSuiteRunner.setup_test_environment(world.test_runner)
world.created_db = DjangoTestSuiteRunner.setup_databases(world.test_runner)
call_command('syncdb', interactive=False, verbosity=0)
# Uncomment if you are using South
# call_command('migrate', interactive=False, verbosity=0)
#after.runserver
def teardown_database(actual_server):
'''
This will destroy your test database after all of your tests have executed.
'''
logger.info("Destroying the test database ...")
DjangoTestSuiteRunner.teardown_databases(world.test_runner, world.created_db)
#before.all
def setup_browser():
world.browser = webdriver.Firefox()
#after.all
def teardown_browser(total):
world.browser.quit()
What could be the problem with the server, why a 500 response error?
I managed to find what the problem is, the migrations were not running on syncdb