HTML report for django tests - django

Question

I have a Django project containing an API (created with rest framework if that counts anywhere). I have added some tests for the API but in order to have an overall view of the tests, either passing, either failing or missing, I need to create an HTML report.
When the tests are finished a HTML table report should be generated which shows the endpoints and HTTP responses covered during tests, the results of the tests plus the combinations which are missing the tests.
Unfortunately I cannot understand how should I do that. I know that coverage can give me a detailed html report, but that's not what I need, I need something like this:
| Endpoint description | 200 | 400 | 403 | 404 |
| GET /endpoint1 | PASS | PASS |PASS | N/A |
| POST /endpoint1 | PASS | FAIL |MISSING| N/A |
Does anybody has any idea about that? Maybe some libs that could help out with that or what strategy should I use for that?
Thank you in advance

Answer 1

Late to the party, but this is my solution to outputting a HTML test report for Django tests. (based on HtmlTestRunner cannot be directly used with Django DiscoverRunner)
The following classes if placed in tests/html_test_reporter.py can be used as a DiscoverRunner which is patched to use HTMLTestRunner.
from django.test.runner import DiscoverRunner
from HtmlTestRunner import HTMLTestRunner
class MyHTMLTestRunner(HTMLTestRunner):
def __init__(self, **kwargs):
# Pass any required options to HTMLTestRunner
super().__init__(combine_reports=True, report_name='all_tests', add_timestamp=False, **kwargs)
class HtmlTestReporter(DiscoverRunner):
def __init__(self, **kwargs):
super().__init__(**kwargs)
# Patch over the test_runner in the super class.
html_test_runner = MyHTMLTestRunner
self.test_runner=html_test_runner
Then this is run with:
python manage.py test -v 2 --testrunner tests.html_test_reporter.HtmlTestReporter
By default Django projects use django.test.runner.DiscoverRunner to search for tests and then use PyTest to run them. HTMLTestRunner can be used with PyTest to output a HTML test report, but it does seem possible to configure PyTest to use HTMLRunner through DiscoverRunner.
Hope this helps.

Answer 2

As Django uses the python's standard unittest library, you'll have to tweak some of its parts.
First, you'll need some way to specify which tests actually test which endpoint. A custom decorator is handy for that:
from functools import wraps
def endpoint(path, code):
"""
Mark some test as one which tests specific endpoint.
"""
def inner(func):
#wraps(func)
def wrapper(*args, **kwargs):
return func(*args, **kwargs)
return wrapper
inner._endpoint_path = path
inner._endpoint_code = code
return inner
class MyTestCase(TestCase):
#endpoint(path='/path/one', code=200)
def test_my_path_is_ok(self):
response = self.client.get('/path/one?foo=bar')
self.assertEqual(response.status_code, 200)
#endpoint(path='/path/one', code=404)
def test_my_path_expected_errors(self):
response = self.client.get('/path/one?foo=qux')
self.assertEqual(response.status_code, 404)
def test_some_other_stuff(self):
# this one will not be included in our results grid.
pass
You could use a "magical" approach (e.g. special methods' names to guess the endpoint they are testing) instead, but explicit is better than implicit, right?
Then, you need a way to collect the results of your tests - specifically, of that which test the endpoints. Here we make a (very draft) subclass of unittest.TestResult to handle it:
class EndpointsTestResult(TestResult):
def __init__(self):
super(EndpointsTestResult, self).__init__()
self.endpoint_results = {}
def addError(self, test, err):
super(EndpointsTestResult, self).addError(test, err)
if hasattr(test, '_endpoint_path'):
branch = self.endpoint_results.setdefault(getattr(test, '_endpoint_path'), {})
branch[getattr(test, '_endpoint_code')] = 'MISSING'
def addFailure(self, test, err):
# similar as addError()
def addSuccess(self, test):
# similar as addError()
Finally it's time to actually output our results. Let's make a sublass of the unittest.TextTestRunner and specify it in our custom runner:
class EndpointsTestRunner(TextTestRunner):
def _makeResult(self):
self._result = EndpointsTestResult()
return self._result
def run(self, test):
super(EndpointsTestRunner).run(test)
# After running a test, print out the table
generate_a_nifty_table(self._result.endpoint_results)
class EndpointsDjangoRunner(django.test.runner.DiscoverRunner):
test_runner = EndpointsTestRunner
Now we have our custom EndpointsDjangoRunner, and we should specify it in the settings.py:
TEST_RUNNER = 'path.to.the.EndpointsDjangoRunner'
That's it. Please let me know if you spot any awkward errors in the code.