I am trying to run tests on django with coverage. It works fine, but it doesn't detect class definitions, because they are defined before coverage is started. I have following test runner, that I use, when I compute coverage:
import sys
import os
import logging
from django.conf import settings
MAIN_TEST_RUNNER = 'django.test.simple.run_tests'
if settings.COMPUTE_COVERAGE:
try:
import coverage
except ImportError:
print "Warning: coverage module not found: test code coverage will not be computed"
else:
coverage.exclude('def __unicode__')
coverage.exclude('if DEBUG')
coverage.exclude('if settings.DEBUG')
coverage.exclude('raise')
coverage.erase()
coverage.start()
MAIN_TEST_RUNNER = 'django-test-coverage.runner.run_tests'
def run_tests(test_labels, verbosity=1, interactive=True, extra_tests=[]):
# start coverage - jeśli włączmy już tutaj, a wyłączymy w django-test-coverage,
# to dostaniemy dobrze wyliczone pokrycie dla instrukcji wykonywanych przy
# imporcie modułów
test_path = MAIN_TEST_RUNNER.split('.')
# Allow for Python 2.5 relative paths
if len(test_path) > 1:
test_module_name = '.'.join(test_path[:-1])
else:
test_module_name = '.'
test_module = __import__(test_module_name, {}, {}, test_path[-1])
test_runner = getattr(test_module, test_path[-1])
failures = test_runner(test_labels, verbosity=verbosity, interactive=interactive)
if failures:
sys.exit(failures)
What can I do, to have classes also included in coverage? Otherwise I have quite a low coverage and I can't easily detect places, that really need to be covered.
The simplest thing to do is to use coverage to execute the test runner. If your runner is called "runner.py", then use:
coverage run runner.py
You can put your four exclusions into a .coveragerc file, and you'll have all of the benefits of your coverage code, without keeping any of your coverage code.
Related
I am trying to configure Jenkins to build my Selenium Webdriver Python code.
When i click Build Now it fails
The Console output shows the following:
Building in workspace C:\Program Files\Jenkins\workspace\ClearCore
[ClearCore] $ cmd /c call C:\Windows\TEMP\hudson6133135491793466847.bat
C:\Program Files\Jenkins\workspace\ClearCore>copy E:\RL Fusion\projects\Jenkins sample\ClearCore501\TestCases\*.py
The system cannot find the file specified.
C:\Program Files\Jenkins\workspace\ClearCore>python smoketests.py
python: can't open file 'smoketests.py': [Errno 2] No such file or directory
C:\Program Files\Jenkins\workspace\ClearCore>exit 2
Build step 'Execute Windows batch command' marked build as failure
Recording test results
ERROR: Publisher 'Publish JUnit test result report' failed: No test report files were found. Configuration error?
Finished: FAILURE
In PyCharm i have a smoketests.py file as follows:
import unittest
from xmlrunner import xmlrunner
from TestCases.LoginPage_TestCase import LoginPage_TestCase
from TestCases.AdministrationPage_TestCase import AdministrationPage_TestCase
from TestCases.DataConfigurationPage_TestCase import DataConfigurationPage_TestCase
login_tests = unittest.TestLoader().loadTestsFromTestCase(LoginPage_TestCase)
admin_tests = unittest.TestLoader().loadTestsFromTestCase(AdministrationPage_TestCase)
dataconf_tests = unittest.TestLoader().loadTestsFromTestCase(DataConfigurationPage_TestCase)
smoke_tests = unittest.TestSuite([login_tests, admin_tests, dataconf_tests])
xmlrunner.XMLTestRunner(verbosity=2, output='test-reports').run(smoke_tests)
I have a test_HTMLRunner.py as follows:
import unittest
import HTMLTestRunner
import os
from TestCases.LoginPage_TestCase import LoginPage_TestCase
from TestCases.AdministrationPage_TestCase import AdministrationPage_TestCase
from TestCases.DataConfigurationPage_TestCase import DataConfigurationPage_TestCase
# get the directory path to output report file
result_dir = os.getcwd()
login_tests = unittest.TestLoader().loadTestsFromTestCase(LoginPage_TestCase)
admin_tests = unittest.TestLoader().loadTestsFromTestCase(AdministrationPage_TestCase)
dataconf_tests = unittest.TestLoader().loadTestsFromTestCase(DataConfigurationPage_TestCase)
smoke_tests = unittest.TestSuite([login_tests, admin_tests, dataconf_tests])
# open the report file
outfile = open(result_dir + "\TestReport.html", "w")
# configure HTMLTestRunner options
runner = HTMLTestRunner.HTMLTestRunner(stream=outfile,
title='Test Report',
description='LADEMO create a basic project test')
# run the suite using HTMLTestRunner
runner.run(smoke_tests)
I have a suite.py as follows:
import sys
import unittest
import HTMLTestRunner
import os
import unittest
import AdministrationPage_TestCase
import LoginPage_TestCase
import DataConfigurationPage_TestCase
class Test_Suite(unittest.TestCase):
def test_main(self):
# suite of TestCases
self.suite = unittest.TestSuite()
self.suite.addTests([
unittest.defaultTestLoader.loadTestsFromTestCase(LoginPage_TestCase.LoginPage_TestCase),
unittest.defaultTestLoader.loadTestsFromTestCase(AdministrationPage_TestCase.AdministrationPage_TestCase),
unittest.defaultTestLoader.loadTestsFromTestCase(DataConfigurationPage_TestCase.DataConfigurationPage_TestCase),
])
runner = unittest.TextTestRunner()
runner.run (self.suite)
import unittest
if __name__ == "__main__":
unittest.main()
In Jenkins I have configured the following:
From the section Build, Execute Windows Batch Command
copy E:\RL Fusion\projects\Jenkins sample\ClearCore501\TestCases\*.py
python smoketests.py
From the section Post-Build Actions, Publish JUnit test result report
test_reports/*..xml
Below test_reports/*..xml it shows:
‘test_reports/*..xml’ doesn’t match anything: even ‘test_reports’ doesn’t exist
How do i get this to work please? What am i doing wrong?
Is there any sample demo I could follow and then I can get my setup to work?
Thanks,
Riaz
The problem looks to be in the copy step of you batch file. Notice how it says it cant find the file. Surround the source and destination paths with double quotes so that windows knows your path has spaces in it.
It also appears the copy operation doesn't have a destination specified. You should may want to specify that too. Although apparently that isn't a requirement, as I just found out :).
Once the copy operation succeeds, check the workspace directory to see if the file(s) you expect are present.
Alternatively, you can tell the Jenkins job to use a custom workspace, the directory where your tests live. With this configuration you don't even have to worry about copying files.
Here's how:
In the job config in Jenkins, open the Advanced Project Options and select use custom workspace and set the directory to E:\RL Fusion\projects\Jenkins sample\ClearCore501\TestCases\.
Then the build command can just be python smoketests.py.
I am learning test driven development...
I wrote a test that should fail but it's not...
(env)glitch:ipals nathann$ ./manage.py test npage/
Creating test database for alias 'default'...
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK
Destroying test database for alias 'default'...
in npage/ I have tests.py:
from django.test import TestCase
from npage.models import Tip
import datetime
# Example
class TipTester(TestCase):
def setUp(self):
print dir(self)
Tip.objects.create(pk=1,
text='Testing',
es_text='Probando')
def tips_in_spanish(self):
my_tip = Tip.objects.get(pk=1)
my_tip.set_language('es')
self.assertEqual(my_tip.text, 'this does not just say \'Probando\'')
What am I doing wrong? I've read this but I still can't figure out what is going wrong here.
Your test functions need to start with test:
def test_tips_in_spanish(self):
Docs here
"When you run your tests, the default behavior of the test utility is to find all the test cases (that is, subclasses of unittest.TestCase) in any file whose name begins with test, automatically build a test suite out of those test cases, and run that suite."
I am trying to write unit tests for a Python application and I want to write a unit test to verify that the entries in a dictionary are correct (based on the OS).
PROBLEM: I cannot mock the os that I am on. I just want to mock the os so that I can verify that the entries of a dictionary are valid. I have tried monkey patching, mocking and a patch but nothing works at the moment (which I do not doubt is due to my limited knowledge of unit-testing). The current code that I have is below. Any (detailed) help as to why this is not working or what I am doing wrong/should be doing instead will be greatly appreciated! :D
So I have:
package.subpackage.module (mymodule.py)
import sys
if platform.system() == 'Linux':
mydictionary = {"1":one, "2":two, "3":three}
elif platform.system() == 'other_os':
mydictionary = {"A":aaa, "B":bee, "c":cee}
I created a unit test here with the following code:
package.subfolder.subsubfolder.module (myunittestmodule.py)
def test_dictionary():
import sys
sys.modules['platform'] = Mock()
import platform
targetPlatform = platform
targetPlatform.system = Mock(return_value="other_os")
from package.subpackage.module import mydictionary
for entry in mydictionary:
print(entry)
# for now I use print to see if this even works
# but it only returns the dictionary for my os (linux), not anything else :(
I created custom django-admin commands
But, I don't know how to test it in standard django tests
If you're using some coverage tool it would be good to call it from the code with:
from django.core.management import call_command
from django.test import TestCase
class CommandsTestCase(TestCase):
def test_mycommand(self):
" Test my custom command."
args = []
opts = {}
call_command('mycommand', *args, **opts)
# Some Asserts.
From the official documentation
Management commands can be tested with the call_command() function. The output can be redirected into a StringIO instance
You should make your actual command script the minimum possible, so that it just calls a function elsewhere. The function can then be tested via unit tests or doctests as normal.
you can see in github.com example
see here
def test_command_style(self):
out = StringIO()
management.call_command('dance', style='Jive', stdout=out)
self.assertEquals(out.getvalue(),
"I don't feel like dancing Jive.")
To add to what has already been posted here. If your django-admin command passes a file as parameter, you could do something like this:
from django.test import TestCase
from django.core.management import call_command
from io import StringIO
import os
class CommandTestCase(TestCase):
def test_command_import(self):
out = StringIO()
call_command(
'my_command', os.path.join('path/to/file', 'my_file.txt'),
stdout=out
)
self.assertIn(
'Expected Value',
out.getvalue()
)
This works when your django-command is used in a manner like this:
$ python manage.py my_command my_file.txt
A simple alternative to parsing stdout is to make your management command exit with an error code if it doesn't run successfully, for example using sys.exit(1).
You can catch this in a test with:
with self.assertRaises(SystemExit):
call_command('mycommand')
I agree with Daniel that the actual command script should do the minimum possible but you can also test it directly in a Django unit test using os.popen4.
From within your unit test you can have a command like
fin, fout = os.popen4('python manage.py yourcommand')
result = fout.read()
You can then analyze the contents of result to test whether your Django command was successful.
If I run the following command:
>python manage.py test
Django looks at tests.py in my application, and runs any doctests or unit tests in that file. It also looks at the __ test __ dictionary for extra tests to run. So I can link doctests from other modules like so:
#tests.py
from myapp.module1 import _function1, _function2
__test__ = {
"_function1": _function1,
"_function2": _function2
}
If I want to include more doctests, is there an easier way than enumerating them all in this dictionary? Ideally, I just want to have Django find all doctests in all modules in the myapp application.
Is there some kind of reflection hack that would get me where I want to be?
I solved this for myself a while ago:
apps = settings.INSTALLED_APPS
for app in apps:
try:
a = app + '.test'
__import__(a)
m = sys.modules[a]
except ImportError: #no test jobs for this module, continue to next one
continue
#run your test using the imported module m
This allowed me to put per-module tests in their own test.py file, so they didn't get mixed up with the rest of my application code. It would be easy to modify this to just look for doc tests in each of your modules and run them if it found them.
Use django-nose since nose automatically find all tests recursivelly.
Here're key elements of solution:
tests.py:
def find_modules(package):
"""Return list of imported modules from given package"""
files = [re.sub('\.py$', '', f) for f in os.listdir(os.path.dirname(package.__file__))
if f.endswith(".py") and os.path.basename(f) not in ('__init__.py', 'test.py')]
return [imp.load_module(file, *imp.find_module(file, package.__path__)) for file in files]
def suite(package=None):
"""Assemble test suite for Django default test loader"""
if not package: package = myapp.tests # Default argument required for Django test runner
return unittest.TestSuite([doctest.DocTestSuite(m) for m in find_modules(package)])
To add recursion use os.walk() to traverse module tree and find python packages.
Thanks to Alex and Paul. This is what I came up with:
# tests.py
import sys, settings, re, os, doctest, unittest, imp
# import your base Django project
import myapp
# Django already runs these, don't include them again
ALREADY_RUN = ['tests.py', 'models.py']
def find_untested_modules(package):
""" Gets all modules not already included in Django's test suite """
files = [re.sub('\.py$', '', f)
for f in os.listdir(os.path.dirname(package.__file__))
if f.endswith(".py")
and os.path.basename(f) not in ALREADY_RUN]
return [imp.load_module(file, *imp.find_module(file, package.__path__))
for file in files]
def modules_callables(module):
return [m for m in dir(module) if callable(getattr(module, m))]
def has_doctest(docstring):
return ">>>" in docstring
__test__ = {}
for module in find_untested_modules(myapp.module1):
for method in modules_callables(module):
docstring = str(getattr(module, method).__doc__)
if has_doctest(docstring):
print "Found doctest(s) " + module.__name__ + "." + method
# import the method itself, so doctest can find it
_temp = __import__(module.__name__, globals(), locals(), [method])
locals()[method] = getattr(_temp, method)
# Django looks in __test__ for doctests to run
__test__[method] = getattr(module, method)
I'm not up to speed on Djano's testing, but as I understand it uses automatic unittest discovery, just like python -m unittest discover and Nose.
If so, just put the following file somewhere the discovery will find it (usually just a matter of naming it test_doctest.py or similar).
Change your_package to the package to test. All modules (including subpackages) will be doctested.
import doctest
import pkgutil
import your_package as root_package
def load_tests(loader, tests, ignore):
modules = pkgutil.walk_packages(root_package.__path__, root_package.__name__ + '.')
for _, module_name, _ in modules:
try:
suite = doctest.DocTestSuite(module_name)
except ValueError:
# Presumably a "no docstrings" error. That's OK.
pass
else:
tests.addTests(suite)
return tests