I am new to using Google test framework and still going through lot of materials to utilize it to full extent.
Is there any way I can dictate/specify a relation between test cases so that it can be executed conditionally? Like lets say I have two tests; Can I run the second test only if the first succeeds? I am not really sure if it falls under the original rule of testing 'units' but was just wondering if its possible.
There no way to do it in source. Possible solution use shell scripts and run tests using filter.
Python example:
from subprocess import call
def runTest(pattern):
return call(['test', '--gtest_filter=%s' % pattern])
if runTest('FirstPriorityTestPattern') == 0:
return runTest('SecondPriorityTestPattern')
return 1
Related
I have a pair of tests like this:
Make sure a task's deletion status initializes as None
def test_initial_task_deletion_status_is_none(self):
unfinished_task = Task.objects.get(tasked_with="Unfinished Test")
self.assertIsNone(unfinished_task.delete_status)
# Make sure a task's deletion status changes appropriately
def test_unfinished_task_deletion_status_updates_appropriately(self):
unfinished_task = Task.objects.get(tasked_with="Unfinished Test")
unfinished_task.timed_delete(delta=.1)
self.assertIs(unfinished_task.delete_status, "Marked for Deletion")
This will go on, but I'll have unfinished_task = Task.objects.get(tasked_with="Unfinished Test") at the beginning of every one. Is there a way to split these types of things into separate tests, but use the same query result?
Assuming you're using Django's testing framework, then you can do this using setUp().
More about unittest.TestCase.setUp() here
So your updated snippet would look like:
from django.test import TestCase
class MyTestCase(TestCase):
def setUp(self):
self.unfinished_task = Task.objects.get(tasked_with="Unfinished Test")
def test_initial_task_deletion_status_is_none(self):
self.assertIsNone(self.unfinished_task.delete_status)
# Make sure a task's deletion status changes appropriately
def test_unfinished_task_deletion_status_updates_appropriately(self):
self.unfinished_task.timed_delete(delta=.1)
self.assertIs(self.unfinished_task.delete_status, "Marked for Deletion")
You can place the repeated line in the setUp method, and that will make your code less repetitive, but as DanielRoseman pointed out, it will still be run for each test, so you won't be using the same query result.
You can place it in the setUpTestData method, and it will be run only once, before all the tests in MyTestCase, but then your unfinished_task object will be a class variable, shared across all the tests. In-memory modifications made to the object during one test will carry over into subsequent tests, and that is not what you want.
In read-only tests, using setUpTestData is a good way to cut out unnecessary queries, but if you're going to be modifying the objects, you'll want to start fresh each time.
For example I want to write several tests cases like this
class Test(APITestCase):
def setUp(self):
....some payloads
def test_create_user(self):
....create the object using payload from setUp
def test_update_user(self):
....update the object created in above test case
In the example above, the test_update_user failed because let's say cannot find the user object. Therefore, for that test case to work, I have to create the user instead test_update_user again.
One possible solution, I found is to run create user in setUp. However, I would like to know if there is a way to chain test cases to run one after another without deleting the object created from previous test case.
Rest framework tests include helper classes that extend Django's existing test framework and improve support for making API requests.
Therefore all tests for DRF calls are executed with Django's built in test framework.
An important principle of unit-testing is that each test should be independent of all others. If in your case the code in test_create_user must come before test_update_user, then you could combine both into one test:
def test_create_and_update_user(self):
....create and update user
Tests in Django are executed in a parallell manner to minimize the time it takes to run all tests.
As you said above if you want to share code between tests one has to set it up in the setUp method
def setUp(self):
pass
pytest allows the creation of fixtures that are automatically applied to every test in a test suite (via the autouse keyword argument). This is useful for implementing setup and teardown actions that affect every test case. More details can be found in the pytest documentation.
In theory, the same infrastructure would also be very useful for verifying post-conditions that are expected to exist after each test runs. For example, maybe a log file is created every time a test runs, and I want to make sure it exists when the test ends.
Don't get hung up on the details, but I hope you get the basic idea. The point is that it would be tedious and repetitive to add this code to each test function, especially when autouse fixtures already provide infrastructure for applying this action to every test. Furthermore, fixtures can be packaged into plugins, so my check could be used by other packages.
The problem is that it doesn't seem to be possible to cause a test failure from a fixture. Consider the following example:
#pytest.fixture(autouse=True)
def check_log_file():
# Yielding here runs the test itself
yield
# Now check whether the log file exists (as expected)
if not log_file_exists():
pytest.fail("Log file could not be found")
In the case where the log file does not exist, I don't get a test failure. Instead, I get a pytest error. If there are 10 tests in my test suite, and all of them pass, but 5 of them are missing a log file, I will get 10 passes and 5 errors. My goal is to get 5 passes and 5 failures.
So the first question is: is this possible? Am I just missing something? This answer suggests to me that it is probably not possible. If that's the case, the second question is: is there another way? If the answer to that question is also "no": why not? Is it a fundamental limitation of pytest infrastructure? If not, then are there any plans to support this kind of functionality?
In pytest, a yield-ing fixture has the first half of its definition executed during setup and the latter half executed during teardown. Further, setup and teardown aren't considered part of any individual test and thus don't contribute to its failure. This is why you see your exception reported as an additional error rather than a test failure.
On a philosophical note, as (cleverly) convenient as your attempted approach might be, I would argue that it violates the spirit of test setup and teardown and thus even if you could do it, you shouldn't. The setup and teardown stages exist to support the execution of the test—not to supplement its assertions of system behavior. If the behavior is important enough to assert, the assertions are important enough to reside in the body of one or more dedicated tests.
If you're simply trying to minimize the duplication of code, I'd recommend encapsulating the assertions in a helper method, e.g., assert_log_file_cleaned_up(), which can be called from the body of the appropriate tests. This will allow the test bodies to retain their descriptive power as specifications of system behavior.
AFAIK it isn't possible to tell pytest to treat errors in particular fixture as test failures.
I also have a case where I would like to use fixture to minimize test code duplication but in your case pytest-dependency may be a way to go.
Moreover, test dependencies aren't bad for non-unit tests and be careful with autouse because it makes tests harder to read and debug. Explicit fixtures in test function header give you at least some directions to find executed code.
I prefer using context managers for this purpose:
from contextlib import contextmanager
#contextmanager
def directory_that_must_be_clean_after_use():
directory = set()
yield directory
assert not directory
def test_foo():
with directory_that_must_be_clean_after_use() as directory:
directory.add("file")
If you absoulutely can't afford to add this one line for every test, it's easy enough to write this as a plugin.
Put this in your conftest.py:
import pytest
directory = set()
# register the marker so that pytest doesn't warn you about unknown markers
def pytest_configure(config):
config.addinivalue_line("markers",
"directory_must_be_clean_after_test: the name says it all")
# this is going to be run on every test
#pytest.hookimpl(hookwrapper=True)
def pytest_runtest_call(item):
directory.clear()
yield
if item.get_closest_marker("directory_must_be_clean_after_test"):
assert not directory
And add the according marker to your tests:
# test.py
import pytest
from conftest import directory
def test_foo():
directory.add("foo file")
#pytest.mark.directory_must_be_clean_after_test
def test_bar():
directory.add("bar file")
Running this will give you:
fail.py::test_foo PASSED
fail.py::test_bar FAILED
...
> assert not directory
E AssertionError: assert not {'bar file'}
conftest.py:13: AssertionError
You don't have to use markers, of course, but these allow controlling the scope of the plugin. You can have the markers per-class or per-module as well.
Python 2.7.1
Nose 1.1.2
I have read related questions on this but they do not help. I have Test cases that look like the below
For example in my_tests.py
def load_tests(loader, tests, pattern):
return unittest.TestSuite(MyTest() for scenario_name in list)
I have several such modules with load_tests method and I run them using unittest as follows
test_loader = unittest.defaultTestLoader.discover( '.', my_pattern_var);
test_runner = unittest.TextTestRunner();
result = test_runner.run(test_loader)
sys.exit(not result.wasSuccessful())
If I replace this with the equivalent nose code nose.main() it finds 0 tests.
Questions
How do I get nose to discover tests? WITHOUT actually losing the ability to just run my tests using python unittest. I would like to use NOSE as an addon to python unittest to get clover and coverage reports
How do I get it to run tests matching only a specific pattern?
sorry for getting back so late. We basically did the same thing you're trying to do here for a console script except we named all of our test modules integration_foo.py. Anyhow the solution is straightforward, just run nose programmatically.
import re
from nose.config import Config
TEST_REGEX = '(?:^|[\\b_\\./-])[Ll]oad'
# Change the test match pattern
nose_config = Config()
nose_config.testMatch = re.compile(TEST_REGEX)
# Specify the use of a Plugin Manager, load plugins
nose_config.plugins = BuiltinPluginManager()
nose_config.plugins.loadPlugins()
run(config=nose_config)
So this basic option changes the regex pattern nose is looking for from all methods labeled test to all methods labeled load. However this is not what you would need completely to run nose, it is also necessary to get some kind of parser object or pass a specific set of argv to nose.
If you want to pass a specific set of argv to be parsed by nose just do
run(config=nose_config, argv=["foo", "bar"])
Otherwise you can probably specify nose specific arguments at the command line and as long as you don't throw in anything funky nose shouldn't error.
Check out the nose source code at https://github.com/nose-devs/nose/tree/master/nose its where I got all the information I needed to write this
I would like to ask on UnitTesting in sikuli.
Is there any way to make settings from one place for all test ? Also there are any posibility to import one script (for example login to page) to other test ??
I have serious trouble with that. For example I make import file (config.py) with:
def setUp(self):
...some code...
def tearDown(self):
...some code...
It's really strange because I run test -> it's work,...and after that it didn't.
Do you have same experience? Or how you config your test ...
Thank you.
Because of many odds in the Sikuli IDE implementation of UnitTest (e.g. as in this case: import does not work), I recommend, to generally use the Python UnitTest module directly (just a few lines more coding, but total freedom to do what's needed)
see: https://answers.launchpad.net/sikuli/+faq/1804
I think that there are a few questions that are being asked here. I'll see if I can provide some insights for each.
Is there any way to make settings from one place for all test ?
Like with all unit testing the setUp and tearDown are run before and after each and every unit test.
import one script (for example login to page) to other test ??
Yes, you can reuse code. See the following documentation:
http://doc.sikuli.org/globals.html#importing-other-sikuli-scripts-reuse-code-and-images
I run test -> it's work,...and after that it didn't. Do you have same experience?
To paraphrase the documentation, save before you run and only hit the run button in the unit test panel.
http://sikuli.org/wiki/UnitTesting
Hope this helps.