I have several TestCase classes in my django application. On some of them, I mock out a function which calls external resources by decorating the class with #mock.patch, which works great. One TestCase in my test suite, let's call it B(), depends on that external resource so I don't want it mocked out and I don't add the decorator. It looks something like this:
#mock.patch("myapp.external_resource_function", new=mock.MagicMock)
class A(TestCase):
# tests here
class B(TestBase):
# tests here which depend on external_resource_function
When I test B independently, things work as expected. However, when I run both tests together, A runs first but the function is still mocked out in B. How can I unmock that call? I've tried reloading the module, but it didn't help.
Patch has start and stop methods. Based on what I can see from the code you have provided, I would remove the decorator and use the setUp and tearDown methods found in the link in your classes.
class A(TestCase):
def setUp(self):
self.patcher1 = patch('myapp.external_resource_function', new=mock.MagicMock)
self.MockClass1 = self.patcher1.start()
def tearDown(self):
self.patcher1.stop()
def test_something(self):
...
>>> A('test_something').run()
Great answer. With regard to Ethereal's question, patch objects are pretty flexible in their use.
Here's one way to approach tests that require different patches. You could still use setUp and tearDown, but not to do the patch.start/stop bit.
You start() the patches in each test and you use a finally clause to make sure they get stopped().
Patches also support Context Manager stuff so that's another option, not shown here.
class A(TestCase):
patcher1 = patch('myapp.external_resource_function', new=mock.MagicMock)
patcher2 = patch('myapp.something_else', new=mock.MagicMock)
def test_something(self):
li_patcher = [self.patcher1]
for patcher in li_patcher:
patcher.start()
try:
pass
finally:
for patcher in li_patcher:
patcher.stop()
def test_something_else(self):
li_patcher = [self.patcher1, self.patcher2]
for patcher in li_patcher:
patcher.start()
try:
pass
finally:
for patcher in li_patcher:
patcher.stop()
Related
I have a need for a uniqueID within my Django code. I wrote a simple model like this
class UniqueIDGenerator(models.Model):
nextID = models.PositiveIntegerField(blank=False)
#classmethod
def getNextID(self):
if(self.objects.filter(id=1).exists()):
idValue = self.objects.get(id=1).nextID
idValue += 1
self.objects.filter(id=1).update(nextID=idValue)
return idValue
tempObj = self(nextID=1)
tempObj.save()
return tempObj.nextID
Then I wrote a unit test like this:
class ModelWorking(TestCase):
def setUp(self):
return None
def test_IDGenerator(self):
returnValue = UniqueIDGenerator.getNextID()
self.assertEqual(returnValue, 1)
returnValue = UniqueIDGenerator.getNextID()
self.assertEqual(returnValue, 2)
return None
When I run this test by itself, it runs fine. No issues.
When I run this test as a suite, which includes a bunch of other unit tests as well (which include calls to getNextID() as well), this test fails. The getNextID() always returns 1. Why would that be happening?
I figured it out.
Django runs each test in a transaction to provide isolation. Doc link.
Since my other tests make a call to getNextID(), the first row gets deleted after the first test that makes such a call is complete. Subsequent tests never find (id=1), due to which all subsequent calls return the value 1.
Even though I don't think I would face that situation in production, I went I ahead and changed my code to use .first() instead of (id=1). Like this
def getNextID(self):
firstRow = self.objects.first()
if(firstRow):
That way I believe it would better handle any future scenario when the database table might be emptied.
I have a flask application which uses a global object data_loader.
The main flask file (let's call it main.py) starts as follow:
app = Flask('my app')
...
data_loader = DataLoader(...)
Later on, this global data_loader object is called in the route methods of the webserver:
class MyClass(Resource):
def get(self):
data_loader.load_some_data()
# ... process data, etc
Using unittest, I want to be able to patch the load_some_data() method. I'm using the flask test_client:
from my_module.main import app
class MyTest(unittest.TestCase):
#classmethod
def setUpClass(cls) -> None:
cls.client = app.test_client('my test client')
How can I patch the data_loader method in subsequent tests in MyTest? I have tried this approach, but it does not work (although the data_loader seems to be replaced at some point):
#unittest.mock.patch('my_module.main.DataLoader')
def my_test(self, DataLoaderMock):
data_loader = DataLoaderMock.return_value
data_loader.my_method.return_value = 'new results (patched)'
with app.test_client() as client:
response = client.get(f'/some/http/get/request/to/MyTest/route',
query_string={...})
# ... some assertions to be tested ...
It seems the data_loader is never truly replaced in the Flask app.
Also, is this considered "good practice" to have a global variable in the Flask server, or is the app supposed to have it stored inside?
Thanks
About mocking, patch.object can be used to modify object attributes:
#unittest.mock.patch.object(data_loader, 'my_method')
def my_test(self, my_method_mock):
my_method_mock.return_value = 'new results (patched)'
with app.test_client() as client:
response = client.get(f'/some/http/get/request/to/MyTest/route',
query_string={...})
my_method_mock.assert_called() # ok!
My solution with interesting insights would be:
import unittest
from unittest.mock import patch
class MyTest(unittest.TestCase):
def test_get(self):
client = app.test_client('my test client')
patcher = patch('{package_here}.{module_here}.DataLoader.load_some_data', return_value={'status': 1})
patcher.start()
self.assertDictEqual(client.get('/').json, {'status': 1})
patcher.stop()
# or
with patch('{package_here}.{module_here}.DataLoader.load_some_data', return_value={'status': 1}):
self.assertDictEqual(client.get('/').json, {'status': 1})
About "good practice" and global variables. Yes, I have seen global variables in various projects. But I don't recommend using global variables. Because:
It can lead to recursive imports and dependency hell. I have worked with large Flask application with recursive imports. It is really pain. And you can't fix all problems for a short time.
Let's imagine you have a tests which mocking a global variables. I think refactoring is more difficult when you have a rather big service.
Separate imports and initialization is really simpler and more configurable. In this case all works in one direction import all dependencies -> load config -> initialization -> run. In other case you will have import -> new instance -> new instance -> import -> ....
Another reason for memory leaks.
Maybe global variables is not bad way for a stand alone packages, modules etc but not for a project. I also want to recommend using some additional tools. This will not only make it easier to write tests, but it will also save you headaches.
Loading spacy models slows down running my unit tests. Is there a way to mock spacy models or Doc objects to speed up unit tests?
Example of a current slow tests
import spacy
nlp = spacy.load("en_core_web_sm")
def test_entities():
text = u"Google is a company."
doc = nlp(text)
assert doc.ents[0].text == u"Google"
Based on the docs my approach is
Constructing the Vocab and Doc manually and setting the entities as tuples.
from spacy.vocab import Vocab
from spacy.tokens import Doc
def test()
alphanum_words = u"Google Facebook are companies".split(" ")
labels = [u"ORG"]
words = alphanum_words + [u"."]
spaces = len(words) * [True]
spaces[-1] = False
spaces[-2] = False
vocab = Vocab(strings=(alphanum_words + labels))
doc = Doc(vocab, words=words, spaces=spaces)
def get_hash(text):
return vocab.strings[text]
entity_tuples = tuple([(get_hash(labels[0]), 0, 1)])
doc.ents = entity_tuples
assert doc.ents[0].text == u"Google"
Is there a cleaner more Pythonic solution for mocking spacy objects for unit tests for entities?
This is a great question actually! I'd say your instinct is definitely right: If all you need is a Doc object in a given state and with given annotations, always create it manually wherever possible. And unless you're explicitly testing a statistical model, avoid loading it in your unit tests. It makes the tests slow, and it introduces too much unnecessary variance. This is also very much in line with the philosophy of unit testing: you want to be writing independent tests for one thing at a time (not one thing plus a bunch of third-party library code plus a statistical model).
Some general tips and ideas:
If possible, always construct a Doc manually. Avoid loading models or Language subclasses.
Unless your application or test specifically needs the doc.text, you do not have to set the spaces. In fact, I leave this out in about 80% of the tests I write, because it really only becomes relevant when you're putting the tokens back together.
If you need to create a lot of Doc objects in your test suite, you could consider using a utility function, similar to the get_doc helper we use in the spaCy test suite. (That function also shows you how the individual annotations are set manually, in case you need it.)
Use (session-scoped) fixtures for the shared objects, like the Vocab. Depending on what you're testing, you might want to explicitly use the English vocab. In the spaCy test suite, we do this by setting up an en_vocab fixture in the conftest.py.
Instead of setting the doc.ents to a list of tuples, you can also make it a list of Span objects. This looks a bit more straightforward, is easier to read, and in spaCy v2.1+, you can also pass a string as a label:
def test_entities(en_vocab):
doc = Doc(en_vocab, words=["Hello", "world"])
doc.ents = [Span(doc, 0, 1, label="ORG")]
assert doc.ents[0].text == "Hello"
If you do need to test a model (e.g. in the test suite that makes sure that your custom models load and run as expected) or a language class like English, put them in a session-scoped fixture. This means that they'll only be loaded once per session instead of once per test. Language classes are lazy-loaded and may also take some time to load, depending on the data they contain. So you only want to do this once.
# Note: You probably don't have to do any of this, unless you're testing your
# own custom models or language classes.
#pytest.fixture(scope="session")
def en_core_web_sm():
return spacy.load("en_core_web_sm")
#pytest.fixture(scope="session")
def en_lang_class():
lang_cls = spacy.util.get_lang_class("en")
return lang_cls()
def test(en_lang_class):
doc = en_lang_class("Hello world")
I am trying to mock elasticsearch data for hosted CI unit-testing purposes.
I have prepared some fixtures that I can successfully load with bulk(), but then, for unknown reason, I cannot match anything, even though the test_index seemingly contains the data (because I can get() items by their IDs).
The fixtures.json is a subset of ES documents that I fetched from real production index. With real world index, everything works as expected and all tests pass.
An artificial example of the strange behaviour follows:
class MyTestCase(TestCase):
es = Elasticsearch()
#classmethod
def setUpClass(cls):
super().setUpClass()
cls.es.indices.create('test_index', SOME_SCHEMA)
with open('fixtures.json') as fixtures:
bulk(cls.es, json.load(fixtures))
#classmethod
def tearDownClass(cls):
super().tearDownClass()
cls.es.indices.delete('test_index')
def test_something(self):
# check all documents are there:
with open('fixtures.json') as fixtures:
for f in json.load(fixtures):
print(self.es.get(index='test_index', id=f['_id']))
# yes they are!
# BUT:
match_all = {"query": {"match_all": {}}}
print('hits:', self.es.search(index='test_index', body=match_all)['hits']['hits'])
# prints `hits: []` like there was nothing in
print('count:', self.es.count(index='test_index', body=match_all)['count'])
# prints `count: 0`
While I can completely understand your pain (everything works except for the tests), the answer is actually quite simple: the tests, in contrast to your experiments, are too quick.
Elasticsearch is near real-time search engine, which means there
is up to 1s delay between indexing a document and it being
searchable.
There is also unpredictable delay (depending on actual
overhead) between creating an index and it being ready.
So the fix would be time.sleep() to give ES some space to create all the sorcery it needs to give you results. I would do this:
#classmethod
def setUpClass(cls):
super().setUpClass()
cls.es.indices.create('test_index', SOME_SCHEMA)
with open('fixtures.json') as fixtures:
bulk(cls.es, json.load(fixtures))
cls.wait_until_index_ready()
#classmethod
def wait_until_index_ready(cls, timeout=10):
for sec in range(timeout):
time.sleep(1)
if cls.es.cluster.health().get('status') in ('green', 'yellow'):
break
While #jsmesami's is very correct in his answer, there is this possibly cleaner way of doing this. If you notice, the issue is because ES has not re-indexed. There are actually functions exposed by the API for this very purpose.
Try something like,
cls.es.indices.flush(wait_if_ongoing=True)
cls.es.indices.refresh(index='*')
To be more specific, you can pass index='test_index' to both these functions. I think this is a cleaner and more specific way than using sleep(..).
I want to check that do_a calls do_b. I'm doing like this:
The code:
def do_a(...):
...
do_b(...)
...
The test:
def test_do_a(self):
...
with patch('...do_b', new_callable=do_nothing()) as mock_do_b:
do_a(...)
mock_do_b.assert_called_once_with(...)
And do_nothing:
def do_nothing():
pass
This is working fine but I had to use do_nothing() which I find hacky. Is there a way to make the same test without the extra useless function do_nothing()?
You could use patch as a decorator
#patch('...do_b')
def test_do_a(self, mock_do_b):
do_a(...)
mock_do_b.assert_called_once_with(...)