Django tests failing when run with all test cases - django

I have a problem with tests. When I run some tests I launch separately, they pass. When all together then fail.
#mock.patch(
'apps.abstract.validators.years_range_is_not_future', new=fake_years_range_is_not_future
)
def test_create_building_with_validation_of_foundation_period(self):
self.c.login(username=self.user.username, password='111')
response = self.c.post(
'/en/api/buildings/',
data={
'name': "New Building",
'foundation_period': {
'lower': MIN_INT_VALUE,
'upper': MAX_INT_VALUE
},
'stream': {
'uuid': s_uuid(self.stream)
}
}
)
self.assertEqual(response.status_code, status.HTTP_201_CREATED)
I read about this problem here
why would a django test fail only when the full test suite is run?
and tried to patch the validator in the serializer file as shown here
#mock.patch(
'apps.buildings.api.serializers.years_range_is_not_future', new=fake_years_range_is_not_future
)
def test_create_building_with_validation_of_foundation_period(self):
..............................................................
but then I get an incomprehensible for me exception
Error
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/mock.py", line 1049, in _dot_lookup
return getattr(thing, comp)
AttributeError: module 'apps.buildings.api' has no attribute 'serializers'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.5/unittest/mock.py", line 1149, in patched
arg = patching.__enter__()
File "/usr/lib/python3.5/unittest/mock.py", line 1205, in __enter__
self.target = self.getter()
File "/usr/lib/python3.5/unittest/mock.py", line 1375, in <lambda>
getter = lambda: _importer(target)
File "/usr/lib/python3.5/unittest/mock.py", line 1062, in _importer
thing = _dot_lookup(thing, comp, import_path)
File "/usr/lib/python3.5/unittest/mock.py", line 1051, in _dot_lookup
__import__(import_path)
File "/home/env/project/apps/buildings/api/serializers.py", line 12, in <module>
from apps.communities.api.serializers import CommunityBriefSerializer
File "/home/env/project/apps/communities/api/serializers.py", line 297, in <module>
class CommunityOfficialRequestBuildingSerializer(BaseCommunityOfficialRequestSerializer):
File "/home/rp/env/project/apps/communities/api/serializers.py", line 299, in CommunityOfficialRequestBuildingSerializer
from apps.buildings.api.serializers import BuildingBriefSerializer
ImportError: cannot import name 'BuildingBriefSerializer'
help please understand what I'm doing wrong
project structure (__init__.py files not listed)
project
|__apps
|__communities
| |_api
| |_serializers.py
|
|__buildings
| |_api
| | |_serializers.py
| |
| |_tests
| |_test.py
|
|_abstract
|_validators.py

Seeing this,
Traceback (most recent call last):
File "/home/rp/env/project/apps/communities/api/serializers.py", line 299, in CommunityOfficialRequestBuildingSerializer
from apps.buildings.api.serializers import BuildingBriefSerializer
suggests that your import statement is inside a class or a def or some other statement.
Maybe the import statement is executed after you mocked apps.buildings.api.serializers. If you move this import to the top of the file, then BuildingBriefSerializer will probably become available before apps.buildings.api.serializers is mocked, and your tests will pass.
This would also explain why the tests run, when you run them individually.

Related

Celery raise MemoryError

I'm using celery 4.3.0,And use it to generate a image which maybe 10M+,Then I got an error like below:
Pool callback raised exception: MemoryError('Process got: ')
Traceback (most recent call last):
File "*/lib/python3.7/site-packages/billiard/pool.py", line 1750, in safe_apply_callback
fun(*args, **kwargs)
File "*/lib/python3.7/site-packages/celery/worker/request.py", line 564, in on_success
return self.on_failure(retval, return_ok=True)
File "*/lib/python3.7/site-packages/celery/worker/request.py", line 351, in on_failure
raise MemoryError('Process got: %s' % (exc_info.exception,))
My server has 20G+ memory left when running this task.And I had test some small images which work well.Do I need set some config to prevent this?

NoneType has no attribute 'select' KerasDML SystemML

Im have some issue while running example Keras2DML code in this page. While running the code, i've got this error:
Traceback (most recent call last):
File "/home/fregy/kerasplayground/sysml/examplenn.py", line 12, in <module>
sysml_model = Keras2DML(spark, keras_model,input_shape=(3,224,224))
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/estimators.py", line 909, in __init__
convertKerasToCaffeNetwork(keras_model, self.name + ".proto")
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 201, in convertKerasToCaffeNetwork
jsonLayers = list(chain.from_iterable(imap(lambda layer: _parseKerasLayer(layer), kerasModel.layers)))
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 201, in <lambda>
jsonLayers = list(chain.from_iterable(imap(lambda layer: _parseKerasLayer(layer), kerasModel.layers)))
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 137, in _parseKerasLayer
ret = { 'layer': { 'name':layer.name, 'type':supportedLayers[layerType], 'bottom':_getBottomLayers(layer), 'top':layer.name, paramName:param[paramName] } }
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 112, in _getBottomLayers
return [ bottomLayer.name for bottomLayer in _getInboundLayers(layer) ]
File "/usr/local/lib/python2.7/dist-packages/systemml/mllearn/keras2caffe.py", line 70, in _getInboundLayers
for node in layer.inbound_nodes: # get inbound nodes to current layer
AttributeError: 'Conv2D' object has no attribute 'inbound_nodes'
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/lib/python2.7/SocketServer.py", line 230, in serve_forever
r, w, e = _eintr_retry(select.select, [self], [], [],
AttributeError: 'NoneType' object has no attribute 'select'
Im using Tensorflow-GPU 1.5 , and Keras 2.1.3 .
Thanks for trying out Keras2DML. The issue arises because the newer Keras versions renamed the attribute inbound_nodes to _inbound_nodes. This issue was fixed in yesterday's commit: https://github.com/apache/systemml/commit/9c3057a34c84d5bf1c698ad0a5c3c34d90412dbb.
Since you are using TensorFlow-GPU, you may want to check if TF grabs onto most of GPU memory when Keras model is compiled using nvidia-smi. If yes, here are two easy workarounds:
a. Hide GPUs from TF:
os.environ['CUDA_DEVICE_ORDER'] = 'PCI_BUS_ID'
os.environ['CUDA_VISIBLE_DEVICES'] = ''
import tensorflow as tf
b. Or minimize the overhead due to TensorFlow:
from keras.backend.tensorflow_backend import set_session
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
set_session(tf.Session(config=tf_config))

ImportError during CircleCI test

I tried to find a solution for this problem but I didn't find anything.
======================================================================
ERROR: Failure: ImportError (No module named mock)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 390, in loadTestsFromName
addr.filename, addr.module)
File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 39, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 86, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/home/ubuntu/client/tests/test_gs_interface.py", line 23, in <module>
import unittest, mock, kiss
ImportError: No module named mock
When I make a test with CIrcleCI using default parameters I always get the same ImportError.
I read at the documentation that I must set the configuration manually but I didn't find how to do.
Anybody could tell me how to import a module in the circle.xml configuration file?
Thank you.
Is mock included in your requirements.txt file? Also, do you have a Python version specified in your circle.yml file, as suggested here?

Debug django tests

I see that TestCase has a method Debug(), but I can't find any example on how to implement it. As far as I've tried, nothing works.
Can anyone provide some code as to how to use it?
debugunit.py
from unittest import TestCase
class MyTest(TestCase):
def test1(self):
print 'before'
self.assertEquals(2+2, 5)
print 'after'
run
python -i debugunit.py
To run a test interactively, create a TestCase instance, giving it the test name as a parameter. To run it, call the resulting object.
>>> print MyTest('test1')()
before
None
The "2+2!=5" exception is consumed by the unittest machinery. To get the set to run (with setUp and tearDown, etc), run the debug() method:
>>> MyTest('test1').debug()
before
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/unittest/case.py", line 400, in debug
getattr(self, self._testMethodName)()
File "debugunit.py", line 6, in test1
self.assertEquals(2+2, 5)
File "/usr/lib/python2.7/unittest/case.py", line 515, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib/python2.7/unittest/case.py", line 508, in _baseAssertEqual
raise self.failureException(msg)
AssertionError: 4 != 5

Scrapy cannot handle bad headers properly [ScrapyHTTPPageGetter,client] Unhandled Error

Environment:
Scrapy 0.16.2
Twisted-12.2.0
python 2.7
macosx-10.6
Okey here is my problem:
I try to run
scrapy shell http://aaa.17domn.com/bt9/file.php/MERH77V.html
Error:
[ScrapyHTTPPageGetter,client] Unhandled Error
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/internet/selectreactor.py", line 150, in _doReadOrWrite
why = getattr(selectable, method)()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/internet/tcp.py", line 202, in doRead
return self._dataReceived(data)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/internet/tcp.py", line 208, in _dataReceived
rval = self.protocol.dataReceived(data)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/protocols/basic.py", line 564, in dataReceived
why = self.lineReceived(line)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.2-py2.7.egg/scrapy/core/downloader/webclient.py", line 50, in lineReceived
return HTTPClient.lineReceived(self, line.rstrip())
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/web/http.py", line 450, in lineReceived
self.extractHeader(self._header)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/web/http.py", line 406, in extractHeader
key, val = header.split(':',1)
exceptions.ValueError: need more than 1 value to unpack
I found the solution from https://groups.google.com/forum/#!msg/scrapy-users/xFKo8ggzPxs/VXDl3CZ4V4cJ
They describe this is caused by twisted. Then I patched function extractHeader in /twisted/web/http.py from http://twistedmatrix.com/trac/ticket/2842. Its WORKS
BUT BUT, Hold on NOt yet!!!
I run another web
scrapy shell http://www1.wkdown.info/fs3/file.php/M994ATR.html
Error:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/internet/defer.py", line 551, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.2-py2.7.egg/scrapy/core/downloader/webclient.py", line 122, in _build_response
status = int(self.status)
ValueError: invalid literal for int() with base 10: 'html'
I think something happen on response headers. Scrapy cannot handle it well.
Any idea?
Thank you!