python caught exception stacktrace vs uncaught exception stacktrace - python-2.7

I have a try except block in my code where I am expecting a django.core.exceptions.ValidationError kind of excepption, catching it and logging it. I have tried various ways to log the exception, from simple logger.excepption to all the functions from traceback library. following is the stacktrace I get everytime:
Traceback (most recent call last):
File "/home/belong/work/code/hulk/hulk/commons/decorators.py", line 48, in func_wrapper
func(*args, **kwargs)
File "/home/belong/work/code/hulk/hulk/job/views.py", line 211, in create
raise DjangoCoreValidationError('test')
ValidationError: [u'test']
Whereas, if I dont catch the exception at all, python logs it as an uncaught exception and I get the following stacktrace:
2019-07-09 06:09:56,445 ERROR MainProcess [hulk.commons.exceptions.handlers: 35] [oid: 706a0f00ea9d476291ba118e61576622] [strail: hulk] [aid: ] Uncaught exception
Traceback (most recent call last):
File "/home/belong/work/code/hulk/venv/local/lib/python2.7/site-packages/rest_framework/views.py", line 480, in dispatch
response = handler(request, *args, **kwargs)
File "/home/belong/work/code/hulk/hulk/job/views.py", line 211, in create
raise DjangoCoreValidationError('test')
ValidationError: [u'test']
Why are these 2 different? Is there any way I can get the same stacktrace as the uncaught one while catching and logging the same exception?

Related

NEOS solver bonmin

did anyone know why this error happen? other solver such as cbc works fine but when sent to NEOS, I got the following error in command line
Traceback (most recent call last):
File "C:\repos\work_packager\Test\Scratch\Networkx_Sample\testss.py", line 409, in
process_data(df, df)
File "C:\repos\work_packager\Test\Scratch\Networkx_Sample\testss.py", line 191, in process_data
pyomo_tool(df,road_year, pit_year, road_estimated_cost, pit_estimated_cost, road_tot_budget, pit_tot_budget,
File "C:\repos\work_packager\Test\Scratch\Networkx_Sample\testss.py", line 291, in pyomo_tool
solver.solve(model)
File "C:\Python31\lib\site-packages\pyomo\opt\results\container.py", line 284, in getattr
raise AttributeError("Unknown attribute "+str(name)+"' for object with type "+str(type(self))) AttributeError: Unknown attribute solve' for object with type <class 'pyomo.opt.results.results_.SolverResults'>

Celery raise MemoryError

I'm using celery 4.3.0,And use it to generate a image which maybe 10M+,Then I got an error like below:
Pool callback raised exception: MemoryError('Process got: ')
Traceback (most recent call last):
File "*/lib/python3.7/site-packages/billiard/pool.py", line 1750, in safe_apply_callback
fun(*args, **kwargs)
File "*/lib/python3.7/site-packages/celery/worker/request.py", line 564, in on_success
return self.on_failure(retval, return_ok=True)
File "*/lib/python3.7/site-packages/celery/worker/request.py", line 351, in on_failure
raise MemoryError('Process got: %s' % (exc_info.exception,))
My server has 20G+ memory left when running this task.And I had test some small images which work well.Do I need set some config to prevent this?

Issue in describing Opsworks stack using boto

Below is the simple code snippet, which I'm trying to run, it gives exception.
I've configured AWS on my local and I'm able to describe the same stack in the AWS Opsworks UI. Can someone help here, with what could be the reason:
import boto3
client=boto3.client('opsworks')
response=client.describe_stack_summary(
StackId="6efce529-0b77-43dc-981b-ff20b906c4ae"
)
print(response)
Stacktrace for error:
Traceback (most recent call last):
File "botoTest.py", line 9, in <module>
StackId="6efce529-0b77-43dc-981b-ff20b906c4ae"
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/botocore/client.py", line 320, in _api_call
return self._make_api_call(operation_name, kwargs)
File
"/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/botocore/client.py", line 623, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.ResourceNotFoundException: An error occurred
(ResourceNotFoundException) when calling the DescribeStackSummary
operation: Unable to find stack with ID 6efce529-0b77-43dc-981b-
ff20b906c4ae

Restarting a Monitor instance

I have a cherrypy app that's got a Monitor instance like so:
mail_checker = Monitor(cherrypy.engine, self.mail_processor.poll_history_feed, frequency=10)
To put it simply it checks a gmail inbox for new emails and processes them. Sometimes poll_history_feed() will throw an exception, I'm guessing right now that its because of our unstable internet, and it ceases to run until I restart the whole app. (sample of the traceback below)
[01/Mar/2016:17:08:29] ENGINE Error in background task thread function <bound method MailProcessor.poll_history_feed of <mailservices.mailprocessor.MailProcessor object at 0x10a2f0250>>.
Traceback (most recent call last):
File "/Users/hashtaginteractive/Projects/.venvs/emaild/lib/python2.7/site-packages/cherrypy/process/plugins.py", line 500, in run
self.function(*self.args, **self.kwargs)
File "/Users/hashtaginteractive/Projects/emaild/emaild-source/mailservices/mailprocessor.py", line 12, in poll_history_feed
labelIds=["INBOX", "UNREAD"]
File "/Users/hashtaginteractive/Projects/.venvs/emaild/lib/python2.7/site-packages/oauth2client/util.py", line 142, in positional_wrapper
return wrapped(*args, **kwargs)
File "/Users/hashtaginteractive/Projects/.venvs/emaild/lib/python2.7/site-packages/googleapiclient/http.py", line 730, in execute
return self.postproc(resp, content)
File "/Users/hashtaginteractive/Projects/.venvs/emaild/lib/python2.7/site-packages/googleapiclient/model.py", line 207, in response
return self.deserialize(content)
File "/Users/hashtaginteractive/Projects/.venvs/emaild/lib/python2.7/site-packages/googleapiclient/model.py", line 262, in deserialize
content = content.decode('utf-8')
File "/Users/hashtaginteractive/Projects/.venvs/emaild/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x8b in position 23: invalid start byte
Is there any way to set this up so that it automatically restarts either the server or this particular Monitor instance whenever an exception happens?
You have to wrap the call to self.mail_processor.poll_history_feed in a try/except block and log the error for convenience.
def safe_poll_history_feed(self):
try:
self.mail_processor.poll_history_feed()
except Exception:
cherrypy.engine.log("Exception in mailprocessor monitor", traceback=True)
And then use the safe_poll_history_feed method

Scrapy cannot handle bad headers properly [ScrapyHTTPPageGetter,client] Unhandled Error

Environment:
Scrapy 0.16.2
Twisted-12.2.0
python 2.7
macosx-10.6
Okey here is my problem:
I try to run
scrapy shell http://aaa.17domn.com/bt9/file.php/MERH77V.html
Error:
[ScrapyHTTPPageGetter,client] Unhandled Error
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/internet/selectreactor.py", line 150, in _doReadOrWrite
why = getattr(selectable, method)()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/internet/tcp.py", line 202, in doRead
return self._dataReceived(data)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/internet/tcp.py", line 208, in _dataReceived
rval = self.protocol.dataReceived(data)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/protocols/basic.py", line 564, in dataReceived
why = self.lineReceived(line)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.2-py2.7.egg/scrapy/core/downloader/webclient.py", line 50, in lineReceived
return HTTPClient.lineReceived(self, line.rstrip())
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/web/http.py", line 450, in lineReceived
self.extractHeader(self._header)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/web/http.py", line 406, in extractHeader
key, val = header.split(':',1)
exceptions.ValueError: need more than 1 value to unpack
I found the solution from https://groups.google.com/forum/#!msg/scrapy-users/xFKo8ggzPxs/VXDl3CZ4V4cJ
They describe this is caused by twisted. Then I patched function extractHeader in /twisted/web/http.py from http://twistedmatrix.com/trac/ticket/2842. Its WORKS
BUT BUT, Hold on NOt yet!!!
I run another web
scrapy shell http://www1.wkdown.info/fs3/file.php/M994ATR.html
Error:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Twisted-12.2.0-py2.7-macosx-10.6-intel.egg/twisted/internet/defer.py", line 551, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Scrapy-0.16.2-py2.7.egg/scrapy/core/downloader/webclient.py", line 122, in _build_response
status = int(self.status)
ValueError: invalid literal for int() with base 10: 'html'
I think something happen on response headers. Scrapy cannot handle it well.
Any idea?
Thank you!