We have a Django 4.0.4 site running. Since upgrading from Python 3.10->3.11 and Psycopg2 from 2.8.6->2.9.3/5 and gunicorn 20.0.4->20.1.0 we've been getting random InterfaceError: cursor already closed errors on random parts of our codebase. Rarely the same line twice. Just kind of happens once every 5-10k runs. So it feels pretty rare, but does keep happening a few times every day. I've been assuming it's related to the ugprade, but it may be something else. I don't have a full grap on why the cursor would be disconnecting and where I should be looking to figure out the true issue.
Psycopg version: 2.9.5 & 2.9.3
Python version: 3.11
PostgreSQL version: 12.11
Gunicorn
The site had been running for 1-2 years without this error. Now it happens a few times every day after a recent upgrade.
We are having the same 'heisenbug' in our system and are attempting to solve it (unsuccessfully so far) ...
Related
I have this issue for a while now where starting django's internal server (runserver) becomes near unusable because there's so many errors like this reported in the console:
Exception ignored in: <generator object SQLCompiler.setup_query.<locals>.<genexpr> at 0x2F2DE360>
Traceback (most recent call last):
File "C:\Python36\lib\site-packages\django\db\models\sql\compiler.py", line 39, in <genexpr>
if all(self.query.alias_refcount[a] == 0 for a in self.query.alias_map):
SystemError: error return without exception set
Basically, these are generators not consumed and Python (at least 3.5 and up) reports this into console. And there are MANY!
This basically hogs the python process serving the app as well as PyCharm process trying to display all these errors in the console view. On a bad day, the app becomes like 10% of its normal speed because of this.
I am currently lessening this problem by implementing a filter on stderr which at least makes the console output usable again. It helps with CPU usage as well, but the problem is that those exceptions still happen and trigger PyCharm's hooks. As a result, CPU usage is still high, though not insane any more.
How can I get rid of this permanently? Any interpretation of "get rid" accepted in proposed solutions.
This seems to be an issue with PyCharm.
Try setting the environment variable PYDEVD_USE_FRAME_EVAL=NO, as suggested in this ticket on the PyCharm Issue Tracker.
I moved five different sites into MAMP Pro 4.0.6 recently, and now the program is running very sluggishly. I get periodic spinning beach balls anytime I touch the interface, and Activity Monitor shows MAMP %CPU shooting up to 75 and even 100 + shortly after. Once I get a site actually open, it seems to run fine in the browser. It's just the MAMP interace has ground to a halt.
I thought this might be a port conflict, but moving away from 80/3306 to program defaults has not solved the problem.
I also thought this might have something to do with using PHP 7.0.12, so I moved back to 5.6.27, but this has had not impact either.
Can anyone suggest other ways to troubleshoot this? I've looked in my logs and have found nothing overly suspicious except for reference to a two.crt files left over from a dev server SSL install. I removed these, but I'm not sure if this would have any impact.
Thanks.
-- Lee
I am using django 1.5 and gunicorn (sync workers)
Workers memory usage grow with time
i try to use Dozer to find the reason, but get:
AssertionError: Dozer middleware is not usable in a multi-process environment
How can i found the reason of leaking, any ideas?
We had this issue recently (with memory usage slowly climbing until the server(s) could not keep up).
We set gunicorn max_requests variable and it solved our problem. We set ours to 1000, although I'm not certain what the "ideal" setting would be.
http://docs.gunicorn.org/en/latest/configure.html#max-requests
Although I'm not sure if there may potentially be some reason why this became a problem to begin with.
I solved it by upgrading django to 1.5.1 (1.5 has some memory leak bugs)
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Django: IE doesn't load locahost or loads very SLOWLY
I just set up a clean dev environment on a computer running Windows 7 64-bit and installed all the latest officially released 64-bit versions of my tools including Django 1.3.1 and Python 2.7.2. I also got all the OS updates from MS and the computer vendor (HP), which I assume include fixes for IE9 bugs.
I am seeing exactly the same problem as reported 6 months ago in this StackOverflow question originally posed back on May 18 2011:
Django: IE doesn't load localhost or loads very SLOWLY
That is, Firefox works fine but IE9 hangs. The Django dev server , which seems to be running single threaded, seems to complete passing the response to the client and then sits waiting for the next request. IE9 however seems to think that it has not gotten the complete response (even though it has, including static pages referenced in the main page, to judge from the fact that it gets into the cache and can be gotten via am “X” disconnect followed by a refresh.)
My question is, is there a definitive resolution for this problem? In a response to the original question dated Aug 23 2011 Catalin Iacob says “I filled ticket 15178 and just confirmed that using the multithreaded development server fixes it. The fix is in revision 16427.” I am running the final Django 1.3.1 but I don’t know what its revision number is. Is the fix in question in 1.3.1? Do I have to enable multithreading with an option in settings.py or whatever?
EDIT: Thanks to user1043838 and nagisa and maybe others to come for posting concrete constructive solutions to the problem. I will try the fix that goes into settings.py because it’s non-invasive and easy to back out, but in general I want to work in as vanilla an environment as possible (Windows env that is). The problem bothers me but is far from a show-stopper at this point – firefox + firebug etc. is better for testing anyway – and if the cause is not outdated or misconfigured software then I can deal.
Disclaimer: I don't use Windows.
I will post multi threaded server fix as answer as I have a hunch that this is purely connection concurrency related problem.
In your project manage.py file add
import settings
# Multithreaded server...
if settings.DEBUG:
import SocketServer
import django.core.servers.basehttp
django.core.servers.basehttp.WSGIServer = \
type('WSGIServer',
(SocketServer.ThreadingMixIn,
django.core.servers.basehttp.WSGIServer,
object),
{})
just before if __name__ == "__main__": line. Then rerun your server by using same manage.py runserver and it should run as multi threaded server.
But be aware, it is even less stable than single threaded server and it tends to not serve files at all sometimes.
It seems to be working fairly well on Internet Explorer 10 Developer Preview on Windows 8 x64 Developer Preview with Django version 1.4 pre-alpha SVN-17202 running Satchmo version 0.9.2-pre hg-unknown.
Maybe upgrade?
I'm embedding iPython shell in a Django script (with development server, e.g. runserver at localhost) like this:
...
from IPython.Shell import IPShellEmbed
ipshell = IPShellEmbed()
ipshell()
...
which gives me interactive shell at the desired place. Now, if modify the source code, Django automatically reloads, probably without correctly quitting iPython shell, and "breaks" my terminal emulator (xterm, konsole) - text becomes invisible, etc. (same effect if iPython running inside Django is terminated with Ctrl+d).
Any suggestions as what could be causing this? (I'm probably using iPython in a wrong way, but who knows).
I cannot answer the question why it's going wrong, but I can tell you how to recover from it: quit the debugging server and give the reset command.
Another way to prevent this from happening is to use the --reload switch on the runserver command. This means that Django will not reload after a change, but it also doesn't break your debugger.
This issue is already fixed: http://code.djangoproject.com/ticket/15565
Thanks Django.