We are set up using Django + mod_wsgi + Apache.
Cccasionally, whilst we are doing site updates/changing the back-end Python code, site users run into Internal Server Error pages that appear to be served by Apache, not by Django (at least they are the horribly ugly ones, not the nice ones). They do seem to generate error reports though (via email) I think, they just don't seem to land on our 500 server error template.
I'd like to get to the bottom of why this is happening. Normally errors result in the Django-served (and nicely-styled) 500 error page, but just occasionally we see the ugly Internal Server Error page instead.
Does anyone have any idea why this is happening and what I can check/test/post here to stop this happening? Help really appreciated, thank you!
If Apache is showing you the 500 page, that suggests that Django threw an error while rendering the 500 page. For example, there could be a template tag on the 500.html template that Django was unable to render.
Have you looked in your Apache error logs? For Apache2 On Debian they are stored in /var/log/apache2/ by default. Your operating system may vary.
I know this behavior, had the same problem on Centos 5.3 with Python 2.5.
After I installed Python 2.7.2 and compiled the latest mod_wsgi (I think it was version 3.3) against it, those errors vanished. The problem is a bug in Python 2.5 as far as I remember in regards to memory allocation.
Be careful when using CentOS, as you can not update Python 2.4, youll have to install versions parallel. Updating Python will break CentOS system features like "yum".
Related
I have been running a Django server on Python 3.8 with Apache and mod_wsgi for some time now, and decided it was time to upgrade to Python 3.10. I installed Python 3.10 on my server, installed the Django and mod_wsgi packages, copied the 3.10 build of mod_wsgi.so to Apache's modules folder, and everything works great ... however, it seems with this build of mod_wsgi, changes to template files do not take effect unless I restart Apache.
As an example, I added a random HTML template file to my website, and started the server with some initial text in that template. Running on the Python 3.8, I am able to change the contents of that template (e.g. echo "More Text" >> test_template.html) and upon refreshing my browser the new text will show up. However doing the same test in 3.10, the new text will not show up. I have tried different browser sessions and hard reloading, it is not a caching issue on the client side, and looking at the response sizes in Apache's access log confirms the data being sent to the client changes in 3.8 but not in 3.10.
I have stood up a test server to isolate the problem, and have narrowed it down to specifically changing the mod_wsgi build (which of course changes the entire Python version used by Django). Still, that confirms it should not be a caching setting of Apache, or any mis-configuration of Django templates, and I have followed the steps here to confirm I am running mod_wsgi in Daemon mode (as I have been for years on this server, this is a long-standing server configured seemingly without issue for Python 3.8).
Lastly, running the Django development server (using base manage.py runserver command) reflects template changes on the fly without issue, and without a server reboot. So as far as I can tell this seems to be a mod_wsgi quirk.
The specific Apache | mod_wsgi | Python version combinations is as follows:
Apache/2.4.41 (Ubuntu) mod_wsgi/4.6.8 Python/3.8
Apache/2.4.41 (Ubuntu) mod_wsgi/4.9.4 Python/3.10
...as reported by Apache's error.log, confirming the modules are loading as expected.
Does anyone know if this is a known issue with Python 3.10 builds of mod_wsgi? Perhaps is there a new setting I'm forgetting? My understanding of Django templates is that they should always reflect changes immediately (without a server restart), however code changes require a restart (or touching of the wsgi.py script); I have never had to restart the server for template changes prior to this change. Any help is appreciated-
Edit: Just tried upgrading my Python 3.8 version of mod_wsgi to the same version (4.9.4), and it still works fine, so there is something about Python 3.10 vs 3.8, or another installed python package. I will keep testing...
I ended up making a post under the mod_wsgi github project, and traced the issue back to a change in Django's behavior in this commit
Full details of that post can be found here
Tl;dr; is there is a caching template loader, which used to only be enabled when DEBUG = False was set, but was updated to always be in effect. I'm not sure why my Python 3.8 build did not have this change, as I had upgraded both builds to the latest Django build available (4.6.1), but my original install was years ago so it's quite possible a fresh install would not have had this issue.
If you still want to disable cached templates as I did, you have to override the template loaders in your Django settings:
TEMPLATES = [
{
'OPTIONS': {
'loaders': ['django.template.loaders.filesystem.Loader', 'django.template.loaders.app_directories.Loader']
},
},
]
I'm a noob trying to learn Django for the first time, I created a project in a virtualenv on Windows 10. It worked well in the beginning where I was able to login to the admin section after running '''python manage.py runserver'''
But now when I run the same command I'm able to see the Django landing page but as soon as I try to hit http://localhost:8000/admin/ or http://127.0.0.1:8000/admin the server automatically disconnects and I get the "This site can’t be reached" error on Chrome.
I tried changing the port number by running python manage.py runserver 0.0.0.0:8001 but it didn't work. I tried to check if the port (8000) is currently in use by running the cmd (as an admin) netstat -a -b but couldn't find any issues.
The server just quits without any error message
Edit: Currently using Python 3.7.0 and django-3.0.1
There's a ticket about this issue: https://code.djangoproject.com/ticket/31067.
This seems to be a bug in Python 3.7.0, and appears to be fixed in Python 3.7.1. It's still unknown what the exact trigger is for this bug.
Since Django officially only supports the latest patch release of a Python series, this won't be fixed in Django. You can either upgrade your Python version to the latest patch release of 3.7, or downgrade Django to 2.2.
It's a Django 3.0 issue as I've seen. There are so many issues on GitHub, regarding this error.
You may try downgrading to Django 2.* versions for now. Version 2.1/2.2 works fine.
I recently migrated a python django application from a debian system to a redhat enterprise distribution. The application is hosted using httpd, mod_wsgi and running in a venv in an daemon process. On large requests I now get following error message in the log file:
"Truncated or oversized response headers received from daemon process"
I have never experienced anything like this and Google is not the key here as well.
I checked configuration of apache, but no config is related to response headers in there.
My httpd.conf configuration looks like this ( pretty standard):
WSGIPassAuthorization On
WSGIScriptAlias / /var/www/myapp/wsgi.py
WSGIDaemonProcess my.name python-path=/path/to/myapp/:/path/to/venv/lib/python2.7/site-packages display-name=%{GROUP}
WSGIProcessGroup my.name
Does any Guru have a hint in which direction I should look?
We recently ran into this issue, and after days of vigorous ugoogleizing and massive headache, we discovered that we were using psycopg2-binary as our database connector dependency (I know, newbs)! It states right in their documentation not to use the package in a production environment.
We did add all the other proposed answers such as adding 'WSGIApplicationGroup %{GLOBAL}' to our settings (which we kept), but all of them alone and together didn't solve the issue.
We also found that other C libraries like numpy, cause issues.
Hope this helps someone some day.
Django Webfaction 'Timeout when reading response headers from daemon process'
http://initd.org/psycopg/docs/install.html#prerequisites
The code used from Apache by mod_wsgi applies a limit on the size of a single response header returned from mod_wsgi daemon mode processes. This would result in a really cryptic error message from Apache which didn't at all point to the problem. From memory the previous error was:
Premature end of script headers
The size limit was also hard coded in Apache and couldn't be changed. This has caused problems for some Python web applications such as Keystone in OpenStack which generates very large authentication headers.
In mod_wsgi 4.1+, the reliance on the Apache code has been removed and the limit is now configurable. The error message is also more specific as you have seen.
The default maximum header size for what is returned from mod_wsgi daemon mode, that is header name and value, is about 8192 bytes. You can override this by using the header-buffer-size option to WSGIDaemonProcess.
Can you indicate what application and what header was so large that the limit was reached as would like to know what other Python web applications besides Keystone are generating such large headers if is a commonly used application.
A second possibility, deriving from the 'truncated' reference in that message, is that your mod_wsgi daemon process crashed. You don't say though that you saw a 'Segmentation fault' or similar message indicating that a crash occurred. Check for that and if there are other messages in the error log at the time, then indicate what they were and can look at that as the primary problem.
Turned out to be not the actual problem. The problem was lying deeper, as I changed Cairo to CairoCffi and the RSVG-Handler couldn't deal with the Context-Object coming from Cffi.
No my actuall problem is having an up-to-date python lib that allows me to convert svg into png. Using svg2png from CairoSVG isn't working for me. I get an
cairo returned CAIRO_STATUS_NO_MEMORY: out of memory
Error, which I'm sure of, that it does not tell the truth again and the problem lies somewhere else. But lets see.
I had an issue with this in CentOS 7 server when deploying Django using httpd with mod_wsgi 4.5.4. I had to revert back to using mod_wsgi 4.3.2 which solved my problem.
I have installed filebeat which changed my ssl version so that psycopy2 needs to be updated and the error was Truncated or oversized response headers received from daemon process
Do the following:-
Uninstall your psycopy2 package using pip. I am using pip3 because my python version is 3.6.8
sudo pip3 uninstall psycopg2
Reinstall psycopy2 using pip.
sudo pip3 install psycopg2
Before psycopg2-2.7.4 now psycopg2-2.8.3
I suddenly had this same problem after an update. The next update fixed the problem... I run arch, as of the date of this post, the WSGI version in repo works.
I had the same error message "Truncated or oversized response headers received from daemon process '...': /var/www/dev.audiocont.com/index.wsgi" in my Django project (without any other error message).
My error was that I changed the virtual environment and forgot to adapt the Apache settings "dev.conf" to the new venv path.
Change Deadlock time out in httpd.conf
I tried everything ,none of the answer work for me .Then i change the deadlock timeout and everything works fine now .Server Goes in to idle state for long processing just change the deadlock timeout.
I got into the same problem "Truncated or Oversized Response Headers".
I Resolved it by adding,
"WSGIDaemonProcess test user=apache group=apache processes=1 display-name=%{GROUP} header-buffer-size=65536"
in app.conf/httpd.conf depends upon your configuration file.
Based on your server's RAM size change the Processes and header-buffer-size. Default is header-buffer-size is 65536
For me SQLite turned out to be the problem.
I migrated a Django application with SQLite, on a new server, and this error started appearing, and the HTTP requests hanging.
I manage to solve the issue with:
WSGIApplicationGroup %{GLOBAL}'
But, I wanted to get to the bottom of the problem, as this application was not using any different python modules than other applications on the same server.
I realized that the only difference is the SQLite database, and after migrating from SQLite to Postgres, the problem went away.
In the past, I've had other grievances with using Django with SQLite, so I would advise against using SQLite for anything that needs to go into production.
As specified in the tutorial (http://www.jython.org/jythonbook/en/1.0/JythonDjango.html), I am using doj.backends.zxjdbc.postgresql as Django's DB engine. However, when I do
jython manage.py syncdb
I get an ImproperlyConfigured error, stating that doj.backends.zxjdbc.postgresql is not an available backend and that no module named postgresql exists. Strangely, if I enter the Jython shell, I can do
import doj.backends.zxjdbc.postgresql
without any error messages.
Ideas on what is going on here?
Solution: django-jython is out of date. 1.4 is not currently supported, so there is no solution to this issue other than to revert to a supported version of django.
Unfortunately it seems like django-jython is not seeing much/any active development anymore...
However, I have found that development versions available at the Google code repository below do work in Django 1.5 (and I assume they may work with 1.4 too, if you pick the right version):
https://code.google.com/p/django-jython/source/list
In particular, I have found that this dev version works with Django 1.5:
https://code.google.com/p/django-jython/source/detail?r=c4a0dd949a6d86a4baf8d1bed3b1926fa5318e87
I have a django application using the SessionMiddleware running in mod_wsgi daemon mode with Apache. My Apache configuration also contains a standard mod_deflate config and mod_cache/mod_disk_cache for performance.
My problem is that the Vary: Cookie set by django's SessionMiddleware isn't being recognized correctly by mod_cache, which is serving cached versions of pages although the request cookies are different. Removing mod_deflate fixes the problem. Setting Vary: Cookie in Apache with mod_headers (regardless of the django middleware) also fixes the problem (this is my temporary workaround).
I have noticed, while playing around with this, that there seem to be cases where headers set inside the mod_wsgi app aren't properly recognized by some Apache modules that come afterwards, does this mean anything to anyone?
I've got very standard wsgi (attached) and Apache configs, let me know if any other data can help. I'm running stock ubuntu natty libapache2-mod-wsgi 3.3-2ubuntu2, Apache 2.2.17, django 1.3
As first step, use:
http://code.google.com/p/modwsgi/wiki/DebuggingTechniques#Tracking_Request_and_Response
to verify what is actually coming out of the WSGI application and being passed to mod_wsgi.
I can't think of any reason why mod_wsgi would cause an issue and sounds more like an issue with Apache output filter ordering.
BTW, this is the sort of problem better handled on mod_wsgi mailing list. Debugging issues via StackOverflow sucks.