I'm under Apache + fastCGI with Django 1.8
My folder on the shared host looks like:
/home/username/
/home/username/MyProject_SRC
/home/username/MyProject_SRC/MyProject/manage.py
and:
/home/username/public_html/.htaccess
/home/username/public_html/index.fcgi
I've edited .htaccess and index.fcgi to look like this:
.htaccess :
AddHandler fcgid-script .fcgi
RewriteEngine On
RewriteCond %{REQUEST_URI} !=/index.fcgi
RewriteRule ^(.*)$ index.fcgi/$1 [QSA,L]
index.fcgi:
#!/home/username/djangoenv/bin/python3
import sys, os
sys.path.insert(0, "/home/username/MyProject_SRC")
os.chdir("/home/username/MyProject_SRC")
os.environ['DJANGO_SETTINGS_MODULE'] = "MyProject.settings"
from django.core.servers.fastcgi import runfastcgi
runfastcgi(method="threaded", daemonize="false")
Now, when I do on the server:
(djangoenv)username#hostname [~/public_html]# ./index.fcgi
Status: 200 OK
Vary: Cookie
X-Frame-Options: SAMEORIGIN
Content-Type: text/html; charset=utf-8
<!DOCTYPE html>
<html lang="en">
.... // Normal webpage
</html>
WSGIServer: missing FastCGI param REQUEST_METHOD required by WSGI!
WSGIServer: missing FastCGI param SERVER_NAME required by WSGI!
WSGIServer: missing FastCGI param SERVER_PORT required by WSGI!
WSGIServer: missing FastCGI param SERVER_PROTOCOL required by WSGI!
Going on my Home webpage returns a 500 internal error without more in the logs.
EDIT I've got the same result when doing:
(djangoenv)username#hostname [~/MyProject_SRC]# ./manage.py runfcgi
I had a similar problem, in my case I did not find a specific source, but I tried to add this in my index.fcgi and doing ./index.fcgi no longer asked me, if it keeps getting 500 check the version of flup you are using, which be 1.0.2 preferably.
os.environ['REQUEST_METHOD'] = "GET"
os.environ['SERVER_NAME'] = "yourdomain.com"
os.environ['SERVER_PORT'] = "80"
os.environ['SERVER_PROTOCOL'] = "HTTP/1.1"
Related
My django webapp runs properly using gunicorn, proxied through nginx; but I keep getting error emails: [Django] ERROR (EXTERNAL IP): Invalid HTTP_HOST header: '112.124.42.80'. You may need to add '112.124.42.80' to ALLOWED_HOSTS. Each time the IP changes, and I suspect this is a security issue.
I've only started to notice this issue a week ago, and the only significant change I've made was to set up using Cloudflare as CDN
Report at /
Invalid HTTP_HOST header: '112.124.42.80'. You may need to add '112.124.42.80' to ALLOWED_HOSTS.
Request Method: HEAD
Request URL: http://112.124.42.80/
Django Version: 2.0.6
Python Executable: /xxx/xxxx/xxxx/.virtualenv/bin/python3.6
Python Version: 3.6.8
META:
HTTP_ACCEPT = 'text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2'
HTTP_ACCEPT_ENCODING = 'gzip'
HTTP_CONNECTION = 'close'
HTTP_HOST = '112.124.42.80'
HTTP_PROXY_CONNECTION = 'keep-alive'
HTTP_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'
HTTP_X_FORWARDED_FOR = '60.191.52.254'
HTTP_X_FORWARDED_PROTO = 'http'
HTTP_X_REAL_IP = '60.191.52.254'
PATH_INFO = '/'
QUERY_STRING = ''
RAW_URI = '/'
REMOTE_ADDR = '127.0.0.1'
REMOTE_PORT = '49562'
REQUEST_METHOD = 'HEAD'
SCRIPT_NAME = ''
SERVER_NAME = '0.0.0.0'
SERVER_PORT = '3030'
SERVER_PROTOCOL = 'HTTP/1.0'
SERVER_SOFTWARE = 'gunicorn/19.9.0'
gunicorn.socket = <gevent._socket3.socket object, fd=13, family=2, type=2049, proto=0>
wsgi.errors = <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7f81a34dccc0>
wsgi.file_wrapper = ''
wsgi.input = <gunicorn.http.body.Body object at 0x7f81a34dc9e8>
wsgi.multiprocess = True
wsgi.multithread = True
wsgi.run_once = False
wsgi.url_scheme = 'http'
wsgi.version = '(1, 0)'
I assume your web app is usually accessed through a fully qualified domain name. It sounds like different IP addresses are pointing to your server, and nginx is routing these through to gunicorn, and so to Django. Django is doing the right thing in blocking them, but it would be better if nginx was not passing them through. You should change your nginx configuration so it only proxies the addresses you expect. You might want to look at the default_server settings.
I have googled but most uses are when person knows their URLs yet i want to redirect all possible URLs that give 404/503 errors with 301 to main domain.
Preferably htaccess since site is html.
So any request to page domain.com/XXXX that doesnt exist should redirect to main instead of giving errors ie:
Not Found
The requested URL /eafsdg was not found on this server.
To redirect all non existent and 503 requests to newdomain, you can use
ErrorDocument 404 http://newdomain.com/
ErrorDocument 503 http://newdomain.com/
In NGINX it would be like this :
NOTE: Since I use and I think a lot of people use NGINX as a proxy server, this won't redirect bad pages if the location is a proxy, that needs to be configured on the actual server, so I ended up using starkeen's answer on the end Apache server, but thought id share for NGINX.
error_page 404 /custom_404.html;
location = /custom_404.html {
root /usr/share/nginx/html;
internal;
}
or
error_page 404 /;
location = / {
root /usr/share/nginx/html;
internal;
}
https://www.digitalocean.com/community/tutorials/how-to-configure-nginx-to-use-custom-error-pages-on-ubuntu-14-04
Try this .htaccess code to redirect all 404 page to homepage:
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule . / [L,R=301]
My Django website is in HTTPS. When I am trying to POST data to the website from a script I get this error : "referer checking failed - no Referer". It seems to be a CSRF issue but I do not know how to solve it.
Example :
import requests
r = requests.post('https://mywebsite/mypage', data = {'key':'value'})
print r.text
gives me this output :
[...]
<p>Reason given for failure:</p>
<pre>
Referer checking failed - no Referer.
</pre>
<p>In general, this can occur when there is a genuine Cross Site Request Forgery, or when
<a
href="https://docs.djangoproject.com/en/1.8/ref/csrf/">Django's
CSRF mechanism</a> has not been used correctly. For POST forms, you need to
ensure:</p>
<ul>
<li>Your browser is accepting cookies.</li>
<li>The view function passes a <code>request</code> to the template's <code>render</code>
method.</li>
<li>In the template, there is a <code>{% csrf_token
%}</code> template tag inside each POST form that
targets an internal URL.</li>
<li>If you are not using <code>CsrfViewMiddleware</code>, then you must use
<code>csrf_protect</code> on any views that use the <code>csrf_token</code>
template tag, as well as those that accept the POST data.</li>
</ul>
[...]
Do I need to pass a referer to my headers before sending the POST data - which would not be convenient ? Or should I disable CSRF for this page ?
Thanks
AFAIK, This is the purpose of CSRF, to avoid posting data from unknown strange sources. You need csrf token to post this which django generates dynamically.
Upgrading Django might fix the missing Referer error.
As of Django 4.0 (release notes), the backend will first check the Origin header before falling back to the Referer header (source):
CsrfViewMiddleware verifies the Origin header, if provided by the browser, against the current host and the CSRF_TRUSTED_ORIGINS setting. This provides protection against cross-subdomain attacks.
In addition, for HTTPS requests, if the Origin header isn’t provided, CsrfViewMiddleware performs strict referer checking. This means that even if a subdomain can set or modify cookies on your domain, it can’t force a user to post to your application since that request won’t come from your own exact domain.
It's possible you have a reverse proxy running, for example an nginx proxy_pass to 127.0.0.1:8000?
In this case, Django expects the Cross-Site Forgery Protection tokens to match hostname 127.0.0.1, but they will be coming from a normal domain (for example example.com).
Expected Source
Actual Source
http://127.0.0.1
https://example.com
HTTP reverse proxy (example.com:80 -> localhost:3000) is a common way to use nginx with NodeJS applications, but it doesn't work well with Django
Client-Facing URL
Server Proxy URL
https://example.com
http://127.0.0.1:3000
It is better to run Django through a Unix socket rather than a port (example.com:80 -> <socket>). You can do this with Gunicorn:
Client-Facing URL
Server Proxy URL
https://example.com
unix:/run/example.com.sock
Here's how to do this with Django, Gunicorn, and nginx:
Let's say you've got a Django project root, which contains a system folder (the one where settings.py and wsgi.py are):
export DJANGO_PROJECT_PATH=/path/to/django/root
export DJANGO_SETTING_FOLDER=system
First, make sure you have Gunicorn installed and that you are using a virtual environment:
cd $DJANGO_PROJECT_PATH
source .venv/bin/activate # <- Use a virtual environment
pip3 install gunicorn # <- install Gunicorn in the venv
Run Gunicorn. This will start the Django project similar to running python3 manage.py runserver, except that you can listen for requests on a Unix socket:
$DJANGO_PROJECT_PATH/.venv/bin/gunicorn \
--workers=3 \
--access-logfile - \
--bind unix:/run/example.com.sock \ # <- Socket
--chdir=$DJANGO_PROJECT_PATH/ \
$DJANGO_SETTING_FOLDER.wsgi:application
Then create an HTTP proxy using nginx that passes HTTP requests from clients through the gunicon-created socket:
/etc/nginx/sites-enabled/example.com:
server {
listen 80;
listen [::]:80;
server_name example.com;
# serve static files directly through nginx
location /static/ {
autoindex off;
root /path/to/django/root;
}
# serve user-uploaded files directly through nginx
location /media/ {
autoindex off;
root /path/to/django/root;
}
# You can do fun stuff like aliasing files from other folders
location /robots.txt {
alias /path/to/django/root/static/robots.txt;
}
# here is the proxy magic
location / {
include proxy_params;
proxy_pass http://unix:/run/example.com.sock; # <- the socket!
}
}
Make sure to restart nginx:
sudo service restart nginx
After all this, your csrf tokens should match the domain name of your site and you'll be able to log in and submit forms.
Many of webmasters use pingdom.com as a monitoring ping service.
But the problem is that /httpd/access_log is full of
208.64.28.194 - - [06/Aug/2015:12:20:22 -0500] "GET / HTTP/1.1" 200 2917 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"
I set
CustomLog "logs/access_log" combined env=!dontlog
and tried to get rid of it using variations like
SetEnvIf Remote_Host "^pingdom\.com$" dontlog
SetEnvIFNoCase Remote_Host "pingdom.com$" dontlog
SetEnvIfNoCase Referer "www\.pingdom\.com" dontlog
SetEnvIFNoCase Host "^pingdom.com$" dontlog
but still no a success with any of them - so thanks for any else hint to try.
I'll put my comment here as answer so anyone will find this easier.
Since one can see from the log file, the host name is not Pingdom.com but a part of the user agent string.
Solutions to try:
First be sure you have enabled the setenvif-module. Write the command
sudo apache2ctl -M | grep setenv
It should return something like "setenvif_module (shared)"
Then you can try setting by remote address
SetEnvIf Remote_Addr "208\.64\.28\.194$" dontlog
The final working solution is this, dont log if the user agent string contains Pingdom string:
SetEnvIfNoCase User-Agent "^Pingdom" dontlog
Edit: enhanced some parts of the answer.
I have a Django project and I wanna delivery it using gunicorn (and apache proxing).
I can't use Nginx, so that's no possible.
I've set the Apache proxy and setup a runner script to gunicorn, but i am get this weird error
2012-08-27 14:03:12 [34355] [DEBUG] GET /
2012-08-27 14:03:12 [34355] [ERROR] Error handling request
Traceback (most recent call last):
File "/home/tileone/venv/lib/python2.6/site-packages/gunicorn/workers/sync.py", line 93, in handle_request
self.address, self.cfg)
File "/home/tileone/venv/lib/python2.6/site-packages/gunicorn/http/wsgi.py", line 146, in create
path_info = path_info.split(script_name, 1)[1]
IndexError: list index out of range
I am running this script
#!/bin/bash
LOGFILE=/var/log/gunicorn/one-project.log
VENV_DIR=/path/to/venv/
LOGDIR=$(dirname $LOGFILE)
NUM_WORKERS=5
# user/group to run as
USER=USER
GROUP=GROUP
BIND=127.0.0.1:9999
cd /path_to_project
echo 'Setup Enviroment'
#some libraries
echo 'Setup Venv'
source $VENV_DIR/bin/activate
export PYTHONPATH=$VENV_DIR/lib/python2.6/site-packages:$PYTHONPATH
#Setup Django Deploy
export DJANGO_DEPLOY_ENV=stage
echo 'Run Server'
test -d $LOGDIR || mkdir -p $LOGDIR
export SCRIPT_NAME='/home/tileone/one-project'
exec $VENV_DIR/bin/gunicorn_django -w $NUM_WORKERS --bind=$BIND\
--user=$USER --group=$GROUP --log-level=debug \
--log-file=$LOGFILE 2>>$LOGFILE
and my apache configuration is like this:
Alias /static/ /hpath_to_static/static/
Alias /media/ /path_to_static/media/
Alias /favicon.ico /path_to/favicon.ico
ProxyPreserveHost On
<Location />
SSLRequireSSL
ProxyPass http://127.0.0.1:9999/
ProxyPassReverse http://127.0.0.1:9999/
RequestHeader set SCRIPT_NAME /home/tileone/one-project/
RequestHeader set X-FORWARDED-PROTOCOL ssl
RequestHeader set X-FORWARDED-SSL on
</Location>
What am i doing wrong?
In case anyone has similar issues, I managed to fix this by removing the equivalent of:
RequestHeader set SCRIPT_NAME /home/tileone/one-project/
And instead adding to settings.py the equivalent of:
FORCE_SCRIPT_NAME = '/one-project'
Of course for this, the apache configuration should be more like:
ProxyPreserveHost On
<Location /one-project/>
SSLRequireSSL
ProxyPass http://127.0.0.1:9999/
ProxyPassReverse http://127.0.0.1:9999/
RequestHeader set X-FORWARDED-PROTOCOL ssl
RequestHeader set X-FORWARDED-SSL on
</Location>
The reason for the fix proposed in the accepted answer is that you need to decide between one of the following two approaches:
Let the HTTP server strip the location sub path BEFORE forwarding the request to the WSGI server (as explained above ... this is done when both the Location and the ProxyPass directive end with a forward slash. Nginx behaves the same way). In this case, you may not use the SCRIPT_NAME HTTP header/gunicorn-env-variable, because gunicorn would try to strip the value from the incoming URL (but that fails because the web server, Apache, already did that). In this case, you're forced to use Django's FORCE_SCRIPT_NAME setting.
Let the request URL passed to gunicorn unmodified (an example of an URL might be /one-project/admin), and use the SCRIPT_NAME HTTP header (or gunicorn-env-variable). Because then gunicorn will modify the request and strip the value of SCRIPT_NAME from the URL before Django handles building the response.
I would prefer option 2, because you only need to change one file, the web server configuration, and all changes are neatly together.