I have a django project running on my localhost and it is working very well, however when I uploaded it to real server, some problem started happening with the url. it happens every time HttpResponseRedirect or any redirect gets called
a page on my local host
http://127.0.0.1:8000/signin
while on the server it becomes
http://xyz.com,%20xyz.com/signin
in firebug i see
GET signin 301 MOVED PERMANENTLY
GET signin http://xyz.com,%20xyz.com/signin
I belive this happens because the urls.py has ^signin/$^ and APPEND_SLASH = True in settings.py because when I visit /signin/ it works!
404 page on my local host
Request URL: http://127.0.0.1:8000/test
on the server
Request URL: http://xyz.com,%20xyz.com/test
for some reason it is adding [comma][space] to url and redirects it.
home page is working without issues
The issue is tracked in the following ticket:
https://code.djangoproject.com/ticket/11877
It has to do with how Django handles proxy redirection. The following middleware will help you out.
class MultipleProxyMiddleware(object):
FORWARDED_FOR_FIELDS = [
'HTTP_X_FORWARDED_FOR',
'HTTP_X_FORWARDED_HOST',
'HTTP_X_FORWARDED_SERVER',
]
def process_request(self, request):
"""
Rewrites the proxy headers so that only the most
recent proxy is used.
"""
for field in self.FORWARDED_FOR_FIELDS:
if field in request.META:
if ',' in request.META[field]:
parts = request.META[field].split(',')
request.META[field] = parts[-1].strip()
If, for example, your Django site is sitting behind a proxy which includes proxy information in the X-Forwarded-For header, and then your web server also does proxying, the header will contain a list (comma separated) of the proxied addresses. By using this middleware, it will strip all but one of the proxied addresses in the headers.
It might not be an answer since I'm working with you on the same application, I fixed it.
It has something to do with nginx to apache redirection, we had proxy_set_header Host $host; and when I disabled it the redirection worked without errors.
Related
My original question was how to enable HTTPS for a Django login page, and the only response, recommended that I - make the entire site as HTTPS-only.
Given that I'm using Django 1.3 and nginx, what's the correct way to make a site HTTPS-only?
The one response mentioned a middleware solution, but had the caveat:
Django can't perform a SSL redirect while maintaining POST data.
Please structure your views so that redirects only occur during GETs.
A question on Server Fault about nginx rewriting to https, also mentioned problems with POSTs losing data, and I'm not familiar enough with nginx to determine how well the solution works.
And EFF's recommendation to go HTTPS-only, notes that:
The application must set the Secure attribute on the cookie when
setting it. This attribute instructs the browser to send the cookie
only over secure (HTTPS) transport, never insecure (HTTP).
Do apps like Django-auth have the ability to set cookies as Secure? Or do I have to write more middleware?
So, what is the best way to configure the combination of Django/nginx to implement HTTPS-only, in terms of:
security
preservation of POST data
cookies handled properly
interaction with other Django apps (such as Django-auth), works properly
any other issues I'm not aware of :)
Edit - another issue I just discovered, while testing multiple browsers. Say I have the URL https://mysite.com/search/, which has a search form/button. I click the button, process the form in Django as usual, and do a Django HttpResponseRedirect to http://mysite.com/search?results="foo". Nginx redirects that to https://mysite.com/search?results="foo", as desired.
However - Opera has a visible flash when the redirection happens. And it happens every search, even for the same search term (I guess https really doesn't cache :) Worse, when I test it in IE, I first get the message:
You are about to be redirected to a connection that is not secure - continue?
After clicking "yes", this is immediately followed by:
You are about to view pages over a secure connection - continue?
Although the second IE warning has an option to turn it off - the first warning does not, so every time someone does a search and gets redirected to a results page, they get at least one warning message.
For the 2nd part of John C's answer, and Django 1.4+...
Instead of extending HttpResponseRedirect, you can change the request.scheme to https.
Because Django is behind Nginx's reverse proxy, it doesn't know the original request was secure.
In your Django settings, set the SECURE_PROXY_SSL_HEADER setting:
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
Then, you need Nginx to set the custom header in the reverse proxy. In the Nginx site settings:
location / {
# ...
proxy_set_header X-Forwarded-Proto $scheme;
}
This way request.scheme == 'https' and request.is_secure() returns True.
request.build_absolute_uri() returns https://... and so on...
Here is the solution I've worked out so far. There are two parts, configuring nginx, and writing code for Django. The nginx part handles external requests, redirecting http pages to https, and the Django code handles internal URL generation that has an http prefix. (At least, those resulting from a HttpResponseRedirect()). Combined, it seems to work well - as far as I can tell, the client browser never sees an http page that the users didn't type in themselves.
Part one, nginx configuration
# nginx.conf
# Redirects any requests on port 80 (http) to https:
server {
listen 80;
server_name www.mysite.com mysite.com;
rewrite ^ https://mysite.com$request_uri? permanent;
# rewrite ^ https://mysite.com$uri permanent; # also works
}
# django pass-thru via uWSGI, only from https requests:
server {
listen 443;
ssl on;
ssl_certificate /etc/ssl/certs/mysite.com.chain.crt;
ssl_certificate_key /etc/ssl/private/mysite.com.key;
server_name mysite.com;
location / {
uwsgi_pass 127.0.0.1:8088;
include uwsgi_params;
}
}
Part two A, various secure cookie settings, from settings.py
SERVER_TYPE = "DEV"
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True # currently only in Dev branch of Django.
SESSION_EXPIRE_AT_BROWSER_CLOSE = True
Part two B, Django code
# mysite.utilities.decorators.py
import settings
def HTTPS_Response(request, URL):
if settings.SERVER_TYPE == "DEV":
new_URL = URL
else:
absolute_URL = request.build_absolute_uri(URL)
new_URL = "https%s" % absolute_URL[4:]
return HttpResponseRedirect(new_URL)
# views.py
def show_items(request):
if request.method == 'POST':
newURL = handle_post(request)
return HTTPS_Response(request, newURL) # replaces HttpResponseRedirect()
else: # request.method == 'GET'
theForm = handle_get(request)
csrfContext = RequestContext(request, {'theForm': theForm,})
return render_to_response('item-search.html', csrfContext)
def handle_post(request):
URL = reverse('item-found') # name of view in urls.py
item = request.REQUEST.get('item')
full_URL = '%s?item=%s' % (URL, item)
return full_URL
Note that it is possible to re-write HTTPS_Response() as a decorator. The advantage would be - not having to go through all your code and replace HttpResponseRedirect(). The disadvantage - you'd have to put the decorator in front of HttpResponseRedirect(), which is in Django at django.http.__init__.py. I didn't want to modify Django's code, but that's up to you - it's certainly one option.
if you stick your entire site behind https, you don't need to worry about it on the django end. (assuming you don't need to protect your data between nginx and django, only between users and your server)
I am in Django HTTP error codes hell. Would be great if an expert can help me out of my misconfiguration.
My Django project runs with nginx as a reverse proxy coupled to a gunicorn application server.
Requirement:
I want a custom Page not found template to render (i.e. 404) when a url pattern is entered that doesn't exist in my urls.py. Sounds simple enough, and is well documented.
I have already gone ahead and implemented this.
The Problem:
Assume example.com is my live project.
1) If I try to access https://example.com/asdfasdf (i.e. unmatched, random gibberish) on my production server, it displays the 500 template instead of 404.
2) Next, if I try to curl the said url pattern via curl -I https://example.com/asdfasdf/, I see 200 OK instead of 404 or 500. Wth?
3) Moreover, if I try the same behavior with Debug = True on localhost, 404 is returned correctly (both template and HTTP error code are in consonance).
These 3 behaviors are quite perplexing.
My configuration:
I created error_views.py and inserted it in the folder where I keep my regular views.py. This error file contains:
from django.shortcuts import render
def server_error(request):
return render(request, '500.html')
def not_found(request):
return render(request, '404.html')
def permission_denied(request):
return render(request, '404.html')
def bad_request(request):
return render(request, '404.html')
In my urls.py (kept in the same folder as settings.py), I added the following after all url patterns:
handler404 = 'my_app.error_views.not_found'
handler500 = 'my_app.error_views.server_error'
handler403 = 'my_app.error_views.permission_denied'
handler400 = 'my_app.error_views.bad_request'
I created 404.html and 500.html, and inserted them in the default /templates/ directory.
In settings.py, I have ALLOWED_HOSTS = ['*']
Lastly, my nginx conf dealing with this is as follows (placed within the server block in the virtual host file):
# Error pages
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/ubuntu/this_proj/project_dir/templates/;
}
location = /too_bad.svg {
root /home/ubuntu/this_proj/project_dir/static/img/;
}
All of this is fairly regular stuff and I'm missing what I've misconfigured here. Can an expert guide me out of this mess?
Thanks in advance, and please ask for more information in case warranted.
Note: I tried solutions provided in similar questions on SO here and here. Needless to say, those misconfigurations were very different, displaying none of the symptoms I'm seeing.
If you use a custom handler, you have to explicitly set the proper http status for the response object. If you don't set the status, the default is 200 OK.
def not_found(request):
return render(request, '404.html', status=404)
I am developing a ExtJS application that uses a Django-rest-framework service. I am using CORS headers to allow fetching the data from the service (https://github.com/OttoYiu/django-cors-headers).
What happens is that at a point in time I want to change the URL from the store. And when I do that I get the following error:
XMLHttpRequest cannot load http://10.98.0.241:8000/reacsearch/as?_dc=1418831884352&page=1&start=0&limit=25. The request was redirected to 'http://10.98.0.241:8000/reacsearch/as/?_dc=1418831884352&page=1&start=0&limit=25', which is disallowed for cross-origin requests that require preflight.
In the settings.oy I define the following properties for the CORS
CORS_ALLOW_METHODS = (
'GET',
'OPTIONS'
)
CORS_ORIGIN_ALLOW_ALL = True
This works fine when I use URLs to list all the elements in my database, however when I change the store for another URL I get the error above. Also the link works fine in the browser.
The store url change is made this way:
var store = Ext.getStore(storeName);
store.getProxy().setUrl(newURL);
store.load();
The difference between the views, is that the two that work on the application are viewsets, while the other is just a generic list
class Example1viewset(viewsets.ModelViewSet):
"""
API endpoing that allows metabolites to be viewed.
"""
queryset = examples1.objects.all()
serializer_class = Example1Serializer
class Example1SearchList(generics.ListAPIView):
serializer_class = Example1Serializer
def get_queryset(self):
queryset = Example.objects.all()
if 'attr' in self.kwargs:
queryset = queryset.filter(Q(attribute1__contains=self.kwargs['attr']) | Q(attribute2__contains=self.kwargs['abbr']))
return queryset
Like I mentioned both examples work fine in the browser (even accessing through other computers in the network), however in the application when changing the URL of the store I get the CORS error. Does anyone has any idea why this is happening?
Thank you.
Edit:
Just for clarification, the problem is not in changing the url of the store. As I tried to set those urls as defaults, but they are not working when accessing from the application.
My urls.py file:
router = routers.DefaultRouter()
router.register(r'example', views.Example1ViewSet)
# Wire up our API using automatic URL routing.
# Additionally, we include login URLs for the browsable API.
urlpatterns = [
url(r'^', include(router.urls)),
url(r'^reacsearch/(?P<attr>.+)/$', Example1SearchList.as_view()),
url(r'^api-auth/', include('rest_framework.urls', namespace='rest_framework'))
Can it be that the problem is related with the fact that I am not adding the search list to the router?
Edit2
Problem solved since I was trying to fetch data from a different domain. I changed the type of store to jsonp in Extjs, and I also allowed my rest service to render data as jsonp.
Just a reminder if anyone comes accross this same problem, it is necessary to add ?format=jsonp to the store url:
http://my/url/?format=jsonp
Since it looks like an alternate solution was found, I'll explain what the issue appeared to be as well as why the alternative works.
XMLHttpRequest cannot load first url. The request was redirected to 'second url', which is disallowed for cross-origin requests that require preflight.
The issue here is that you are telling Django to enforce the trailing slash, which makes it automatically redirect urls without a trailing slash to urls with a trailing slash, assuming that one exists. This is why, as stated in the error, the request was redirected to the second url, which you can tell has the missing trailing slash. This is controlled by the APPEND_SLASH Django setting which is True by default.
The problem is that when CORS is doing a preflight request, which is what allows it to determine if the request can be made, there must be a valid response at the requested URL. Because you are redirecting the request, the preflight request fails and you're stuck without your information.
You can fix this by adding the trailing slash in your code. There appear to be a few solutions for doing this with ext, but I personally can't recommend a specific one. You can also manually set the url to use the trailing slash, which sounds like what you were doing previously.
Or you can use JSONP...
You've found the alternative solution, which is to use JSONP to make the request instead of relying on CORS. This gets around the preflight issue and works in all major browsers, but there are some drawbacks to consider. You can find more information on CORS vs JSONP by looking around.
You're going to need CORS if you want to push any changes to your API, as JSONP only supports GET requests. There are other advantages, such as the ability to abort requests, that also comes with CORS.
Using the Django-auth application (Django version 1.3), I want to have my login page go to https://mysite.com/login/. Currently, I'm using:
# urls.py
from django.contrib.auth.views import login
urlpatterns = patterns('', url(r'^login/$', login, name='login-view'),)
# navbar.html
<li id="nav-login"><a href="{% url login-view %}" ><b>Login</b></a></li>
which works nicely, but goes to http://mysite.com/login/.
Is there some way to tell Django-auth what prefix (https) to use, when it reverses the view name? I've read the entire manual page, and haven't found anything that covers it. Or maybe some way to tell the url tag to go to https?
Or is the only option to specify the entire URL manually? I hope not :) And given how powerful Django has been so far, I can't believe it wouldn't have that ability - I must be overlooking it. :)
Set OS environmental variable HTTPS to on
You need to enable the OS environmental variable HTTPS to 'on' so django will prepend https to fully generated links (e.g., like with HttpRedirectRequests). If you are using mod_wsgi, you can add the line:
os.environ['HTTPS'] = "on"
to your wsgi script. You can see the need for this by reading django/http/__init__.py:
def build_absolute_uri(self, location=None):
"""
Builds an absolute URI from the location and the variables available in
this request. If no location is specified, the absolute URI is built on
``request.get_full_path()``.
"""
if not location:
location = self.get_full_path()
if not absolute_http_url_re.match(location):
current_uri = '%s://%s%s' % (self.is_secure() and 'https' or 'http',
self.get_host(), self.path)
location = urljoin(current_uri, location)
return iri_to_uri(location)
def is_secure(self):
return os.environ.get("HTTPS") == "on"
Secure your cookies
In settings.py put the lines
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
and cookies will only be sent via HTTPS connections. Additionally, you probably also want SESSION_EXPIRE_AT_BROWSER_CLOSE=True. Note if you are using older versions of django (less than 1.4), there isn't a setting for secure CSRF cookies. As a quick fix, you can just have CSRF cookie be secure when the session cookie is secure (SESSION_COOKIE_SECURE=True), by editing django/middleware/csrf.py:
class CsrfViewMiddleware(object):
...
def process_response(self, request, response):
...
response.set_cookie(settings.CSRF_COOKIE_NAME,
request.META["CSRF_COOKIE"], max_age = 60 * 60 * 24 * 7 * 52,
domain=settings.CSRF_COOKIE_DOMAIN,
secure=settings.SESSION_COOKIE_SECURE or None)
Direct HTTP requests to HTTPS in the webserver
Next you want a rewrite rule that redirects http requests to https, e.g., in nginx
server {
listen 80;
rewrite ^(.*) https://$host$1 permanent;
}
Django's reverse function and url template tags only return relative links; so if you are on an https page your links will keep you on the https site.
As seen in other StackOverflow questions, you could implement middleware that would automatically redirect the login page to a secure version.
If you are really serious about security, you should probably migrate the entire website to SSL. From the EFF's How to Deploy HTTPS Correctly:
You must serve the entire application domain over HTTPS. Redirect HTTP requests with HTTP 301 or 302 responses to the equivalent HTTPS resource.
Some site operators provide only the login page over HTTPS, on the theory that only the user’s password is sensitive. These sites’ users are vulnerable to passive and active attack.
When developing with Django without a web server (serving directly from Django) I have a problem with external urls that lack the domain part and have parameters.
Let's say I'm using a javascript library that does an ajax call to "/prefix/foo/bar?q=1" (the url is not something I can change). It is not a problem for the production server but only a problem when not using a web server. I can redirect by adding the following pattern to my urlpatters:
(r'^prefix/(?P<path>.*)$', 'django.views.generic.simple.redirect_to', {'url': 'htttp://example.com/prefix/%(path)s'}),
but of course %(path)s will only contain "foo/bar" not "foo/bar?q=1".
Is there a way to handle this problem with Django?
You'll have to write your own redirect:
def redirect_get(request, url, **kwargs):
if request.META['QUERY_STRING']:
url += '?%s' % request.META['QUERY_STRING']
return redirect_to(request, url, **kwargs)