How do i configure the django rest framework pagination url - django

when I get an object in django rest framework the urls always come absolute with localhost, but in production im going through a proxy on nginx, is there a way to set this url in the settings
Example
count: 11
next: "http://localhost:8000/api/accounts/?ordering=-date_registered&page=2"
previous: null
I need it to be
count: 11
next: "http:/example.com/api/accounts/?ordering=-date_registered&page=2"
previous: null
---------- edit --------------------------
please see my complete nginx config
server {
listen 80;
server_name 123.123.123.123;
root /home/admin/www/site-web/dist;
index index.html;
charset utf-8;
location /static/ {
alias /home/admin/www/site/static/;
}
location /media/ {
alias /home/admin/www/site/media/;
}
location /nginx_status/ {
# Turn on nginx stats
stub_status on;
# I do not need logs for stats
access_log off;
# Security: Only allow access from 192.168.1.100 IP #
# allow 192.168.1.100;
# Send rest of the world to /dev/null #
# deny all;
}
location / {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
try_files $uri $uri/ /index.html;
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';
#
# Om nom nom cookies
#
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
#
# Custom headers and headers various browsers *should* be OK with but aren't
#
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
#
# Tell client that this pre-flight info is valid for 20 days
#
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain charset=UTF-8';
add_header 'Content-Length' 0;
return 204;
}
if ($request_method = 'POST') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}
if ($request_method = 'GET') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}
}
location /docs/ {
proxy_pass http://127.0.0.1:8000/docs/;
break;
}
location /api/ {
underscores_in_headers on;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://127.0.0.1:8000/api/;
break;
}
location /admin/ {
proxy_pass http://127.0.0.1:8000/admin/;
break;
}
}
==== super edit====
Sorry guys, i had 'underscores_in_headers on;' i removed it and all is working
================

It sounds like your Host header is not being set properly, which would be an issue in your nginx configuration. The issue is that your Host header that is being sent includes the port number, so Django is including the port number when building out urls. This will cause future issues with CSRF, because CSRF checks do strict port checking when you are not debugging.
This is known to cause issues with SSL for similar reasons.
You can fix this by setting the Host header within Nginx to not include the proxied port.
proxy_set_header Host $http_host;
Note that I used the $http_host variable instead of $host or $host:$server_port. This will ensure that Django will still respect CSRF requests on non-standard ports, while still giving you the correct absolute urls.

Set USE_X_FORWARDED_HOST in your settings to True and make sure you pass it along using your web server(proxy) as well.
When django does build_absolute_uri() it calls get_host() - see below in django.http.request:
def get_host(self):
"""Returns the HTTP host using the environment or request headers."""
# We try three options, in order of decreasing preference.
if settings.USE_X_FORWARDED_HOST and (
'HTTP_X_FORWARDED_HOST' in self.META):
host = self.META['HTTP_X_FORWARDED_HOST']
...
See Real life usage of the X-Forwarded-Host header?

Related

CORS Issues with Nginx as Reverse Proxy for Django

I am running Nginx as my webserver to server the static frontend files and as a reverse proxy for my Django Server. The Problem is that i am having CORS Issues, when doing request from the frontend. My Design-Plan was to use the Apex Domain example.com for the frontend and all API Calls go to api.example.com.
I already did a tone of research and tried to catch OPTIONS request etc. but i still have CORS Errors. Also Django has the django-cors package installed and Axios is using .withDefaultCredentials = true. My current nginx config looks like this:
server {
listen 80;
listen [::]:80;
server_name $DOMAIN_HOSTNAME;
root /var/www/example.com/dist;
location /media {
alias /var/www/example.com/media/;
}
location / {
try_files $uri $uri/ /index.html =404;
}
}
upstream djangoserver {
server django:8000;
}
server {
listen 80;
listen [::]:80;
server_name $API_DOMAIN_HOSTNAME;
location / {
proxy_pass http://djangoserver;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header 'Access-Control-Allow-Origin' $http_origin;
proxy_set_header 'Access-Control-Allow-Credentials' 'true';
proxy_redirect off;
add_header 'Access-Control-Allow-Origin' 'https://example.com';
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
add_header 'Access-Control-Allow-Credentials' 'true';
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' 'https://example.com';
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
add_header 'Access-Control-Allow-Credentials' 'true';
return 204;
}
}
}
Also so no confusion is happening internally the nginx is serving on port 80, but its behind cloudflare which is establishing a https connection from client to cloudflares network. In the Django setting there is the domain configured:
# Allowed hosts this django app is allowed to run on.
ALLOWED_HOSTS = [
"localhost",
"127.0.0.1",
os.environ.get('DOMAIN_IP'),
os.environ.get('FRONTEND_DOMAIN_HOSTNAME')
]
# Cors policy (allowed origins) options...
CORS_ALLOWED_ORIGINS = [
"http://127.0.0.1",
"http://127.0.0.1" + ":" + os.environ.get("FRONTEND_PORT"),
"https://127.0.0.1",
"https://127.0.0.1" + ":" + os.environ.get("FRONTEND_PORT"),
"http://localhost",
"http://localhost" + ":" + os.environ.get("FRONTEND_PORT"),
"https://localhost",
"https://localhost" + ":" + os.environ.get("FRONTEND_PORT"),
"http://" + os.environ.get('FRONTEND_DOMAIN_HOSTNAME'),
"http://" + os.environ.get('FRONTEND_DOMAIN_HOSTNAME') + ":" + os.environ.get("FRONTEND_PORT"),
"https://" + os.environ.get('DOMAIN_IP'),
"https://" + os.environ.get('DOMAIN_IP') + ":" + os.environ.get("FRONTEND_PORT"),
]
CORS_ALLOW_CREDENTIALS = True
CSRF_TRUSTED_ORIGINS = CORS_ALLOWED_ORIGINS
Am i doing something wrong with my design or am i just missing something? I'm quite new to nginx configs.

How to return Django project's error messages when I use nginx in production mode

I have developed the Django project and deployed it to the Amazon's free tier EC2 services. Everything is fine except errors message that are not returning back. I am using the project in production mode.
Explanation for above image [Console Log]:
Successful request and response - it was made for existing url
Second request is made intentionaly to non existing url and did not receive any response.
I want to get at least 404 response, the problem I have is not having any response from server. When I run it on server I saw it is logging the results to the server.
Question:
How to return the response that Django is generating when something is wrong.
Extra Info: Those error messages and response are getting generated in djangorestframework's built in template.
Extra Details:
Let me know if I am missing anything.
Brain does really interesting things when it is tired. Thanks to #iklinac. He was right, better I would have used django-cors-headers correctly. It was already installed and working on heroku, when I moved to amazon aws I thought anything was related to NGINX.
Notes to take.
pip install django-cors-headers
Make sure it is in your installed apps.
INSTALLED_APPS = [
...
'corsheaders',
...
]
You will also need to add a middleware class to listen in on responses: # which I have missed
MIDDLEWARE = [ # Or MIDDLEWARE_CLASSES on Django < 1.10
...
'corsheaders.middleware.CorsMiddleware',
'django.middleware.common.CommonMiddleware',
...
]
A list of origins that are authorized to make cross-site HTTP requests
CORS_ORIGIN_WHITELIST = [
"https://example.com",
"https://sub.example.com",
"http://localhost:8080",
"http://127.0.0.1:9000"
]
Then there are few other things you can tweak and use as you want.
Eventually I have changed my nginx.conf to following
upstream hello_django {
server web:8000;
}
server {
listen 80;
location / {
proxy_pass http://hello_django;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /staticfiles/ {
alias /home/app/web/staticfiles/;
}
location /mediafiles/ {
alias /home/app/web/mediafiles/;
}
}
Happy coding.)
credits to testdriven.io and django-cors-headers
You are proxy passing requests and they don't get add_header correctly, you should proxy pass after you add headers
location / {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS, PATCH, PUT, DELETE';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Headers' 'Authorization,Accept,Origin,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range';
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS, PATCH, PUT, DELETE';
#
# Custom headers and headers various browsers *should* be OK with but aren't
#
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range, Authorization';
#
# Tell client that this pre-flight info is valid for 20 days
#
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain; charset=utf-8';
add_header 'Content-Length' 0;
return 204;
}
proxy_pass http://app;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
Other way around would be to add django-cors-headers into your application

nginx dynamic server_name with variable

Dear StackOverflow community I am working with a third party that does not support dynamic GET requests (eg example.com?variable=somethingDynamic) thus I resotred to using custom sub-domains, however I prefer not to make a sub domain for each and every dynamic request so I have been wondering:
how can I write server_name in a way to catch two or three dynamic variables?
here is my example server block:
server {
listen 80;
server_name someSecretUrl_$variable1_$variable2.example.com;
root /usr/share/campagins/campagin1;
client_max_body_size 10000m;
proxy_connect_timeout 30000;
location /funnel_webhooks/test {
return 200;
}
location / {
if ($request_method = 'OPTIONS') {
# Tell client that this pre-flight info is valid for 20 days
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain charset=UTF-8';
add_header 'Content-Length' 0;
add_header 'Access-Control-Allow-Origin' "$http_origin" always;
add_header 'Access-Control-Allow-Credentials' 'true' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With,Etag,Last-Modified,HTTP_IF_MODIFIED_SINCE,HTTP_IF_NONE_MATCH' always;
return 204;
}
add_header 'Access-Control-Allow-Origin' "$http_origin" always;
add_header 'Access-Control-Allow-Credentials' 'true' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With,Etag,Last-Modified,HTTP_IF_MODIFIED_SINCE,HTTP_IF_NONE_MATCH,ETag,Retry-After' always;
add_header 'Access-Control-Expose-Headers' 'ETag,Retry-After' always;
add_header 'Cache-Control' "must-revalidate, post-check=0, pre-check=0" always;
rewrite ^(.*)$ $1?preMadeDataParsers=$variable1&preMadeDataParsersOnResponse=$variable2&$args break;
proxy_buffering off;
proxy_pass http://localhost:3000; #proxy_set_header X-Real-IP $remote_addr;
proxy_set_header REMOTE_ADDR $remote_addr;
}
}
Looking at the code above you will notice I am trying to move $variable1 into a GET variable, and $variable2 into another get variable aswell, how can I achieve such a thing?
thanks!
I wouldn't do this in nginx, I'd do this in your application. (Especially if you expect to expand on this resource.)
I would configure the server to listen on the IP with no virtual hosts at all, so that it answers any request made to the IP. Just leave out the server_name directive:
server {
listen 1.2.3.4:80;
...
Then configure your DNS with a wildcard entry so that *.example.com points to that IP. Now you can hit any_string.example.com and it will resolve to your IP, get answered by the main server block, and passed to your app.
Then, inside your app, look at what hostname was requested. (In PHP for example, this is available via $_SERVER['HTTP_HOST'].) If your app determines that the requested hostname is invalid, just issue a 404 and exit. Otherwise, decode the hostname and process the request.
This way, you can add new variables and new features without editing the nginx config. You could even encode your variables in JSON then BASE64 encode them:
$vars = [
'var1' => 'one',
'var2' => 'two',
'var3' => 'three',
];
$url = base64_encode(json_encode($vars));
eyJ2YXIxIjoib25lIiwidmFyMiI6InR3byIsInZhcjMiOiJ0aHJlZSJ9.example.com
Now you can pass any number of variables, with any names, including indexed and associative arrays. (Though note there is a limit to the domain name length, and you'll have to do something about the + and / characters which I'm pretty sure aren't valid in domain names.)

Bypass nginx cache if user is authenticated Django

I'm implementing nginx as a reverse proxy for a Django project. I'm now trying implement nginx's cache config below:
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:5m max_size=1g inactive=60m;
proxy_temp_path /var/cache/nginx/tmp;
server {
...
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
# cache
proxy_cache my_cache;
proxy_cache_min_uses 1;
proxy_cache_valid 200 302 10m;
proxy_cache_valid 404 1m;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
proxy_cache_use_stale error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_cache_lock on;
proxy_hide_header "Set-Cookie";
add_header X-Proxy-Cache $upstream_cache_status;
}
}
All works great, however I'd like it so that authenticated users are able to bypass the cache, as otherwise when they update the site content, they won't see the changes, only the cached content until it expires.
What would be the best way to approach this?
Any help would be much appreciated.
Thanks
Add a middleware to set cookie (eg cachedisable) in response if user is autheticated else remove that cookie. So that you can put $cookie_ cachedisable in nginx conf
location / {
proxy_cache_bypass $cookie_cachedisable $arg_nocache;
...
}
Whenever you want bypass cache do requests like
http://www.example.com/?nocache=true #or if cookie `cachedisable` is present
For more info https://www.nginx.com/blog/nginx-caching-guide/?utm_source=nginx-caching-guide&utm_medium=blog#caching-guide-faq-hole-punch

Access-Control-Allow-Origin is not allowed by Access-Control-Allow-Headers

I have two separate server,one is nginx with node,and another one is django with django-rest-framework for build ding REST API,nginx is responsible for the REST API request,node takes care of client request, also i use polymer for the frontend .Below are a brief description:
machine one:
nginx:192.168.239.149:8888 (API listening address) forward to 192.168.239.147:8080
node:192.168.239.149:80 (client listening address)
machine two:
unicorn:192.168.239.147:8080(listening address)
The process is when a request comes in,node server(192.168.239.149:80) responses to return html,in html an AJAX request ask for API server(nginx:192.168.239.149:8888 forward to unicorn:192.168.239.147:8080),and then unicorn(192.168.239.147:8080) returns the result.
but there is a CORS problem,I read a lot article,and many people met the same questions,I tried many methods,but no help.still error.
what i get is :
that is:
XMLHttpRequest cannot load http://192.168.239.149:8888/article/. Request header field Access-Control-Allow-Origin is not allowed by Access-Control-Allow-Headers.
What i do is :
core-ajax
<core-ajax auto headers='{"Access-Control-Allow-Origin":"*","X-Requested-With": "XMLHttpRequest"}' url="http://192.168.239.149:8888/article/" handleAs="json" response="{{response}}"></core-ajax>
nginx:
http {
include mime.types;
default_type application/octet-stream;
access_log /tmp/nginx.access.log;
sendfile on;
upstream realservers{
#server 192.168.239.140:8080;
#server 192.168.239.138:8000;
server 192.168.239.147:8080;
}
server {
listen 8888 default;
server_name example.com;
client_max_body_size 4G;
keepalive_timeout 5;
location / {
add_header Access-Control-Allow-Origin *;
try_files $uri $uri/index.html $uri.html #proxy_to_app;
}
location #proxy_to_app{
add_header Access-Control-Allow-Origin *;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
#proxy_set_header X-Real-IP $remote_addr;
proxy_redirect off;
proxy_pass http://realservers;
}
}
}
node:
app.listen(80, function() {
console.log('server.js running');
});
unicorn:
return Response(serializer.data,headers={'Access-Control-Allow-Origin':'*',
'Access-Control-Allow-Methods':'GET',
'Access-Control-Allow-Headers':'Access-Control-Allow-Origin, x-requested-with, content-type',
})
Because,I have not much experience on CORS,and I want to understand it thoroughly,can anyone point out what i was doing wrong here,I will thank you very much!
Wow,so excited,I sovled this all by my self,what i do wrong here is that the request header i sent is not included in the nginx config add_header 'Access-Control-Allow-Headers'
complete nginx config:
http {
include mime.types;
default_type application/octet-stream;
access_log /tmp/nginx.access.log;
sendfile on;
upstream realservers{
#server 192.168.239.140:8080;
#server 192.168.239.138:8000;
server 192.168.239.147:8080;
}
server {
listen 8888 default;
server_name example.com;
client_max_body_size 4G;
keepalive_timeout 5;
location / {
add_header Access-Control-Allow-Origin *;
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Access-Control-Allow-Orgin,XMLHttpRequest,Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Mx-ReqToken,X-Requested-With';
try_files $uri $uri/index.html $uri.html #proxy_to_app;
}
location #proxy_to_app{
add_header Access-Control-Allow-Origin *;
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Access-Control-Allow-Orgin,XMLHttpRequest,Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Mx-ReqToken,X-Requested-With';
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
#proxy_set_header X-Real-IP $remote_addr;
proxy_redirect off;
proxy_pass http://realservers;
}
}
}
because my request is :
core-ajax auto headers='{"Access-Control-Allow-Origin":"*","X-Requested-With": "XMLHttpRequest"}' url="http://192.168.239.149:8888/article/" handleAs="json" response="{{response}}"></core-ajax>
i didnt include the Access-Control-Allow-Origin and XMLHttpRequest header into the nginx config Access-Control-Allow-Headers,so that is the problem.
I hope its useful to whom has the same problem!
You do not have to include CORS header into request manualy. The browser takes care of it, you just need to allow it on the api server