logout not working, caching on nginx, how to allow logout? - django

I have everything cached, if I logged into my account, you will not be able to log out any more) how do you get out when you quit? i need to know how to delete cookies and session! when i'll logout!
P.S. if i'll disable caching on nginx level, everything works fine,
problem in nginx
nginx conf
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
proxy_connect_timeout 5;
proxy_send_timeout 10;
proxy_read_timeout 10;
proxy_buffering on;
proxy_buffer_size 16k;
proxy_buffers 24 16k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_temp_path /tmp/nginx/proxy_temp;
add_header X-Cache-Status $upstream_cache_status;
proxy_cache_path /tmp/nginx/cache levels=1:2 keys_zone=first_zone:100m;
proxy_cache one;
proxy_cache_valid any 30d;
proxy_cache_key $scheme$proxy_host$request_uri$cookie_US;
server conf
upstream some site {
server unix:/webapps/some/run/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name server name;
expires 7d;
client_max_body_size 4G;
access_log /webapps/some/logs/nginx-access.log;
error_log /webapps/some/logs/nginx-error.log;
error_log /webapps/some/logs/nginx-crit-error.log crit;
error_log /webapps/some/logs/nginx-debug.log debug;
location /static/ {
alias /webapps/some/static/;
}
location /media/ {
alias /webapps/some/media/;
}
location ~* ^(?!/media).*.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
root root_path;
expires 7d;
add_header Pragma public;
add_header Cache-Control "public, must-revalidate, proxy-revalidate";
access_log off;
}
location ~* ^(?!/static).*.(?:css|js|html)$ {
root root_path;
expires 7d;
add_header Pragma public;
add_header Cache-Control "public, must-revalidate, proxy-revalidate";
access_log off;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_cache one;
proxy_cache_min_uses 1;
proxy_cache_use_stale error timeout;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://some;
break;
}
}
error_page 404 /404.html;
location = /error_404.html {
root /webapps/some/src/templates;
}
error_page 500 502 503 504 /500.html;
location = /error_500.html {
root /webapps/some/src/templates;
}
}

Instead of logging out with a GET request, change your logout view to accept a form POST.
POST requests should not be cached.
This has the added security benefit of preventing users from being logged out with iframes or malicious links (ie: https://example.com/logout/, assuming you have not disabled django's CSRF protection).
Note: there is a ticket on django's bug tracker related to this issue.

You have the following question:
i need to know how to delete cookies and session! when i'll logout!
With the following code:
proxy_cache_key $scheme$proxy_host$request_uri$cookie_US;
We first have to know what's in $cookie_US?
If it's simply the name of the login, then you need to realise that anyone who knows the name of the login, and sets their own cookie as such, and knows the complete URL of a hidden resource that such user (and only such user) has access to, and which has been accessed recently (thus freshly cached), can now gain ‘unauthorised’ access to the given resource, since it'll be served straight from cache, and likely without any sort of re-validation.
Basically, for caching user-specific content, you have to make sure that you set http://nginx.org/r/proxy_cache_key to represent an actually secret non-guessable value, which could then be cleared on the user's end to logout. Subsequently, if the user does logout, then your cache is still subject to the replay attacks by anyone who somehow still posesses such secret value, but it'd usually be minimised by a short expiration time of the cache, plus, the secret is still supposed to stay a secret even after logout.
And clearing the session is as easy as simply re-setting the variable to something that wouldn't be giving the access to the user, e.g., you can even implement the whole logout thing entirely within nginx, too:
proxy_cache_key $scheme$proxy_host$request_uri$cookie_US;
location /logout {
add_header Set-Cookie "US=empty; Expires=Tue, 19-Jan-2038 03:14:07 GMT; Path=/";
return 200 "You've been logged out!";
}
P.S. Note that above code technically opens you up to XSS attacks — any other page can simply embed an iframe with /logout on your site, and your users would be logged out. Ideally, you might want to use a confirmation of logout, or check $http_referer to ensure the link is clicked from your own site.

Related

Nginx + uWSGI + Django too slow

I have django running on nginx and uwsgi. The cached response loads very fast but at other times the website takes more than 30s to load. I am unable to diagnose the root cause of slowing down. Here's what I can provide as info to help narrow down the issue -
GTMetrix - For what I can conclude from waterfall report is that the waiting time for static files is too much alongwith the initial server response time. Here is a more detailed breakdown:
Link to the lighthouse parameters Waterfall report
nginx.conf - Here is the nginx config file:
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 75;
types_hash_max_size 2048;
client_max_body_size 5M;
sendfile_max_chunk 512;
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format upstream_time '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"'
'rt="$request_time" uct="$upstream_connect_time"
uht="$upstream_header_time" urt="$upstream_response_time"';
access_log /var/log/nginx/access.log upstream_time;
error_log /var/log/nginx/error.log;
gzip on;
gzip_disable msie6;
# And all the gzip mime types here
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
proxy_cache_path /data/cache levels=1:2 keys_zone=my_cache:10m max_size=10g
inactive 60m use_temp_path off;
server {
location ~* \.(jpg|jpeg|png|gif|ico|css|js){
proxy_cache my_cache;
proxy_cache_revalidate on;
proxy_cache_min_uses 3;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503
http_504;
proxy_cache_lock on;
expires 365d;
proxy_pass http://example.net;
}
}
}
Nginx Project Config -
map $sent_http_content_type $expires{
default on;
text/html epoch;
text/css max;
appplication/javascript max;
~image/ max;
}
server{
listen 80;
server_name example.com;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/mysite/project_dir/app_dir;
expires $expires;
}
location /images/ {
expires $expires;
root /home/mysite/project_dir/app_dir/static/images/;
}
location /media/ {
expires $expires;
root /home/mysite/project_dir/;
}
location / {
include uwsgi_params;
uwsgi_pass unix:/run/uwsgi/mysite.sock;
gzip_static on;
proxy_buffering off;
proxy_cache my_cache;
proxy_cache_revalidate on;
proxy_cache_min_uses 3;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503
http_504;
proxy_cache_lock on;
expires 365d;
proxy_set_header X-Real-IP $remote-addr;
proxy_set_header Host $http-host;
proxy_set_header Connection "";
}
listen 443 ssl http2;#Managed by certbot
#All the subsequent certbot settings not tampered with
}
Logs - So, when I log nginx using the above config, the access logs show upstream_response_time perfectly only if the website was cached loaded. When it takes >30s to load, the upstream_response_time including all parameters except response_time show hyphen '-'.
UPDATE:
django-debug-toolbar- Resource Usage:
Resource
Value
User CPU time
964.000 msec
System CPU time
52.000 msec
Total CPU time
1016.000 msec
System CPU time
1019.185 msec
All the SQL queries are taking minimal time(10.78ms). Logger too shows 0 errors.
I would highly appreciate if anyone could help me diagnose the root cause of this slowdown. Thank you!
Phew! So I figured out the solution. I used - https://www.webpagetest.org and arrived to a conclusion that the initial connection time was very high (~30s). When it happens, it is most likely some dns/firewall issue. My issue was dns based. I had 2 ips added as A record to my domain. One was a private ip. So the browser actually took ~30s to load that ip and when the website got loaded, the browser cached the response so the subsequent response times were low. Simply removing the private ip worked for me.

nginx dynamic server_name with variable

Dear StackOverflow community I am working with a third party that does not support dynamic GET requests (eg example.com?variable=somethingDynamic) thus I resotred to using custom sub-domains, however I prefer not to make a sub domain for each and every dynamic request so I have been wondering:
how can I write server_name in a way to catch two or three dynamic variables?
here is my example server block:
server {
listen 80;
server_name someSecretUrl_$variable1_$variable2.example.com;
root /usr/share/campagins/campagin1;
client_max_body_size 10000m;
proxy_connect_timeout 30000;
location /funnel_webhooks/test {
return 200;
}
location / {
if ($request_method = 'OPTIONS') {
# Tell client that this pre-flight info is valid for 20 days
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain charset=UTF-8';
add_header 'Content-Length' 0;
add_header 'Access-Control-Allow-Origin' "$http_origin" always;
add_header 'Access-Control-Allow-Credentials' 'true' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With,Etag,Last-Modified,HTTP_IF_MODIFIED_SINCE,HTTP_IF_NONE_MATCH' always;
return 204;
}
add_header 'Access-Control-Allow-Origin' "$http_origin" always;
add_header 'Access-Control-Allow-Credentials' 'true' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With,Etag,Last-Modified,HTTP_IF_MODIFIED_SINCE,HTTP_IF_NONE_MATCH,ETag,Retry-After' always;
add_header 'Access-Control-Expose-Headers' 'ETag,Retry-After' always;
add_header 'Cache-Control' "must-revalidate, post-check=0, pre-check=0" always;
rewrite ^(.*)$ $1?preMadeDataParsers=$variable1&preMadeDataParsersOnResponse=$variable2&$args break;
proxy_buffering off;
proxy_pass http://localhost:3000; #proxy_set_header X-Real-IP $remote_addr;
proxy_set_header REMOTE_ADDR $remote_addr;
}
}
Looking at the code above you will notice I am trying to move $variable1 into a GET variable, and $variable2 into another get variable aswell, how can I achieve such a thing?
thanks!
I wouldn't do this in nginx, I'd do this in your application. (Especially if you expect to expand on this resource.)
I would configure the server to listen on the IP with no virtual hosts at all, so that it answers any request made to the IP. Just leave out the server_name directive:
server {
listen 1.2.3.4:80;
...
Then configure your DNS with a wildcard entry so that *.example.com points to that IP. Now you can hit any_string.example.com and it will resolve to your IP, get answered by the main server block, and passed to your app.
Then, inside your app, look at what hostname was requested. (In PHP for example, this is available via $_SERVER['HTTP_HOST'].) If your app determines that the requested hostname is invalid, just issue a 404 and exit. Otherwise, decode the hostname and process the request.
This way, you can add new variables and new features without editing the nginx config. You could even encode your variables in JSON then BASE64 encode them:
$vars = [
'var1' => 'one',
'var2' => 'two',
'var3' => 'three',
];
$url = base64_encode(json_encode($vars));
eyJ2YXIxIjoib25lIiwidmFyMiI6InR3byIsInZhcjMiOiJ0aHJlZSJ9.example.com
Now you can pass any number of variables, with any names, including indexed and associative arrays. (Though note there is a limit to the domain name length, and you'll have to do something about the + and / characters which I'm pretty sure aren't valid in domain names.)

nginx upstream timed out (110: Connection timed out) while reading response header from upstream fixed by server restart

We are running nginx as a front end proxy server, back to our api servers.
Nginx proxies traffic to an Amazon application load balancer (ALB) which has three plus servers attached to it. Here are the applicable parts of our nginx.conf:
sendfile on;
tcp_nopush on;
tcp_nodelay on;
client_header_timeout 30;
client_body_timeout 30;
client_max_body_size 350M;
gzip on;
gzip_vary on;
gzip_comp_level 2;
gzip_buffers 4 8k;
gzip_min_length 1024;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain application/javascript application/x-javascript text/xml text/css application/json;
gzip_disable "MSIE [1-6]\.";
client_body_buffer_size 500k;
client_header_buffer_size 64k;
large_client_header_buffers 4 64k;
types_hash_max_size 2048;
keepalive_timeout 30;
keepalive_requests 1024;
proxy_connect_timeout 300;
proxy_read_timeout 300;
proxy_send_timeout 300;
server {
listen 8888 default_server;
server_name _;
root /usr/share/nginx/html;
include /etc/nginx/default.d/*.conf;
index index.html;
location /api/ {
proxy_pass http://internal-ALB:8080/api/;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header Access-Control-Allow-Origin $http_origin;
}
}
Periodically we will get a server slowdown, and the site will eventually go down. We will see the following errors in the logs:
upstream timed out (110: Connection timed out) while reading response header from upstream
However, when we look at all of the upstream components they are working. The ALB is working, and all targets are healthy. The individual servers are up and healthy. It appears that "something happened" causing it to break, but whatever that event was it was resolved. However, after the resolution the proxy connections were not reset, still resulting in timeouts.
We can resolve the issue by restarting nginx, However, we want nginx to reset the proxy connection on it's own without restarting the server.
Based on our setup, we are having a very difficult problem replicating this issue, because it may be AWS related. However, we would like our system to be fault tolerant and resolve this issue without restarting the server.
thanks.
myles.

nginx: serve static content with alias url

I'm writing a web service using django and rest-framework and I use nginx as my web server.
The clients will work with the API using standard REST methods. One of the modules lets users to upload a photo and the service changes the name to random string and saves it under: /home/myProject/files/user-content/, like:
/home/myProject/files/user-content/bb7dfb34336d4b638e50040cf91b8d9d.png
At the API level everything works just fine. But I want the users to be able to get the files they have uploaded from: /image/:filename like:
http://mydomain/images/bb7dfb34336d4b638e50040cf91b8d9d.png
To achieve that, I this is the configuration I have for nginx:
server {
listen 80;
error_page 404 /index.html;
root /home/webworker/landingpage/Page;
index index.html index.htm;
server_name localhost;
location / {
proxy_pass http://0.0.0.0:8001;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Real-IP $remote_addr;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
}
location ~ ^/images/(.*)$ {
alias /home/myProject/files/user-content/$1;
access_log on;
index index.html;
autoindex off;
gzip_static on;
# expires max;
sendfile off;
add_header Cache-Control no-cache;
}
}
nginx service starts with no error, but entering the URL I think should work returns with 404 not found. BTW the files have read permission for all users.
replace this line
location ~ ^/images/(.*)$ {
with
location /images/ {
This should satisfy your need.

How do i configure the django rest framework pagination url

when I get an object in django rest framework the urls always come absolute with localhost, but in production im going through a proxy on nginx, is there a way to set this url in the settings
Example
count: 11
next: "http://localhost:8000/api/accounts/?ordering=-date_registered&page=2"
previous: null
I need it to be
count: 11
next: "http:/example.com/api/accounts/?ordering=-date_registered&page=2"
previous: null
---------- edit --------------------------
please see my complete nginx config
server {
listen 80;
server_name 123.123.123.123;
root /home/admin/www/site-web/dist;
index index.html;
charset utf-8;
location /static/ {
alias /home/admin/www/site/static/;
}
location /media/ {
alias /home/admin/www/site/media/;
}
location /nginx_status/ {
# Turn on nginx stats
stub_status on;
# I do not need logs for stats
access_log off;
# Security: Only allow access from 192.168.1.100 IP #
# allow 192.168.1.100;
# Send rest of the world to /dev/null #
# deny all;
}
location / {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
try_files $uri $uri/ /index.html;
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';
#
# Om nom nom cookies
#
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
#
# Custom headers and headers various browsers *should* be OK with but aren't
#
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
#
# Tell client that this pre-flight info is valid for 20 days
#
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain charset=UTF-8';
add_header 'Content-Length' 0;
return 204;
}
if ($request_method = 'POST') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}
if ($request_method = 'GET') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}
}
location /docs/ {
proxy_pass http://127.0.0.1:8000/docs/;
break;
}
location /api/ {
underscores_in_headers on;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://127.0.0.1:8000/api/;
break;
}
location /admin/ {
proxy_pass http://127.0.0.1:8000/admin/;
break;
}
}
==== super edit====
Sorry guys, i had 'underscores_in_headers on;' i removed it and all is working
================
It sounds like your Host header is not being set properly, which would be an issue in your nginx configuration. The issue is that your Host header that is being sent includes the port number, so Django is including the port number when building out urls. This will cause future issues with CSRF, because CSRF checks do strict port checking when you are not debugging.
This is known to cause issues with SSL for similar reasons.
You can fix this by setting the Host header within Nginx to not include the proxied port.
proxy_set_header Host $http_host;
Note that I used the $http_host variable instead of $host or $host:$server_port. This will ensure that Django will still respect CSRF requests on non-standard ports, while still giving you the correct absolute urls.
Set USE_X_FORWARDED_HOST in your settings to True and make sure you pass it along using your web server(proxy) as well.
When django does build_absolute_uri() it calls get_host() - see below in django.http.request:
def get_host(self):
"""Returns the HTTP host using the environment or request headers."""
# We try three options, in order of decreasing preference.
if settings.USE_X_FORWARDED_HOST and (
'HTTP_X_FORWARDED_HOST' in self.META):
host = self.META['HTTP_X_FORWARDED_HOST']
...
See Real life usage of the X-Forwarded-Host header?