I'm implementing nginx as a reverse proxy for a Django project. I'm now trying implement nginx's cache config below:
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:5m max_size=1g inactive=60m;
proxy_temp_path /var/cache/nginx/tmp;
server {
...
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
# cache
proxy_cache my_cache;
proxy_cache_min_uses 1;
proxy_cache_valid 200 302 10m;
proxy_cache_valid 404 1m;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
proxy_cache_use_stale error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_cache_lock on;
proxy_hide_header "Set-Cookie";
add_header X-Proxy-Cache $upstream_cache_status;
}
}
All works great, however I'd like it so that authenticated users are able to bypass the cache, as otherwise when they update the site content, they won't see the changes, only the cached content until it expires.
What would be the best way to approach this?
Any help would be much appreciated.
Thanks
Add a middleware to set cookie (eg cachedisable) in response if user is autheticated else remove that cookie. So that you can put $cookie_ cachedisable in nginx conf
location / {
proxy_cache_bypass $cookie_cachedisable $arg_nocache;
...
}
Whenever you want bypass cache do requests like
http://www.example.com/?nocache=true #or if cookie `cachedisable` is present
For more info https://www.nginx.com/blog/nginx-caching-guide/?utm_source=nginx-caching-guide&utm_medium=blog#caching-guide-faq-hole-punch
Related
I'm using wso2am-4.1.0 and I changed the default hostname and defined a proxy port. Further, I fronted the APIM using an Nginx.
However, once I try to login to the publisher or access the devportal, it fails due to authenticationendpoint being not found (probably the redirection to a certain endpoint fails).
deployment.toml
hostname = "<mydomain>"
base_path = "${carbon.protocol}://${carbon.host}:${carbon.management.port}"
server_role = "default"
[transport.https.properties]
proxyPort = 443
Error in the browser network console
Any viable solution for this is highly appreciated. Thanks in advance.
Update:
Nginx Configuration
server {
listen 80;
server_name <MY_DOMAIN>;
return 301 https://<MY_DOMAIN>$request_uri;
}
server {
listen 443 ssl;
ssl_certificate /etc/nginx/ssl/<MY_CER>.cer;
ssl_certificate_key /etc/nginx/ssl/<MY_KEY>.key;
access_log /var/log/nginx/<CONF>.access.log;
error_log /var/log/nginx/<CONF>.error.log debug;
server_name <MY_DOMAIN>;
add_header X-Frame-Options "SAMEORIGIN";
add_header Strict-Transport-Security "max-age=31536000; includeSubdomains; preload";
add_header Content-Security-Policy "default-src 'self' http: https: data: blob: 'unsafe-inline'" always;
add_header X-XSS-Protection "1; mode=block";
add_header 'Referrer-Policy' 'origin';
#add_header Content-Security-Policy "default-src 'self';" always;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
location / {
return 301 https://<MY_DOMAIN>/authenticationendpoint/;
}
location /authenticationendpoint {
limit_except GET HEAD POST { deny all; }
proxy_pass https://xx.x.x.x:9443;
proxy_set_header x-Real-IP $remote_addr;
proxy_set_header Host $host;
}
location /logincontext {
limit_except GET HEAD POST { deny all; }
proxy_pass https://xx.x.x.x:9443;
proxy_set_header x-Real-IP $remote_addr;
proxy_set_header Host $host;
}
location /oidc {
limit_except GET HEAD POST { deny all; }
proxy_pass https://xx.x.x.x:9443;
proxy_set_header x-Real-IP $remote_addr;
proxy_set_header Host $host;
}
location /oauth2 {
limit_except GET HEAD POST { deny all; }
proxy_pass https://xx.x.x.x:9443;
proxy_set_header x-Real-IP $remote_addr;
proxy_set_header Host $host;
}
location /commonauth {
limit_except GET HEAD POST { deny all; }
proxy_pass https://xx.x.x.x:9443;
proxy_set_header x-Real-IP $remote_addr;
proxy_set_header Host $host;
}
location /publisher {
limit_except GET HEAD POST PUT { deny all; }
proxy_pass https://xx.x.x.x:9443;
proxy_set_header x-Real-IP $remote_addr;
proxy_set_header Host $host;
}
location /devportal {
limit_except GET HEAD POST PUT { deny all; }
proxy_pass https://xx.x.x.x:9443;
proxy_set_header x-Real-IP $remote_addr;
proxy_set_header Host $host;
}
}
From your config, I can't exactly tell what's going on. But your 404 is being generated from the authenticationendpoint it seems. For example, if you try to call the authenticationendpoint with an invalid path you will see a 404.
curl https://localhost:9443/authenticationendpoint => 302
curl https://localhost:9443/authenticationendpoint/xxx => 404
So I suspect given you have a default localtion block(location /) in your NginX to redirect to authenticationendpoint in your request flow something is redirected to the default block. Hence you are getting the 404. I would suggest checking this in the Network traces in the browser console.
Also, another important thing you have missed is, you have to add a location block to capture requests to /api as well.
Currently infra has NGINX play as reverse proxy with ELB(classic load balance). would i know whether AWS ALB can achieve similar reverse proxy feature. such as something as below in NGINX:
location /v1/ {
proxy_pass http://tomcat/serviceA/v1/;
proxy_next_upstream error timeout invalid_header http_500 http_502
http_503 http_504;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
it looks ALB can support http redirects but only support GET method. Is it possible for POST?
when I get an object in django rest framework the urls always come absolute with localhost, but in production im going through a proxy on nginx, is there a way to set this url in the settings
Example
count: 11
next: "http://localhost:8000/api/accounts/?ordering=-date_registered&page=2"
previous: null
I need it to be
count: 11
next: "http:/example.com/api/accounts/?ordering=-date_registered&page=2"
previous: null
---------- edit --------------------------
please see my complete nginx config
server {
listen 80;
server_name 123.123.123.123;
root /home/admin/www/site-web/dist;
index index.html;
charset utf-8;
location /static/ {
alias /home/admin/www/site/static/;
}
location /media/ {
alias /home/admin/www/site/media/;
}
location /nginx_status/ {
# Turn on nginx stats
stub_status on;
# I do not need logs for stats
access_log off;
# Security: Only allow access from 192.168.1.100 IP #
# allow 192.168.1.100;
# Send rest of the world to /dev/null #
# deny all;
}
location / {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
try_files $uri $uri/ /index.html;
if ($request_method = 'OPTIONS') {
add_header 'Access-Control-Allow-Origin' '*';
#
# Om nom nom cookies
#
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
#
# Custom headers and headers various browsers *should* be OK with but aren't
#
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
#
# Tell client that this pre-flight info is valid for 20 days
#
add_header 'Access-Control-Max-Age' 1728000;
add_header 'Content-Type' 'text/plain charset=UTF-8';
add_header 'Content-Length' 0;
return 204;
}
if ($request_method = 'POST') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}
if ($request_method = 'GET') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
}
}
location /docs/ {
proxy_pass http://127.0.0.1:8000/docs/;
break;
}
location /api/ {
underscores_in_headers on;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://127.0.0.1:8000/api/;
break;
}
location /admin/ {
proxy_pass http://127.0.0.1:8000/admin/;
break;
}
}
==== super edit====
Sorry guys, i had 'underscores_in_headers on;' i removed it and all is working
================
It sounds like your Host header is not being set properly, which would be an issue in your nginx configuration. The issue is that your Host header that is being sent includes the port number, so Django is including the port number when building out urls. This will cause future issues with CSRF, because CSRF checks do strict port checking when you are not debugging.
This is known to cause issues with SSL for similar reasons.
You can fix this by setting the Host header within Nginx to not include the proxied port.
proxy_set_header Host $http_host;
Note that I used the $http_host variable instead of $host or $host:$server_port. This will ensure that Django will still respect CSRF requests on non-standard ports, while still giving you the correct absolute urls.
Set USE_X_FORWARDED_HOST in your settings to True and make sure you pass it along using your web server(proxy) as well.
When django does build_absolute_uri() it calls get_host() - see below in django.http.request:
def get_host(self):
"""Returns the HTTP host using the environment or request headers."""
# We try three options, in order of decreasing preference.
if settings.USE_X_FORWARDED_HOST and (
'HTTP_X_FORWARDED_HOST' in self.META):
host = self.META['HTTP_X_FORWARDED_HOST']
...
See Real life usage of the X-Forwarded-Host header?
TLDR: How to share cookies between subdomains for a backend application sever that I cannot "configure" using nginx (1.8.x) as a proxy - some magical combination of proxy_*?
A tornado web server is running on "127.0.0.1:9999/ipython" that I cannot configure (it's running as part of an ipython notebook server). I'm using nginx to proxy from "www.mysite.com" to 127.0.0.1:9999 successfully (http traffic at least).
However, part of the backend application requires Websockets. Because I am using CloudFlare, I have to use a separate domain for Websockets ("Websockets are currently only available for Enterprise customers ... All other customers ... should create a subdomain for Websockets in their CloudFlare DNS and disable the CloudFlare proxy"). I'm using "ws.mysite.com".
When a user logs in at "https :// www.mysite.com", a cookie is set by the tornado web server for "www.mysite.com" (I can't seem to configure it, otherwise I would just set it to ".mysite.com"). When the websocket part of the application kicks in, it sends a request to "wss :// ws.mysite.com", but fails to authenticate because the cookie is set for a different domain("www.mysite.com").
Is it possible for nginx to "spoof" the domain so the tornado webserver registers it for ".mysite.com"? proxy_cookie_domain doesn't seem to work as I'd expect... Should I hard code "proxy_set_header Host"?
I was thinking a nginx conf similar to....
upstream ipython_server {
server 127.0.0.1:8888;
}
server {
listen 443;
server_name www.mysite.com;
ssl_certificate cert.crt;
ssl_certificate_key cert.key;
ssl on;
# **** THIS DOESN'T WORK ??? ****
proxy_cookie_domain www.mysite.com .mysite.com;
location /ipython/static {
proxy_pass https://ipython_server$request_uri;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /ipython/api/sessions {
proxy_pass https://ipython_server$request_uri;
proxy_set_header Host $host;
proxy_set_header Origin "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /ipython {
proxy_pass https://ipython_server$request_uri;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location / {
try_files $uri $uri/ =404;
}
}
server {
listen 443;
server_name ws.azampagl.com;
ssl_certificate cert.crt;
ssl_certificate_key cert.key;
ssl on;
# **** THIS DOESN'T WORK ??? ****
proxy_cookie_domain ws.mysite.com .mysite.com;
# This is the websocket location
location /ipython/api/kernels/ {
proxy_pass https://ipython_server$request_uri;
proxy_redirect off;
proxy_http_version 1.1;
chunked_transfer_encoding off;
proxy_buffering off;
proxy_read_timeout 86400;
proxy_set_header Host $host;
proxy_set_header Origin "";
proxy_set_header Upgrade websocket;
proxy_set_header Connection "upgrade";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-NginX-Proxy true;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
I've been looking in the nginx lua module? It looks like you can set cookie domains, but it looks hackish...
Thanks greatly in advance for your assistance!
(Side note: I do technically have access to the tornado configuration, but there is zero documentation on how to set the "cookie domain" for the server. i.e.
c.NotebookApp.tornado_settings = {'cookie_domain????':'.mysite.com'}
)
I have two separate server,one is nginx with node,and another one is django with django-rest-framework for build ding REST API,nginx is responsible for the REST API request,node takes care of client request, also i use polymer for the frontend .Below are a brief description:
machine one:
nginx:192.168.239.149:8888 (API listening address) forward to 192.168.239.147:8080
node:192.168.239.149:80 (client listening address)
machine two:
unicorn:192.168.239.147:8080(listening address)
The process is when a request comes in,node server(192.168.239.149:80) responses to return html,in html an AJAX request ask for API server(nginx:192.168.239.149:8888 forward to unicorn:192.168.239.147:8080),and then unicorn(192.168.239.147:8080) returns the result.
but there is a CORS problem,I read a lot article,and many people met the same questions,I tried many methods,but no help.still error.
what i get is :
that is:
XMLHttpRequest cannot load http://192.168.239.149:8888/article/. Request header field Access-Control-Allow-Origin is not allowed by Access-Control-Allow-Headers.
What i do is :
core-ajax
<core-ajax auto headers='{"Access-Control-Allow-Origin":"*","X-Requested-With": "XMLHttpRequest"}' url="http://192.168.239.149:8888/article/" handleAs="json" response="{{response}}"></core-ajax>
nginx:
http {
include mime.types;
default_type application/octet-stream;
access_log /tmp/nginx.access.log;
sendfile on;
upstream realservers{
#server 192.168.239.140:8080;
#server 192.168.239.138:8000;
server 192.168.239.147:8080;
}
server {
listen 8888 default;
server_name example.com;
client_max_body_size 4G;
keepalive_timeout 5;
location / {
add_header Access-Control-Allow-Origin *;
try_files $uri $uri/index.html $uri.html #proxy_to_app;
}
location #proxy_to_app{
add_header Access-Control-Allow-Origin *;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
#proxy_set_header X-Real-IP $remote_addr;
proxy_redirect off;
proxy_pass http://realservers;
}
}
}
node:
app.listen(80, function() {
console.log('server.js running');
});
unicorn:
return Response(serializer.data,headers={'Access-Control-Allow-Origin':'*',
'Access-Control-Allow-Methods':'GET',
'Access-Control-Allow-Headers':'Access-Control-Allow-Origin, x-requested-with, content-type',
})
Because,I have not much experience on CORS,and I want to understand it thoroughly,can anyone point out what i was doing wrong here,I will thank you very much!
Wow,so excited,I sovled this all by my self,what i do wrong here is that the request header i sent is not included in the nginx config add_header 'Access-Control-Allow-Headers'
complete nginx config:
http {
include mime.types;
default_type application/octet-stream;
access_log /tmp/nginx.access.log;
sendfile on;
upstream realservers{
#server 192.168.239.140:8080;
#server 192.168.239.138:8000;
server 192.168.239.147:8080;
}
server {
listen 8888 default;
server_name example.com;
client_max_body_size 4G;
keepalive_timeout 5;
location / {
add_header Access-Control-Allow-Origin *;
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Access-Control-Allow-Orgin,XMLHttpRequest,Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Mx-ReqToken,X-Requested-With';
try_files $uri $uri/index.html $uri.html #proxy_to_app;
}
location #proxy_to_app{
add_header Access-Control-Allow-Origin *;
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Access-Control-Allow-Orgin,XMLHttpRequest,Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Mx-ReqToken,X-Requested-With';
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
#proxy_set_header X-Real-IP $remote_addr;
proxy_redirect off;
proxy_pass http://realservers;
}
}
}
because my request is :
core-ajax auto headers='{"Access-Control-Allow-Origin":"*","X-Requested-With": "XMLHttpRequest"}' url="http://192.168.239.149:8888/article/" handleAs="json" response="{{response}}"></core-ajax>
i didnt include the Access-Control-Allow-Origin and XMLHttpRequest header into the nginx config Access-Control-Allow-Headers,so that is the problem.
I hope its useful to whom has the same problem!
You do not have to include CORS header into request manualy. The browser takes care of it, you just need to allow it on the api server