How can I intercept all outgoing responses within nginx and modify them? - web-services

I have a nginx server that serves a forum via php.
I want to add an upstream to nginx that intercepts all responses generated by the php application and modifies the response.
Is it possible to achieve this by modfying the nginx conf file?
Here is my existing configuration:
server {
listen 443 ssl;
server_name example.net www.example.net;
index index.php index.html;
root /var/www/html;
location / {
try_files $uri $uri/ /index.php?$uri&$args;
}
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass xf2-php-fpm:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
}
}
How can I modify this configuration to pass all responses generated by php to an upstream?

You could add a new upstream block outside of your current ones that defines the server/servers you want to use as the upstream
upstream upStream{
server upStream_server:4000;
}
Then modify the location block that handles PHP requests to proxy the requests to the upstream server
location ~ \.php$ {
proxy_pass http://upStream; #URL to which the request should be proxied
proxy_set_header Host $host; #Hostname of the server to which the request is being sent to
proxy_set_header X-Real-IP $remote_addr; #IP address of the client that made the request
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; #IP address of the client that made the request and any other IP addresses that the request was forwarded through
//
}
This should pass all requests to the upStream server and the response will be sent back to the client. Tell me if you need more help

Related

How to config nginx proxy pass using subfolder domain whith gunicorn Django

How can I configure ngnix to redirect a proxypass from a domain with subfolder to /?
Example:
https://example.com/yoursub/
to
localhost without /yoursub/ prefix
At the moment the direct access to the server ip http://xxx.xxx.xxx.xx/ from the intranet works without problems.
my nginx config file:
upstream app_server {
server unix:/home/myname/APP-Server/gunicorn/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
# add here the ip address of your server
# or a domain pointing to that ip (like example.com or www.example.com)
server_name 123.456.789.1;
keepalive_timeout 5;
client_max_body_size 4G;
access_log /home/myname/APP-Server/logs/nginx-access.log;
error_log /home/myname/APP-Server/logs/nginx-error.log;
location /static/ {
alias /home/myname/APP-Server/static-root/;
}
# checks for static file, if not found proxy to app
location / {
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server;
}
}
If it's relevant: The backend is a django app with a gunicorn server. Do I have to consider anything about the redirect from https to http? I have no control over the base domain.
If I understand correctly, you want to remove the first part of the URI. There are multiple ways you can do that, but the easiest is probably with the alias directive, which will remove the portion of the URI that matches the current location block:
location /foo/ {
alias /home/myname/APP-Server/static-root/; # It doesn't really matter what you put here, since you're proxying everything.
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server;
}
If your Nginx server is running on foobar.example and you request http://foobar.example/foo/bar, the upstream server will see a request for http://foobar.example/bar.
The alias directive can be a bit buggy/unintuitive, so it's best to keep your location directive top-level (not nested within other location blocks) and as simple as possible.
If instead you want to add a prefix to the URI, you can do that within the proxy_pass directive itself:
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server/foo$uri$is_args$args;
}
If your Nginx server is running on foobar.example and you request http://foobar.example/bar, the upstream server will see a request for http://foobar.example/foo/bar
Try this:
server {
...
location #proxy_to_app {
...
proxy_pass http://app_server/; # note the trailing slash
}
}
Explanation
As per the nginx docs:
If the proxy_pass directive is specified with a URI, then when a request is passed to the server, the part of a normalized request URI matching the location is replaced by a URI specified in the directive.
Since / location matches anything, then everything will be replaced by / (the trailing slash in proxy_pass) before being proxied to the upstream.
The way I handle this is with a rewrite rule:
location /yoursub {
rewrite /yoursub(.*) $1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server;
}
The rewrite will convert your url from /yoursub/path/to/view to /path/to/view for your app server.

Force WWW behind an AWS EC2 Load Balancer

I've come up with a small issue, we're using a load balancer for a new project, but we cannot force the www. without having a redirect loop between requests.
We're currently using NGINX, and the snippet to redirect is the following:
LOAD BALANCER NGINX CONFIG
# FORGE CONFIG (DOT NOT REMOVE!)
include forge-conf/mywebsite.com/before/*;
# FORGE CONFIG (DOT NOT REMOVE!)
include upstreams/mywebsite.com;
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name .mywebsite.com;
if ($host !~* ^www\.){
rewrite ^(.*)$ https://www.mywebsite.com$1;
}
# FORGE SSL (DO NOT REMOVE!)
ssl_certificate /etc/nginx/ssl/mywebsite.com/225451/server.crt;
ssl_certificate_key /etc/nginx/ssl/mywebsite.com/225451/server.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
charset utf-8;
access_log off;
error_log /var/log/nginx/mywebsite.com-error.log error;
# FORGE CONFIG (DOT NOT REMOVE!)
include forge-conf/mywebsite.com/server/*;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://370308_app/;
proxy_redirect off;
# Handle Web Socket Connections
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
# FORGE CONFIG (DOT NOT REMOVE!)
include forge-conf/mywebsite.com/after/*;
HTTP SERVER NGINX CONFIG
# FORGE CONFIG (DOT NOT REMOVE!)
include forge-conf/mywebsite.com/before/*;
server {
listen 80;
listen [::]:80;
server_name .mywebsite.com;
root /home/forge/mywebsite.com/public;
if ($host !~* ^www\.){
rewrite ^(.*)$ https://www.mywebsite.com$1;
}
# FORGE SSL (DO NOT REMOVE!)
# ssl_certificate;
# ssl_certificate_key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/nginx/dhparams.pem;
add_header X-Frame-Options "SAMEORIGIN";
add_header X-XSS-Protection "1; mode=block";
add_header X-Content-Type-Options "nosniff";
index index.html index.htm index.php;
charset utf-8;
# FORGE CONFIG (DOT NOT REMOVE!)
include forge-conf/mywebsite.com/server/*;
location / {
try_files $uri $uri/ /index.php?$query_string;
}
location = /favicon.ico { access_log off; log_not_found off; }
location = /robots.txt { access_log off; log_not_found off; }
access_log off;
error_log /var/log/nginx/mywebsite.com-error.log error;
error_page 404 /index.php;
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/var/run/php/php7.1-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
}
location ~ /\.(?!well-known).* {
deny all;
}
}
# FORGE CONFIG (DOT NOT REMOVE!)
include forge-conf/mywebsite.com/after/*;
Thing is, with this config I'm only getting redirect loops from the server.
Help please :D <3
After writing the prior general-purpose answer, I Googled "FORGE CONFIG (DOT NOT REMOVE!)", and this was the first result:
https://laracasts.com/discuss/channels/forge/forge-how-to-disable-nginx-default-redirection
inside nginx/forge-conf/be106.net/before/redirect.conf file there is this simple config:
…
server_name www.my-domain.net;
return 301 $scheme://my-domain.net$request_uri;
…
is there a simple way of removing this without altering the file itself(as it look like bad idea).
So, it appears that the redirect is being caused by the application you're using, so, we found the most likely cause of the loop!
In turn, the appropriate way to configure your application to avoid said loop would be outside of the score of StackOverflow.
However, as a workaround:
consider whether you actually need all those forge-conf include directives at the load-balancer level; subsequently, you could fake the appropriate domain to be passed to the backend that would not cause a redirect (provided you remove your own redundant redirects):
- proxy_set_header Host $http_host;
+ proxy_set_header Host example.com;
note that the reason the forge-conf/example.com/before/redirect.conf directive takes precedence over your own configuration for .example.com is the order of the directive — you could potentially move the /before/* include to be after your own configuration, if such a move would otherwise make sense.
I don't think the nginx snippets you provided would cause a redirect loop by themselves.
First, you have to figure out whether it's an actual redirect — very often in these questions, the 301 Moved Permanently response gets cached in your browser, and subsequently you see a cached version, instead of a fresh one.
Subsequently, you'd have to figure out what is causing the redirect loop:
Try adding unique strings to each redirect directive, to see which one would be causing the loop.
if ($host !~* ^www\.) {return 301 $scheme://www.$host/levelX$request_uri}
Ask yourself why do you have so many redirect directives in the first place — there doesn't seem to be much of a valid reason to have redirect directives both at the front-end load balancer, as well as the backend.
If the above doesn't resolve the issue, then you know that the redirect loop is not coming from the files you've provided, and you have to dig deeper — it's possible for it to come from some other files, perhaps one of your include directives, or perhaps a default server of www.example.com is defined elsewhere, which redirects to example.com, or perhaps the redirect is done at the application layer.

nginx - not to redirect to https

Im trying to setup nginx as a proxy server for my django server and here is my configuration.
For some reason when I send a request to the server http://ipaddress it automatically redirects me to https://ipaddress even though I have included anything to redirect.
I want to disable the redirect to https as its a dev server
upstream app_server {
# For a TCP configuration:
server 127.0.0.1:8000 fail_timeout=0;
}
# configuration of the server
server {
#add_header HTTP_X_FORWARDED_PROTO https;
# the port your site will be served on
listen 80 default_server;
# the domain name it will serve for
charset utf-8;
#server_name localhost;
# max upload size
client_max_body_size 75M; # adjust to taste
location / {
# checks for static file, if not found proxy to app
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server;
}
}

Django #login_required dropping https

I'm trying to test my Django app locally using SSL. I have a view with the #login_required decorator. So when I hit /locker, I get redirected to /locker/login?next=/locker. This works fine with http.
However, whenever I use https, the redirect somehow drops the secure connection, so I get something like https://cumulus.dev/locker -> http://cumulus.dev/locker/login?next=/locker
If I go directly to https://cumulus.dev/locker/login?next=locker the page opens fine over a secure connection. But once I enter the username and password, I go back to http://cumulus.dev/locker.
I'm using Nginx to handle the SSL, which then talks to runserver. My nginx config is
upstream app_server_djangoapp {
server localhost:8000 fail_timeout=0;
}
server {
listen 80;
server_name cumulus.dev;
access_log /var/log/nginx/cumulus-dev-access.log;
error_log /var/log/nginx/cumulus-dev-error.log info;
keepalive_timeout 5;
# path for static files
root /home/gaurav/www/Cumulus/cumulus_lightbox/static;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://app_server_djangoapp;
break;
}
}
}
server {
listen 443;
server_name cumulus.dev;
ssl on;
ssl_certificate /etc/ssl/cacert-cumulus.pem;
ssl_certificate_key /etc/ssl/privkey.pem;
access_log /var/log/nginx/cumulus-dev-access.log;
error_log /var/log/nginx/cumulus-dev-error.log info;
keepalive_timeout 5;
# path for static files
root /home/gaurav/www/Cumulus/cumulus_lightbox/static;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://app_server_djangoapp;
break;
}
}
}
Django is running on plain HTTP only behind the proxy, so it will always use that to construct absolute URLs (such as redirects), unless you configure it how to see that the proxied request was originally made over HTTPS.
As of Django 1.4, you can do this using the SECURE_PROXY_SSL_HEADER setting. When Django sees the configured header, it will treat the request as HTTPS instead of HTTP: request.is_secure() will return true, https:// URLs will be generated, and so on.
However, note the security warnings in the documentation: you must ensure that the proxy replaces or strips the trusted header from all incoming client requests, both HTTP and HTTPS. Your nginx configuration above does not do that with X-Forwarded-Ssl, making it spoofable.
A conventional solution to this is to set X-Forwarded-Protocol to http or https, as appropriate, in each of your proxy configurations. Then, you can configure Django to look for it using:
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTOCOL', 'https')

nginx rewrite leads to redirect loop

I'm using nginx with a reverse proxy to my gunicorn django server at local to run my django app.
I am trying to force ssl from any http request, as the site is only intended https access. The app will ONLY listen to port 8888, (80 and 443 taken), so the site should only be accessible when specified the 8888 port.
I tried to rewrite with this rewrite ^ https://domain.net:8888$request_uri? permanent; in the server block and location block. Not only does it not redirect http requests to the same url, it also causes a redirect loop when there is a https request.
server {
listen 8888;
server_name sy-system.net;
rewrite ^ https://domain.net:8888$request_uri? permanent;
ssl on;
ssl_certificate /path/to/domain.pem;
ssl_certificate_key /path/to/domain.key;
# serve directly - analogous for static/staticfiles
location /media/ {
root /path/to/root;
}
location /static/ {
root /path/to/root;
}
location / {
#rewrite ^ https://sy-system.net:8888$request_uri? permanent;
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_connect_timeout 10;
proxy_read_timeout 10;
proxy_pass http://127.0.0.1:8881/;
proxy_set_header X-Forwarded-Protocol https;
}
# what to serve if upstream is not available or crashes
error_page 500 502 503 504 /media/50x.html;
}
You are unconditionally redirecting from domain.net:8888 to domain.net:8888. An infinite redirect loop is the only possible result.
In any case, what you are attempting to do isn't possible. Nginx is talking SSL, your browser is not, so no data will be transferred between them (which is why you don't get the redirect loop when connecting with plain HTTP). When they do talk (via SSL) your redirect loop takes over.