how to break a variable into seperate variables in nginx using regex - regex

I have a development server setup that does some dynamic rooting to allow me to set up quick test projects by detecting domain and subdomain in server_name and using it to set the root.
server_name ~^(?<subdomain>\w*?)?\.?(?<domain>\w+\.\w+)$;
This works well, allowing me to set the root path based on variables $subdomain and $domain
For a specific type of project though, I also need to be able to further split the subdomain variable into two variables base on if the subdomain contains a dash.
e.g.
mysubdomain should not split but remain as variable $subdomain,
but mysubdomain-tn would be seperated into 2 variable $subdomain and $version

You need to complicate your regular expression a bit more:
server_name ~^(?<subdomain>\w*?)(-(?<version>\w*?)?)?\.?(?<domain>\w+\.\w+)$;
EDIT:
There are several ways to debug an Nginx configuration, including the debugging log, echo module and, in some extreme situations, even using a real debugger. However, in most cases adding custom headers to the response is enough to get the necessary information.
For example, I tested the regular expression above using this simple configuration:
server {
listen 80;
server_name ~^(?<subdomain>\w*?)(-(?<version>\w*?)?)?\.?(?<domain>\w+\.\w+)$;
# Without this line your browser will try to download
# the response as if it were a file
add_header Content-Type text/plain;
# You can name your headers however you like
add_header X-subdomain "$subdomain";
add_header X-domain "$domain";
add_header X-version "$version";
return 200;
}
Then I added domains mydomain.local, mysubdomain-tn.mydomain.local and mysubdomain-tn.mydomain.local to my hosts file, opened them in a browser with open debug panel (F12 in most browsers) and got the results.

Related

Use nginx to block certain requests with regex in location doesn't work

We use an Tomcat application which has been migrated to another system. Also attached is a MSSQL report server. The application needs to be read-only eventually. At this moment we still need to enter some data for certain customers. I use nginx so that only one server is needed for both Tomcat service and Report service. Now I'd like to use nginx location method to block certain requests.
With the help of regex101.com I created a regex to use in nginx. I already tried to change the order of the location blocks. However, I can't get it to work. Here's the link to the regex where you can see that only the blocked URL's are colored: https://regex101.com/r/w3oy0M/1/
Here's the code:
server {
listen 443 ssl;
server_name subdomain.domainname.com;
ssl_certificate c:\cert\public.crt;
ssl_certificate_key c:\cert\private.rsa;
location / {
proxy_pass https://subdomain.domainname.com:8443;
}
location ~ "https:\/\/subdomain\.domainname\.com:8443\/(?!.*customer\/510|.*customer\/10638)(.*\/edit|.*\/new|.*\/archive)" {
# TEST URL FOR NOT ALLOWED REQUESTS
return 301 https://www.google.com;
}
location /Reports {
return 301 https://subdomain.domainname.com:7443/Reports;
}
}
Try
location ~ /.+?(?=customer\/)(?!customer\/510|customer\/10638)(.*\/edit|.*\/new|.*\/archive) {
# TEST URL FOR NOT ALLOWED REQUESTS
return 301 https://www.google.com;
}
However please note anything after # will be dropped as it's an anchor element so a request like
https://subdomain.domainname.com/#/customer/521/installations/new will be sent to nginx as GET / and gets routed to your first location block
But, a request like https://subdomain.domainname.com/hello/customer/521/installations/new will be routed to https://google.com

serve different content according to header value in Nginx

I have some web content behind the follwoing arch, for which I need to serve different content for desktop and mobile clients.
client > AWS Cloudfront > Nginx > ...
cloudfront has the builtin ability to identify the user-agent and they kind of unify it into 4 special headers (for example CloudFront-Is-Desktop-Viewer) which are either true or false.
on Nginx I'm trying to decide which content to serve according to those headers.
for example:
location / {
if ($http_CloudFront-Is-Desktop-Viewer = true) {
proxy_pass http://upstream;
break;
}
root /var/www/static/en-US;
try_files $uri /index.html;
}
so in the above case it should go to the upstream if the CloudFront-Is-Desktop-Viewer value is true, and get the static files from nginx if it's false or not present.
but for some reason I always get the static files from nginx.
I'm sure this header is being forwarded, I've even tried to send it directly from chrome with a header modifier.
what am i missing?
many thanks
Try this variable:
$http_cloudfront_is_desktop_viewer
From manual about embedded variables:
$http_name nginx
arbitrary request header field; the last part of a variable name is
the field name converted to lower case with dashes replaced by
underscores

How to mediate Django-Tastypie URI with Nginx routing

Specifically, I think I need Nginx to not consume (capture?) a piece of the URI I'm using to route a location. But I don't know if such a thing is possible.
Let me back up. I'm transitioning my app's setup. Before, I had an Nginx config file with a single location block matching everything:
server {
listen 80;
server_name ec2-54-234-175-21.compute-1.amazonaws.com;
location / {
...
proxy_pass http://localhost:8000/;
}
With this setup, up until now, I've just been running a Django app. After Nginx routes the user to the only endpoint, Django consumes the whole URI. What this means is that internally, Django chops off /api/ and then just has 1.0, which it also needs.
Now, I'm going to use Nginx to proxy for multiple servers, each hosting an app. Because of the aforementioned internal routing, the Django app needs to receive (at least) a URI of /1.0, while the other (on Flask) needs to get /api/2.0. However, in order for the Nginx location directives to make any sense, I have to differentiate the two applications. What I've worked out is that I'll just have the following two location directives:
server {
listen 80;
server_name ec2-54-234-175-21.compute-1.amazonaws.com;
location /api/[1.0] {
...
proxy_pass http://localhost:8000/;
}
location / {
...
proxy_pass http://localhost:8080/;
}
However, note the [1.0] in brackets. If that is there, what I've realized is that in order to actually access the intended resource, I have to enter a URI of /api/1.0/1.0. So, somehow I need a non-consuming location in my nginx conf. If I knew how to express this in simpler terms, I would. Is what I want possible?
About a week after asking this question, a coworker pointed out the answer I wanted, and it's very simple. The key is in the malleability that Nginx gives you with its routing. Because I wanted to access a resource that was at /api/1.0, but still needed to differentiate the two apps in my config, I do the following:
server {
listen 80;
server_name ec2-54-234-175-21.compute-1.amazonaws.com;
location /api/1.0 {
...
proxy_pass http://localhost:8000/api/1.0;
}
location /api/2.0 {
...
proxy_pass http://localhost:8080/api/2.0;
}
This effectively makes the URL "non-consuming", as I wrote above, because I use the desired resource URL to route with, and then duplicate it in referencing the actual location within the specific app.
Maybe this will help someone else.

Is Host header poisoning possible in such case?

Django team considers host header poisoning (CVE-2011-4139 and CVE-2012-4520) as a security issue that must be resolved at a framework level. Pyramid, for instance (that is, its underlying low-level request wrapper—webob) does not consider this as an issue.
On production & development machines I have nginx which seems to pass correct SERVER_NAME even if Host header contains complete garbage, and responds with 444 No response if there is no matching server_name.
Question: should I worry about Host header poisoning in such case, if I use SERVER_NAME to build absolute URLs?
If you use nginx to sanitize the HTTP_HOST and SERVER_NAME fields, you are doing the right thing and do not need to worry about Host header poisining.
Like Django, Pyramid considers a large part of this the task of the WSGI host environment. And nginx does an excellent, battle-hardened job of sanitizing the HTTP request information.

Missing custom header with django, nginx and gunicorn

Disclaimer:
I'm working in a project where exist an "huge" webapp that have an api for mobiles, so change the api is not an option.
This application was developed time ago and several developers have worked on it,
Having said that, the problem is this;
In the api for mobile of this site (just views than returns json data), the code is looking for a token but does in the headers of request:
token = request.META.get('HTTP_TOKEN')
When I test this api locally, works fine, but in production doesn't, so, I try to figure out whats going on and found this:
django converts headers, even custom headers to keys in request.META, I use urllib2 and requests for test the api and the problem in production is that in production server the request.META never has a key called HTTP_TOKEN, so, doing a little of debug I seriously think the problem is the way we serve the django application.
We are using django1.3, nginx, gunicorn, virtualenvwrapper, python2.7.
My prime suspect is nginx, I think, in someway nginx receive the header but don' forward it to django, I try to do some research about this, but I only found infor about security headers and custom headers from nginx, but I dont find doc or something about how to tell nginx that allows that header and don't remove it.
I need help here, the first thing is test if nginx receives the header, but I just know a little about nginx and I don't know how to tell it to log the headers of requests.
Thanks
Update
nginx conf file
If Django is accessed using uwsgi_pass, then in the appropriate location(s) ...
# All request headers should be passed on by default
# Make sure "Token" response header is passed to user
uwsgi_pass_header Token;
If Django is accessed using fastcgi_pass, then in the appropriate location(s) ...
# All request headers should be passed on by default
# Make sure "Token" response header is passed to user
fastcgi_pass_header Token;
If Django is accessed using proxy_pass, then in the appropriate location(s) ...
# All request headers should be passed on by default
# but we can make sure "Token" request header is passed to Django
proxy_set_header Token $http_token;
# Make sure "Token" response header is passed to user
proxy_pass_header Token;
These should help eliminate the possibility that Nginx is not passing things along from your issue.
In your nginx configuration file (f.e. mysite_nginx.conf) in the server section add this parameter: uwsgi_pass_request_headers on;.
For example:
server {
# the port your site will be served on
listen 8000;
...
underscores_in_headers on;
}
And if access to Django goes through uwsgi_pass, you need to add this one parameter uwsgi_pass_request_headers on; in location section.
For example:
location / {
include /etc/nginx/uwsgi_params; # the uwsgi_params file you installed
uwsgi_pass_request_headers on;
uwsgi_pass django;
}
I think this is what you need:
log_format combined '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" "$http_http_token" "$upstream_http_http_token"'
to log what is happening.
You might look deeper into the proxy_set_header section on the upstream proxy module to see how to pass on the headers you need.
You can find the documentation here:
http://wiki.nginx.org/HttpLogModule
http://wiki.nginx.org/NginxHttpUpstreamModule
http://wiki.nginx.org/NginxHttpProxyModule#proxy_set_header
http://wiki.nginx.org/HttpProxyModule#proxy_pass_request_headers
The last entry seems to indicate that nginx passes most headers by default
I didn't find a real answer, but was able to make a workaround. I was having the same problem with RFC standard headers if-none-match and if-modified-since, so my solution is tested for those headers.
Added to my nginx config:
uwsgi_param HTTP_IF_NONE_MATCH $http_if_none_match;
uwsgi_param HTTP_IF_MODIFIED_SINCE $http_if_modified_since;
I cannot explain why nginx refuses to pass these headers to uwsgi by default. This config forces it. Pages generate 304s as appropriate now.
For the original question about the non-standard "token" header, this should do the trick:
uwsgi_param HTTP_TOKEN $http_token;
The answers above are enough for us to figure out the way. But there is still an annoying point we need to know. Yes, it is very annoying.
proxy_set_header X_FORWARDED_FOR # oops it never works. 1.16.1 on centos7
proxy_set_header X-FORWARDED-FOR # this will do the job
So you get it. Underscore could never appear in the customized variable name. Use a hyphen instead.
Maybe Nginx uses underscores in some grammar cases. Someone pointed out the official reference will be appreciated.
It depends on how the custom header is named. My was in format "SomethingLike.this", it contains a dot. It was not possible to rename the header in the request, because it is not our code. So writing this would not work:
proxy_set_header SomethingLike.this $http_somethinglike.this;
proxy_pass_header SomethingLike.this;
Also this would not work:
underscores_in_headers on;
because I would need dots_in_headers directive which does not exist.
But I found I can pass ALL headers simply by adding:
ignore_invalid_headers off;
It may be insecure to pass all headers, please use with caution.