I have the following code which fires on with a *.m3u8 file is in the url.
Example URL: http://example.com/live/username/password/stream.m3u8
location ~ [^/]\.m3u8(/|$)(?!token$) {
expires 10s;
resolver 1.1.1.1 1.0.0.1 8.8.8.8 8.8.4.4 208.67.222.222 208.67.220.220 valid=60s;
resolver_timeout 5s;
proxy_pass http://origin_server;
proxy_redirect http://1.2.3.4 http://lb_server.example.com;
}
It matches the m3u8 file ext but I need it to NOT use this 'location' if the url looks like *.m3u8?toek=blah_blah
I cannot seem to get it to work for the life of me.
Related
I want to have a private media folder on my django website, accessible only to logged in users, so I got to know that I should handle authentication part on the django side, and file serving on the nginx side. However following internal location config examples I find it impossible to make it work. Nginx ignores django completely (only for the internal location case). Even if I don't have the url allowed in my urls.py and I have it listed as internal location in nginx, it will still be freely accessible to everybody.
I am posting my nginx configuration in hope that someone can find a mistake in it.
My expectation is that everything in /internal/ folder will not be accessible to anonymous users and it will only be accessible by the django application through X-Accel-Redirect header. Right now if I go to /internal/test.png in an incognito window it will show me the picture.
I am not posting my django code for now, since it is ignored anyway by nginx, so it must be the nginx config problem.
server {
server_name XXX.XX.XX.XXX example.com www.example.com;
location = /favicon.ico {
access_log off;
log_not_found off;
alias /home/user/myproject/static/favicon4.ico;
}
location /static/ {
root /home/user/myproject;
}
location /media/ {
root /home/user/myproject;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
location /internal/ {
internal;
root /home/user/myproject;
}
root /home/user/myproject;
location ~* \.(jpg|jpeg|png|webp|ico|gif)$ {
expires 30d;
}
location ~* \.(css|js|pdf)$ {
expires 1d;
}
client_max_body_size 10M;
# below in this server block is only my Certbot stuff
}
P.S. I swapped identifiable data to X characters and basic names.
I had 2 more problems in this config and I will show everything I did to make it work. The original problem why nginx was ignoring django was in how nginx chooses which location block to use, as suggested by Richard Smith.
From nginx.org we can read:
To find location matching a given request, nginx first checks locations defined using the prefix strings (prefix locations). Among them, the location with the longest matching prefix is selected and remembered. Then regular expressions are checked, in the order of their appearance in the configuration file. The search of regular expressions terminates on the first match, and the corresponding configuration is used. If no match with a regular expression is found then the configuration of the prefix location remembered earlier is used.
And also:
If the longest matching prefix location has the “^~” modifier then regular expressions are not checked.
So regular expressions, if available, will be chosen first. ^~ modifier before prefix makes it chosen instead of regular expressions.
I changed location /internal/ { line to location ^~ /internal/ { and then I got 404 errors every time and no matter how I tried to access the files, but at least I knew nginx was going to this location.
The 2nd mistake was thinking that I can get away with using the same url as the folder name, or in other words, that I can put in my urls.py
path('internal/<path>', views.internal_media, name='internal_media')
together with
location ^~ /internal/ {
internal;
root /home/user/myproject;
}
in my nginx config.
I can't. The url must be different, because otherwise the url doesn't lead to django urls.py - it still leads to /internal/ location through nginx (again, due to how nginx chooses locations).
I changed my urls.py line to point to private url instead:
path('private/<path>', views.internal_media, name='internal_media')
and in the views.py file I redirect to /internal/:
def internal_media(request, path):
if request.user.groups.filter(name='team-special').exists():
response = HttpResponse()
response['X-Accel-Redirect'] = '/internal/' + path
del response['Content-Type'] # without this your images will open as text
return response
else:
raise PermissionDenied()
Aaaand this still didn't work. 404 errors every time. The 3rd mistake was forgetting about the combo of those two:
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
location ~* \.(jpg|jpeg|png|webp|ico|gif)$ {
expires 30d;
}
Now if I went to the url /private/test.jpg nginx didn't let me go to django, because location / is lower in priority than regular expressions, so location ~* took precedence and I never got to django. I noticed it by accident after a lot of time being frustrated, when I put the url incorrectly in incognito mode. When I went to /private/test.jp now I got a 403 forbidden error instead of 404.
It started working immediately when I commented out this.
location ~* \.(jpg|jpeg|png|webp|ico|gif)$ {
expires 30d;
}
location ~* \.(css|js|pdf)$ {
expires 1d;
}
So now internal files worked nicely, but I didn't have caching...
To fix that, I modified my /static/ and /media/ locations, but maybe I won't go into that here, since it is a different topic. I'll just post my full nginx config that works :)
Well, what you might want to also know is that:
~* tells nginx that we are writing a regular expression that is case insensitive
~ would tell nginx that we were writing a regular expression that is case sensitive
server {
server_name XXX.XX.XX.XXX example.com www.example.com;
location = /favicon.ico {
access_log off;
log_not_found off;
alias /home/user/myproject/static/favicon4.ico;
expires 30d;
}
location /static/ {
root /home/user/myproject;
expires 30d;
}
location /media/ {
root /home/user/myproject;
expires 30d;
}
location ~* \/(static|media)\/\S*\.(css|js|pdf) {
root /home/user/myproject;
expires 1d;
}
location ^~ /internal/ {
root /home/user/myproject;
internal;
expires 1d;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
client_max_body_size 10M;
# certbot stuff
}
I am trying to exclude some paths in my nginx proxypass and want everything else to go to my proxypass.
i.e I dont want to give proxy_pass to any url which starts with 'tiny' or 'static', but want everythign else to go to my proxypass location.
and I am using following regex to achieve this:
~ ^((?!tiny|static).)*$
But I always get 404 error.
If I navigate to following url in browser
localhost:8080/xyz
I want it to go to
localhost:8000/api/tiny/records/xyz
Can someone please help me in pointing out what is the issue ?
Here is my full nginx conf file:-
server {
listen 8080;
server_name localhost;
location ~ ^((?!tiny|static).)*$ {
proxy_pass http://localhost:8000/api/tiny/records/$1;
}
location / {
proxy_pass http://localhost:8000;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
Thanks a lot.
You are missing a / and have the * in the wrong place. The regular expression should be:
^(/(?!tiny|static).*)$
But you do not need to use a regular expression with a negative lookahead assertion. Instead, place a normal regular expression on the other location block.
For example:
location / {
proxy_pass http://localhost:8000/api/tiny/records/;
}
location ~ ^/(tiny|static) {
proxy_pass http://localhost:8000;
}
I am trying to proxy_pass traffic depending on location:
server {
listen 80
server servername.com.eu
(...)
location ~ /examples {proxy_pass host #1}
location ~* /?ip {proxy_pass host #1}
location / proxy_pass host #2
My problem is that location with /? is not working as expected. It is supposed to be proxy_pass for queries ?ip=X.X.X.X
If this location ? works (tried different regex), then other locations does not.
If it does not work other 2 work just fine.
Any ideas?
I have tried already:
location ~* "/?ip" location ~ "/?ip" location = /?ip and even match for last for ip digits which actually crashes nginx (?:[0-9]{1,3}\.){3}[0-9]{1,3}$ - i suppose sth needs to be escaped
nginx: [emerg] invalid number of arguments in "proxy_pass" directive in /etc/nginx/sites-enabled/django_direct:12
My nginx conf file:
server {
listen 80;
server_name 94.154.13.214;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /root/django_direct/main_app/;
}
location / {
include proxy_params;
proxy_pass unix: /root/django_direct/django_direct.sock;
}
}
What do I do?
UPD:
I have changed file like this:
proxy_pass http://unix:/root/django_direct/django_direct.sock;
But didn't help, I've restarted nginx and now
I am getting now a 502 Bad Gateway error.
If someone else finds this and has this error. Check that you have a semi-colon at the end of the parameters line. goes for all of the lines, not just proxy_pass.
Well the nginx sees two parameters: unix, and /root/django_redirect/.... I have the idea however that you want to specify a UNIX domain socket path. You can do this with:
proxy_pass http://unix:/root/django_direct/django_direct.sock;
As is described in the documentation.
Your argument is wrong.
It needs an URL:
Sets the protocol and address of a proxied server and an optional URI to which a location should be mapped. As a protocol, “http” or “https” can be specified. The address can be specified as a domain name or IP address, and an optional port:
proxy_pass http://localhost:8000/uri/;
or as a UNIX-domain socket path specified after the word “unix” and enclosed in colons:
proxy_pass http://unix:/tmp/backend.socket:/uri/;
See the documentation: http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
I can also happen when you mistakenly put a = character in, trying to assign a value that way.
proxy_pass = http://other_node;
The same thing might happen to you with for example auth_basic = "restricted site".
Just remove the =.
I am trying to match specific parts of $uri using proxy_cache_key in nginx 1.12 for these requests where different file names (with no arguments) actually have the same file contents. :(
2018-04-02T21:25:37+00:00 MISS /bein1/1/media_w1751476191_2333.ts
2018-04-02T21:25:37+00:00 MISS /bein1/1/media_w2137194067_2333.ts
2018-04-02T21:25:38+00:00 MISS /bein1/1/media_w1023873154_2333.ts
I have tried the following:
location ~ ^/bein1/(.*)/media_(.*)_(.*).(ts)$ {
proxy_cache_valid 200 302 60s;
proxy_cache_key "/bein1/$1/media_$3.ts";
proxy_pass http://origin;
add_header "X-Hls-Cache-Status" "Cached TS";
}
But I still could not match the specified URI.
Can any one can help me please?
Try with something like this:
location ~ ^/bein1/([-_a-zA-Z0-9/]+)/media_([-_a-zA-Z0-9/]+)_([0-9]+).ts$ {
proxy_cache_key "/bein1/$1/media_$3.ts";
proxy_cache_valid 200 302 60s;
proxy_pass http://origin;
}