I want to maintain one single URL for all pages and I'm using the index index.html directive to have a page at /writing/index.html be displayed when someone visits /writing/. However, with this index directive /writing/index.html is still a valid URL that nginx serves a page at.
I want /writing/index.html to 301 redirect to /writing/, and so forth for the root path (/index.html -> /) and all other URLS too (/foo/bar/index.html -> /foo/bar/).
I want to use a regular expression that only matches the /index.html ending such as: ^(.*/)index\.html$
But if if I add
rewrite "^(.*)/index\.html$" $1 last;
to my nginx conf I'm seeing /writing/index.html 301 redirect to /writing/ which is good but I also see /writing/ 301 redirect to /writing/ in an infinite loop.
So my question is why does that above rewrite regex match /writing/ when it does not end in index.html? Is it because of the internal index directive in the nginx conf?
I've seen other one off solutions on StackOverflow for redirecting a single path, but not a solution that does it in a clean/generic way like this.
Below is my current nginx.conf
server {
listen 80;
server_name example.com *.example.com;
charset utf-8;
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
rewrite "^(.*/)index\.html$" $1 permanent;
location / {
root /srv/www/example.com/;
index index.html;
}
error_page 404 /404/;
}
So the solution to this problem was to have the rewrite inside a location that checks against the actual request's $request_uri which avoids internal re-routing with the index directive.
Pretty much use this instead:
if ($request_uri ~ "^(.*/)index\.html$") {
rewrite "^(.*/)index\.html$" $1 permanent;
}
I believe a location block with a return would be more efficient and easier to read:
location ~ ^(.*/)index\.html$ {
return 301 $1;
}
Related
I'm working on an nginx reverse proxy container image to proxy frontend files from s3, and Im trying to access these files from a specific folder location, instead of just the base path of the s3 bucket. As of yet I can only serve up the index.html which I'm using a rewrite for, but I'm getting a 403 on the js and css files.
I've tried including mime.types
include mime.types;
I've tried adding an s3 folder bucket param
proxy_pass http://YOURBUCKET.s3-website.eu-central-1.amazonaws.com/$1;
And then various regex options
Here is my nginx conf file
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/ssl/nginx-server.crt;
ssl_certificate_key /etc/ssl/nginx-server.key;
server_name timemachine.com;
sendfile on;
default_type application/octet-stream;
resolver 8.8.8.8;
server_tokens off;
location ~ ^/app1/(.*) {
set $s3_bucket_endpoint "timemachineapp.s3-us-east-1.amazonaws.com";
proxy_http_version 1.1;
proxy_buffering off;
proxy_ignore_headers "Set-Cookie";
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-meta-s3cmd-attrs;
proxy_hide_header Set-Cookie;
proxy_set_header Authorization "";
proxy_intercept_errors on;
rewrite ^/app1/?$ /dev/app1/index.html; <-- I can only access index.html and the other js and css files throw a 403
proxy_pass https://timemachineapp.s3-us-east-1.amazonaws.com;
break;
}
}
As you can see, I'm trying to make this so that the user goes to https://timemachine/app1 that this will go to the homepage and load all the css and js files. Again, what im getting is a 403 and sometimes a 404. Insight appreciated.
From the question it looks like
There's a constant request-url prefix /app1/
There's a constant proxied-url prefix /dev/app1/
On that basis...
First, enable the debug log
There will already be an error_log directive in the nginx config, locate it and temporarily change to debug:
error_log /dev/stderr debug;
This will allow you to see how these requests are being processed.
Try naive-simple first
Let's use this config (other header directives omitted for brevity):
location = /app1 { # redirect for consistency
return 301 /app1/;
}
location = /app1/ { # explicitly handle the 'index' request
proxy_pass https://example.com/dev/app1/index.html;
}
location /app1/ {
proxy_pass https://example.com/dev/;
}
And emit a request to it:
$ ~ curl -I http://test-nginx/app1/some/path/some-file.txt
HTTP/1.1 403 Forbidden
...
Note that S3 returns a 403 for requests that don't exist, nginx is just proxying that response here.
Let's look in the logs to see what happened:
2023/01/28 14:46:10 [debug] 15#0: *1 test location: "/"
2023/01/28 14:46:10 [debug] 15#0: *1 test location: "app1/"
2023/01/28 14:46:10 [debug] 15#0: *1 using configuration "/app1/"
...
"HEAD /dev/some/path/some-file.txt HTTP/1.0
Host: example.com
Connection: close
User-Agent: curl/7.79.1
Accept: */*
"
So our request became https://example.com/dev/some/path/some-file.txt
That's because the way proxy_pass works is:
If the proxy_pass directive is specified with a URI, then when a request is passed to the server, the part of a normalized request URI matching the location is replaced by a URI specified in the directive
Meaning:
Nginx receives:
/app1/some/path/some-file.txt
^ the normalized path starts here
Proxied-upstream receives:
/dev/some/path/some-file.txt
^ and was appended to proxy-pass URI
I point this out as renaming/moving things on s3 may lead to a simpler nginx setup.
Rewrite all paths, not specific requests
Modifying the config above like so:
location = /app1 { # redirect for consistency
return 301 /app1/;
}
location = /app1/ { # explicitly handle the 'index' request
proxy_pass https://example.com/dev/app1/index.html;
}
location /app1/ {
rewrite ^/(.*) /dev/$1 break; # prepend with /dev/
# rewrite ^/app1/(.*) /dev/app1/$1 break; # OR this
proxy_pass https://example.com/; # no path here
}
And trying that test-request again yields the following logs:
"HEAD /dev/app1/some/path/some-file.txt HTTP/1.0
Host: example.com
Connection: close
User-Agent: curl/7.79.1
Accept: */*
"
In this way the index request works, but also arbitrary paths - and there's no need to modify this config to handle each individual url requested.
Alright so found a solution. Unless I'm missing something, this is easier than thought. For my use case, all I had to do was simply add multiple writes with those css files passed in (I'm sure there's a simpler way to just specify any .css file extension regardless of the naming of the file. Anyway, here is solution at the moment:
server {
listen 80;
listen 443 ssl;
ssl_certificate /etc/ssl/nginx-server.crt;
ssl_certificate_key /etc/ssl/nginx-server.key;
server_name timemachine.com;
sendfile on;
default_type application/octet-stream;
resolver 8.8.8.8;
server_tokens off;
location ~ ^/app1/(.*) {
set $s3_bucket_endpoint "timemachineapp.s3-us-east-1.amazonaws.com";
proxy_http_version 1.1;
proxy_buffering off;
proxy_ignore_headers "Set-Cookie";
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-meta-s3cmd-attrs;
proxy_hide_header Set-Cookie;
proxy_set_header Authorization "";
proxy_intercept_errors on;
rewrite ^/app1/?$ /dev/app1/index.html;
rewrite ^/app1/?$ /dev/app1/cssfile.css; <- and keep adding, if needed
proxy_pass https://timemachineapp.s3-us-east-1.amazonaws.com;
break;
}
}
I don't have much experience with regex, and less in an Nginx context, so this one is a difficult one for me.
What I want is that when I'm on a especific page (not subfolder) the server takes images from a different root.
My current configuration to handle images in all the pages of my website right now is this one.
location ~* \.(png|jpg|jpeg|gif) {
root /usr/share/nginx/nginxTestPHP/media;
}
And I'd like that when I'm in the page example.com/My-Profile the server handles the images from /usr/share/nginx/nginxTestPHP/media/uploads, my idea is something like this, if it makes any sense.
location ~* /My-Profile/*\.(png|jpg|jpeg|gif) {
root /usr/share/nginx/nginxTestPHP/media/uploads;
}
Obviously this one doesn't work otherwise I wouldn't be here asking this, so for the regex pros out there, what would be the solution for this one ?
Also how would I apply this in 2 different pages in the same regex, something like (My-Profile|Configuration) is my idea.
My Nginx configuration
server {
listen 81;
listen [::]:81;
server_name IP;
client_max_body_size 4M;
charset UTF-8;
include /etc/nginx/mime.types;
error_log /var/log/nginx/error_log warn;
access_log /var/log/nginx/access_log main;
rewrite ^(/.*)\.html(\?.*)?$ $1$2 permanent;
rewrite ^/(.*)/$ /$1 permanent;
root /usr/share/nginx/nginxTestPHP/PHP;
index index.html index.htm Inicio.php Index.php;
location / {
try_files $uri/index.html $uri.html $uri/ #extensionless-php;
}
location #extensionless-php {
rewrite ^(.*)$ $1.php last;
}
# pass the PHP scripts to FastCGI
location ~ \.php$ {
try_files $uri =404;
fastcgi_intercept_errors on;
fastcgi_pass app:9000;
fastcgi_index Inicio.php;
fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
include fastcgi_params;
}
location ^~ /vendor/ {
deny all;
return 403;
}
# deny access to .htaccess files
location ~ /\.ht {
deny all;
}
location ^~ /en/ {
try_files $uri/.php $uri.php $uri/ #rewrite;
fastcgi_intercept_errors on;
fastcgi_pass app:9000;
fastcgi_index Index.php;
fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
include fastcgi_params;
}
location ^~ /Blog/ {
try_files $uri/.php $uri.php $uri/ #rewrite;
fastcgi_intercept_errors on;
fastcgi_pass app:9000;
fastcgi_index Index.php;
fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
include fastcgi_params;
}
location #rewrite {
rewrite ^/(en|Blog)(/.*\.(png|jpg|svg|ico))$ $2 last;
return 404;
}
location ~ \.css {
root /usr/share/nginx/nginxTestPHP/CSS;
default_type text/css;
add_header Content-Type text/css;
}
location ~ \.js {
root /usr/share/nginx/nginxTestPHP/JavaScript;
default_type text/javascript;
add_header Content-Type application/x-javascript;
}
location ~ \.txt {
root /usr/share/nginx/nginxTestPHP/PHP;
}
location ~* ^/Mi-Perfil/([^/]+\.(png|jpg|jpeg|gif))$ {
alias /usr/share/nginx/nginxTestPHP/media/uploads/$1;
}
location ~* \.(png|jpg|svg|ico|jpeg|gif) {
root /usr/share/nginx/nginxTestPHP/media;
}
error_page 405 =200 $uri;
# redirect server error pages to the static page /50x.html
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/nginxTestPHP/PHP;
try_files $uri /Inicio.php;
}
}
Your regex is broken, it is a dot (.) in PCRE patterns that acts as a wildcard, not an asterisk one. If you want to match any non-zero number of characters except a / one, you can use [^/]+ regex pattern:
location ~* /My-Profile/[^/]+\.(png|jpg|jpeg|gif)$ { ... }
or to ensure that an URI started exactly with /My-Profile/ add a ^ anchor (matches start of the string):
location ~* ^/My-Profile/[^/]+\.(png|jpg|jpeg|gif)$ { ... }
Check the difference between root and alias nginx directives. With your configuration nginx will search the files inside the /usr/share/nginx/nginxTestPHP/media/uploads/My-Profile folder. Use an alias directive instead:
location ~* ^/My-Profile/[^/]+\.(png|jpg|jpeg|gif)$ {
alias /usr/share/nginx/nginxTestPHP/media/uploads;
}
Update
I think I made a serious mistake. Since this is a regex matching location an alias directive argument should contain full path to image file with filename:
location ~* ^/My-Profile/([^/]+\.(png|jpg|jpeg|gif))$ {
alias /usr/share/nginx/nginxTestPHP/media/uploads/$1;
}
Little experience with Nginx here but the regex part looks relatively simple to fix - try using:
location ~* /My-Profile/.+\.(png|jpg|jpeg|gif)$ {
root /usr/share/nginx/nginxTestPHP/media/uploads;
}
Note the .+ after My-Profile/ and the end-of-string anchor ($) - I'm assuming you want to only match images, not a URI like stackoverflow.com/test.png/somepage.
From the little research I did, you may also want to look into nginx maps, especially if you're looking to do this for multiple profile pages all following the same URI pattern.
I'm trying to find a way to load an html that shows "Server Down" or something similar when I'm building my application.
Right now I every time I build my backend and frontend there are couple of seconds when I see the below template if I go to the site:
I will like to customize that page or show a different template saying : Server Down at the moment or Building
My nginx.conf is the following. Where should I put the location for a 403.html template to load ?: This needs to be outside of the build folder I think, since the 403 page appears while it's building.
server { # [ASK]: is this what's causing the problem ?
root /home/smiling/smiling-frontend/website/build; ## development build
index index.html;
server_name frontend.develop.smiling.be; ## development domain
charset utf-8;
gzip on;
gzip_vary on;
gzip_disable "msie6";
gzip_comp_level 6;
gzip_min_length 1100;
gzip_buffers 16 8k;
gzip_proxied any;
gzip_types
text/plain
text/css
text/js
text/xml
text/javascript
application/javascript
application/x-javascript
application/json
application/xml
application/xml+rss;
location / {
try_files $uri $uri/ /index.html;
}
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc|svg|woff|woff2|ttf)\$ {
expires 1M;
access_log off;
add_header Cache-Control "public";
}
location ~* \.(?:css|js)\$ {
expires 7d;
access_log off;
add_header Cache-Control "public";
}
location ~ /\.well-known {
allow all;
}
location ~ /\.ht {
deny all;
}
add_header Access-Control-Allow-Origin '*/';
add_header Access-Control-Allow-Headers 'origin, x-requested-with, content-type, accept, authorization';
add_header Access-Control-Allow-Methods 'GET, POST';
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/backend.develop.smiling.be/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/backend.develop.smiling.be/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
Your last sentence is kind of inconsistent... You like not to do something, but want to do the same thing nevertheless.
You could define your own page or string to be served on errors:
error_page 403 /403.html;
location = /403.html {
internal;
return 403 "Server Down at the moment"; # <- this could also contain an HTML string if your nginx defaults to text/html as content type.
}
You could also put a 403.html file in your root folder and skip the location part in order to serve a full HTML file here.
I'm trying to change my website over from the default /year/month/day/post_title permalinks, to a simple /post_title/ link, however, when changing it, all of my older links are broken. I've read a site on how to do it on Apache with .htaccess, but need some help on figuring out how to make it work with nginx's location instead of mod_rewrite.
This is the site that details how to do it on Apache http://www.rickbeckman.org/how-to-update-your-wordpress-permalinks-without-causing-link-rot/
And I tried using this htaccess to nginx converter http://winginx.com/htaccess however, the regex is probably causing a problem and I get this error when starting nginx
[emerg]: unknown directive "4}/[0-9]" in /usr/local/nginx/sites-enabled/website.com:19
And this is my configuration file
server {
listen 80;
server_name website.com;
rewrite ^/(.*) http://www.website.com/$1 permanent;
}
server {
listen 80;
server_name www.website.com;
error_log /home/user/public_html/website.com/log/error.log;
client_max_body_size 10M;
client_body_buffer_size 128k;
location / {
root /home/user/public_html/website.com/public/;
index index.php index.html;
try_files $uri $uri/ /index.php;
}
location ~ ^/[0-9]{4}/[0-9]{2}/[0-9]{2}/([a-z0-9\-/]+) {
rewrite ^(.*)$ http://website.com/$1 permanent;
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
location ~ \.php$
{
fastcgi_read_timeout 120;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include /usr/local/nginx/conf/fastcgi_params;
fastcgi_param SCRIPT_FILENAME /home/user/public_html/website.com/public/$fastcgi_script_name;
}
}
Would anyone know how to fix it? Thanks.
the fix is easy:
location ~ ^/[0-9]{4}/[0-9]{2}/[0-9]{2}/([a-z0-9\-/]+) {
rewrite ^(.*)$ http://website.com/$1 permanent;
}
the problem with the above is that the rewrite resets the backreferences so the $1 now matches the entire url from your rewrite match, not the $1 from the location match.
the following should work:
location ~ ^/[0-9]{4}/[0-9]{2}/[0-9]{2}/([a-z0-9\-/]+) {
set $post_title $1;
rewrite ^(.*)$ http://website.com/$post_title permanent;
}
(alternatively match the regex in the rewrite rule again)
Fixed it by adding quotes around the regex.
rewrite "^/[0-9]{4}/[0-9]{2}/[0-9]{2}/([a-z0-9\-/]+)" http://www.website.com/$1;
First of all, I have tried to search for similar questions, but the solutions to those questions were specific lines of code, that I couldn't customise to fit my needs.
I have a Codeigniter installation, and I'm trying to migrate from Apache to nginx. However, in Apache the .htaccess was pretty simple: it would take a whitelist, and rewrite everything else to index.php.
RewriteEngine on
RewriteCond $1 !^(index\.php|css|images|core|uploads|js|robots\.txt|favicon\.ico)
RewriteRule ^(.*)$ /index.php/$1 [L]
However in nginx, I have tried out the if and try_files directives, as well as messing around with locations, to no avail. I'm still new to how nginx reads the server config, and the tutorials online were somewhat confusing to follow through.
Additionally, the index.php will not be in the web root, but in a subdirectory server.
Because of this, I also need to make sure even URI requests beginning with /server do not go to the directory, but to index.php
This is my nginx virtual host configuration so far:
server {
listen 80;
server_name example.com;
root /home/example/public_html;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
location / {
index index.htm index.html index.php;
}
location ~ \.php$ {
fastcgi_pass unix:/var/run/php-fpm/example.sock;
include fastcgi_params;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
}
location ~* ^.*(/|\..*) {
try_files $uri $uri/ /server/index.php;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
Which helps to redirect requests to index.php, but doesn't have a whitelist. I would appreciate it if anyone could generate a working example with a brief explanation of what each part does.
I would use the $uri variable and if in a location block to achieve this.
location / {
if ( $uri !~ ^/(index\.php|css|images|core|uploads|js|robots\.txt|favicon\.ico) ) {
rewrite ^ /server/index.php last;
}
}
Also, as for the pathinfo security problems, (discussion) it's a good practice to add
try_files $uri =403;
fastcgi_split_path_info ^(.+.php)(.*)$;
to the location ~ \.php$ block.