Nginx caching of uwsgi served static files - django

The django project is deployed using uwsgi as application server, it also serves static files from a specified directory(as shown in the below command) and nginx is used as reverse proxy server. This is deployed using docker.
The uwsgi command to run the server is as follows:
uwsgi -b 65535 --socket :4000 --workers 100 --cpu-affinity 1 --module wui.wsgi --py-autoreload 1 --static-map /static=/project/static;
The application is working fine at this point. I would like to cache the static files into nginx server. So i have referred blog https://www.nginx.com/blog/maximizing-python-performance-with-nginx-parti-web-serving-and-caching and i have included following configuration in my nginx.conf :
location ~* .(ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|css|rss|atom|js|jpg
|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid
|midi|wav|bmp|rtf)$ {
expires max;
log_not_found off;
access_log off;
}
After adding this into my Nginx conf, the Nginx server container exits with the following error:
[emerg] 1#1: invalid number of arguments in "location" directive in /etc/nginx/nginx.conf:43
Is this how uwsgi served static files can be cached into nginx? If yes please suggest me what is gone wrong here.
My complete nginx.conf is as follows:
events {
worker_connections 1024; ## Default: 1024
}
http {
include conf/mime.types;
# the upstream component nginx needs to connect to
upstream uwsgi {
server backend:4000; # for a web port socket (we'll use this first)
}
# configuration of the server
server {
# the port your site will be served on
listen 8443 ssl http2 default_server;
# the domain name it will serve for
server_name _; # substitute your machine's IP address or FQDN
charset utf-8;
ssl_certificate /secrets/server.crt;
ssl_certificate_key /secrets/server.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
add_header Strict-Transport-Security "max-age=31536000" always;
# Redirect HTTP to HTTPS
error_page 497 https://$http_host$request_uri;
# max upload size
client_max_body_size 75M; # adjust to taste
uwsgi_read_timeout 600s;
# Finally, send all non-media requests to the Django server.
location / {
uwsgi_pass uwsgi;
include /config/uwsgi_params; # the uwsgi_params file you installed
}
location ~* .(ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|css|rss|atom|js|jpg
|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid
|midi|wav|bmp|rtf)$ {
expires max;
log_not_found off;
access_log off;
}
}
}
Nginx version: 1.16

The problem with your config is that the location block has newlines in the list of filenames. I tried nginx -t -c <filename> with a modified version of your location block:
location ~* .(ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|css|rss|atom|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid|midi|wav|bmp|rtf)$ {
expires max;
log_not_found off;
access_log off;
}
... and this passes the test!

Related

Hosting two different django project in the same droplet with different subdomains (NGINX, Gunicorn, ubuntu)

As the title says I want to host two different django project in the same droplet (NGINX, Gunicorn, ubuntu) with different subdomains. One will be our main site example.com. which is up and running and working perfectly. We want to host the staging site staging.example.com in the same droplet.
We have created new sockets and service files for the staging site and activated and enabled them but the issue is nginx still points to the files in main domain directory rather than the staging directory and hence we get this error below even though these domains have been added in the allowed hosts of settings.py of the staging site
DisallowedHost at /
Invalid HTTP_HOST header: 'staging.example.com'. You may need to add 'staging.example.com' to ALLOWED_HOSTS
Here is our staging.guinicorn.service file
[Unit]
Description=staging.gunicorn daemon
Requires=staging.gunicorn.socket
After=network.target
[Service]
User=admin
Group=www-data
WorkingDirectory=/home/admin/example1staging
ExecStart=/home/admin/example1staging/venv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/staging.gunicorn.sock djangoproject.wsgi:application
[Install]
WantedBy=multi-user.target
Here is our staging.guicorn.socket file
[Unit]
Description=staging.gunicorn socket
[Socket]
ListenStream=/run/staging.gunicorn.sock
[Install]
WantedBy=sockets.target
Lastly here is our nginx config
server {
listen 80;
listen [::]:80;
server_name example.com www.example.com;
return 302 https://$server_name$request_uri;
}
server {
# SSL configuration
listen 443 ssl http2;
listen [::]:443 ssl http2;
ssl_certificate /etc/ssl/cert.pem;
ssl_certificate_key /etc/ssl/key.pem;
ssl_client_certificate /etc/ssl/cloudflare.crt;
ssl_verify_client on;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/admin/example1;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
}
server {
listen 80;
listen [::]:80;
server_name staging.example.com www.staging.example.com;
return 302 https://$server_name$request_uri;
}
server {
# SSL configuration
listen 443 ssl http2;
listen [::]:443 ssl http2;
ssl_certificate /etc/ssl/cert.pem;
ssl_certificate_key /etc/ssl/key.pem;
ssl_client_certificate /etc/ssl/cloudflare.crt;
ssl_verify_client on;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/admin/example1staging;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/staging.gunicorn.sock;
}
}
Some help here would be extremely welcome.

HTTPS protocol not working with django and nginx on ec2 instance

I have been trying to auto transfer all requests to https protocol using nginx in django in ec2 instance but i am unable to do so.. here is my nginx file..
please suggest me the problem.
nginx file
server{
listen 443 ssl;
server_name www.priyamarya.com;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/ubuntu/project/aryapriyam/;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/ubuntu/project/aryapriyam/project.sock;
}
}
server{
listen 80;
server_name priyamarya.com;
return 301 https://www.priyamarya.com;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/ubuntu/project/aryapriyam/;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/ubuntu/project/aryapriyam/project.sock;
}
}
i have also added this in settings.py
settings.py
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
SECURE_SSL_REDIRECT =True
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True
gunicorn.service
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=ubuntu
Group=www-data
WorkingDirectory=/home/ubuntu/project/aryapriyam
ExecStart=/home/ubuntu/project/venv/bin/gunicorn --access-logfile - --
workers 3 --chdir /home/ubuntu/project/aryapriyam/ --bind
unix:/home/ubuntu/project/aryapriyam/project.sock
project.wsgi:application
[Install]
WantedBy=multi-user.target
i have also set my hostedzone A type record set to the alias provided by elb load balancer.
i have tried a lot of things like returning the https://sitename , and creating a different server block for both protocols but than it starts a loop in between requests. this is why i am posting the code from which i initially started. i have searched a lot but nothing is helping in regards of nginx and django both, please help.. i want all my forms request also to go through https only.
You need to add other server block for ssl and use following config for ssl
This config also redirects http request to https (i.e ssl port 443)
server {
listen 80;
server_name testing.com;
return 301 https://testing.com;
location = /favicon.ico { access_log off; log_not_found off; }
location / {
include proxy_params;
proxy_pass http://unix:/home/ubuntu/sample_project/sample_project.sock;
}
}
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name testing.com;
ssl on;
ssl_certificate /etc/nginx/ssl/nginx.crt;
ssl_certificate_key /etc/nginx/ssl/nginx.key;
location /static/ {
root /home/ubuntu/sample_project;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/ubuntu/sample_project/sample_project.sock;
}
}

Nginx not serving files from /var/www/html after deploying Django app

I have a Django application stored at /var/www/w_gm .
Now I have used Nginx + Gunicorn to deploy it.
My default conf file at
root#dev:/etc/nginx/sites-enabled# ls
default w_gm
Default conf file :
server {
listen 80 default_server;
listen [::]:80 ipv6only=on default_server;
# SSL configuration
#
# listen 443 ssl default_server;
# listen [::]:443 ssl default_server;
#
# Note: You should disable gzip for SSL traffic.
# See: https://bugs.debian.org/773332
#
# Read up on ssl_ciphers to ensure a secure configuration.
# See: https://bugs.debian.org/765782
#
# Self signed certs generated by the ssl-cert package
# Don't use them in a production server!
#
# include snippets/snakeoil.conf;
root /var/www/html;
# Add index.php to the list if you are using PHP
index index.php index.html index.htm index.nginx-debian.html;
server_name _;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
location ~ \.php$ {
include snippets/fastcgi-php.conf;
# With php7.0-cgi alone:
#fastcgi_pass 127.0.0.1:9000;
# With php7.0-fpm:
fastcgi_pass unix:/run/php/php7.0-fpm.sock;
}
w_gm conf file :
server {
listen 80;
server_name 119.00.00.100;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /var/www/w_gm/w_gm/;
}
location / {
include proxy_params;
proxy_pass http://unix:/var/www/w_gm/w_gm.sock;
}
}
Now the issue is that, when am tying in my IP address i.e, it's redirecting me to the Django app and it's working perfectly fine. But my other files which are in var/www/html doesn't get served i.e if I have <ip.addr>/work1 , it gives me error.
Now if I edit the w_gm conf file and add a suffix, let's say server_name 119.00.00.100/abc; , which obviously is wrong, my /var/www/html files start working.
I need a solution where if I type in <ip.addr>/something, then it should redirect to Django app, else serve the files which are in var/www/html.
You currently have two servers, but it sounds like you only want one server.
<ip.addr>/something is ambiguous, so you need Nginx to look for files in one root and send requests to the proxy only if they are not found.
You would need to combine your two server blocks into something like this:
root /var/www/html;
location / {
try_files $uri $uri/ #proxy;
}
location ~ \.php$ {
...
}
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /var/www/w_gm/w_gm/;
}
location #proxy {
include proxy_params;
proxy_pass http://unix:/var/www/w_gm/w_gm.sock;
}
This assumes that the /static/ URI is unambiguous, and that no URIs ending with .php are sent to the proxy. See this docuemnt for more.

HTTP request 408 error(Client request timeout) over mobile network. Django + Gunicorn + Nginx

I have faced with strange issue. My API has endpoint on which i am using PATCH(tried also PUT and POST - same result) to send json with jsonarray(request body size typically is 40KB) from mobile phone. When i am doing this using Wi-Fi - all works fine. But if i am using mobile network, i am facing with SocketConnection Timeout(in Retrofit/Kotlin) with some phones(iOS works perfect).
From nginx access.log :
domain - - [21/Nov/2017:16:33:01 +0000] "PATCH /endpoint/ HTTP/1.1" 408 0 "-" "Dalvik/2.1.0 (Linux; U; Android 6.0.1; Nexus 5 Build/M4B30Z)"
Nginx config :
server {
listen port;
server_name domain;
client_max_body_size 32m;
proxy_connect_timeout 75s;
proxy_read_timeout 300s;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/user/apps/api;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/user/apps/api/api.sock;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/domain/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/domain.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
Gunicorn config :
respawn
setuid root
setgid www-data
chdir /home/user/apps/api
exec /home/user/virtualenvs/apienv/bin/gunicorn --workers 3 --timeout=90 --bind unix://api.sock api.wsgi:application
The problem was in network connection. We changed location of our clouds from Frankfurt to Amsterdam and now all works perfect

Nginx gunicorn redirect not working and empty files

I am setting up a production server with nginx and gunicorn. I used the nginx.conf from the gunicorn examples pages and modified it:
worker_processes 1;
user mypolls webapps;
# 'user nobody nobody;' for systems with 'nobody' as a group instead
pid /var/run/nginx.pid;
error_log /webapps/mypolls/logs/nginx.error.log;
events {
worker_connections 1024; # increase if you have lots of clients
accept_mutex off; # set to 'on' if nginx worker_processes > 1
# 'use epoll;' to enable for Linux 2.6+
# 'use kqueue;' to enable for FreeBSD, OSX
}
http {
# fallback in case we can't determine a type
default_type application/octet-stream;
access_log /webapps/mypolls/logs/nginx.access.log combined;
sendfile on;
upstream app_server {
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response
# for UNIX domain socket setups
server unix:/webapps/mypolls/run/gunicorn.sock fail_timeout=0;
# for a TCP configuration
# server 192.168.0.7:8000 fail_timeout=0;
}
server {
# if no Host match, close the connection to prevent host spoofing
listen 80 default_server;
return 444;
}
server {
# use 'listen 80 deferred;' for Linux
# use 'listen 80 deferred;' for Linux
# use 'listen 80 accept_filter=httpready;' for FreeBSD
listen 80;
client_max_body_size 4G;
# set the correct host(s) for your site
server_name 192.168.1.17;
keepalive_timeout 5;
# path for static files
root /webapps/mypolls/app/static/;
location / {
include /etc/nginx/mime.types;
# checks for static file, if not found proxy to app
try_files $uri #proxy_to_app;
}
location / {
include /etc/nginx/mime.types;
# checks for static file, if not found proxy to app
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# enable this if and only if you use HTTPS
# proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $http_host;
# we don't want nginx trying to do something clever with
# redirects, we set the Host: header above already.
proxy_redirect off;
proxy_pass http://app_server;
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /webapps/mypolls/app/static/;
}
}
}
For some reason the request for static files is still passed to gunicorn:
[2017-02-08 15:54:33 +0100] [2207] [DEBUG] GET /static/polls/style.css
The feels seem to be found but empty files are served. Is something wrong with the configuration?