Nginx reverse proxy to django application, images styles are not showing - django

Hello i have a complete django app which I'll be hosting on aws ec2 with gunicorn and nginx. But when I run the app with gunicorn or python3 my app runs and shows all images with the css and styles. But whenever I try to serve my django app with nginx by proxy_pass the image or style or anything on static folder could not be accessed. my error log shows permission issue. But I have set the app with user and group both nginx. but nothing is showing.
nginx conf on nginx.conf
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 4096;
include /etc/nginx/mime.types;
default_type application/octet-stream;
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
server_name example.com;
root /home/ec2-user/sites/softopark-django-cms;
access_log /var/log/nginx/example.com.access.log;
error_log /var/log/nginx/example.com.error.log;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /usr/share/nginx/html/softopark-django-cms/static/;
}
location /media/ {
root /usr/share/nginx/html/softopark-django-cms/media/;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://0.0.0.0:8000/;
}
}
my gunicorn service file
[Unit]
Description=test
After=network.target
[Service]
User=nginx
Group=nginx
WorkingDirectory=/usr/share/nginx/html/softopark-django-cms
Environment="/home/ec2-user/Env/cms5-XkwMz5k7/bin"
ExecStart=/home/ec2-user/Env/cms5-XkwMz5k7/bin/gunicorn --workers 3 --bind 0.0.0.0:8000 cms_project.wsgi
[Install]
WantedBy=multi-user.target
can anyone help me with this?
I need to host my django app on aws but with nginx I'm not being able to do it. Please help.

Well, I didn't find any specific answer to this but I fixed it after running collectstatic command.
One thing, I missed that whenever we create any Django projects and at the time of deploying to another place we need to run python manage.py collectstatic so that our app can detect the static files and update their path. So I updated the codebase and deployed it on the server the but didn't used collectstatic, so i was unable see those images.
The python manage.py collectstatic did the work.

Related

DJango + nginx + gunicorn: how to start caching and which way is more standard?

I deployed my Django project using Gunicorn, but now can't use Nginx caching.
How to start caching on a project that use Gunicorn and which caching method is standard for Django.
I try to use Ngnix caching but it won't work.
Here is an example of my Ngnix conf and Caching Conf
Ngnix Conf
/etc/nginx/sites-available/my-project-config
upstream daphne_server {
server unix:/var/www/my-project-config/env/run/daphne.sock fail_timeout=0;
}
upstream gunicorn_server {
server unix:/var/www/my-project-config/env/run/gunicorn.sock fail_timeout=0;
}
server {
listen 8000;
server_name my_server_ip_address or domain name;
client_max_body_size 100M;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/ubuntu/my_project_dir/public_html;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
location /ws/ {
proxy_pass http://localhost:8001;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
}
}
Cache Config
/etc/nginx/sites-available/my-project-cache-config
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=custom_cache:10m inactive=60m;
upstream origin_server {
server 127.0.0.1:8000;
}
server {
listen 80;
server_name _;
location / {
include proxy_params;
proxy_pass http://origin_server;
proxy_cache custom_cache;
proxy_cache_valid any 10m;
add_header X-Proxy-Cache $upstream_cache_status;
}
}
Gunicorn Socket
/etc/systemd/system/gunicorn.socket
[Unit]
Description=gunicorn socket
[Socket]
ListenStream=/run/gunicorn.sock
[Install]
WantedBy=sockets.target
Gunicorn Service
sudo nano /etc/systemd/system/gunicorn.service
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=ubuntu
Group=www-data
WorkingDirectory=/home/ubuntu/my_project_dir
ExecStart=/home/ubuntu/my_project_dir/venv/bin/gunicorn \
--access-logfile - \
--workers 3 \
--bind unix:/run/gunicorn.sock \
my_project.wsgi:application
[Install]
WantedBy=multi-user.target
I am also using gunicorn to host a django webserver on Nginx, and I did not run into this problem. I don't believe it should matter that you are hosting with gunicorn. As long as you configure caching for the directory that contains all of your static files it should work.
Here's an example where the static files are cached for a year:
location /static {
root [location of /static folder];
expires 1y;
access_log off;
add_header Cache-Control "public";
}
If this doesn't solve the problem please add your cache settings for Nginx to your question
If you are trying to cache django views or templates, look into django-cahceops or a similar package

(Django REST Framework) Basic authentication doesn't work when app is served through NGINX

I'm building an API and I finally got it working to serve it through Gunicorn and NGINX. Nginx proxies incoming requests to a socket binded to Gunicorn. The problem is this:
When I try to access the API directly by running the 'gunicorn command or by using the builtin 'runserver' command from Django, and having configured Django REST Framework's BasicAuthentication as the default authentication class in the settings.py-file, everything works fine. Each time I try to access an endpoint, it asks me for a valid username/password-combo, just like you would expect.
However, when I try to access the API through NGINX, which has a proxy_pass configured to the unix socket which Gunicorn is bound to, BasicAuthentication doesn't work anymore. All requests are granted, without providing a username and password.
I know basic authentication should be avoided, but it's a requirement for a project I'm working on. Does anyone know why this happens and how to solve this?
settings.py:
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.BasicAuthentication',
),
}
nginx.conf:
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
upstream *** {
server unix:/***/***/***/***/***/***/***.sock
fail_timeout=0;
}
server {
server_name <server-name>;
satisfy all;
allow <IP-address>
deny all;
# location = /favicon.ico {access_log off; log_not_found off;}
location /static/ {
autoindex on;
alias ../static/;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://<upstream>;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/***/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/***/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
Okay so I managed to solve it myself. I don't know what the cause was because I also had the same problem before using Gunicorn, but restarting it solved it somehow.

Nginx as reverse proxy django gunicorn returning 400 bad request

I'm trying to use nginx as a reverse proxy to receive incoming calls, then, depending on the server_name, redirect those calls to different computers (hosts), running nginx Django and Gunicorn. So far, I've tried different configurations for the conf file on the host, but none of them are working. Is there anything wrong with my conf files?
This is the nginx.conf in 192.168.0.13 that will function as a reverse proxy:
server {
listen 80;
server_name www.coding.test;
location / {
proxy_pass http://192.168.0.8:80;
proxy_redirect off;
# app1 reverse proxy follow
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
This is the nginx.conf in 192.168.0.8 that is intended to run the django app:
upstream django {
server unix:///home/pi/coding-in-dfw/mysocket.sock fail_timeout=0;
}
server {
listen 80 default_server;
server_name www.coding.test
client_max_body_size 4G;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
location /static/ {
alias /home/pi/coding-in-dfw/static/;
}
location /media/ {
alias /home/pi/coding-in-dfw/media/;
}
# Finally, send all non-media requests to the Django server.
location / {
uwsgi_pass django;
include /etc/nginx/uwsgi_params; # the uwsgi_params file you installed
}
location /.well-known {
alias /home/pi/coding-in-dfw/.well-known;
}
}
Finally this is the way I'm running gunicorn:
gunicorn --workers 5 --bind unix:///home/pi/coding-in-dfw/mysocket.sock codingindfw.wsgi:application && sudo service nginx restart
Any help is appreciated.

Where is robots.txt coming from (Nginx + Gunicorn + Django)?

I am running a server with Nginx + Gunicorn + Django.
I have configured Django to show a robots.txt file, and this works just fine when I test Django on its own, without Nginx+Gunicorn.
However, when I run the whole stack, /robots.txt suddenly gets replaced with a different file that I didn't write.
Why is this happening? I never wrote anything to tell Nginx or Gunicorn to do such a thing. Is there a default somewhere that I need to overwrite or deactivate?
The folder /etc/nginx/sites-enabled contains only my own sites file, and it doesn't mention /robots explicitly. It just has a location block targeting / which gets routed to Gunicorn+Django, and for every site except robots.txt this works just fine.
My Gunicorn config:
[Unit]
Description=gunicorn daemon
After=network.target
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=root
Group=www-data
WorkingDirectory=/root/lod/collab/collab
ExecStart=/root/lod/CollabVenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/root/lod/collab.sock collab.wsgi:application
[Install]
WantedBy=multi-user.target
My Nginx config:
server {
# if no Host match, close the connection to prevent host spoofing
listen 80 default_server;
listen [::]:80 default_server;
return 444;
}
server {
# redirect any http requests to https
listen 80;
listen [::]:80;
server_name example.com www.example.com localhost;
return 301 https://example.com$request_uri;
}
server {
# this is the main server application, serving example.com with https
listen 443;
listen [::]:443;
server_name example.com;
# SSL certificate
ssl on;
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
# allow large request bodies, which may be necessary for some JSON commands
client_max_body_size 4G;
# ignore a missing favicon
location = /favicon.ico { access_log off; log_not_found off; }
# Django media
location /media {
alias /root/lod/CollabData/media_public; # your Django project's media files
}
# Django static
location /static {
autoindex on;
alias /root/lod/collab/collab/static_processed; # your Django project's static files
}
# send all non-media requests to the Django server
location / {
# necessary for SSL
# note:
# I have no idea what most of these do. I just followed tutorials on the internet.
# Feel free to improve this if you know what you are doing.
proxy_redirect off;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Host $host:443;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-Port 443;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
# the connection to Gunicorn
proxy_pass http://unix:/root/lod/collab.sock;
}
}
EDIT:
It just started working correctly, even though I made no changes since the last time I opened the site. Maybe some background process or cache took a while to update? I'm leaving this question open in case someone can explain what caused this, so that other people won't have to spend as much time with this problem as I did.

How to listen nginx port 80 to django server?

Why nginx run default page ? how to listen my django server ?
First inside the sites-availabe folder i created example.com file then i
[root#instance-4 sites-available]# ls -al /etc/nginx/sites-enabled/example.com
lrwxrwxrwx. 1 root root 21 Dec 22 11:03 /etc/nginx/sites-enabled/example.com -> example.com
/etc/nginx/sites-available/example.com
server {
listen 80;
server_name example.com;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
Then when i run gunicorn example.wsgi in my app folder and later i visited the example.com but you know what i am still getting nginx default page.
What i am missing here ?
Updated :
Now this time i created example.com file in my Django root folder then after Symlink
[root#instance-4 Staging]# ln -s example.com /etc/nginx/sites-enabled/
after the nginx restart still same ...
Updated 2 :
nginx.conf file
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
include /etc/nginx/sites-enabled/*;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
Check for a default in /etc/nginx/site-enabled/ and remove it if it's there. Then reload or restart your nginx server.
You can also check gunicorn is serving requests by visiting example.com:8000.
It's worthwhile noting that you'll probably also want nginx to be serving your static files so put in a /static/ block:
location /static/ {
alias /path/to/your/app/static/;
if ($query_string) {
# If using GET params to control versions, set to max expiry.
expires max;
}
access_log off;
}
From what i remember of nginx, there is 2 places where you can find the index.html of nginx, try to do a "find / -name index.html" you will prolly find the 2nd .html i am talking about, and regarding the path u should be able to fix this