nginx 404 on root/homepage with Django + Gunicorn - django

I'm migrating my website from one host to another. The migration has been mostly 99% successful except for the fact that whenever I navigate to the home page of the site, I get nginx's 404 page. Every other page, static & media file is rendered properly. I've wracked my brain trying to find a solution but I haven't found any, let alone someone having a similar issue (instead, others have a working home page but 404 on all others).
I have two different domains (one that my company owns, the other is owned by the city we're in). I've updated the DNS on the domain I own to ensure it's working with the new host and it is. When I navigate using the host (example.com) or the server's IP address, the website loads all pages correctly - except the homepage, which - again - returns nginx's 404. All other 404 errors display the 404 page I've set up via Django.
Whenever this issue crops up, the gunicorn error log adds a line stating that the service is 'Booting worker with pid: ...'
nginx.conf --> save for the paths and port, this conf is identical to the conf running on my current host
http {
client_max_body_size 30M;
open_file_cache max=1000 inactive=300s;
open_file_cache_valid 360s;
open_file_cache_min_uses 2;
open_file_cache_errors off;
upstream cgac_server {
server unix:/root/cgac/cinema_backend/cinema_backend.sock fail_timeout=0;
}
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
gzip on;
server {
listen 80 default_server;
location /static/ {
expires 365d;
root /root/cgac/cinema_backend/static/;
}
location /media/ {
expires 365d;
root /root/cgac/cinema_backend/media/;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://cgac_server;
break;
}
}
}
gunicorn.service --> gunicorn is setup differently from the active server but the parameters in this file are identical, and it's working.
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=root
Group=www-data
WorkingDirectory=/root/cgac/cinema_backend
ExecStart=/root/cgac-venv/bin/gunicorn --access-logfile /root/logs/gunicorn-access.log --error-logfile /root/logs/gunicorn-error.log --workers 3 --bind unix:/root/cgac/cinema_backend/cinema_backend.sock cinema_backend.wsgi:application
[Install]
WantedBy=multi-user.target
Django URL patterns --> also 100% identical to the working site
urlpatterns = patterns('',
url(r'^admin/', include(admin.site.urls)), # NOQA
url(r'^fobi/', include('fobi.urls.view')),
url(r'^fobi/', include('fobi.urls.edit')),
url(r'^captcha/', include('captcha.urls')),
url(r'^sitemap\.xml$', 'django.contrib.sitemaps.views.sitemap',
{'sitemaps': {'cmspages': CMSSitemap}}),
url(r'^select2/', include('django_select2.urls')),
url(r'^', include('cms.urls')),
)

many option are there to fix it
put your upstream over http {}
reload your daemon
restart your gunicorn.service
your cinema_backend must be /run/cinema_backend.sock(what is normal)
try to use different config like site-availabe

The issue had absolutely nothing to do with the nginx or gunicorn configuration. It turns out that an image in a carousel on the home page somehow got corrupted or something and it was driving Django crazy. I deleted the entry in the CMS which fixed the issue, then proceeded to reupload the file and it's all good.

Related

Nginx responds 404 not found on Django media URL in preprod, dev ok

I have a quite standard Django application with a Vuejs frontend.
I have different environments (preprod/dev) in which I have file upload/download features.
For files, everything works fine because they are returned through standard API views in attachment (Content-Disposition: attachment). When it comes to images though, like profile pictures, there is a problem.
In development (DEBUG=True), I have this :
from django.conf import settings
from django.conf.urls.static import static
from django.urls import include, path
from backend.applications.base.views.authentication_views import LoginAPIView, LogoutAPIView
urlpatterns = [
path("api/login", LoginAPIView.as_view()),
path("api/logout", LogoutAPIView.as_view()),
path("api/base/", include("backend.applications.base.urls")),
path("api/contact/", include("backend.applications.contact.urls")),
path("api/helpdesk/", include("backend.applications.helpdesk.urls")),
path("api/inventory/", include("backend.applications.inventory.urls")),
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) # For serving media files when DEBUG=True
and images are correctly served (no nginx in dev mode, just frontend and backend dev servers django's runserver).
My preprod however, is made of a nginx container which serves my built vuejs frontend, and a backend container which contains my Django (DEBUG=False) application (which runs with gunicorn this time, like this : gunicorn backend.wsgi:application --bind 0.0.0.0:8000 --access-logfile="-").
Before trying to serve images, I had this nginx configuration :
http {
client_max_body_size 5M;
upstream backend_api {
server backend:8000;
# 'backend' is the name of the backend service in my docker-compose config
}
server {
listen 80;
include /etc/nginx/mime.types;
root /usr/share/nginx/html;
index index.html;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /api {
proxy_pass http://backend_api;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $http_x_forwarded_proto;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
}
location / {
try_files $uri $uri/ /index.html;
}
}
}
Then I thought that /media requests should also be passed to the backend and I changed
location /api
into
location ~ ^/(api|media)/ {
My /api URLs are still handled correctly but /media URLs are answered by a 404 :
(trying to load profile pictures of my user(s) in a kanban view).
Also trying directly http://localhost/media/base/users/8/picture.jpg directly in my browser doesn't work :
From here I don't know what to do to solve the issue. If something is missing, mention it and I'll update the post.
Django does not serve static- and media files with runserver, you will need WhiteNoise for that. See http://whitenoise.evans.io/en/stable/
Whitenoise however is not suitable for serving user-uploaded media files. See http://whitenoise.evans.io/en/stable/django.html#serving-media-files
(Optionally, skip whitenoise, and host static/media files through NGINX.)
You really shouldn't be hosting your server with py manage.py runserver. This is not secure. See Why not use "runserver" for production at Django? and https://docs.djangoproject.com/en/dev/ref/django-admin/#runserver
Use something like Gunicorn instead.
See https://docs.djangoproject.com/en/4.1/howto/deployment/wsgi/gunicorn/
(Or waitress, the windows alternative)
https://pypi.org/project/django-waitress/
To host static/media files with nginx, paste this into your nginx conf:
location /media {
alias /PATH/TO/DIRECTORY; #Absolute path.
}
And in your settings.py, set the media root to that same directory.

Moving a Django app into production using Nginx and Gunicorn, there are no errors but the page doesn't show

I'm trying to setup an app on my Digital Ocean production server, I've followed these instructions, testing gunicorn and nginx,I could access the app in gunicorn and both services start fine with no errors logged. However when I go to the site it does not show anything. This is a subdomain of my main site. Mostly I'm looking to find a place to start troubleshooting this especially since everything looks fine.
Configs:
Nginx config for subdomain (Django) site:
server {
# use 'listen 80 deferred;' for Linux
# use 'listen 80 accept_filter=httpready;' for FreeBSD
listen 80;
# set the correct host(s) for your site
server_name subdomain.domain_name.com www.subdomain.domain_name.com;
location = /favicon.ico {access_log off; log_not_found off;}
location /static/ {
root /path/to/static/files;
}
location / {
include proxy_params;
proxy_pass http://unix:/path/to/.sock/file;
}
}
Nginx config for main (static) site:
server {
listen 80 default_server;
listen [::]:80 default_server;
root /var/www/main_site_folder;
index index.html index.htm index.nginx-debian.html;
server_name domain_name www.domain_name;
location / {
try_files $uri $uri/ =404;
}
}
[Unit]
Description=Description of the app
After=network.target
[Service]
User=MyUserName
Group=www-data
WorkingDirectory=/var/www/app_directory/
ExecStart=/path/to/venv/and/gunicorn --access-logfile - --workers 3 --bind unix:/var/www/app_dir/.sock_filename app_name.wsgi:application
[Install]
WantedBy=multi-user.target
You could start by changing ALLOWED_HOSTS = ["*"] in settings.py. Also try accessing your URL through CURL.
Solved by adding an A record for the sub-domain 🙄, this was a classic case of not being able to find the answer because it was right in front of my face. 😅

Multiple Django Project + Nginx on subpath

I am trying to run multiple dashboards written in Django to run on my server, but am not getting it up and running. Followed this digital ocean tutorial and modified it according to this SO answer. Now everything is up and running and but when am pointing to my URL, it shows Nginx welcome page http://ipaddr/first_dashboard
Below is the gunicorn_fdab.socket file :
[Unit]
Description=gunicorn socket
[Socket]
ListenStream=/run/gunicorn_fdab.sock
[Install]
WantedBy=sockets.target
Below is gunicorn_fdab.service file :
[Unit]
Description=gunicorn daemon for fdab
Requires= gunicorn_fdab.socket
After=network.target
[Service]
User=root
Group=root
WorkingDirectory=/opt/fdab
ExecStart=/opt/anaconda/envs/fdab/bin/gunicorn \
--access-logfile - \
--workers 3 \
--bind unix:/run/gunicorn_fdab.sock \
fdab.wsgi:application
[Install]
WantedBy=multi-user.target
Now this is my Nginx conf file :
server {
listen 80;
server_name 111.11.11.111;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /opt/fdab/fdab;
}
location /fdab {
include proxy_params;
rewrite /fdab(.*) $1;
proxy_pass http://unix:/run/gunicorn_fdab.sock;
}
}
Am not able to understand where am doing it wrong.
If am doing curl --unix-socket /run/gunicorn_fdab.sock localhost , it just returning nothing.
(base) root#virtualserver01:~# curl --unix-socket /run/gunicorn_fdab.sock localhost
(base) root#virtualserver01:~#
Project is stored at /opt/fdab.
Additional information:
Basically, my project structure for both the project is like this :
/opt/fdab
/fdab
/fdab_dashboard
/opt/pdab
/pdab
/pdab_dashboard
The structure for the project is like this because I intend to have multiple apps in fbad and fdab2(second project name.
EDIT
Updated conf file for Nginx :
server {
listen 80;
server_name 111.11.11.111;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /opt/fdab/fdab;
}
location /fdab {
include proxy_params;
rewrite /fdab/(.*) /$1 break;
proxy_pass http://unix:/run/gunicorn_fbad.sock;
}
location /pdab/static/ {
alias /opt/pdab/pdab/static/;
}
location /pdab {
include proxy_params;
rewrite /pdab/(.*) /$1 break;
proxy_pass http://unix:/run/gunicorn_pdab.sock;
}
}
Now I have added FORCE_SCRIPT_NAME = '/exampleproject' in both the project.
Now what's happening is that, if am typing in, http://<ipaddr>/fdab/fdab_dashboard it's working fine, but if am typing in http://<ipaddr>/fdab/ or http://<ipaddr>/pdab/, am getting redirected to http://<ipaddr>/fdab_dashboard and http://<ipaddr>/pdab_dashboard , this is not what is required and moreover, http://<ipaddr>/fdab_dashboard seems to be working properly. But the fdab part of the url is missing, once I get into the app after logging in, the url seems fine, maybe because of the FORCE_SCRIPT_NAME = '/fdab' , but the url http://<ipaddr>/pdab_dashboard gives me 404 error page.
So the good news is that your gunicorn and nginx configs as posted look correct.
(1) Problem #1 default web page shows:
This is almost always caused by the default nginx config file default.conf. Just remove that file and you should see your site popping up instead. The only other thing to check for is to test and reload nginx to make sure your configuration is valid and loaded:
sudo nginx -t
sudo systemctl reload nginx
(2) Problem #2 curl to the unix socket doesn't return what you'd expect. The curl command looks slightly off: try something like:
curl -v --no-buffer --unix-socket /run/gunicorn_fdab.sock http://localhost/route/available/in/django
You can pair that curl while tailing the gunicorn logs with journalctl --since today -u gunicorn -f
I suggest you try not doing any URL rewrites in the nginx config. Do the proxy_pass to the sockets as required, and then adapt your Django URL configs to match the URLs you want to use in the different apps.

Deploy one django project with multiple app on multiple subdomains on AWS EC2

I have one django project which contains 2apps namely, admin and api.
The admin app is dependent on api app to access the models.
I have 2 subdomains like: admin.xxxx.com and api.xxxx.com.
This project is currently deployed in AWS EC2 using gunicorn + nginx.
UPDATE
All admin requests pass to : some.ip.address.0:8000/admin/, and all api requests pass through some.ip.address.0:8000/
Is there any way I can point my some.ip.address.0:8000/admin/ to admin.xxxx.com and some.ip.address.0:8000/ to api.xxxx.com?
UPDATE 2:
myproject_nginx.conf file:
upstream myproject_backend_server {
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response (in case the Unicorn master nukes a
# single worker for timing out).
server unix:/home/ubuntu/myproject_backend/myproject_backend.sock fail_timeout=0;
}
server{
listen 80;
listen [::]:80;
server_name admin.mydomain.in;
location / {
proxy_pass http://13.***.***.***:8000/admin/;
}
location /static/ {
alias /home/ubuntu/myproject_backend/static/;
}
location /media/ {
alias /home/ubuntu/myproject_backend/media/;
}
}
server {
listen 80;
server_name 13.***.***.***:8000 api.mydomain.in www.api.mydomain.in;
client_max_body_size 4G;
location /static/ {
alias /home/ubuntu/myproject_backend/static/;
}
location /media/ {
alias /home/ubuntu/myproject_backend/media/;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
# Try to serve static files from nginx, no point in making an
# *application* server like Unicorn/Rainbows! serve static files.
if (!-f $request_filename) {
proxy_pass http://myproject_backend_server;
break;
}
}
}
myproject urls.py file:
from django.urls import path, re_path, include
from django.conf.urls.static import static
from django.conf import settings
from django.views.static import serve
urlpatterns = [
re_path(r'^', include('api_app.urls')),
...
path('admin/', include('admin_app.urls')),
...
re_path(r'^static/(?P<path>.*)$', serve,
{'document_root': settings.STATIC_ROOT}),
re_path(r'^media/(?P<path>.*)$', serve,
{'document_root': settings.MEDIA_ROOT}),
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
It opening my admin login page, but I try to login it says: /admin/admin not found on this server.
Please suggest what's wrong?
As I can understand you want to display an admin page when the user types this address http://admin.mydomain.in in their browser and this admin page is handled by your django app so you are using nginx to proxy to http://13.***.***.***:8000/admin/ from where your admin page can be accessed.
But the problem here is that your app doesn't know how to do that. So it needs a middleman ( which in your case is Gunicorn ) that is specifically meant for this purpose. Moreover nginx cannot directly communicate with your django app simply because it was not meant to serve dynamic content.
So to fix this problem you need to configure gunicorn so that it listen to address http://13.***.***.***:8000 to where nginx will forward the request. And then run gunicorn on this address with parameter as your app name. You can read the second answer to the post serving a request from gunicorn to configure your gunicorn file.
Yes, in order to do that you have to point two of the domains to EC2 instance hosting your django application (or ELB if you're using one) and configure Nginx so it redirects requests from one domain to admin and from other to api path.

Passing Static Content from Django/Gunicorn to Nginx

I've been playing around with Nginx & Gunicorn/Django for the past two days. Thought I'd share my troubles on here in hopes someone might have a potential solution
I have two separate machines. On one, I have a public facing box with nginx installed which acts as a reverse proxy for my other internal box with Django and Gunicorn installed.
When web requests come from the outside, Nginx simply acts as the middle man and forwards the request to Django/Gunicorn. Django/Gunicorn returns a response to Nginx. Nginx then takes that response and forwards back to where the web request came from. It's a pretty neat set up because Nginx can handle slow clients and take the hit in the case of a DDOS attack. Your actual application server is safe from any sort of damage :)
Although my application doesn't serve actual static content (pictures, videos etc ...). I would still like to pass static content from Gunicorn/Django to Nginx. Currently when I access the Django admin, it's just plain text with no graphics or images. It really sucks. I did however notice that when I accessed the Gunicorn/Django server directly, all the images for the Django admin showed up. It took me a while to understand what was happening here, and I realized that the static content wasn't being passed.
Here is my current Django urls.py configuration:
from django.conf.urls import patterns, include, url
from django.contrib import admin
from django.conf.urls.static import static
from django.conf import settings
from django.views.decorators.csrf import csrf_exempt
admin.autodiscover()
urlpatterns = patterns('',
# Examples:
# url(r'^$', 'django.views.home', name='home'),
# url(r'^blog/', include('blog.urls')),
url(r'^admin/', include(admin.site.urls)),
) + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
This automatically finds the Django admin files and serves them to me through Gunicorn. I don't know how but it works. My debug settings is set to True so I don't know if that has anything to do with it.
My nginx conf looks like this:
worker_processes 1;
events {
worker_connections 1024;
}
http {
sendfile on;
gzip on;
gzip_http_version 1.0;
gzip_proxied any;
gzip_min_length 1000;
gzip_disable "MSIE [1-6]\.";
gzip_types text/plain text/xml text/css
text/comma-separated-values
text/javascript
application/x-javascript
application/atom+xml;
# Configuration containing list of application servers
upstream app_servers {
server 10.101.010.111:8000;
# server 127.0.0.1:8081;
# ..
# .
}
# I just added this in so that it will redirect all requests to HTTPs
server {
listen 80;
server_name *.mydomain.com;
rewrite ^/(.*) https://*.mydomain.com/$1 permanent;
}
# Configuration for Nginx
server {
# Running port
listen 443;
ssl on;
server_name *.mydomain.com;
### SSL log files ###
#access_log logs/ssl-access.log;
#error_log logs/ssl-error.log;
### SSL cert files ###
ssl_certificate ssl/server.crt;
ssl_certificate_key ssl/server.key;
### Add SSL specific settings here ###
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4:HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
keepalive_timeout 15;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
# Settings to serve static files
location ^~ /static/ {
# Example:
# root /full/path/to/application/static/file/dir;
root /app/static/;
}
# Serve a static file (ex. favico)
# outside /static directory
location = /favico.ico {
root /app/favico.ico;
}
# Proxy connections to the application servers
# app_servers
location / {
proxy_pass http://app_servers;
proxy_redirect off;
proxy_set_header Accept-Encoding "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Forwarded-Proto $scheme;
add_header Front-End-Https on;
}
}
}
How can I pass static content from Django/Gunicorn to Nginx?
Looking forward to hearing your suggestions! I'm a self taught noob so any help would be extremely appreciated!
If you tell nginx to recognize the URI /static/, it will serve all files under that directory. In your mysite.conf:
location /static/ {
alias /path/to/your/static/;
}
Edit: I'm going to try to explain why this works. Originally, Django is managing all your URLs. When it receives a request for a specific URL, it generates the appropriate response. However, when accessing something like an image, the initial response contains something like an <img> tag, which includes a source. The client then sends back a request to this source URL and asks for the image. The important part here is that the image is not served with the page initially. The other key is that nginx sees all requests first, and only passes to Django when it has to. If you tell nginx to recognize requests to /static/, you are telling it to intercept and answer all requests to this URI. Therefore, it will answer requests for static files without even passing to Gunicorn or Django. I'm new to this too, so this may not be completely accurate, but it's my best shot at an explanation!
Hope this helps.