Django+Nginx+Gunicorn "Connection reset by peer" Error - django

I'm using the Digital Ocean tutorial (here) to set up a app that allows for a file upload (videos ranging from 5 mb to 1 GB). I know that large file uploads is not an ideal use case, but the client and the server are sitting in neighboring buildings connected via LAN (fast speeds in transfer) and FTP was not an option provided to me.
When the files are small enough (30-40 mb), the app works fine. With 100 mb+ videos, I get a "502 - bad gateway" error on the client side.
Nginx error log shows the following:
2017/07/17 15:52:18 [error] 18503#18503: *9 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: <client-ip>, server: <my-hostname>, request: "POST /videos HTTP/1.1", upstream: "http://unix:/www/app/app.sock:/videos", host: "<my-hostname>", referrer: "<app-domain>/videos"
The Gunicorn error log shows no errors.
My gunicorn settings:
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=django
Group=www-data
WorkingDirectory=/www/app
EnvironmentFile=/www/app/.env2
ExecStart=/home/django/.pyenv/versions/django/bin/gunicorn --access-logfile /backup/logs/app_gunicorn_access.log --error-logfile /backup/logs/app_gunicorn_errors.log --workers 3 --worker-class=tornado --timeout=600 --graceful-timeout=10 --log-level=DEBUG --capture-output --bind unix:/www/app/app.sock app.wsgi:application
[Install]
WantedBy=multi-user.target
What am I doing wrong?
EDIT ->
NGINX config
server {
listen 80;
server_name <my-hostname>;
rewrite ^/(.*) https://<my-domain>/$1 permanent;
}
server {
listen 443 ssl;
proxy_read_timeout 600s;
keepalive_timeout 5;
fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;
server_name <my-hostname>;
client_max_body_size 0;
ssl_certificate /ssl_certs/hostname_bundle.cer;
ssl_certificate_key /ssl_certs/hostname.key;
root /www/app;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
alias /www/app/staticfiles/;
}
location /media/ {
alias /backup/app_media/;
}
location / {
include proxy_params;
proxy_pass http://unix:/www/app/app.sock;
include /etc/nginx/mime.types;
}
location /robots.txt {
alias /www/app/robots.txt;
}
}

Fixed the issue by changing the worker class from tornado to default (sync) workers while keeping the timeout long enough for large uploads to complete. Obviously there might be more elegant solutions to this issue, but I'm yet to come across anything. Exec line in gunicorn.service is as follows:
ExecStart=/home/django/.pyenv/versions/django/bin/gunicorn --access-logfile /backup/logs/app_gunicorn_access.log --error-logfile /backup/logs/app_gunicorn_errors.log --workers 3 --timeout=600 --graceful-timeout=10 --log-level=DEBUG --capture-output --bind unix:/www/app/app.sock app.wsgi:application

Related

502 bad gateway: Nginx, gunicorn, Django

Here is my nginx.conf
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
server_name 0.0.0.0;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /myproject;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://unix:/root/myproject/myproject.sock;
}
}
}
And my project have this structure:
This is my gunicorn.service file
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=root
Group=nginx
WorkingDirectory=/root/myproject/myproject
ExecStart=/root/myproject/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/gunicorn.sock myproject.wsgi:application
[Install]
WantedBy=multi-user.target
error.log
2022/04/13 12:45:30 [crit] 20474#20474: *8 connect() to unix:/root/myproject/myproject.sock failed (2: No such file or directory) while connecting to upstream, client: 10.81.234.10, server: 0.0.0.0, request: "GET / HTTP/1.1", upstream: "http://unix:/root/myproject/myproject.sock:/", host: "10.10.89.8"
My gunicorn service is working properly and nginx.conf is also working properly with no errors.
All permissions have been granted to myproject directory
Still I am getting a 502 Bad Gateway error.
Please let me know if any additional information needed
The issue is you are creating the Socket file in different location and pointing the nginx to different location. Your socket file is created at
/run/gunicorn.sock as you can see from this command
ExecStart=/root/myproject/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/gunicorn.sock myproject.wsgi:application
and referring to it in different location(/root/myproject/myproject.sock) in nginx configuration as you can
see here
proxy_pass http://unix:/root/myproject/myproject.sock;
You can solve this in 2 ways
by changing gunicorn conf from
ExecStart=/root/myproject/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/gunicorn.sock myproject.wsgi:application
to
ExecStart=/root/myproject/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/root/myproject/myproject.sock myproject.wsgi:application
By changing nginx conf from
proxy_pass http://unix:/root/myproject/myproject.sock;
to
proxy_pass http://unix:/run/gunicorn.sock;
I recommend using the first approach
Restart both gunicorn and nginx after these changes

Gunicorn, nginx, django, inside of a docker container. Gunicorn successfully runs on port 80 but nginx fails

I'm trying to set up a simple blogging site that I wrote using the django framework. The website works except that it isn't serving static files. I imagine that's because nginx isn't running. However, when I configure it to run on any port other than 80 I get the following error:
nginx: [emerg] bind() to 172.17.0.1:9000 failed (99: Cannot assign requested address)
When I run it on a port that is already being used by gunicorn I get the following error:
nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)
My nginx configuration file is as follows:
upstream django {
server 127.0.0.1:8080;
}
server {
listen 172.17.0.1:9000;
server_name my.broken.blog;
index index.html;
location = /assets/favicon.ico { access_log off; log_not_found off; }
location /assets {
autoindex on;
alias /var/www/html/mysite/assets;
}
location / {
autoindex on;
uwsgi_pass unix:///run/uwsgi/django/socket;
include /var/www/html/mysite/mysite/uwsgi_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
}
But if I run nginx without starting guincorn it runs properly but I get a 403 forbidden error.
Going with the suggested answer I don't get any errors, but the site is returning 403 forbidden and doesn't present the part of the website gunicorn is supposed to deliver.
Configure Nginx and Gunicorn in following way to make it work,
Use unix socket to comminicate between nginx and gunicron rather than running gunicorn in some port
Create a unit file for gunicorn in the following location
sudo nano /etc/systemd/system/gunicorn.service
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=root
Group=nginx
WorkingDirectory=/path-to-project-folder
ExecStart=/<path-to-env>/bin/gunicorn --workers 9 --bind unix:/path-to-sockfile/<blog>.sock app.wsgi:application
Restart=on-failure
[Install]
WantedBy=multi-user.target
Then start and enable gunicorn service
It will generate a sock file in the specified path.
sudo systemctl enable gunicorn
sudo systemctl start gunicorn
Note: Choose the suitable number of workers for gunicorn, can sepecify log files as follows
ExecStart=//bin/gunicorn --workers 9 --bind unix:/path-to-sockfile/.sock app.wsgi:application --access-logfile /var/log/gunicorn/access.log --error-logfile /var/log/error.log
create a new configuration file specific to the project in /etct/nginx rather than edit the default nginx.conf
nano /etc/nginx/blog.conf
and add the following lines ( can also add the file in /etc/nginx/default.d/)
server {
listen 80;
server_name 172.17.0.1; # your Ip
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /path-to-static-folder;
}
location /media/ {
root /path-to-media-folder;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://unix:/path-to-sock-file/<blog-sock-file>.sock;
}
}
include the /etc/nginx/blog.conf to nignx.conf
---------
----------
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/blog.conf; # the new conf file added
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
root /usr/share/nginx/html;
------
-------
run sudo nginx -t // to check for errors in nginx configuration.
run sudo systemctl nginx restart
refer: https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04

Django + Nginx + Gunicorn setup with error: (104: Connection reset by peer) while reading response header from upstream

I have a server set up with Ubuntu 20.04, Nginx and running a Django app with Gunicorn. My gunicorn.socket file is like so:
[Unit]
Description=gunicorn socket
[Socket]
ListenStream=/run/gunicorn.sock
[Install]
WantedBy=sockets.target
And My gunicorn.service file is so:
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=root
Group=www-data
WorkingDirectory=/root/backend
ExecStart=/root/.local/share/virtualenvs/backend-Lu9kB6KY/bin/gunicorn \
--access-logfile /var/log/nginx/backend.log \
--workers 3 \
--bind unix:/run/gunicorn.sock \
backend.wsgi:application
[Install]
WantedBy=multi-user.target
And in a file at /etc/nginx/sites-available/backend, I have:
server {
listen 80;
listen [::]:80;
server_name domain.com;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /root/backend/;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
}
When I visit the homepage, I see a 502 Bad Gateway error, and the logs at /var/log/nginx/error.log shows:
[error] 15866#15866: *86 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 82.102.24.9, server: domain.com, request: "GET / HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/", host: "domain.com"
What have I done wrong? What to do?

Nginx Gunicorn Django -- upstream prematurely closed connection Error

I am pretty new to NGINX, GUNICORN, DJANGO setup. I am using supervisor between nginx, gunicorn. Without NGINX, setup works well with supervisor and gunicorn and I can see the result through my server IP. But when i am using nginx to serve the requests, the error "upstream prematurely closed connection while reading response header from upstream" occurs. please anyone help me in this?
Supervisor command I am using:
sudo /path/to/gunicorn/gunicorn -k gevent --workers 4 --bind unix:/tmp/gunicorn.sock --chdir /path/to/application wsgi:application --timeout 120
below is the nginx.conf i am currently using and it is working as expected. but i am not sure it is up to the mark. Please look into this. Thanks.
==============Update=============
upstream xxxx {
server unix:/tmp/gunicorn.sock;
}
server{
listen 80;
listen [::]:80;
server_name xxx.in www.xxx.in;
return 301 https://$host$request_uri;
}
server{
listen 443 ssl;
ssl_certificate /etc/letsencrypt/live/xxx.in/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/xxx.in/privkey.pem;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx';
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root path/to/project;
}
location / {
include uwsgi_params;
proxy_pass http://unix:/tmp/gunicorn.sock;
}
}
Please check this steps:
First of all be sure that gunicorn is creating .sock file running with supervisor. You can ensure it with
$sudo supervisorctl status <name-of-supervisor-task-for-it>
(check if service is RUNNING)
$ls /tmp
(There should be a gunicorn.sock file existing there)
Also be aware of user that you are assigning to supervisor config. In this case you don't need to set root before command, just give privilege of root user to config file. like this:
[program:myprogram]
command=/path/to/gunicorn/gunicorn -k gevent --workers 4 --bind unix:/tmp/gunicorn.sock --chdir /path/to/application wsgi:application --timeout 120
<other commands>
user=root
And you nginx config should should look like this:
upstream django {
server unix://tmp/gunicorn.sock;
}
server {
listen 80;
server_name <your_app_domain_here>;
location / {
include uwsgi_params;
proxy_pass http://django/;
}

Django+gunicorn+nginx site runs perfect when using example.com:8000 but not when using example.com

I have a site on Digital Ocean VPS.
But I get a bad gateway when accessing example.com but not when example.com:8000.
Also when visiting the django-admin at example.com:8000/admin the default formatting is not coming. I guess nginx is not being able to serve the static content.
Below is the service file for gunicorn:
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
Group=www-data
WorkingDirectory=/FINlit
ExecStart=/FINlit/venvforfinlit/bin/gunicorn --access-logfile - --workers 3 --bind unix:/FINlit/Finlit.sock Finlit.wsgi:application
[Install]
WantedBy=multi-user.target
and the nginx config:
server {
listen 80;
server_name my_ip;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /FINlit;
}
location / {
include proxy_params;
proxy_pass http://unix:/FINlit/Finlit.sock;
}
}
Thanks
After almost searching for a couple of hours, my instincts led me to replace the IP with the domain name in the nginx config and now it works fine.
Actually I don't know, there might be a possibility that something else did the job for me but anyways.