502 Bad Gateway | Gunicorn - ngnix - django - django

I wanted to that my dango app run in server and I tried with gunciron. When i run my app with gunicorn, server is working.
I mean
`# gunicorn --bind 0.0.0.0:8000 myapp.wsgi`
is working
But if i disconnect with server, server is not working. So i used ngnix.
I followed this source https://github.com/satssehgal/URLShortnerDjango- . I did all them. I controlled my path but all path is right. What can i do for server work?
systemctl status nginx
enter image description here
ps aux | grep php-fpm
enter image description here
ngnix.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/x$
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
fastcgi_buffers 8 16k;
fastcgi_buffer_size 32k;
fastcgi_connect_timeout 300;
fastcgi_send_timeout 300;
fastcgi_read_timeout 300;
}
/etc/nginx/sites-available/myapp:
server {
listen 80;
location = /favicon.ico { access_log off; log_not_found off; }
keepalive_timeout 10;
charset utf-8;
location /static/ {
root /root/ayzguci/myapp;
}
location / {
include proxy_params;
proxy_pass http://unix:/root/ayzguci/myapp/myapp.sock;
proxy_redirect off;
proxy_set_header X-Forwarded-Host $5.105.5.140;
}
}
ps ax|grep gunicorn outout:
1782 ? Ss 0:00 /root/ayzguci/myenv/bin/python3 /root/ayzguci/myenv/bin/gunicorn --access-logfile - --timeout 1000 --workers 3 --bind unix:/root/ayzguci/myapp/myapp.sock myapp.wsgi:application
1812 ? S 0:00 /root/ayzguci/myenv/bin/python3 /root/ayzguci/myenv/bin/gunicorn --access-logfile - --timeout 1000 --workers 3 --bind unix:/root/ayzguci/myapp/myapp.sock myapp.wsgi:application
1813 ? S 0:00 /root/ayzguci/myenv/bin/python3 /root/ayzguci/myenv/bin/gunicorn --access-logfile - --timeout 1000 --workers 3 --bind unix:/root/ayzguci/myapp/myapp.sock myapp.wsgi:application
1814 ? S 0:00 /root/ayzguci/myenv/bin/python3 /root/ayzguci/myenv/bin/gunicorn --access-logfile - --timeout 1000 --workers 3 --bind unix:/root/ayzguci/myapp/myapp.sock myapp.wsgi:application
1900 pts/0 S+ 0:00 grep --color=auto gunicorn
netstat -an|grep 8000 output:
root#localhost:~# sudo apt install net-tools
Reading package lists... Done
Building dependency tree
Reading state information... Done
net-tools is already the newest version (1.60+git20161116.90da8a0-1ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 72 not upgraded.
root#localhost:~# netstat -an|grep 8000
root#localhost:~#

Related

Django Send_mail not working in production

I have the current set up and emails are pushed locally fine but it doens't in production. It is a django, docker, dokku and nginx setup hosted on a VPS.
Locally I run it in docker with docker compose up and in production also with the same dockerfile. This is all the info that i thought was useful does anyone know what i have to do?
settings.py
...
EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
EMAIL_USE_TLS = True
EMAIL_HOST_USER = os.environ.get("Email_Username", "")
EMAIL_HOST_PASSWORD = os.environ.get("Email_Password", "")
EMAIL_HOST = os.environ.get("Email_Host", "mail.domain.nl")
EMAIL_PORT = os.environ.get("Email_Port", 587)
...
actions.py
def sendRegisterMail(modeladmin, request, queryset):
for lid in queryset:
print("email sending")
achievements = '...'
subject = "..."
message = f""" ... """
send_mail(
subject,
message,
settings.EMAIL_HOST_USER,
[lid.email],
fail_silently=True,
)
print("email sent")
dokku nginx:show-config app
server {
listen [::]:80;
listen 80;
server_name app.dispuutstropdas.nl;
access_log /var/log/nginx/app-access.log;
error_log /var/log/nginx/app-error.log;
include /home/dokku/app/nginx.conf.d/*.conf;
location / {
return 301 https://$host:443$request_uri;
}
}
server {
listen [::]:443 ssl http2;
listen 443 ssl http2;
server_name app.dispuutstropdas.nl;
access_log /var/log/nginx/app-access.log;
error_log /var/log/nginx/app-error.log;
ssl_certificate /home/dokku/app/tls/server.crt;
ssl_certificate_key /home/dokku/app/tls/server.key;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers off;
keepalive_timeout 70;
location / {
gzip on;
gzip_min_length 1100;
gzip_buffers 4 32k;
gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/json application/xml application/rss+xml font/truetype application/x-font-ttf font/opentype application/vnd.ms-fontobject image/svg+xml;
gzip_vary on;
gzip_comp_level 6;
proxy_pass http://app-8000;
http2_push_preload on;
proxy_http_version 1.1;
proxy_read_timeout 60s;
proxy_buffer_size 4096;
proxy_buffering on;
proxy_buffers 8 4096;
proxy_busy_buffers_size 8192;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Request-Start $msec;
}
error_page 400 401 402 403 405 406 407 408 409 410 411 412 413 414 415 416 417 418 420 422 423 424 426 428 429 431 444 449 450 451 /400-error.html;
location /400-error.html {
root /var/lib/dokku/data/nginx-vhosts/dokku-errors;
internal;
}
error_page 404 /404-error.html;
location /404-error.html {
root /var/lib/dokku/data/nginx-vhosts/dokku-errors;
internal;
}
error_page 500 501 503 504 505 506 507 508 509 510 511 /500-error.html;
location /500-error.html {
root /var/lib/dokku/data/nginx-vhosts/dokku-errors;
internal;
}
error_page 502 /502-error.html;
location /502-error.html {
root /var/lib/dokku/data/nginx-vhosts/dokku-errors;
internal;
}
include /home/dokku/app/nginx.conf.d/*.conf;
}
upstream app-8000 {
server 172.17.0.7:8000;
}
ports in dokku
DOKKU_PROXY_PORT_MAP: http:80:8000 https:443:8000
Dockerfile (which works)
FROM python:3.10.7-alpine
COPY backend/requirements.txt ./
RUN apk update \
&& apk add postgresql-dev gcc python3-dev musl-dev \
&& apk add -u zlib-dev jpeg-dev gcc musl-dev \
&& pip install -r requirements.txt
RUN mkdir /code
WORKDIR /code
ADD . /code/
ENV PYTHONUNBUFFERED 1
EXPOSE 8000
CMD gunicorn core.wsgi:application --chdir backend -b 0.0.0.0:8000 --log-file -
Thanks for your help in advance

(nginx + gunicorn) small server instance drops/timeouts connections on +60 simple API requests / second. Can it be improved?

I'm setting up the first production architecture for my Django-based app. I'm using nginx + gunicorn + remote postgres database setup.
After performing simple API load tests with https://loader.io I've found out that when increasing the number of clients sending api requests over 60 clients/second for 30 seconds-long test the tool shows errors that the connections timeout.
When using double server setup with a load balancer I can double the clients/second number but I would expect a single 3vCPU /1 GB ram setup to be able to handle more than 30 requests/second - am I right?
I've tried a lot of differente gunicorn / nginx config parameters but nothing seems to help.
This is the content of my /etc/nginx/nginx.conf file:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
worker_rlimit_nofile 100000;
events {
worker_connections 4000;
multi_accept on;
use epoll;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
server_names_hash_bucket_size 512;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
reset_timedout_connection on;
keepalive_requests 100000;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
This is the content of my /etc/nginx/sites-available/MY_DOMAIN file:
server {
listen 80;
listen [::]:80;
server_name MY_DOMAIN www.MY_DOMAIN;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
listen [::]:443 ssl;
ssl on;
client_max_body_size 5M;
server_name MY_DOMAIN www.MY_DOMAIN;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /var/www/backend;
}
location /loaderio-b061bddf86a67379411d4ef54f7ee430/ {
root /var/www/backend;
}
location / {
include proxy_params;
proxy_pass http://unix:/var/www/backend/MY_SOCKET.sock;
}
location /ws/ {
include proxy_params;
proxy_pass http://unix:/var/www/backend/ws.sock;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
send_timeout 300;
}
ssl_certificate /etc/letsencrypt/live/MY_DOMAIN/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/MY_DOMAIN/privkey.pem; # managed by Certbot
This is the content of my supervisor file:
[program:gunicorn]
directory=/var/www/backend
command=/root/.pyenv/versions/VENV_NAME/bin/gunicorn --workers 5 --keep-alive 15 --worker-class gevent --bind unix:/var/www/backend/SOCK_NAME.sock config.wsgi:application
autostart=true
autorestart=true
log_level=debug
stderr_logfile=/var/log/gunicorn/gunicorn.out.log
stdout_logfile=/var/log/gunicorn/gunicorn.err.log
user=root
group=www-data
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8
[program:daphne]
directory=/var/www/backend
command=/root/.pyenv/versions/VENV_NAME/bin/daphne -u /var/www/backend/ws.sock config.asgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/daphne/daphne.out.log
stdout_logfile=/var/log/daphne/daphne.err.log
user=root
group=www-data
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8
[group:GROUP_NAME]
programs:gunicorn,daphne
When performing the load test the CPU vCores are between 10-18% load and the RAM usage is around 70%
Is it possible to enhance this single server instance performance above the 60 req/sec or is it just hardware limitation (I've already tried digital ocean 16vCPU 8GB ram droplet size and the results where pretty much the same (no matter if 5 or 15 workers were used)?

Nginx 502 Bad Gateway - Django - Gunicorn - When running MySQL reports (Stored Procedures) that take longer than 30 seconds

I have a Django, Nginx, Gunicorn, and MySQL on AWS.
Running a postback from django which calls a stored procedure that takes longer than 30 seconds to complete causes a return of "502 Bad Gateway" nginx/1.4.6 (Ubuntu).
It sure looks like a timeout issue and that this post should resolve it.
But alas, it doesn't seem to be working.
Here is my gunicorn.conf file:
description "Gunicorn application server handling formManagement django app"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
setuid ubuntu
setgid www-data
chdir /home/ubuntu/AARC-ServiceManager/ServerSide/formManagement
exec ve/bin/gunicorn --timeout 300 --workers 3 --bind unix:/home/ubuntu/AARC-ServiceManager/ServerSide/formManagement/formManagement.sock formManagement.wsgi:application
And my Nginx.conf:
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
# set client body size (max http request size) #
client_max_body_size 50M;
#upping the timeouts to allow time for the DB to return from a long running sproc
proxy_connect_timeout 300s;
proxy_read_timeout 300s;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
Any thoughts?
UPDATE:
This is the error in the nginx error log:
[error] 14316#0: *13 upstream prematurely closed connection while reading response header from upstream ...
I found the resolution!
I was updating the wrong gunicorn.conf file.
I saved to config file to my source control and when I was in the server, was updating that file.
However, I needed to be changing the file at location:
/etc/init/gunicorn.conf
... and I learned a lesson about having more than one config file on the server.
Thanks all who were offering help.

Django, Ubuntu-server, Nginx, Gunicorn Server Error 500

The issue I am having currently is that after I have launched my project with Gunicorn and do a symbolical link to Nginx’s sites-available to sites-enabled directory, followed with the shell command ... sudo service nginx reload … I’m given an error page html that says “ Server Error 500” when I attempt to connect locally and the project isn’t even talking to the outside world seeing as my domain is a blank canvas. I’m at a lost as to whether this error lies in my Nginx, Gunicorn, or Django. Any help, tips or good criticism will be gladly welcomed.
Project specifics:
This is being lauched off a small pc I built at home as a way to test the waters for different web ideas I have, by testing their potential locally first I can decide as to whether or not they're valuable.
Python 3.4
Virtualenv latest
Django 1.8
Gunicorn latest
Nginx latest
Ubuntu server 14.04.4 LTS
DB type: sqlite3(I have a very html based project)
settings.py
DEBUG = False
ALLOWED_HOSTS = ["*",]
wsgi.py
import os
from django.core.wsgi import get_wsgi_application
os.environ['DJANGO_SETTINGS_MODULE'] = "gartp.settings"
application = get_wsgi_application()
start_gunicorn.bash
this file has been … chmod +x … to make it excutable and is lauched with ./start_gunicorn.bash
#!/bin/bash
set -e
LOGFILE=/home/workarea/gart/gartp/gartp.log
ERRORFILE=/home/workarea/gart/gartp/error.log
LOGDIR=$(dirname $LOGFILE)
NUM_WORKERS=4
#The below address:port info will be used later to configure Nginx with Gunicorn
ADDRESS=127.0.0.1:8002
# user/group to run as
#USER=your_unix_user
#GROUP=your_unix_group
cd /home/workarea/gart/gartp/
source /home/workarea/gart/bin/activate
test -d $LOGDIR || mkdir -p $LOGDIR
exec /home/workarea/gart/bin/gunicorn -w $NUM_WORKERS --bind=$ADDRESS gartp.wsgi \
--log-level=debug \
--log-file=$LOGFILE 2>>$LOGFILE 1>>$ERRORFILE &
nginx sites-available file ‘gartp’
linked with … sudo ln -s /etc/nginx/sites-available/myproject /etc/nginx/sites-enabled...
upstream app_server_djangoapp {
server localhost:8002 fail_timeout=0;
}
server {
#EC2 instance security group must be configured to accept http connections over Port 80
listen 80;
server_name mydomain.com; # changed for a touch of privacy
#server_name ec2;
access_log /var/log/nginx/guni-access.log;
error_log /var/log/nginx/guni-error.log info;
keepalive_timeout 5;
# path for static files
root /home/workarea/gart/gartp/task/static;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://app_server_djangoapp;
break;
}
}
}
nginx.conf
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
server_names_hash_bucket_size 128;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
##
# nginx-naxsi config
##
# Uncomment it if you installed nginx-naxsi
##
#include /etc/nginx/naxsi_core.rules;
##
# nginx-passenger config
##
# Uncomment it if you installed nginx-passenger
##
#passenger_root /usr;
#passenger_ruby /usr/bin/ruby;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/gartp;
}
static ip configuration, values are not my own but from the example I used. I should state as well that I use a NIC on this pc.
/etc/network/interfaces
auto eth0
iface eth0 inet static
address 192.168.1.128
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1
/etc/resolvconf/resolv.conf.d/base
search (domain name)
nameserver 8.8.8.8
nameserver 8.8.4.4

nginx/uwsgi don't serve any request

I have nginx, uwsgi, django stack. There use to be an https serving from the nginx but I'm trying to disable it.
this is my nginx configuration files:
nginx.conf:
events {
worker_connections 768;
# multi_accept on;
}
http {
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
##
# nginx-naxsi config
##
# Uncomment it if you installed nginx-naxsi
##
#include /etc/nginx/naxsi_core.rules;
##
# nginx-passenger config
##
# Uncomment it if you installed nginx-passenger
##
#passenger_root /usr;
#passenger_ruby /usr/bin/ruby;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
site-available/mysite.com:
upstream django {
server 127.0.0.1:44000;
}
server {
listen 80 default;
server_name www.mysite.com mysite.com;
charset utf-8;
access_log /var/log/nginx/mysite_access.log;
error_log /var/log/nginx/mysite_error.log;
location /static/ {
root /opt/myapp/myapp/;
index index.html index.htm;
}
location / {
uwsgi_pass django;
include uwsgi_params;
}
}
When I run it with uwsgi i get this error:
django.core.exceptions.SuspiciousOperation: Invalid HTTP_HOST header (you may need to set ALLOWED_HOSTS): www.mysite.com
[pid: 12865|app: 0|req: 1/2] 84.94.36.146 () {36 vars in 571 bytes} [Mon May 26 07:12:55 2014] GET /favicon.ico => generated 0 bytes in 457 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 0)
I put 'www.mysite.com' on my ALLOWED_HOSTS section in settings.py and it didn't help.
these are the lines I added in settings.py:
ALLOWED_HOSTS = [
'.mysite.com', # Allow domain and subdomains
'.mysite.com.', # Also allow FQDN and subdomains
]
When I run it with runserver i get this error:
[26/May/2014 06:53:09] code 400, message Bad request syntax ('\x00;\x02\x00\x0c\x00QUERY_STRING\x00\x00\x0e\x00REQUEST_METHOD\x03\x00GET\x0c\x00CONTENT_TYPE\x00\x00\x0e\x00CONTENT_LENGTH\x00\x00\x0b\x00REQUEST_URI\x0c\x00/favicon.ico\t\x00PATH_INFO\x0c\x00/favicon.ico\r\x00DOCUMENT_ROOT\x0f\x00/etc/nginx/html\x0f\x00SERVER_PROTOCOL\x08\x00HTTP/1.1\x0b\x00REMOTE_ADDR\x0c\x0084.94.36.146\x0b\x00REMOTE_PORT\x05\x0051792\x0b\x00SERVER_PORT\x02\x0080\x0b\x00SERVER_NAME\x10\x00www.mysite.com\t\x00HTTP_HOST\x10\x00www.mysite.com\x0f\x00HTTP_CONNECTION')
[26/May/2014 06:53:09] ";
QUERY_STRINGREQUEST_METHODGET
CONTENT_TYPECONTENT_LENGTH
REQUEST_URI
/favicon.ico PATH_INFO
DOCUMENT_ROOT/etc/nginx/htmlSERVER_PROTOCOHTTP/1.1 /favicon.ico
REMOTE_ADDR
84.94.36.146
REMOTE_PORT51792
SERVER_PORT80
SERVER_NAMEwww.mysite.com HTTP_HOSTwww.mysite.comHTTP_CONNECTION" 400 -
I read that this error may be caused by HTTPS request, but it doesn't seem to me that I redirect HTTP requests to HTTPS anywhere, cause I deleted all those configurations.
any ideas?
Thanks
It's working now.
The problem was that the ALLOWED_HOSTS was already defined in the local_settings.py with the old domains. I updated the domains and it worked.
All the nginx configurations are correct, so the files attached here are useful.
Thanks for your time.
I solved it by using
uwsgi --socket :8001 --wsgi-file test.py
instead of
python manage.py runserser