I have a django project named MySite, and this project has some application inside as below:
- MySite
-- app
-- venv
-- media
-- django_project
--- wsgi.py
--- settings.py
--- urls.py
--- asgi.py
To deploy on aws, I am in the phase of gunicorn configuring. However I face with this error:
guni:gunicorn BACKOFF Exited too quickly (process log may have details)
However, first time status is like:
gunicorn STARTING
this is my gunicorn.conf:
[program:gunicorn]
directory=/home/ubuntu/MySite
command=/usr/bin/gunicorn --workers 3 --bind unix:/home/ubuntu/MySite/app.sock django_project.wsgi.application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
program:gunicorn
in gunicorn.err.log it says problem is in:
usage: gunicorn [OPTIONS] [APP_MODULE]
gunicorn: error: unrecognized arguments: django_project.wsgi.application
when I try this:
gunicorn --bind 0.0.0.0:8000 django_project.wsgi:application
I get this error:
SyntaxError: invalid syntax
[2021-02-10 10:12:40 +0000] [6914] [INFO] Worker exiting (pid: 6914)
[2021-02-10 10:12:40 +0000] [6912] [INFO] Shutting down: Master
[2021-02-10 10:12:40 +0000] [6912] [INFO] Reason: Worker failed to boot.
The entire process to install and run gunicorn which I did:
**********************************START********************************
sudo apt-get upgrade -y
sudo apt-get update -y
1) Clone the git project
git clone https://github.com/XX/MyProj.git
2) cd /MySite ## there is a venv with django installed in
3) Activate venv
source venv/bin/activate
5. Instal NGINX and GUNICORN
pip3 install gunicorn ## install without sudo..
sudo apt-get install nginx -y
pip install psycopg2-binary
6. Connect gunicorn (#Error: Worker failed to boot.)
gunicorn --bind 0.0.0.0:8000 django_project.wsgi:application
7. Install supervisor
sudo apt-get install -y supervisor ## This command holds the website after we logout
8. Config supervisor
cd /etc/supervisor/conf.d
sudo touch gunicorn.conf
9) ##In the file file do following###
[program:gunicorn]
directory=/home/ubuntu/MySite
command=/usr/bin/gunicorn --workers 3 --bind unix:/home/ubuntu/MySite/app.sock django_project.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/var/log/gunicorn/gunicorn.err.log
stdout_logfile=/var/log/gunicorn/gunicorn.out.log
[group:guni]
Program:gunicorn
####endfile####
10). Config supervisor
cd /etc/supervisor/conf.d
sudo touch gunicorn.conf
11). Connect file to supervisor
sudo mkdir -p /var/log/gunicorn
sudo supervisorctl reread
sudo supervisorctl reread
sudo supervisorctl update
12. Check if gunicorn is running in background
sudo supervisorctl status
**********************************END********************************
Your issue could be a combination of a few things:
Verify you django settings.py is set up properly for deployment. Pay very close attention to the STATIC_ROOT and STATICFILES_DIR variables as they are critical to serving your projects static files.
Make sure your virtualenv is activated and you have ran pip install -r requirements.txt.
Note: At this point try to run your project with your servers public ip python manage.py runserver server_public_ip:8000 a lot of people assume that, because their project ran locally, it will run on the server. Something always goes wrong.
Make sure you run python manage.py collectstatic on your server. This collects your static files and makes a directory for it. Take note of the path it tells you it's going to copy them to, you're going to need it for your /static/ location block in your nginx sites-available configuration file.
Make sure your gunicorn.conf command variable points to your virtualenv path, and that it points to a .sock file that gunicorn and nginx can access (sudo chown user:group). Here is an example:
[program:gunicorn]
command=/home/user/.virtualenvs/djangoproject/bin/gunicorn -w3 --bind unix:/etc/supervisor/socks/gunicorn.sock djangoproject.wsgi:application --log-level=info
directory=/home/user/djangoproject
numprocs=3
process_name=%(program_name)s_%(process_num)d
user=user
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8,HOME="/home/user/djangoproject", USER="user"
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/gunicorn.err.log
stdout_logfile=/var/log/supervisor/gunicorn.out.log
There are a couple ways you can set up your nginx configuration files. Some docs have you do it all in the nginx.conf file, however, you should break it up into sites-available with a symbolic link to sites-enabled. Either way should get you the same result. Here is an example:
upstream django {
server unix:/etc/supervisor/socks/gunicorn.sock;
}
server {
listen 80;
server_name www.example.com;
client_max_body_size 4G;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_headers_hash_max_size 1024;
proxy_read_timeout 300s;
proxy_connect_timeout 75s;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log debug;
location /media/ {
autoindex off;
alias /home/user/djangoproject/media/;
}
location /static/ {
autoindex off;
alias /home/user/djangoproject/static/;
}
location / {
proxy_pass http://django;
proxy_redirect off;
}
}
Make sure that you run sudo service nginx restart after every nginx configuration change.
Make sure you run sudo supervisorctl reread, sudo supervisorctl update all, and sudo supervisorctl restart all after every supervisor configuration file change. In your case every gunicorn.conf file change.
Lastly, as a general rule, make sure all of your directory paths match to their respected processes. For example: Let's say your gunicorn command variable points to a sock file that is in /path/to/project/gunicorn.sock, but your nginx configuration has a proxy_pass to /etc/supervisor/socks/gunicorn.sock, nginx doesn't know where to pass your requests to, so gunicorn never sees the request. Also, you can add this to your nginx location blocks so you can see where each request gets to in your browser dev tools response header: add_header X-debug-message "The port 80, / location was served from django" always;
Note: If you are getting the "Welcome to Nginx" page, it means nginx doesn't know where to send the request. A lot of times you have a static root directory path problem. However, there are other issues, but situational to how things are set up. You'll have to debug with some trial and error. Also, try adding a location block to a known url like http://example.com/login, if you get there, you know you have an nginx configuration issue. If you get 404 not found, then you most likely have a django project problem or a gunicorn problem.
This guide should work just fine :-)
Install dependencies
[user#machine]$ sudo apt update && sudo apt upgrade -y
[user#machine]$ sudo apt install python3-dev python3-pip supervisor nginx -y
[user#machine]$ sudo systemctl enable nginx.service
[user#machine]$ sudo systemctl restart nginx.service
[user#machine]$ sudo systemctl status nginx.service
Create new site
/etc/nginx/sites-available/<site-name>
server{
listen 80;
server_name <ip or domain>;
location = /favicon.ico {
access_log off;
log_not_found off;
}
location /static/ {
root /path/to/static/root;
}
# main django application
location / {
include proxy_params;
proxy_pass http://unix:/path/to/project/root/run.sock;
}
}
Create symbolic link
[user#machine]$ ln -s /etc/nginx/sites-available/<site-name> /etc/nginx/sites-enabled
Setup supervisor
/etc/supervisor/conf.d/<project-name>.conf
[program:web]
directory=/path/to/project/root
user=<user>
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/path/to/logs/gunicorn-error.log
command=/path/to/gunicorn --access-logfile - --workers 3 --bind unix:/path/to/project/root/run.sock <project-name>.wsgi:application
Then
[user#machine]$ sudo supervisorctl reread
[user#machine]$ sudo supervisorctl update
[user#machine]$ sudo supervisorctl status web
[user#machine]$ sudo systemctl restart nginx.service
The error tells you that the way you start gunicorn is wrong.
It looks like you are using supervisor. As per the docs, the correct syntax is:
[program:gunicorn]
command=/path/to/gunicorn main:application -c /path/to/gunicorn.conf.py
directory=/path/to/project
user=nobody
autostart=true
autorestart=true
redirect_stderr=true
Related
I've worked with Nginx and uWSGI numerous times, but I've never seen socket not being created even when specified explicitly.
My uWSGI config (/etc/uwsgi/sites/my_site.ini) is quite simple:
[uwsgi]
project = my_site
base = /root
chdir = %(base)/%(project)
home = %(base)/Env/%(project)
module = %(project).wsgi:application
master = true
processes = 5
socket = /run/uwsgi/%(project).sock
chmod-socket = 664
vacuum = true
As visible, uWSGI must create socket with 644 permissions (although some suggest it should be 666).
Nginx server configuration (/etc/nginx/nginx.conf) is quite simple as well:
server {
listen 80;
server_name mysite.com www.mysite.com;
location /static/ {
root /root/my_site;
}
location / {
include uwsgi_params;
uwsgi_pass unix:/run/uwsgi/my_site.sock;
}
}
uwgsi.service (startup script) (/etc/systemd/system/uwsgi.service on CentOS):
[Unit]
Description=uWSGI Emperor service
After=syslog.target
[Service]
ExecStart=/usr/uwsgi --emperor /etc/uwsgi/sites
Restart=always
KillSignal=SIGQUIT
Type=notify
StandardError=syslog
NotifyAccess=all
[Install]
WantedBy=multi-user.target
After making these configurations, I executed following commands:
sudo systemctl daemon-reload
sudo systemctl restart uwsgi
sudo systemctl restart nginx
sudo systemctl status uwsgi
sudo systemctl status nginx
Everything was fine according to commands above, but Nginx returned 502 bad gateway) error when visiting website.
Thus I checked /var/log/nginx/error.log/:
2018/08/25 19:33:10 [crit] 14920#0: *3 connect() to unix:/run/uwsgi/my_site.sock failed (2: No such file or directory) while connecting to upstream, client: xx.xxx.xxx.xx, server: my_site.com, request: "GET / HTTP/1.1", upstream: $
As you can see above, uWSGI doesn't create socket at /run/uwsgi/ directory for some reason...
I'm sure that socket is not being created at /run/uwsgi/, since ls /run/uwsgi/ returns nothing.
Permissions:
sudo usermod -a -G root nginx;chmod 710 /root
Main problem:
uWSGI is not creating a unix socket at the specified path in the configuration above, what could be the problem?
Is there any possible cause?
I have followed the DigitalOcean tutorial to deploy a django app at DigitalOcean, the guide is:
https://www.digitalocean.com/community/tutorials/how-to-deploy-a-local-django-app-to-a-vps
Question:
The problem is that when I go to the IP with the browser, I see the Welcome to nginx page and not my django app.
Tutorial important points
Respect the tutorial, I have not seen the following error as tutorial says: server_names_hash, you should increase server_names_hash_bucket_size: 32
Another important difference between what I did and tutorial is that gunicorn_django --bind yourdomainorip.com:8001 did not work for me.
I use this statement to start gunicorn:
web: gunicorn --chdir code/computationalMarketing computationalMarketing.wsgi --log-file -
My configuration
At /etc/nginx/sites-enabled I have symlink called computationalMarketing that refers to /etc/nginx/sites-available/computationalMarketing
This files has the following lines:
server {
listen 127.0.0.1;
server_name 159.65.18.211;
error_log /var/log/nginx/localhost.error_log info;
root /var/www/localhost/htdocs;
location /static/ {
alias /opt/computationalMarketing/static/;
}
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Real-IP $remote_addr;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
}
}
I have a virtualenv at /opt/computationalMarketing and inside this I have another computationalMarketing folder with the Git repo file.
This repo has the following structure:
My installations are:
sudo pip3 install numpy==1.13.3
sudo pip3 install pandas==0.22.0
sudo pip3 install scikit-learn==0.19.1
sudo pip3 install pymysql==0.8.1
sudo pip3 install psycopg2==2.7.3.2
sudo pip3 install django==2.0.5
sudo pip3 install django-connection-url==0.1.2
sudo pip3 install whitenoise==3.3.1
sudo pip3 install gunicorn==19.7.1
The database is a Postgresql, which I can connect without problem.
Can anyone guess why I am seeing the nginx page and not my django app?
You've told nginx to listen for this particular config on the localhost only. Don't do that. Remove that listen line altogether.
There are a few other weird things in your question. The command you claim to be using to start gunicorn is a Procfile instruction, it's not something you could actually run at the command line. What command are you actually using to start gunicorn? Whatever you use, you do need to tell it to serve on the same port that nginx is proxying to - in your case 8001.
Hi so I am trying to deploy my portfolio to Ubuntu Server 16.04 but I keep getting internal server error.
To walk through what i have done i created the instance and changed the HTTP and HTTPs security settings to anywhere.
After that i launched the instance then running these commands
ubuntu#ip-172-31-41-27:~ sudo apt-get update
ubuntu#ip-172-31-41-27:~$ sudo apt-get install python-pip python-dev
nginx git
ubuntu#ip-172-31-41-27:~$ sudo apt-get update
ubuntu#ip-172-31-41-27:~$ sudo pip install virtualenv
ubuntu#ip-172-31-41-27:~/portfolio$ virtualenv venv --python=python3
ubuntu#ip-172-31-41-27:~/portfolio$ source venv/bin/activate
(venv)ubuntu#ip-172-31-41-27:~/portfolio$ pip install -r
requirements.txt
(venv) ubuntu#ip-172-31-41-27:~/portfolio$ pip install django bcrypt
django-extensions
(venv) ubuntu#ip-172-31-41-27:~/portfolio$ pip install gunicorn
i edited the settings.py
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = False
ALLOWED_HOSTS = ['52.14.89.55']
STATIC_ROOT = os.path.join(BASE_DIR, "static/")
then run,
(venv) ubuntu#ip-172-31-41-27:~/portfolio$ python manage.py
collectstatic
followed by,
ubuntu#ip-172-31-41-27:~/portfolio$ sudo vim
/etc/systemd/system/gunicorn.service
where I add
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=ubuntu
Group=www-data
WorkingDirectory=/home/ubuntu/portfolio
ExecStart=/home/ubuntu/portfolio/venv/bin/gunicorn --workers 3 --bind
unix:/home/ubuntu/portfolio/portfolio.sock portfolio.wsgi:application
[Install]
WantedBy=multi-user.target
followed by a gunicorn reboot
ubuntu#ip-172-31-41-27:~/portfolio$ sudo systemctl daemon-reload
ubuntu#ip-172-31-41-27:~/portfolio$ sudo systemctl start gunicorn
ubuntu#ip-172-31-41-27:~/portfolio$ sudo systemctl enable gunicorn
finally,
ubuntu#54.162.31.253:~$ sudo vim /etc/nginx/sites-
available/portfolio
adding
server {
listen 80;
server_name 52.14.89.55;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/ubuntu/portfolio;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/ubuntu/portfolio/portfolio.sock;
}
}
creating the link
ubuntu#ip-172-31-41-27:/etc/nginx/sites-enabled$ sudo ln -s
/etc/nginx/sites-available/portfolio /etc/nginx/sites-enabled
ubuntu#ip-172-31-41-27:/etc/nginx/sites-enabled$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
deleting the default
then restarting
ubuntu#ip-172-31-41-27:/etc/nginx/sites-enabled$ sudo service nginx
restart
EDIT**
I changed the following codes to match exactly what I have done.
I have a single Docker container deployment to AWS Elastic Beanstalk.
When I visit the site, it returns a 502 error, which makes me think the port inside the Docker container is not exposed.
These are my settings:
Dockerrun.aws.json:
{
"AWSEBDockerrunVersion": "1",
"Volumes": [
{
"ContainerDirectory": "/var/app",
"HostDirectory": "/var/app"
}
],
"Logging": "/var/eb_log",
"Ports": [
{
"containerPort": 80
}
]
}
Dockerfile
FROM ubuntu:16.04
# Install Python Setuptools
RUN rm -fR /var/lib/apt/lists/*
RUN apt-get update
RUN apt-get install -y software-properties-common
RUN add-apt-repository ppa:jonathonf/python-3.6
RUN apt-get update && apt-get install -y python3-pip
RUN apt-get install -y python3.6
RUN apt-get install -y python3-dev
RUN apt-get install -y libpq-dev
RUN apt-get install libffi-dev
RUN apt-get install -y git
# Add and install Python modules
ADD requirements.txt /src/requirements.txt
RUN cd /src; pip3 install -r requirements.txt
# Bundle app source
ADD . /src
# Expose
EXPOSE 80
# Run
CMD ["python3", "/src/app.py"]
app.py
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "Hello World!"
# run the app.
if __name__ == "__main__":
# Setting debug to True enables debug output. This line should be
# removed before deploying a production app.
app.debug = False
app.run(port=80)
I see this in my docker-ps.log:
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
ead221e6d2c6 2eb62af087be "python3 /src/app.py" 34 minutes ago Up 34 minutes 80/tcp peaceful_lamport
and:
/var/log/eb-docker/containers/eb-current-app/ead221e6d2c6-stdouterr.log
-------------------------------------
* Running on http://127.0.0.1:80/ (Press CTRL+C to quit)
and this error:
2017/07/06 05:57:36 [error] 15972#0: *10 connect() failed (111: Connection refused) while connecting to upstream, client: 172.5.154.225, server: , request: "GET / HTTP/1.1", upstream: "http://172.17.0.3:80/", host: "bot-platform.us-west-2.elasticbeanstalk.com"
What am I doing wrong?
After looking up your error code I think you could give the following solution a try. It seems that you have to edit the nginx config of elastic beanstalk. Thereto you add the file nginx.config to the directory .ebextionsions in elastic beanstalk. Put the following content into the file:
files:
"/etc/nginx/conf.d/000_my_config.conf":
content: |
upstream nodejsserver {
server 127.0.0.1:8081;
keepalive 256;
}
server {
listen 8080;
location / {
proxy_pass http://nodejsserver;
proxy_set_header Connection "";
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /myconfig {
proxy_pass http://my_proxy_pass_host;
}
}
Maybe you have to adjust it a little but this seems to be the proper way to solve your problem. If you google your error you will find a lot slightly different solutions on how to adjust nginx in order to resolve this problem.
I'm new to Django and I want to configure my app with apache2. I just follow the guide and the other question, but I can't figure out!
My simple configuration file sites-available/000-default.conf:
<VirtualHost *:80>
WSGIScriptAlias / /var/www/html/mysite/mysite/wsgi.py
WSGIPythonPath /var/www/html/mysite
<Directory /var/www/html/mysite/mysite>
<Files wsgi.py>
Require all granted
</Files>
</Directory>
</VirtualHost>
after apache2 restart there is a syntax error in WSGIPythonPath
if I put utside WSGIPythonPath inside apache2.conf file, the application does not work. What is the problem?
I follow https://docs.djangoproject.com/en/1.10/howto/deployment/wsgi/modwsgi/ but WSGIDaemonProcess seems not to work
Some things to check: Do you load the wsgi module?
LoadModule wsgi_module modules/mod_wsgi.so
Also look for this answer and a potential gotcha in the Apache2 configuration: 403 Forbidden error with Django and mod_wsgi
Ok, recommending it not very helpful, so here is a basic configuration for your nginx and gunicorn.
In order ot make it simple, let's assume your app is located in this directory: /home/root/app/src/ and we're gonna use root user (but you should create separate user for your app)
First your nginx. You have to insert a new file to your /etc/nginx/sites-enabled/yourapp.conf, if there is a file named default.conf - remove it.
Bellow I'm posting a nginx conf file, which will try to run your service with using gunicorn.sock:
upstream yourappname {
server unix:/home/root/app/src/gunicorn.sock fail_timeout=0;
}
server {
root /home/root/app/src/;
listen 80;
server_name yourdomain.com *.yourdomain.com
charset utf-8;
client_max_body_size 100m;
access_log /home/root/app/src/logs/nginx-access.log; #you have to have logs folder in src
error_log /home/root/app/src/logs/nginx-error.log;
location /static/ {
alias /home/root/app/src/static/;
}
location /media/ {
alias /home/root/app/src/media/;
}
}
so now on gunicorn start script .
#!/bin/bash
ME="root"
DJANGODIR=/home/root/app/src
SOCKFILE=/home/root/app/src/gunicorn.sock
USER=root
GROUP=webapps
NUM_WORKERS=3
DJANGO_SETTINGS_MODULE=yourapp.yoursettings
DJANGO_WSGI_MODULE=yourapp.wsgi
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source /home/root/app/env/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec /home/root/app/env/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name root \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP \
--bind=unix:$SOCKFILE \
--log-level=debug \
--log-file=-
ok now in order to be able to run your gunicorn start script it has to have execution model enabled so
sudo chmod u+x gunicorn_start
now you will be able to start your gunicorn server with just using ./gunicorn_start
in top of that I'll post also supervisor conf, which will try to start your app whenever it fail, or just when system boots.
At first install supervisor.
Then create a .conf file in your main directory /etc/supervisor/conf.d/your_conf_file.conf
inside insert:
[program:yourappname]
command = /home/root/app/src/gunicorn_start
user = root
stdout_logfile = /home/root/app/src/logs/gunicorn_supervisor.log
redirect_stderr = true
so having that done we have to tell our supervisor that we have just added new configuration file. Simply run those commands:
sudo supervisorctl reread
sudo supervisorctl update
and in order to check if your app is running correctly just run
sudo supervisorctl status yourappname
I hope this is helpful, if you have any questions just ask