Resolving Fatal status in Supervisor and Django - django

I am using Digital Ocean Ubuntu server to deploy my Django project and follow this guide to set it all up: A Complete Beginner's Guide to Django - Part 7
I am the process of configurig Gunicorn and Supervisor and I get the following error:
I am logged in as non-root but sudo user that I have created called betofolio. My django project is called betofolio.
Below is a screenshot of what my folders look like:
Following the steps from the tutorial:
Create a new file named gunicorn_start inside /home/betofolio:
vim gunicorn_start
I insert the following:
#!/bin/bash
NAME="betofolio"
DIR=/home/betofolio/betofolio
USER=betofolio
GROUP=betofolio
WORKERS=3
BIND=unix:/home/betofolio/run/gunicorn.sock
DJANGO_SETTINGS_MODULE=betofolio.settings
DJANGO_WSGI_MODULE=betofolio.wsgi
LOG_LEVEL=error
cd $DIR
source ../venv/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DIR:$PYTHONPATH
exec ../venv/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $WORKERS \
--user=$USER \
--group=$GROUP \
--bind=$BIND \
--log-level=$LOG_LEVEL \
--log-file=-
Then I save and exit.
Make this file executable:
chmod u+x gunicorn_start
Create two empty folders, one for the socket file and one to store the logs:
mkdir run logs
Configuring Supervisor
Create an empty log file inside the /home/betofolio/logs/ folder:
touch logs/gunicorn.log
Now create a new supervisor file:
sudo vim /etc/supervisor/conf.d/betofolio.conf
[program:betofolio]
command=/home/betofolio/gunicorn_start
user=betofolio
autostart=true
autorestart=true
redirect_stderr=true
stdout_logfile=/home/betofolio/logs/gunicorn.log
Then:
sudo supervisorctl reread
sudo supervisorctl update
Now check the status:
sudo supervisorctl status betofolio
I get:
I am new to linux and networking, so I am not sure how to resolve this

I was following the same guide, and ran into the same problem. What solved it for me was realizing I didn't have gunicorn installed in my virtual environment. I had done a pip3 install gunicorn, but you need to do pip install gunicorn. You may have to say sudo before if it says you don't have permission (sudo pip install gunicorn)
go into the logs folder, and open gunicorn.logs. This file is in /home/(user)/logs, and type less gunicorn.logs to see it. It will show you why it's failing. In my case it said it couldn't find gunicorn in the location I had specified within the gunicorn_start file.
Make sure you have the virtual environment activated The command line turns into (name of virtual env) $...
cd into /home/(user)/(venv)/bins, and type ls to see the files in the directory, and see if gunicorn is in there.
After fixing this it still wouldn't run, and I realized in the log files I had some typos in the gunicorn_start file, so that logs file is very handy to troubleshoot.

Related

Why django codes not updating automatically using supervisorctl and Gunicorn?

Using Supervisor, I deployed Django. Django codes are not updated, instead I must restart Nginx or Supervisor. If you could help me with this issue, that would be great.
Supervisor configuration
[program:program_name]
directory=/home/user/django/dir/django_dir/
command=/home/user/django/dir/venv/bin/gunicorn --workers 3 --bind unix:/home/user/django/dir/name.sock ingen>
#command=/home/user/django/dir/venv/bin/ssh_filename.sh
numprocs=3
process_name=name%(process_num)d
autostart=true
autorestart=true
stopasgroup=true
user=user
Group=www-data
stderr_logfile=/home/user/django/dir/logs/supervisor_err.log
stdout_logfile=/home/user/django/dir/logs/supervisor.log
redirect_stderr=true
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8
I am trying to restart the supervisor process with the below command in order to update the Django backend codes, but it doesn't always work.
sudo supervisorctl restart program_name:*
//'program_name' refers to the name of the program in the supervisor configuration file
METHOD 2
A second trail with a ssh_filename.sh file and the second command in the above supervisor configuration is used to run the ssh_filename.sh script.
#!/bin/bash
NAME="django_dir" #Django application name
DIR_PARENT=/home/user/django/dir
DIR=${DIR_PARENT}/django_dir #Directory where project is located
USER=user #User to run this script as
GROUP=www-data #Group to run this script as
WORKERS=3 #Number of workers that Gunicorn should spawn
SOCKFILE=unix:${DIR_PARENT}/dir.sock #This socket file will communicate with Nginx
DJANGO_SETTINGS_MODULE=django_dir.settings #Which Django setting file should use
DJANGO_WSGI_MODULE=django_dir.wsgi #Which WSGI file should use
LOG_LEVEL=debug
cd $DIR
source ${DIR_PARENT}/venv/bin/activate #Activate the virtual environment
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DIR_PARENT:$PYTHONPATH
#Command to run the progam under supervisor
exec ${DIR_PARENT}/venv/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $WORKERS \
--user=$USER \
--group=$GROUP \
--bind=$SOCKFILE \
--log-level=$LOG_LEVEL \
--log-file=-
Nginx web server throws a "BAD GATEWAY"" error when using the second method. If the above ssh_filename.sh script is run manually, the website works. Following are the codes I use to manually run the above ssh_filename.sh script
cd /home/user/django/dir
source venv/bin/activate
sudo venv/bin/sh_filename.sh //It works only if run with sudo
Questions
I activated the virtual environment in the above ssh_filename.sh script. Is it a smart idea?
What is the best way to update Django codes automatically without restarting Supervisor?
How to prevent the "BAD GATEWAY" error using the second method?

How to Deploy Multiple Django Apps on Single server using Ngnix & GUnicorn?

I am trying to Deploy Two Django apps with single AWS EC2 Instance having same IP.
But it always failed when I have added the second App.sock and test Supervisor.
I fond some body asked similar question before. but Not answered properly, and my use case is little different. ( Run multiple django project with nginx and gunicorn )
I have followed these steps:
.
Cloned my project from Git *
pip install -r requiernments.txt
pip3 install gunicorn
sudo apt-get install nginx -y
sudo apt-get install supervisor -y
cd /etc/supervisor/conf.d
sudo touch testapp2.conf
sudo nano testapp2.conf
Updated config file same as below
[program:gunicorn]
directory=/home/ubuntu/projects/testapp2/testerapp
command=/home/ubuntu/projects/testapp2/venv/bin/gunicorn --workers 3 --bind unix:/home/ubuntu/projects/testapp2/testerapp/app.sock testerapp.wsgi:application
autostart=true
autorestart=true
stderr_logfile=/home/ubuntu/projects/testapp2/log/gunicorn.err.log
stdout_logfile=/home/ubuntu/projects/testapp2/log/gunicorn.out.log
[group:guni]
programs:gunicorn
*----------
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl status
The below steps will work and Site available on browser if there is only One configuration above. But when i have added an additional configuration, it shows 502 Bad Gateway on the Browser. Please help me to solve this issue.
You can add one more config file in supervisor conf.d and use different port numbers in the different Django apps.

Docker file throwing error when i try to run it "AH00111: Config variable ${APACHE_RUN_DIR} is not defined"

I am trying my hands with Docker.
I am trying to install apche2 into ubuntu images.
FROM ubuntu
RUN echo "welcome to yellow pages"
RUN apt-get update
RUN apt-get install -y tzdata
RUN apt-get install -y apache2
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
RUN echo 'Hello, docker' > /var/www/index.html
ENTRYPOINT ["/usr/sbin/apache2"]
CMD ["-D", "FOREGROUND"]
I found a reference online reference
I have added this line "RUN apt-get install -y tzdata" because it was asking for an option of tzdata and stopping image creation.
Now when I run my image I am getting the below error
[Thu Jan 07 09:43:57.213998 2021] [core:warn] [pid 1] AH00111: Config variable ${APACHE_RUN_DIR} is not defined
apache2: Syntax error on line 80 of /etc/apache2/apache2.conf: DefaultRuntimeDir must be a valid directory, absolute or relative to ServerRoot
I am new to docker and it's a bit of a task for me to understand it.
Could anyone help me out of this?
This seems to be Apache issue, not docker issue. Your conf seems to have errors. You have a parameter there called DefaultRuntimeDir which is pointing ad directory which does not exist in docker. Review your config file and ensure directories you specified in there exist in docker.
You can play within docker by simply:
docker build -t my_image_name .
docker run -it --rm --entrypoint /bin/bash my_image_name
# now you are in your docker container, you can check if your directories exist
Without knowing your config I would simply add one more RUN (I made this path up, you can change it to whatever you like)
ENV APACHE_RUN_DIR /var/lib/apache/runtime
RUN mkdir -p ${APACHE_RUN_DIR}
As a side note I would also combine all RUN into single like this:
RUN echo "welcome to yellow pages" \
&& apt-get update \
&& apt-get install -y tzdata apache2 \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /var/www \
&& echo 'Hello, docker' > /var/www/index.html

Correct gunicorn.conf to get environmental variables for Django use

I am deploying a Django app on my VPS using Nginx as the web server and Gunicorn installed in virtualenv. (I am using virtualenv with virtualenvwrapper.)
When I run Gunicorn like this, environment variables (such as database password, name) can be found:
workon virtual_env_name
# from my_project's root path
gunicorn my_project.wsgi --bind localhost:9000 --daemon
# this works
I exported the environment variables this way:
# /home/user_name/Envs/virtual_env_name/bin/postactivate
export DATABASE_PASSWORD="secret_password"
However the way below does not (whether or not virtual_env_name is activated):
sudo service gunicorn start
# env variables can't be found - KeyError thrown
This is how my gunicorn.conf script looks like:
start on (local-filesystems and net-device-up IFACE=eth0)
stop on runlevel [!12345]
# If the process quits unexpectadly trigger a respawn
respawn
setuid user_name
setgid www-data
chdir /home/user_name/my_project
exec /home/user_name/Envs/virtual_env_name/bin/gunicorn \
--name=my_project \
--pythonpath=/home/user_name/my_project \
--bind=127.0.0.1:9000 \
my_project.wsgi:application
I can confirm this gunicorn.conf works if I hard code all the password, keys into my Django's settings.py instead of using os.environ[..].
What do I need to do to make my environment variables found when I start Gunicorn with sudo service start? What's the difference between the first and second way? Thanks.
You need to define those variables inside gunicorn.conf.
env DATABASE_PASSWORD="secret_password"
You don't run the code in virtualenv.
Instead of exec use
pre-start script
workon virtual_env_name
end script
script
gunicorn \
--name=my_project \
--pythonpath=/home/user_name/my_project \
--bind=127.0.0.1:9000 \
my_project.wsgi:application
end script
workon update Your $PATH env variable.

gunicorn not starting workers

When i run this command
[jenia#arch app]../bin/gunicorn zones.wsgi:application --bind localht:8000
The gunicorn server runs at localhost:8000. It doesnt return anything to the console as I assume it should. Just runs silently.
When I run my script in bin/gunicorn_start the server still runs silently and features odd behaviour. If I input an address that django can't resolve it gives me internal server error and that's it. no stack trace no nothing.
This is the bin/gunicorn_start script:
#!/bin/bash
NAME="hello_app" # Name of the application
DJANGODIR=/srv/http/proj05/app # Django project directory
SOCKFILE=/srv/http/proj05/app/run/gunicorn.sock # we will communicte using this unix socket
USER=jenia # the user to run as
GROUP=jenia # the group to run as
NUM_WORKERS=3 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=zones.settings # which settings file should Django use
DJANGO_WSGI_MODULE=zones.wsgi # WSGI module name
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
echo "about to exec exec is" $DJANGO_WSGI_MODULE
exec ../bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP \
--log-level=debug \
--bind=unix:$SOCKFILE
By the way, I created a virtualen at by doing:
cd proj05
virtualenv .
source bin/activate
pip install django
pip install gunicorn
...
Can anyone tell me how to make gunicorn output the debug information instead of just internal server error?
Thanks in advance.
gunicorn doesn't return to the console by default now. Use the option --log-file=- to do it.
Also the error should be fixed in https://github.com/benoitc/gunicorn/issues/785 .
I will make a release tomorrow.
I was able to fix this problem by reverting back to Gunicorn 18.0.0.
pip uninstall gunicorn
pip install gunicorn==18.0.0
Not the ideal solution. Perhaps it's worth making a bug ticket about this problem. My concern is that I can't actually identify what the problem is...so how do I make a proper bug ticket? haha
You should use --log-file=- option
For more information see: http://gunicorn-docs.readthedocs.org/en/latest/faq.html#why-i-don-t-see-any-logs-in-the-console