Gunicorn error 203 while setting up django project - django

I try setting up a django project with nginx and gunicorn but get a gunicorn error 203:
My installation is located at path /webapps/sarlex and all project files are owned by user "sarlex" and group "webapps".
The virtualenv is located at /webapps/sarlex/environment.
Gunicorn is located at /webapps/sarlex/environment/bin/ and owned by user sarlex.
Gunicorn configuration is set in /webapps/sarlex/gunicorn_start.sh:
NAME="sarlex" # Name of the application
DJANGODIR=/webapps/sarlex/ # Django project directory
SOCKFILE=/webapps/sarlex/run/gunicorn.sock # we will communicte using this unix socket
USER=sarlex # the user to run as
GROUP=webapps # the group to run as
NUM_WORKERS=9 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=sarlex.settings # which settings file should Django use
DJANGO_WSGI_MODULE=sarlex.wsgi # WSGI module name
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source /webapps/sarlex/environment/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
exec gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP \
--bind=unix:$SOCKFILE \
--log-level=debug \
--log-file=-
Executing gunicorn_start.sh works with environment activated and deactivated.
/etc/systemd/system/gunicorn.service is owned by root and has this content:
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=sarlex
Group=webapps
WorkingDirectory=/webapps/sarlex
ExecStart=/webapps/sarlex/gunicorn_start.sh
[Install]
WantedBy=multi-user.target
I tried to replace "User=Sarlex" with "User=root" but still get the same error.
The correct gunicorn it taken
Thanks for your help

What exactly is your question?
The export for DJANGO_SETTINGS_MODULE was awkward.
Enclose expressions and variable references in double quotes
NAME="sarlex" # Name of the application
DJANGODIR=/webapps/sarlex/ # Django project directory
SOCKFILE=/webapps/sarlex/run/gunicorn.sock # we will communicte using this unix socket
USER=sarlex # the user to run as
GROUP=webapps # the group to run as
NUM_WORKERS=9 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=sarlex.settings # which settings file should Django use
DJANGO_WSGI_MODULE=sarlex.wsgi # WSGI module name
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
source /webapps/sarlex/environment/bin/activate
cd "$DJANGODIR"
export DJANGO_SETTINGS_MODULE
export PYTHONPATH="$DJANGODIR":$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR="$(dirname $SOCKFILE)"
test -d "$RUNDIR" || mkdir -p "$RUNDIR"
# Start your Django Unicorn
exec gunicorn "${DJANGO_WSGI_MODULE}:application" \
--name "$NAME" \
--workers "$NUM_WORKERS" \
--user="$USER" --group="$GROUP" \
--bind="unix:$SOCKFILE" \
--log-level=debug \
--log-file=-
Please show /var/log/nginx/ error messages.

Django failed starting because in one of my scripts I wrote "admin" in a path instead of "Admin".
This didn't cause a problem on my windows PC where I am developing my code.

Related

How to config supervisor with Django channels and server daphne

I have a problem with my configuration supervisor, my file is in etc/supervisor/conf.d/realtimecolonybit.conf,
When I try command supervisorctl reread, show me the "No config updates to processes" and when I try the other command like this
supervisorctl status realtimecolonybit
Shows me this error
realtimecolonybit FATAL can't find command '/home/ubuntu/realtimecolonybit/bin/start.sh;'
And when try the supervisorctl start realtimecolonybit
show me this error
realtimecolonybit: ERROR (no such file)
My configuration in my file realtimecolonybit.conf is below
[program:realtimecolonybit]
command = /home/ubuntu/realtimecolonybit/bin/start.sh;
user = root
stdout_logfile = /home/ubuntu/realtimecolonybit/logs/realtimecolonybit.log;
redirect_strderr = true;
My configuration from my file start.sh is below
#!/bin/bash
NAME="realtimecolonybit"
DJANGODIR=/home/ubuntu/realtimecolonybit/colonybit
SOCKFILE=/home/ubuntu/realtimecolonybit/run/gunicorn.sock
USER=root
GROUP=root
NUM_WORKERS=3
DJANGO_SETTINGS_MODULE=colonybit.settings
echo "Starting $NAME as `whoami`"
cd $DJANGODIR
source /home/ubuntu/realtimecolonybit/bin/activate
# workon realtimecolonybit
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPAHT=$DJANGODIR:$PYTHONPATH
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
exec daphne -b 0.0.0.0 -p 8001 colonybit.asgi:application
When I run without supervisor like this
(realtimecolonybit)realtimecolonybit/#/ ./bin/start.sh
it's run ok and working well, but sometimes down the server
I try to run a Django 1.11 and django_channel with supervisor my app is in aws.
I solve my problem, the error was in file .conf, I remove ; and remove the .sh, change the start.sh to start
wrong command
command = /home/ubuntu/realtimecolonybit/bin/start.sh;
correct command
command = /home/ubuntu/realtimecolonybit/bin/start

Celery with Supervisord workers are not working Isolated

.conf file
[program:task1]
directory=/home/ubuntu/proj1
command=/usr/bin/python3 /usr/local/bin/celery -A proj1 worker -l info --concurrency=10 -n proj1_worker#%%h
user=ubuntu
numprocs=1
stdout_logfile=/var/log/proj1_celeryd.log
stderr_logfile=/var/log/proj1_celeryd.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=600
priority=998
[program:task2]
directory=/home/ubuntu/proj2/
command=/usr/bin/python3 /usr/local/bin/celery -A proj2 worker -l info --concurrency=10 -n proj2_worker#%%h
user=ubuntu
numprocs=1
stdout_logfile=/var/log/proj2_celeryd.log
stderr_logfile=/var/log/proj2_celeryd.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs=600
priority=998
[group:celeryworkers]
programs=task1,task2
proj1_worker and proj2_worker are not getting isolated.
At first, always proj1_worker is called even I called proj2_worker
I don't know where I am going wrong. Kindly assist.
Thank you in advance
First of all I really recommend you to use virtualenv for each project. Create 2 separate virtualenvs (you can specify your own location), see https://docs.python.org/3/library/venv.html.
python3 -m venv /home/ubuntu/virtualenvs/proj1
python3 -m venv /home/ubuntu/virtualenvs/proj2
Activate virtualenv and install celery:
source /home/ubuntu/virtualenvs/proj1/bin/activate
pip install --upgrade celery
source /home/ubuntu/virtualenvs/proj2/bin/activate
pip install --upgrade celery
Your supervisor configuration should then look like this:
[program:task1]
directory=/home/ubuntu/proj1
command=/home/ubuntu/virtualenvs/proj1/bin/celery worker -A proj1 -l info --concurrency=10 -n proj1_worker#%%h
# ...
[program:task2]
directory=/home/ubuntu/proj2
command=/home/ubuntu/virtualenvs/proj1/bin/celery worker -A proj2 -l info --concurrency=10 -n proj2_worker#%%h
# ...
Next, create 2 separate virtual hosts for your projects:
rabbitmqctl add_user proj_1 <PASSWORD>
rabbitmqctl add_vhost proj_1_vhost
rabbitmqctl set_permissions -p proj_1_vhost proj_1 ".*" ".*" ".*"
rabbitmqctl add_user proj_2 <PASSWORD>
rabbitmqctl add_vhost proj_2_vhost
rabbitmqctl set_permissions -p proj_2_vhost proj_2 ".*" ".*" ".*"
Finally modify celery configuration to use newly created virtual hosts:
app = Celery('proj1_celery_app')
app.conf.update(
# ...
broker_url='amqp://proj1:<PASSWORD>#localhost:5672/proj_1_vhost'
# ...
)
app = Celery('proj2_celery_app')
app.conf.update(
# ...
broker_url='amqp://proj2:<PASSWORD>#localhost:5672/proj_2_vhost'
# ...
)
For more info about rabbit vhosts see this SO post: Running multiple instances of celery on the same server.

Gunicorn settings for virtualenvwrapper

I am trying to deploy a test site which made using Django and virtualenvwrapper. I want to use nginx for requests.I used Taskbuster's tutorial. So my project layer is similar as below :
--abctasarim **main folder
--manage.py **django manage file
----/yogavidya ** project folder
----/yogavidya/wsgi.py **wsgi file
----/yogavidya/settings/base.py ***settings
I prepared a script to use with gunicorn. I addressed virtualenv to virtualenvwrapper envs
#!/bin/bash
NAME="yogavidya" #Name of the application (*)
DJANGODIR=/home/ytsejam/public_html/abctasarim # Django project directory (*)
SOCKFILE=/home/ytsejam/public_html/abctasarim/run/gunicorn.sock # we will communicate using this unix socket (*)
USER=ytsejam # the user to run as (*)
GROUP=webdata # the group to run as (*)
NUM_WORKERS=1 # how many worker processes should Gunicorn spawn (*)
DJANGO_SETTINGS_MODULE=yogavidya.settings.base # which settings file should Django use (*)
DJANGO_WSGI_MODULE=yogavidya.wsgi # WSGI module name (*)
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source /home/ytsejam/.virtualenvs/yv_dev/bin/activate
#export /home/ytsejam/.virtualenvs/yv_dev/bin/postactivate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec /home/ytsejam/public_html/abctasarim/gunicorn \
--name $NAME \
--workers $NUM_WORKERS \
--env DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE \
--pythonpath $DJANGODIR \
--user $USER \
--bind=unix:$SOCKFILE yogavidya.wsgi:application
When I try to run it , I am getting an error for my service file :
...
ImportError: No module named ' '
...
How can I fix my script to serve the site correctly ?
Thanks
virtualenvwrapper is supposed to be what you use in development. You want to deploy using the package that virtualenvwrapper is built on—the virtualenv package. The best suggestion I have for you for how to make this work would be to try the steps that you would normally use to start your virtualenvwrapper environment, namely sourcing the shell script and then using workon:
NAME="yogavidya" #Name of the application (*)
DJANGODIR=/home/ytsejam/public_html/abctasarim # Django project directory (*)
SOCKFILE=/home/ytsejam/public_html/abctasarim/run/gunicorn.sock # we will communicate using this unix socket (*)
USER=ytsejam # the user to run as (*)
GROUP=webdata # the group to run as (*)
NUM_WORKERS=1 # how many worker processes should Gunicorn spawn (*)
DJANGO_SETTINGS_MODULE=yogavidya.settings.base # which settings file should Django use (*)
DJANGO_WSGI_MODULE=yogavidya.wsgi # WSGI module name (*)
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGO_DIR
source /path/to/virtualenvwrapper.sh
workon yv_dev
You should also just try invoking gunicorn from the command line after activating your virtualenv.
Here’s how you can do it with virtualenv:
cd /home/ytsejam/public_html/abctasarim
sudo pip install virtualenv
virtualenv .
. bin/activate
pip install -r requirements.txt
pip install gunicorn
gunicorn script:
NAME="yogavidya" #Name of the application (*)
DJANGODIR=/home/ytsejam/public_html/abctasarim # Django project directory (*)
SOCKFILE=/home/ytsejam/public_html/abctasarim/run/gunicorn.sock # we will communicate using this unix socket (*)
USER=ytsejam # the user to run as (*)
GROUP=webdata # the group to run as (*)
NUM_WORKERS=1 # how many worker processes should Gunicorn spawn (*)
cd $DJANGO_DIR
. bin/activate
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
exec /home/ytsejam/public_html/abctasarim/bin/gunicorn \
--name $NAME \
--workers $NUM_WORKERS \
--user $USER \
--bind=unix:$SOCKFILE yogavidya.wsgi:application

couldn't exec gunicorn_start EACCES

#!/bin/bash
NAME="trueguild" # Name of the applica$
DJANGODIR=/webapps/trueguild_django/trueguild # Django project dire$
SOCKFILE=/webapps/trueguild_django/run/gunicorn.sock # we will communicte $
USER=trueguild # the user to run as
GROUP=webapps # the group to run as
NUM_WORKERS=3 # how many worker pro$
DJANGO_SETTINGS_MODULE=trueguild.settings # which settings file$
DJANGO_WSGI_MODULE=trueguild.wsgi # WSGI module name
echo "Starting $NAME as $NAME"
# Activate the virtual environment
cd $DJANGODIR
source ../bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do$
exec ../bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER --group=$GROUP \
--bind=unix:$SOCKFILE \
--log-level=debug \
--log-file=-
I have the above gunicorn_start file.But when i start using supervisorctl then it throws this error
supervisor: couldn't exec /webapps/trueguild_django/bin/gunicorn_start: EACCES
supervisor: child process was not spawned
i am using django along with nginx and gunicorn.
How can i start the server using supervisor?
here is the supervisor.d conf file
; supervisor config file
[unix_http_server]
file=/var/run/supervisor.sock ; (the path to the socket file)
chmod=0700 ; sockef file mode (default 0700)
[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor ; ('AUTO' child log dir, default $TEMP)
; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL for a unix socket
; The [include] section can just contain the "files" setting. This
; setting can list multiple files (separated by whitespace or
; newlines). It can also contain wildcards. The filenames are
; interpreted as relative to this file. Included files *cannot*
; include files themselves.
[include]
files = /etc/supervisor/conf.d/*.conf
and here is the trueguild.conf file inside conf.d directory
[program:trueguild]
command = /webapps/trueguild_django/bin/gunicorn_start ; Command to start app
user = trueguild ; User to run as
stdout_logfile = /webapps/trueguild_django/logs/gunicorn_supervisor.log ; Where to write log messages
redirect_stderr = true ; Save stderr in the same log
environment=LANG=en_US.UTF-8,LC_ALL=en_US.UTF-8 ; Set UTF-8 as default encoding
In the trueguild.conf file
add sh after command...
it'll be like
command =sh /webapps/trueguild_django/bin/gunicorn_start
It'll work.
Please make sure you have permission to execute gunicorn command under trueguild

supervisor, gunicorn and django only work when logged in

I'm using this guide to setting up an intranet server. Everything goes ok, the server works and I can checked it is working in my network.
But when I logout, I get 404 error.
The sock file is in the path indicated in gunicorn_start.
(cmi2014)javier#sgc:~/workspace/cmi/cmi$ ls -l run/
total 0
srwxrwxrwx 1 javier javier 0 mar 10 17:31 cmi.sock
Actually I can se the workers when listed the process list.
(cmi2014)javier#sgc:~/workspace/cmi/cmi$ ps aux | grep cmi
javier 17354 0.0 0.2 14652 8124 ? S 17:27 0:00 gunicorn: master [cmi]
javier 17365 0.0 0.3 18112 10236 ? S 17:27 0:00 gunicorn: worker [cmi]
javier 17366 0.0 0.3 18120 10240 ? S 17:27 0:00 gunicorn: worker [cmi]
javier 17367 0.0 0.5 36592 17496 ? S 17:27 0:00 gunicorn: worker [cmi]
javier 17787 0.0 0.0 4408 828 pts/0 S+ 17:55 0:00 grep --color=auto cmi
And supervisorctl responds that the process is running:
(cmi2014)javier#sgc:~/workspace/cmi/cmi$ sudo supervisorctl status cmi
[sudo] password for javier:
cmi RUNNING pid 17354, uptime 0:29:21
There is an error in nginx logs,
(cmi2014)javier#sgc:~/workspace/cmi/cmi$ tail logs/nginx-error.log
2014/03/10 17:38:57 [error] 17299#0: *19 connect() to
unix:/home/javier/workspace/cmi/cmi/run/cmi.sock failed (111: Connection refused) while
connecting to upstream, client: 10.69.0.174, server: , request: "GET / HTTP/1.1",
upstream: "http://unix:/home/javier/workspace/cmi/cmi/run/cmi.sock:/", host:
"10.69.0.68:2014"
Again, the error appears only when I logout or close session, but everything works fine when run or reload supervisor and stay connected.
By the way, ngnix, supervisor and gunicorn run on my uid.
Thanks in advance.
Edit Supervisor conf
[program:cmi]
command = /home/javier/entornos/cmi2014/bin/cmi_start
user = javier
stdout_logfile = /home/javier/workspace/cmi/cmi/logs/cmi_supervisor.log
redirect_stderr = true
autostart=true
autorestart=true
Gnunicor start script
#!/bin/bash
NAME="cmi" # Name of the application
DJANGODIR=/home/javier/workspace/cmi/cmi # Django project directory
SOCKFILE=/home/javier/workspace/cmi/cmi/run/cmi.sock # we will communicte using this unix socket
USER=javier # the user to run as
GROUP=javier # the group to run as
NUM_WORKERS=3 # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=cmi.settings # which settings file should Django use
DJANGO_WSGI_MODULE=cmi.wsgi # WSGI module name
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source /home/javier/entornos/cmi2014/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
export CMI_SECRET_KEY='***'
export CMI_DATABASE_HOST='***'
export CMI_DATABASE_NAME='***'
export CMI_DATABASE_USER='***'
export CMI_DATABASE_PASS='***'
export CMI_DATABASE_PORT='3306'
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec /home/javier/entornos/cmi2014/bin/gunicorn ${DJANGO_WSGI_MODULE}:application --name $NAME --workers $NUM_WORKERS --user=$USER --group=$GROUP --log-level=debug --bind=unix:$SOCKFILE