I have a Django site running in Docker containers, which uses docker-compose to manage the various containers (database, nginx, etc.). There are a few Django tasks that I use for site maintenance using the Django manage.py command. They commands take the form of:
manage.py updateflickr --settings=mysite.myproj.prod
Running under docker-compose, they look like:
docker-compose run --rm app manage.py updateflickr --settings=mysite.myproj.prod
My problem is that when I try to run these same commands using Fabric, it appears that the settings file I am specifying is not being used. Django is returning database connection errors, which typically mean that it is not getting the correct database information, or in this case the connection specified in mysite.myprod.prod
My Fabric file looks like:
import os
from fabric.api import *
env.hosts = ['myserver.com']
env.user = "myuser"
env.key_filename = '~/.ssh/do_rsa'
env.shell = '/bin/bash -c'
#task
def updateflickr():
run('docker-compose run --rm app python manage.py updateflickr --settings=mysite.myproj.prod')
I have also expirimented with setting the DJANGO_SETTINGS_MODULE environment variable in my docker-compose.yml but am getting the same results. Finally, the last thing I tried was wrapping the command in a shell script. Same results - if I run on the server, it runs fine. If I run the shell script from Fabric, I get database connection issues.
UPDATE
I am not so sure this is so much a question about Fabric, then a question about how docker-compose runs. If I try the following:
ssh -t me#myserver.com 'docker-compose run --rm app python manage.py updateflickr --settings=mysite.myproj.prod'
I still get the same results. There must be something different about loading up an interactive shell with just sending a command. I have tried using ssh with and without a -t flag, because docker-compose might need a pty active.
Related
I try to debug an app inside of the container and I need to run django to run tests I do that by:
docker-compose exec django bash
But as a result I get:
service "django" is not running container #1
I don't really understand what this response means and I didn't find any information regarding that. This issue prevents me from being able to debug a code inside of the container with the database up and running.
if docker ps show a container name "django"
then run if you want to get bash prompt in:
docker exec -ti django bin/bash
if bash not available you can try bin/sh
if container isn't running you can't go in.
Try to add in settings.py:
ALLOWED_HOSTS = ['*']
and
Debug = True
I'm developing a module with the crontab.
Actually, the framework I'm using is django so I did install 'django-crontab'
I did test as the instruction did and make it with localhost environment.
When I deployed("sudo service apache2 restart") it on AWS after doing a command 'python manage.py crontab add', it didn't work.
I thiknk it's working on only localhost environment, isn't it?
How can I solve this problem?
If you have more than one profile in your django settings, you shuold specify one before add crontab. if not specified, django crontab run as default environment, which is develop mostly. To run it on product environment , you should do these:
specify crontab enviromen in settings.product.py, something like
CRONTAB_DJANGO_SETTINGS_MODULE = 'gold.settings.product'
specify settings profile and add crontab
export MYPROJECT_PROFILE = product
python manage.py crontab add
I am building a Python+Django development environment using docker. I defined Dockerfile files and services in docker-compose.yml for web server (nginx) and database (postgres) containers and a container that will run our app using uwsgi. Since this is a dev environment, I am mounting the the app code from the host system, so I can easily edit it in my IDE.
The question I have is where/how to run migrate command.
In case you don't know Django, migrate command creates the database structure and later changes it as needed by the project. I have seen people run migrate as part of the compose command directive command: python manage.py migrate && uwsgi --ini app.ini, but I do not want migrations to run on every container restart. I only want it to run once when I create the containers and never run again unless I rebuild.
Where/how would I do that?
Edit: there is now an open issue with the compose team. With any luck, one time command containers will get supported by compose. https://github.com/docker/compose/issues/1896
You cannot use RUN because as you mentioned in the comments your source is mounted during running of the container.
You cannot use CMD either since you don't want it to run everytime you restart the container.
I recommend using docker exec manually after running the container. I do not think there is a way to automate this inside a dockerfile or docker-compose because of the two reasons I gave above.
It sounds like what you need is a tool for managing project tasks. dobi is a tool designed to handle these tasks (disclaimer: I am the author of this tool).
You can see an example of how to run a migration here: https://github.com/dnephin/dobi/tree/master/examples/init-db-with-rails. The example uses rails, but it's basically the same idea as django.
You could setup a task called migrate which would run the command in a container and write the data to a volume. Then when you start your docker-compose containers, use that volume as the source for your database service.
https://github.com/docker/compose/issues/1896 is finally resolved now by the new service profiles introduced with docker-compose 1.28.0. With profiles you can mark services to be only started in specific profiles:
services:
nginx:
# ...
postgres:
# ...
uwsgi:
# ...
migrations:
profiles: ["cli-only"] # profile name chosen freely
# ...
docker-compose up # start only your app services, no migrations
docker-compose run migrations # run migrations on-demand
docker exec -it container-name bash
Then you will be inside the container and you can run any command you normally do when you develop without using docker.
We use jenkins as continious integration system. We have two django servers validated by jenkins.
jenkins validates successully the first server. The second server depends on the first one. Thus we would like to launch at the end of the first server validation the first server itself.
We are using python, virtualenv and django and defined the Virtualenv Builder as follow:
pip install -r requirements.txt
rm -f .coverage
fab localhost test
coverage xml
nohup python manage.py runserver 9090 &
The issue is that the build never ends due to the nohup.
How can I launch the server after a successful build?
I had the same problem.
Ken,
I tried using fabric, but again python manage.py runserver - runs continuosly, so the next command is not starting.
And just few mins ago my collegue showed me how to use nohup and with variable BUILD_ID of Jenkins it would be like this to get Success from the build and leave the Django server running:
BUILD_ID=dontKillMe nohup python manage.py runserver host_server &
This worked for our Django project testing.
Since you are using fabric to test, I would recommend defining another fabric task, say, deploy, which you could call assuming the build succeeds.
Much like the call to fab completes for a successful build such that you get to the nohup line, I would expect the deploy task to return also.
You may also want to consider making the server a service (either via an /etc/init.d style script, or upstart if Ubuntu), and have the fabric task stop the currently running one, copy over whatever new files it needs (or similar process), and then restart it.
Assuming what you have above is a bash script or similar, you may want to also define set -e so that, in case any of the commands returns a non-success code, the script will fail, and in turn, fail the build.
I've been trying to trudge through the docs and examples to get my Django running through upstart so I can have it running all the time but am unable to so.
Here's my upstart configuration file located at /etc/init/myapp.conf:
start on startup
#expect daemon
#respawn
console output
script
chdir /app/env/bin
exec source activate
exec /app/env/bin/python /app/src/manage.py runserver 0.0.0.0:8000 > /dev/null 2>&1 &
end script
When I type sudo service myapp start, the console says that it has started but it doesn't seem to be running.
Is it possible to see some debugging output to see what's going wrong?
I need to run my Django application as another user — i.e. djangouser. How can I do so?
(I've been commenting out some lines to test where the service is going wrong). This is not for production use but my internal development use only.
Thanks.
Edit #1:
I have wrapped both my commands into a simple script at /app/run.sh
#!/bin/bash
cd /app/env/bin
source activate
cd /app/src
python manage.py runserver 0.0.0.0:8000 > /dev/null 2>&1 &
..and I've modified my /etc/init/myapp.conf to
start on startup
expect daemon
exec su - djangouser -c "bash /app/run.sh"
When executing sudo service myapp start — the application starts but the PID is wrong and I can't seem to kill it with sudo service myapp stop
Any ideas?
Change:
exec source activate
By just:
source activate
This will load the virtual environment. You should probably drop the other "exec". If that doesn't work, please post your upstart logs.
A couple of remarks:
logging the output to somewhere else than /dev/null might be useful :)
runserver is not ment to be stable, I see it crashing sometimes and in that case i guess you'll need to force upstart to reload, or put the runserver call in a while loop
you will not be able to use an interactive debugger like ipdb with this setup
How about using nginx and uwsgi with your virtualenv. this will give you a production like environment but will also start your django app at start up. if you are using ubuntu 10 you should take a look at uwsgi-python, otherwise just install the latest uwsgi. i usually start my virtualenv in uwsgi like so : sudo nano /etc/uwsgi-python/apps-available/app.xml
<uwsgi>
<socket>127.0.0.1:8889</socket>
<pythonpath>/home/user/code/</pythonpath>
<virtualenv>/home/user/code</virtualenv>
<pythonpath>/home/user/code/app</pythonpath>
<app mountpoint="/">
<script>uwsgiApp</script>
</app>
</uwsgi>
also setup yournginx files at /etc/nginx/apps-available/default (the file is a bit straight forward). this will help you have your django app at all times,
su is problematic becouse it forks the process. You can use sudo -u djangouser instead or simply add
setuid djangouser
in your conf file.
This should work on Ubuntu 14.04 and possibly other versions as well:
root#vagrant-ubuntu-trusty-64:/etc/init# service my_app start
my_app start/running, process 7799
root#vagrant-ubuntu-trusty-64:/etc/init# cat /var/log/upstart/my_app.log
Performing system checks...
System check identified no issues (0 silenced).
You have unapplied migrations; your app may not work properly until they are applied.
Run 'python manage.py migrate' to apply them.
June 30, 2015 - 06:54:18
Django version 1.8.2, using settings 'my_test.settings'
Starting development server at http://0.0.0.0:8080/
Quit the server with CONTROL-C.
root#vagrant-ubuntu-trusty-64:/etc/init# service my_app status
my_app start/running, process 7799
root#vagrant-ubuntu-trusty-64:/etc/init# service my_app stop
my_app stop/waiting
root#vagrant-ubuntu-trusty-64:/etc/init# service my_app status
my_app stop/waiting
Here is the config to make it work:
root#vagrant-ubuntu-trusty-64:/etc/init# cat my_app.conf
description "my_app upstart script"
start on runlevel [23]
respawn
script
su vagrant -c "source /home/vagrant/dj_app/bin/activate; /home/vagrant/dj_app/bin/python /home/vagrant/my_test/manage.py runserver 0.0.0.0:8080"
end script