I need your help!
I started using docker this week, launched all containers for a new Django project. In this project there are several databases, python, django web server + redis, celery, etc. These all are served by separated docker containers and are launched by docker-compose up command.
This is my probjem: when I type docker-compose up in the console, it starts all services. Then I need to restore my databases dumps for each database (it takes about an hour). But when I use pycharm tools for docker-compose, it recreates some containers. And also it recreates all my postgres databases with ALL MY DATA!
Sometimes is doesn't recreate containers and I can do my job, but if I do any wrong move -then docker-compose erases my databases! I have tired to restore them!
Is there way to protect containers from erasing, to forbid recreate my postgres containers?
PS: I've also tried to export postgres containers to .tar file, but when I import it back, database insight the container is ok and container importing is faster than restoring data from sql, but metadata of docker image is different, so I can't use it.
Please, give me any ideas)
Try to use volumes to store your data. Volumes can keep data after containers recreating.
https://docs.docker.com/storage/volumes/
Related
Why do I have to restart my docker container every time I make a change in my Django python files? I'm running a Django app via Docker and it seems any changes I make in my Views(and possibly elsewhere) are not reflected until I restart my container.
For example, if I log an output to the terminal or make changes, then refresh, there's no change. If I restart my container and then refresh, I get the result I would expect.
As I know you can consider Docker environment like immutable type doesn’t allow any change in the object once it has been created.
And that will give us more security.
If you want a solution there is some vs code extensions can run script contain some command after you edit your files, you can insert docker command in this script to reload environment after you edit any file.
I am developing Django Wagtail application on my local machine connected to a local postgres server.
I have a test server and a production server.
However when I develop locally and try upload it there is always some issue with makemigration and migrate e.g. KeyError etc.
What are the best practices of ensuring I do not get run into these issues? What files do I need to port across?
so ill tell you what i do and what most of the companies that i worked as a django developer did and i can tell you by experience that worked pretty well.
First containerize your application, this will make your life much more easy and you will remove external influence in your code, also will get you an easy way to reproduce your environment.
Your Dockerfile should be from some python image and should do 3 basically things:
Install your requirements dependencies
Run the python manage.py migrate --noinput command
Run a http server such as gunicorn with gunicorn -c /gunicorn.py wsgi:application
You ill do the makemigration in your local machine and make sure that everything is working before commit then to the repo.
In your gunicorn.py you ill put your settings to run the app such as the number of CPU, the binding port, the folder that your app is, something like:
import os
import multiprocessing
# Chdir to specified directory before apps loading.
# https://docs.gunicorn.org/en/stable/settings.html#chdir
chdir = '/app/'
# Bind the application on localhost both on ipv6 and ipv4 interfaces.
# https://docs.gunicorn.org/en/stable/settings.html#bind
bind = '0.0.0.0:8000'
Second containerize your other stuff, for example the postgres database, the redis (for cache), a connection pooler for the database depending on the size of your application.
Its highly recommend that you have a step in the pipeline to do tests, they need to run before everything, maybe just after lint
Ok what now? now you need a way to deploy that stuff, the best for that scenario is: pull your image to github registry, and you can add a tag to that for example:
IMAGE_ID=ghcr.io/${{ github.repository_owner }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
docker tag $IMAGE_NAME $IMAGE_ID:staging
docker push $IMAGE_ID:staging
This can be add in a github action in the build step for example.
After having your new code in a new image inside github you just need to update the current one, this can be done by creaaing a script to do it in the server and running that script from github action, is something like:
docker pull ghcr.io/${{ github.repository_owner }}/$IMAGE_NAME
echo 'Restarting Application...'
docker stop {YOUR_CONTAINER} && docker up -d
sudo systemctl restart nginx
echo 'Cleaning old images'
sudo docker system prune -af
You can see that i create the image with a staging tag, you can create a rule in github actions for example to trigger that action when you create a new release for example, and create another action to be trigger in every new commit and build/deploy for a dev tag.
For the migration problem, the first thing is, when your application go live squash every migration to the first one (you can drop the database and all the migration then create the database and run the makemigration command again to reach this), so you can have a clean migration in the server. Never creates unnecessary relation between the tables, prefer always doing cached properties instead of add new columns, use UUID for unique ids, and try to not do breaking changes in the database, its hard but if you plan the database before is not so difficult to do.
Hit me if you have any questions. A lot of the stuff that i said can be done in a lot of other platforms such as gitlab, travis, circle ci, but i use the github action in the example because i think is more simple to picture.
EDIT:
I forgot to tell you to have a cron in your server doing backups of your databases, the migrate command ill apply the changes only after the verification but if something else break the database this can save your life.
My Django deployment has x number of pods (3 currently)running a Django backend REST API server. We're still in the development/staging phase. I wanted to ask for advice regarding DB migration. Right now the pods simply start by launching the webserver, assuming the database is migrated and ready. This assumption can be wrong of course.
Can I simply put python manage.py migrate before running the server? What happens if 2 or 3 pods are started at the same time and all run migrations at the same time? would there be any potential damage or problem from that? Is there a best practice pattern to follow here to ensure that all pods start the server with a healthy migrated database?
I was thinking about this:
During initial deployment, define a Kubernetes Job object that'll run once, after the database pod is ready. It will be using the same Django container I have, and will simply run python manage.py migrate. the script that deploys will kubectl wait for that job pod to finish, and then apply the yaml that creates the full Django deployment. This will ensure all django pods "wake up" with a database that's fully migrated.
In subsequent updates, I will run the same job again before re-applying the Django deployment pod upgrades.
Now there is a question of chicken and egg and maintaining 100% uptime during migration, but this is a question for another post: How do you apply data migrations that BREAK existing container Version X when the code to work with the new migrations is updated in container Version X+1. Do you take the entire service offline for the duration of the update? is there a pattern to keep service up and running?
Well you are right about the part that multiple migrate commands will run against your database by multiple pods getting started.
But this will not cause any problems. When you are going to make actual changes to your database, if the changes are already applied, your changes will be ignored. So, say 3 pods start at the same time and run the migrate command. Only One of those commands will end up applying changes to the database. Migrations normally need to lock the database for different actions (this is highly related to your DBMS). The lock will happen by one of the migrate commands (one of the pods) and other commands should wait until the work of the first one is over. After the job is done by the first one, others' commands will be ignored automatically. So each migration will happen once.
You can however, change your deployment strategy and ask kubernetes to first, spin up only 1 pod and when the first pod's health check succeeds, others will spin up too. In this case, you can be sure that the lock time for the migration, will happen only once and others will just check that migrations are already applied and ignore them automatically.
You may use Kubernetes init containers, which are specialized containers that run before app containers in a Pod. The init containers stop after successfully executing the commands you want, so they won't occupy unnecessary resources.
Here is the official link:
https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
The best thing you can do is to use kind:Jobs, an run this before the rollout.
Please check: Django migrations by Kubernetes Job and persistent Volume Claim
As I understand In K8s you can't control the order of the pods.
Let's say your DB host is a docker container (Dev environment) and not an RDS/ static host, migrations command could not be executed before the DB is up and ready.
I try to to use the ready function in app.py to execute run_migrations.py that execute shell script of run migrations but run migrations trigger app.py and it's infinity loop.
In my organization We use circleCI which build the image first and then push it to AWS ECR with tags before deploy it to EKS. At the build stage I can't assume the DB pod is ready so run migrations will cause an error.
When the DB for Django is also a K8s pod, the best way to execute it will be a job.
I am building a Python+Django development environment using docker. I defined Dockerfile files and services in docker-compose.yml for web server (nginx) and database (postgres) containers and a container that will run our app using uwsgi. Since this is a dev environment, I am mounting the the app code from the host system, so I can easily edit it in my IDE.
The question I have is where/how to run migrate command.
In case you don't know Django, migrate command creates the database structure and later changes it as needed by the project. I have seen people run migrate as part of the compose command directive command: python manage.py migrate && uwsgi --ini app.ini, but I do not want migrations to run on every container restart. I only want it to run once when I create the containers and never run again unless I rebuild.
Where/how would I do that?
Edit: there is now an open issue with the compose team. With any luck, one time command containers will get supported by compose. https://github.com/docker/compose/issues/1896
You cannot use RUN because as you mentioned in the comments your source is mounted during running of the container.
You cannot use CMD either since you don't want it to run everytime you restart the container.
I recommend using docker exec manually after running the container. I do not think there is a way to automate this inside a dockerfile or docker-compose because of the two reasons I gave above.
It sounds like what you need is a tool for managing project tasks. dobi is a tool designed to handle these tasks (disclaimer: I am the author of this tool).
You can see an example of how to run a migration here: https://github.com/dnephin/dobi/tree/master/examples/init-db-with-rails. The example uses rails, but it's basically the same idea as django.
You could setup a task called migrate which would run the command in a container and write the data to a volume. Then when you start your docker-compose containers, use that volume as the source for your database service.
https://github.com/docker/compose/issues/1896 is finally resolved now by the new service profiles introduced with docker-compose 1.28.0. With profiles you can mark services to be only started in specific profiles:
services:
nginx:
# ...
postgres:
# ...
uwsgi:
# ...
migrations:
profiles: ["cli-only"] # profile name chosen freely
# ...
docker-compose up # start only your app services, no migrations
docker-compose run migrations # run migrations on-demand
docker exec -it container-name bash
Then you will be inside the container and you can run any command you normally do when you develop without using docker.
I'm trying to find a workflow with Docker and Django. Currently, I'm using the basic configuration from the docker documentation.
I'd like to use manage.py startapp directly from the container to start a new app using:
docker-compose run web ./manage.py startapp myapp
But all the files created in the volume are owned by the root user and not by myself, so I can't edit them from the host.
My idea is to avoid installing all the requirements on my host machine but maybe I should not create app from the container?
One possible solution is to create a user and make it having the same UID/GID than my user on my host machine but it won't work if I try to use an other account on my host machine...
Any suggestion?
What worked best for me was avoiding (or minimizing) file creation inside the containers.
My Dockerfile would just copy the requirements.txt and install them;
and the container would access the app files through a mounted volume.
I pass the env var PYTHONDONTWRITEBYTECODE=1 to the containers, so python does not create *.pyc/*.pyo files.
The few times I cannot avoid it (like, ./manage.py makemigrations), I run chown afterwards.
It's not ideal, but as this happens rarely for my case, I don't bother.