service refers to undefined volume - django

I want to share temp files between Django project and celery worker (it works with TemporaryUploadedFiles, so I want to have access to these files from celery worker to manage them). I've read about shared volumes, so I tried to imlement it in my docker-compose file and run it, but the command gave me this error:
$ docker compose up --build
service "web" refers to undefined volume shared_web_volume/: invalid compose project
And sometimes "web" replace with "celery", so both celery and django have no access to this volume.
Here is my docker-compose.yml file:
volumes:
shared_web_volume:
postgres_data:
services:
db:
image: postgres:12.0-alpine
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- ./.env
web:
build:
context: ./MoreEnergy
dockerfile: Dockerfile
entrypoint: sh ./entrypoint.sh
command: python manage.py runserver 0.0.0.0:8000
volumes:
- "shared_web_volume/:/MoreEnergy/"
ports:
- 1337:8000
env_file:
- ./.env
depends_on:
- db
celery:
build:
context: ./MoreEnergy
dockerfile: Dockerfile
entrypoint: sh ./entrypoint.sh
command: celery -A MoreEnergy worker --loglevel=info
volumes:
- "shared_web_volume/:/MoreEnergy/"
env_file:
- ./.env
depends_on:
- web
- redis
redis:
image: redis:5-alpine
What am I doing wrong?
Upd: temp dir is my project folder (I've set it with FILE_UPLOAD_TEMP_DIR variable in settings file), so I don't need to make one more volume only for shared temp files (If I have to, tell me).

your shared volume name is shared_web_volume and it's different from the one that you use in the volumes section which is shared_web_volume/
so my suggestion is to erase the forward slash to be like this
volumes:
- "shared_web_volume:/MoreEnergy/"
Don't forget to do the same thing in celery container

Related

Images loss every time on server when run "docker-compose up --build" + Django

I had a setup docker on Django project, images are stored on server but whenever I upload changes on server and then run "docker-compose up --build" then it loss every images that I uploaded, path showing images there but I am not able to view it. I don't know want happened there. if anyone have idea what to do for that ? How to resolved this issue ? is there any things need to add in docker-compose file ?
Below is my docker compose file.
version: '3'
volumes:
web-django:
web-static:
services:
redis:
image: "redis:alpine"
web:
build: .
image: project_web
command: python manage.py runserver 0.0.0.0:8000
restart: always
volumes:
- web-django:/usr/src/app
- web-static:/usr/src/app/mediafiles
ports:
- "8000:8000"
depends_on:
- redis
celery:
image: project_web
command: celery -A project_name worker -l info
restart: always
depends_on:
- web
- redis
celery-beat:
image: citc_web
command: celery -A project_name beat -l info
restart: always
depends_on:
- web
- redis
I want solution of it, what i missing on above file and want to resolve it.

django.db.utils.OperationalError: could not translate host name "db" to address: Name does not resolve. How to solve this issue?

Can some body help me solve this issue. Why am i getting this error? I have db in .env host and links, network in docker-compose file too. I am not being to figure out where the issue is being raised.
Here is my docker-compose file.
version: "3.9"
volumes:
dbdata:
networks:
django:
driver: bridge
services:
web:
build:
context: .
volumes:
- .:/home/django
ports:
- "8000:8000"
command: gunicorn Django.wsgi:application --bind 0.0.0.0:8000
container_name: django_web
restart: always
env_file: .env
depends_on:
- db
links:
- db:db
networks:
- django
db:
image: postgres
volumes:
- dbdata:/var/lib/postgresql
environment:
- POSTGRES_DB=${DB_NAME}
- POSTGRES_USER=${DB_USER}
- POSTGRES_PASSWORD=${DB_PASSWORD}
ports:
- 5430:5432
networks:
- django
container_name: django_db
here is my .env with database settings
DB_USER=admin
DB_NAME=test
DB_PASSWORD=admin
DB_HOST=db
DB_PORT=5432
DB_SCHEMA=public
CONN_MAX_AGE=60
The problem is that when you are using:
docker compose up --build
As docker compose document describes :
flag
reference
--build
Build images before starting containers.
That means that during the build time there is no "db" container running, so there is no possible to exist a "db" host name.
A suggestion would be to to not engage any DB transaction during the build phase.
You can make any database "rendering" **as a seeding process in your app start

Docker auto reload not working on remote server

I've setup a Django project that runs in Docker. I want to have my dev environment on a remote server and have it auto reload as I change my files locally. I've set up remote deployment in Pycharm and it works fine. All local changes are reflected on my remote server, I can also see that files get changed inside the Docker container as I've setup a volume in my docker-compose file. However auto reloading is not working, and I cannot figure out why.
My docker-compose file:
version: '3.7'
services:
web:
container_name: my_project_ws
restart: always
build:
context: .
dockerfile: Dockerfile
command: gunicorn my_project.wsgi:application --bind 0.0.0.0:8000
volumes:
- static_volume:/my_project/staticfiles
- media_volume:/my_project/mediafiles
- .:/my_project
expose:
- 8000
env_file:
- .env
depends_on:
- db
db:
container_name: my_project_db
restart: always
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- .env
nginx:
restart: always
build: ./nginx
volumes:
- static_volume:/my_project/staticfiles
- media_volume:/my_project/mediafiles
ports:
- 80:80
depends_on:
- web
volumes:
postgres_data:
static_volume:
media_volume:
Running Gunicorn with --reload solved the problem.

Why does this docker compose file build the same image four times?

When I run docker-compose build on the following docker-compose file, which is for a django server with celery, it builds an identical image four times (for the web service, celeryworker, celerybeat and flower).
The entire process is repeated four times
I thought the point of inheriting from other service descriptions in docker-compose was so that you could reuse the same image for different services?
How can I reuse the web image in the other services, to reduce my build time by 75%?
version: '3'
services:
web: &django
image: myorganisation/myapp
container_name: myapp_web
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
# This is a multistage build installing private dependencies, hence this arg is needed
args:
PERSONAL_ACCESS_TOKEN_GITHUB: ${PERSONAL_ACCESS_TOKEN_GITHUB}
command: /start
volumes:
- .:/app
ports:
- 8000:8000
depends_on:
- db
- redis
environment:
- DJANGO_SETTINGS_MODULE=backend.settings.local
- DATABASE_URL=postgres://postgres_user:postgres_password#db/postgres_db
- REDIS_URL=redis://:redis_password#redis:6379
- CELERY_FLOWER_USER=flower_user
- CELERY_FLOWER_PASSWORD=flower_password
env_file:
- ./.env
celeryworker:
<<: *django
container_name: myapp_celeryworker
depends_on:
- redis
- db
ports: []
command: /start-celeryworker
celerybeat:
<<: *django
container_name: myapp_celerybeat
depends_on:
- redis
- db
ports: []
command: /start-celerybeat
flower:
<<: *django
container_name: myapp_flower
ports:
- 5555:5555
command: /start-flower
volumes:
postgres_data:
driver: local
pgadmin_data:
driver: local
Because you are specifying the build section in all the services (using the django anchor), it is getting built for every service.
If you want to use the same image for all services but build it only once, you can specify the build section in only one service which would be started first (i.e., service with no dependencies) and then specify just the image field without build section the in other services and make these services depend on the first service which builds the image.

Developing with celery and docker

I have noticed that developing with celery in container, something like this:
celeryworker:
build: .
user: django
command: celery -A project.celery worker -Q project -l DEBUG
links:
- redis
- postgres
depends_on:
- redis
- postgres
env_file: .env
environment:
DJANGO_SETTINGS_MODULE: config.settings.celery
if I want to do some changes on some celery task, I have to completly rebuild the docker image in order to have the latest changes.
So I tried:
docker-compose -f celery.yml down
docker-compose -f celery.yml up
Nothing changed, then:
docker-compose -f celery.yml down
docker-compsoe -f celery.yml build
docker-compose -f celery.yml up
I have the new changes.
Is this the way to do it? seems very slow to me, all the time rebuilding the image, is then better have the local celery outsite docker containers?
Mount your . (that is, your working copy) as a volume within the container you're developing in.
That way you're using the fresh code from your working directory without having to rebuild (unless, say, you're changing dependencies or something else that requires a rebuild).
The idea is explained here by Heroku, emphasis mine:
version: '2'
services:
web:
build: .
ports:
- "5000:5000"
env_file: .env
depends_on:
- db
volumes:
- ./webapp:/opt/webapp # <--- Whatever code your Celery workers need should be here
db:
image: postgres:latest
ports:
- "5432:5432"
redis:
image: redis:alpine
ports:
- "6379:6379"