Failing when running celery from docker - django

Question

I'm trying to get celery to work with Django and docker and the building works well but I celery won't run. Any ideas?
Here is are the docker-compose logs -f errors
Starting django-celery_redis_1 ... done
Starting django-celery_db_1 ... done
Starting django-celery_flower_1 ... done
Starting django-celery_celery_beat_1 ... done
Starting django-celery_celery_worker_1 ... done
Starting django-celery_web_1 ... done
Attaching to django-celery_db_1, django-celery_redis_1, django-celery_celery_worker_1, django-celery_flower_1, django-celery_celery_beat_1, django-celery_web_1
celery_beat_1 | standard_init_linux.go:219: exec user process caused: exec format error
db_1 | 2021-03-28 18:18:15.611 UTC [1] LOG: starting PostgreSQL 12.0 on x86_64-pc-linux-musl, compiled by gcc (Alpine 8.3.0) 8.3.0, 64-bit
db_1 | 2021-03-28 18:18:15.613 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2021-03-28 18:18:15.616 UTC [1] LOG: listening on IPv6 address "::", port 5432
celery_worker_1 | standard_init_linux.go:219: exec user process caused: exec format error
db_1 | 2021-03-28 18:18:15.648 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
redis_1 | 1:C 28 Mar 2021 18:18:15.425 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 28 Mar 2021 18:18:15.425 # Redis version=5.0.12, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 28 Mar 2021 18:18:15.425 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1 | 1:M 28 Mar 2021 18:18:15.427 * Running mode=standalone, port=6379.
redis_1 | 1:M 28 Mar 2021 18:18:15.427 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1 | 1:M 28 Mar 2021 18:18:15.427 # Server initialized
redis_1 | 1:M 28 Mar 2021 18:18:15.427 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1 | 1:M 28 Mar 2021 18:18:15.428 * DB loaded from disk: 0.000 seconds
redis_1 | 1:M 28 Mar 2021 18:18:15.428 * Ready to accept connections
flower_1 | standard_init_linux.go:219: exec user process caused: exec format error
web_1 | standard_init_linux.go:219: exec user process caused: exec format error
db_1 | 2021-03-28 18:18:15.777 UTC [19] LOG: database system was shut down at 2021-03-28 18:16:52 UTC
db_1 | 2021-03-28 18:18:15.791 UTC [1] LOG: database system is ready to accept connections
django-celery_celery_worker_1 exited with code 1
django-celery_flower_1 exited with code 1
django-celery_celery_beat_1 exited with code 1
django-celery_web_1 exited with code 1
UPDATED: added docker-compose.yml file for better references on the stack problem. The build succeeds but when running docker-compose up it doesn't and throws the celery error
docker-compose.yml
version: '3.8'
services:
web:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: django_celery_example_web
# '/start' is the shell script used to run the service
command: /start
# this volume is used to map the files and folders on the host to the container
# so if we change code on the host, code in the docker container will also be changed
volumes:
- .:/app
ports:
- 8010:8000
# env_file is used to manage the env variables of our project
env_file:
- ./.env/.dev-sample
depends_on:
- redis
- db
db:
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_DB=hello_django_dev
- POSTGRES_USER=hello_django
- POSTGRES_PASSWORD=hello_django
redis:
image: redis:5-alpine
celery_worker:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: django_celery_example_celery_worker
command: /start-celeryworker
volumes:
- .:/app
env_file:
- ./.env/.dev-sample
depends_on:
- redis
- db
celery_beat:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: django_celery_example_celery_beat
command: /start-celerybeat
volumes:
- .:/app
env_file:
- ./.env/.dev-sample
depends_on:
- redis
- db
flower:
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: django_celery_example_celey_flower
command: /start-flower
volumes:
- .:/app
env_file:
- ./.env/.dev-sample
ports:
- 5557:5555
depends_on:
- redis
- db
volumes:
postgres_data:

Answer 1

I'm not sure it applies to your case, but in docker i start celery with this command:
command: celery -A my_proj worker -l DEBUG
Since the error it gives you is "exec format error" it might just be this.