running celery tasks and celery beat in ECS with Django - django

Question

I'm using ECS for the first time. I have dockerized my Django 2.2 application and using ECS and uwsgi to run the Django application in production.
While in the development environment, I had to run three commands to run Django server, celery and celery beat
python manage.py runserver
celery -A qcg worker -l info
celery beat -A qcg worker -l info
Where qcg is my application.
My Dockerfile has following uwsgi configuration
EXPOSE 8000
ENV UWSGI_WSGI_FILE=qcg/wsgi.py
ENV UWSGI_HTTP=:8000 UWSGI_MASTER=1 UWSGI_HTTP_AUTO_CHUNKED=1 UWSGI_HTTP_KEEPALIVE=1 UWSGI_LAZY_APPS=1 UWSGI_WSGI_ENV_BEHAVIOR=holy
ENV UWSGI_WORKERS=2 UWSGI_THREADS=4
ENV UWSGI_STATIC_MAP="/static/=/static_cdn/static_root/" UWSGI_STATIC_EXPIRES_URI="/static/.*\.[a-f0-9]{12,}\.(css|js|png|jpg|jpeg|gif|ico|woff|ttf|otf|svg|scss|map|txt) 315360000"
USER ${APP_USER}:${APP_USER}
ENTRYPOINT ["/app/scripts/docker/entrypoint.sh"]
The entrypoint.sh file has
exec "$#"
I have created the ECS task definition and in the container's command input, I have
uwsgi --show-config
This starts the uwsgi server.
Now I'm running 1 EC2 instance in the cluster and running one service with two instances of the task definition.
I couldn't figure out how to run the celery task and celery beat in my application.
Do I need to create separate tasks for running celery and celery-beat?

Answer 1

Yes, you need to run separate ECS tasks (or separate ECS services) for celery beat and celery worker. Celery Beat will send the Celery tasks to the Celery worker.
I use separate Dockerfiles for Celery, Celery beat, and Django.
Worker Dockerfile something like this:
FROM python:3.8
ENV PYTHONUNBUFFERED 1
ADD requirements.txt /requirements.txt
RUN pip install -r /requirements.txt
ADD . /src
WORKDIR /src
CMD ["celery", "-A", "<appname>", "worker"]
and Beat Dockerfile:
FROM python:3.8
ENV PYTHONUNBUFFERED 1
ADD requirements.txt /requirements.txt
RUN pip install -r /requirements.txt
ADD . /src
WORKDIR /src
CMD ["celery", "-A", "<appname>", "beat"]