Dockerize django app along side with cucumber test - django

Here is the case. I have simple django app with cucumber tests. I dockerized the django app and it works perfect, but I want to dockerize the cucumber test too and run them. Here is my project sturcutre:
-cucumber_drf_tests
-feature
-step_definitions
axiosinst.js
config.js
package.json
cucumber.js
Dockerfile
package-lock.json
-project_apps
-common
docker-compose.yaml
Dockerfile
manage.py
requirements.txt
Here is my cucumber_drf_tests/Dockerfile
FROM node:12
WORKDIR /app/src
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 8000
CMD ["yarn", "cucumber-drf"] (this is how I run my test locally)
My second Dockerfile
FROM python:3.8
ENV PYTHONUNBUFFERED=1
RUN mkdir -p /app/src
WORKDIR /app/src
COPY requirements.txt /app/src
RUN pip install -r requirements.txt
COPY . /app/src
And my docker-compose file
version: "3.8"
services:
test:
build: ./cucumber_drf_tests
image: cucumber_test
container_name: cucumber_container
ports:
- 8000:8000
depends_on:
- app
app:
build: .
image: app:django
container_name: django_rest_container
ports:
- 8000:8000
volumes:
- .:/django #describes a folder that resides on our OS within the container
command: >
bash -c "python manage.py migrate
&& python manage.py loaddata ./project_apps/fixtures/dummy_data.json
&& python manage.py runserver 0.0.0.0:8000"
depends_on:
- db
db:
image: postgres
container_name: postgres_db
volumes:
- ./data/db:/var/lib/postgresql/data
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=bla
- POSTGRES_PASSWORD=blaa
If I remove I remove test service and run the tests locally everything is fine, but otherwise I got different errors the last one is:
Bind for 0.0.0.0:8000 failed: port is already allocated
It is logic I know, but how to tell to test_container to make the API calls to the address of the running django_rest_container. Maybe this dummy question but I am new of containers world so every sharing of good practice is wellcomed

The issue is in exposing the ports. You are exposing both app and test on the same port (8000). For container you can keep it same. But for host it has to be different.
<host port> : <container port>
This is how ports are mapped in docker. So either change the host port in app or test to different port like below.
For app keep below ports:
7500:8000
Now your app will be accessible at port 7500 and test at 8000

Related

Hot reload in Django app inside docker containers not working

So today is my first day ever to use docker, I tried use it many times but I noticed that hot reload does not work
I opened the container using vscode and navigate through the files and tried changing files and nothing happens
here's Dockerfile
FROM python:3.8-slim-buster
WORKDIR /usr/project
ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1
COPY . .
RUN pip install -r requirements.txt
and here's docker-compose.yml
version: '3.7'
services:
web:
restart : always
build:
context: .
dockerfile: Dockerfile
command: sh -c "python manage.py runserver 0.0.0.0:8000"
ports:
- "8000:8000"
env_file:
- .env
volumes:
- .:/user/project
even unchecked docker-compose 2 from desktop docker and restarted the app and the containers, still nothing happens, so what am I doing wrong?
I kind of figured it out, apparently I was using docker wrong
I was using docker-compose run instead of docker-compose up
and I had a typo, I copied the code to a dir use and mounted another dir user :"D

Dockerized django container not producing local migrations file

Question
I am a beginner with docker; this being the first project I have set up with it and don't particularly know what I am doing. I would very much appreciate if someone could give me some advice on what the best way to get migrations from a dockerized django app to store locally
What I have tried so far
I have a local django project setup with the following file structure:
Project
.docker
-Dockerfile
project
-data
-models
- __init__.py
- user.py
- test.py
-migrations
- 0001_initial.py
- 0002_user_role.py
...
settings.py
...
manage.py
Makefile
docker-compose.yml
...
In the current state the migrations for the test.py model have not been run; so I attempted to do so using docker-compose exec main python manage.py makemigrations. This worked successfully returning the following:
Migrations for 'data':
project/data/migrations/0003_test.py
- Create model Test
But produced no local file. However, if I explore the file system of the container I can see that the file exists on the container itself.
Upon running the following:
docker-compose exec main python manage.py migrate
I receive:
Running migrations:
No migrations to apply.
Your models in app(s): 'data' have changes that are not yet reflected in a migration, and so won't be applied.
Run 'manage.py makemigrations' to make new migrations, and then re-run 'manage.py migrate' to apply them.
I was under the impression that even if this did not create the local file it would at least run the migrations on the container.
Regardless, my intention was that when I run docker-compose exec main python manage.py makemigrations it store the file locally in the project/data/migrations folder and then I just run migrate manually. I can't find much documentation on how to do this; the only post I have seen suggested bind mounts (Migrations files not created in dockerized Django) which I attempted by adding the following to my docker-compose file:
volumes:
- type: bind
source: ./data/migrations
target: /var/lib/migrations_test
but I was struggling to get it to work and following from this I had no idea how to run commands through this volume using docker-compose and I was questioning whether this was even a good idea as I had read somewhere it was not best practice to use bind mounts.
Project setup:
The docker-compose.yml file looking like so:
version: '3.7'
x-common-variables: &common-variables
ENV: 'DEV'
DJANGO_SETTINGS_MODULE: 'project.settings'
DATABASE_NAME: 'postgres'
DATABASE_USER: 'postgres'
DATABASE_PASSWORD: 'postgres'
DATABASE_HOST: 'postgres'
CELERY_BROKER_URLS: 'redis://redis:6379/0'
volumes:
postgres:
services:
main:
command:
python manage.py runserver 0.0.0.0:8000
build:
context: ./
dockerfile: .docker/Dockerfile
target: main
environment:
<<: *common-variables
ports:
- '8000:8000'
env_file:
- dev.env
networks:
- default
postgres:
image: postgres:13.6
volumes:
- postgres:/var/lib/postgresql/data
ports:
- '25432:5432'
environment:
POSTGRES_PASSWORD: 'postgres'
command: postgres -c log_min_messages=INFO -c log_statement=all
wait_for_dependencies:
image: dadarek/wait-for-dependencies
environment:
SLEEP_LENGTH: '0.5'
redis:
image: redis:latest
ports:
- '16379:6379'
worker:
build:
context: .
dockerfile: .docker/Dockerfile
target: main
command: celery -A project worker -l INFO
environment:
<<: *common-variables
volumes:
- .:/code/delegated
env_file:
- dev.env
networks:
- default
beat:
build:
context: .
dockerfile: .docker/Dockerfile
target: main
command: celery -A project beat -l INFO
environment:
<<: *common-variables
volumes:
- .:/code/delegated
env_file:
- dev.env
networks:
- default
networks:
default:
Makefile:
build: pre-run
build:
docker-compose build --pull
dev-deps: pre-run
dev-deps:
docker-compose up -d postgres redis
docker-compose run --rm wait_for_dependencies postgres:5432 redis:6379
migrate: pre-run
migrate:
docker-compose run --rm main python manage.py migrate
setup: build dev-deps migrate
up: dev-deps
docker-compose up -d main
Dockerfile:
FROM python:3.10.2 as main
ENV PYTHONUNBUFFERED 1
COPY ./requirements.txt /requirements.txt
RUN pip install -r /requirements.txt
RUN mkdir -p /code
WORKDIR /code
ADD . ./
RUN useradd -m -s /bin/bash app
RUN chown -R app:app .
USER app
EXPOSE 8000
Follow up based on diptangsu-goswami's response
I tried adding the following:
volumes:
- type: bind
source: C:\dev\Project\project
target: /code/
This creates an empty directory in my Project folder; named C:\dev\Project\project but the app doesn't run as it cannot find the manage.py file... I assumed this was because it was in the parent directory Project and tried again with:
volumes:
- type: bind
source: C:\dev\Project
target: /code/
But the same problem occured. Why is it creating the empty directory? surely it should just be binding the existing directory with the container directory? Also using this method, would I need to change my Dockerfile to not copy the codebase to the container in the first place and just mount it on instead?
I managed to fix it by adding the following to my 'main' service in my docker compose:
volumes:
- .:/code:delegated

How to sync local host with docker host?

I have a hello world Django project and i want to dockerize it. My OS is windows 8.1 and I'm using docker toolbox. Using volumes I could persist data in docker container and what I want to do is to sync the code in docker container with the code in my local host in the directory where my project code is stored and so far I couldn't do it.
Here is my docker-compose.yml:
version: '3.7'
services:
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- myvol1:/code
ports:
- 8000:8000
volumes:
myvol1:
and Dockerfile:
# Pull base image
FROM python:3.7
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Set work directory
WORKDIR /code
# Install dependencies
COPY requirement.txt /code/
RUN pip install -r requirement.txt
# Copy project
COPY . /code/
without using volumes I can run my code in the container but the data is not persisted.
I'd be grateful for your help.
Maybe try
version: '3.7'
services:
web:
build: .
command: python manage.py runserver 127.0.0.1:8000
volumes:
- myvol1:/code
ports:
- 8000:8000
volumes:
myvol1:
I thought maybe changing to the localhost IP might help or the ports could also be changed following the format of
<port-number-host> : <port-number-container>
"your listening port : container's listening port"
The port might be busy, but these are things that I would troubleshoot and try.
My resources/references: Udemy Class from Bret Fisher

Docker pull Django image and run container

So, I have followed this tutorial by Docker to create a Django image.
It completely works on my local machine by just running a docker-compose up command from the root directory of my project.
But, after pushing the image to docker hub https://hub.docker.com/repository/docker/vivanks/firsttry
I am pulling the image to another machine and then running:
docker run -p 8020:8020 vivanks/firsttry
But it's not getting started and showing this error:
EXITED(0)
Can anyone help me on how to pull this image and run it?
My Dockerfile
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY . /code/
My docker-compose.yml
version: '3'
services:
db:
image: postgres
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
As #larsks mentioned in his answer your problem is that your command is in the Compose file, rather than in Dockerfile.
To run your project on another machine as-is, use the following docker-compose.yml:
version: '3'
services:
db:
image: postgres
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
web:
image: vivanks/firsttry:latest
command: python manage.py runserver 0.0.0.0:8000
ports:
- "8000:8000"
depends_on:
- db
If you already added CMD python manage.py runserver 0.0.0.0:8000 to your Dockerfile and rebuilt the image, the above can be further simplified to:
version: '3'
services:
db:
image: postgres
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
web:
image: vivanks/firsttry:latest
ports:
- "8000:8000"
depends_on:
- db
Using docker run will fail in either case, since it won't set up a database.
Edit:
OP, I admire your persistence, but at the same time do not understand the insistence on using Docker CLI rather than docker-compose. I recommend using one of the above docker-compose.yml files to start your app.
Nevertheless, I accept the challenge of running it without docker-compose.
Your application fails to start when you use docker run command, because it tries to connect to database on host db, which does not exist. In your (and mine) docker-compose.yml there is a definition of a service called db. Docker-compose uses that definition to set up a database container for you and makes it available for your application under hostname db.
To start your application without using docker-compose, you need to manually do everything it does for you automatically (the commands below assume you have added CMD... to your Dockerfile:
docker network create --driver bridge django-test-network
docker run --detach --env POSTGRES_DB=postgres --env POSTGRES_USER=postgres --env POSTGRES_PASSWORD=postgres --network django-test-network --name db postgres:latest
docker run -it --rm --network django-test-network --publish 8080:8000 vivanks/firsttry:latest
The above 3 commands create a new bridged network, create and start a detached (background) container with properly configured database connected to that network and finally create and start an attached (foreground) container based on your image, also attached to that new network. Since both containers are on the same, non-default bridged network, your application will be able to resolve hostname db to internal IP address of the database container and start properly.
Once you shut it down with Ctrl+C, the container with your application will delete itself (as it was started with option --rm), but you need to also manually clean up the rest. To do so run the following commands:
docker stop db
docker rm -v db
docker network remove django-test-network
The first one stops the database container, the second one removes it and its anonymous volume and the third one removes the network.
I hope this explains everything.
Your Dockerfile doesn't specify a CMD or ENTRYPOINT. When you run...
docker run -p 8020:8020 vivanks/firsttry
...the container has nothing to do (which means it will actually try to start a Python interactive shell, but since you're not allocating a terminal with -t, the shell just exits. Successfully). In your docker-compose.yml, you're passing in an explicit command:
command: python manage.py runserver 0.0.0.0:8000
So the equivalent docker run command line would look like:
docker run -docker run -p 8020:8020 vivanks/firsttry python manage.py runserver 0.0.0.0:8000
But you probably want to bake that into your Dockerfile like this:
CMD python manage.py runserver 0.0.0.0:8000

Django, and React inside Docker inside a single Digital Ocean Droplet: 400 Bad Request for Django, React works fine

I have Django and React inside the same Docker container using docker-compose.yml and running this container inside a Digital Ocean Droplet running Ubuntu. When I navigate to http://my_ip_address:3000 which is the React app, it works just fine, but when I navigate to http://my_ip_address:8000 which is the Django app, I get a 400 Bad Request error from the server.
project/back-end/Dockerfile
FROM python:3.7
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /nerdrich
COPY Pipfile Pipfile.lock /nerdrich/
RUN pip install pipenv && pipenv install --system
COPY . /nerdrich/
EXPOSE 8000
project/front-end/Dockerfile
# official node.js runtime for Docker
FROM node:12
# Create and set the directory for this container
WORKDIR /app/
# Install Application dependencies
COPY package.json yarn.lock /app/
RUN yarn install --no-optional
# Copy over the rest of the project
COPY . /app/
# Set the default port for the container
EXPOSE 3000
CMD yarn start
project/docker-compose.yml
version: "3"
services:
web:
build: ./back-end
command: python /nerdrich/manage.py runserver
volumes:
- ./back-end:/nerdrich
ports:
- "8000:8000"
stdin_open: true
tty: true
client:
build: ./front-end
volumes:
- ./front-end:/app
- /app/node_modules
ports:
- '3000:3000'
stdin_open: true
environment:
- NODE_ENV=development
depends_on:
- "web"
command:
yarn start
project/back-end/nerdrich/.env
ALLOWED_HOSTS=['165.227.82.162']
I can provide any additional information if needed.