In my Dockerfile I want to add a dir and remove some existing dirs. Add works but Remove does not.
My Dockerfile:
FROM wordpress:5.5.1-php7.4-apache
COPY themes/twentyseventeen-b /var/www/html/wp-content/themes/twentyseventeen-b
COPY images/header.jpg /var/www/html/wp-content/uploads/2020/01/a.jpg
#This is line that causes an error:
RUN rm -r /var/www/html/wp-content/themes/twentytwenty
-COPY works and my dirs are created and the files uploaedd
-RUN rm -r /var/www/html/wp-content/themes/twentytwenty does not work:
rm: cannot remove '/var/www/html/wp-content/themes/twentytwenty': No such file or directory
But I know that dir exists (if I comment out the RUN rm.... line and let the build complete I can CLI over to the container and see the dir is there).
I tried using single quotes around the dir name -- but the same problem exists.
I also tried to rm the twentyseventeen-b dir right after creation. It does not give a build error but it also does not delete the dir (I still see the dir in CLI):
FROM wordpress:5.5.1-php7.4-apache
COPY themes/twentyseventeen-b /var/www/html/wp-content/themes/twentyseventeen-b
RUN rm -r /var/www/html/wp-content/themes/twentyseventeen-b
If I use the CLI I can rm any dirs in /var/www/html.
The docker-compose.yml that runs this Dokerfile:
version: '3.3'
services:
db:
build: ./mariadb
volumes:
- db_data:/var/lib/mysql
restart: always
wordpress:
depends_on:
- db
build: ./wordpress
ports:
- "8000:80"
restart: always
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: wordpress
WORDPRESS_DB_NAME: wordpress
volumes:
db_data: {}
you need to add -f argument to rm command which means: "ignore nonexistent files and arguments, never prompt"
for example:
RUN rm -rf /var/www/html/wp-content/themes/twentytwenty
Related
Im new to docker.
I have a docker-compose.yml file like this :
version: '3.7'
services:
nginx_sarahmaso:
build:
context: .
dockerfile: ./compose/production/nginx/Dockerfile
restart: always
volumes:
- staticfiles_sarahmaso:/app/static
- mediafiles_sarahmaso:/app/media
ports:
- 4000:80
depends_on:
- web_sarahmaso
networks:
spa_network_sarahmaso:
web_sarahmaso:
build:
context: .
dockerfile: ./compose/production/django/Dockerfile
restart: always
command: /start
volumes:
- staticfiles_sarahmaso:/app/static
- mediafiles_sarahmaso:/app/media
- sqlite_sarahmaso:/app/db
env_file:
- ./env/prod-sample
networks:
spa_network_sarahmaso:
networks:
spa_network_sarahmaso:
volumes:
sqlite_sarahmaso:
staticfiles_sarahmaso:
mediafiles_sarahmaso:
I'm deploying this on a server with a sh script running these commands :
mkdir -p /app
rm -rf /app/* && tar -xf /tmp/project.tar -C /app
sudo docker-compose -f /app/docker-compose.yml build
sudo supervisorctl restart react-wagtail-project
sudo ufw allow port
However the supervisorctl doesnt run correctly. But the console tells me "Successfully built dc10bd26b175"after the docker build.
But when i run docker-compose ps or docker ps -a i dont see any containers.
Docker-compose ps asks me for a docker-compose.yml file and if i do docker-compose ps -f path_to/docker-compose.yml the console shows me the help slug :
List containers.
Usage: ps [options] [SERVICE...]
Options:
-q, --quiet Only display IDs
--services Display services
--filter KEY=VAL Filter services by a property
-a, --all Show all stopped containers (including those created by the run command)
How come i dont see my containers?
It seems your containers are not started.
With your line sudo docker-compose -f /app/docker-compose.yml build you are building your container, as the console message tells you.
I do not exactly know what this line does sudo supervisorctl restart react-wagtail-project, but to me, it does not look like a command to START your newly built containers.
Try to explicitely start your containers by adding
./path_to_compose/docker-compose up or
./path_to_compose/docker-compose up -d to your script.
Having this dockerfile:
FROM python:3.8.3-alpine
ENV MICRO_SERVICE=/home/app/microservice
# RUN addgroup -S $APP_USER && adduser -S $APP_USER -G $APP_USER
# set work directory
RUN mkdir -p $MICRO_SERVICE
RUN mkdir -p $MICRO_SERVICE/static
# where the code lives
WORKDIR $MICRO_SERVICE
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install psycopg2 dependencies
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev \
&& apk add postgresql-dev gcc python3-dev musl-dev \
&& apk del build-deps \
&& apk --no-cache add musl-dev linux-headers g++
# install dependencies
RUN pip install --upgrade pip
# copy project
COPY . $MICRO_SERVICE
RUN pip install -r requirements.txt
COPY ./entrypoint.sh $MICRO_SERVICE
CMD ["/bin/bash", "/home/app/microservice/entrypoint.sh"]
and the following docker-compose.yml file:
version: "3.7"
services:
nginx:
build: ./nginx
ports:
- 1300:80
volumes:
- static_volume:/home/app/microservice/static
depends_on:
- web
restart: "on-failure"
web:
build: . #build the image for the web service from the dockerfile in parent directory
command: sh -c "python manage.py collectstatic --no-input &&
gunicorn djsr.wsgi:application --bind 0.0.0.0:${APP_PORT}"
volumes:
- .:/microservice:rw # map data and files from parent directory in host to microservice directory in docker containe
- static_volume:/home/app/microservice/static
env_file:
- .env
image: wevbapp
expose:
- ${APP_PORT}
restart: "on-failure"
volumes:
static_volume:
I need to reference the following files (in the docker-compose.yml file) being in other directories rather than the .devcontainer:
manage.py
requirements.txt
.env
This is my folder structure:
An easy solution would be to move the dockerfile, docker-compose.yml, and .env in the django directory djsr, but I am trying to keep the files structured like this. How can I do reference those files in docker-compose.yml?
It is fairly common to put the couple of Docker-related files in the project root directory, and that can potentially save you some trouble; I'd recommend that as a first choice.
If you do want to keep it all in a subdirectory, it's possible, though. When you run docker-compose, you can specify the location of the configuration file. It will consider all paths as relative to this file's directory.
# Either:
docker-compose -f .devcontainer/docker-compose.yml up
cd .devcontainer && docker-compose up
When you go to build the image, the build reads in a context directory, and COPY statements are always interpreted relative to this directory. For your setup, you need the context directory to be the top of your source tree, and then specify an alternate Dockerfile in a subdirectory.
services:
web:
build:
context: ..
dockerfile: .dockerenv/Dockerfile
For the most part the Dockerfile itself is fine, but where the entrypoint script is in a subdirectory, the COPY command needs to reflect that too. Since you're copying the entire source directory, you could also rearrange things inside the image to be the layout you want.
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . ./
# Either:
COPY .dockerenv/entrypoint.sh ./
# Or:
RUN mv .dockerenv/entrypoint.sh .
# Or:
CMD ["./dockerenv/entrypoint.sh"]
I don't recommend the volume structure you have, but if you want to keep it, you also need to change the source path of the bind mount to be the parent directory. (Note particularly, in the previous Dockerfile fragment, a couple of the options involve moving files inside the image, and a bind mount will hide that change.)
services:
web:
volumes:
# Ignore the application built into the container, and use
# whatever's checked out on the host system instead.
- ..:/home/app/microservice
# Further ignore the static assets on the host system and
# use the content in a named volume instead.
- static_volume:/home/app/microservice/static
why don't you mount the same as you did with the folders only for these files?
The source of the mount. For bind mounts, this is the path to the file or directory on the
Docker daemon host. May be specified as source or src.
I want to initialize Docker for my Django project with postreSQL. I followed instrunctions from https://docs.docker.com/compose/django/
I also want to be sure that db runs before web so I use wait_for_db.sh. When I try to execute command docker-compose up
I see following respond:
web_1 | chmod: cannot access 'wait_for_db.sh': No such file or directory
pipingapi_web_1 exited with code 1
Before I try to use "docker-compose run", I Change directory to project root. I tried also to write
$ docker-compose run web django-admin startproject pipingapi . even though project was created before with venv.
I guess its not exactly about .sh file because when I erase lines reffering to that file, Docker cant find manage.py then (look at command order in docker-compose.yml). I also tried to put code/ before wait_for_db.sh in docker-compose.yml but it did not work.
My project tree:
.
L apienv/
L docker-compose.yml
L Dockerfile
L manage.py
L project/
L README.md
L requirements.txt
L restapi/
L wait_for_db.sh
Dockerfile:
FROM python:3.6
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY . /code/
RUN apt-get update -q
RUN apt-get install -yq netcat
docker-compose.yml
version: '3'
services:
db:
image: postgres:12.3
volumes:
- /var/lib/postgresql/data
env_file:
- ./.env
web:
build: .
command:
sh -c "chmod +x wait_for_db.sh
&& ./wait_for_db.sh
&& python manage.py makemigrations
&& python manage.py migrate
&& python manage.py runserver 0.0.0.0:8000"
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
env_file:
- ./.env
If it matters: I use Docker Toolbox on win 8.1
EDIT(SOLVED):
It looked like I was overwritting my tree with "code" directory so I deleted
volumes:
- .:/code
and it works
As the image building stage is complete, you could drop into the docker image and interactively run the commands you are trying to fix.
That should give you some hints
docker run -it web_1 bash
My guess is, as you are setting WORKDIR before you run the COPY, you are probably in the wrong directory.
It looked like I was overwritting my tree with "code" directory so I deleted
volumes:
- .:/code
and it works
I'm trying to Dockerize my local Django/MySql setup. I have this directory and file structure ...
apache
docker-compose.yml
web
- manage.py
- venv
- requirements.txt
- ...
Below is the docker-compose.yml file I'm using ...
version: '3'
services:
web:
restart: always
build: ./web
expose:
- "8000"
links:
- mysql:mysql
volumes:
- web-django:/usr/src/app
- web-static:/usr/src/app/static
#env_file: web/venv
environment:
DEBUG: 'true'
command: [ "python", "./web/manage.py runserver 0.0.0.0:8000" ]
mysql:
restart: always
image: mysql:5.7
environment:
MYSQL_DATABASE: 'maps_data'
# So you don't have to use root, but you can if you like
MYSQL_USER: 'chicommons'
# You can use whatever password you like
MYSQL_PASSWORD: 'password'
# Password for root access
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3406:3406"
expose:
# Opens port 3406 on the container
- '3406'
volumes:
- my-db:/var/lib/mysql
volumes:
web-django:
web-static:
my-db:
However when I run
docker-compose up
I get errors like the below
maps_web_1 exited with code 2
web_1 | python: can't open file './web/manage.py runserver 0.0.0.0:8000': [Errno 2] No such file or directory
maps_web_1 exited with code 2
maps_web_1 exited with code 2
web_1 | python: can't open file './web/manage.py runserver 0.0.0.0:8000': [Errno 2] No such file or directory
maps_web_1 exited with code 2
Is there another way I'm supposed to be referencing the manage.py file?
Edit: Added info requested in comments ...
FROM python:3.7-slim
RUN apt-get update && apt-get install
RUN apt-get install -y libmariadb-dev-compat libmariadb-dev
RUN apt-get update \
&& apt-get install -y --no-install-recommends gcc \
&& rm -rf /var/lib/apt/lists/*
RUN python -m pip install --upgrade pip
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
COPY . .
As others suggested, this is most probably because of running the manage.py runserver from a wrong directory or something very similar to this.
You are not using WORKDIR directive in your Dockerfile, at all. It is much safer if you do use them. Change your Dockerfile and docker-compose.yml files as below, and you problem should be solved.
Dockerfile
FROM python:3.7-slim
RUN apt-get update && apt-get install
RUN apt-get install -y libmariadb-dev-compat libmariadb-dev
RUN apt-get update \
&& apt-get install -y --no-install-recommends gcc \
&& rm -rf /var/lib/apt/lists/*
RUN python -m pip install --upgrade pip
RUN mkdir -p /app/
WORKDIR /app/
COPY requirements.txt requirements.txt
RUN python -m pip install -r requirements.txt
COPY . /app/
docker-compose.yml
version: '3'
services:
web:
restart: always
build: ./web
expose:
- "8000"
links:
- mysql:mysql
volumes:
- web-django:/usr/src/app
- web-static:/usr/src/app/static
#env_file: web/venv
environment:
DEBUG: 'true'
command: [ "python", "manage.py", "runserver", "0.0.0.0:8000" ]
...
Notice
You should be able to fix the problem by simply deleting web from your command for running the server. That's because when you are building the Dockerfile, you are inside the web directory. So when you do COPY . . you are copying contents inside web directory, and not the web directory itself. Actually, your file structure inside the docker image, should look something similar to this:
- root
- home
- var
- ...
- manage.py
- venv
- requirements.txt
- ...
In the command: directive, if you're using the array syntax, you're responsible for breaking up the command into words. As you've shown it you're running the equivalent of python "manage.py runserver 0.0.0.0:8000" at the shell prompt, and it's dutifully considering the entire command and options as the filename of a script to be run, including spaces. If you break this up into single words it will work better
command: ["python", "manage.py", "runserver", "0.0.0.0:8000"]
But there's not really a reason to specify this in docker-compose.yml at all. This is the default command you'd want to run to launch the container no matter how you ran it, so it should be the default command in your image's Dockerfile
...
EXPOSE 8000
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
You don't need links: at all on modern Docker (Docker Compose automatically sets up inter-container networking for you). You definitely don't want to mount named volumes over your application code: this hides what's in your image, and (since you've told Docker this is critical user data) it forces Docker to use an old version of your application if you try to update your image.
That leaves you with a simpler docker-compose.yml file:
version: '3'
services:
web:
restart: always
build: ./web
ports: # to access the container from outside
- "8000:8000"
environment:
DEBUG: 'true'
mysql:
restart: always
image: mysql:5.7
environment:
MYSQL_DATABASE: 'maps_data'
MYSQL_USER: 'chicommons'
MYSQL_PASSWORD: 'password'
MYSQL_ROOT_PASSWORD: 'password'
ports:
- "3406:3306" # second port is always container-internal port
volumes:
- my-db:/var/lib/mysql
volumes:
my-db:
Let us try to debug this error:
maps_web_1 exited with code 2
web_1 | python: can't open file './web/manage.py runserver 0.0.0.0:8000': [Errno 2] No such file or directory
Looks like either code is not not copied to container (named 'web') or command is triggered from root/home directory, where manage.py is not accessible.
1. Is the code available on container? How to check?
Usually, docker will just commands in container execute and exit unless there is unfinished running task (like server running in background).
To stop exiting and enable debugging it, let us add a running command, so that you can login to container and see if code is present.
command: tail -f /dev/null #trick to keep the docker alive for debug mode.
docker-compose.yml
web:
restart: always
build: ./web
expose:
- "8000"
links:
- mysql:mysql
volumes:
- web-django:/usr/src/app
- web-static:/usr/src/app/static
#env_file: web/venv
environment:
DEBUG: 'true'
command: tail -f /dev/null #trick to keep the docker alive for debug mode.
Login to container 'web', from command line run docker exec -it web bash
Check if project files are present, now you can run python manage.py runserver 8000 command manually. If it works, then we can be sure of that the server can be run on container. Now, we can analyse initial working directory.
If code is present, check why manage.py is not found? Is the working directory set? meaning, does the container know what is the base directory to run command?
Specify which is the working directory, in Dockerfile, before you copy the project files in to container.
Dockerfile in web directory
ENV PYTHONUNBUFFERED 1
ARG PROJ_DIR=/usr/project/web
RUN mkdir -p $PROJ_DIR
WORKDIR $PROJ_DIR
COPY . $WORKDIR
docker-compose.yml
restart: always
build: ./web
expose:
- "8000"
links:
- mysql:mysql
volumes:
- web-django:/usr/src/app
- web-static:/usr/src/app/static
#env_file: web/venv
environment:
DEBUG: 'true'
command: python manage.py runserver 0.0.0.0:8000 #note this command is triggered from $WORKDIR that we set in Dockerfile.
I think this should resolve the issue or help you to figure out the problem.
I have a Django project and I've been struggling with the automation of the static files generation. My project structure has a docker-compose.yml file and a Dockerfile for every container image.
The docker-compose.yml file for my project:
version: '3'
services:
web:
build: ./dispenser
command: gunicorn -c gunicorn.conf.py dispenser.wsgi
volumes:
- ./dispenser:/dispenser
ports:
- "8000:8000"
restart: on-failure
nginx:
build: ./nginx/
depends_on:
- web
command: nginx -g 'daemon off;'
ports:
- "80:80"
volumes:
- ./dispenser/staticfiles:/var/www/static
restart: on-failure
The Dockerfile for the Django project I'm using:
FROM python:3.7.4
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
WEBAPP_DIR=/dispenser \
GUNICORN_LOG_DIR=/var/log/gunicorn
WORKDIR $WEBAPP_DIR
RUN mkdir -p $GUNICORN_LOG_DIR \
mkdir -p $WEBAPP_DIR
ADD pc-requirements.txt $WEBAPP_DIR
RUN pip install -r pc-requirements.txt
ADD . $WEBAPP_DIR
RUN python manage.py makemigrations && \
python manage.py migrate && \
python manage.py collectstatic --no-input
After several hours of test and research I've found out that running the collectstatic and migrations commands from the Dockerfile doesn't produce the same result as doing it via the command argument on the docker-compose.yml file.
If I do it as shown above, when time for running the collectstatic command comes, only the "staticfiles" folder is generated (no files inside it). Also database migrations weren't applied (note that I'm using the default .sqlite3 db). Even though the stdout when creating the container said that migrations were applied and staticfiles generated.
The only workaround I found to make it work was executing bash from the container and then running those commands from there.
But later I've found out that if I specify those commands on the docker-file.yml everything works as expected. Leaving the files as follows:
docker-compose.yml
version: '3'
services:
web:
build: ./dispenser
command: bash -c "python manage.py makemigrations && python manage.py migrate && python manage.py collectstatic --no-input && gunicorn -c gunicorn.conf.py dispenser.wsgi"
volumes:
- ./dispenser:/dispenser
ports:
- "8000:8000"
restart: on-failure
nginx:
build: ./nginx/
depends_on:
- web
command: nginx -g 'daemon off;'
ports:
- "80:80"
volumes:
- ./dispenser/staticfiles:/var/www/static
restart: on-failure
Dockerfile
FROM python:3.7.4
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
WEBAPP_DIR=/dispenser \
GUNICORN_LOG_DIR=/var/log/gunicorn
WORKDIR $WEBAPP_DIR
RUN mkdir -p $GUNICORN_LOG_DIR \
mkdir -p $WEBAPP_DIR
ADD pc-requirements.txt $WEBAPP_DIR
RUN pip install -r pc-requirements.txt
ADD . $WEBAPP_DIR
Can anyone explain me why does this occur? And if is there another way of achieving what I intend without having to specify the commands on the docker-compose.yml file?
When you mount a host directory into a container, the contents of host directory shadow the contents of the container.
volumes:
- ./dispenser:/dispenser
So when you run your container, the initial contents of /dispenser inside container will be the contents of ./dispenser from host machine. Any content already at /dispenser inside the container is shadowed. So the content generated during image build time by the RUN instructions inside your Dockerfile will be lost.
In your second approach of using command in compose file, you are mounting the volume first and then generating the content and hence it works.
The command instruction in Dockerfile is used to override the default command in the Docker image which can be set using CMD instruction in Dockerfile. Since you want to use the first approach of running your python script during image build time using RUN instructions, you can RUN them in a different directory(say /tmp/dispenser) and as part of the command in compose or CMD in Dockerfile, you can move the generated content from /tmp/dispenser to /dispenser.