I have following file structure
Django/
- IoT.yml
- Dockerfile_Build_Django
- requirements.txt
My dockerfile (Dockerfile_Build_Django) for budiding image is as below:
FROM python:3.10.0a7-alpine3.13
ENV PYTHONUNBUFFERED 1
RUN mkdir /code/
WORKDIR /usr/src/app
COPY . .
RUN pip install -r requirements.txt
My docker-compose file as below:
Django_2.2:
build:
context: ./
dockerfile: Dockerfile_Build_Django
# Give an image name/tag
image: python:3.10.0a7-alpine3.13
container_name: Django_2.2
depends_on:
- Django_Mongo_4.2.12
tty: true
after "docker-compose -f IoT.yml up" to setup and run container, then I use "docker exec -it Django_2.2 /bin/sh" to SSH access the Django_2.2 container, I found:
no folder "/code" was created according to "RUN mkdir /code/" in docker file
nothing was copied over to working directory according to dockerfile.
Django was not installed according to above dockerfile.
[root#Mysite Django]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
python 3.10.0a7-alpine3.13 bba91cdea5a1 3 days ago 44.9MB
django_2.2 iot 37c35b93c3d8 6 days ago 86.3MB
[root#MySite Django]# docker history python:3.10.0a7-alpine3.13
IMAGE CREATED CREATED BY SIZE COMMENT
bba91cdea5a1 3 days ago /bin/sh -c #(nop) CMD ["python3"] 0B
<missing> 3 days ago /bin/sh -c set -ex; wget -O get-pip.py "$P… 8.01MB
<missing> 3 days ago /bin/sh -c #(nop) ENV PYTHON_GET_PIP_SHA256… 0B
<missing> 3 days ago /bin/sh -c #(nop) ENV PYTHON_GET_PIP_URL=ht… 0B
<missing> 3 days ago /bin/sh -c #(nop) ENV PYTHON_PIP_VERSION=21… 0B
<missing> 3 days ago /bin/sh -c cd /usr/local/bin && ln -s idle3… 32B
<missing> 3 days ago /bin/sh -c set -ex && apk add --no-cache --… 29.5MB
<missing> 3 days ago /bin/sh -c #(nop) ENV PYTHON_VERSION=3.10.0… 0B
<missing> 10 days ago /bin/sh -c #(nop) ENV GPG_KEY=A035C8C19219B… 0B
<missing> 10 days ago /bin/sh -c set -eux; apk add --no-cache c… 1.76MB
<missing> 10 days ago /bin/sh -c #(nop) ENV LANG=C.UTF-8 0B
<missing> 11 days ago /bin/sh -c #(nop) ENV PATH=/usr/local/bin:/… 0B
<missing> 11 days ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
<missing> 11 days ago /bin/sh -c #(nop) ADD file:7119167b56ff1228b… 5.61MB
[root#MySite Django]# docker-compose -f IoT.yml build
Django_Mongo_4.2.12 uses an image, skipping
Building Django_2.2
Sending build context to Docker daemon 2.832MB
Step 1/7 : FROM python:3.10.0a7-alpine3.13
---> bba91cdea5a1
Step 2/7 : ENV PYTHONUNBUFFERED 0
---> Running in 4f04eff80f44
Removing intermediate container 4f04eff80f44
---> 894fa6105e9b
Step 3/7 : RUN mkdir /code/
---> Running in 41a547193037
Removing intermediate container 41a547193037
---> 08d8a6605228
Step 4/7 : WORKDIR /usr/src/app
---> Running in 1d33ed47543f
Removing intermediate container 1d33ed47543f
---> f4f22bc9b3bb
Step 5/7 : COPY . .
---> 317348f86075
Step 6/7 : RUN pip install --upgrade pip
---> Running in e43752bea2dd
Requirement already satisfied: pip in /usr/local/lib/python3.10/site-packages (21.0.1)
Removing intermediate container e43752bea2dd
---> 1ce0ff9deb41
Step 7/7 : RUN pip install -r requirements.txt
---> Running in 87fb8a9db240
Collecting Django==2.2.17
Downloading Django-2.2.17-py3-none-any.whl (7.5 MB)
Collecting pytz
Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB)
Collecting sqlparse>=0.2.2
Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
Installing collected packages: sqlparse, pytz, Django
Successfully installed Django-2.2.17 pytz-2021.1 sqlparse-0.4.1
Removing intermediate container 87fb8a9db240
---> 8471de36ee4d
Successfully built 8471de36ee4d
Successfully tagged python:3.10.0a7-alpine3.13
[root#MySite Django]#
[root#MySite Django]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
python 3.10.0a7-alpine3.13 8471de36ee4d 8 minutes ago 86.5MB
python <none> bba91cdea5a1 3 days ago 44.9MB
django_2.2 iot 37c35b93c3d8 6 days ago 86.3MB
Am I missing some steps from the Docker documentation?
delete my previously-built image, and run docker-compose again, it now build a new image and executing all RUN / COPY / CMD commands in dockerfile. All work perfectly.
Based on above behavior, it seems that "image: " line in docker-compose file asks docker daemon to check if this image exist locally. If yes, then directly use it without building a new one. When I delete that image manually, docker daemon can not find that image, hence build a new one.
According to Docker official guide, https://docs.docker.com/compose/compose-file/compose-file-v3/#build,
"image: " line is only for rename final image, never say it would check if image already existed, and possibly impacting image building. The official guide is not precise and accurate in this regard.
Related
I am trying to deploy a model on AWS Sagemaker and using the following docker file:
FROM ubuntu:16.04
#MAINTAINER Amazon AI <sage-learner#amazon.com>
RUN apt-get -y update && apt-get install -y --no-install-recommends \
wget \
python3.5-dev \
gcc \
nginx \
ca-certificates \
libgcc-5-dev \
&& rm -rf /var/lib/apt/lists/*
# Here we get all python packages.
# There's substantial overlap between scipy and numpy that we eliminate by
# linking them together. Likewise, pip leaves the install caches populated which uses
# a significant amount of space. These optimizations save a fair amount of space in the
# image, which reduces start up time.
RUN wget https://bootstrap.pypa.io/3.3/get-pip.py && python3.5 get-pip.py && \
pip3 install numpy==1.14.3 scipy lightfm scikit-optimize pandas==0.22.0 flask gevent gunicorn && \
rm -rf /root/.cache
# Set some environment variables. PYTHONUNBUFFERED keeps Python from buffering our standard
# output stream, which means that logs can be delivered to the user quickly. PYTHONDONTWRITEBYTECODE
# keeps Python from writing the .pyc files which are unnecessary in this case. We also update
# PATH so that the train and serve programs are found when the container is invoked.
ENV PYTHONUNBUFFERED=TRUE
ENV PYTHONDONTWRITEBYTECODE=TRUE
ENV PATH="/opt/program:${PATH}"
# Set up the program in the image
COPY lightfm /opt/program
WORKDIR /opt/program
The docker container is built successfully, but when I write the following command:
docker run XYZ train
on my local or even on Sagemaker, I am getting the following error:
standard_init_linux.go:207: exec user process caused "permission denied"
In the docker file I am copying a folder called Lightfm and there is a file called "train" in it.
Can anyone help?
OUTPUT OF MY DOCKER BUILD:
$ docker build -t lightfm .
Sending build context to Docker daemon 41.47kB
Step 1/9 : FROM ubuntu:16.04
---> 5e13f8dd4c1a
Step 2/9 : RUN apt-get -y update && apt-get install -y --no-install-recommends wget python3.5-dev gcc nginx ca-certificates libgcc-5-dev && rm -rf /var/lib/apt/lists/*
---> Using cache
---> 14ae3a1eb780
Step 3/9 : RUN wget https://bootstrap.pypa.io/3.3/get-pip.py && python3.5 get-pip.py && pip3 install numpy==1.14.3 scipy lightfm scikit-optimize pandas==0.22.0 flask gevent gunicorn && rm -rf /root/.cache
---> Using cache
---> 5a2727e27385
Step 4/9 : ENV PYTHONUNBUFFERED=TRUE
---> Using cache
---> 43bf8c5e8414
Step 5/9 : ENV PYTHONDONTWRITEBYTECODE=TRUE
---> Using cache
---> 7d2c45d61cec
Step 6/9 : ENV PATH="/opt/program:${PATH}"
---> Using cache
---> f3cc6313c0d9
Step 7/9 : COPY lightfm /opt/program
---> ad929ba84692
Step 8/9 : WORKDIR /opt/program
---> Running in a040dd0bab03
Removing intermediate container a040dd0bab03
---> 8f53c5a3ba63
Step 9/9 : RUN chmod 755 serve
---> Running in 5666abb27cd0
Removing intermediate container 5666abb27cd0
---> e80aca934840
Successfully built e80aca934840
Successfully tagged lightfm:latest
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.
Assuming train is the executable you want to run, give it exec permission. After COPY lightfm /opt/program line, add RUN chmod +x /opt/program/train.
I have a dockerfile like this:
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
ADD reports /code/
RUN pip install -r requirements.txt
ADD . /code/
RUN ls -l /code/reports/report/manage.py # gives expected result
RUN ls -l /code/reports/build_static/ # gives expected result
RUN python /code/reports/report/manage.py build full_report.views.RenderView # does not work
Everything works fine except for the last command which runs a python package (django-bakery) through manage.py build. I don't get any errors.
This command should output some files inside build_static directory in the container.
If I ssh into the container and run the command manually then it is working. I inserted the full path with /code/ to make sure that they match and created all necessary directories beforehand.
This is how I build the container:
docker-compose run django /bin/bash
This is my docker-compose:
version: '3'
services:
django:
build: .
volumes:
- .:/code
ports:
- "8000:8000"
I wonder how come it is working when I run the command manually through bash inside the container, but not working with the command in the dockerfile.
Thanks!
Update (it seems that the files are created, but then if I check on them they aren't there):
Step 12/12 : RUN ls -l /code/reports/build_static/
---> Running in e294563d26d5
total 11080
-rw-r--r-- 1 root root 11339956 Apr 30 10:53 index.html
drwxr-xr-x 7 root root 4096 Apr 30 10:53 static
Removing intermediate container e294563d26d5
---> b8e72da8ee5c
Successfully built b8e72da8ee5c
Successfully tagged image_django:latest
WARNING: Image for service django was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
root#7483853ecc45:/code# ls -l reports/build_static/
total 0
Try following steps and let me know output:
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
ADD reports /code/
RUN pip install -r requirements.txt
ADD . /code/
RUN ls -l /code/reports/report/manage.py # gives expected result
RUN ls -l /code/reports/build_static/ # gives expected result
RUN python /code/reports/report/manage.py build full_report.views.RenderView
RUN ls -l /code/reports/build_static/ # should give you expected list of files
Give me output for the last step.
I'll help you out based on output.
The following Dockerfile, copies your current directory content to a code folder (if it doesn't exist, it creates it), then sets it as the workdir.
The WORKDIR instruction sets the working directory for any RUN, CMD,
ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile
Then, in order to reduce you docker image size to the maximum, we regroup all your commands in one RUN command so that it reduces the number of layers.
FROM python:3
ENV PYTHONUNBUFFERED 1
COPY . /code
WORKDIR /code
RUN pip install -r requirements.txt && \
ls -l reports/report/manage.py && \
ls -l reports/build_static/ && \
python reports/report/manage.py build full_report.views.RenderView
I haven't tried it with a full Django app example, but it should help you narrow down the problem !
I have a local docker image running with a django project that i am running. The docker image is up but when i go to the local host with port 8000 which is set, gives a message that the page is not working and that 127.0.0.1 didn’t send any data.
Here is the docker code:
omars-mbp:Split omarjandali$ docker build -t split .
Sending build context to Docker daemon 222.2kB
Step 1/7 : FROM python:3
---> 79e1dc9af1c1
Step 2/7 : WORKDIR user
---> 15e014da5b80
Removing intermediate container f4081817276f
Step 3/7 : COPY requirements.txt ./
---> 1f444390862b
Step 4/7 : EXPOSE 8000
---> Running in f75a6674ade2
---> 7641865ffc85
Removing intermediate container f75a6674ade2
Step 5/7 : RUN pip install -r requirements.txt
---> Running in 40738a20f481
Collecting Django==1.11.5 (from -r requirements.txt (line 1))
Downloading Django-1.11.5-py2.py3-none-any.whl (6.9MB)
Collecting gunicorn==19.6.0 (from -r requirements.txt (line 2))
Downloading gunicorn-19.6.0-py2.py3-none-any.whl (114kB)
Collecting pytz (from Django==1.11.5->-r requirements.txt (line 1))
Downloading pytz-2017.3-py2.py3-none-any.whl (511kB)
Installing collected packages: pytz, Django, gunicorn
Successfully installed Django-1.11.5 gunicorn-19.6.0 pytz-2017.3
---> 371a95617f78
Removing intermediate container 40738a20f481
Step 6/7 : COPY . .
---> fa31f9520063
Step 7/7 : CMD python manage.py runserver 0.0.0.0:8000
---> Running in 28b6a097dac4
---> c0d19bca8c2d
Removing intermediate container 28b6a097dac4
Successfully built c0d19bca8c2d
Successfully tagged split:latest
omars-mbp:Split omarjandali$ docker run -d -p 8000:8000 split
d70035b14ab0c2046f2bfc8418960a0b1e5dd8bb75e09a045b41e94bcb097aa4
omars-mbp:Split omarjandali$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d70035b14ab0 split "python manage.py ..." 4 seconds ago Up 2 seconds 0.0.0.0:8000->8000/tcp focused_shirley
docker file:
FROM python:3
WORKDIR user
COPY requirements.txt ./
EXPOSE 8000
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
I would like install auditserver on nodejs server , So my auditserver with rpm . it is working fine as a manual steps.
I write a Dockerfile like below.
FROM centos:centos6
# Enable EPEL for Node.js
RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
# Install Node.js and npm
RUN yum install -y npm
# ADD rpm into container
ADD auditserver-1-1.x86_64.rpm /opt/
RUN mkdir -p /opt/auditserver
RUN cd /opt
RUN rpm -Uvh auditserver-1-1.x86_64.rpm
# cd to auditserver
RUN cd /opt/auditserver
# Install app dependencies
RUN npm install
# start auditserver
RUN node server
EXPOSE 8080
while building the docker file I see below issue.
root#CloudieBase:/tmp/sky-test# docker build -t sky-test .
Sending build context to Docker daemon 38.4 kB
Step 1 : FROM centos:centos6
---> 9c95139afb21
Step 2 : RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
---> Using cache
---> fd5b1bb647fc
Step 3 : RUN yum install -y npm
---> Using cache
---> b7c2908fc583
Step 4 : ADD auditserver-1-1.x86_64.rpm /opt/
---> 26ace798f98c
Removing intermediate container 5ea6221797f5
Step 5 : RUN mkdir -p /opt/auditserver
---> Running in 8f7292364245
---> 9b340033f6b7
Removing intermediate container 8f7292364245
Step 6 : RUN cd /opt
---> Running in c7d20fd251f3
---> 0cdf90b6cb2e
Removing intermediate container c7d20fd251f3
Step 7 : RUN rpm -Uvh auditserver-1-1.x86_64.rpm
---> Running in 4473241e5077
error: open of auditserver-1-1.x86_64.rpm failed: No such file or directory
The command '/bin/sh -c rpm -Uvh auditserver-1-1.x86_64.rpm' returned a non-zero code: 1
root#CloudieBase:/tmp/sky-test#
Can any help on this to made perfect Dockerfile. thanks.
The problem is that you are not in the /opt directory when executing the rpm command (step 7). See this answer to find out why it happens. Quote:
Each time you RUN, you spawn a new container and therefore the pwd is '/'.
For how to fix it see this question. To summarize: you can use the WORKDIR dockerfile command or change this part:
RUN cd /opt
RUN rpm -Uvh auditserver-1-1.x86_64.rpm
to this:
RUN cd /opt && rpm -Uvh auditserver-1-1.x86_64.rpm
So I pulled down the opensuse Docker--only 94M, nice!
I created a Docker file like this:
FROM opensuse
RUN zypper --non-interactive install tar
RUN zypper --non-interactive clean -a
RUN rm -rf /var/log/zypp /var/log/zypper.log
The 'tar' command and its dependencies require ~3M, but the resulting image is 140M! I've cleaned the cache and killed the logs, so what else is causing the roughly 40M bloat here?
You need to do the whole operation in one RUN command
RUN zypper --non-interactive install tar \
&& zypper --non-interactive clean -a \
&& rm -rf /var/log/zypp /var/log/zypper.log
As docker does a commit after each line
redacted history
$ docker history test
IMAGE CREATED CREATED BY SIZE COMMENT
f0c1173538b0 2 minutes ago /bin/sh -c zypper --non-interactive install t 7.429 MB
80bd0f661aef 3 weeks ago /bin/sh -c #(nop) ADD file:4c338e205c079dbf5d 97.78 MB
da0e7dee81d7 8 weeks ago /bin/sh -c #(nop) MAINTAINER Flavio Castelli 0 B