Building on Docker Hub ignores context path when copying files - django

I am trying to upload a Django app to Docker Hub. On the local machine (Ubuntu 18.04) everything works fine, but on Docker Hub there is an issue that the requirements.txt file cannot be found.
Local machine:
sudo docker-compose build --no-cache
Result (it's okay):
Step 5/7 : COPY . .
---> 5542d55caeae
Step 6/7 : RUN file="$(ls -1 )" && echo $file
---> Running in b85a55aa2640
Dockerfile db.sqlite3 hello_django manage.py requirements.txt venv
Removing intermediate container b85a55aa2640
---> 532e91546d41
Step 7/7 : RUN pip install -r requirements.txt
---> Running in e940ebf96023
Collecting Django==3.2.2....
But, Docker Hub:
Step 5/7 : COPY . .
---> 852fa937cb0a
Step 6/7 : RUN file="$(ls -1 )" && echo $file
---> Running in 281d9580d608
README.md app config docker-compose.yml
Removing intermediate container 281d9580d608
---> 99eaafb1a55d
Step 7/7 : RUN pip install -r requirements.txt
---> Running in d0e180d83772
[91mERROR: Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
Removing intermediate container d0e180d83772
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1
app/Dockerfile
FROM python:3.8.3-alpine
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /code
COPY . .
RUN file="$(ls -1 )" && echo $file
RUN pip install -r requirements.txt
docker-composer.yml
version: '3'
services:
web:
build:
context: app
dockerfile: Dockerfile
volumes:
- ./app/:/code/
ports:
- "8000:8000"
env_file:
- ./config/.env.dev
command: python manage.py runserver 0.0.0.0:8000
Project Structure:
UPDATE:
Docker is building from Github.
File requirements.txt is in the GitHub repository (app folder), but for some reason during build Docker Hub copies files from the project root folder and not the contents of the app folder.
Github:
https://github.com/sigalglebru/django-on-docker

The problem is that you need to tell Docker Hub where to find your build context.
When you run docker-compose build locally, docker-compose reads your docker-compose.yml file and knows to build inside the app directory, because you've explicitly set the build context:
build:
context: app
dockerfile: Dockerfile
When you build on Docker Hub, by default it will assume the build
context is the top level of your repository. If you set the path to
your Dockerfile to, e.g., app/Dockerfile, this is equivalent to
running:
docker build -f app/Dockerfile .
If you try that, you'll see if fail the same way. Rather than setting
the path to the Dockerfile, you need to set the path to the build
context to the app directory. For example:
(Look at the "Build Context" column).
When configured correct, your repository builds on Docker Hub without errors.

Thank you, I found solution:
I just copied files from./app to the mounted volume, and little changed context, but still don't understand why it worked fine on the local machine
Dockerfile:
FROM python:3.8.3-alpine
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /code
COPY ./app .
RUN pip install -r requirements.txt
docker-compose.yml
version: "3.6"
services:
python:
restart: always
build:
context: .
dockerfile: docker/Dockerfile
expose:
- 8000
ports:
- 8000:8000
command: "python manage.py runserver 0.0.0.0:8000"

Related

CIDC with BitBucket, Docker Image and Azure

EDITED
I am learning CICD and Docker. So far I have managed to successfully create a docker image using the code below:
Dockerfile
# Docker Operating System
FROM python:3-slim-buster
# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE=1
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1
#App folder on Slim OS
WORKDIR /app
# Install pip requirements
COPY requirements.txt requirements.txt
RUN python -m pip install --upgrade pip pip install -r requirements.txt
#Copy Files to App folder
COPY . /app
docker-compose.yml
version: '3.4'
services:
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
ports:
- 8000:8000
My code is on BitBucket and I have a pipeline file as follows:
bitbucket-pipelines.yml
image: atlassian/default-image:2
pipelines:
branches:
master:
- step:
name:
Build And Publish To Azure
services:
- docker
script:
- docker login -u $AZURE_USER -p $AZURE_PASS xxx.azurecr.io
- docker build -t xxx.azurecr.io .
- docker push xxx.azurecr.io
With xxx being the Container registry on Azure. When the pipeline job runs I am getting denied: requested access to the resource is denied error on BitBucket.
What did I not do correctly?
Thanks.
The Edit
Changes in docker-compose.yml and bitbucket-pipeline.yml
docker-compose.yml
version: '3.4'
services:
web:
build: .
image: xx.azurecr.io/myticket
container_name: xx
command: python manage.py runserver 0.0.0.0:80
ports:
- 80:80
bitbucket-pipelines.yml
image: atlassian/default-image:2
pipelines:
branches:
master:
- step:
name:
Build And Publish To Azure
services:
- docker
script:
- docker login -u $AZURE_USER -p $AZURE_PASS xx.azurecr.io
- docker build -t xx.azurecr.io/xx .
- docker push xx.azurecr.io/xx
You didnt specify CMD or ENTRYPOINT in your dockerfile.
There are stages when building a dockerfile
Firstly you call an image, then you package your requirements etc.. those are stages that are being executed while the container is building. you are missing the last stage that executes a command inside the container when its already up.
Both ENTRYPOINT and CMD are essential for building and running Dockerfiles.
for it to work you must add a CMD or ENTRYPOINT at the bottom of your dockerfile..
Change your files accordingly and try again.
Dockerfile
# Docker Operating System
FROM python:3-slim-buster
# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE=1
# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1
#App folder on Slim OS
WORKDIR /app
# Install pip requirements
COPY requirements.txt requirements.txt
RUN python -m pip install --upgrade pip pip install -r requirements.txt
#Copy Files to App folder
COPY . /app
# Execute commands inside the container
CMD manage.py runserver 0.0.0.0:8000
Check you are able to build and run the image by going to its directory and running
docker build -t app .
docker run -d -p 80:80 app
docker ps
See if your container is running.
Next
Update the image property in the docker-compose file.
Prefix the image name with the login server name of your Azure container registry, .azurecr.io. For example, if your registry is named myregistry, the login server name is myregistry.azurecr.io (all lowercase), and the image property is then myregistry.azurecr.io/azure-vote-front.
Change the ports mapping to 80:80. Save the file.
The updated file should look similar to the following:
docker-compose.yml
Copy
version: '3'
services:
foo:
build: .
image: foo.azurecr.io/atlassian/default-image:2
container_name: foo
ports:
- "80:80"
By making these substitutions, the image you build is tagged for your Azure container registry, and the image can be pulled to run in Azure Container Instances.
More in documentation

How can I install Django in my Docker file?

I am a newbie to Docker. I have created one Django project and can run it in Docker. However, I have started a second project and have encountered a problem.
I created a virtual env and entered it
pipenv install django~=3.1.0 && pipenv shell
I created a Django project
django-admin startproject config .
I ran it within the virtualenv
python manage.py runserver
and could see the Django spaceship
I then exited the virtualenv and created a Dockerfile
Dockerfile
# Pull base image
FROM python:3.8
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Set work directory
WORKDIR /code
# Install dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system
# Copy project
COPY . /code/
I ran
docker build .
and it reported a successful build
I created a docker-compose.yml file
docker-compose.yml
version: '3.8'
services:
web:
build: .
command: python /code/manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- 8000:8000
When I run
docker-compose up
it complains
ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?
I have read in the comments to this question that virtual envs should not be used in docker files, so I replaced
RUN pip install pipenv && pipenv install --system
with
RUN pip install django~=3.1.0
but I still get the same error.
What is wrong?
Have you tried installing your list of requirements from a separate file, something like this?
COPY requirements.txt /code/requirements.txt
WORKDIR /code
RUN pip install -r requirements.txt
Once you have it installed you can run docker-compose run web /bin/sh to start a shell and then use django-admin startproject to create a django project. You may need to change the path in the docker-compose file so that it reflects where your manage.py file ended up (I moved mine to the root). I was able to get it working with the following:
requirements.txt
django==3.1.0
docker-compose.yml
version: '3.8'
services:
web:
build:
context: .
dockerfile: Dockerfile
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- 8000:8000
Dockerfile
# Pull base image
FROM python:3.8
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Set work directory
WORKDIR /code
# Install dependencies
COPY requirements.txt /code/requirements.txt
WORKDIR /code
RUN pip install -r requirements.txt
# Copy project
COPY . /code/
File tree looks like this:

Docker ERROR: Couldn't find env file: /home/sam/code/docker/.env.dev

I'm learning how to use docker with Django. So first step is you setup the Dockerfile and here's the content of the file.
FROM python:3.8.0-alpine
# set work directory
WORKDIR /usr/src/app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install dependencies
RUN pip install --upgrade pip
COPY ./requirements.txt /usr/src/app/requirements.txt
RUN pip install -r requirements.txt
# copy project
COPY . /usr/src/app/
another file the docker-compose.yml file content below
version: '3.7'
services:
web:
build: ./project
command: python manage.py runserver 0.0.0.0:8000
volumes:
- ./project/:/usr/src/app/
ports:
- 8000:8000
env_file:
- ./.env.dev
now these two files are in the folder docker which also has my django project folder called project keeping it simple :)
When I run docker-compose build I get back the error
ERROR: Couldn't find env file: /home/sam/code/docker/.env.dev
You are specifying that .env file can be found in the location where your docker-compose.yml file is located.
env_file:
- ./.env.dev
Make sure that your .env file is available there. If you don't need any environment variables to be set just remove above line from the docker-compose.yml file.

Django web app Docker - unable to connect

I am new on Django and Docker and I have a problem to enter site localhost:8000.
I built django app and it is working on my local server but I'd like to dockerize my app. So I created two files:
Dockerfile :
RUN python:3.6.7-alpine
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD ./ /code/
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
and docker-compose.yml
version: '3'
services:
web:
build: .
command: python mysite/manage.py runserver 8000
ports:
- "8000:8000"
My next steps:
docker built --tag django_docker:latest .
and:
docker run django_docker
It's open server, but when I want to open localhost:8000 from my browser I can't because of "Unable to connect"
Where is my fault?
More about django app : it's project from book Python Crash Course : Learning_log. I'd like to build an image and push it to hub docker, but I am stuck. Thanks for help!
You are using a docker-compose.yml file, therefore you need to use the docker-compose command to run it:
docker-compose up
That's all you need, and you can read more about it in the official docs.
To run it without using docker compose, then your docker command needs to be:
docker run --publish 8000:8000 django_docker
If you want to restrict the site to be available only on your localhost, then bind to 127.0.0.1:
docker run --publish 127.0.0.1:8000:8000 django_docker
Try these
update dockerfile
# Pull base image
FROM python:3.7
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Set work directory
WORKDIR /code
# Install dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system
# Copy project
COPY . /code/
update dockor-compose.yml
version: '3.7'
services:
web:
build: .
command: python /code/manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- 8000:8000
after updating just run one commands in terminal
docker-compose up -d --build
to stop it use
docker-compose down

Docker compose is not copying any files

Quite simple: the docker-compose configuration below does not allow any files to persist after running. So when I do docker exec -i -t aas-job-hunter_web_1 ls /app -alt, I see nothing.
Here is the (non-)working minimal example: https://github.com/philastrophist/test-docker
I'm on Windows 10, I've allowed mounted drives and enabled the TLS connection. I'm not sure what else to do. The thing that most confuses me is that requirements.txt is clearly copied over (since it installs it all) but it isn't there when I have a look docker exec.
My directory structure is:
parent/
website/
manage.py
...
Dockerfile
docker-compose.yml
...
Dockerfile:
FROM python:3.6
#WORKDIR /app
# By copying over requirements first, we make sure that Docker will cache
# our installed requirements rather than reinstall them on every build
COPY requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
# Now copy in our code, and run it
COPY . /app
EXPOSE 8000
CMD python website/manage.py runserver 0.0.0.0:8000
# CMD tail -f /dev/null # use when testing
docker-compose.yml:
version: '2'
services:
web:
build: .
ports:
- "8000:8000"
volumes:
- .:/app
links:
- db
db:
image: "postgres:9.6"
ports:
- "5432:5432"
environment:
POSTGRES_PASSWORD: hunter2
Traceback:
> docker-compose -f docker-compose.yml up --build
Building web
Step 1/6 : FROM python:3.6
---> 0668df180a32
Step 2/6 : COPY requirements.txt /app/requirements.txt
---> Using cache
---> 3073d0bef876
Step 3/6 : RUN pip install -r /app/requirements.txt
---> Using cache
---> 8ad63bbb3de5
Step 4/6 : COPY . /app
---> 16390cdd6c2c
Step 5/6 : EXPOSE 8000
---> Running in f628000e8961
Removing intermediate container f628000e8961
---> 80e6994cfbd2
Step 6/6 : CMD python website/manage.py runserver 0.0.0.0:8000
---> Running in acb6b25eb558
Removing intermediate container acb6b25eb558
---> da8876d78103
Successfully built da8876d78103
Successfully tagged aas-job-hunter_web:latest
Starting aas-job-hunter_db_1 ... done
Recreating aas-job-hunter_web_1 ... done
Attaching to aas-job-hunter_db_1, aas-job-hunter_web_1
db_1 | LOG: database system was shut down at 2019-05-24 21:23:31 UTC
db_1 | LOG: MultiXact member wraparound protections are now enabled
db_1 | LOG: database system is ready to accept connections
web_1 | python: can't open file 'website/manage.py': [Errno 2] No such file or directory
aas-job-hunter_web_1 exited with code 2
Actually it copies files.
Solution 1
Change CMD to :
CMD python /app/website/manage.py runserver 0.0.0.0:8000
Solution 2
You call WORKDIR before the /app folder is created. So change your Dockerfile to :
FROM python:3.6.2
# By copying over requirements first, we make sure that Docker will cache
# our installed requirements rather than reinstall them on every build
COPY requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
# Now copy in our code, and run it
COPY . /app
WORKDIR /app
#EXPOSE 8000
CMD python ./website/manage.py runserver 0.0.0.0:8000
# CMD tail -f /dev/null # use when testing
And call it after.
Moreover remember that in your current docker-compose file you are using bind mounts, not volumes, so the context - . will replace entirely the content of /app in your container.
Uncomment #WORKDIR /app.
I also cleaned the other parts a bit up to utilize the WORKDIR more.
FROM python:3.6
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
EXPOSE 8000
CMD python website/manage.py runserver 0.0.0.0:8000
I think there is nothing wrong with COPY. But, you need to set the work directory to /app as your manage.py file is inside /app/website, not in /website inside Docker.
So, I think your Dockerfile should be like this:
FROM python:3.6
RUN mkdir /app
COPY requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
COPY . /app
WORKDIR /app
EXPOSE 8000
CMD python website/manage.py runserver 0.0.0.0:8000