Docker Django installation error of Pillow Package - django

I am dockerising my django apps, you know all if you use django image field, you need to use Pillow package but currently my docker installing all the package and show error when it try install pillow
my Dockerfile
# pull official base image
FROM python:3.7-alpine
# set work directory
WORKDIR /app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV DEBUG 0
# install psycopg2
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev \
&& apk add postgresql-dev \
&& pip install psycopg2 \
&& apk del build-deps
# install dependencies
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# copy project
COPY . .
# collect static files
RUN python manage.py collectstatic --noinput
# add and run as non-root user
RUN adduser -D myuser
USER myuser
# run gunicorn
CMD gunicorn projectile.wsgi:application --bind 0.0.0.0:$PORT
and this is requirements.txt file
Django==2.2.2
Pillow==5.0.0
dj-database-url==0.5.0
gunicorn==19.9.0
whitenoise==4.1.2
psycopg2==2.8.4
I am not getting whats wrong with it, why pilow not installing, it throws an error, this is below:
The headers or library files could not be found for zlib,
remote: a required dependency when compiling Pillow from source.
remote:
remote: Please see the install instructions at:
remote: https://pillow.readthedocs.io/en/latest/installation.html
remote:
remote:
remote: ----------------------------------------
remote: ERROR: Command errored out with exit status 1: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-k4_gcdwn/Pillow/setup.py'"'"'; __file__='"'"'/tmp/pip-install-k4_gcdwn/Pillow/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-qgvai9fm/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.
Can anyone help me to fix this error?
Thanks

You can add zlib-dev in your apk add section and install pillow there. For example(explanations are in comment section):
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev zlib-dev postgresql-dev jpeg-dev \ # will be removed after dependent python packages are installed
&& apk add postgresql zlib jpeg \ # these packages won't be deleted from docker image
&& pip install psycopg2 Pillow==5.0.0 \ # Here I am installing these python packages which have dependencies on the libraries installed in build-deps, because later build-deps will be deleted
&& apk del build-deps # for reducing the size of the Docker Image, we are removing the build-deps folder
# install dependencies
COPY ./requirements.txt .
RUN pip install -r requirements.txt
# Rest of the code

Related

Exception Value: libFreeCADApp.so: cannot open shared object file: No such file or directory

I can't figure out how to use external library as python module in production. Any help on this issue is much appreciated.
I am importing FreeCAD as python module in my django app as follow.
views.py
import sys
sys.path.append('freecad/lib')
import FreeCAD
import Part
And Freecad bin and library files reside at root directory where manage.py file is as shown below.
Everything works fine on my local sever. I can import FreeCad and do data processing on CAD files.
But things start to break when I deploy app on google cloud engine. After deployment it threw this error.
Exception Value: libFreeCADApp.so: cannot open shared object file: No such file or directory
I also built docker image of this application to make sure consistent dependencies. But same result local sever finds Freecad library and runs fine, but docker throws this error.
ModuleNotFoundError: No module named 'FreeCAD'.
Docker file content
# Base Image
FROM python:3.8.5
# set default environment variables # Dont wory about this
ENV PYTHONUNBUFFERED 1
ENV LANG C.UTF-8
ENV DEBIAN_FRONTEND=noninteractive
# create and set working directory
RUN mkdir /app
WORKDIR /app
# Add current directory code to working directory
ADD . /app/
# Pass requirements to Docker
COPY ./requirements.txt /requirements.txt
# set project environment variables
# grab these via Python's os.environ
# these are 100% optional here
ENV PORT=8888
# Install system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
tzdata \
python3-setuptools \
python3-pip \
python3-dev \
python3-venv \
git \
&& \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# install environment dependencies
RUN pip3 install --upgrade pip
RUN pip3 install pipenv
RUN pip3 install -r /requirements.txt
# Install project dependencies
RUN pipenv install --skip-lock --system --dev
EXPOSE 8888
CMD gunicorn mod_project.wsgi:application --bind 0.0.0.0:$PORT
requirements.txt
asgiref==3.3.1
cachetools==4.1.1
certifi==2020.12.5
cffi==1.14.4
chardet==3.0.4
cycler==0.10.0
Django==3.1.4
django-storages==1.10.1
google-api-core==1.23.0
google-auth==1.23.0
google-cloud-core==1.4.4
google-cloud-storage==1.33.0
google-crc32c==1.0.0
google-resumable-media==1.1.0
googleapis-common-protos==1.52.0
gunicorn==20.0.4
idna==2.10
kiwisolver==1.3.1
matplotlib==3.3.3
numpy==1.19.4
numpy-stl==2.13.0
Pillow==8.0.1
protobuf==3.14.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
PyMySQL==0.10.1
pyparsing==2.4.7
python-dateutil==2.8.1
python-utils==2.4.0
pytz==2020.4
requests==2.25.0
rsa==4.6
six==1.15.0
sqlparse==0.4.1
stripe==2.55.1
urllib3==1.26.2

Docker Compose Up is running while with the same image its not running the image?

docker-compose up working fine. Screenshot attached.
Here is the docker-compose file
version: '3.0'
services:
web:
build: .
command: python manage.py runserver 0.0.0.0:9090
ports:
- 9090:9090
env_file:
- .env
Dockerfile
FROM python:3.7-alpine
RUN mkdir -p /app
COPY . /app
COPY .env /app
WORKDIR /app/
RUN apk --update add python3-dev
RUN apk add mariadb-dev mariadb-client
RUN apk --update add python3 py-pip openssl ca-certificates py-openssl wget
RUN apk update && \
apk upgrade --no-cache && \
apk add --no-cache \
gcc g++ make libc-dev linux-headers
RUN pip install --upgrade pip
RUN pip install uwsgi
RUN pip install -r requirements.txt --default-timeout=10000 --no-cache-dir
EXPOSE 9090
docker run testbackend_web:latest
Above Command not working with the build
Can someone help in the same?
Error in Container

resolve pipenv install weasyprint error in Docker

I'm using Alpine linux for my Docker setup.
Here is the Dockerfile.
# pull official base image
FROM python:3.7.4-alpine
# set work directory
WORKDIR /usr/src/app
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
RUN apk --update --upgrade --no-cache add cairo-dev pango-dev gdk-pixbuf
RUN apk update \
&& apk add --virtual build-deps gcc python3-dev musl-dev jpeg-dev zlib-dev libffi-dev\
&& apk add postgresql \
&& apk add postgresql-dev \
&& pip install psycopg2 \
&& apk add jpeg-dev zlib-dev libjpeg \
&& pip install Pillow \
&& apk del build-deps
# install dependencies
RUN pip install --upgrade pip
RUN pip install pipenv
COPY ./Pipfile /usr/src/app/Pipfile
RUN pipenv install --skip-lock --system --dev
# copy entrypoint.sh
COPY ./entrypoint.sh /usr/src/app/entrypoint.sh
# copy project
COPY . /usr/src/app/
# run entrypoint.sh
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]
which results in stalling on the installation of cairocffi and giving the error of
unable to execute 'gcc': No such file or directory.
This page is gonna save your life, no matter if you're using pipenv or not,
For Alpine(>=3.6), use
apk --update --upgrade add gcc musl-dev jpeg-dev zlib-dev libffi-dev cairo-dev pango-dev gdk-pixbuf-dev
I found this link however, which recommends adding the line:
RUN apk add --update python python-dev py-pip build-base
to the build file and works.

How to setup docker-compose, pytest, Selenium Driver and geckodriver/chromedriver correctly

I am testing my django app with pytest. Now I want to use Selenium for system tests. I am also using docker so to execute my tests I do docker-compose -f local.yml run django pytest
When I try to use Selenium I get the message that 'geckodriver' executable needs to be in PATH..
I read many questions here on SO and I followed the instructions. Yet, I think my path can't be found because I am executing everything in a docker-container. Or am I mistaken?
I added the geckodriver path to my bash-profile by checking my path with:
which geckodriver. The result of this is:/usr/local/bin/geckodriver
Then I added this to my profile and when I type echo $PATH I can see this: /usr/local/bin/geckodriver/bin:. So I assume all is set correctly. I also tried setting it manually doing:
self.selenium = webdriver.Firefox(executable_path=r'/usr/local/bin/geckodriver')
Since I still get the error I assume it has sth to do with docker. That's why my specific question is: How can I setup the PATH for geckodriver with pytes, selenium and docker-compose?
Any help or hint into the right direction is very much appreciated! Thanks in advance!
Here is my docker-compose file as requested:
FROM python:3.6-alpine
ENV PYTHONUNBUFFERED 1
RUN apk update \
# psycopg2 dependencies
&& apk add --virtual build-deps gcc python3-dev musl-dev \
&& apk add postgresql-dev \
# Pillow dependencies
&& apk add jpeg-dev zlib-dev freetype-dev lcms2-dev openjpeg-dev tiff-dev tk-dev tcl-dev \
# CFFI dependencies
&& apk add libffi-dev py-cffi \
# Translations dependencies
&& apk add gettext \
# https://docs.djangoproject.com/en/dev/ref/django-admin/#dbshell
&& apk add postgresql-client
# Requirements are installed here to ensure they will be cached.
COPY ./requirements /requirements
RUN pip install -r /requirements/local.txt
COPY ./compose/production/django/entrypoint /entrypoint
RUN sed -i 's/\r//' /entrypoint
RUN chmod +x /entrypoint
COPY ./compose/local/django/start /start
RUN sed -i 's/\r//' /start
RUN chmod +x /start
WORKDIR /app
ENTRYPOINT ["/entrypoint"]
When I am debugging by doing
for p in sys.path:
print(p)
I get:
usr/local/bin
/usr/local/lib/python36.zip
/usr/local/lib/python3.6
/usr/local/lib/python3.6/site-packages
So my geckodriver is not there I assume?

gcc error while building docker image for django on windows

I am trying to build a docker image using Visual Studio Code following this tutorial "https://code.visualstudio.com/docs/python/tutorial-deploy-containers".
I created a django app with a connection to a MSSQLserver on azure with the package pyodbc.
During the build of the docker image i receive the following error messages:
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
Failed building wheel for pyodbc
and
unable to execute 'gcc': No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
Failed building wheel for typed-ast
I read solutions for linux systems where one should install python-dev, but since i am working on a windows machine this is no solution.
Then i read that on windows all the needed files are in the 'include' directory of the python installation. But in a venv installation this directory is empty... so i created a directory junction to the original 'include'. The error still exists.
My docker file is included below.
# Python support can be specified down to the minor or micro version
# (e.g. 3.6 or 3.6.3).
# OS Support also exists for jessie & stretch (slim and full).
# See https://hub.docker.com/r/library/python/ for all supported Python
# tags from Docker Hub.
FROM tiangolo/uwsgi-nginx:python3.6-alpine3.7
# Indicate where uwsgi.ini lives
ENV UWSGI_INI uwsgi.ini
# Tell nginx where static files live (as typically collected using Django's
# collectstatic command.
ENV STATIC_URL /app/static_collected
# Copy the app files to a folder and run it from there
WORKDIR /app
ADD . /app
# Make app folder writable for the sake of db.sqlite3, and make that file also writable.
# RUN chmod g+w /app
# RUN chmod g+w /app/db.sqlite3
# If you prefer miniconda:
#FROM continuumio/miniconda3
LABEL Name=hello_django Version=0.0.1
EXPOSE 8000
# Using pip:
RUN python3 -m pip install -r requirements.txt
CMD ["python3", "-m", "hello_django"]
# Using pipenv:
#RUN python3 -m pip install pipenv
#RUN pipenv install --ignore-pipfile
#CMD ["pipenv", "run", "python3", "-m", "hello_django"]
# Using miniconda (make sure to replace 'myenv' w/ your environment name):
#RUN conda env create -f environment.yml
#CMD /bin/bash -c "source activate myenv && python3 -m hello_django"
I could use some help in building the image without the errors.
Based on the answer of 2ps i added these lines almost at the top of the docker file
FROM tiangolo/uwsgi-nginx:python3.6-alpine3.7
RUN apk update \
&& apk add apk add gcc libc-dev g++ \
&& apk add libffi-dev libxml2 libffi-dev \
&& apk add unixodbc-dev mariadb-dev python3-dev
and received a new error...
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
v3.7.1-98-g2f2e944c59 [http://dl-cdn.alpinelinux.org/alpine/v3.7/main]
v3.7.1-105-g7db92f4321 [http://dl-cdn.alpinelinux.org/alpine/v3.7/community]
OK: 9053 distinct packages available
ERROR: unsatisfiable constraints:
add (missing):
required by: world[add]
apk (missing):
required by: world[apk]
The command '/bin/sh -c apk update && apk add apk add gcc libc-dev g++ && apk add libffi-dev libxml2 libffi-dev && apk add unixodbc-dev mariadb-dev python3-dev' returned a non-zero code: 2
Found out that adding
RUN echo "ipv6" >> /etc/modules
helped with the errors above. Taken from: https://github.com/gliderlabs/docker-alpine/issues/55
The app now works, exept that the intended connection to the MsSQL database still not works.
Error at /
('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 13 for SQL Server' : file not found (0) (SQLDriverConnect)")
I think i should get my hands dirty on some docker documentation.
I gave up on the solution with alpine and switched to debian
FROM python:3.7
# needed files for pyodbc
RUN apt-get update
RUN apt-get install gcc libc-dev g++ libffi-dev libxml2 libffi-dev unixodbc-dev -y
# MS SQL driver 17 for debian
RUN apt-get install apt-transport-https \
&& curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -\
&& curl https://packages.microsoft.com/config/debian/9/prod.list > /etc/apt/sources.list.d/mssql-release.list \
&& apt-get update \
&& ACCEPT_EULA=Y apt-get install msodbcsql17 -y
You'll need to use apk to install gcc and other native dependencies needed to build your pip dependencies. For the ones that you listed (typedast and pyodbc), I think they would be:
RUN apk update \
&& apk add apk add gcc libc-dev g++ \
&& apk add libffi-dev libxml2 libffi-dev \
&& apk add unixodbc-dev mariadb-dev python3-dev