This is the Dockerfile for my Django project:
FROM python:3.10.5-alpine
ENV PYTHONDONTWRITEBYTECODEBYDEFAULT=1
ENV PYTHONUNBUFFERED=1
RUN adduser --disabled-password appuser
USER appuser
WORKDIR /home/appuser/app
COPY requirements.txt .
USER root
RUN python -m pip install --no-cache-dir --disable-pip-version-check --requirement requirements.txt
USER appuser
COPY . .
ENTRYPOINT [ "./entrypoint.sh" ]
And Django settings regarding static assets:
STATIC_URL = 'static/'
STATIC_ROOT = BASE_DIR / 'static/'
And entrypoint.sh
#!/bin/sh
python manage.py makemigrations
python manage.py migrate
python manage.py collectstatic --no-input
gunicorn project.wsgi:application --bind=0.0.0.0:8000 --workers=4 --timeout=300 --log-level=debug --log-file=-
exec "$#"
When I start the container I shell into it and see that static folder is created and populated with admin staff.
However browsing http://127.0.0.1:8000/admin brings up the admin login page without any CSS and I get lots of 404 errors in the developer console.
I also changed STATIC_ROOT to /home/appuser/app/static/ and got the same.
Please assist.
Did you check for the permissions of the created static folder?
I had to manually change the permissions of the folders.
you could try with following Dockerfile for nginx:
FROM nginx:latest
RUN apt-get update && apt-get install -y procps
RUN mkdir -p /home/app/staticfiles
RUN chmod -R 755 /home/app/staticfiles
Could you try using this config.
Django:
STATIC_URL = "/static/"
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
STATICFILES_DIRS = (os.path.join(BASE_DIR, "static"),)
Dockerfile:
RUN mkdir -p /home/appuser/app/staticfiles
Nginx.conf
location /static/ {
alias /home/appuser/app/staticfiles/;
}
docker-compouse.yml something like:
web:
container_name: test_table_django
build: .
command:
sh -c "python manage.py collectstatic --no-input --clear &&
python manage.py migrate &&
gunicorn --workers=4 --bind=0.0.0.0:8000 test_table.wsgi:application"
volumes:
- .:/home/appuser/app
- static_volume:/home/appuser/app/staticfiles/
env_file:
- .env.prod
expose:
- 8000
depends_on:
- db
restart: always
nginx:
build: ./nginx
container_name: test_table_nginx
ports:
- 80:80
volumes:
- static_volume:/home/appuser/app/staticfiles/
The static URL should start with /.
You can also check the logs to see where it is trying to reach to.
BASE_DIR = Path(__file__).resolve().parent.parent.parent
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, '..', 'static')
Related
I am building a Django app with Docker. I run the command collectstatic in my entrypoint when database is ready. When I check my container, the /static/ folder is empty. Thus, Nginx cannot load the static files.
# settings.py
STATIC_URL = '/static/'
STATIC_ROOT = '/static/'
Here is my docker-compose file
version: "3.9"
services:
db:
image: postgis/postgis:14-3.3
container_name: db
volumes:
- ./data/db:/var/lib/postgresql/data
env_file:
- prod.env
backend:
container_name: backend
build:
dockerfile: ./django/Dockerfile
command: gunicorn api.wsgi:application --bind 0.0.0.0:8000
volumes:
- static:/usr/src/app/static
ports:
- "8000:8000"
env_file:
- prod.env
depends_on:
- db
nginx:
container_name: nginx
build:
dockerfile: ./nginx/Dockerfile
volumes:
- static:/usr/src/app/static
ports:
- "80:80"
depends_on:
- backend
restart: always
redis:
container_name: redis
restart: unless-stopped
image: redis:alpine
expose:
- 6379
worker:
container_name: worker
build:
dockerfile: ./django/Dockerfile
command: celery -A api worker -l INFO
volumes:
- static:/usr/src/app/static
env_file:
- prod.env
depends_on:
- db
- backend
- redis
volumes:
static:
My Nginx configuration:
upstream api {
server backend:8000;
}
server {
listen 80;
location / {
proxy_pass http://api;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /usr/src/app/static/;
}
}
backend Dockerfile:
# syntax=docker/dockerfile:1
FROM python:3
WORKDIR /usr/src/app
RUN apt-get update
RUN apt-get install -y libgdal-dev gdal-bin netcat
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
COPY /django/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY /django/django-entrypoint.sh /django-entrypoint.sh
RUN chmod +x /django-entrypoint.sh
COPY django /usr/src/app
ENTRYPOINT ["/django-entrypoint.sh"]
And the entrypoint:
#!/bin/sh
if [ "$POSTGRES_NAME" = "postgres" ]
then
echo "Waiting for Postgres..."
while ! nc -z $POSTGRES_HOST $POSTGRES_PORT; do
sleep 0.1
done
echo "PostgreSQL started"
fi
ls
# python manage.py flush --no-input
python manage.py migrate --no-input
python manage.py collectstatic --no-input
exec "$#"
In my files (local), I do not seem to see the '/static/' folder to be generated. How is this? I have check that static in backend and nginx by ssh in the container and the static folders were empty. In the logs, collectstatic was executed without an error with this message:
backend | 173 static files copied to '/static'.
You use a Docker named volume to hold the static files
volumes:
- static:/usr/src/app/static
# ^^^^^^
# a volume name, not a host path
This named volume only exists inside Docker's storage; you will not see its content on your host system or in your local source tree.
This isn't a problem for the setup you're describing here: since you're re-running collectstatic every time the container starts up, and the volume contents hide the image contents in this directory, there's no particular need for the files to exist in source control or your host filesystem. If you did need them, you could presumably run manage.py collectstatic in a non-Docker virtual environment.
Try by adding the following command in Dockerfile and re-build image.
RUN python manage.py collectstatic --noinput
You can place it after RUN pip install --no-cache-dir -r requirements.txt
I'm using Docker, PostgreSQL, Gunicorn and nginx. I've searched for the answer everywhere. All works fine on the development server.
When I publish to the production server the images do not display in wagtail admin.
Dockerfile.prod
###########
# BUILDER #
###########
# pull official base image
FROM python:3.8.3-alpine as builder
# set work directory
WORKDIR /usr/src/wa-cms
# set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# install Wagtail, Django and psycopg2 dependencies
RUN apk update \
&& apk add \
postgresql-dev \
gcc \
python3-dev \
musl-dev \
build-base \
jpeg-dev \
zlib-dev \
libwebp-dev \
openjpeg-dev
# lint
RUN pip install --upgrade pip --no-cache-dir
RUN pip install flake8
COPY . .
# install dependencies
COPY ./requirements.txt .
RUN pip wheel --no-cache-dir --no-deps --wheel-dir /usr/src/wa-cms/wheels -r requirements.txt
######################
# FINAL - PRODUCTION #
######################
# pull official base image
FROM python:3.8.3-alpine
# create directory for the app user
RUN mkdir -p /home/app
# create the app user
RUN addgroup -S app && adduser -S app -G app
# create the appropriate directories
ENV HOME=/home/app
ENV APP_HOME=/home/app/web
RUN mkdir $APP_HOME
RUN mkdir $APP_HOME/static
RUN mkdir $APP_HOME/media
WORKDIR $APP_HOME
# install dependencies
RUN apk update && apk add libpq \
postgresql-dev \
gcc \
python3-dev \
musl-dev \
build-base \
jpeg-dev \
zlib-dev \
libwebp-dev \
openjpeg-dev
RUN pip install --upgrade pip --no-cache-dir
COPY --from=builder /usr/src/wa-cms/wheels /wheels
COPY --from=builder /usr/src/wa-cms/requirements.txt .
RUN pip install --no-cache /wheels/*
# copy entrypoint-prod.sh
COPY ./entrypoint.prod.sh $APP_HOME
# copy project
COPY . $APP_HOME
# chown all the files to the app user
RUN chown -R app:app $APP_HOME
# change to the app user
USER app
# run entrypoint.prod.sh
ENTRYPOINT ["/home/app/web/entrypoint.prod.sh"]
docker-compose.prod.yml
version: "3.8"
services:
web:
build:
context: ./wa-cms
dockerfile: Dockerfile.prod
command: gunicorn backend.wsgi:application --bind 0.0.0.0:8000
volumes:
- static_volume:/home/app/web/static
- media_volume:/home/app/web/media
restart: unless-stopped
expose:
- 8000
env_file:
- ./.env.prod
depends_on:
- db
db:
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- ./.env.prod.db
restart: unless-stopped
nginx:
build: ./nginx
volumes:
- static_volume:/home/app/web/static
- media_volume:/home/app/web/media
ports:
- 80:80
depends_on:
- web
volumes:
postgres_data:
static_volume:
media_volume:
nginx.conf
upstream wtwacmsdev01 {
server web:8000;
}
server {
client_max_body_size 50M;
listen 80;
location / {
proxy_pass http://wtwacmsdev01;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /home/app/web/static/;
}
location /media/ {
alias /home/app/web/media/;
}
}
I run the commands:
chmod +x wa-cms/entrypoint.prod.sh
docker-compose -f docker-compose.prod.yml up -d --build
winpty docker-compose -f docker-compose.prod.yml exec web python manage.py migrate --noinput
winpty docker-compose -f docker-compose.prod.yml exec web python manage.py collectstatic --no-input --clear
If anybody can give any indication as to what I am doing wrong that would be much appreciated.
I am struggling to suggest options because I can't really imagine using a Docker volume for storing user-uploaded content in production. Dev, yes. But for production I usually have the files stored in S3 so they are always available and have backups. Something like the code below:
AWS_STORAGE_BUCKET_NAME = env('AWS_STORAGE_BUCKET_NAME', default=None)
if AWS_STORAGE_BUCKET_NAME:
# https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
MEDIA_ROOT = ''
MEDIA_URL = "http://{}.s3.amazonaws.com/".format(AWS_STORAGE_BUCKET_NAME)
AWS_S3_REGION_NAME = env('AWS_DEFAULT_REGION', default='us-west-2')
AWS_S3_FILE_OVERWRITE = True
AWS_DEFAULT_ACL = 'private'
else:
# Use a docker volume
DEFAULT_FILE_STORAGE = 'catalog.core.utils.FileSystemStorage'
# https://docs.djangoproject.com/en/4.0/ref/settings/#media-root
MEDIA_ROOT = '/media' # noqa
# https://docs.djangoproject.com/en/4.0/ref/settings/#media-url
MEDIA_URL = ''
When i try to access http://127.0.0.1:8000/admin, i get this.
my folder structure is :
django-react-nginx
|
|_ _ docker-compose.yml
|
> backend
|
|_ Dockerfile
|
|_ entrypoint.sh
> languages
|
|_ settings.py
> media
> static # This folder appears after running docker-compose -d --build
> nginx
|
|_ default.conf
|
|_ Dockerfile
now
here is the files
Django
settings.py
DEBUG = True
ALLOWED_HOSTS = ['*']
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
MEDIA_URL = '/media/'
Docker file
FROM python:3.8
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
WORKDIR /backend
COPY requirements.txt /backend/
RUN pip install -r requirements.txt && \
pip install --upgrade pip
COPY ./entrypoint.sh /
ENTRYPOINT ["sh", "/entrypoint.sh"]
entrypoint.sh
#!/bin/sh
python manage.py migrate --no-input
python manage.py collectstatic --no-input
gunicorn languages.wsgi:application --bind 0.0.0.0:8000
Nginx
default.conf
upstream django {
server backend:8000;
}
server {
listen 80;
location / {
proxy_pass http://django;
}
location /static/ {
autoindex on;
alias /backend/static;
}
location /media/ {
autoindex on;
alias /backend/static;
}
}
Dockerfile
FROM nginx:1.19.8-alpine
COPY ./default.conf /etc/nginx/conf.d/default.conf
Root Folder
Docker-compose
version: "3.7"
services:
backend:
build: ./backend/languages
stdin_open: true # interactive
tty: true # interactive
restart: "on-failure"
env_file:
.env
volumes:
- ./backend/languages:/backend
- ./backend/languages/static:/backend/static
ports:
- 8000:8000
networks:
- nginx_network
- db_network
depends_on:
- db
db:
image: postgres:11
restart: "on-failure"
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
volumes:
- postgres_data:/var/lib/postgresql/data/
networks:
- db_network
nginx:
build: ./nginx
restart: always
volumes:
- ./backend/languages:/backend
- ./backend/languages/static:/static
ports:
- 80:80
networks:
- nginx_network
depends_on:
- backend
networks:
nginx_network:
driver: bridge
db_network:
driver: bridge
volumes:
postgres_data:
As mentioned above i am not sure how to make the static files work, they been copied to the folder mentioned above because when i can see the folder created and when i check the logs i get this 165 static files copied to '/backend/static'.
Nginx config
Nginx config looks fine. Container is mapped to /backend/languages/static with /static and the alias points to the same folder inside container - /static.
nginx.conf
location /static/ {
autoindex on;
alias /static;
}
upd issue detected: an alias must have the same ending slash to work properly. so it has to be alias /static/
nginx in compose
nginx:
...
volumes:
- ./backend/languages:/backend
- ./backend/languages/static:/static
Django config
But Django config looks broken. Here you've configured to collect static to static folder within BASE_DIR
settings.py
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
and this is exactly what collectstatic is reporting to you:
165 static files copied to '/backend/static'
files are in the /backend/static, not /static. Hovewer the container is configured to root folder /static:
django in compose
backend
...
volumes:
- ./backend/languages:/backend
- ./backend/languages/static:/static
looks like the issue can be fixed by pointing to
volumes:
- ./backend/languages/static:/backend/static
Nevertheless there is still some work to do: media files are not supposed to be served by Django as well, so it is recommended to configure Nginx to serve media files too.
Django Dockerfile
I believe the mapping ./backend/languages:/backend works but what's the point of dockerizing then? There is no app in this docker image, just dependencies. It is "better" to include sources in the image so the deployment in the end would require to update images and restart containers only.
So, as a side note, I'd suggest to (at least try to):
update configs to serve media files with Nginx
include django app sources inside your docker image
collect static files during the build and not to include them into docker image
remove collectstatic from the entrypoint
supply static files and treat them as a separate product, think of them if you were going to deliver them to a CDN, a separate hosting - it is a very common solution
whilst they are still on the same hosting, keep mapping them to the container as you do now, but in a "reversed" way: deliver static files to the host folder, access them from the container in ro mode
You can use whitenoise by configuring it in your settings file. Set up Whitenoise with Django here
I'm trying to understand what am I doing wrong when trying to copy my static/media folder to staticfiles/medialfiles on docker.
This is what I have:
settings.py
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
STATICFILES_DIRS = [os.path.join(BASE_DIR, 'static')]
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, "mediafiles")
Dockerfile
FROM python:3.6
ENV PYTHONUNBUFFERED 1
ENV C_FORCE_ROOT true
RUN mkdir /www
WORKDIR /www
COPY . /www/
RUN pip install -r requirements.txt
RUN python3 /www/manage.py collectstatic --noinput
docker-compose
# use this file only for live production
# docker-compose up -d web
version: '3'
volumes:
db_psql:
postgis-data:
esdata:
services:
web:
build: .
restart: always
container_name: django_web
command: gunicorn --bind 0.0.0.0:8080 LG__CXS4.wsgi
depends_on:
- nginx
volumes:
- .:/www
ports:
- "8080:8080"
links:
- redis
nginx:
restart: always
image: "nginx"
ports:
- "80:80"
volumes:
- ./nginx/conf.d:/etc/nginx/conf.d
- ./staticfiles:/static
- ./mediafiles:/media
When I run my docker file I get: 5355 static files copied to '/www/staticfiles'. but when I look into the container's staticfiles -it's empty.
Edit: see the ngnix config file in the answer below that clarify the mistake.
if you are bringing up the containers via docker-compose you can remove
RUN mkdir /www
WORKDIR /www
COPY . /www/
from your web Dockerfile since you're mounting the volume in the composer file
I would think you'd be trying to serve static files from your nginx container, so you'd docker exec -it <nginx_container> bash and making sure the files were copied to /static and /media there right?
the suspects are neither the Dockerfile nor the docker-compose. The problem is with the nginx. It was missing the www in the alias.
what works is:
server {
listen 80;
server_name localhost;
# serve static files
location /static/ {
alias /www/static/;
}
# serve media files
location /media/ {
alias /www/media/;
}
# pass requests for dynamic content to gunicorn
location / {
proxy_pass http://web:8080;
}
}
I change the value of the STATIC_ROOT variable by the address inside the container and not by the project address
STATIC_ROOT = '/webapp/static' #'./static' <-Both work
And then throw the next command in another command window, while the container was running
docker-compose exec name_service_web python manage.py collectstatic --noinput
This is probably the millionth django+nginx post, but since I didn't find an answer after more than 5 hours here it goes:
My issue is that not all static files get served, only some. The whole thing runs inside Docker where I run
RUN python manage.py collectstatic --noinput;
RUN python manage.py makemigrations;
RUN python manage.py migrate;
on each start, but it does not serve my new .js and .css files although they are in the same directories as the old ones.
I also see the message:
132 static files copied to '/static'.
and above that is the list of files being copied there, including the new ones.
Project Structure:
/djangoapp
/app
/static
/css
/js
/django
/Dockerfile
/docker-compose
settings.py:
DEBUG = False
STATIC_URL = '/static/'
STATIC_ROOT = '/static'
nginx.conf:
upstream web {
ip_hash;
server web:8000;
}
server {
location /static {
autoindex on;
alias /static;
}
location / {
proxy_connect_timeout 3600;
proxy_send_timeout 3600;
proxy_read_timeout 3600;
proxy_pass http://web/;
}
listen 8000;
server_name localhost;
}
Why aren't all static files being served?
EDIT:
Dockerfile:
FROM python:3.6.4-onbuild
RUN mkdir /config;
RUN mkdir /src;
COPY . /src
WORKDIR /src
RUN python manage.py collectstatic --noinput;
RUN python manage.py makemigrations;
RUN python manage.py migrate;
RUN chmod 775 -R /static
#this shows that the new files reside with the others in the same directory
RUN ls -R /static/
CMD gunicorn WebInterface.wsgi -b 0.0.0.0:8000 --timeout 3600
docker-compose.yml:
version : '3'
services:
web:
build: ./WebInterface
container_name: WebDocker
volumes:
- static-content:/static
expose:
- "80"
nginx:
image: nginx:1.12.2
container_name: NGINXDocker
ports:
- "8000:8000"
volumes:
- ./WebInterface:/src
- ./config/nginx:/etc/nginx/conf.d/
- static-content:/static
depends_on:
- web
volumes:
static-content:
Ok, with the additional info I think I know what's going on.
You run your python script to generate static files (which I assume it does) on build. On run, you mount the directory, so it will overwrite everything in the static directory.
Either don't mount it or add the scripts
python manage.py collectstatic --noinput;
python manage.py makemigrations;
python manage.py migrate;
in the entrypoint (https://docs.docker.com/engine/reference/builder/#entrypoint)
Also, the mounting of static into the nginx container doesn't do anything since you are only reverse proxying into the django container and not giving a root path. Or if it does work by some coincident, I'm not sure if it would get the newly generated files on startup.
Hope this helps, had a lot of fun with that stuff when I started with docker...
A workaround for now is to create the static folder on my local machine and then transfer the files to the desired position inside docker-compose.
When I find a real solution, on why collect static is not working within docker I will update this
Edit: Solution provided by LevinM
updated the Dockerfile (This only provides the image and installs python libraries from requirements.txt due to the 'onbuild' parameter):
FROM python:3.6.4-onbuild
updated docker-compose.yml:
version : '3'
services:
web:
build: ./WebInterface
entrypoint: bash -c "python manage.py collectstatic --noinput && python manage.py makemigrations && python manage.py migrate && gunicorn WebInterface.wsgi -b 0.0.0.0:8000 --timeout 3600";
container_name: WebDocker
volumes:
- ./WebInterface:/src
- static-content:/static
expose:
- "80"
nginx:
image: nginx:1.12.2
container_name: NGINXDocker
ports:
- "8000:8000"
volumes:
- ./WebInterface:/src
- ./config/nginx:/etc/nginx/conf.d/
- static-content:/static
depends_on:
- web
volumes:
static-content: