I am having problem with Nginx default 80 port when I upload media files - django

I upload a file to the server using the post method, but it appears on the default port 80. However, I run Nginx on port 8001.
This is what I see when I check my endpoint with 8001 port.
File look like port 80.
http://xxx.xxx.x.xx/media/files/excel/macro/data.xlsx
But I check with 8000 port
http://xxx.xxx.x.xx:8000/media/files/excel/macro/data.xlsx
My nginx/default.conf file :
upstream django_asgi {
server django_asgi:8000;
}
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
client_max_body_size 100M;
location / {
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_pass http://django_asgi;
client_max_body_size 100M;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 3600;
proxy_connect_timeout 3600;
proxy_send_timeout 3600;
keepalive_timeout 3600;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
location /static/ {
alias /code/staticfiles/;
}
location /media/ {
alias /code/media/;
}
}
My Dockerfile :
FROM python:3.8
ENV PYTHONUNBUFFERED 1
ENV REDIS_HOST "redis"
RUN mkdir /code
RUN mkdir code/staticfiles
RUN mkdir code/mediafiles
WORKDIR /code
RUN pip install --upgrade pip
RUN pip install psycopg2
COPY requirements.txt /code/
RUN pip install uwsgi
RUN apt-get update && apt-get install -y tdsodbc unixodbc-dev
RUN pip install -r requirements.txt
ADD . /code/
RUN python manage.py collectstatic --no-input
RUN python manage.py migrate --no-input
My docker-compose.yml file :
version: "3.5"
services:
django_asgi:
container_name: django_asgi
build: .
command: daphne -b 0.0.0.0 -p 8000 core.asgi:application
env_file:
- .env
volumes:
- .:/code
restart: always
ports:
- "8000:8000"
networks:
- redisnet
- djangonet
links:
- redis
redis:
image: redis:latest
restart: always
networks:
- redisnet
- djangonet
nginx:
container_name: django_nginx
image: nginx:1.15.0
depends_on:
- django_asgi
volumes:
- ./nginx:/etc/nginx/conf.d
- static_volume:/code/staticfiles
- media_volume:/code/media
networks:
- djangonet
restart: always
ports:
- "8001:80"
adminer:
image: adminer
restart: always
ports:
- 8080:8080
volumes:
static_volume:
media_volume:
networks:
redisnet:
driver: bridge
djangonet:
driver: bridge
My settings.py static and media file structues :
STATIC_URL = 'static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, "static")]
STATIC_ROOT = BASE_DIR / "staticfiles"
MEDIA_URL = "/media/"
MEDIA_ROOT = BASE_DIR / "media"
I check media and static files structure, I think they are same. Static files work with Nginx server but media files do not.

Related

I use django, postgresql, redis, channels, Docker tools for my project.I have some problem with Docker postgresql image

When I run docker-compose up command, this error occurs:
django.db.utils.OperationalError: could not translate host name "postgres_db" to address: Temporary failure in name resolution
My docker-compose.yml file:
version: "3.5"
services:
redis:
image: redis:latest
networks:
- redisnet
postgres_db:
image: postgres
volumes:
- ./data/db:/var/lib/postgresql/data
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
django_wsgi:
container_name: django_wsgi
build:
context: .
command: uwsgi --socket=:9000 --module=core.wsgi:application --py-autoreload=1
volumes:
- .:/code
env_file:
- .env
networks:
- webnet
- redisnet
links:
- redis
- postgres_db:postgres_db
depends_on:
- postgres_db
django_asgi:
container_name: django_asgi
build: .
# command: python manage.py runserver 0.0.0.0:8002
command: daphne -b 0.0.0.0 -p 8002 core.asgi:application
env_file:
- .env
volumes:
- .:/code
networks:
- webnet
- redisnet
links:
- redis
- postgres_db:postgres_db
depends_on:
- postgres_db
nginx:
image: nginx:1.15.0
depends_on:
- django_asgi
- django_wsgi
volumes:
- ./nginx:/etc/nginx/conf.d
- ./static:/static
networks:
- webnet
ports:
- "80:80"
networks:
webnet:
redisnet:
My Dockerfile:
FROM python:3.8
ENV PYTHONUNBUFFERED 1
ENV REDIS_HOST "redis"
RUN mkdir /code
WORKDIR /code
RUN pip install --upgrade pip
RUN pip install psycopg2
COPY requirements.txt /code/
RUN pip install uwsgi
RUN apt-get update && apt-get install -y tdsodbc unixodbc-dev
RUN pip install -r requirements.txt
ADD . /code/
My nginx default.conf file:
upstream django_wsgi {
server django_wsgi:9000;
}
upstream django_asgi {
server django_asgi:8002;
}
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 80 default_server;
root /usr/src/app;
index index.html;
client_max_body_size 60M;
location / {
include /etc/nginx/uwsgi_params;
uwsgi_pass django_wsgi;
uwsgi_param Host $host;
uwsgi_param X-Real-IP $remote_addr;
uwsgi_param X-Forwarded-For $proxy_add_x_forwarded_for;
uwsgi_param X-Forwarded-Proto $http_x_forwarded_proto;
}
location /chat/stream/ {
proxy_pass http://django_asgi;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
location /static {
alias /static;
}
}
I think that for some reason my postgresql image is not visible - but this image is created, I can see it in Docker Desktop.
The problem is that you declared a network for the djangouwsgi service with:
networks:
- webnet
- redisnet
This means that this container will only be able to contact the containers in the webnet or redisnet network.
On the other hand, you did not set any network for the postgres_db service, so this service is not able to comunicate with the services.
I will suggest adding a network to the postgres service and including the djangouwsgi under the same network.
Another option is to just remove all networks definitions, so all the containers inside of the compose file will be on the same network and they will be able to comunicate which each other.

not able to serve static files with nginx on ec2 with django and gunicorn running inside docker container

I am using the below docker compose 'local.yml' to run django on ec2 server
services:
django: &django
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: name_docker
container_name: django
depends_on:
- mariadb
- mailhog
volumes:
- .:/app:z
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.mariadb
oom_kill_disable: True
deploy:
resources:
limits:
cpus: '0.50'
memory: '3G'
ports:
- "8000:8000"
command: /start
it starts with start.sh script which is written as
#!/bin/bash
set -o errexit
set -o pipefail
set -o nounset
# python /app/manage.py collectstatic --noinput
/usr/local/bin/gunicorn config.wsgi --bind 0.0.0.0:8000 --timeout 10000 --workers 5 --threads 5 --chdir=/app
On ec2 after deployment, server is running fine with gunicorn.
Now I added nginx configuration as
server {
listen 3000;
server_name domain_name.in;
access_log /var/log/nginx/django_project-access.log;
error_log /var/log/nginx/django_project-error.log info;
add_header 'Access-Control-Allow-Origin' '*';
keepalive_timeout 5;
# path for staticfiles
location /static {
autoindex on;
alias /static;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://0.0.0.0:8000;
}
}
This configuration is also running fine and I can access it locally on example.in:3000.
But when I try to open admin page then I am not able to see static files there.
Also I tried to collectstatic with the below command
docker-compose -f local.yml run --rm django python manange.py collectstatic --no-input
The files are collected successfully.
What should i do to serve the static files ?
Map your /app/static/ folder from the container into a folder on the host, lets say: /home/ec2/static/ and make sure your nginx has access there.
volumes:
- /home/ec2/static/:/app/static
nginx.conf
...
location /home/ec2/static/ {
autoindex on;
alias /static/;
}
...

NGINX does not serve the collected static files

I am running a dockerized Django application on an AWS EC2 instance, and the page loads without the static files, even though NGINX has them collected. I find this pretty strange, as it had not happened to me until now (at least on my local machine). It would be greatly appreciated if you could help me solve this issue.
This is my docker-compose.yml file:
version: "3"
services:
web:
restart: always
build: .
command: bash -c "
python manage.py makemigrations &&
python manage.py migrate &&
python manage.py creategroups &&
python manage.py initializesuperuser &&
python manage.py collectstatic --noinput &&
daphne -b 0.0.0.0 -p 80 DL.asgi:application"
expose:
- 80
environment:
- DJANGO_SETTINGS_MODULE=DL.production_settings
volumes:
- static_volume:/usr/src/app/DL/static/
- media_volume:/usr/src/app/media_cdn
nginx:
restart: always
build: ./nginx
ports:
- "80:80"
volumes:
- static_volume:/static/
- media_volume:/media/
depends_on:
- web
volumes:
static_volume:
media_volume:
Here's production_settings.py:
STATIC_URL = '/static/'
STATIC_ROOT = "/usr/src/app/DL/static/"
A snippet of the file structure inside the web container (DL_app is the name of the Django app):
usr
|
|──src
|
|──app
|
|──DL
|
|──DL
|
|──DL_app
| |
| |──static
|
|──manage.py
|
|──static
And, lastly, here's nginx.conf:
upstream web {
server web:80;
}
server {
listen 80;
location / {
proxy_pass http://web;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_redirect off;
}
location /static/ {
alias /usr/src/app/DL/static/;
}
location /protected/ {
alias /media/;
}
}
You have mounted static files in nginx container at /static path:
nginx:
restart: always
build: ./nginx
ports:
- "80:80"
volumes:
- static_volume:/static/ # here
But your NGINX configuration defines they are at /usr/src/app/DL/static/
location /static/ {
alias /usr/src/app/DL/static/; # there is no such path in NGINX container
#alias /static/; # this should work
}

I can't seem to get a Docker-Nginx-Django app to refresh the static files

I currently have a Django website that runs on a server using Nginx an Docker. Every time I update my main css file and deploy it, nothing changes, despite hard refreshing the site. When new static files are added, they end up working, but modified files don't seem to change.
This led me to believing that nginx or Docker was caching the file.
I've tried the following:
Clearing the cache in the nginx container, setting the lines expires -1 and sendfile off, rebuilding the containers, with no luck.
The last time I had this problem, I believe I had to delete the volume for it to refresh, which I don't think is a good solution.
Can someone explain what I am doing wrong?
Settings:
STATIC_URL = '/static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, 'static'),]
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_FINDERS = [
"django.contrib.staticfiles.finders.FileSystemFinder",
"django.contrib.staticfiles.finders.AppDirectoriesFinder",
]
nginx configuration:
upstream my_server {
server web:80;
}
server {
listen 80;
server_name mywebsite.com;
location / {
proxy_pass http://my_server;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $Host;
proxy_redirect off;
return 301 https://$host$request_uri;
}
location /.well-known/acme-challenge/ {
root /var/www/certbot;
}
location /static/ {
alias /code/staticfiles/;
expires -1;
}
}
server {
listen 443 ssl;
server_name mywebsite.com;
ssl_certificate /etc/letsencrypt/live/mywebsite.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mywebsite.com/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
location / {
proxy_pass http://my_server;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /code/staticfiles/;
expires -1;
}
}
Docker Compose File
version: '3.7'
services:
web:
build:
context: .
dockerfile: Dockerfile.prod
environment:
- ENVIRONMENT=production
- SECRET_KEY=#^la7#2kl%8$$o9rl)8$$aqvrwq5_x1=r!xc-1p#+(!m4r2vue*&
- DEBUG=0
- EMAIL_HOST_USER=apikey
- EMAIL_HOST_PASSWORD=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
volumes:
- .:/code
- static_volume:/code/staticfiles
networks:
- nginx_network
- db_network
depends_on:
- db
nginx:
image: nginx:1.17.0
ports:
- 80:80
- 443:443
volumes:
- ./config/nginx/conf.d:/etc/nginx/conf.d
- static_volume:/code/staticfiles
- ./data/certbot/conf:/etc/letsencrypt
- ./data/certbot/www:/var/www/certbot
depends_on:
- web
networks:
- nginx_network
command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s
reload; done & nginx -g \"daemon off;\"'"
db:
image: postgres:11
env_file:
- config/db/db_env
networks:
- db_network
volumes:
- db_volume:/var/lib/postgresql/data/
certbot:
image: certbot/certbot
volumes:
- ./data/certbot/conf:/etc/letsencrypt
- ./data/certbot/www:/var/www/certbot
entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew;
sleep 12h & wait $${!}; done;'"
networks:
nginx_network:
driver: bridge
db_network:
driver: bridge
volumes:
db_volume:
static_volume:
Dockerfile:
# Pull base image
FROM python:3.7
# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Set work directory
WORKDIR /code
# Install dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system
# Copy project
COPY . /code/
RUN apt update
RUN apt install -y cron
CMD /code/script_prod.sh

Django and nginx with docker compose - static files are not loaded

I know that several similar questions have been asked, but I have not managed to find a proper answer anywhere.
I want to dockerize my Django app. The app works fine in the docker container but the static files (images, css etc.) are not loaded.
My docker-compose.yml:
version: '2'
services:
web:
build: .
command: ./start.sh
volumes:
- .:/app
- /app/www/static
ports:
- "8000:8000"
env_file: .env
nginx:
build: ./nginx
links:
- web
ports:
- "0.0.0.0:80:80"
volumes_from:
- web
The start.sh script
python manage.py makemigrations --noinput
python manage.py migrate --noinput
python manage.py collectstatic --noinput
exec gunicorn audiobridge.wsgi:application --bind 0.0.0.0:8000 --workers 3
The Dockerfile for the web container:
FROM python:2.7
ENV PYTHONUNBUFFERED 1
RUN mkdir /app
WORKDIR /app
ADD requirements.txt /app/
RUN pip install -r requirements.txt
ADD . /app/
Then I have an nginx/ directory in my project which has the following Dockerfile:
FROM nginx
COPY sites-enabled/audiobridge /etc/nginx/sites-enabled/
and inside nginx/sites-enabled there is the nginx configuration file:
server {
listen 80;
server_name 127.0.0.1;
charset utf-8;
location /static {
alias /app/www/static/;
}
location / {
proxy_pass http://web:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
enter image description here
volumes:
- "../nginx/conf.d:/etc/nginx/conf.d/templates"
- "/static:/static"
Did you try add this code to nginx and django app?