I'm trying to deploy Django on Docker in AWS Lightsail, and the pages are stuck loading with no connection to server - django

I have been trying to deploy a Django app on Lightsail with Gunicorn, NginX, and Docker. I've looked at multiple tutorials, all without success. I'm not familiar with most of the concepts, and I've pretty much been following blindly. So far, everything seems to work on the server itself, but I can't see the results on a webpage. I have configured it for "production" (not sure if I'm even doing it right), and I've added a record to my domain which redirects to this server. The webpage just buffers continuously, even when I try to set it to port 8000 (for development). I think I've gotten a few instances where I saw a "301 5" permanently moved log show up on the docker-compose logs, but that's about it.
Here are the Dockerfile, docker-compose.yml, and nginx conf.d file (which are probably the most important.
docker-compose.yml
version: '3.7'
services:
web:
build:
environment:
- ENVIRONMENT=production
- SECRET_KEY=NOT IMPORTANT
- DEBUG=0
- EMAIL_HOST_USER=EMAIL
- EMAIL_HOST_PASSWORD=PASSWORD
volumes:
- .:/code
- static_volume:/code/staticfiles
depends_on:
- db
networks:
- nginx_network
- db_network
db:
image: postgres:11
env_file:
- config/db/db_env
networks:
- db_network
volumes:
- db_volume:/var/lib/postgresql/data
nginx:
image: nginx:1.17.0
ports:
- 80:80
volumes:
- ./config/nginx/conf.d:/etc/nginx/conf.d
- static_volume:/code/staticfiles
depends_on:
- web
networks:
- nginx_network
networks:
nginx_network:
driver: bridge
db_network:
driver: bridge
volumes:
db_volume:
static_volume:
Dockerfile:
# Pull base image
FROM python:3.7
# Environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
# Work directory
WORKDIR /code
# Dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system
# Copy project
COPY . /code/
# expose port
EXPOSE 80
# gunicorn
CMD ["gunicorn", "--chdir", "my_project", "--bind", ":80", "mbdebate_project.wsgi:application"]
conf.d:
upstream hello_server {
server web:80;
}
server {
listen 80;
server_name mydomain.com;
location / {
proxy_pass http://hello_server;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /code/staticfiles/;
}
}
The settings are pretty standard, and I don't think the problem is there. Any help would truly be appreciated :).
The tutorial I followed: tpawamoy.github.io/2018/02/01/docker-compose-django-postgres-nginx.html

When working on a flask application I had a similar issue connecting with nginx.
On nginx.conf, I used the same host configuration in my app.run.
Change web:80; in your conf.d to the host exposed by Django application.**
My case:-
app.py
if __name__ == "__main__":
app.run(host='0.0.0.0', port=5000)
nginx.conf
upstream flask-app {
server 0.0.0.0:5000;
}
server {
listen 80 default_server;
# https for production
location / {
proxy_pass http://flask-app;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
}

Related

Publishing Two Sites with Django/Docker/Nginx Reverse Proxy?

How can I publish two different Django projects with two domains to one host using Docker in my DigitalOcean Droplet? Should I set Nginx in the docker-compose.yml file, or should I set it on the host system (etc/nginx/…)? Or should I run nginx in a separate container?
When I search on the internet, people who publish django projects with docker usually write their nginx settings in the docker-compase.yml file, but I couldn't understand how to set the reverse proxy in this way to publish two different sites.
Nginx settings - Dockerfile, docker-compose.yml would be very appreciated if you give information about settings. Thanks in advance.
I'm sorry for my bad English :( :(
docker-compose.yml
version: '3'
services:
djangoapp:
build: .
volumes:
- .:/opt/services/djangoapp/src
- static_volume:/opt/services/djangoapp/static
- media_volume:/opt/services/djangoapp/media
networks:
- nginx_network
- database1_network
depends_on:
- database1
nginx:
image: nginx
ports:
- 8000:80
volumes:
- ./config/nginx/conf.d:/etc/nginx/conf.d
- static_volume:/opt/services/djangoapp/static
- media_volume:/opt/services/djangoapp/media
depends_on:
- djangoapp
networks:
- nginx_network
database1:
image: postgres
env_file:
- config/db/database1_env
networks:
- database1_network
volumes:
- database1_volume:/var/lib/postgresql/data
networks:
nginx_network:
driver: bridge
database1_network:
driver: bridge
volumes:
database1_volume:
static_volume:
media_volume:
Nginx - local.conf
upstream hello_server {
server djangoapp:8000;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://hello_server;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8000;
}
location /static/ {
alias /opt/services/djangoapp/static/;
}
location /media/ {
alias /opt/services/djangoapp/media/;
}
}
Dockerfile
FROM python:3.8
RUN mkdir -p /opt/services/djangoapp/src
WORKDIR /opt/services/djangoapp/src
COPY Pipfile Pipfile.lock /opt/services/djangoapp/src/
RUN pip install pipenv && pip install --upgrade pip && pipenv install --system
RUN pip install django-ckeditor
RUN pip install Pipfile
COPY . /opt/services/djangoapp/src
EXPOSE 8000
CMD ["gunicorn", "--chdir", "hello", "--bind", ":8000", "hello.wsgi:application"]

can't access to server. Nginx is not working on docker, Django Project, Daphne, Gunicorn, Redis

I am configuring Nginx for ASGI(with Daphne), WSGI(Gunicorn) as well as Redis and Postgres to deploy Django Project.
it was working find but all of sudden the browser is not able to access the server(172.0.0.1:1337) after set up django settings.py for check --deploy with following Nginx logs, I don't think it can cause problems because I just added some configurations in local compose file but compose.prod.yml for production.
I didn't set the .sh file yet, every pages that I googled for below messages mention those files for running docker-compose in production. would it be a problem not configuring that .sh file? but It was working even without the file.
which part should I check for this problem and any configurations that I need to add?
Thanks in advance!
if more information needed, let me know.
nginx | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
nginx | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
nginx | 10-listen-on-ipv6-by-default.sh: /etc/nginx/conf.d/default.conf is not a file or does not exist, exiting
nginx | /docker-entrypoint.sh: Configuration complete; ready for start up
nginx Dockerfile
FROM nginx:1.19.0-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d
#########
EDIT
** here I changed 2nd line COPY ./nginx.conf /etc/nginx/conf.d/default.conf and now nginx logs display like below**
nginx | 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf
nginx | 10-listen-on-ipv6-by-default.sh: info: /etc/nginx/conf.d/default.conf differs from the packaged version
#########
Nginx.conf
upstream web {
server web:8000 fail_timeout=0;
}
upstream daphne {
server daphne:8001 fail_timeout=0;
}
server {
listen 80;
location / {
proxy_pass http://web;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /ws/ {
proxy_pass http://daphne;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
location /staticfiles/ {
alias /Users/kz/Desktop/projects/community_app/staticfiles/;
}
location /media/ {
alias /Users/kz/Desktop/projects/community_app/media/;
}
}
docker-compose.prod.yml
version: '3.8'
services:
web:
build: .
container_name: web
command: gunicorn config.wsgi:application --bind 0.0.0.0:8000
volumes:
- static_volume:/address/
- media_volume:/address/
expose:
- 8000
depends_on:
- commu_db
- redis
environment:
- "DJANGO_SETTINGS_MODULE=config.settings"
- "DJANGO_SECRET_KEY=##"
daphne:
build: .
restart: always
container_name: daphne
command: daphne -b 0.0.0.0 -p 8001 config.asgi:application
volumes:
- static_volume:/address/
- media_volume:/address/
expose:
- 8001
# ports:
# - "8001:8001"
environment:
- "DJANGO_SETTINGS_MODULE=config.settings"
- "DJANGO_SECRET_KEY=###"
depends_on:
- redis
- commu_db
- web
nginx:
build: ./nginx
container_name: nginx
volumes:
- static_volume:/address/
- media_volume:/address/
ports:
- 1337:80
depends_on:
- web
- daphne
commu_db:
image: postgres:13
container_name: postgres
volumes:
- postgres_test_data:/var/lib/postgresql/data/
ports:
- 5433:5432
environment:
- "POSTGRES_PASSWORD=postgres"
redis:
container_name: redis
hostname: comm_redis
restart: always
image: redis:5
ports:
- "6379:6379"
volumes:
postgres_test_data:
static_volume:
media_volume:

Nginix configuration for django nuxt app with docker hosted on ec2

I have a dockerized app that works fine in development mode on my host machine. I'm trying to figure out how I can host my app on ec2 using the default ip address created when I launch my instance.
My folder structure is as follows.
backend
|---projectname
|---Dockerfile
|---requirements.txt
|---wait-for-it.sh
config/nginx
|---app.conf
frontend
|---nuxt folders
|---Dockerfile
This is my current docker compose file I'm using
docker-compose.yml
version: '3.4'
services:
db:
restart: always
image: postgres
volumes:
- pgdata:/var/lib/postgresql/data
env_file: .env
ports:
- "5432:5432"
expose:
- 5432
redis:
restart: always
image: redis
volumes:
- redisdata:/data
django:
build:
context: ./backend
env_file: .env
command: >
sh -c "./wait-for-it.sh db:5432 &&
cd autobets && python manage.py collectstatic --noinput &&
gunicorn --workers=2 --bind=0.0.0.0:8000 autobets.wsgi:application"
ports:
- "8000:8000"
volumes:
- ./backend:/app
depends_on:
- db
restart: on-failure
nuxt:
build:
context: ./frontend
environment:
- API_URI=http://django:8000/api
command: bash -c "npm install && npm run dev"
volumes:
- ./frontend:/app
ports:
- "3000:3000"
depends_on:
- django
- redis
volumes:
pgdata:
redisdata:
config/nginx/app.config
upstream django {
ip_hash;
server django:8000;
}
upstream nuxt {
ip_hash;
server nuxt:3000;
}
server {
location ~ /(api|admin|static)/ {
proxy_pass http://django;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
}
location / {
proxy_pass http://nuxt;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
}
listen 8000;
server_name localhost;
}
Say if my ec2 domain name pointer is ec2-52-204-122-132.compute-1.amazonaws.com
How do I set nginx up in my app to accept http connections to the frontend of my app?
On the backend localhost:8000/admin is my admin page I'd like to access this also using the ec2 domain name too.
What's the best way to alter my config so when I push my app after add the domain name pointer I can access my app hosted on ec2?
I've been reading documentation but can't find any helpful info for a dockerized django vue type app running on ec2.
firstly you need to make sure the security group attach to your box is open for incoming connection on port that NGinx listen
For each container you want to put on NGinx config you will need to find their do to so do:
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id
NGinx is not inside a container,docker-compose so, nginx.conf
upstream django {
ip_hash;
server <container IP>:8000; # <-- Change this, with container IP
}
upstream nuxt {
ip_hash;
server <container IP>:3000; # <-- Change this, with container IP
}
server {
location ~ /(api|admin|static)/ {
proxy_pass #django # <-- Change this, with container IP
proxy_pass <container IP>:8000; # <-- OR Change this, with container IP
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
}
location / {
proxy_pass #nuxt # <-- Change this, add port
proxy_pass <container IP>:3000 # <-- OR Change this, add port
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
}
listen 8000; # <-- Change port number, django already use this host port
server_name localhost ec2-52-204-122-132.compute-1.amazonaws.com; # <-- change this line, add EC2 public domain
}
docker-compose.yml
version: '3.4'
services:
db:
restart: always
image: postgres
volumes:
- pgdata:/var/lib/postgresql/data
env_file: .env
ports:
- "5432:5432"
expose:
- 5432
networks: # <-- Add this
- random_name # <-- Add this
redis:
restart: always
image: redis
volumes:
- redisdata:/data
networks: # <-- Add this
- random_name # <-- Add this
django:
build:
context: ./backend
env_file: .env
command: >
sh -c "./wait-for-it.sh db:5432 &&
cd autobets && python manage.py collectstatic --noinput &&
gunicorn --workers=2 --bind=0.0.0.0:8000 autobets.wsgi:application"
ports:
- "8000:8000"
volumes:
- ./backend:/app
depends_on:
- db
restart: on-failure
networks: # <-- Add this
- random_name # <-- Add this
nuxt:
build:
context: ./frontend
environment:
- API_URI=http://django:8000/api # <-- Wrong
# From you Javascript on the client point of view,
# you will request the public server, not the internal name of
# you backend inside a container, that the public will never see
- API_URI=http://ec2-52-204-122-132.compute-1.amazonaws.com/api # <-- Right
command: bash -c "npm install && npm run dev"
volumes:
- ./frontend:/app
ports:
- "3000:3000"
depends_on:
- django # <-- will become useless if change API_URI=
- redis # <-- bad design
networks: # <-- Add this
- random_name # <-- Add this
volumes:
pgdata:
redisdata:
networks: # <-- Add this
- random_name: # <-- Add this wanto make sure container can communicate
Security group on AWS should be open on the port you'll use for listening on NGinx, host 8000 is already used by django, so use another on
From app architecture POV, your backend should do the cache stuff with Redis, not the Frontend. Complexity or backend response caching should be cache on the backend somewhere on the controllers. You client should cache only statis assets. but you here to make you have working on a server, not to speak archi.

Set nginx proxy for react multi-stage build with docker compose

I am - once again - very confused, with nginx, docker and react.
Here is what is what I want: 1.) a Django REST api that exposes a port only locally 2.) I want the staticfiles of the REST api handled by nginx 3.) a ReactJS front end that is served via nginx on port 80 (I dont know if it is nessesary to serve react via nginx - but I've heard it reduces image size).
The problem: It does not work. All containers can run individually but serving them via docker compose will not run properly. I dont seem to be able to proxy to the api and frontend.
I hint to my problem: What I am seeing is that the image I am layering in reactJS "tiangolo/node-frontend:10" also copies a nginx.conf file that may overwrite mine.
Using a million tutorials, I am here:
nginx.conf
upstream website_rest {
server restapi:8000;
}
upstream website_frontend {
server frontend:8080;
}
server {
listen 80;
location /rest_call/ {
proxy_pass http://website_rest;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location / {
proxy_pass http://frontend;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /rest_call/staticfiles/ {
alias /usr/src/website-dj/staticfiles/;
}
}
This is the dockerfile for react:
# Stage 0, "build-stage", based on Node.js, to build and compile the frontend
FROM tiangolo/node-frontend:10 as build-stage
WORKDIR /website-rj
COPY package*.json /website-rj/
RUN npm install
COPY ./ /website-rj/
RUN npm run build
# Stage 1, based on Nginx, to have only the compiled app, ready for production with Nginx
FROM nginx:1.15
COPY --from=build-stage /website-rj/build/ /usr/share/nginx/html
# Copy the default nginx.conf provided by tiangolo/node-frontend
COPY --from=build-stage /nginx.conf /etc/nginx/conf.d/default.conf
Dockercompose:
version: '3.7'
services:
frontend:
expose:
- 8080
build: "./website_rj_docker"
volumes:
- ./website_rj_docker/build:/usr/src/website-rj/
restapi:
build: "./website_dj_docker/"
# command: python /usr/src/website-dj/manage.py runserver 0.0.0.0:8000 --settings=rest.settings.production
command: gunicorn rest.wsgi:application --bind 0.0.0.0:8000
volumes:
- ./website_dj_docker/:/usr/src/website-dj/
- static_volume:/usr/src/website-dj/staticfiles
expose:
- 8000
environment:
- SECRET_KEY='something...'
- SQL_ENGINE=django.db.backends.postgresql
- SQL_DATABASE=postgres
- SQL_USER=something...
- SQL_PASSWORD=something...
- SQL_HOST=db
- SQL_PORT=5432
- DATABASE=postgres
depends_on:
- db
db:
image: postgres:10.5-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
nginx:
build: ./nginx
volumes:
- static_volume:/usr/src/website-dj/staticfiles
ports:
- 1337:80
depends_on:
- restapi

Why isn't nginx handling requests in this docker-compose/django setup?

I'm trying to learn docker and docker-compose, and have run into a roadblock.
The stack is a python:2.7 base image serving up django pages, and this part works fine. I now want to put nginx in front of it as a reverse proxy. When I access localhost:8000 I get django pages as expected. When I load up localhost with no port, I get nothing ("Problem loading page"). I assume the connection between django container's port 8080 and nginx port 80 isn't happening, and I'm so new to docker that it's probably something simple that I'm not seeing.
docker-compose.yml
version: '2'
services:
web:
build: ./app/
command: python manage.py runserver 0.0.0.0:8000
volumes:
- ./app/:/code
ports:
- "8000:8000"
depends_on:
- db
nginx:
restart: always
build: ./nginx/
ports:
- "80:80"
volumes:
- /www/static
volumes_from:
- web
links:
- web:web
# probably not relevant to my issue
db:
image: postgres
redis:
restart: always
image: redis:latest
ports:
- "6380:6380"
volumes:
- ./redisdata/:/data
nginx config:
server {
listen 80;
server_name 127.0.0.1;
charset utf-8;
location /static {
alias /code/static;
}
location / {
proxy_pass http://web:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
The nginx service starts successfully and appears to be running. I checked it's log using
docker-compose logs nginx
And it was empty.