KeyError: REQUEST_METHOD with Django + uWSGI + nginx + docker [closed] - django

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 3 days ago.
Improve this question
I've built a django REST-API and I want to deploy it on my server. Everything works fine in local dev environment. But on my server I have an error with nginx doesn't pass uwsgi_params correctly (I think).
The error
It seems to me that my configs work fine, since when I access my domain name, the logs appear on the server. However, the error seems to me that nginx doesn't pass the reqeust_method to wsgi in django.
I did found another post here which had the same error, however, I double checked the uwsgi_params file and it's not empty. So I have no clue.
simuzones_server_web | Traceback (most recent call last):
simuzones_server_web | File "/home/tx7093/.local/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 130, in __call__
simuzones_server_web | request = self.request_class(environ)
simuzones_server_web | File "/home/tx7093/.local/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 78, in __init__
simuzones_server_web | self.method = environ["REQUEST_METHOD"].upper()
simuzones_server_web | KeyError: 'REQUEST_METHOD'
simuzones_server_web | [pid: 11|app: 0|req: 1/1] () {48 vars in 977 bytes} [Sun Jan 22 13:28:53 2023] => generated 0 bytes in 5 msecs ( 500) 0 headers in 0 bytes (0 switches on core 0)
simuzones_server_web | Traceback (most recent call last):
simuzones_server_web | File "/home/tx7093/.local/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 130, in __call__
simuzones_server_web | request = self.request_class(environ)
simuzones_server_web | File "/home/tx7093/.local/lib/python3.8/site-packages/django/core/handlers/wsgi.py", line 78, in __init__
simuzones_server_web | self.method = environ["REQUEST_METHOD"].upper()
simuzones_server_web | KeyError: 'REQUEST_METHOD'
simuzones_server_web | [pid: 11|app: 0|req: 2/2] () {44 vars in 855 bytes} [Sun Jan 22 13:28:53 2023] => generated 0 bytes in 4 msecs ( 500) 0 headers in 0 bytes (0 switches on core 0)
Django app docker
I've created a dockerfile for my django app which runs and serves the application using uwsgi. The dockerfile is as follow
FROM python:3.8.15-slim-buster
ARG user
ARG DEBIAN_FRONTEND=noninteractive
ENV PATH="/docker_scripts:/home/${user}/.local/bin:${PATH}"
ENV PYTHONUNBUFFERED 1
...
RUN mkdir /simuzones_server
COPY ./simuzones_server /simuzones_server
WORKDIR /simuzones_server
COPY ./docker_scripts /docker_scripts
RUN chmod +x /docker_scripts/*
RUN mkdir -p /vol/web/media
RUN mkdir -p /vol/web/static
RUN adduser --disabled-login $user
RUN chown -R $user:$user /vol
RUN chmod -R 755 /vol/web
USER $user
COPY requirements.txt /simuzones_server/
RUN /usr/local/bin/python -m pip install --upgrade pip
RUN cd /simuzones_server/
RUN pip install -r requirements.txt
CMD ["entrypoint.sh"]
The entrypoint.sh file simply launches the uwsgi server.
#!/bin/sh
set -e
python manage.py collectstatic --noinput
uwsgi --socket :8000 --master --enable-threads --module simuzones_server.wsgi
Nginx docker
The Dockerfile
FROM nginxinc/nginx-unprivileged:1-alpine
COPY ./default.conf /etc/nginx/conf.d/default.conf
COPY ./uwsgi_params /etc/nginx/uwsgi_params
USER root
RUN mkdir -p /vol/static
RUN chmod 755 /vol/static
USER nginx
And the default.conf file
server {
listen 8080;
listen [::]:8080;
server_name simuzones.tk www.simuzones.tk;
return 301 https://$server_name$request_uri;
}
server {
# SSL configuration
listen 443 ssl http2;
listen [::]:443 ssl http2;
ssl_certificate /etc/ssl/cert.pem;
ssl_certificate_key /etc/ssl/key.pem;
server_name simuzones.tk www.simuzones.tk;
location /static {
alias /vol/static;
}
location / {
include /etc/nginx/uwsgi_params;
uwsgi_pass web:8000;
}
}
And the uwsgi_params file on host machine (server)
uwsgi_param QUERY_STRING $query_string;
uwsgi_param REQUEST_METHOD $request_method;
uwsgi_param CONTENT_TYPE $content_type;
uwsgi_param CONTENT_LENGTH $content_length;
uwsgi_param REQUEST_URI $request_uri;
uwsgi_param PATH_INFO $document_uri;
uwsgi_param DOCUMENT_ROOT $document_root;
uwsgi_param SERVER_PROTOCOL $server_protocol;
uwsgi_param REMOTE_ADDR $remote_addr;
uwsgi_param REMOTE_PORT $remote_port;
uwsgi_param SERVER_ADDR $server_addr;
uwsgi_param SERVER_PORT $server_port;
uwsgi_param SERVER_NAME $server_name;
The docker-compose (if needed)
version: '3.7'
services:
web:
build:
args:
user: tx7093
context: .
ports:
- "8000:8000"
volumes:
- ./simuzones_server:/simuzones_server
- static_data:/vol/web
container_name: simuzones_server_web
environment:
- POSTGRES_DB=${POSTGRES_DB}
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- PGDATA=/var/lib/postgresql/data/pgdata
- DATABASE_URL=${DATABASE_URL}
- DEBUG=1
- SECRET_KEY=${SECRET_KEY}
- ALLOWED_HOSTS=${ALLOWED_HOSTS}
depends_on:
- db
nginx:
build:
context: ./nginx
volumes:
- static_data:/vol/static
- /etc/ssl:/etc/ssl
ports:
- "8080:8080"
depends_on:
- web
db:
image: postgres:11.18-bullseye
environment:
- POSTGRES_DB=${POSTGRES_DB}
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
- PGDATA=/var/lib/postgresql/data/pgdata
- DATABASE_URL=${DATABASE_URL}
ports:
- "5432:5432"
volumes:
- type: bind
source: ./tmp/postgres-data
target: /var/lib/postgresql/data
volumes:
static_data:
I would like to say thanks in advance, this problem takes me days and I couldn't figure it out yet.

Related

Nginx error with django staticfiles "no such file or directory"

I'm trying to make nginx process a request for static files in a django application, but it throws an error
[error] 20#20: *1 open() "/home/app/web/staticfiles/admin/css/nav_sidebar.css" failed (2: No
such file or directory), client: 172.20.0.1, server: , request: "GET
/staticfiles/admin/css/nav_sidebar.css HTTP/1.1", host: "127.0.0.1:1337", referrer:
"http://127.0.0.1:1337/admin/"
What did i do wrong? I understand that the request comes to the right place, but for some reason it is not possible to make a response from this directory
nginx.conf
upstream hello_django {
server web:8000;
}
server {
listen 80;
location / {
proxy_pass http://hello_django;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /staticfiles/ {
alias /home/app/web/staticfiles/;
}
}
docker-compose.yml
version: '3.9'
services:
web:
build: .
command: gunicorn pets.wsgi:application --bind 0.0.0.0:8000
volumes:
- static_volume:/home/app/web/staticfiles
expose:
- 8000
env_file:
- .env
depends_on:
- db
db:
image: postgres:12.0-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file:
- .env
nginx:
build: ./nginx
volumes:
- static_volume:/home/app/web/staticfiles
ports:
- 1337:80
depends_on:
- web
volumes:
postgres_data:
static_volume:
Dockerfile
FROM python:3.10.0-alpine
WORKDIR /usr/src/app
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
RUN pip install --upgrade pip
RUN apk update
RUN apk add postgresql-dev gcc python3-dev musl-dev
COPY ./requirements.txt .
RUN pip install -r requirements.txt
COPY ./entrypoint.sh .
RUN sed -i 's/\r$//g' /usr/src/app/entrypoint.sh
RUN chmod +x /usr/src/app/entrypoint.sh
COPY . .
ENV HOME=/home/app
ENV APP_HOME=/home/app/web
RUN mkdir $HOME
RUN mkdir $APP_HOME
RUN mkdir $APP_HOME/staticfiles
ENTRYPOINT ["/usr/src/app/entrypoint.sh"]
entrypoint.sh
#!/bin/sh
if [ "$DATABASE" = "postgres" ]
then
echo "Waiting for postgres..."
while ! nc -z $POSTGRES_HOST $POSTGRES_PORT; do
sleep 0.1
done
echo "PostgreSQL started"
fi
cd pets
exec "$#"
Not completely sure this is the issue, but remove the trailing slash in your nginx config. Instead of:
location /staticfiles/ {
alias /home/app/web/staticfiles/;
}
use this
location /staticfiles/ {
alias /home/app/web/staticfiles;
}

WebSocket connection getting rejected in production when using docker container

My WebSocket connection getting rejected from the Django Channels server deployed in AWS EC2. I am using Docker for server side which contains PostgreSQL and Redis also
WebSocket HANDSHAKING /ws/notifications/
WebSocket REJECT /ws/notifications/
WebSocket DISCONNECT /ws/notifications/
front-end is built using React Js is deployed in netlify.
server side is secured using cer-bot.
my nginx configuration files looks like this
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
server_name IP_address example.com ;
location = /favicon.ico { access_log off; log_not_found off; }
location /staticfiles/ {
root /home/ubuntu/social-network-django-server;
}
location / {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/_____; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/____; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = example.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name IP_address example.com ;
return 404; # managed by Certbot
}
everything other than the websocket connection is working fine without any issues. I have no idea what is happening here. I am using d]Django channels first time,
I am using wss:// instead of ws://
my error in the browser looks like this
WebSocket connection to 'wss://_____________________' failed:
if any one have any idea what is happening please share!
my github repo links server client
my Dockerfile looks like this
FROM python:3.11.1
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
postgresql-client \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
my docker-compose.yaml looks like this
version: '3'
services:
db:
image: postgres
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=2221
- POSTGRES_DB=social-network
volumes:
- db-data:/var/lib/postgresql/data/
ports:
- "5432"
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
environment:
- DATABASE_URL=postgres://postgres:2221#db:5432/social-network
- REDIS_URL=redis://redis:6379/0
redis:
image: redis
ports:
- "6379"
volumes:
db-data:
I think this is maybe related to redis, can someone explain what is this error is all about, I am new to Redis and first time I'm using it
my websocket connection is rejected by server in the production environment.
most of the time it is getting rejected, but one time I got the error like this
future: <Task finished name='Task-2137' coro=<Connection.disconnect() done, defined at /usr/local/lib/python3.11/site-packages/redis/asyncio/connection.py:720> exception=RuntimeError('Event loop is closed')>
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/redis/asyncio/connection.py", line 729, in disconnect
self._writer.close() # type: ignore[union-attr]
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/streams.py", line 344, in close
return self._transport.close()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 839, in close
self._loop.call_soon(self._call_connection_lost, None)
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
self._check_closed()
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
raise RuntimeError('Event loop is closed')

502 Bad Gateway for NGINX USWGI and Django app

I am having issues running this locally. I have two containers in my app. The Django app and the nginx server. Below are the config files and dockerfiles. I am getting a 502 on the localhost:8000 and the error message is
2021/11/16 01:18:29 [error] 23#23: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.30.0.1, server: localhost, request: "GET /favicon.ico HTTP/1.1", upstream: "uwsgi://127.0.0.1:8888", host: "localhost:8000", referrer: "http://localhost:8000/"
docker-compose.yml
version: "3.9"
services:
web:
build: .
container_name: test_deploy_web
hostname: blindspot
command: uwsgi --ini uwsgi.ini
volumes:
- .:/app/
- staticfiles:/app/static/
nginx:
build: ./nginx
container_name: test_deploy_nginx
volumes:
- staticfiles:/app/static/
ports:
- 8000:80
depends_on:
- web
volumes:
staticfiles:
app dockerfile
FROM python:3
ENV PYTHONUNBUFFERED=1
RUN mkdir /app
WORKDIR /app
RUN pip install --upgrade pip
COPY requirements.txt /app/
RUN pip install -r requirements.txt
COPY . /app
RUN python manage.py collectstatic --noinput
nginx dockerfile
FROM nginx:latest
COPY nginx.conf /etc/nginx/nginx.conf
COPY my_nginx.conf /etc/nginx/sites-available/
RUN mkdir -p /etc/nginx/sites-enabled/\
&& ln -s /etc/nginx/sites-available/my_nginx.conf /etc/nginx/sites-enabled/\
&& rm /etc/nginx/conf.d/default.conf
CMD ["nginx", "-g", "daemon off;"]
my_nginx.conf
# the upstream component nginx needs to connect to
upstream blindspot {
server 127.0.0.1:8888;
#server unix:/app/app.sock; # for a file socket
}
# configuration of the server
server {
# the port your site will be served on
listen 80;
# the domain name it will serve for
server_name localhost;
# Django media
# location /media {
# alias /usr/src/app/static/media; # your Django project's media files - amend as required
# }
location /static {
alias /app/static/;
}
# Finally, send all non-media requests to the Django server.
location / {
uwsgi_pass blindspot;
include uwsgi_params; # the uwsgi_params file you installed
}
}
uwsgi.ini
[uwsgi]
wsgi-file = /app/blindspot/wsgi.py
socket = 127.0.0.1:8888
master=true# maximum number of worker processes
processes = 4
threads = 2# Django's wsgi file
module = blindspot.wsgi:application
vacuum=true

nginx: [emerg] host not found in upstream when dockerizing a django/react project

I'm trying to dockerize a django/react project but i'm running into this error when running docker-compose up.
i don't understand where the error come from.
i'm new to docker.
my goal is to deploy the frontend and the backend separately in one server.
nginx: [emerg] host not found in upstream
[emerg] 1#1: host not found in upstream "backend:8000" in /etc/nginx/conf.d/default.conf:2
nginx_1 | nginx: [emerg] host not found in upstream "backend:8000" in /etc/nginx/conf.d/default.conf:2
here is my code
django's Dockerfile
ENV DJANGO_SECRET_KEY $DJANGO_SECRET_KEY
ENV DJANGO_CORS_ORIGIN_WHITELIST $DJANGO_CORS_ORIGIN_WHITELIST
RUN mkdir /backend
WORKDIR /backend
COPY requirements.txt /backend/
EXPOSE 8000
RUN pip install -r requirements.txt
COPY . /backend/
RUN python manage.py makemigrations
RUN python manage.py migrate
react's Dockerfile
FROM node
USER root
WORKDIR /frontend
COPY . /frontend
ARG API_URL
ENV REACT_APP_HOST_IP_ADDRESS $API_URL
RUN yarn
RUN yarn config set ignore-engines true
RUN yarn build
server's config
upstream api {
server backend:8000;
}
server {
listen 8080;
location /api/ {
proxy_pass http://api$request_uri;
}
# ignore cache frontend
location ~* (service-worker\.js)$ {
add_header 'Cache-Control' 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
expires off;
proxy_no_cache 1;
}
location / {
root /var/www/frontend;
try_files $uri $uri/ /index.html;
}
}
docker-copose.yml
version: '3'
services:
backend:
build:
context: ./ruelle_backend
args:
DJANGO_ALLOWED_HOSTS: http://ruelle-online.fr
DJANGO_SECRET_KEY: anis1807
DJANGO_CORS_ORIGIN_WHITELIST: http://ruelle-online.fr
command: gunicorn ivan_backend.wsgi --bind 0.0.0.0:8000
ports:
- "8000:8000"
frontend:
build:
context: ./ruelle_frontend
args:
API_URL: http://ruelle-online.fr
volumes:
- build_folder:/frontend/build
nginx:
image: nginx:latest
ports:
- 80:8080
volumes:
- ./webserver/nginx-proxy.conf:/etc/nginx/conf.d/default.conf:ro
- build_folder:/var/www/frontend
depends_on:
- backend
- frontend
volumes:
build_folder:

App accessible via port 3000 after deploy to server

I have a django app running in docker containers (please see docker compose and dockerfile below). I have removed port exposure from my docker-compose however when i deploy the code onto an ubuntu server, I can still access the app via port 3000. I am also using nginx to do the proxing (see nginx file below).
services:
rabbitmq:
restart: always
image: rabbitmq:3.7
...
db:
restart: always
image: mongo:4
...
cam_dash:
restart: always
build: .
command: python3 manage.py runserver 0.0.0.0:3000
...
celery:
restart: always
build: .
command: celery -A dashboard worker -l INFO -c 200
...
celery_beat:
restart: always
build: .
command: celery beat -A dashboard -l info --scheduler django_celery_beat.schedulers:DatabaseScheduler
...
FROM python:3.7
COPY requirements.txt /
RUN pip3 install -r /requirements.txt
ADD ./ /dashboard
WORKDIR /dashboard
COPY ./docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 3000
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
return 301 https://$host$request_uri;
root /var/www/html;
index index.html;
}
server {
listen 443;
server_name camonitor.uct.ac.za;
ssl on;
ssl_certificate /etc/ssl/certs/wildcard.crt;
ssl_certificate_key /etc/ssl/private/wildcard.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
location / {
root /var/www/html;
index index.html;
}
location /dash/ {
proxy_pass http://127.0.0.1:3000/dash/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
}
...
I am expecting that if I try access https://example.com:3000/dash/, it should not be accessible. https://example.com/dash/ works just fine.
Thanks for the help.
You should prevent access to port 3000 using the system's firewall.
I had same issue hosting more than one web server on same machine and proxying with Nginx, I solved using this port configuration in docker-compose.yml, binding the port only to localhost, maybe you could apply same configuration to python server.
"127.0.0.1:3000:3000"
version: '3'
services:
myService:
image: "myService/myService:1"
container_name: "myService"
ports:
- "127.0.0.1:3000:3000"