docker nginx connection refused while connecting to upstream - web-services

I use shiny server to build a web-app on port 3838, when i use nginx in my server it works well. But when I stop nginx on my server and try to use docker nginx, I find the site comes to a '502-Bad Gate Way' error and nginx log shows:
2016/04/28 18:51:15 [error] 8#8: *1 connect() failed (111: Connection refused) while connecting to upstream, ...
I install docker-nginx by this command:
sudo docker pull nginx
My docker command line is something like (for clear i add some indent):
sudo docker run --name docker-nginx -p 80:80
-v ~/docker-nginx/default.conf:/etc/nginx/conf.d/default.conf
-v /usr/share/nginx/html:/usr/share/nginx/html nginx
I create a folder name 'docker-nginx' in my home dir, move my nginx conf file in this folder, and then remove my original conf in etc/nginx dir just in case.
My nginx conf file looks like this:
server {
listen 80 default_server;
# listen [::]:80 default_server ipv6only=on;
root /usr/share/nginx/html;
index index.html index.htm;
# Make site accessible from http://localhost/
server_name localhost;
location / {
proxy_pass http://127.0.0.1:3838/;
proxy_redirect http://127.0.0.1:3838/ $scheme://$host/;
auth_basic "Username and Password are required";
auth_basic_user_file /etc/nginx/.htpasswd;
# enhance the performance
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}

You have to define upstream directly. Currently your nginx can not proxy to your web application.
http://nginx.org/en/docs/http/ngx_http_upstream_module.html
upstream backend {
server backend1.example.com weight=5;
server backend2.example.com:8080;
server unix:/tmp/backend3;
server backup1.example.com:8080 backup;
server backup2.example.com:8080 backup;
}
server {
location / {
proxy_pass http://backend;
}
}

My situation was running 3 containers, a nginx container and two containerized services. I was using the Nginx container as a reverse proxy for my go services.
Issue is the nginx container was looking for microservice ports in its own container environment. I didn't realize that at the time and I didn't use the docker-compose.yml then. When using docker-compose.yml file you specify a network and that's that.
So when running the containers you should use --net=host.
Info on that: What does --net=host option in Docker command really do?
This worked for me, I hope it saves someone the pain :):
docker run --net=host nginx:someTag
docker run --net=host service1:someTag

Related

AWS w/nginx redirect to docker containers; getting bad gateway 502

The setup is a RHEL AWS instance. On this instance, nginx is installed and working. That means if I go to http://[root] I get the html in the nginx folder like I'm supposed to. If I go to http://[root]/[sub1] I also get the other html in the nginx folder like I'm supposed to.
Now, http://[root]/[sub2] is a django server in a docker container. When runserver, the django app is listening on http://127.0.0.1:8000. My docker container translates :38000->:8000 via docker-compose.yml.
My nginx.conf file looks like this:
server {
listen 80;
root /usr/share/nginx/html;
location / {}
location /test {
index text.html;
alias /usr/share/nginx/html;
}
location /mod {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_pass http://127.0.0.1:38000;
}
}
While the root and /test (sub1) locations work, whenever I go to /mod (sub2), I get a 502 Bad Gateway.
My docker-compose.yml (version 2) contains ports: 38000:8000. When I create the container, I use docker-compose run --name mod mod.
Any suggestions?
Two things needed to be fixed before working correctly.
First, the proxy_pass should have been 'http://localhost:38000',
and second, I had to run the Django app to listen to 0:8000 (i.e. python manage.py runserver 0:8000).
Using just runserver or runserver 0.0.0.0:8000 didn't allow NGINX to send to the docker container.

Django Channels Nginx production

I have a django project and recently added channels to use websockets. This seems to all work fine, but the problem I have is to get the production ready.
My setup is as follows:
Nginx web server
Gunicorn for django
SSL enabled
Since I have added channels to the mix. I have spent the last day trying to get it to work.
On all the turtotials they say you run daphne on some port then show how to setup nginx for that.
But what about having gunicorn serving django?
So now I have guncorn running this django app on 8001
If I run daphne on another port, lets say 8002 - how should it know its par of this django project? And what about run workers?
Should Gunicorn, Daphne and runworkers all run together?
This question is actually addressed in the latest Django Channels docs:
It is good practice to use a common path prefix like /ws/ to
distinguish WebSocket connections from ordinary HTTP connections
because it will make deploying Channels to a production environment in
certain configurations easier.
In particular for large sites it will be possible to configure a
production-grade HTTP server like nginx to route requests based on
path to either (1) a production-grade WSGI server like Gunicorn+Django
for ordinary HTTP requests or (2) a production-grade ASGI server like
Daphne+Channels for WebSocket requests.
Note that for smaller sites you can use a simpler deployment strategy
where Daphne serves all requests - HTTP and WebSocket - rather than
having a separate WSGI server. In this deployment configuration no
common path prefix like is /ws/ is necessary.
In practice, your NGINX configuration would then look something like (shortened to only include relevant bits):
upstream daphne_server {
server unix:/var/www/html/env/run/daphne.sock fail_timeout=0;
}
upstream gunicorn_server {
server unix:/var/www/html/env/run/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name _;
location /ws/ {
proxy_pass http://daphne_server;
}
location / {
proxy_pass http://gunicorn_server;
}
}
(Above it is assumed that you are binding the Gunicorn and Daphne servers to Unix socket files.)
I have created an example how to mix Django Channels and Django Rest Framework. I set nginx routing that:
websockets connections are going to daphne server
HTTP connections (REST API) are going to gunicorn server
Here is my nginx configuration file:
upstream app {
server wsgiserver:8000;
}
upstream ws_server {
server asgiserver:9000;
}
server {
listen 8000 default_server;
listen [::]:8000;
client_max_body_size 20M;
location / {
try_files $uri #proxy_to_app;
}
location /tasks {
try_files $uri #proxy_to_ws;
}
location #proxy_to_ws {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_pass http://ws_server;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Url-Scheme $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app;
}
}
I recently answered a similiar question, have a look there for an explanation on how django channels work.
Basically, you don't need gunicorn anymore. You have daphne which is the interface server that accepts HTTP/Websockets and you have your workers that run django views. Then obviously you have your channel backend that glues everything together.
To make it work you have to configure CHANNEL_LAYERS in settings.py and also run the interface server: $ daphne my_project.asgi:channel_layer
and your worker:
$ python manage.py runworker
NB! If you chose redis as the channel backend, pay attention to file sizes you're serving. If you have large static files make sure NGINX serves them or otherwise clients will experience cryptic errors that may occur due to redis running out of memory.

Docker django nginx gunicorn URL dropping port

Firstly apologies if this is a duplicate but i have not found a solution through similar posts shown in SO
I have a Docker Django image which is using nginx and gunicorn.
Gunicorn script:
exec /var/www/venv/bin/gunicorn wsgi:application \
--bind 0.0.0.0:8001 \
--access-logfile /var/log/gunicorn/access.log \
--error-logfile /var/log/gunicorn/error.log
Nginx config:
server {
server_name 172.0.0.1;
access_log off;
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header Host $host:$server_port;
}
location /static/ {
autoindex on;
alias /var/www/django/assets/;
expires 7d;
}
}
I am exposing port 80 and mapping to 49260.
When browsing to the docker host external ip including the port the site is published and serves the static files.
http://xxx.xx.xx.xxx:49260/
The issue is when i navigate to any other page in the django site, the mapped port is dropped from the URL which is then picked up by the host server ngnix config.
What i am trying to achieve is maintain the port in the URL which i can later reverse proxy from the host server.
Any advice would be really appreciated.
The answer was adding:
proxy_set_header Host $http_host;
to the nginx conf which prints hostname:portnumber
See serverfault.com link here: Original thread

gunicorn, nginx, and using port 80 for running a django web application

I have django, nginx, and gunicorn installed on a web server.
Nginx listens on port 80
Gunicorn runs django project on port 8000
This works fine. If I go to www.mysite.com:8000/myapp/ the django application comes up OK. But what if I want users to go to www.mysite.com/myapp/ to view the django application? I don't think getting rid of Nginx is the answer, and I'm hoping I missed some configuration tweak i can apply to make this work.
Any advice is appreciated.
You can use the following configuration, so you can access your website normally on port 80:
this is your nginx configuration file, sudo vim /etc/nginx/sites-available/django
upstream app_server {
server 127.0.0.1:9000 fail_timeout=0;
}
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
root /usr/share/nginx/html;
index index.html index.htm;
client_max_body_size 250M;
server_name _;
keepalive_timeout 15;
# Your Django project's media files - amend as required
location /media {
alias /home/xxx/yourdjangoproject/media;
}
# your Django project's static files - amend as required
location /static {
alias /home/xxx/yourdjangoproject/static;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server;
}
}
and configure gunicorn as
description "Gunicorn daemon for Django project"
start on (local-filesystems and net-device-up IFACE=eth0)
stop on runlevel [!12345]
# If the process quits unexpectadly trigger a respawn
respawn
setuid yourdjangousernameonlinux
setgid yourdjangousernameonlinux
chdir /home/xxx/yourdjangoproject
exec gunicorn \
--name=yourdjangoproject \
--pythonpath=yourdjangoproject \
--bind=0.0.0.0:9000 \
--config /etc/gunicorn.d/gunicorn.py \
yourdjangoproject.wsgi:application
No, getting rid of nginx is definitely not the answer. The answer is to follow the very nice documentation to configure nginx as a reverse proxy to gunicorn.

nginx can't listen on port 80

I'm trying to set nginx with gunicorn but I keep getting the "Welcome to nginx!" page. I am able to successfully listen to other ports (like 8080) but port 80 does not work at all.
server {
listen 80;
server_name host.ca www.host.ca;
access_log /var/log/nginx/example2.log;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://127.0.0.1:8000;
}
}
I'm running the server as root.
I can't seem to see anything running in port 80.
Diagnosing the Problem
Make sure to check your logs (likely /var/log/nginx or some variant).
Check to see what might be hogging port 80
netstat -nlp | grep 80
Sites-enabled, port hogging
Then, make sure you have the Django site enabled in sites-enabled. Delete any old symlinks if you created one first.
rm /etc/nginx/sites-enabled/django
ln -s /etc/nginx/sites-available/django /etc/nginx/sites-enabled/django
Double check your /etc/nginx/nginx.conf to make sure it's loading sites-enabled and not loading some other default.
http {
...
include /etc/nginx/sites-enabled/*;
}
After you do all this, shut down and restart the nginx service.
Either service nginx restart or service nginx stop && service nginx start