I have a django project and recently added channels to use websockets. This seems to all work fine, but the problem I have is to get the production ready.
My setup is as follows:
Nginx web server
Gunicorn for django
SSL enabled
Since I have added channels to the mix. I have spent the last day trying to get it to work.
On all the turtotials they say you run daphne on some port then show how to setup nginx for that.
But what about having gunicorn serving django?
So now I have guncorn running this django app on 8001
If I run daphne on another port, lets say 8002 - how should it know its par of this django project? And what about run workers?
Should Gunicorn, Daphne and runworkers all run together?
This question is actually addressed in the latest Django Channels docs:
It is good practice to use a common path prefix like /ws/ to
distinguish WebSocket connections from ordinary HTTP connections
because it will make deploying Channels to a production environment in
certain configurations easier.
In particular for large sites it will be possible to configure a
production-grade HTTP server like nginx to route requests based on
path to either (1) a production-grade WSGI server like Gunicorn+Django
for ordinary HTTP requests or (2) a production-grade ASGI server like
Daphne+Channels for WebSocket requests.
Note that for smaller sites you can use a simpler deployment strategy
where Daphne serves all requests - HTTP and WebSocket - rather than
having a separate WSGI server. In this deployment configuration no
common path prefix like is /ws/ is necessary.
In practice, your NGINX configuration would then look something like (shortened to only include relevant bits):
upstream daphne_server {
server unix:/var/www/html/env/run/daphne.sock fail_timeout=0;
}
upstream gunicorn_server {
server unix:/var/www/html/env/run/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
server_name _;
location /ws/ {
proxy_pass http://daphne_server;
}
location / {
proxy_pass http://gunicorn_server;
}
}
(Above it is assumed that you are binding the Gunicorn and Daphne servers to Unix socket files.)
I have created an example how to mix Django Channels and Django Rest Framework. I set nginx routing that:
websockets connections are going to daphne server
HTTP connections (REST API) are going to gunicorn server
Here is my nginx configuration file:
upstream app {
server wsgiserver:8000;
}
upstream ws_server {
server asgiserver:9000;
}
server {
listen 8000 default_server;
listen [::]:8000;
client_max_body_size 20M;
location / {
try_files $uri #proxy_to_app;
}
location /tasks {
try_files $uri #proxy_to_ws;
}
location #proxy_to_ws {
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_pass http://ws_server;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Url-Scheme $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app;
}
}
I recently answered a similiar question, have a look there for an explanation on how django channels work.
Basically, you don't need gunicorn anymore. You have daphne which is the interface server that accepts HTTP/Websockets and you have your workers that run django views. Then obviously you have your channel backend that glues everything together.
To make it work you have to configure CHANNEL_LAYERS in settings.py and also run the interface server: $ daphne my_project.asgi:channel_layer
and your worker:
$ python manage.py runworker
NB! If you chose redis as the channel backend, pay attention to file sizes you're serving. If you have large static files make sure NGINX serves them or otherwise clients will experience cryptic errors that may occur due to redis running out of memory.
Related
The setup is a RHEL AWS instance. On this instance, nginx is installed and working. That means if I go to http://[root] I get the html in the nginx folder like I'm supposed to. If I go to http://[root]/[sub1] I also get the other html in the nginx folder like I'm supposed to.
Now, http://[root]/[sub2] is a django server in a docker container. When runserver, the django app is listening on http://127.0.0.1:8000. My docker container translates :38000->:8000 via docker-compose.yml.
My nginx.conf file looks like this:
server {
listen 80;
root /usr/share/nginx/html;
location / {}
location /test {
index text.html;
alias /usr/share/nginx/html;
}
location /mod {
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
proxy_pass http://127.0.0.1:38000;
}
}
While the root and /test (sub1) locations work, whenever I go to /mod (sub2), I get a 502 Bad Gateway.
My docker-compose.yml (version 2) contains ports: 38000:8000. When I create the container, I use docker-compose run --name mod mod.
Any suggestions?
Two things needed to be fixed before working correctly.
First, the proxy_pass should have been 'http://localhost:38000',
and second, I had to run the Django app to listen to 0:8000 (i.e. python manage.py runserver 0:8000).
Using just runserver or runserver 0.0.0.0:8000 didn't allow NGINX to send to the docker container.
I have Django react app. I am using drf for apis and react as front-end. i want to deploy them on digital ocean separately on same server. any suggestions?
First of all, I must say that my main stack is Django-uWSGI-NGNIX. There is a great instruction, which you can use with DigitalOcean:
How To Serve Django Applications with uWSGI and Nginx
And this one for Gunicorn:
How To Set Up Django with Postgres, Nginx, and Gunicorn
I recommend trying to deploy your Django App on the server first, using these instructions. Maybe it will be just a simple one-page project.
After that you can modify your ngnix configs. In my case it will be:
upstream my_backend_server_name_or_whatever {
# server unix:///path/to/your/mysite/mysite.sock; -- if you want to use sockets
server 127.0.0.1:5000;
}
server {
listen 80;
server_name yourdomain.com/api; #or /back, /backend, /what_you_like
charset utf-8;
client_max_body_size 75M;
location /media #your locantions configs
....
location / {
uwsgi_pass my_backend_server_name_or_whatever;
include /path/to/file/uwsgi_params;
}
}
Also, you need to run your CGI server on port 5000. After that, you can access to your Django app through yourdomain.com/api and it will upstream to localhost:5000.
Try this step with your current Django app. After that, you can configure your DRF to work with these links.
If it will work, the next step. Run your NodeJS server on the other port, like 5100. You can find the same instructions for Webpack or raw NodeJS. After that use the same technics, but for the NodeJS server. In my case:
upstream my_frontend_server_or_whatever {
server localhost:5100;
}
server {
listen 80;
server_name yourdomain.com;
access_log /var/log/nginx/ide.access.log;
error_log /var/log/nginx/ide.error.log;
client_max_body_size 75M;
location / {
proxy_pass http://my_frontend_server_or_whatever;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_redirect off;
}
}
Now you can save both files as backend.conf and frontend.conf, run ngnix and check all configs like in instructions above. After that, you can use yourdomain.com/api links in your React App.
In my opinion, this is the simplest way to try React+DRF. But these configs only for development!! Not for production.
Hope it's helpful.
I have a Flask application running behind NGINX and I am using Gunicorn to deploy. When I deploy, everything works perfectly fine, I can hit my servers IP and see the app running with no issues, however when I execute an action that uses socketio, the action does not get passed to the backend and I believe this is an issue with my configuration on NGINX. My conf.d file has the following
server {
listen 80;
server_name MY_SERVER_IP;
location / {
proxy_pass http://127.0.0.1:8000;
}
location /socket.io {
include proxy_params;
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_pass http://127.0.0.1:8000/socket.io;
}
}
I deploy the app with
gunicorn -k geventwebsocket.gunicorn.workers.GeventWebSocketWorker -w 1 app:app
Within my app.py I am running the socketio server with
socketio.run(app, host='127.0.0.1', port=80, debug=True)
Also seeing this within console...
socket.io.min.js:2 GET http://127.0.0.1:8000/socket.io/?EIO=3&transport=polling&t=MuA1z9K net::ERR_CONNECTION_REFUSED
Everything works locally. Please keep in mind, I am fairly new to Flask deployments with socketio.
Your server is running on port 8000, so your proxy_pass statement should go to that port, not 5000:
proxy_pass http://127.0.0.1:8000/socket.io;
Also note that when you run your server via gunicorn, the socketio.run() line does not execute, that is used when you don't use a third party web server such as gunicorn or uwsgi, and I'm guessing is what you use locally.
My connection to socket.io was set incorrectly. I should have had var socket = io(); in my js script as opposed to var socket = io.connect('127.0.0.1:8000'); per documentation
Firstly apologies if this is a duplicate but i have not found a solution through similar posts shown in SO
I have a Docker Django image which is using nginx and gunicorn.
Gunicorn script:
exec /var/www/venv/bin/gunicorn wsgi:application \
--bind 0.0.0.0:8001 \
--access-logfile /var/log/gunicorn/access.log \
--error-logfile /var/log/gunicorn/error.log
Nginx config:
server {
server_name 172.0.0.1;
access_log off;
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header Host $host:$server_port;
}
location /static/ {
autoindex on;
alias /var/www/django/assets/;
expires 7d;
}
}
I am exposing port 80 and mapping to 49260.
When browsing to the docker host external ip including the port the site is published and serves the static files.
http://xxx.xx.xx.xxx:49260/
The issue is when i navigate to any other page in the django site, the mapped port is dropped from the URL which is then picked up by the host server ngnix config.
What i am trying to achieve is maintain the port in the URL which i can later reverse proxy from the host server.
Any advice would be really appreciated.
The answer was adding:
proxy_set_header Host $http_host;
to the nginx conf which prints hostname:portnumber
See serverfault.com link here: Original thread
It's probably related to this question: How to run more than one app on one instance of EC2
But that question only seemed to be talking about multiple node.js apps.
I am trying learn several different things, so I'm building different websites to learn Ruby on Rails, LAMP, and node.js. Along with my personal website and blog.
Is there any way to run all these on the same EC2 instance?
First, there's nothing EC2-specific about setting up multiple web apps on one box. You'll want to use nginx (or Apache) in "reverse proxy" mode. This way, the web server listens on port 80 (and 443), and your apps run on various other ports. Each incoming request reads the "Host" header to map the request to a backend. So different DNS names/domains show different content.
Here is how to setup nginx in reverse proxy mode: http://www.cyberciti.biz/tips/using-nginx-as-reverse-proxy.html
For each "back-end" app, you'll want to:
1) Allocate a port (3000 in this example)
2) write an upstream stanza that tells it where your app is
3) write a (virtual) server stanza that maps from the server name to the upstream location
For example:
upstream app1 {
server 127.0.0.1:3000; #App1's port
}
server {
listen *:80;
server_name app1.example.com;
# You can put access_log / error_log sections here to break them out of the common log.
## send request to backend
location / {
proxy_pass http://app1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
I prefer to have Nginx in front of Apache for two reasons: 1) nginx can serve static files with much less memory, and 2) nginx buffers data to/from the client, so people on slow internet connections don't clog your back-ends.
When testing your config, use nginx -s reload to reload the config, and curl -v -H "Host: app1.example.com" http://localhost/ to test a specific domain from your config
Adding to the #Brave answer, I would like to mention the configuration of my nginx for those who are looking for the exact syntax in implementing it.
server {
listen 80;
server_name mysite.com;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_pass http://127.0.0.1:3000;
}
}
server {
listen 80;
server_name api.mysite.com;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_pass http://127.0.0.1:4500;
}
}
Just create two server objects with unique server name and the port address.
Mind proxy_pass in each object.
Thank you.