Deploy Django\Tornado on Heroku - django

I want to deploy some app on Heroku, it's Django with Tornado(Tornadio2) server for implement WebSockets with socket.io protocol.
So, at now it's working fine on my VPS server,
I use Nginx with using location section for routing requests for Django or Tornado.
Nginx config looks like this:
location /socket.io {
# Tornado app
proxy_pass http://localhost:8088;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
...
}
location / {
# Django app
...
}
So, how simple way to routing requests for Django/Tornado on Heroku?
Looks like necessary use custom Buildpack for install Nginx?
Or may be have good way to implement async socket.io in Django to avoid having to route requests?

Let me start with your last question:
Or may be have good way to implement async socket.io in Django to avoid having to route requests?
Django is essentially a library for processing of HTTP requests into appropriate HTTP responses. It does not provide an execution context like uWSGI, Apache mod_wsgi, Tornado, Flask, gunicorn etc. So you can not really use django to serve web sockets; there will always be an execution context around django.
When you deploy a django site to Heroku, it will normally use gunicorn as an execution environment. Performance wise this is not so cool: performance of python servers. Because the poor performance of gunicorn has to do with blocking I/O, some people install nginx as a non-blocking layer in front of gunicorn. This has led to the nginx buildpack.
I don't think this fits your needs. Nginx is an awesome web server, but does not contain a python execution environment. So you end up introducing a third server into your stack:
nginx for HTTP
gunicorn for django
server #3 for the websockets
My best suggestion is to leave gunicorn and nginx and to bring everything together in Tornado: web server, WSGI execution context for django, async context for websockets.
This link shows how to run Tornado on Heroku. And the next link shows how to run django in Tornado.

Related

How to server HTTP/2 Protocol with django

I am planing to deploy my Django application with HTTP/2 protocol but I'm unable to find the proper solution. How can I serve my Django web application with HTTP/2, the only thing that I find is hyper-h2.
I read the documentation but unable to setup the connections.
You can do with Nginx proxy
if you have existing nginx config. you do by just adding a word .http2 in listen
listen 443 ssl http2 default_server;
full document avaliable in
https://www.digitalocean.com/community/tutorials/how-to-set-up-nginx-with-http-2-support-on-ubuntu-16-04
One option is to use Apache httpd server with mod_wsgi. Apache supports terminating HTTP/2. The link to your Django application is still via WSGI API so you don't really get any access to HTTP/2 specific features in your application. You can though configure Apache to do things like server push on your behalf.
https://httpd.apache.org/docs/2.4/howto/http2.html
https://httpd.apache.org/docs/2.4/mod/mod_http2.html
To support HTTP 2.0, You can deploy Django apps on web servers like Daphne using ASGI (which is the spiritual successor to WSGI).
you can read more about deploying Django with ASGI in the official documentaion
to read more about ASGI and what is it, introduction to ASGI
to read more about Daphne server, official repository

Django app running in EC2, but trying to visit elastic URL returns page not found

I'm just starting out with EC2, and I've pulled down a git repo that I started on my local machine and so I know that it works running the server from there, and it seems to works when I run my server from the EC2 instance I have running, but for some reason, when I visit the elastic IP address of that instance I get a page-not-found. Any idea on why that might be?
So, I've now started using nginx, and made a conf file following the instructions here: https://code.djangoproject.com/wiki/DjangoAndNginx that is as follows:
server {
listen 80;
server_name ec2-54-242-149-154.compute-1.amazonaws.com;
access_log /var/log/nginx/USBag.access.log;
error_log /var/log/nginx/USBag.error.log;
location /basicMap/ {
alias /home/www/ec2-54-242-149-154.compute-1.amazonaws.com/basicMap/;
expires 30d;
}
location / {
include fastcgi_params;
fastcgi_pass 127.0.0.1:8080;
}
}
basicMap is a place that I have already defined in my django app, and the linked ec2 ip is the one my server is running on. I am having a lot of difficulty finding documentation on how to proceed or how to determine if my conf file is correct or not. Using the standard python manage.py runserver doesn't work however. Advice on how to proceed?
There is a lot of info about setting up a production django server out there, and I'll give you my personal preferences below, but before all that let's backup and see if we can just get any response from the production server.
To start the development server on your EC2 instance run:
manage.py runserver 0.0.0.0:8000
That command will cause runserver to bind to all interfaces and serve files to the external world. You'll never want to do this outside of development, but it is a good way just to test if your django app is setup before complicating things. Now try hitting your EC2 instance and see if you get a response.
If that's still not working, make sure you allow incoming connections to the server's port (8000 in the command above, 80 once live). You could test that you have ports open using netcat (nc -l).
Once you are satisfied that you have your app setup, I'd recommend you use nginx as your front end webserver and gunicorn as your django webserver in production. You'll likely want to look into setting up a virtualenv, supervisord etc for your production setup (here is a tutorial: http://senko.net/en/django-nginx-gunicorn/), but all that depends on the specifics of your project.

What is the purpose of NGINX and Gunicorn running in parallel?

A lot of Django app deployments over Amazon's EC2 use HTTP servers NGINX and Gunicorn.
I was wondering what they actually do and why both are used in parallel. What is the purpose of running them both in parallel?
They aren't used in parallel. NGINX is a reverse proxy. It's first in line. It accepts incoming connections and decides where they should go next. It also (usually) serves static media such as CSS, JS and images. It can also do other things such as encryption via SSL, caching etc.
Gunicorn is the next layer and is an application server. NGINX sees that the incoming connection is for www.domain.com and knows (via configuration files) that it should pass that connection onto Gunicorn. Gunicorn is a WSGI server which is basically a:
simple and universal interface between web servers and web applications or frameworks
Gunicorn's job is to manage and run the Django instance(s) (similar to using django-admin runserver during development)
The contrast to this setup is to use Apache with the mod_wsgi module. In this situation, the application server is actually a part of Apache, running as a module.
Nginx and Gunicorn are not used in parrallel.
Gunicorn, is a Web Server Gateway Interface (WSGI) server
implementation that is commonly used to run Python web applications.
NGINX is a free, open-source, high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server.
Nginx responsible for serving static content, gzip compression, ssl,
proxy_buffers and other HTTP stuff.While gunicorn is a Python HTTP server that interfaces with both nginx and your actual python web-app code to serve dynamic content.
The following diagrams shows how nginx and Gunicorn interact.

Configuration for Django, Apache and Nginx

I've setup my Django application on Apache+mod_wsgi. To serve the static files I'm using Nginx, as suggested on Django's project website. http://docs.djangoproject.com/en/dev/howto/deployment/modwsgi/
Apache is running on port 8081 and nginx is on port 80. Now some people have suggested that my configuration is wrong and I should reverse the roles of Apache and Nginx. I'm not sure why that should be. And if indeed my configuration is wrong, why would django website suggest the wrong method?
The django docs you linked to do not suggest you use apache as a reverse proxy. They simply suggest you use a separate web server, so I'd say the docs are not clear on that subject -- they are not suggesting anything wrong.
My initial answer was assuming you had nginx as a reverse proxy because port 80 is the HTTP port, the one used when a browser attempts to go to a url with no port specified.
There are numerous complete guides to setting up nginx + apache via a quick google search but here is the gist for setting up nginx:
location / {
# proxy / requests to apache running django on port 8081
proxy_pass http://127.0.0.1:8081/;
proxy_redirect off;
}
location /media/ {
# serve static media directly from nginx
root /srv/anuva_project/www/;
expires 30d;
break;
}
All you need to do is remove the proxy lines from your apache configuration and add the proxy statements to your nginx.conf instead.
If you really want to serve your site from port 8081, you could potentially have nginx listen on port 8081 and have apache listen on a different port.
The point is, apache sits in some obscure port, only serving requests sent to it from nginx, while static file serving is handled by nginx.

Running Multiple Sites with Django's Sites Framework through Gunicorn/Nginx

I have a Django based CMS that uses Django's sites framework and Nginx/Apache/mod_wsgi virtual hosts to run a number of websites on different domains. We're assessing other options for a Django stack and have the CMS running with a single site on a new server with Nginx proxying to Gunicorn (gunicorn_django, specifically).
Although this works great for a single site, I'm not sure how to configure Gunicorn for multiple sites. The problem is that with Apache/mod_wsgi, we could set the DJANGO_SETTINGS_MODULE for mod_wsgi to the appropriate site's settings.py
import os, sys
def inflight(filename):
"""
Calculate absolute path to the folder containing "myfile.wsgi", then
append to the PYTHONPATH.
"""
ROOT = ('/').join(os.path.abspath(os.path.dirname(filename)).split('/')[0:-1])
sys.path.append(ROOT)
sys.path.append(os.path.join(ROOT, 'website'))
sys.stdout = sys.stderr
# Each website should have a settings file: /www/mysite.com/website/settings.py
os.environ['DJANGO_SETTINGS_MODULE'] = 'website.settings'
import django.core.handlers.wsgi
return django.core.handlers.wsgi.WSGIHandler()
At the moment I'm thinking that I have to have a different instance of Gunicorn for each virtual host site we run but that seems overkill for the traffic we get to most of our sites.
Does anyone run Gunicorn with Django's sites framework and can give a hint to how it's configured?
I had the same problem and stumbled upon this question in search of the same answer.
I know the question is old and you've probably figured it out by now, but since it might be useful for someone else, here's how I solved it:
You do need to run separate gunicorn processes to make django sites framework work because you can only point a gunicorn instance at one settings.py file. If you're sites don't get much traffic, I'd only create 1 or 2 gunicorn workers per site. (I know, still probably overkill).
Ideally you would want to manage these different processes with something like supervisord to make it easier to manage starting/stoping/restarting your different sites, but I couldn't get it working.
First, start up your gunicorn servers on local host at different ports using the command line. Ex:
gunicorn_django -w 2 -b 127.0.0.1:1000 /path/to/my/django/project/site1_settings.py --daemon
gunicorn_django -w 2 -b 127.0.0.1:1001 /path/to/my/django/project/site2_settings.py --daemon
You now have 2 django sites running on localhost at ports 1000 and 1001 (you can use whatever ports suite you).
Now you need to create two separate nginx server configuration files to point each domain name at it's respective django site.
Ex:
server {
listen 80;
server_name website1.com;
client_max_body_size 4G;
keepalive_timeout 4;
location /media {
root /path/to/my/django/project/media; #this servers your static files
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header HOST $http_host;
proxy_redirect off;
if (!-f $request_filename){
proxy_pass http://127.0.0.1:1000; #point to the django site already running on local host
break;
}
}
#add in any additional nginx configuration such as support for 500 errors or proxy apache to server php files etc.
}
Then create a duplicate of the nginx configuration for your 2nd site, but change the server name and the proxy_pass to the values for site 2.
Make sure your server configuration files are included in the main nginx.conf file.
Reload nginx and you should be good to go.
If anyone has an easier/better way to go about this please post it.