EC2 Django App Only Accessible without www. prefix - django

if I visit:
myapp.com (it works)
if I visit:
www.myapp.com (throws http 500 error)
or
the fully-qualified version: http://www.myapp.com (throws http 500 error)
that http 500 error is:
xx.xx.xxx.xxx - - [26/Oct/2013:18:33:10 +0000] "GET / HTTP/1.1" 500 460 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:24.0) Gecko/20100101 Firefox/24.0"
I am getting this error, from my access.log used as part of my nginx configuration (note error.log has nothing new in it):
server {
#listen 8001;
listen 80;
#listen 127.0.0.1;
server_name myapp.com www.myapp.com; #*.myapp.com;
#server_name ec2-xx-xxx-xxx-xx.compute-1.amazonaws.com;
access_log /home/ubuntu/virtualenv/myapp/error/access.log;
error_log /home/ubuntu/virtualenv/myapp/error/error.log warn;
connection_pool_size 2048;
fastcgi_buffer_size 4K;
fastcgi_buffers 64 4k;
root /home/ubuntu/virtualenv/myapp/homelaunch/;
location /static/ {
alias /home/ubuntu/virtualenv/myapp/homelaunch/static/;
#alias /static/;
#root /home/ubuntu/virtualenv/myapp/homelaunch/;
}
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
#proxy_set_header X-Forwarded-Host $server_name;
#proxy_set_header X-Real-IP $remote_addr;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
}
}
my ec2 security policy looks like:
What am I doing wrong here?
Thank you!

This didn't appear to be a problem with any settings as much as it was an issue with settings.py, the settings from settings.py weren't being applied because I didn't:
1) Stop Gunicorn and Stop Nginx first
2) Then start them up again using proper commands like:
sudo /usr/local/bin/gunicorn -c /home/ubuntu/virtualenv/gunicorn_config.py myapp.wsgi; sudo nginx -c /etc/nginx/nginx.conf;
Special thanks to user Steve above for the point out.
If anyone runs into a problem like this where your app is accessible via <domainhere>.com, but not www.<domainhere>.com in the future check the following:
1) Make sure your ALLOWED_HOSTS=[] has settings like:
ALLOWED_HOSTS = ['www.myapp.com','myapp.com','<server-ip-here>','ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com']
2) That you are properly starting both gunicorn and nginx using commands like:
sudo /usr/local/bin/gunicorn -c /home/ubuntu/virtualenv/gunicorn_config.py myapp.wsgi;
sudo nginx -c /etc/nginx/nginx.conf;
3) Use a command like this to check that both are properly running:
ps ax|grep nginx; ps ax|grep gunicorn;
Good luck!

Related

deploying channels with nginx

I have deployed django with nginx following the tutorials in digital ocean. Then I blindly followed the section "Example Setup" in the channels document after installation.
My confusions are:
When setting up the configuration file for supervisor, it says to set the directory as
directory=/my/app/path
Should I write down the path where the manage.py is or the path where the settings.py is?
When I reload nginx after changing nginx configuration file, I get an error saying that
host not found in upstream "channels-backend" in
/etc/nginx/sites-enabled/mysite:18 nginx: configuration file
/etc/nginx/nginx.conf test failed
I did replace "mysite" by the name of my website. I had another error earlier saying that
no live upstreams while connecting to upstream
but could not recreate the situation.
I am new to using the channels, so any additional information on upstream would be helpful. Please let me know if I need to provide more information.
Edit:
Here is the nginx.conf file. I changed some sensitive data inside the <>.
upstream channels-backend {
server localhost:8000;
}
server {
listen 80;
server_name <domain name> <ip address>;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root <root to static>;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_pass http://channels-backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
This passes nginx -t. The error message I have in the error.log
connect() failed (111: Connection refused) while connecting to upstream, client: <some ip>, server: <domain name>, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "<domain name>"
The problem actually in supervisor configuration file.
[fcgi-program:asgi]
# TCP socket used by Nginx backend upstream
socket=tcp://localhost:8000
# Directory where your site's project files are located
directory=/my/app/path
# Each process needs to have a separate socket file, so we use process_num
# Make sure to update "mysite.asgi" to match your project name
command=daphne -u /run/daphne/daphne%(process_num)d.sock --fd 0 --access-log - --proxy-headers mysite.asgi:application
# Number of processes to startup, roughly the number of CPUs you have
numprocs=4
# Give each process a unique name so they can be told apart
process_name=asgi%(process_num)d
# Automatically start and recover processes
autostart=true
autorestart=true
# Choose where you want your log to go
stdout_logfile=/your/log/asgi.log
redirect_stderr=true
To check if supervisor was running correctly, I ran
sudo supervisorctl status
This gave me a FATAL status. The problem was that I am currently using a virtual environment, and daphne was only installed inside the virtual environment. Therefore your command should be something like
command= /my/project/virtualenv/path/bin/daphne -u /run/daphne/daphne%(process_num)d.sock --fd 0 --access-log - --proxy-headers mysite.asgi:application

ssl with django gunicorn and nginx

I am currently working on deploying my project over https however I am running into some issues. I have it working with http but when I try to incorporate the ssl it breaks. I think I am misconfiguring the gunicorn upstream client in my nginx block but I am uncertain. Could the issue be in the unix binding in my gunicorn service file? I am very new to gunicorn so I'm a little lost.
Here is my configuration below.
Gunicorn:
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
Environment=PYTHONHASHSEED=random
User=USER
Group=www-data
WorkingDirectory=/path/to/project
ExecStart=/path/to/project/project_env/bin/gunicorn --workers 3 --bind unix:/path/to/project/project.sock project.wsgi:application
[Install]
WantedBy=multi-user.target
Nginx (working-http):
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name server_domain;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /path/to/project;
}
location / {
include proxy_params;
proxy_pass http://unix:/path/to/project/project.sock;
}
}
Nginx (https):
upstream server_prod {
server unix:/path/to/project/project.sock fail_timeout=0;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name server_domain;
}
server {
server_name server_domain;
listen 443;
ssl on;
ssl_certificate /etc/ssl/server_domain.crt;
ssl_certificate_key /etc/ssl/server_domain.key;
location /static/ {
root /path/to/project;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://server_prod;
break;
}
}
}
Your gunicorn systemd unit file seems OK. Your nginx is generally OK too. You have posted too little info to get an appropriate diagnostic. I'm guessing you are missing passing the X-Forwarded-Proto header to gunicorn, but it could be something else. Here's an nginx configuration file that works for me:
upstream gunicorn{
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response (in case the Unicorn master nukes a
# single worker for timing out).
# for UNIX domain socket setups:
server unix:/path/to/project/project.sock fail_timeout=0;
# for TCP setups, point these to your backend servers
# server 127.0.0.1:9000 fail_timeout=0;
}
server {
listen 80;
listen 443 ssl http2;
server_name server_domain;
ssl_certificate /etc/ssl/server_domain.crt;
ssl_certificate_key /etc/ssl/server_domain.key;
# path for static files
root /path/to/collectstatic/dir;
location / {
# checks for static file, if not found proxy to app
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# When Nginx is handling SSL it is helpful to pass the protocol information
# to Gunicorn. Many web frameworks use this information to generate URLs.
# Without this information, the application may mistakenly generate http
# URLs in https responses, leading to mixed content warnings or broken
# applications. In this case, configure Nginx to pass an appropriate header:
proxy_set_header X-Forwarded-Proto $scheme;
# pass the Host: header from the client right along so redirects
# can be set properly within the Rack application
proxy_set_header Host $http_host;
# we don't want nginx trying to do something clever with
# redirects, we set the Host: header above already.
proxy_redirect off;
# Try to serve static files from nginx, no point in making an
# *application* server like Unicorn/Rainbows! serve static files.
proxy_pass http://gunicorn;
}
}

Django, Gunicorn Setup

I am trying to setup a django project on my server and cannot get it to run. I am using virtualenv, gunicorn and nginx for static files. I am not sure were I am going wrong. My current setup is as follows:
myenv
- project(my django project)
- bin(and all it contains)
- lib(and all it contains)
- include(and all it contains)
- gunicorn_config.py
gunicorn_config.py:
command = '/home/me/django/myenv/bin/gunicorn'
pythonpath = '/home/me/django/myenv/project'
bind = '127.0.0.1:80'
workers = 2
nginx project.conf:
upstream project_server {
server unix:/tmp/gunicorn_project.sock fail_timeout=0;
}
server {
listen 80;
client_max_body_size 4G;
# set the correct host(s) for your site
server_name project.com www.project.com;
keepalive_timeout 5;
# path for static files
root /home/me/django/myenv/assets;
location / {
# checks for static file, if not found proxy to app
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://project_server;
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/me/django/myenv/project/project/templates;
}
}
And I run the following to try start it all up:
/home/me/django/myenv/bin/gunicorn -c /home/me/django/myenv/gunicorn_config.py project/project/wsgi.py
But it just says "Can't connect to ('127.0.0.1', 80)"
You've configured gunicorn to bind on a TCP port, but gunicorn is binding on a unix socket. You should use the same thing; preferably the socket, so it doesn't conflict with the port nginx is actually listening on.
In gunicorn_config.py:
bind = 'unix:/tmp/gunicorn_project.sock'
Basically, I would guess nginx spins up before gunicorn. It takes port 80 (from your listen). gunicorn comes next, also wants port 80 (from your bind), and finds it occupied so it errors out. Run gunicorn on a different port and use proxy_pass to tell nginx about it.
Gunicorn
bind = '127.0.0.1:8000'
Nginx
proxy_pass http://127.0.0.1:8000/;

Django Rest Framework app, served by Gunicorn, proxied by Nginx (SSL)

Goal: I want to serve django-rest-framework(drf) responses over https.
I'm trying to understand why my setup isn't working, whether it can be fixed by a simple settings change in one of the components, or if I should try a different approach.
I have a drf application served by gunicorn. Gunicorn is behind an NGINX proxy. I also have some static content served by NGINX. This setup has worked fine for plain old http.
Then I set NGINX to listen over ssl (,and installed certs, etc.). The static content works over https. I can still get to drf via http, but I get nothing / timed out over https. The gunicorn logs haven't been that helpful to me, but django dev server gives "dev server doesn't work over https" when sent a request.
Here is my first attempt at the nginx config (anonymized). This was mostly from reading the gunicorn and nginx manuals.
server {
server_name example.com;
listen 443 ssl;
ssl_certificate /path/to/cert;
ssl_certificate_key /path/to/key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers ...
location /static/{
alias /path/to/static/content/;
}
location / {
return 301 $scheme://example.com/static;
}
location /drf/{
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_set_header X-Real_IP $remote_addr;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://127.0.0.1:6565; # bound to gunicorn, can be reached from http://example.com:6565 - ideally want to be available from https://example.com:6565
}
}
server {
listen 80;
server_name example.com;
return 301 https://example.com
}
I don't think there is an additional step for gunicorn. On the DRF side, I read the relevant Django security section and tried using the SECURE_SSL_REDIRECT = True (new in 1.8 - not addressed in a lot of existing questions) and SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https').
I also tried with a redirect, but this was sort of a shot in the dark:
...after `proxy_pass` above, and commenting out proxy_redirect off:
proxy_redirect http://127.0.0.1:6565 https://example.com/drf
Ubuntu 14, Gunicorn 19, Nginx 1.1,
Django 1.8 (DRF 3.2)
Update:
NGINX Error logging:
I get nothing if I go to https://example.com:6565. Here is an error from nginx's error log for a request to https://example.com/drf/endpoint.
2015/09/08 13:53:52 [error] 12564#0: *14 connect() failed (111: Connection refused while connecting to upstream, client: 155.xxx.xxx.xx, server:example.com, request:"GET /drf/endpoint", upstream: "http://127.0.0.1:6565/drf/endpoint", host: "example.com"

How to call a flask application from a web browser?

I have a flask application that works on my PC.
When I deployed it on the virtual server (CentOs 6.5) I used nginx according to the article:
https://www.digitalocean.com/community/tutorials/how-to-deploy-flask-web-applications-using-uwsgi-behind-nginx-on-centos-6-4
I had to change the port in /etc/nginx/nginx.conf because it created conflict with apache port (caused nginx to fail to start because port is already in use). So my nginx.conf is:
worker_processes 1;
events {
worker_connections 1024;
}
http {
sendfile on;
gzip on;
gzip_http_version 1.0;
gzip_proxied any;
gzip_min_length 500;
gzip_disable "MSIE [1-6]\.";
gzip_types text/plain text/xml text/css
text/comma-separated-values
text/javascript
application/x-javascript
application/atom+xml;
# Configuration containing list of application servers
upstream uwsgicluster {
server 127.0.0.1:8081;
# server 127.0.0.1:8081;
# ..
# .
}
# Configuration for Nginx
server {
# Running port
listen 81;
# Settings to by-pass for static files
location ^~ /static/ {
# Example:
# root /full/path/to/application/static/file/dir;
root /app/static/;
}
# Serve a static file (ex. favico) outside static dir.
location = /favico.ico {
root /app/favico.ico;
}
# Proxying connections to application servers
location / {
include uwsgi_params;
uwsgi_pass uwsgicluster;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
}
}
My application is located under 12.12.15.16/cgi-bin/My_app (the ip address here is dummy).
I used the following command line to start the app:
env/bin/uwsgi --socket 127.0.0.1:8081 --protocol=http --wsgi-file main.py --callable app
My question is: How can I now call my app from a web browser?
Thank you for your help!
Your Flask app is listening on port 8081 but only for localhost connections (127.0.0.1). Nginx is listening on port 81 for connections and piping them to Flask on 8081. So from your own browser you want to access http://your-digital-ocean-ip:81/