Internal Server Error 500 -- EC2, django, nginx, uwsgi - django

I'm developing and deploying django based web-app through EC2 with nginx and uwsgi.
Because of developing Front-side (based on react), I send several http request to my server's public IPv4.
And the first few moment, my request is normally answered by server. All data is safely delivered to "http://localhost:3000".
But in some reason, after rebooting my EC2 instance, my nginx returned "500 (Internal Server Error)". More detailed information is descripted below.
[log]
/var/nginx/access.log --> any further information than internal server error
3.21.102.102 - - [22/Dec/2022:04:07:08 +0000] "GET /beta/.env HTTP/1.1" 500 32 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246" "-"
3.21.102.102 - - [22/Dec/2022:04:07:08 +0000] "GET /beta/.env.local HTTP/1.1" 500 32 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246" "-"
3.21.102.102 - - [22/Dec/2022:04:07:09 +0000] "GET /beta/.env.production HTTP/1.1" 500 32 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246" "-"
3.21.102.102 - - [22/Dec/2022:04:07:09 +0000] "GET /beta/.env.staging HTTP/1.1" 500 32 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246" "-"
3.21.102.102 - - [22/Dec/2022:04:07:09 +0000] "GET /live/.env HTTP/1.1" 500 32 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246" "-"
/var/nginx/access.log --> empty
[configure files and django settings.py]
nginx.configure
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 4096;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
listen [::]:80;
server_name {{server_address_of_mine}};
root /home/ec2-user/cury_engine;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
include uwsgi_params;
uwsgi_pass unix:/home/ec2-user/cury.sock;
}
}
django settings.py
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = False
WSGI_APPLICATION = 'cury_engine.wsgi.application'
CORS_ORIGIN_WHITELIST = [
'http://localhost:3000',
"http://localhost:8000",
'{{my_domain}}',
"http://127.0.0.1:8000",
]
[What I tried after reboot server]
sudo systemctl restart nginx
sudo systemctl restart nginx.service
sudo systemctl restart uwsgi (my django server is automatically run server with this command)
sudo systemctl restart uwsgi.service
-> I check nginx and uwsgi is running safely. (with sudo systemctl status XXX command)
[What I tried differently to test server status]
postman request --> same result: 500 (Internal Server Error)
curl request inside EC2 instance terminal --> same result: 500 (Internal Server Error)
i expect to recover my server and know how to debug this error

Related

How to prevent these malicious requests with nginx config?

My nginx server (using django) is getting hit with thousands of these types of requests per second:
199.127.61.178 - - [09/Nov/2022:08:20:42 +0000] "GET http://www.wuqiaoxianzajituan.com/ HTTP/1.1" 500 186 "http://www.wuqiaoxianzajituan.com" "Mozilla/5.0 (Linux; U; Android 2.3.5; en-in; Micromax A87 Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1"
104.238.222.87 - - [09/Nov/2022:08:20:42 +0000] "GET http://www.wuqiaoxianzajituan.com/ HTTP/1.1" 400 55440 "http://www.wuqiaoxianzajituan.com" "Mozilla/5.0 (Linux; Android 8.0.0; SM-G950F Build/R16NW) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Mobile Safari/537.36"
172.93.110.55 - - [09/Nov/2022:08:20:42 +0000] "GET http://tucgd.lixil-kitchen.cn/ HTTP/1.1" 400 55373 "http://tucgd.lixil-kitchen.cn" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/601.4.4 (KHTML, like Gecko) Version/9.0.3 Safari/537.86.4"
104.243.37.94 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55205 "http://you.br-sx.com" "Mozilla/5.0 (iPhone; CPU iPhone OS 11_2 like Mac OS X) AppleWebKit/604.4.7 (KHTML, like Gecko) Version/11.0 Mobile/15C114 Safari/604.1"
104.238.222.9 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?nf91C2a99VqP4D43fy6uPrgt0 HTTP/1.1" 400 55722 "https://skype.gmw.cn" "Mozilla/5.0 (iPad; U; CPU OS 4_3_5 like Mac OS X; de-de) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5"
104.238.205.70 - - [09/Nov/2022:08:20:42 +0000] "GET http://eqksp.drtjy.com/ HTTP/1.1" 400 55224 "http://eqksp.drtjy.com" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.87 Safari/537.36"
103.195.103.32 - - [09/Nov/2022:08:20:42 +0000] "GET http://eqksp.drtjy.com/ HTTP/1.1" 400 55192 "http://eqksp.drtjy.com" "Mozilla/5.0 (Windows NT 5.1; rv:40.0) Gecko/20100101 Firefox/40.0"
104.243.37.94 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55133 "http://you.br-sx.com" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20100101 Firefox/17.0"
173.208.239.195 - - [09/Nov/2022:08:20:42 +0000] "GET http://eqksp.drtjy.com/ HTTP/1.1" 500 588 "http://eqksp.drtjy.com" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.63 Safari/537.36"
104.238.205.70 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55147 "http://you.br-sx.com" "Mozilla/5.0 (Windows NT 5.1; rv:33.0) Gecko/20100101 Firefox/33.0"
104.238.205.70 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?7n62R0Ocp5h8ymbM74co76w370m0Cv HTTP/1.1" 400 55741 "https://skype.gmw.cn" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36"
104.238.222.9 - - [09/Nov/2022:08:20:42 +0000] "GET http://tucgd.lixil-kitchen.cn/ HTTP/1.1" 500 588 "http://tucgd.lixil-kitchen.cn" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.67 Safari/537.36"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET http://www.wuqiaoxianzajituan.com/ HTTP/1.1" 400 55372 "http://www.wuqiaoxianzajituan.com" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0"
103.195.103.32 - - [09/Nov/2022:08:20:42 +0000] "GET http://tucgd.lixil-kitchen.cn/ HTTP/1.1" 400 55366 "http://tucgd.lixil-kitchen.cn" "Mozilla/5.0 (iPad; CPU OS 10_3_3 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) CriOS/61.0.3163.73 Mobile/14G60 Safari/602.1"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?DnV7mPJ19L1Li6bwt39aVP59oDDi6bJxPb8Pj0 HTTP/1.1" 400 55775 "https://skype.gmw.cn" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:59.0) Gecko/20100101 Firefox/59.0"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?3as00v0ydeeRx5sXVa3wMoQ6 HTTP/1.1" 400 55676 "https://skype.gmw.cn" "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36"
104.194.8.226 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55219 "http://you.br-sx.com" "Mozilla/5.0 (Linux; Android 6.0.1; SM-G532M Build/MMB29T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.91 Mobile Safari/537.36"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET http://www.wuqiaoxianzajituan.com/ HTTP/1.1" 400 55434 "http://www.wuqiaoxianzajituan.com" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.106 Safari/537.36"
104.243.37.94 - - [09/Nov/2022:08:20:42 +0000] "GET https://skype.gmw.cn/?I5RaB0sIBAt7W9i7iWueXU9104kJ HTTP/1.1" 400 55714 "https://skype.gmw.cn" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
104.194.8.226 - - [09/Nov/2022:08:20:42 +0000] "GET http://you.br-sx.com/ HTTP/1.1" 400 55238 "http://you.br-sx.com" "Mozilla/5.0 (iPhone; CPU iPhone OS 10_0_2 like Mac OS X) AppleWebKit/602.1.50 (KHTML, like Gecko) Version/10.0 Mobile/14A456 Safari/602.1"
104.238.222.88 - - [09/Nov/2022:08:20:42 +0000] "GET http://px9i1.jntmzg.cn/ HTTP/1.1" 400 55245 "http://px9i1.jntmzg.cn" "Mozilla/5.0 (Linux; Android 8.0.0; FLA-LX3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.80 Mobile Safari/537.36"
...
199.127.60.99 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?s8023k1FRBDIvK6Rxw6q8h0e5S HTTP/1.1" 502 166 "https://d518b.com" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:55.0) Gecko/20100101 Firefox/55.0/Nutch-1.12"
185.150.189.223 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?xCtRIk9u13N80G1J8xaWTF1GSLo80M6 HTTP/1.1" 502 568 "https://d518b.com" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36"
104.238.221.227 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?25HwH49w9C9UqapejfQ3HQCX02EKbegWgvG4 HTTP/1.1" 502 166 "https://d518b.com" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/600.3.18 (KHTML, like Gecko) Version/8.0.3 Safari/600.3.18"
104.243.34.218 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?v8iHIV5uWx4540GvN4apQ3dG3 HTTP/1.1" 502 166 "https://d518b.com" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:26.0) Gecko/20100101 Firefox/26.0"
103.195.103.32 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?4FrrkvoMux8HR162L324b2 HTTP/1.1" 502 568 "https://d518b.com" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; InfoPath.1)"
199.127.63.156 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?A5cAxLa81Q52KCL752QK010X3NuQ HTTP/1.1" 502 568 "https://d518b.com" "Mozilla/5.0 (Linux; NetCast; U) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/38.0.2125.122 Safari/537.31 SmartTV/7.5"
103.195.103.32 - - [09/Nov/2022:13:19:50 +0000] "GET https://d518b.com/?G2nga9Dw9q1Xy9bR7qBXB HTTP/1.1" 502 568 "https://d518b.com" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322)"
They all seem to requests external URLs, (which I don't think my server needs to do for any legitimate reason), so I tried to deny them with my nginx config by denying matches to http, .com, cn, with this config:
upstream app_server {
server unix:/home/django/gunicorn.socket fail_timeout=0;
}
limit_req_zone $binary_remote_addr zone=one:10m rate=10r/s;
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
root /usr/share/nginx/html;
index index.html index.htm;
client_max_body_size 4G;
server_name _;
keepalive_timeout 5;
# Your Django project's media files - amend as required
location /media {
alias /home/django/django_project/django_project/media;
}
# your Django project's static files - amend as required
location /static {
alias /home/django/django_project/django_project/static;
}
# Proxy the static assests for the Django Admin panel
location /static/admin {
alias /usr/lib/python3/dist-packages/django/contrib/admin/static/admin/;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://app_server;
limit_req zone=one;
limit_except GET HEAD POST {
deny all;
}
}
# ADDED THESE NEW LINES
location ~ http {
deny all;
}
location ~ .com {
deny all;
}
location ~ .cn {
deny all;
}
}
server {
root /usr/share/nginx/html;
index index.html index.htm;
client_max_body_size 4G;
server_name <MYURLHIDDEN>; # managed by Certbot
keepalive_timeout 5;
# Your Django project's media files - amend as required
location /media {
alias /home/django/django_project/django_project/media;
}
# your Django project's static files - amend as required
location /static {
alias /home/django/django_project/django_project/static;
}
# Proxy the static assests for the Django Admin panel
location /static/admin {
alias /usr/lib/python3/dist-packages/django/contrib/admin/static/admin/;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://app_server;
limit_req zone=one;
limit_except GET HEAD POST {
deny all;
}
}
# ADDED THESE NEW LINES
location ~ http {
deny all;
}
location ~ .com {
deny all;
}
location ~ .cn {
deny all;
}
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/<MYURLHIDDEN>/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/<MYURLHIDDEN>/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = <MYURLHIDDEN>) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80 ;
listen [::]:80 ;
server_name <MYURLHIDDEN>;
return 404; # managed by Certbot
}
But these types of requests are still hitting my access log, and my servers CPU keeps hitting 100% and site keeps going down. What am I doing wrong?

What is the best practice using KQL to filter desired attack signature over (web)logs?

Recently I'm experimenting with logstach and Kibana on top of elastic over (web-)server logs. I tried to extract some attack signature like XSS & SQL injection like the following examples when logs contain < $ ' " ! .\ %22, and so on:
<script>foo</script>
<script>document.cookie=%22testkzcp=XXX;%22</script>
<meta%20http-equiv=Set-Cookie%20content=%22testvpmi=XXXX%22>
${XXXXXXXXXX+5}.action
'.print(md5(XXXXX)).'
${#print(md5(XXXXX))}\
";print(md5(XXXXX));$a="
!(()&&!|*|*|
.\.\.\.\.\.\.\.\.\.\/windows/win.ini
The following is the common error I get when using"((", ".\", "OR" or "$" and so on using KQL:
KQLSyntaxError: Expected ":", "<", "<=", ">", ">=", AND, OR, end of input, whitespace but ")" found.
I checked The Kibana Query Language (KQL) and tried to use * as wildcard_queries beside of interesting term "</script>" or "%22</script>" through my desired timestamp but it was unsuccessful. I also checked Escaping special characters in elasticsearch.
So The question is, What is the best practice for using KQL to filter/search desired string-based attack signature over logs. Please give an example for the above-mentioned attack signatures.
Edit1: I found the post that says it's possible to solve this problem using Regex in KQL as well as some workaround here & here, So I'm also interested in finding Regex-based solution to find the afore-mentioned pattern in KQL.
Example web requests within the above-mentioned patterns:
[21/Jan/2021:02:02:23 +0000] XX.XXX.XXX.X "-" "GET / HTTP/1.1" 403 "-b" 0b 1ms "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4512.0 Safari/537.36" XXX.XXX.XX.XX 42109 "'>"></title></style></textarea></noscript></template></script><script/src="//bxss.me/s?u=074623&r=74172-18&h=74172-7bf88-2&"></script>" "'>"></title></style></textarea></noscript></template></script><script/src="//bxss.me/s?u=074623&r=74172-18&h=74172-7bf88-2&"></script>" - - TLSv1.2 -,-,-
[19/Jan/2021:23:02:37 +0000] XXX.XXX.XXX.XX "-" "GET / HTTP/1.1" 403 "-b" 0b 1ms "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4512.0 Safari/537.36" XXX.XXX.XX.XX 42109 "-1" OR 2+190-190-1=0+0+0+1 --" "-1" OR 2+190-190-1=0+0+0+1 --" - - TLSv1.2 -,-,-
[10/Jan/2021:01:11:02 +0000] XXX.XXX.XX.XX "-" "GET / HTTP/1.1" 403 "-b" 0b 1ms "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4512.0 Safari/537.36" XXX.XXX.XX.XX 42133 "${#print(md5(31337))}" "${#print(md5(31337))}" - - TLSv1.2 -,-,-
[18/Jan/2022:09:13:00 +0000] XXX.XXX.XX.XX "-" "GET / HTTP/1.1" 403 "-b" 0b 1ms "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4512.0 Safari/537.36" XXX.XXX.XX.XX 42133 ")))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))" ")))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))))" - - TLSv1.2 -,-,-

Static files not being utilized by Django on docker with nginx

Background
I am working on a docker-compose app made of 4 services: a django app, a wagtail django app, nginx, and postgresql. My main issue is with static files: they work fine with the development server, but not with nginx. The really strange part is that nginx shows that it is serving the static files, and they are accessible through their URL on a browser. How can I get them to show up?
From settings.py in wagtail app
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
]
STATICFILES_DIRS = [
os.path.join(PROJECT_DIR, 'static'),
''
]
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
STATIC_URL = '/static/'
From settings.py in django app
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
docker-compose.yml
version: '3.7'
services:
nginx:
image: nginx:latest
container_name: production_nginx
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
- ./nginx/error.log:/etc/nginx/error_log.log
#- /etc/letsencrypt/:/etc/letsencrypt/
- cms_static_volume:/usr/src/cms/static
- core_static_volume:/usr/src/core/static
ports:
- 80:80
- 443:443
depends_on:
- core
- cms
core:
build:
context: ./cirrus_core
dockerfile: Dockerfile.prod
command: gunicorn cirrus_core.wsgi:application --bind 0.0.0.0:8000
volumes:
- core_static_volume:/usr/src/core/static
expose:
- "8000"
env_file: .env
depends_on:
- db
cms:
build:
context: ./cirrus_cms
dockerfile: Dockerfile.prod
command: gunicorn cirrus_cms.wsgi:application --bind 0.0.0.0:8001
volumes:
- cms_static_volume:/usr/src/cms/static
expose:
- '8001'
env_file: .env
depends_on:
- db
db:
image: postgres:11.5-alpine
volumes:
- postgres_data:/var/lib/postgresql/data/
env_file: .env.db
volumes:
postgres_data:
core_static_volume:
cms_static_volume:
nginx.conf
events{}
http {
upstream cms {
server cms:8001;
}
upstream core {
server core:8000;
}
server {
listen 80;
server_name dashboard.sentimentco.com 192.168.86.1;
location / {
proxy_pass http://cms/admin/login/?next=/admin/;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /usr/src/cms/static/;
}
}
server {
listen 80;
server_name admin.sentimentco.com 192.168.86.1;
location / {
proxy_pass http://core/admin/login/?next=/admin/;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
}
location /static/ {
alias /usr/src/core/static/;
}
}
}
nginx logs
77.247.108.110 - - [19/Aug/2019:20:56:51 +0000] "GET / HTTP/1.1" 400 37 "-" "-"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET / HTTP/1.1" 200 6387 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor/modernizr-2.6.2.min.76e933e876d5.js HTTP/1.1" 200 10583 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/css/vendor/jquery-ui/jquery-ui-1.10.3.verdant.767104e7b415.css HTTP/1.1" 200 27112 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/css/core.a28f9631bc68.css HTTP/1.1" 200 78445 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/css/normalize.aa75c284e281.css HTTP/1.1" 200 2359 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/css/vendor/jquery.tagit.1060b2909c82.css HTTP/1.1" 200 832 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/css/layouts/login.ea7b231e7652.css HTTP/1.1" 200 1842 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor/jquery.autosize.a300cc7b9e37.js HTTP/1.1" 200 5571 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor/jquery-3.2.1.min.c9f5aeeca3ad.js HTTP/1.1" 200 86659 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor/bootstrap-transition.7b8c2e460984.js HTTP/1.1" 200 2087 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor/bootstrap-modal.b411788be2b6.js HTTP/1.1" 200 7031 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor/jquery.datetimepicker.280df8b3286d.js HTTP/1.1" 200 108981 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor/tag-it.a60812d95e27.js HTTP/1.1" 200 21758 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor/bootstrap-tab.f9e4e160df7e.js HTTP/1.1" 200 3563 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/core.7690c99ae674.js HTTP/1.1" 200 17602 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor/jquery-ui-1.12.1.min.c15b1008dec3.js HTTP/1.1" 200 253669 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/wagtailadmin.8edb9675af94.js HTTP/1.1" 200 58279 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
192.168.86.1 - - [19/Aug/2019:20:56:59 +0000] "GET /static/wagtailadmin/js/vendor.a08a4452c3bc.js HTTP/1.1" 200 194518 "http://dashboard.sentimentco.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36
result
django admin screenshot
Can anyone tell what's going on here?
It turns out I just needed to include mime.types in my nginx.conf. Big thanks to gasman for helping me figure out the problem.

Deploying docker on AWS Beanstalk only serves html files, no other files

I have tried to get this project deployed to AWS Elastic Beanstalk: https://github.com/coralproject/talk The dockerfile exposes port 5000 and I have defined environment variables also all using port 5000.
When I run the project locally with the recommended docker-compose file (https://coralproject.github.io/talk/installation-from-docker/#installing) everything works fine locally.
But when I deploy the app to Beanstalk, the html page is served and loads fine, however other files that are referenced locally such as my bundle.js and favicon files return a 502.
What am I missing?
Logs that may be relevant:
/var/log/eb-activity.log
cat: /var/app/current/Dockerrun.aws.json: No such file or directory
8c17e6ddb0f842e592940a3aa67d0f39ec8702eb4ad6c3f9b876fc33b7f02ddc
[2018-02-11T08:29:26.836Z] INFO [24507] - [Application update
app-5d978-180211_092600#12/AppDeployStage1/AppDeployEnactHook/01flip.sh]
: Starting activity... [2018-02-11T08:29:28.428Z] INFO [24507] -
[Application update
app-5d978-180211_092600#12/AppDeployStage1/AppDeployEnactHook/01flip.sh]
: Completed activity. Result: nginx: [warn] duplicate MIME type
"text/html" in
/etc/nginx/sites-enabled/elasticbeanstalk-nginx-docker-proxy.conf:11
Stopping nginx: [ OK ] Starting nginx: nginx: [warn] duplicate
MIME type "text/html" in
/etc/nginx/sites-enabled/elasticbeanstalk-nginx-docker-proxy.conf:11
[ OK ] cat: /var/app/current/Dockerrun.aws.json: No such file or
directory /opt/elasticbeanstalk/hooks/common.sh: line 95: [: 1:
unary operator expected iptables: Saving firewall rules to
/etc/sysconfig/iptables: [ OK ]
The nginx access log only shows the html requests not the other files
/var/log/nginx/access.log
95.90.245.122 - - [11/Feb/2018:22:43:00 +0000] "GET / HTTP/1.1" 302 72 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
95.90.245.122 - - [11/Feb/2018:22:43:00 +0000] "GET /admin/install HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132
Safari/537.36"
95.90.245.122 - - [11/Feb/2018:22:45:57 +0000] "GET /admin/install HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132
Safari/537.36"
95.90.245.122 - - [11/Feb/2018:22:46:04 +0000] "GET / HTTP/1.1" 302 72 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36"
95.90.245.122 - - [11/Feb/2018:22:46:04 +0000] "GET /admin/install HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132
Safari/537.36"
I see this when I access the page:
install:45 GET
https://talk-now.us-east-1.elasticbeanstalk.com:5000/static/coral-admin/bundle.js
net::ERR_CONNECTION_REFUSED :5000/public/img/favicon-32x32.png:1 GET
https://talk-now.us-east-1.elasticbeanstalk.com:5000/public/img/favicon-32x32.png
net::ERR_CONNECTION_REFUSED :5000/public/img/favicon-16x16.png:1 GET
https://talk-now.us-east-1.elasticbeanstalk.com:5000/public/img/favicon-16x16.png
net::ERR_CONNECTION_REFUSED :5000/public/img/favicon-96x96.png:1 GET
https://talk-now.us-east-1.elasticbeanstalk.com:5000/public/img/favicon-96x96.png
net::ERR_CONNECTION_REFUSED
The problem is that you are missing a Dockerrun.aws.json file at the root level of your repository. This file is necessary for Beanstalk to determine how to execute the set of containers in your project.
Also note that the format of the sections in this file is similar to that of Amazon ECS Task definitions

Bad request 400: nginx / gunicorn

I have followed this tutorial: http://blog.wercker.com/2013/11/25/django-16-part3.html and I am just trying to make it work locally with Vagrant for now. I am not trying to use Wercker.
After everything is installed, I try to access the website but I get a Bad Request (400) error every time. I do not know if that is due to a problem in nginx or in gunicorn.
They both have a log entry so at least I know that the request goes all the way through gunicorn and is not stopped at the nginx level.
Where is the problem located? Gunicorn? nginx?
Here are the logs of gunicorn and nginx.
I see that the favicon is missing but that only should not stop the page from being displayed right?
Gunicorn:
>>> cat /var/local/sites/hellocities/run/gunicorn.error.log
10.0.0.1 - - [28/Jan/2014:07:05:16] "GET / HTTP/1.0" 400 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
10.0.0.1 - - [28/Jan/2014:07:09:43] "GET / HTTP/1.0" 400 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
Nginx:
>>> cat /var/log/nginx/hellocities-access.log
10.0.0.1 - - [28/Jan/2014:07:05:16 +0000] "GET / HTTP/1.1" 400 37 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
10.0.0.1 - - [28/Jan/2014:07:05:20 +0000] "GET /favicon.ico HTTP/1.1" 404 200 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
10.0.0.1 - - [28/Jan/2014:07:09:43 +0000] "GET / HTTP/1.1" 400 37 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
10.0.0.1 - - [28/Jan/2014:07:09:44 +0000] "GET /favicon.ico HTTP/1.1" 404 200 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.76 Safari/537.36"
>>> cat /var/log/nginx/hellocities-error.log
2014/01/28 07:05:20 [error] 13886#0: *1 open() "/var/local/sites/hellocities/static/favicon.ico" failed (2: No such file or directory), client: 10.0.0.1, server: _, request: "GET /favicon.ico HTTP/1.1", host: "10.0.0.200"
2014/01/28 07:09:44 [error] 13886#0: *3 open() "/var/local/sites/hellocities/static/favicon.ico" failed (2: No such file or directory), client: 10.0.0.1, server: _, request: "GET /favicon.ico HTTP/1.1", host: "10.0.0.200"
I had the same problem and adding ALLOWED_HOSTS = ("yourdomain.com",) to settings fixed it.
UPDATE: there few other possibilities:
Nginx (or whatever web server you use) doesn't pass the $host variable to the app
Host contains underscores
See details: https://blog.anvileight.com/posts/how-to-fix-bad-request-400-in-django/
As I was having the same issue (400 error code when trying to share with vagrant share), I stumble upon this question. The answer and comments are right, as the obvious solution is to set ALLOWED_HOSTS list, but I was already setting it correctly (I thought).
I can't speak for nginx as I'm running this on apache2, but here's what solved the issue:
Take a look at the ALLOWED_HOSTS doc to find what's best for your case.
With vagrant, you might find it useful to accept all the vagrantshare.com subdomain, so just add '.vagrantshare.com' (notice the dot) to the ALLOWED_HOSTS list.
Not sure if it is really necessary, but I changed the modified date of the wsgi.py file
touch wsgi.py
As I'm using apache2, I needed to restart the service.
sudo service apache2 restart
And then it worked.
I ran into this issue. It was because I forgot to add the proxy_set_header settings in the nginx config:
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
So Django didn't see the original hostname that was requested, so it didn't match with what was in ALLOWED_HOSTS. Then it gave back the 400 response.
After adding this to my nginx config (at the spot where you do the proxy_pass to Gunicorn) and then restarting nginx, it worked.
More info: https://docs.gunicorn.org/en/stable/deploy.html#nginx-configuration