I am developing a web application using AWS, and Django Rest Framework.(Django:v1.8, DRF:v3)
I have kept getting django.request: Forbidden (Referer checking failed - no Referer.) for POST multipart form request.
I am using AWS ELB(Elastic load balancer), NGINX on my ec2(in autoscailing group) and Gunicorn.
AWS ELB listener setting is like below(HTTPS only):
elb https only listener setting
NGINX setting is like below:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/conf.d/*.conf;
index index.html index.htm;
upstream my_server {
server localhost:8000;
}
server {
listen 80;
server_name <server name>;
access_log /etc/nginx/log/local-wc.access.log;
error_log /etc/nginx/log/local-wc.error.log;
root /usr/share/nginx/html;
location /api/v1 {
proxy_pass http://my_server/api/v1;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Protocol $scheme;
}
}
}
<server name> is the CNAME which point to elb DNS name.
In other words, <server name> => xxxx-123456789.us-west-2.elb.amazonaws.com (A Record).
Every API call is made by https://<server name>/api/v1/*
Finally Gunicorn is running by:
gunicorn my_django_app.wsgi:application -w 1 -b 127.0.0.1:8000 -t 300 --max-requests=100
and Django Setting is:
ALLOWED_HOSTS = ['*']
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
MIDDLEWARE_CLASSES = (
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.security.SecurityMiddleware',
)
View function is like below(with CSRF exempt):
class UserViewSet(CsrfExemptMixin, mixins.CreateModelMixin,
mixins.ListModelMixin,
mixins.RetrieveModelMixin,
mixins.UpdateModelMixin,
viewsets.GenericViewSet):
# already tried #csrf_exempt
def create(self, request, *args, **kwargs):
self.parser_classes = (FormParser, MultiPartParser, )
.........
Problem again:
When I send
curl -i -k -X POST -H "Accept: application/json" \
-F "email=myemail#email.com" \
-F "profile_img=#profile.jpg" \
https://<server name>/api/v1/users/
and in my Django log:
[WARNING] django.request: Forbidden (Referer checking failed - no Referer.): /api/v1/users/
It worked with POST on HTTP or GET method on HTTPS.
I wonder whether ELB configuration is wrong or Nginx Configuration is wrong with referer...
I would appreciate if some one help me to solve this problem..
I think DRF ignores csrf_exempt decorator and I am not sure where the CsrfExemptMixin is defined. The easiest thing you can do is add Referrer: yourhost to your curl headers.
Related
My app is an AWS EC2 instance, I have some problems with a CORS error.
It works great on a local server but not on a production server. My app is an angular 11 app in the frontend. Its a Spring Boot API. It is deployed in aws elastic beanstalk, I understand that aws uses nginx and this is what is generating the cors error for me
Then I tried a solution I found around: file path .ebextensions\nginx file nginx.conf
user nginx;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
worker_processes auto;
worker_rlimit_nofile 33282;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
include conf.d/*.conf;
map $http_upgrade $connection_upgrade {
default "upgrade";
}
server {
listen 80;
server_name dominio.com;
root /var/app/current/public;
location / {
# Simple requests
if ($request_method ~* "(GET|POST)") {
add_header "Access-Control-Allow-Origin" *;
}
# Preflighted requests
if ($request_method = OPTIONS ) {
add_header "Access-Control-Allow-Origin" *;
add_header "Access-Control-Allow-Methods" "GET, POST, OPTIONS, HEAD";
add_header "Access-Control-Allow-Headers" "Authorization, Origin, X-Requested-With, Content-Type, Accept";
return 200;
}
}
location /api {
proxy_pass http://127.0.0.1:5000;
proxy_http_version 1.1;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
access_log /var/log/nginx/access.log main;
client_header_timeout 60;
client_max_body_size 10M;
client_body_timeout 60;
keepalive_timeout 60;
gzip off;
gzip_comp_level 4;
# Include the Elastic Beanstalk generated locations
include conf.d/elasticbeanstalk/01_static.conf;
include conf.d/elasticbeanstalk/healthd.conf;
}
}
Nginx has a configuration file by default located in /etc/nginx/nginx.conf. This file has a lot of default properties for all requests and one of these is client_max_body_size. The default value for that property is 1MB.
in the file I am updating the client_max_body_size to 10MB, but the cors error when I send a file larger than 1 MB still persists:
Access to XMLHttpRequest at 'https://dominio.com/api' from origin 'https://www.dominio.frontend.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
CORS configuration in spring:
enter image description here
The application works fine in production except when I use a field to add files larger than 1MB, at which point it generates the CORS error.
None of this has solved the problem. What am I missing here?
Thanks so much for any help!
you should configure CORS in your Spring application.
Create a #Configuration class implementing WebMvConfigurer interface
Override method addCorsMappings(), like sample below, which allow all methods, all origins and apply to all API, aka: Global CORS configuration.
#Configuration
public class CorsConfig implements WebMvcConfigurer {
#Override
public void addCorsMappings(CorsRegistry registry) {
registry.addMapping("/**").allowedMethods("*");
}
}
See the guide below if you want to apply selectively to specific APIs
Ref:
https://spring.io/guides/gs/rest-service-cors/
I'm building an API and I finally got it working to serve it through Gunicorn and NGINX. Nginx proxies incoming requests to a socket binded to Gunicorn. The problem is this:
When I try to access the API directly by running the 'gunicorn command or by using the builtin 'runserver' command from Django, and having configured Django REST Framework's BasicAuthentication as the default authentication class in the settings.py-file, everything works fine. Each time I try to access an endpoint, it asks me for a valid username/password-combo, just like you would expect.
However, when I try to access the API through NGINX, which has a proxy_pass configured to the unix socket which Gunicorn is bound to, BasicAuthentication doesn't work anymore. All requests are granted, without providing a username and password.
I know basic authentication should be avoided, but it's a requirement for a project I'm working on. Does anyone know why this happens and how to solve this?
settings.py:
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.BasicAuthentication',
),
}
nginx.conf:
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
upstream *** {
server unix:/***/***/***/***/***/***/***.sock
fail_timeout=0;
}
server {
server_name <server-name>;
satisfy all;
allow <IP-address>
deny all;
# location = /favicon.ico {access_log off; log_not_found off;}
location /static/ {
autoindex on;
alias ../static/;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://<upstream>;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/***/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/***/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
Okay so I managed to solve it myself. I don't know what the cause was because I also had the same problem before using Gunicorn, but restarting it solved it somehow.
Hello i have a complete django app which I'll be hosting on aws ec2 with gunicorn and nginx. But when I run the app with gunicorn or python3 my app runs and shows all images with the css and styles. But whenever I try to serve my django app with nginx by proxy_pass the image or style or anything on static folder could not be accessed. my error log shows permission issue. But I have set the app with user and group both nginx. but nothing is showing.
nginx conf on nginx.conf
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 4096;
include /etc/nginx/mime.types;
default_type application/octet-stream;
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
server_name example.com;
root /home/ec2-user/sites/softopark-django-cms;
access_log /var/log/nginx/example.com.access.log;
error_log /var/log/nginx/example.com.error.log;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /usr/share/nginx/html/softopark-django-cms/static/;
}
location /media/ {
root /usr/share/nginx/html/softopark-django-cms/media/;
}
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://0.0.0.0:8000/;
}
}
my gunicorn service file
[Unit]
Description=test
After=network.target
[Service]
User=nginx
Group=nginx
WorkingDirectory=/usr/share/nginx/html/softopark-django-cms
Environment="/home/ec2-user/Env/cms5-XkwMz5k7/bin"
ExecStart=/home/ec2-user/Env/cms5-XkwMz5k7/bin/gunicorn --workers 3 --bind 0.0.0.0:8000 cms_project.wsgi
[Install]
WantedBy=multi-user.target
can anyone help me with this?
I need to host my django app on aws but with nginx I'm not being able to do it. Please help.
Well, I didn't find any specific answer to this but I fixed it after running collectstatic command.
One thing, I missed that whenever we create any Django projects and at the time of deploying to another place we need to run python manage.py collectstatic so that our app can detect the static files and update their path. So I updated the codebase and deployed it on the server the but didn't used collectstatic, so i was unable see those images.
The python manage.py collectstatic did the work.
I have a centos server with nginx installed where i would to load a django app.
After install python34, nginx, django and gunicorn i configure nginx.conf file like this:
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
index index.html index.htm;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name 54.17X.2XX.11X;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location /static/ {
#alias /opt/cath/static/;
alias /home/ec2-user/test/endpoint/website/static/;
}
location / {
proxy_pass http://127.0.0.1:8000/site;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Real-IP $remote_addr;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
}
well at this point, when i start gunicorn and nginx seervic for all the resource under /home/ec2-user/test/endpoint/website/static/ i receive the 403 forbidden access error.
I try with:
sudo chmod -R 777 <Path to static>
also tried with
sudo chown -R nginx:nginx <path to static>
but error 403 persist
How can i resolve this issue?
Thanks in advance
but the problem persist
i think it is related to SELinux you can disable it by:
nano /etc/selinux/config
change the value of SELINUX to disabled
location / { include proxy_params; proxy_pass http://unix:/run/gunicorn.sock; }
Why nginx run default page ? how to listen my django server ?
First inside the sites-availabe folder i created example.com file then i
[root#instance-4 sites-available]# ls -al /etc/nginx/sites-enabled/example.com
lrwxrwxrwx. 1 root root 21 Dec 22 11:03 /etc/nginx/sites-enabled/example.com -> example.com
/etc/nginx/sites-available/example.com
server {
listen 80;
server_name example.com;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
Then when i run gunicorn example.wsgi in my app folder and later i visited the example.com but you know what i am still getting nginx default page.
What i am missing here ?
Updated :
Now this time i created example.com file in my Django root folder then after Symlink
[root#instance-4 Staging]# ln -s example.com /etc/nginx/sites-enabled/
after the nginx restart still same ...
Updated 2 :
nginx.conf file
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
include /etc/nginx/sites-enabled/*;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
Check for a default in /etc/nginx/site-enabled/ and remove it if it's there. Then reload or restart your nginx server.
You can also check gunicorn is serving requests by visiting example.com:8000.
It's worthwhile noting that you'll probably also want nginx to be serving your static files so put in a /static/ block:
location /static/ {
alias /path/to/your/app/static/;
if ($query_string) {
# If using GET params to control versions, set to max expiry.
expires max;
}
access_log off;
}
From what i remember of nginx, there is 2 places where you can find the index.html of nginx, try to do a "find / -name index.html" you will prolly find the 2nd .html i am talking about, and regarding the path u should be able to fix this