Bad Request on File upload - django

I have setup my python project using nginx and gunicorn as described in this tutorial https://djangocentral.com/deploy-django-with-nginx-gunicorn-postgresql-and-lets-encrypt-ssl-on-ubuntu/
so far everything works perfectly fine.
My project is in /root/project_folder/
I want to upload files via the admin page, but I get a bad request error 400. The media folder I want to add the files to is /var/www/my_domain/media (owned by the group www-data). This is also properly configured since I can see the images when I move them manually into that folder.
Do you guys maybe have any idea why the issue may be ?
the main problem is when uploading I need to save the image to the media root:
/var/www/domain/media/images/dad/ProfilePicture/offline_screen.png
But when I send request to view the image the web server returns the following:
/media/var/www/domain/media/images/dad/ProfilePicture/offline_screen.png
The upload_to path needs to be images/dad/ProfilePicture/offline_screen.png for correct request but then the image is uploaded to the wrong folder
Any ideas ? Thanks in advance!
UPDATE:
Here is my nginx config
`
server {
server_name domain.net www.domain.net ip_address;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /var/www/domain;
}
location = /media/ {
root /var/www/domain;
}
location / {
include proxy_params;
proxy_pass http://unix:/var/log/gunicorn/domain.sock;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/domain/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/domain/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = www.domain) {
return 301 https://$host$request_uri;
} # managed by Certbot
if ($host = domain) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name domain www.domain ip_address;
return 404; # managed by Certbot
}
`
Here is my media root / media url:
MEDIA_ROOT = '/var/www/domain/media' #os.path.join(BASE_DIR, 'media')
MEDIA_URL = '/media/'
The url.py:
urlpatterns = [
path('admin/', admin.site.urls),
path('', include('webpages.urls')),
]
if settings.DEBUG:
urlpatterns += static(settings.MEDIA_URL,document_root=settings.MEDIA_ROOT)
urlpatterns += static(settings.STATIC_URL,document_root=settings.STATIC_ROOT)
And finally how I serve and upload Images:
def get_path_of_content_image(instance, filename):
return os.path.join(settings.MEDIA_ROOT, "images", str(instance.course.topic), str(instance.course), filename)
class Topic(models.Model):
title = models.CharField(max_length=100)
description = models.TextField(max_length=445)
image = models.ImageField(upload_to=get_topic_profile_path, default="images/default.jpeg")
def __str__(self):
return self.title
def image_tag(self):
if self.image:
return mark_safe('<img src="%s" style="width: 200px; height:200px;" />' % self.image.get_image())
def get_image(self):
if self.image:
return str(self.image.url).replace('/media/var/www/domain', '') ########1
else:
return settings.STATIC_ROOT + 'webpages/img/default.jpeg'
image_tag.short_description = 'Current Image'
I wrote this line #######1 since set the upload path to the full media root but then in order to return the correct path I need to remove the media root prefix. And also very importantly this only works if DEBUg set to TRUE.

I'm not certain this will solve your issue, but there are 2 things that need to be configured correctly here. If nginx is serving the files in your media directory ok, I'm guessing your nginx config is already ok and you just need to edit settings.py, but I'll include the nginx config in case it helps someone else.
UPDATE
I don't think those methods in your models are necessary, although I don't really know how they are used. Here's an example of how I configure a model with an ImageField:
Firstly, how to upload and save images
(letting django do most of the work)
models.py
...
class Post(models.Model):
"""Represents a single blog post"""
# model fields
title = models.CharField(max_length=150, unique=True)
content = models.TextField()
title_image = models.ImageField(upload_to='image/', blank=True, null=True)
...
forms.py
...
class PostModelForm(forms.ModelForm):
class Meta:
model = Post
fields = ['title', 'title_image', 'content']
widgets = {
'content': Textarea(attrs={'cols': 80, 'rows': 25}),
}
...
views.py
...
def new_post(request):
context = {}
template = 'new_post.html'
# POST
if request.method == 'POST':
form = PostModelForm(request.POST or None, request.FILES)
if form.is_valid():
new_post = form.save()
return redirect('list_post')
else:
context['form'] = form
return render(request, template, context)
# GET
else:
context['form'] = PostModelForm()
return render(request, template, context)
...
new_post.html (note the enctype of the form - important if you want to upload files)
...
<div class="card bg-light">
<div class="card-header">
<h3>New Post</h3>
</div>
<div class="card-body">
<form action="{% url 'blog:new_post' %}" method="POST" class="w-100" enctype="multipart/form-data">
<div class="col-12">
{% csrf_token %}
{{ form.as_p }}
</div>
<div class="col-12">
<ul class="actions">
<button class="primary" type="submit">Save</button>
</ul>
</div>
</form>
</div>
</div>
...
Secondly, how to serve your images
Now you want to display the image in you markup
In views.py
...
def detail(request, id):
context = {}
template_name = 'post.html'
post = get_object_or_404(Post, id=id)
context['post'] = post
return render(request, template_name, context)
...
In your markup in post.html
...
{% if post.title_image %}
<span class="img"><img src="{{ post.title_image.url }}" alt="image for {{ post.title }}" /></span>
{% endif %}
...
(1) Define MEDIA_ROOT and MEDIA_URL in settings.py
In settings.py add
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
MEDIA_URL = '/media/'
Note: Read the django docs on MEDIA_ROOT and MEDIA_URL
(2) define media location in nginx server block
In that tutorial the nginx configuration doesn't define a location for media. Here's the server block from the tutorial:
server {
listen 80;
server_name server_domain_or_IP;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /path/to/staticfiles;
}
location / {
include proxy_params;
proxy_pass http://unix:/var/log/gunicorn/project_name.sock;
}
}
I think it should look something like this:
server {
listen 80;
server_name server_domain_or_IP;
location = /favicon.ico { access_log off; log_not_found off; }
location /media/ {
root /path/to/media;
}
location /static/ {
root /path/to/staticfiles;
}
location / {
include proxy_params;
proxy_pass http://unix:/var/log/gunicorn/project_name.sock;
}
}
If that configuration doesn't work, I've posted what works for me below
In my own experience with django, nginx and gunicorn however, my nginx configuration looks like this:
server {
listen 443 ssl;
listen [::]:443 ssl;
ssl_certificate /etc/letsencrypt/live/website_name.com-0001/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/website_name.com-0001/privkey.pem; # managed by Certbot
server_name website_name.com www.website_name.com;
location = /favicon.ico { access_log off; log_not_found off; }
location /media {
alias /home/user_name/website_name/project_name/media;
}
location /static/ {
alias /home/user_name/website_name/project_name/static_root/;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
}
Note 1: Here are the nginx docs for alias and root directives.

So after spending an eternity on debugging, I have the solution to my problem:
First Thing that was problematic is the '=' between 'location' and '/media/' in the nginx file. Be sure that this is not root of the error. It should look like this:
location /media {
alias /home/user_name/website_name/project_name/media;
}
and not like this:
location = /media {
alias /home/user_name/website_name/project_name/media;
}
The file upload to a custom local directory can be done like this:
from django.core.files.storage import FileSystemStorage
fs = FileSystemStorage(location=settings.MEDIA_ROOT)
and for the model add the file storage option on upload (or change the name of the image with the updated path afterwards):
image = models.ImageField(upload_to=get_topic_profile_path, default="images/default.jpeg", storage=fs)
thanks for the other answers. I learned quite a lot by fixing this erros :D

Related

Nginx : How to serve multiple contents with a single internal directive after authentication

I want to display my documentation (a single-page application built with React) only after authentication with my backend.
My configuration :
Nginx acts as a reverse proxy for the backend (Django) and serves static files like single-page-applications.
Django, the backend, identifies the user and makes a request to Nginx using X-Accel-Redirect.
So I proceed as follows:
1) Authentication on Django
views.py
def get_doc(request):
if request.method == 'POST':
form = PasswordForm(request.POST)
if form.is_valid():
if form.cleaned_data['password'] == 'foo':
response = HttpResponse()
response['Content-Type'] = ''
response['X-Accel-Redirect'] = '/docs-auth/'
return response
else:
return HttpResponse("Wrong password")
else:
form = PasswordForm()
return render(request, 'docs/form.html', {'form': form})
urls.py
urlpatterns = [
path('docs/', views.get_doc, name='documentation'),
]
2) Nginx serves the single-page application
upstream backend {
server web:8000;
}
server {
location = /favicon.ico {access_log off;log_not_found off;}
...
location /docs {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_redirect off;
proxy_pass http://backend;
}
location /docs-auth/ {
internal;
alias /home/foo/docs/;
index index.html;
try_files $uri $uri/ /docs/index.html;
}
location / {
alias /home/foo/landing_page/;
error_page 404 /404.html;
index index.html;
try_files $uri $uri/ =404;
}
}
My problem is that the index.html file is served to the user but then the browser requests to access CSS and Javascript files are blocked because the browser cannot access the internal url.
Do you have any ideas to solve my problem?
I am also open to another way to serve a single-page application after backend authentication.
Thanks a lot.
You want to use the auth_request tag to make your life easier. Here is an example that you will need to retrofit unto your config. You could make your whole server require auth my just moving the auth_request tag to the top level outside of location
server {
...
location /docs {
auth_request /docs-auth;
...// Add your file redering here
}
location = /docs-auth {
internal;
proxy_pass http://auth-server;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URI $request_uri;
}
}

Django Uploaded images not displayed in production

I am aware of this question: Django Uploaded images not displayed in development , I have done everything that it is described, but still can't find a solution. I also have used for reference this: GeeksForGeeks and Uploaded Files and Uploaded Handlers - Django documentation, however, none of them solved my problem.
I have deployed a Django App on a Ubuntu server for the first time using Nginx and gunicorn.
Before deployment, I used port 8000 to test if everything runs as it is supposed to and all was fine. Since I allowed 'Nginx Full' my database images are not showing up.
This is my django project structure:
My virtual environment folder and my main project folder are both in the same directory. I have separated them.
# Create your models here.
class Project(models.Model):
project_name = models.CharField(max_length=120)
project_description = models.CharField(max_length=400)
project_link = models.CharField(max_length=500)
project_image = models.ImageField(upload_to='')
def __str__(self):
return self.project_name
I have set up my settings.py to :
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(os.path.dirname(__file__), '..', 'media').replace('\\','/')
My view gets all the project object from a database and passes those to the template. The template renders successfully all other information related to project model except the image field . In my template I do:
<div class="row text-center mx-auto">
{% for project in projects %}
{% if forloop.counter|mod:2 == 0 %}
<div class="col projects pb-3 pt-3 mb-3 ml-2">
{% else %}
<div class="col projects pb-3 pt-3 mb-3 mr-2">
{% endif %}
<img class="card-img-top pt-2 pl-2 pr-2" src="{{ project.project_image.url}}"
alt="Image could not be found :(" style="height:120px; width:166px !important;" /><br>
<div class="card-body">
<h3 class="card-title ">{{ project.project_name }}</h3>
<p class="card-text dates">{{ project.project_description}}</p>
Link
</div>
</div>
{% if forloop.counter|mod:2 == 0 %}
<div class="w-100"></div>
{% endif %}
{% endfor%}
</div>
</div>
Uploading the images works, it sends them in the project's media directory, the problem is that they are not showing up, the alt="" is activated.
My main urls.py:
urlpatterns = [
path('', include('project.urls')),
path('admin/', admin.site.urls),
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
The gunicorn system file:
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=myusername
Group=www-data
WorkingDirectory=/home/myusername/myproject
ExecStart=/home/myusername/venv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/home/myusername/myproject/myproject.sock myproject.wsgi:application
[Install]
WantedBy=multi-user.target
Nginx setup:
server {
listen 80;
server_name <my IP> ;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/myusername/myproject;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/myusername/myproject/myproject.sock;
}
}
EDIT: When inspecting the image element of the webpage the source of the image it is "/media/imageNmae.png".
Any help would be appreciated!
EDIT: The solution has been found, the Nginx was not serving the media as Daniel suggested. There is some uWSGI documentation which is worth reading to avoid further similar problems Documentation
Your Nginx is not serving the MEDIA_URL i.e. /media/. You need an Nginx configuration section like you have for /static/
server {
listen 80;
server_name <my IP> ;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/myusername/myproject;
}
location /media/ {
root /home/myusername/myproject;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/myusername/myproject/myproject.sock;
}
}
You may need to run this command :
python manage.py collectstatic from the shell of your platform
if you are using heroku here is the command
heroku run python manage.py collectstatic

X-Accel-Redirect Nginx configuration not working

I'm now deploying django projects on CentOS and have a problem with X-Accel-Redirect for protected file serving.
Here is my nginx.conf
server
{
listen 80;
server_name example;
index index.html index.htm index.php;
root /www/server/example;
charset UTF-8;
access_log /var/log/nginx/myproject_access.log;
error_log /var/log/nginx/myproject_error.log;
client_max_body_size 75M;
location /public/ {
root /www/wwwroot/myproject/;
}
location /media/ {
root /www/wwwroot/myproject/;
internal;
}
location / {
include uwsgi_params;
uwsgi_pass django;
}
...
}
Of course, protected files are on /www/wwwroot/myproject/media.
And corresponding python view file is following.
class ProtectedView(generics.GenericAPIView):
permission_classes = [IsAuthenticated]
def get(self, request, id, target):
file_name = "1.png"
response = HttpResponse()
response["X-Accel-Redirect"] = "/media/{0}".format(file_name)
response["Content-Disposition"] = "attachment; filename={0}".format(file_name)
return response
But server returns 404 error.
And the myproject_error.log is like this.
[error] 24570#0: *5 open() "/www/server/example/media/1.png" failed (2: No such file or directory), client: 174.11.13.81, server: example, request: "GET /protected-view/ HTTP/1.1", upstream: "uwsgi://0.0.0.0:8008", host: "40.1.12.23"
Maybe location /media/ {} block is not working. What problem? I have changed the permission but not working.
PS: I'm using django rest framework.

main view keeps redirecting to login page even for logged-in user in django

my main view is,
def main(request):
if request.user.is_authenticated:
return render(request,'main/main.html',{'use':use,'query':query,'noter':noter,'theme':request.user.profile.theme})
else:
return redirect('home')
the home page is responsible for serving login form is
def home(request):
if request.user.is_authenticated:
return redirect('main')
else:
return render(request,'home.html')
the "home.html" uses ajax to submit login page
$(document).ready(function(){
$('#form2').submit(function(event){
event.preventDefault();
$('#username_error').empty();
$("#password_error").empty();
var csrftoken = $("[name=csrfmiddlewaretoken]").val();
var formdata={
'username':$('input[name=username2]').val(),
'password':$('input[name=loginpassword]').val(),
};
$.ajax({
type:'POST',
url:'/Submit/logging',
data:formdata,
dataType:'json',
encode:true,
headers:{
"X-CSRFToken": csrftoken
},
})
.done(function(data){
if(!data.success){//we will handle error
if (data.password){
$('#password_error').text(data.password);
}
if(data.Message){
$('#password_error').text("You can't login via pc");
}
blocker();
return false;
}
else{
window.location='/';
}
});
event.preventDefault();
});
this configuration is working fine but main page keeps redirecting to home.html even when though user is already logged in. I'm using nginx server whose configuration is
server {
server_name host.me www.host.me;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/username/projectname;
}
location /media/ {
root /home/username/projectname/appname;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/username/projectname/projectname.sock;
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/www.host.me/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/www.host.me/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}server {
if ($host = www.host.me) {
return 301 https://$host$request_uri;
} # managed by Certbot
if ($host = host.me) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name host.me www.host.me;
return 404; # managed by Certbot
}
Edit 1
The urls of corresponding views are,
path('home',views.home,name='home'),
path('',views.main,name='main'),
path('Submit/logging',views.Loging,name='Loging'),
while my view for performing login is
def Loging(request):
if request.user_agent.is_pc:
return JsonResponse({'Message':'pc'})
else:
username1=request.POST['username']
print(username1)
username=username1.lower()
password=request.POST['password']
print(password)
user=authenticate(request,username=username,password=password)
if user is not None:
login(request,user)
return JsonResponse({'success':True})
else:
return JsonResponse({'password':'The user credentials do not exist in our database'})
Also if a user tries to access website from another url endpoint(say mywebsite.me/another) & already authenticated , then it will lead to the correct view which means login is performed correctly.

Django serve static files with user verification [duplicate]

I have been fumbling around with trying to protect Django's media files with no luck so far! I am simply trying to make it where ONLY admin users can access the media folder. Here is my Nginx file.
server {
listen 80;
server_name xxxxxxxxxx;
location = /favicon.ico {access_log off; log_not_found off;}
location /static/ {
alias /home/{site-name}/static_cdn/;
}
location /media/ {
internal;
root /home/{site-name}/;
}
location / {
this is setup and working. Didn't include Code though
}
My Url File
urlpatterns = [
url(r'^media/', views.protectedMedia, name="protect_media"),
]
And my view
def protectedMedia(request):
if request.user.is_staff:
response = HttpResponse()
response['Content-Type'] = ''
response['X-Accel-Redirect'] = request.path
return response
else:
return HttpResponse(status=400)
This is producing a 404 Not Found Nginx error. Does anything look blatantly wrong here? Thanks!
BTW, I have tried adding /media/ to the end of the root URL in the Nginx settings.
This is what fixed this issue thanks to #Paulo Almeida.
In the nginx file I changed what I previosly had too...
location /protectedMedia/ {
internal;
root /home/{site-name}/;
}
My url is...
url(r'^media/', views.protectedMedia, name="protect_media"),
And the View is...
def protectedMedia(request):
if request.user.is_staff:
response = HttpResponse(status=200)
response['Content-Type'] = ''
response['X-Accel-Redirect'] = '/protectedMedia/' + request.path
return response
else:
return HttpResponse(status=400)
This works perfectly! Now only admin users can access the media files stored in my media folder.
This helped me a lot, just a small update and modification:
urls.py:
re_path(r'^media/(?P<path>.*)', protectedMedia, name="protect_media")
views.py:
from urllib.parse import quote
from django.http import HttpResponse
from django.contrib.admin.views.decorators import staff_member_required
#staff_member_required
def protectedMedia(request, path):
response = HttpResponse(status=200)
response["Content-Type"] = ''
response['X-Accel-Redirect'] = '/protectedMedia/' + quote(path)
return response
I had to change the nginx config to the following:
location /protectedMedia/ {
internal;
alias /home/{site-name}/;
}
Notes:
I prefer using the decorator, as it automatically redirects to the login page (when specified in settings) and sets the "next" page.
url() gets deprecated in Django 3.1 so just use re_path() instead
alias instead of root in nginx config: I don't want to have "/protectedMedia/" appear in the url (and it didn't work), see also nginx docs
edit from 2021-06-14: German-Umlauts (äüö etc.) didn't work in media paths so I edited the above answer to include quote from urllib.parse
If you're still stuck somewhere, this gave me further backround information: https://wellfire.co/learn/nginx-django-x-accel-redirects/