Absolute paths on images uploaded by django-ckeditor - django

I am using django-rest-framework in conjuntion with django-ckeditor. I'm serving some images with absolute url-s without any problem. But images and files uploaded by ckeditor are served as relative paths, and they can't be displayed client side since it is in a different domain.
Here is an example of what I'm getting:
{
image: "http://example.com/media/myimage.png",
body: "<p>download my file</p>"
}
And this is what I woul like to get:
{
image: "http://example.com/media/myimage.png",
body: "<p>download my file</p>"
}
Edit:
This would be the model of my example:
from django.db import models
from ckeditor_uploader.fields import RichTextUploadingField
image: models.ImageField()
body: RichTextUploadingField(blank=True,null=True)

I would use a custom serializer to fix that:
from rest_framework import serializers
def relative_to_absolute(url):
return 'http://127.0.0.1:8000' + url
class FileFieldSerializer(serializers.Field):
def to_representation(self, value):
url = value.url
if url and url.startswith('/'):
url = relative_to_absolute(url)
return url
When filefield.url contains a relative url, relative_to_absolute() is called to prepend the domain.
Here I just used a constant string; you can either save it in your settings, or, if Django Site framework is installed, retrieve it as follows:
from django.contrib.sites.models import Site
domain=Site.objects.get_current().domain
Sample usage of the custom serializer:
class Picture(BaseModel):
...
image = models.ImageField(_('Image'), null=True, blank=True)
...
class PictureSerializer(serializers.ModelSerializer):
image = FileFieldSerializer()
class Meta:
model = Picture
fields = '__all__'
Variation for RichTextUploadingField
If, on the other hand, you're using RichTextUploadingField by CKEditor, your field is, basically, a TextField where an HTML fragment is saved upon images
upload.
In this HTML fragment, CKEditor will reference the uploaded images with a relative path, for very good reasons:
your site will still work if the domain is changed
the development instance will work in localhost
after all, we're using Django, not WordPress ;)
So, I wouldn't touch it, and fix the path at runtime in a custom serializer instead:
SEARCH_PATTERN = 'href=\\"/media/ckeditor/'
SITE_DOMAIN = "http://127.0.0.1:8000"
REPLACE_WITH = 'href=\\"%s/media/ckeditor/' % SITE_DOMAIN
class FixAbsolutePathSerializer(serializers.Field):
def to_representation(self, value):
text = value.replace(SEARCH_PATTERN, REPLACE_WITH)
return text
Alternatively, domain can be saved in settings:
from django.conf import settings
REPLACE_WITH = 'href=\\"%s/media/ckeditor/' % settings.SITE_DOMAIN
or retrieved from Django Site framework as follows:
from django.contrib.sites.models import Site
REPLACE_WITH = 'href=\\"{scheme}{domain}/media/ckeditor/'.format(
scheme="http://",
domain=Site.objects.get_current().domain
)
You might need to adjust SEARCH_PATTERN according to your CKEditor configuration; the more specific, the better.
Sample usage:
class Picture(BaseModel):
...
body = RichTextUploadingField(blank=True,null=True)
...
class PictureSerializer(serializers.ModelSerializer):
body = FixAbsolutePathSerializer()
class Meta:
model = Picture
fields = '__all__'

Related

django-rest-framework "This field is required" on POST

Whenever I POST to my django-rest-framework (DRF) endpoints, I keep receiving a "HTTP 400 Bad Request" {"offeror_organization":["This field is required."]} response. But, given the curl example below, I'm clearly specifying a value.
This happens regardless of the Content-Type (application/json, application/x-www-form-urlencoded, multipart/form-data). The only time it works is when I submit using the "HTML form" (vs. the "Raw Data") tab on the DRF web interface.
There's a few similar SO posts (like this and this), but none of the solutions seem to be working for me.
Model:
class OrganizationManager(models.Manager):
def get_by_natural_key(self, offeror_organization):
return self.get(offeror_organization=offeror_organization)
class Organization(models.Model):
idorganization = models.AutoField(primary_key=True)
offeror_organization = models.CharField(max_length=250, null=False, blank=False, verbose_name='Offeror Organization')
created_at = models.DateTimeField(auto_now_add=True, null=False)
updated_at = models.DateTimeField(auto_now=True, null=False)
objects = OrganizationManager()
def natural_key(self):
return "%s" % (self.offeror_organization)
def __str__(self):
return self.offeror_organization
Serializer:
class OrganizationSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = Organization
fields = ['offeror_organization']
# I've tried both with and without a create function
def create(self, validated_data):
organization_data = validated_data.pop('offeror_organization', None)
if organization_data:
organization = Organization.objects.get_or_create(**organization_data)[0]
validated_data['offeror_organization'] = organization
views/api.py:
from webapp.models import Organization
from webapp.serializers import OrganizationSerializer
from rest_framework import viewsets
class OrganizationViewSet(viewsets.ModelViewSet):
queryset = Organization.objects.all().order_by('offeror_organization')
serializer_class = OrganizationSerializer
urls.py:
from django.urls import include, path
from rest_framework import routers
from . import views
router = routers.DefaultRouter()
router.register(r'organization', views.OrganizationViewSet)
urlpatterns = [
...
path('api/', include(router.urls)),
path('api-auth/', include('rest_framework.urls', namespace='rest_framework')),
]
Curl Command:
curl -X POST -H 'Content-Type: application/json' -d '{"offeror_organization":"Test2"}' 10.101.10.228:29000/webapp/api/organization/
settings.py MIDDLEWARE:
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.RemoteUserMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'csp.middleware.CSPMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware'
]
settings.py REST_FRAMEWORK
# currently have all API authentication disabled while troubleshooting this issue
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': [],
'DEFAULT_PERMISSION_CLASSES': [],
}
In my case, fixing this issue required "maneuvering" around a few different implementation constraints.
nginx + uWSGI Socket + Django REMOTE_USER Authentication:
As mentioned in this post's comments/chat, I've got both an nginx proxy and a uWSGI application server fronting my Django application. Since I'm relying upon REMOTE_USER Authentication, my uwsgi/nginx configuration must use uWSGI sockets (vs. http) so that I may pass the REMOTE_USER from nginx to Django as an environment variable. When using http (coupled w/ nginx proxy_pass), although proxy_pass can set headers or cookies, those seemingly cannot translate over to Django (which requires the environment variable).
I think there's a few issues at play when trying to POST to a Django/DRF application served using uWSGI sockets. Per the uWSGI Things to know (best practices), "TL/DR: if you plan to expose uWSGI directly to the public, use --http, if you want to proxy it behind a webserver speaking http with backends, use --http-socket". In my case, having both a web application and a DRF-based API (that I want other services and systems to talk to), I need both! As a (hopefully temporary) workaround, I'm currently spawning two uWSGI processes - one using --socket, and one using --http (for API POST calls). If you POST while using ---socket, you'll likely get an Empty Response error from DRF.
As an aside, I initially saw some "promise" in utilizing uwsgi_curl (from uwsgi_tools) to POST over the uWSGI socket (which resulted in the "field is required" error (vs. the Empty Response error), but that's when I started to run into my second issue...
POST nested application/json w/ simultaneous file upload: The "Organization" model referenced in the post was mostly proof-of-concept, as it's the least complex model in my Django application. In reality, I need to post to a more complex model with nested serialization, as the model contains Foreign Key's to other models. But that's totally do-able with DRF. Except in my case, where one of my model attributes is a FileUpload field. As noted in other SO Questions (like this one), there's also a few issues in trying to POST nested (i.e. not "flat") application/json with a file upload in a single request. While I was never able to fully understand the issue at play (at least using drf_writable_nested.serializers.WritableNestedModelSerializer in my case), I simplified the problem at-hand by writing my own custom Serializer (serializers.Serializer), this way I could avoid nested JSON objects (like { "offeror_organization": {"offeror_organization: "Test"}} in my POST requests. This fixed my issue.
With the custom serializer in place to mitigate the nested JSON + file upload issue, I bet the uwsgi_curl POST would work. Although then external client systems/services are limited to using that Python package. Anyways, I'll update my answer once I try it out. Thanks to #Michael for his comments and helping to lead me down the right "road".
I have the same setup (nginx + gunicorn + django + rest-framework + drf-writeable-nested) but I could figure out a valid format for the POST request containing multipart/form-data:
It needs to be like this:
json:
{
'name': 'test-name',
'files': [
{
'file': 'test-file-1'
},
{
'file': 'test-file-2'
},
...
],
...
}
must be formatted to:
FormData:
name: test-name
files[0]file: test-file-1
files[1]file: test-file-2
...
Some libraries would use a dot after the brackets for nested lists, which would lead to the This field is required error. Or even another bracket after the list bracket would lead to the same error.
This is wrong:
files[0].file
This is also wrong:
files[0][file]
My example assumes the following Django-classes:
# views.py
from rest_framework.viewsets import ModelViewSet
from rest_framework.parsers import MultiPartParser, FormParser
from .serializers import YourCustomModelSerializer
class YourCustomModelViewSet(ModelViewSet):
queryset = YourCustomModel.objects.all()
parser_classes = [FormParser, MultiPartParser]
permission_classes = []
serializer_class = YourCustomModelSerializer
# serializers.py
from rest_framework.serializers import ModelSerializer
from drf_writable_nested.serializers import WritableNestedModelSerializer
from .models import YourCustomModel, File
class FileSerializer(ModelSerializer):
class Meta:
model = File
fields = ['file']
class YourCustomModelSerializer(WritableNestedModelSerializer):
# Reverse foreign key
files = FileSerializer(many=True)
class Meta:
model = YourCustomModel
read_only_fields = ('id', )
fields = [
'id',
'name',
'files'
]
# models.py
from django.db import models
class File(models.Model):
file = models.FileField()
class YourCustomModel(models.Model):
name = models.CharField(max_length=200)
I used the following javascript/typescript frontend code to pack my json data into a FormData request:
const requestBody = {
name: 'test-name',
files: [
{ file: file1 }, // File object
{ file: file2 }, // File object
{ file: file3 } // File object
]
}
// # use your own code to serialize the above javascript dict/json/object
// into form-data object (I used the library https://www.npmjs.com/package/object-to-formdata but I had to write a patch for the exact reason described above: files[0].file is not correctly parsed by Django and files[0][file] also won't work, therefore I slightly changed the library code so that it will format my object to files[0]file for every nested item:
// 2. prepare a serializer-library to convert the dict into a special form which can be parsed by django.
const options = {
indices: true,
allowEmptyArrays: true,
useDotOrBracketsOrNothingForNestedObjects: 2 // Option 0: DOT-Notation, 1: Brackets, 2: Nothing (this option is from my custom patch)
}
// use npx patch: https://stackoverflow.com/a/62567504/3433137
// (I patched this serialize library and the patch is somewhere stored as a file in this project)
const formData = serialize(
requestBody,
options
)
// 3. upload the data
api.post(url, formData)

Django url path converter not working in production

I'm using path converter in my django app like so:
# urls.py
from . import views
from django.urls import path
urlpatterns = [
path('articles/<str:collection>', views.ArticleView),
]
# views.py
#login_required
def ArticleView(request, collection):
print(collection)
if collection == "None":
articles_query = ArticleModel.objects.all()
...
This works fine in development for a url suck as : http://localhost:8000/articles/My Collection which gets encoded to http://localhost:8000/articles/My%20Collection, and is decoded properly in the ArticleView. However, in development, I have to edit the view like so to get it to work:
# views.py
import urllib.parse
#login_required
def ArticleView(request, collection):
collection = urllib.parse.unquote(collection)
print(collection)
if collection == "None":
articles_query = ArticleModel.objects.all()
...
Otherwise, the print(collection) shows My%20Collection and the whole logic in the rest of the view fails.
requirements.txt
asgiref==3.2.10
Django==3.1.1
django-crispy-forms==1.9.2
django-floppyforms==1.9.0
django-widget-tweaks==1.4.8
lxml==4.5.2
Pillow==7.2.0
python-pptx==0.6.18
pytz==2020.1
sqlparse==0.3.1
XlsxWriter==1.3.3
pymysql
What am I doing wrong here?
Thanks in advance!
The URL is being urlencoded which encodes spaces as %20. There are a number of other encodings. As you've discovered you need to decode that parameter in order to compare it to what you'd expect. As you've likely realized, if you have a value that actually wants The%20News and not The News, you have no recourse. To handle this people will create a slug field. Django has a model field for this in the framework.
This is typically a URL-friendly, unique value for the record.
Assuming you add a slug = models.SlugField() to ArticleModel, your urls and view can change into:
urlpatterns = [
# Define a path without a slug to identify the show all code path and avoid the None sentinel value.
path('articles', views.ArticleView, name='article-list'),
path('articles/<slug:slug>' views.ArticleView, name='article-slug-list'),
]
#login_required
def ArticleView(request, slug=None):
articles_query = ArticleModel.objects.all()
if slug:
articles_query = articles_query.filter(slug=slug)

Embedding Video File in Django Site

I have a Django site that I'm creating, and I want some of the pages to have videos embedded in them. These videos aren't part of a model. I just want to be able to use the view to figure out which video file to play, and then pass the file path into the template. All the files are hosted locally (for now, at least).
Is it possible to do with Django? And if so, how do I do it?
There are two ways you can do this -
Method 1: Pass parameter in URL and display video based on that parameter -
If you don't want to use models at any cost, use this, else try method 2.
Assuming you have saved all videos in your media directory and they all have unique names (serving as their ids).
your_app/urls.py -
from django.conf.urls import url
from . import views
urlpatterns = [
url(r'^video/(?P<vid>\w+)/$',views.display_video)
# \w will allow alphanumeric characters or string
]
Add this in the project's settings.py -
#Change this as per your liking
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
your_app/views.py -
from django.conf import settings
from django.shortcuts import render
from django.http import HttpResponse
import os
import fnmatch
def display_video(request,vid=None):
if vid is None:
return HttpResponse("No Video")
#Finding the name of video file with extension, use this if you have different extension of the videos
video_name = ""
for fname in os.listdir(settings.MEDIA_ROOT):
if fnmatch.fnmatch(fname, vid+".*"): #using pattern to find the video file with given id and any extension
video_name = fname
break
'''
If you have all the videos of same extension e.g. mp4, then instead of above code, you can just use -
video_name = vid+".mp4"
'''
#getting full url -
video_url = settings.MEDIA_URL+video_name
return render(request, "video_template.html", {"url":video_url})
Then in your template file, video_template.html, display video as -
<video width="400" controls>
<source src="{{url}}" type="video/mp4">
Your browser does not support HTML5 video.
</video>
Note: There can be performance issue, iterating through all the files in the folder using os.listdir(). Instead, if possible, use a common file extension or use the next method (strongly recommended).
Method 2 : Storing video ids and correspondig file names in database -
Use same settings.py, urls.py and video_template.html as in method 1.
your_app/models.py -
from django.db import models
class videos(models.Model):
video_id = models.CharField(blank=False, max_length=32)
file_name = models.CharField(blank=False, max_length=500)
def __str__(self):
return self.id
your_app/views.py -
from django.conf import settings
from django.shortcuts import render, get_object_or_404
from django.http import HttpResponse
from .models import videos
def display_video(request,vid=None):
if vid is None:
return HttpResponse("No Video")
try:
video_object = get_object_or_404(videos, pk = vid)
except videos.DoesNotExist:
return HttpResponse("Id doesn't exists.")
file_name = video_object.file_name
#getting full url -
video_url = settings.MEDIA_URL+file_name
return render(request, "video_template.html", {"url":video_url})
So if you want to access any page with video id 97veqne0, just goto - localhost:8000/video/97veqne0

How to upload image to custom location using models.ImageField and attribute?

I am an absolute beginner with Django so here is my question:
I created a model and I have image = models.ImageField(). When I use the admin interface, every uploaded image is placed in the root directory.
I read through this https://docs.djangoproject.com/en/dev/topics/files/ and if I use the below example it still doesn't work.
from django.core.files.storage import FileSystemStorage
fs = FileSystemStorage(location='/static/images/gallery')
class Car(models.Model):
...
photo = models.ImageField(storage=fs)
The new entry is correctly added but when I click the image name, the following error is displayed, and the image is not placed in /static/images/gallery
Page not found (404)
Request Method: GET
Request URL: http://127.0.0.1:8000/admin/Library/book/12/change/002_user_experience_remastered.jpg/change/
Raised by: django.contrib.admin.options.change_view
book object with primary key '12/change/002_user_experience_remastered.jpg' does not exist.
You're seeing this error because you have DEBUG = True in your Django settings file. Change that to False, and Django will display a standard 404 page.
My code as I wrote it:
from django.core.files.storage import FileSystemStorage
fs = FileSystemStorage(location='/static/images/gallery')
class book(models.Model):
name = models.CharField(max_length=128, unique=True)
authors = models.CharField(max_length=128)
language = models.CharField(max_length=128)
otherDetails = models.URLField()
availableCopies = models.IntegerField()
borrowUntilDate = models.DateField()
image = models.ImageField(storage=fs)
commentName = models.CharField(max_length=128)
commentText = models.CharField(max_length=2048)
slug = models.SlugField(unique=True)
My project has the following configuration:
..\Services
Services
Library (my app)
static
templates
venvs
From the admin interface I load pics from C:\pics and I want them to be stored in ..\Services\Library\static\images\gallery folder.
What am I doing wrong? What am I missing?
Thank you!
From the documentations:
During development, you can serve user-uploaded media files from MEDIA_ROOT using the django.contrib.staticfiles.views.serve() view.
This is not suitable for production use! For some common deployment strategies, see Deploying static files.
For example, if your MEDIA_URL is defined as /media/, you can do this by adding the following snippet to your urls.py:
from django.conf import settings
from django.conf.urls.static import static
urlpatterns = [
# ... the rest of your URLconf goes here ...
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
You need to specify the absolute path of your folder in the FileSystemStorage location and also provide a base_url if you have not set MEDIA_URL in settings.
fs = FileSystemStorage(
location='C:/.../Services/Library/static/images/gallery',
base_url='/gallery/'
)
References:
FileSystemStorage

Django cms accessing extended property

I've extended the Django cms Page model into ExtendedPage model and added page_image to it.
How can I now acces the page_image property in a template.
I'd like to access the page_image property for every child object in a navigation menu... creating a menu with images...
I've extended the admin and I have the field available for editing (adding the picture)
from django.db import models
from django.utils.translation import ugettext_lazy as _
from cms.models.pagemodel import Page
from django.conf import settings
class ExtendedPage(models.Model):
page = models.OneToOneField(Page, unique=True, verbose_name=_("Page"), editable=False, related_name='extended_fields')
page_image = models.ImageField(upload_to=settings.MEDIA_ROOT, verbose_name=_("Page image"), blank=True)
Thank you!
BR
request.current_page.extended_fields.page_image
should work if you are using < 2.4. In 2.4 they introduced a new two page system (published/draft) so you might need
request.current_page.publisher_draft.extended_fields.page_image
I usually write some middleware or a template processor to handle this instead of doing it repetitively in the template. Something like:
class PageOptions(object):
def process_request(self, request):
request.options = dict()
if not request.options and request.current_page:
extended_fields = None
try:
extended_fields = request.current_page.extended_fields
except:
try:
custom_settings = request.current_page.publisher_draft.extended_fields
except:
pass
if extended_fields:
for field in extended_fields._meta.fields:
request.options[field.name] = getattr(extended_fields, field.name)
return None
will allow you to simply do {{ request.options.page_image }}