I am using Cloudinary in my Django project to upload and store my images.
Something like this:
class MyModel(models.Model):
logo = CloudinaryField('Logo', blank=True, null=True)
In my serializers, if I call something like:
mymodel = MyModel.objects.get(pk=1)
return mymodel.logo.url
What is being returned is a cloudinary url but only with http. How do I fix this? How do I get https?
The Cloudinary response holds both url (HTTP) and secure_url (HTTPS).
Please try:
return mymodel.logo.secure_url
Instead of
return mymodel.logo.url
Set secure in settings:
cloudinary.config(cloud_name = "", api_key = "", api_secret = "", secure = True)
For anyone that's getting errors with secure_url, modifying the url_options dictionary did the trick for me:
mymodel = MyModel.objects.get(pk=1)
# Adding 'secure' to url_options
# Cloudinary source code seems to look for this key when building urls
mymodel.logo.url_options.update({'secure':True})
return mymodel.logo.url
Related
Whenever I POST to my django-rest-framework (DRF) endpoints, I keep receiving a "HTTP 400 Bad Request" {"offeror_organization":["This field is required."]} response. But, given the curl example below, I'm clearly specifying a value.
This happens regardless of the Content-Type (application/json, application/x-www-form-urlencoded, multipart/form-data). The only time it works is when I submit using the "HTML form" (vs. the "Raw Data") tab on the DRF web interface.
There's a few similar SO posts (like this and this), but none of the solutions seem to be working for me.
Model:
class OrganizationManager(models.Manager):
def get_by_natural_key(self, offeror_organization):
return self.get(offeror_organization=offeror_organization)
class Organization(models.Model):
idorganization = models.AutoField(primary_key=True)
offeror_organization = models.CharField(max_length=250, null=False, blank=False, verbose_name='Offeror Organization')
created_at = models.DateTimeField(auto_now_add=True, null=False)
updated_at = models.DateTimeField(auto_now=True, null=False)
objects = OrganizationManager()
def natural_key(self):
return "%s" % (self.offeror_organization)
def __str__(self):
return self.offeror_organization
Serializer:
class OrganizationSerializer(serializers.HyperlinkedModelSerializer):
class Meta:
model = Organization
fields = ['offeror_organization']
# I've tried both with and without a create function
def create(self, validated_data):
organization_data = validated_data.pop('offeror_organization', None)
if organization_data:
organization = Organization.objects.get_or_create(**organization_data)[0]
validated_data['offeror_organization'] = organization
views/api.py:
from webapp.models import Organization
from webapp.serializers import OrganizationSerializer
from rest_framework import viewsets
class OrganizationViewSet(viewsets.ModelViewSet):
queryset = Organization.objects.all().order_by('offeror_organization')
serializer_class = OrganizationSerializer
urls.py:
from django.urls import include, path
from rest_framework import routers
from . import views
router = routers.DefaultRouter()
router.register(r'organization', views.OrganizationViewSet)
urlpatterns = [
...
path('api/', include(router.urls)),
path('api-auth/', include('rest_framework.urls', namespace='rest_framework')),
]
Curl Command:
curl -X POST -H 'Content-Type: application/json' -d '{"offeror_organization":"Test2"}' 10.101.10.228:29000/webapp/api/organization/
settings.py MIDDLEWARE:
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.RemoteUserMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'csp.middleware.CSPMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware'
]
settings.py REST_FRAMEWORK
# currently have all API authentication disabled while troubleshooting this issue
REST_FRAMEWORK = {
'DEFAULT_AUTHENTICATION_CLASSES': [],
'DEFAULT_PERMISSION_CLASSES': [],
}
In my case, fixing this issue required "maneuvering" around a few different implementation constraints.
nginx + uWSGI Socket + Django REMOTE_USER Authentication:
As mentioned in this post's comments/chat, I've got both an nginx proxy and a uWSGI application server fronting my Django application. Since I'm relying upon REMOTE_USER Authentication, my uwsgi/nginx configuration must use uWSGI sockets (vs. http) so that I may pass the REMOTE_USER from nginx to Django as an environment variable. When using http (coupled w/ nginx proxy_pass), although proxy_pass can set headers or cookies, those seemingly cannot translate over to Django (which requires the environment variable).
I think there's a few issues at play when trying to POST to a Django/DRF application served using uWSGI sockets. Per the uWSGI Things to know (best practices), "TL/DR: if you plan to expose uWSGI directly to the public, use --http, if you want to proxy it behind a webserver speaking http with backends, use --http-socket". In my case, having both a web application and a DRF-based API (that I want other services and systems to talk to), I need both! As a (hopefully temporary) workaround, I'm currently spawning two uWSGI processes - one using --socket, and one using --http (for API POST calls). If you POST while using ---socket, you'll likely get an Empty Response error from DRF.
As an aside, I initially saw some "promise" in utilizing uwsgi_curl (from uwsgi_tools) to POST over the uWSGI socket (which resulted in the "field is required" error (vs. the Empty Response error), but that's when I started to run into my second issue...
POST nested application/json w/ simultaneous file upload: The "Organization" model referenced in the post was mostly proof-of-concept, as it's the least complex model in my Django application. In reality, I need to post to a more complex model with nested serialization, as the model contains Foreign Key's to other models. But that's totally do-able with DRF. Except in my case, where one of my model attributes is a FileUpload field. As noted in other SO Questions (like this one), there's also a few issues in trying to POST nested (i.e. not "flat") application/json with a file upload in a single request. While I was never able to fully understand the issue at play (at least using drf_writable_nested.serializers.WritableNestedModelSerializer in my case), I simplified the problem at-hand by writing my own custom Serializer (serializers.Serializer), this way I could avoid nested JSON objects (like { "offeror_organization": {"offeror_organization: "Test"}} in my POST requests. This fixed my issue.
With the custom serializer in place to mitigate the nested JSON + file upload issue, I bet the uwsgi_curl POST would work. Although then external client systems/services are limited to using that Python package. Anyways, I'll update my answer once I try it out. Thanks to #Michael for his comments and helping to lead me down the right "road".
I have the same setup (nginx + gunicorn + django + rest-framework + drf-writeable-nested) but I could figure out a valid format for the POST request containing multipart/form-data:
It needs to be like this:
json:
{
'name': 'test-name',
'files': [
{
'file': 'test-file-1'
},
{
'file': 'test-file-2'
},
...
],
...
}
must be formatted to:
FormData:
name: test-name
files[0]file: test-file-1
files[1]file: test-file-2
...
Some libraries would use a dot after the brackets for nested lists, which would lead to the This field is required error. Or even another bracket after the list bracket would lead to the same error.
This is wrong:
files[0].file
This is also wrong:
files[0][file]
My example assumes the following Django-classes:
# views.py
from rest_framework.viewsets import ModelViewSet
from rest_framework.parsers import MultiPartParser, FormParser
from .serializers import YourCustomModelSerializer
class YourCustomModelViewSet(ModelViewSet):
queryset = YourCustomModel.objects.all()
parser_classes = [FormParser, MultiPartParser]
permission_classes = []
serializer_class = YourCustomModelSerializer
# serializers.py
from rest_framework.serializers import ModelSerializer
from drf_writable_nested.serializers import WritableNestedModelSerializer
from .models import YourCustomModel, File
class FileSerializer(ModelSerializer):
class Meta:
model = File
fields = ['file']
class YourCustomModelSerializer(WritableNestedModelSerializer):
# Reverse foreign key
files = FileSerializer(many=True)
class Meta:
model = YourCustomModel
read_only_fields = ('id', )
fields = [
'id',
'name',
'files'
]
# models.py
from django.db import models
class File(models.Model):
file = models.FileField()
class YourCustomModel(models.Model):
name = models.CharField(max_length=200)
I used the following javascript/typescript frontend code to pack my json data into a FormData request:
const requestBody = {
name: 'test-name',
files: [
{ file: file1 }, // File object
{ file: file2 }, // File object
{ file: file3 } // File object
]
}
// # use your own code to serialize the above javascript dict/json/object
// into form-data object (I used the library https://www.npmjs.com/package/object-to-formdata but I had to write a patch for the exact reason described above: files[0].file is not correctly parsed by Django and files[0][file] also won't work, therefore I slightly changed the library code so that it will format my object to files[0]file for every nested item:
// 2. prepare a serializer-library to convert the dict into a special form which can be parsed by django.
const options = {
indices: true,
allowEmptyArrays: true,
useDotOrBracketsOrNothingForNestedObjects: 2 // Option 0: DOT-Notation, 1: Brackets, 2: Nothing (this option is from my custom patch)
}
// use npx patch: https://stackoverflow.com/a/62567504/3433137
// (I patched this serialize library and the patch is somewhere stored as a file in this project)
const formData = serialize(
requestBody,
options
)
// 3. upload the data
api.post(url, formData)
I have created a model that uploads img to cloudinary, however, shows a wrong URL in the Django template which has 2 'https://' parts inside of it. Please help.
models.py:
class Product(models.Model):
img = models.ImageField(upload_to='product', height_field=None, width_field=None, max_length=100,
default='https://res.cloudinary.com/dgdhnbsae/image/upload/vxxxx/product/xxxx.jpg')
def get_img(self):
return f"{'/media/product/'+self.img}"
I have set the related configuration according to the tutorial
setting.py:
INSTALLED_APPS = [
xxxx
'cloudinary_storage',
'cloudinary',
]
CLOUDINARY_STORAGE = {
'CLOUD_NAME': 'xxxx',
'API_KEY': 'xxxxx',
'API_SECRET': 'xxxxx'
}
DEFAULT_FILE_STORAGE = 'cloudinary_storage.storage.MediaCloudinaryStorage'
Template:
<div class="featured__item__pic set-bg" data-setbg="{{ product.img.url }}">
The output:
https://res.cloudinary.com/xxxx/image/upload/v1/media/https://res.cloudinary.com/vxxxx/image/upload/xxxxx/xxxxx.jpg
So if I understood your problem correctly,
youll need to do the below changes
in your model.py file, add below
from cloudinary.models import CloudinaryField
instead of using the built in function for the image upload, you'd want to use the cloudinary function.
eg:
img = CloudinaryField('attributes')
you can later call this in views as usual and then pass it on to your django template. or you can use cloudinary template call, which will allow you to parse additional image manipulation options into the url string.
Some references:
https://jszczerbinski.medium.com/django-web-app-and-images-cloudinary-straightforward-study-ae8b5bb03e37
an easier reference:
https://alphacoder.xyz/image-upload-with-django-and-cloudinary/
I am using django-rest-framework in conjuntion with django-ckeditor. I'm serving some images with absolute url-s without any problem. But images and files uploaded by ckeditor are served as relative paths, and they can't be displayed client side since it is in a different domain.
Here is an example of what I'm getting:
{
image: "http://example.com/media/myimage.png",
body: "<p>download my file</p>"
}
And this is what I woul like to get:
{
image: "http://example.com/media/myimage.png",
body: "<p>download my file</p>"
}
Edit:
This would be the model of my example:
from django.db import models
from ckeditor_uploader.fields import RichTextUploadingField
image: models.ImageField()
body: RichTextUploadingField(blank=True,null=True)
I would use a custom serializer to fix that:
from rest_framework import serializers
def relative_to_absolute(url):
return 'http://127.0.0.1:8000' + url
class FileFieldSerializer(serializers.Field):
def to_representation(self, value):
url = value.url
if url and url.startswith('/'):
url = relative_to_absolute(url)
return url
When filefield.url contains a relative url, relative_to_absolute() is called to prepend the domain.
Here I just used a constant string; you can either save it in your settings, or, if Django Site framework is installed, retrieve it as follows:
from django.contrib.sites.models import Site
domain=Site.objects.get_current().domain
Sample usage of the custom serializer:
class Picture(BaseModel):
...
image = models.ImageField(_('Image'), null=True, blank=True)
...
class PictureSerializer(serializers.ModelSerializer):
image = FileFieldSerializer()
class Meta:
model = Picture
fields = '__all__'
Variation for RichTextUploadingField
If, on the other hand, you're using RichTextUploadingField by CKEditor, your field is, basically, a TextField where an HTML fragment is saved upon images
upload.
In this HTML fragment, CKEditor will reference the uploaded images with a relative path, for very good reasons:
your site will still work if the domain is changed
the development instance will work in localhost
after all, we're using Django, not WordPress ;)
So, I wouldn't touch it, and fix the path at runtime in a custom serializer instead:
SEARCH_PATTERN = 'href=\\"/media/ckeditor/'
SITE_DOMAIN = "http://127.0.0.1:8000"
REPLACE_WITH = 'href=\\"%s/media/ckeditor/' % SITE_DOMAIN
class FixAbsolutePathSerializer(serializers.Field):
def to_representation(self, value):
text = value.replace(SEARCH_PATTERN, REPLACE_WITH)
return text
Alternatively, domain can be saved in settings:
from django.conf import settings
REPLACE_WITH = 'href=\\"%s/media/ckeditor/' % settings.SITE_DOMAIN
or retrieved from Django Site framework as follows:
from django.contrib.sites.models import Site
REPLACE_WITH = 'href=\\"{scheme}{domain}/media/ckeditor/'.format(
scheme="http://",
domain=Site.objects.get_current().domain
)
You might need to adjust SEARCH_PATTERN according to your CKEditor configuration; the more specific, the better.
Sample usage:
class Picture(BaseModel):
...
body = RichTextUploadingField(blank=True,null=True)
...
class PictureSerializer(serializers.ModelSerializer):
body = FixAbsolutePathSerializer()
class Meta:
model = Picture
fields = '__all__'
Title is a bit confusing, but basically I have an s3 path stored as a string
class S3Stuff(Model):
s3_path = CharField(max_length=255, blank=True, null=True)
# rest is not important
There are existing methods to download the content given the url, so I want to utilize that
def download_from_s3(bucket, file_name):
s3_client = boto3.client(bleh_bleh)
s3_response = s3_client.get_object(Bucket=s3_bucket, Key=file_name)
return {'response': 200, 'body': s3_response['Body'].read()}
s3_path can be broken into bucket and file_name. This works very easily when I use my own frontend because I can do whatever I want with it, but I don't know how to apply this to admin
class S3StuffAdmin(admin.StackedInline):
model = S3Stuff
fields = ('s3_path', )
Now how do I call that method and make the display a link that says "download"
I don't think this function will be much useful for generating download links, instead use the boto3's presigned_url like this:
from django.utils.html import format_html
class S3StuffAdmin(admin.StackedInline):
model = S3Stuff
fields = ('s3_path', )
readonly_field = ('download',)
def download(self, obj):
s3_client = boto3.client(bleh_bleh)
url = s3_client.generate_presigned_url('get_object', Params = {'Bucket': 'bucket', 'Key': obj.s3_path}, ExpiresIn = 100)
return format_html('<a href={}>download</a>'.format(url))
My application has a tastypie endpoint that received post requests with a file(not big...40-70kb) and is supposed to save the file and then process it. However, it seems the file never gets saved after upload and thus trying to retrieve the file after save fails. I have read in many places about it but it seems tastypie has no one standard way of doing it. Here are my models and resources:
models
class Tag(models.Model):
track = models.FileField(upload_to='tags/', max_length=250)
def __unicode__(self):
return self.track.url
api_resource
class TagResource(MultipartResource, ModelResource):
track = fields.FileField(attribute="track", null=True, blank=True)
class Meta:
queryset = Tag.objects.all()
resource_name = 'tag'
authorization = Authorization()
object_class = Tag
always_return_data = True
def dehydrate(self, bundle):
bundle.obj.save()
#the processing operation on the saved file
result = recognize(bundle.obj.track)
bundle.data['tag'] = result
return bundle
When I post a file with Curl:
curl -F "track=/path/to/track/track.mp3" http://127.0.0.1:8000/api/tag/
I get an error:
SuspiciousFileOperation: Attempted access to '/path/to/track/track.mp3'
and upon further investigation, i realized the error is being caused by the intended operation working on the url of the source file instead of the uploaded file's url.
What i'm I doing wrong??
I finally got it to work. Seems like I was not posting correctly using CURL. I now use this to upload the file:
curl -F "track=#/path/to/file.mp3" http://1**.**.***.**:8000/api/tag/
The '#' before the path was missing before. Adding that fixed it.