I implemented a file upload endpoint with DRF the issue is the documents do not show the absolute url of the file as shown below on the screenshot. I expect the absolute url start with http://localhost ....
Here is my django settings
STATIC_URL = '/static/'
# The folder hosting the files
STATICFILES_DIRS = [os.path.join(BASE_DIR, 'static'),]
## Serving the STATIC FILES
# As declared in NginX conf, it must match /src/static/
STATIC_ROOT = os.path.join(os.path.dirname(os.path.dirname(BASE_DIR)), 'static')
MEDIA_URL = '/media/'
# do the same for media files, it must match /opt/services/djangoapp/media/
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
models.py
class Document(models.Model):
"""This represents document class model."""
file = models.FileField(upload_to='documents/inspections/%Y/%m/%d')
timestamp = models.DateTimeField(auto_now_add=True)
#property
def name(self):
name = self.file.name[33:]
return name
Django doesn't provide absoluteurl for image stored in models.ImageField. But you can modify your serializer and do this with serializer.SerializerMethodField. as you didn't provide your serializer so i assume it maybe look like this:
class Document(seralizer.ModelSerializer):
file_url = serializer.SerializerMethodField()
class Meta:
model = Document
fields = ('id','name','timestamp','file_url')
def get_file_url(self, document):
request = self.context.get('request')
file_url = document.file.url
return request.build_absolute_uri(file_url)
Django rest framework, by default, builds absolute URLs for file field if you pass request in serializer context. Try to pass the request in serializer like this.
serializer = YourSerializer(<OTHER_ARGS>, context={'request': request})
You will get the absolute URL automatically.
To make things easier I used cloudinary to host my images since eventually I will store them in the cloud in production.
Therefore:
Installed required cloudinary python packages i.e cloudinary and dj3-cloudinary-storage
Added cloudinary in installed apps:
INSTALLED_APPS = [
......
'rest_framework',
'django_extensions',
'corsheaders',
'cloudinary', # Add cloudinary
'drf_yasg',
'celery',
'django_prometheus',
'debug_toolbar',
]
Changed the default file storage in settings.py and added cloudinary configs:
CLOUDINARY_STORAGE = {
'CLOUD_NAME': '*******',
'API_KEY': '*********',
'API_SECRET': '*********',
}
DEFAULT_FILE_STORAGE = 'cloudinary_storage.storage.MediaCloudinaryStorage'
Removed below don't need it anymore in root urls.py
urlpatterns+= static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
After testing an upload here are the results:
Related
I'm working on a Django backend deployed on a server, here's my settings:
DEBUG = False
STATIC_URL = "/staticfiles/"
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
MEDIA_URL = "/media/"
MEDIA_ROOT = os.path.join(BASE_DIR, "media")
And I want to save the users' images outside the django folder, so I've created a custom FileSystemStorage in this way:
from django.core.files.storage import FileSystemStorage
key_store = FileSystemStorage(location="/home/usr/project/media/", base_url="/media")
Where I put the absolute path of ubuntu server.
def profile_picture_url(instance, *_):
return f"{instance.user.uuid}/profile.picture.png"
picture = models.FileField(
storage=key_store, default=None, upload_to=profile_picture_url
)
But it doesn't create any file inside media folder.
Any solution?
I have a directory '/media/profile_image' where profile images uploaded by users are saved. In the template I built the url by using the user object
<img id="profile_image" src="{{ request.user.profile_image.url }}"></div>
# Model
def get_profile_image_filepath(self, filename):
return f'profile_image/{self.pk}/{"profile_image.png"}'
...
profile_image = models.ImageField(max_length=255, upload_to=get_profile_image_filepath, null=True, blank=True, default=get_default_profile_image())
...
which creates the correct url to the desired directory. But it doesn't show the image, why's that?
# rendered url
<img id="profile_image" src="/media/profile_image/1/profile_image.png">
settings.py
MEDIA_URL = '/media/'
MEDIA_ROOT = BASE_DIR / 'media'
dir
Dealing with Django and media files on development like this :
First install pillow
pip install pillow on your activated env.
Specify the MEDIA_ROOT variable in the settings file
MEDIA_ROOT = os.path.join(BASE_DIR, 'media/') or MEDIA_ROOT = BASE_DIR / 'media' in Django 3.
Tell Django to serve media file (in development mode)
urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT) in the project urls.py
file.
(Optional) Specify the MEDIA_URL variable too
URL that handles the media served from MEDIA_ROOT.
Exemple :
MEDIA_URL = '/media/'
MEDIA_ROOT = BASE_DIR / 'media' # or os.path.join(BASE_DIR, 'media/') in Django 2
And in your model, you can define a media file like this :
class Profile(models.Model):
# Others fields
photo = models.ImageField(upload_to="photos/")
Here the upload_to is used to designate the location where the images assigned to the photo attribute will be saved on the hard drive for all instances of the model. If you don't specify a value for upload_to, the images will be saved to the root of MEDIA_ROOT.
I get the following error whenever a custom domain for the S3 endpoint is used.
# WORKS
AWS_S3_CUSTOM_DOMAIN = 'example.fra1.digitaloceanspaces.com/{}'.format(AWS_STORAGE_BUCKET_NAME)
# DOES NOT WORK ⁉️
AWS_S3_CUSTOM_DOMAIN = 'cdn.example.com/{}'.format(AWS_STORAGE_BUCKET_NAME)
CommandError: An error occurred during rendering /home/<user>/<app>/templates/public/index.html: 'https://cdn.example.com/storage/static/node_modules/nouislider/distribute/nouislider.min.css' isn't accessible via COMPRESS_URL ('https://example.fra1.digitaloceanspaces.com/storage/static/') and can't be compressed
If I go to either url the file is accessible, hence its likely that the CDN is ok, URLs are correctly defined, CORS are fine too.
Also without django-compressor subdomain delivery had been working fine, leading me to believe the issue is not with django-storages
I've been trying for several hours and ultimately had to do a temporary fix by setting the AWS_S3_CUSTOM_DOMAIN to be the same as the AWS_S3_ENDPOINT_URL. However this is not ideal.
Please see the implementation below.
/requirements.txt
Django==3.1.4
...
boto3~=1.16.46
botocore~=1.19.46
s3transfer~=0.3.3
...
django-storages~=1.11.1
django-compressor~=2.4
/config/settings.py
...
# ==========================================================
# DJANGO-STORAGES
# ==========================================================
if LOCALHOST_MODE:
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
STATIC_URL = '/static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, 'static/'), ]
else:
AWS_ACCESS_KEY_ID = os.getenv("AWS_ACCESS_KEY_ID")
AWS_SECRET_ACCESS_KEY = os.getenv("AWS_SECRET_ACCESS_KEY")
AWS_STORAGE_BUCKET_NAME = 'storage'
AWS_S3_ENDPOINT_URL = 'https://example.fra1.digitaloceanspaces.com'
# WORKS ⚠️
AWS_S3_CUSTOM_DOMAIN = 'example.fra1.digitaloceanspaces.com/{}'.format(AWS_STORAGE_BUCKET_NAME)
# DOES NOT WORK ⁉️
AWS_S3_CUSTOM_DOMAIN = 'cdn.example.com/{}'.format(AWS_STORAGE_BUCKET_NAME)
AWS_S3_OBJECT_PARAMETERS = {
'CacheControl': 'max-age=86400',
}
AWS_LOCATION = 'static'
AWS_DEFAULT_ACL = 'public-read'
STATICFILES_DIRS = (os.path.join(BASE_DIR, "static"), )
STATIC_URL = '{}/{}/{}/'.format(AWS_S3_ENDPOINT_URL, AWS_STORAGE_BUCKET_NAME, AWS_LOCATION)
STATICFILES_STORAGE = 'storage_backends.CachedS3Boto3Storage'
MEDIA_ROOT = os.path.join(BASE_DIR, "media")
MEDIA_URL = '{}/{}/{}/'.format(AWS_S3_ENDPOINT_URL, AWS_STORAGE_BUCKET_NAME, 'media')
DEFAULT_FILE_STORAGE = 'storage_backends.MediaStorage'
# ==========================================================
# DJANGO-COMPRESSOR
# ==========================================================
COMPRESS_ENABLED = True
STATIC_DEPS = True
COMPRESS_ROOT = os.path.join(BASE_DIR, "static")
COMPRESS_FILTERS = {
'css': ['compressor.filters.css_default.CssAbsoluteFilter', 'compressor.filters.cssmin.rCSSMinFilter'],
'js': ['compressor.filters.jsmin.JSMinFilter']
}
if LOCALHOST_MODE:
COMPRESS_OFFLINE = False
else:
COMPRESS_OFFLINE = True
COMPRESS_STORAGE = STATICFILES_STORAGE
COMPRESS_URL = STATIC_URL
STATICFILES_FINDERS = [
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
'compressor.finders.CompressorFinder',
]
...
/storage_backends.py
from django.core.files.storage import get_storage_class
from storages.backends.s3boto3 import S3Boto3Storage
from config import settings
class MediaStorage(S3Boto3Storage):
bucket_name = settings.AWS_STORAGE_BUCKET_NAME
location = 'media'
class CachedS3Boto3Storage(S3Boto3Storage):
def __init__(self, *args, **kwargs):
super(CachedS3Boto3Storage, self).__init__(*args, **kwargs)
self.local_storage = get_storage_class(
"compressor.storage.CompressorFileStorage")()
def save(self, name, content):
self.local_storage._save(name, content)
super(CachedS3Boto3Storage, self).save(name, self.local_storage._open(name))
return name
How to debug this error:
Looking at where the error is raised (in django-compressor.compressor.base) we find the following:
def get_basename(self, url):
try:
base_url = self.storage.base_url
except AttributeError:
base_url = settings.COMPRESS_URL
base_url = str(base_url)
if not url.startswith(base_url):
raise UncompressableFileError(
"'%s' isn't accessible via "
"COMPRESS_URL ('%s') and can't be "
"compressed" % (url, base_url)
)
Looking at your settings:
your COMPRESS_URL is set equal to STATIC_URL,
which in turn is equal to '{}/{}/{}/'.format(AWS_S3_ENDPOINT_URL, AWS_STORAGE_BUCKET_NAME, AWS_LOCATION).
This is https://example.fra1.digitaloceanspaces.com/storage/static/
But the url being passed in is https://cdn.example.com/storage/static/...
This is the route of the problem.
So what's happening??
COMPRESS_URL controls the URL that linked files will be read from and compressed files will be written to.
When you have a CSS or JS file with a link, rel or src attribute somewhere, django-compressor calls get_basename.
get_basename takes full path to a static file (eg. "/static/css/style.css") and
returns path with storage's base url removed (eg. "css/style.css").
Since the base path of some link, rel or src attribute doesn't begin with COMPRESS_URL (which it is expecting to be the base url) it can't remove it and so raises an error.
How to solve this
It's difficult to know exactly what is going wrong without seeing the whole project, however the below might help:
If you updated your AWS_S3_ENDPOINT_URL to match your AWS_S3_CUSTOM_DOMAIN. This would in turn update your STATIC_URL (which I suspect is affecting the filenames in your files), and will also update your COMPRESS_URL. I think this might fix it.
Alternatively, just update COMPRESS_URL to use the correct url (although I think this needs to match STATIC_URL with your current setup).
If any (link, rel or src) urls in files that are being compressed are hard-coded they need to be updated to match your COMPRESS_URL.
Django Compressor has to be configured for remote storage. Section 2.5.1 of the Django Compressor Documentation covers S3 and explains that you need to set
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
COMPRESS_URL = f'https://{AWS_S3_CUSTOM_DOMAIN}/'
and
COMPRESS_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
in settings.py. Before I could get my app working, I still had to debug further such that I had to run collectstatic locally so that my_project/static matched my S3 storage. Then, I had to set
STATICFILES_LOCATION = ''
The latter two bugs may be unrelated issues, but I thought I should mention them.
I have created model - Profile - which presents user's profile. I have one field which is models.ImageFiled and I have method to get absolute url to this image. I have server in development role, so I've exposed /media folder.
When I use full server URL it works. Is there any method to avoid put static web server address?
# Profile
class Profile(models.Model):
user = models.OneToOneField(settings.AUTH_USER_MODEL)
date_of_birth = models.DateField(blank=True, null=True)
photo = models.ImageField(upload_to='users/%Y/%m/%d',blank=True)
def __str__(self):
return 'Profile for user {}'.format(self.user.username)
#property
def get_absolute_image_url(self):
return "http://127.0.0.1:8000/{0}".format(self.photo.url)
in settings.py
MEDIA_URL = 'media/'
MEDIA_ROOT = os.path.join(BASE_DIR,'media/')
You need an absolute path with a leading slash:
MEDIA_URL = '/media/'
After some digging I found the solution.
It based on two things:
1) Media url has to be inside static directory to available to display.
2) Uploaded media has to be inside static directory.
My settings.py looks like below:
STATIC_URL = '/static/'
MEDIA_URL = '/static/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, "shop", "static", "media")
A help came from here [a link] Differences between STATICFILES_DIR, STATIC_ROOT and MEDIA_ROOT
model.py:
class Album(models.Model):{
poster = models.ImageField upload_to='static/images/album/%Y/%m/%d')
}
serializer.py
class AlbumSerializer(serializers.ModelSerializer):
doc = DoctorSerializer()
class Meta:
model = Album
fields = '__all__'
setting.py
STATIC_URL = '/static/'
STATICFILES_DIRS=[
os.path.join(BASE_DIR,'static')
]
views.py
class index(viewsets.ModelViewSet):
serializer_class = AlbumSerializer
queryset = Album.objects.all()
When I upload the image under http://127.0.0.1:8008/admin/qa/album/, the image show as 'static/images/album/2018/06/27/xxxxxx.jpg' at the poster field in the database. And I can access to the image through http://127.0.0.1:8008/static/images/album/2018/06/27/xxxxxx.jpg.
However, in the index view, the Django Rest Framework API return the url of the image as:
http://127.0.0.1:8008/api/index/static/images/album/2018/06/27/xxxxxx.jpg, which make the image 404.
Why the /api/index/ has been added to the url? What's wrong with my setting? Need your help...
To propely serve media files you need to add MEDIA_URL and MEDIA_ROOT to your settings also:
MEDIA_URL = '/static/'
MEDIA_ROOT = os.path.join(BASE_DIR,'static')
Note if you set static directory with MEDIA_ROOT setting in upload_to path you can skip static:
poster = models.ImageField(upload_to='images/album/%Y/%m/%d')
In urls.py:
from django.conf import settings
urlpatterns = [
# your urls here
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)
UPD
As #brunodesthuilliers said in comment, you'd better divide media and static files and use media url and directory instead of static:
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR,'medial')