I have a demo Wagtail website. A site is generated via a Cookiecutter. To populate the CMS with initial content I have added a load_initial_data command that can be run once Wagtail is installed. This populates text content from a fixtures JSON file, and moves images from the fixtures folder to the Wagtail site's media_root folder. The code looks like:
# load_initial_data.py
import os, shutil
from django.conf import settings
from django.core.management.base import BaseCommand
from django.core.management import call_command
class Command(BaseCommand):
def handle(self, **options):
fixtures_dir = os.path.join(settings.PROJECT_DIR, 'fixtures')
fixture_file = os.path.join(fixtures_dir, 'db.json')
image_src_dir = os.path.join(fixtures_dir, 'images')
image_dest_dir = os.path.join(settings.MEDIA_ROOT, 'original_images')
call_command('loaddata', fixture_file, verbosity=0)
if not os.path.isdir(image_dest_dir):
os.makedirs(image_dest_dir)
for filename in os.listdir(image_src_dir):
shutil.copy(os.path.join(image_src_dir, filename), image_dest_dir)
This works to the extent that the images are copied to the correct directory, and on the templates the images appear as expected when requested. The problem is within /admin/images/ where the requested version of the image is unavailable, and so the browser shows a broken image icon.
The admin page is looking for a specific size of the image ({your-image-name}.max-165x165.{.jpg|.png|.gif}.
Watching how images move from original_images to images makes it appear that they are only processed after the template they are on is first requested. One idea then might be to create a template listing all the images (with the correct styling) to process them after the data has been loaded in. However doing something like
{% image page.image max-165x165 as test_photo %}
<img src="{{ test_photo.url }}" width="{{ test_photo.width }}" height="{{ test_photo.height }}" alt="{{ test_photo.alt }}" />
Still returns a broken image, and doesn't process the image from the original_images folder to the images folder as I'd have expected. I tried this after the initial data load, and am presuming it's because the image size needs to have a reference within both the database and template?
Is there a way to programmatically force Wagtail to reprocess all the images to generate the size and file name that the image admin page is looking for?
(To mention quickly, if it's relevant, that the images currently sit within the project repo, but will ultimately be a zip file stored on a cloud store and only be imported to the project once requested. Currently, regardless of whether the user wants them or not, the images are included with the Cookiecutter)
Whenever a template (either front-end or within the admin) requires an image at a particular size, it will look in the wagtailimages.Rendition model (or the project-specific Rendition model, if custom image models are in use) to see if one has previously been generated. If so, it will use the existing file; if not, it will generate a new one and add a Rendition record.
If you're getting a broken image, it most likely means that a Rendition record exists (because it's been included in your initial data fixture) but the corresponding image file isn't present in MEDIA_ROOT/images. The proper fix would be to remove the rendition records from your fixture. To fix this after the fact and force all image renditions to be recreated, you can simply delete the contents of the wagtailimages_rendition table.
Related
I try to build my first application with Django, and use django-storage to store my media on dropbox:
DEFAULT_FILE_STORAGE = 'storages.backends.dropbox.DropBoxStorage'
DROPBOX_OAUTH2_TOKEN = ---- mytoken ----
DROPBOX_ROOT_PATH = 'media'
It works well to upload and display the images using the models directly (e.g. {{ object.photo.url }} in the html template works)
However, how can I create a link to the image in the template if I have no object.
Something like
<img src="{{media_url}}myimage.jpg">
does not work as it links to localhost (on my local server). Is it needed to redefine media_url or is there another tag to use?
I am not sure about the existence of any tag to use in the case of accessing the image from dropbox. I would solve this issue using custom filters in Django since it is the case of distinguishing between local and remote and forming the right URL to pass to the src attribute.
You can read from the docs here on how custom filter works. Essentially, you would need a custom function that prepends the remote/local URL.
Using a function like this for e.g.
#register.filter
#stringfilter
def prepend(str):
url = "" #defien your remote url here
return "{}{}".format(url, str)
You then make calls from the template to this function which would give you the correct url (as you defined it).
I'm working on a Django app which parses xlsx input, processes series of REST API queries and returns the result of the queries as a table in one of my django templates. The HTML code containing the results table is generated with Pandas.to_html() functionality. The HTML code is stored in a variable ("table") and passed to the html template, where it is displayed as {{ table | safe }}. This mechanism works just fine. However, I'm now struggling to add a button which would generate a pdf file to be downloaded by the user.
NOTE: I'm aware it would probably make more sense to use JS to render the PDF on the client side, but at the moment the point is to avoid doing so.
Upon some research, I decided to go with the django-easy-pdf library. I based my solution on the example included in the documentation, but so far to no avail.
In urls.py:
urlpatterns = [
[...]
path('result.pdf', views.PDFView.as_view(), name='PDFview'),
]
In views.py:
class PDFView(PDFTemplateView):
template_name = 'whitelist/listresult.html'
table = None
def get_context_data(self, **kwargs):
return super(PDFView, self).get_context_data(pagesize='A4', title='Hi there!', table=self.table, date=date.today)
Please note that the template "listresult.html" is the one expecting the {{ table }} variable.
Last but not least, in the very listresult.html (where I want to place the button to render the PDF file), I added a simple button object:
<a class="btn btn-secondary btn-lg" href="{% url 'PDFview' table %}">Download PDF</a>
My expectation was that since I'm sending the "table" variable along with the request for PDF url, the view would process it nicely and prompt the user with a download pop-up for the PDF file looking exactly (or almost exactly) as the page which presented the result in the first place. However, I'm cought in a vicious circle where if I add "table" to the url reference on the site I get an error during template rendering for the very website displaying results (NoReverseMatch for ..., Reverse for 'PDFview' with arguments '['table' contents go here] not found). On the other hand, if I remove the "table" argument from the url reference, the results website renders OK with the "Download PDF" button, but upon clicking it I'm left with a Runtime Error (since context is missing).
I'm 97% confident my mistake is ostentatiously stupid, but after x hours of struggle I'm ready for StackOverflow's judgement.
I am trying to use thumbnail packages to generate thumbnail images from base images. Originally, I had my source images in my static dir, and as the thumbnail packages want to generate them to my media dir, I thought that was the cause of the SuspiciousFileOperation error I was getting.
No problem, I just copied my images to my media dir, which I thought would solve the issue, but no, the issue remains.
From what I can tell, it seems to be a problem with having a relative path vs a full path?
The full error is:
SuspiciousFileOperation at /toys/
The joined path (/media/images/test.jpg) is located outside of the base path component (/home/username/django/first_webapp/my_site/media)
The path /home/username/django/first_webapp/my_site/media/images/test.jpg is valid and test.jpg is a valid jpg image.
The abridged code I am using in my template, with sorl-thumbnail (although I have also tried with easy_thumbnails) is:
{% for instance in prods %}
<img src=" {% thumbnail instance.image_url 300x300 %} ">
{% endfor %}
instance.image_url, in this case, is set to /media/images/test.jpg
My media directory settings from my settings.py
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
MEDIA_URL = '/media/'
I am unsure where to begin to troubleshoot this.
I can't really understand how you think the type of the field is not relevant. Of course it is.
In your case, you have (for some reason) used a TextField to store the path of your image file. TextFields have no special knowledge of media files, and sort-thumbnail just treats the contents as a path component, which it then joins with MEDIA_ROOT. But since your path begins with a leading slash, the result of os.path.join(MEDIA_ROOT, path) is just path; the leading slash means exactly "start from the filesystem root". So the result is a path outside your project, which Django disallows for security reasons.
A quick fix is to remove the leading slash - as well as the duplicate "media" prefix - and just store "images/test.jpg". But the real fix is to use the appropriate field for the content you are storing, and let that field manage it for you.
For anyone who lands here off Google because upgrading their Django and easy-thumbnails caused the same error to start happening, check that your template is referencing your ImageField as {{image_field|thumbnail_url:'default'}}, not {{image_field.url|thumbnail_url:'default'}}.
Apparently it got less forgiving of that mistake.
I'm facing difficulty to download (the already uploaded) files as I am a newbie in django. Also I could not find a way to do that (the uploaded files are of different formats). May someone post a minimal but complete (Model, View, Template) example code to do so?
So if you alredy have model like:
class MyFile(models.Model):
file_field = models.FileField()
In your view add your uploaded file in template context like:
{'myfile': MyFile.objects.get(id=1)}
In your template just use <a> tag with:
href="{{ myfile.file_field.url }}"
Having issues with SORL Thumbnail and deleting thumbnails files or refreshing thumbnails when a file is overwritten. The scenario is that I have a file that for each entry is always the same but can be overwritten. Need the thumbnail to be recreated when a new file is uploaded and the old file is overwritten.
This is at the model + form level so I'm using the low level API to generate thumbs.
Have tried using:
from sorl.thumbnail import delete
delete(filename)
But with no success, the thumbnail is never deleted or overwritten.
I have even tried:
from sorl.thumbnail.images import ImageFile
from sorl.thumbnail import default
image_file = ImageFile(filename)
default.kvstore.delete_thumbnails(image_file)
Again with no success.
Please help!
Update:
I found a work around by creating an alternate ThumbnailBackend and a new _get_thumbnail_filename method. The new method uses a file's SHA-1 hash to always have a thumbnail specific to the current file.
Here's the backend for anyone else that might encounter a similar scenario.
class HashThumbnailBackend(ThumbnailBackend):
def _get_thumbnail_filename(self, source, geometry_string, options):
"""
Computes the destination filename.
"""
import hashlib
# hash object
hash = hashlib.sha1()
# open file and read it in as chunks to save memory
f = source.storage.open(u'%s' % source, 'rb')
while True:
chunk = f.read(128)
if not chunk:
break
hash.update(hashlib.sha1(chunk).hexdigest())
# close file
f.close()
hash.update(geometry_string)
hash.update(serialize(options))
key = hash.hexdigest()
# make some subdirs
path = '%s/%s/%s' % (key[:2], key[2:4], key)
return '%s%s.%s' % (settings.THUMBNAIL_PREFIX, path,
self.extensions[options['format']])
Its a little hard to explain so I made this awesome table. the first column's commands are listed below, the other columns marks wheter it deletes using an X. Original is the original file, thumbnails the thumbnails for the original and KV means the Key Value store reference.
| Command | Original | Thumbnails | KV Original | KV Thumbnails |
| #1 | X | X | X | X |
| #2 | | X | | X |
| #3 | | X | X | X |
sorl.thumbnail.delete(filename)
sorl.thumbnail.default.kvstore.delete_thumbnails(image_file)
sorl.thumbnail.delete(filename, delete_file=False)
As I understand it you really want to do #3. Now, your problem... a guess is that filename does not refer to a filename relative to MEDIA_ROOT (if you are using another storage backend the situation would be similar). But I think I need to know what you are doing besides this to get a better picture, note that ImageFields and FileFields do not overwrite, also note that django changed the deletion behaviour in 1.2.5, see release notes.
Update:
Anyone reading this should note that the above way to generate thumbnail filenames is extremely inefficient, please do not use if you care anything at about performance.
I'm not completely sure whether this answers your question, but I was having the same problem and this was my solution.
I have a model with a FileField on it, like such:
material = models.FileField(upload_to='materials')
When handling an uploaded file, I use get_thumbnail() to generate the thumbnail, passing the FileField in as the parameter vs the python level file behind it. ie:
thumb = get_thumbnail(modelinstance.material, '%dx%d' % (thumb_width, thumb_height))
As with your issue, I also found that when a file had the same name, sorl would just grab the thumbnail from the cache instead of generating a new one. Aggravating!
What worked was using sorl's delete method and passing the FileField. I first tried passing in the python file behind the FileField object, which is possibly what you were trying? Going from this:
sorl.thumbnail.delete(modelinstance.material.file)
To this:
sorl.thumbnail.delete(modelinstance.material)
Seemed to line up with sorl-thumbnail's KV Store, and would properly get the cached thumbnail out of the way so the new one could be created from the new file. Yay!
This was helpful for me: http://sorl-thumbnail.readthedocs.org/en/latest/operation.html
Also, even after running ./manage.py thumbnail cleanup and ./manage.py thumbnail clear, I couldn't get Django to stop looking for the old thumbnails in the same place. I had to manually clear the Django cache (I'm using memcached). Here's how you can do that:
import os
# Set the DJANGO_SETTINGS_MODULE environment variable.
os.environ['DJANGO_SETTINGS_MODULE'] = "yourproject.settings"
from django.core.cache import cache
# Flush!
cache._cache.flush_all()
This is my first SO answer. Hope it helps someone :)
The thing is you cannot use the shortcut delete(file) with a File class that is different to the one you employed to generate that very thumbnail through get_thumbnail() or the {% thumbnail ...%} templatetag.
The reason is ImageFile instances constructed from the file objects will get differents keys (ImageFile.key) and delete() will never be able to retrieve the good thumbnails to remove because the keys don't match.
I'm not sure that it won't works if you use a Python File object and then a Django File object for instance, but in Django, if you generate the thumbnail with a FileField object and try to delete it (and its thumbnails) with a File instance, it will not works for sure.
So, in your templates, don't do:
{% load thumbnail %}
{% thumbnail image "100" as im %}
<img src="{{ im.url }}" width="{{ im.width }}" height="{{ im.height }}">
{% endthumbnail %}
Where image is a models.ImageField instance, but use its file attribute:
{% load thumbnail %}
{% thumbnail image.file "100" as im %}
And to delete it in your Python code (the following is an example of Storage to overwrite the existing file if the name is the same):
from django.core.files.storage import FileSystemStorage
from django.core.files import File
from sorl.thumbnail import delete
class OverwriteStorage(FileSystemStorage):
def _save(self, name, content):
if self.exists(name):
img = File(open(os.path.join(self.location, name), "w"))
delete(img)
return super(OverwriteStorage, self)._save(name, content)
Not sure if its a bug in sorl or if there is a good reason to generate different keys.
I saw this problem. It was happening because Sorl was being used oddly.
All the thumbnail were got in the following style:
sorl.thumbnail.get_thumbnail(self.picture.url, geometry_string, **options)
# picture being a FieldFile
And when deleting the thumbnail (removing them from cache) it was being done like this:
sorl.thumbnail.delete(self.picture.name, delete_files=False)
Shortly, we were using the image's URL to generate and fetch the thumbnails, and when deleting we were using the image's name. Although Sorl didn't complain about it, the KV Store and the FS weren't never cleaned up.
The fix was to just change the get_thumbnail name argument to self.picture.name.