Pillow png compressing - django

Im making a simple app that able to compress images with jpeg and png format using Pillow library, python3 and Django. Made a simple view that able to identify formats, save compress images and give some statistics of compressing. With images in jpeg format it works really fine, i got compressicons close to 70-80% of original size, and it works really fast, but if i upload png i works much worse. Compression takes a long time, and it only 3-5% of original size. Trying to find some ways to upgrade compress script, and stuck on it.
Right now ive got this script in my compress django view:
from django.shortcuts import render, redirect, get_object_or_404, reverse
from django.contrib.auth import login, authenticate, logout
from django.contrib.auth.models import User
from django.http import HttpResponse, HttpResponseRedirect
from django.http import JsonResponse
from django.contrib import auth
from .forms import InputForm, SignUpForm, LoginForm, FTPForm
import os
import sys
from PIL import Image
from .models import image, imagenew, FTPinput
from django.views import View
import datetime
from django.utils import timezone
import piexif
class BasicUploadView(View):
def get(self, request):
return render(self.request, 'main/index.html', {})
def post(self, request):
form = InputForm(self.request.POST, self.request.FILES)
if form.is_valid():
photo = form.save(commit=False)
photo.name = photo.image.name
photo.delete_time = timezone.now() + datetime.timedelta(hours=1)
photo.user = request.user
photo.size = photo.image.size
photo = form.save()
name = (photo.name).replace(' ', '_')
picture = Image.open(photo.image)
if picture.mode in ('RGB'):
piexif.remove('/home/andrey/sjimalka' + photo.image.url)
picture.save('media/new/'+name,"JPEG",optimize=True,quality=75)
newpic = 'new/'+name
new = imagenew.objects.create(
name = name,
image = newpic,
delete_time = timezone.now() + datetime.timedelta(hours=1),
user = request.user,
)
if new.image.size < photo.image.size:
diff = round((new.image.size-photo.image.size)/float(photo.image.size)*100, 2)
else:
diff = str(round((new.image.size-photo.image.size)/float(photo.image.size)*100, 2))+' Не удалось сжать файл'
oldsize = round(photo.image.size/1000000, 2)
newsize = round(new.image.size/1000000, 2)
id = new.pk
imagenew.objects.filter(pk=id).update(size=new.image.size)
elif picture.mode != ('RGB'):
picture.save('media/new/'+name,"PNG", optimize=True, quality=75)
newpic = 'new/'+name
new = imagenew.objects.create(
name = name,
image = newpic,
delete_time = timezone.now() + datetime.timedelta(hours=1),
user = request.user,
)
if new.image.size < photo.image.size:
diff = round((new.image.size-photo.image.size)/float(photo.image.size)*100, 2)
else:
diff = str(round((new.image.size-photo.image.size)/float(photo.image.size)*100, 2))+' Не удалось сжать файл'
oldsize = round(photo.image.size/1000000, 2)
newsize = round(new.image.size/1000000, 2)
id = new.pk
imagenew.objects.filter(pk=id).update(size=new.image.size)
data = {'is_valid': True, 'name': new.image.name, 'url': new.image.url, 'diff': diff,
'oldsize':oldsize, 'newsize':newsize,}
else:
alert = 'Данный формат не поддерживается. Пожалуйста загрузите картинки форматов png или jpg(jpeg)'
data = {'is_valid': False, 'name': alert,}
return JsonResponse(data)
The question: is there any ways to make script with png upload work faster, and (that much more important) make png size compressions closer to jpeg? Maybe i should use another python library?

how tinypng works then? They compressing same png files with 50-60%
They probably reduce the colour palette from 24-bit to 8-bit. Here's a detailed answer about that - https://stackoverflow.com/a/12146901/1925257
Basic method
You can try that in Pillow like this:
picture_8bit = picture.convert(
mode='P', # use mode='PA' for transparency
palette=Image.ADAPTIVE
)
picture_8bit.save(...) # do as usual
This should work similar to what tinypng does.
If you don't want transparency, it's better to first convert RGBA to RGB and then to P mode:
picture_rgb = picture.convert(mode='RGB') # convert RGBA to RGB
picture_8bit = picture_rgb.convert(mode='P', ...)
Getting better results
Calling convert() as shown above will actually call quantize() in the background and Median Cut algorithm will be used by default for reducing the colour palette.
In some cases, you'll get better results with other algorithms such as MAXCOVERAGE. To use a different algorithm, you can just call the quantize() method directly:
picture_rgb = picture.convert(mode='RGB') # convert RGBA to RGB
picture_8bit = picture.quantize(colors=256, method=Image.MAXCOVERAGE)
You have to understand that downsizing the colour palette means that if the image has lots of colours, you will be losing most of them because 8-bit can only contain 256 colours.

The document of Pillow Image.quatize displays a more convenient way to compress png files. In a personal experiment, the following code could make png about 70% compression of the original size, which is also close to the result created by ImageMagick.
# Image.quantize(colors=256, method=None, kmeans=0, palette=None)
# method: 0 = median cut; 1 = maximum coverage; 2 = fast octree
img = img.quantize(method=2)

Related

How to save pillow processed image in already existing Django object

I have created a object model as below
from django.db import models
# Create your models here.
class ImageModel(models.Model):
image = models.ImageField(upload_to='images/')
editedImg = models.ImageField(upload_to='images/')
def delete(self, *args, **kwargs):
self.image.delete()
self.editedImg.delete()
super().delete(*args, **kwargs)
And here is what i am trying to do in a function
from django.shortcuts import render
from EditorApp.forms import ImageForm
from EditorApp.models import ImageModel
from django.http import HttpResponseRedirect
from PIL import Image
def edit_column(request):
codArr = request.POST.getlist('codArr[]')
imgs = ImageModel.objects.first()
orgImage = ImageModel.objects.first().image
orgImage = Image.open(orgImage)
croppedImg = orgImage.crop((int(codArr[0]), int(codArr[1]), int(codArr[2]), int(codArr[3])))
# croppedImg.show()
# imgs.editedImg = croppedImg
# imgs.save()
return HttpResponseRedirect("/editing/")
What i am trying to do is the codArr consists of coordinates of top(x, y) and bottom(x, y) in the array form(Which is not an issue and is tested(croppedImg.show() showed the desired cropped image) and handled and used to crop the image). Image crop is working fine. But what i am trying to do is to save the cropped image in editedImg of the model used above. The above commented one is what i tried but throw a error AttributeError: _committed
As i have not used any name for image in model as its not required.
Kindly help please, Would be very thankfull.
you should do it like this:
from io import BytesIO
from api.models import ProductPicture
from django.core import files
codArr = request.POST.getlist('codArr[]')
img_obj = ImageModel.objects.first()
orgImage = img_obj.image
orgImage = Image.open(orgImage)
croppedImg = orgImage.crop((int(codArr[0]), int(codArr[1]), int(codArr[2]), int(codArr[3])))
thumb_io = BytesIO() # create a BytesIO object
croppedImg.save(thumb_io, 'png')
editedImg = files.File(thumb_io, name=file_name)
img_obj.editedImg = editedImg
img_obj.save()
You can use Python's context manager to open the image and save it to the desired storage in that case I'm using the images dir.
Pillow will crop the image and image.save() will save it to the filesystem and after that, you can add it to Django's ImageField and save it into the DB.
The context manager takes care of the file opening and closing, Pillow
takes care of the image, and Django takes care of the DB.
from PIL import Image
with Image.open(orgImage) as image:
file_name = image.filename # Can be replaced by orgImage filename
cropped_path = f"images/croped-{file_name}"
# The crop method from the Image module takes four coordinates as input.
# The right can also be represented as (left+width)
# and lower can be represented as (upper+height).
(left, upper, right, lower) = (20, 20, 100, 100)
# Here the image "image" is cropped and assigned to new variable im_crop
im_crop = image.crop((left, upper, right, lower))
im_crop.save(cropped_path)
imgs.editedImg = cropped_path
imgs.save()
Pillow's reference

Failing to display images

I am writing this Django program which is a clone of Craigslist but displaying images of the searched products. The issue is I failing to display the actual image on the card, I am only getting the image icon at the top left corner of the card
import requests
from bs4 import BeautifulSoup
from django.shortcuts import render
from urllib.parse import quote_plus
from . import models
BASE_CRAIGSLIST_URL = 'https://losangeles.craigslist.org/d/services/search/bbb?query={}'
BASE_IMAGE_URL = 'https://images.craigslist.org/{}_300x300.jpg'
# Create your views here.
def home(request):
return render(request, 'base.html')
def new_search(request):
search = request.POST.get('search')
models.Search.objects.create(search=search)
final_url = BASE_CRAIGSLIST_URL.format(quote_plus(search))
response = requests.get(final_url)
data = response.text
soup = BeautifulSoup(data, features='html.parser')
post_listings = soup.find_all('li', {'class': 'result-row'})
final_postings = []
for post in post_listings:
post_title = post.find(class_='result-title').text
post_url = post.find('a').get('href')
if post.find(class_='result-price'):
post_price = post.find(class_='result-price').text
else:
post_price = 'N/A'
if post.find(class_='result-image').get('data-ids'):
post_image_id = post.find(class_='result-image').get('data-ids').split(',')[0].split(':')
post_image_url = BASE_IMAGE_URL.format(post_image_id)
print(post_image_url)
else:
post_image_url = 'https://craigslist.org/images/peace.jpg'
final_postings.append((post_title, post_url, post_price, post_image_url))
stuff_for_frontend = {
'search': search,
'final_postings': final_postings,
}
return render(request, 'my_app/new_search.html', stuff_for_frontend)
so i have figured it out, i was trying to access a single image yet the url had like a slide of images so i had to select the first image and display that one like this
post_image_id = post.find(class_='result-image').get('data-ids').split(',')[0].split(':')[1]

Upload Image to Amazon S3 with Flask-admin

I am using Flask-Admin and is very happy with it. However, the sample in Flask-Admin only provides to upload the image to static folder. Is it possible to upload it to S3 directly with Flask-Admin? Thanks.
Regards
Alex
Thanks for your sample code, Alex Chan. I needed this functionality too, so I decided to write more complete S3FileUploadField and S3ImageUploadField classes, based on your code and various other snippets.
You can find my code at:
https://github.com/Jaza/flask-admin-s3-upload
Also up on pypi, so you can install with:
pip install flask-admin-s3-upload
I've documentated a basic usage example in the readme (can see it on the github project page). Hope this helps, for anyone else who needs S3 file uploads in flask-admin.
Here it is but not clean the code..
import os
import os.path as op
import cStringIO
import logging
import config
from flask import url_for
from werkzeug import secure_filename
from werkzeug.datastructures import FileStorage
import boto
from boto.s3.key import Key
from wtforms import ValidationError, fields
from wtforms.widgets import HTMLString, html_params
from flask.ext.admin.babel import gettext
from flask.ext.admin._compat import string_types, urljoin
from flask.ext.admin.form.upload import ImageUploadField
try:
from PIL import Image, ImageOps
except ImportError:
Image = None
ImageOps = None
__all__ = ['FileUploadInput', 'FileUploadField',
'ImageUploadInput', 'ImageUploadField',
'namegen_filename', 'thumbgen_filename']
class ImageUploadInput(object):
"""
Renders a image input chooser field.
You can customize `empty_template` and `data_template` members to customize
look and feel.
"""
empty_template = ('<input %(file)s>')
data_template = ('<div class="image-thumbnail">'
' <img %(image)s>'
' <input type="checkbox" name="%(marker)s">Delete</input>'
'</div>'
'<input %(file)s>')
def __call__(self, field, **kwargs):
kwargs.setdefault('id', field.id)
kwargs.setdefault('name', field.name)
args = {
'file': html_params(type='file',
**kwargs),
'marker': '_%s-delete' % field.name
}
if field.data and isinstance(field.data, string_types):
url = self.get_url(field)
args['image'] = html_params(src=url)
template = self.data_template
else:
template = self.empty_template
return HTMLString(template % args)
def get_url(self, field):
if field.thumbnail_size:
filename = field.thumbnail_fn(field.data)
else:
filename = field.data
if field.url_relative_path:
filename = urljoin(field.url_relative_path, filename)
return field.data
#return url_for(field.endpoint, filename=field.data)
class s3ImageUploadField(ImageUploadField):
"""
Image upload field.
Does image validation, thumbnail generation, updating and deleting images.
Requires PIL (or Pillow) to be installed.
"""
widget = ImageUploadInput()
keep_image_formats = ('PNG',)
"""
If field detects that uploaded image is not in this list, it will save image
as PNG.
"""
def __init__(self, label=None, validators=None,
base_path=None, relative_path=None,
namegen=None, allowed_extensions=None,
max_size=None,
thumbgen=None, thumbnail_size=None,
permission=0o666,
url_relative_path=None, endpoint='static',
**kwargs):
"""
Constructor.
:param label:
Display label
:param validators:
Validators
:param base_path:
Absolute path to the directory which will store files
:param relative_path:
Relative path from the directory. Will be prepended to the file name for uploaded files.
Flask-Admin uses `urlparse.urljoin` to generate resulting filename, so make sure you have
trailing slash.
:param namegen:
Function that will generate filename from the model and uploaded file object.
Please note, that model is "dirty" model object, before it was committed to database.
For example::
import os.path as op
def prefix_name(obj, file_data):
parts = op.splitext(file_data.filename)
return secure_filename('file-%s%s' % parts)
class MyForm(BaseForm):
upload = FileUploadField('File', namegen=prefix_name)
:param allowed_extensions:
List of allowed extensions. If not provided, will allow any file.
:param max_size:
Tuple of (width, height, force) or None. If provided, Flask-Admin will
resize image to the desired size.
:param thumbgen:
Thumbnail filename generation function. All thumbnails will be saved as JPEG files,
so there's no need to keep original file extension.
For example::
import os.path as op
def thumb_name(filename):
name, _ = op.splitext(filename)
return secure_filename('%s-thumb.jpg' % name)
class MyForm(BaseForm):
upload = ImageUploadField('File', thumbgen=prefix_name)
:param thumbnail_size:
Tuple or (width, height, force) values. If not provided, thumbnail won't be created.
Width and height is in pixels. If `force` is set to `True`, will try to fit image into dimensions and
keep aspect ratio, otherwise will just resize to target size.
:param url_relative_path:
Relative path from the root of the static directory URL. Only gets used when generating
preview image URLs.
For example, your model might store just file names (`relative_path` set to `None`), but
`base_path` is pointing to subdirectory.
:param endpoint:
Static endpoint for images. Used by widget to display previews. Defaults to 'static'.
"""
# Check if PIL is installed
if Image is None:
raise Exception('PIL library was not found')
self.max_size = max_size
self.thumbnail_fn = thumbgen or thumbgen_filename
self.thumbnail_size = thumbnail_size
self.endpoint = endpoint
self.image = None
self.url_relative_path = url_relative_path
if not allowed_extensions:
allowed_extensions = ('gif', 'jpg', 'jpeg', 'png', 'tiff')
super(ImageUploadField, self).__init__(label, validators,
base_path=base_path,
relative_path=relative_path,
namegen=namegen,
allowed_extensions=allowed_extensions,
permission=permission,
**kwargs)
def pre_validate(self, form):
super(ImageUploadField, self).pre_validate(form)
if self.data and isinstance(self.data, FileStorage):
try:
self.image = Image.open(self.data)
except Exception as e:
raise ValidationError('Invalid image: %s' % e)
# Deletion
def _delete_file(self, filename):
super(ImageUploadField, self)._delete_file(filename)
self._delete_thumbnail(filename)
def _delete_thumbnail(self, filename):
path = self._get_path(self.thumbnail_fn(filename))
if op.exists(path):
os.remove(path)
# Saving
def _save_file(self, data, filename):
path = self._get_path(filename)
if not op.exists(op.dirname(path)):
os.makedirs(os.path.dirname(path), self.permission)
# Figure out format
filename, format = self._get_save_format(filename, self.image)
if self.image and (self.image.format != format or self.max_size):
if self.max_size:
image = self._resize(self.image, self.max_size)
else:
image = self._resize(self.image, (500, 500))
#image = self.image
self._save_image(image, filename, format)
else:
data.seek(0)
data.save(path)
savedUrl=self._save_image(self.image, filename, format)
self._save_thumbnail(data, filename, format)
return savedUrl
def _save_thumbnail(self, data, filename, format):
if self.image and self.thumbnail_size:
path = self._get_path(self.thumbnail_fn(filename))
savedUrl=self._save_image(self._resize(self.image, self.thumbnail_size),
thumbgen_filename(filename),
format)
return savedUrl
def _resize(self, image, size):
(width, height, force) = size
if image.size[0] > width or image.size[1] > height:
if force:
return ImageOps.fit(self.image, (width, height), Image.ANTIALIAS)
else:
thumb = self.image.copy()
thumb.thumbnail((width, height), Image.ANTIALIAS)
return thumb
return image
def _save_image(self, image, path, format='JPEG'):
if image.mode not in ('RGB', 'RGBA'):
image = image.convert('RGBA')
conn =boto.connect_s3( config.AWS_KEY, config.AWS_SECRET,)
bucket = conn.get_bucket("vipbutton")
k = Key(bucket)
k.key= path
tempFile = cStringIO.StringIO()
image.save(tempFile,format)
#image.seek(0)
#tempFile.seek(0)
#k.set_contents_from_string('This is a test of S3')
k.set_contents_from_string(tempFile.getvalue())
k.set_acl('public-read')
#k.set_contents_from_file(tempFile.getValue())
#with open(path, 'wb') as fp:
# image.save(fp, format)
return k.generate_url(expires_in=0, query_auth=False)
def _get_save_format(self, filename, image):
if image.format not in self.keep_image_formats:
name, ext = op.splitext(filename)
filename = '%s.jpg' % name
return filename, 'JPEG'
return filename, image.format
# Helpers
def namegen_filename(obj, file_data):
"""
Generate secure filename for uploaded file.
"""
return secure_filename(file_data.filename)
def thumbgen_filename(filename):
"""
Generate thumbnail name from filename.
"""
name, ext = op.splitext(filename)
return '%s_thumb%s' % (name, ext)

Django - Getting PIL Image save method to work with Amazon s3boto Storage

In order to resize images upon upload (using PIL), I'm overriding the save method for my Article model like so:
def save(self):
super(Article, self).save()
if self.image:
size = (160, 160)
image = Image.open(self.image)
image.thumbnail(size, Image.ANTIALIAS)
image.save(self.image.path)
This works locally but in production I get an error:
NotImplementedError: This backend doesn't support absolute paths.
I tried replacing the image.save line with
image.save(self.image.url)
but then I get an IOError:
[Errno 2] No such file or directory: 'https://my_bucket_name.s3.amazonaws.com/article/article_images/2.jpg'
That is the correct location of the image though. If I put that address in the browser, the image is there. I tried a number of other things but so far, no luck.
You should try and avoid saving to absolute paths; there is a File Storage API which abstracts these types of operations for you.
Looking at the PIL Documentation, it appears that the save() function supports passing a file-like object instead of a path.
I'm not in an environment where I can test this code, but I believe you would need to do something like this instead of your last line:
from django.core.files.storage import default_storage as storage
fh = storage.open(self.image.name, "w")
format = 'png' # You need to set the correct image format here
image.save(fh, format)
fh.close()
For me default.storage.write() did not work, image.save() did not work, this one worked. See this code if anyone is still interested. I apologize for the indentation. My project was using Cloudinary and Django small project.
from io import BytesIO
from django.core.files.base import ContentFile
from django.core.files.storage import default_storage as storage
def save(self, *args, **kargs):
super(User, self).save(*args, **kargs)
# After save, read the file
image_read = storage.open(self.profile_image.name, "r")
image = Image.open(image_read)
if image.height > 200 or image.width > 200:
size = 200, 200
# Create a buffer to hold the bytes
imageBuffer = BytesIO()
# Resize
image.thumbnail(size, Image.ANTIALIAS)
# Save the image as jpeg to the buffer
image.save(imageBuffer, image.format)
# Check whether it is resized
image.show()
# Save the modified image
user = User.objects.get(pk=self.pk)
user.profile_image.save(self.profile_image.name, ContentFile(imageBuffer.getvalue()))
image_read = storage.open(user.profile_image.name, "r")
image = Image.open(image_read)
image.show()
image_read.close()
If you are working with cloud storages for files in Django
NotImplementedError: This backend doesn't support absolute paths
To fix it you need to replace file.path with file.name
For code in the the question: image.save(self.image.path) with image.save(self.image.name)
Here how it looks like in the console
>>> c = ContactImport.objects.last()
>>> c.json_file.name
'protected/json_files/data_SbLN1MpVGetUiN_uodPnd9yE2prgeTVTYKZ.json'
>>> c.json_file
<FieldFile: protected/json_files/data_SbLN1MpVGetUiN_uodPnd9yE2prgeTVTYKZ.json>
>>> c.json_file.url
'https://storage.googleapis.com/super-secret/media/api/protected/json_files/data_SbLN1MpVGetUiN_uodPnd9yE2prgeTVTYKZ.json?Expires=1631378947&GoogleAccessId=secret&Signature=ga7...'

Django admin upload and image to s3 and then resize the image and save a thumb problem

I am having error after error trying to upload and resize images to s3 with pil and botos3 and the django default_storage. I am trying to do this on save in the admin.
here is the code:
from django.db import models
from django.forms import CheckboxSelectMultiple
import tempfile
from django.conf import settings
from django.core.files.base import ContentFile
from django.core.files.storage import default_storage as s3_storage
from django.core.cache import cache
from datetime import datetime
import Image, os
import PIL.Image as PIL
import re, os, sys, urlparse
class screenshot(models.Model):
title = models.CharField(max_length=200)
slug = models.SlugField(max_length=200)
image = models.ImageField(upload_to='screenshots')
thumbnail = models.ImageField(upload_to='screenshots-thumbs', blank=True, null=True, editable=False)
def save(self):
super(screenshot, self).save() # Call the "real" save() method
if self.image:
thumb = Image.open(self.image.path)
thumb.thumbnail(100, 100)
filename = str(self.slug)
temp_image = open(os.path.join('tmp',filename), 'w')
thumb.save(temp_image, 'JPEG')
from django.core.files import File
thumb_data = open(os.path.join('/tmp',filename), 'r')
thumb_file = File(thumb_data)
new_file.thumb.save(str(self.slug) + '.jpg', thumb_file)
def __str__(self):
return self.title
This is just one of the many ways I have tried to get it working, and I either get (2, 'No such file or directory') or some other error.
Please can someone help me to get it working. I want it to use the django backend to get the image uploaded to be resized and saved as the thumbnail and then saved. Let me know if you need to know any information. I would be happy to use the django snippet - http://djangosnippets.org/snippets/224/ but I don't know what data to feed it. I get the same IOErrors and 'no such path/filename' even though the main image is uploading to s3 fine. I have also tried things like:
myimage = open(settings.MEDIA_URL + str(self.image))
myimage_io = StringIO.StringIO()
imageresize = myimage.resize((100,100), Image.ANTIALIAS)
imageresize.save('resize_100_100_aa.jpg', 'JPEG', quality=75)
It's been 3 days of looking now so I am starting to go spare! Thanks
I had a similar problem, but in my case using sorl-thumbnail was not an option. I found that I can open an Image directly from S3BotoStorage by passing in a file descriptor instead of a path.
So instead of
thumb = Image.open(self.image.path)
use
thumb = Image.open(s3_storage.open(self.image.name))
Then you can process and save the new file locally as you were doing before.
Why don't you try sorl-thumbnail. It has the exact same interface as the default ImageField django provides and it seems like it would be a lot nicer to work with than the roll-your-own support.
Storage support
Pluggable Engine support (PIL, pgmagick)
Pluggable Key Value Store support (redis, cached db)
Pluggable Backend support
Admin integration with possibility to delete
Dummy generation
Flexible, simple syntax, generates no html
ImageField for model that deletes thumbnails
CSS style cropping options
Margin calculation for vertical positioning