I am having the following issue with filedownload from product views in Django product site:
The issue: Filesize of downloaded file is practically 1kb whilst it should be a normal image filesize (20kb in my example).
So the to-download file is present in static folder of the product instance.id (static_cdn/protected/instance.id/image.jpg -- context: product site where user can upload a file to the corresponding product view).
However, whenever I try to download it from the product view, it downloads the file with the right filename (including the added instance.id number before the filename), but the filesize is almost null. I think it has to do the with the class ProductDownloadView.
Please find the relevant codes below:
views.py:
class ProductDownloadView(MultiSlugMixin, DetailView):
model = Product
def get(self, request, *args, **kwargs):
obj = self.get_object()
filepath = os.path.join(settings.PROTECTED_ROOT, obj.media.path)
response = HttpResponse(file(filepath), content_type="application/force-download")
response["Content-Disposition"] = "attachment;filename=%s" % (obj.media.name)
response["X-SendFile"] = str(obj.media.name)
return response
models.py
class Product(models.Model):
user = models.ForeignKey(settings.AUTH_USER_MODEL, null=True, blank=True)
managers = models.ManyToManyField(settings.AUTH_USER_MODEL, related_name="managers_product")
media = models.FileField(blank=True,
null=True,
upload_to=download_media_location,
storage=FileSystemStorage(location=settings.PROTECTED_ROOT))
def __unicode__(self):
return self.title
def get_absolute_url(self):
view_name = "products:detail_slug"
return reverse(view_name, kwargs={"slug": self.slug})
def get_download(self):
view_name = "products:download_slug"
url = reverse(view_name, kwargs={"slug": self.slug})
return url
Please find below the printed obj, filepath and response variables:
print obj:
pr8
print filepath:
C:\Users\xx\xx\xx\market_place\static_cdn\protected\8\Beach.jpg
print response:
Content-Type: application/force-download
Content-Disposition: attachment;filename=8/Beach.jpg
X-SendFile: 8/Beach.jpg
���� JFIF �� C
[21/Jun/2017 02:17:05] "GET /products/pr8/download/ HTTP/1.1" 200 52
I think I've found the answer. I got the file-download to work by using the open method instead of the file method. Because of this solution I am deviating from a tutorial, but at least I got the job done.
So I got it working by changing the following rule:
response = HttpResponse(file(filepath), content_type="application/force-download")
into:
response = HttpResponse(open(filepath, "rb"), content_type="application/force-download")
So basically adding a mode to the function. Even the file method works after adding the mode "rb".
try this:
response = HttpResponse(content_type="image/jpeg")
response['X-Sendfile'] = obj.media.path
response['Content-Disposition'] = 'attachment; filename=%s' % smart_str(photo.image.name)
Related
400 Error
{"detail":"Missing filename. Request should include a Content-Disposition header with a filename parameter."}
I want to upload file via DRF FileUploadParser
But Error occurs below
Bad Request: /api/activities/40/chapter/1/upload
[09/Nov/2021 16:44:33] "POST /api/activities/40/chapter/1/upload HTTP/1.1" 400 109
And my codes in views.py about that error are this.
class FileView(APIView):
parser_classes = (FileUploadParser,)
def post(self, request, format=None, *args, **kwargs):
if 'file' not in request.FILES:
raise ParseError("Empty content")
f = request.FILES.get('file')
print(f)
print(dir(request))
print(request.__dict__)
addAttr = request.data.dict()
file_name = request.data['filename']
new_file_full_name = file_upload_path(file_name.name)
file_path = '/'.join(new_file_full_name.split('/')[0:-1])
#model Attr
addAttr['activityid'] = request.parser_context['kwargs']['pk']
addAttr['chapterid'] = request.parser_context['kwargs']['chapterid']
addAttr['filepath'] = file_path
addAttr['filename'] = file_name
addAttr['fileext'] = os.path.splitext(file_name.name)[1]
addAttr['create_date'] = datetime.datetime.now()
addAttrDict = QueryDict('', mutable=True)
addAttrDict.update(addAttr)
fileSerializer = ChapterfileSerializer(data = addAttrDict, files=request.FILES)
if fileSerializer.is_valid():
fileSerializer.save()
print(fileSerializer.data)
return Response(status=status.HTTP_201_CREATED)
else:
print(fileSerializer.errors)
return Response(fileSerializer.errors, status=status.HTTP_400_BAD_REQUEST)
If I add a parameter "filename", 500 error occured.
TypeError: post() missing 1 required positional argument: 'parameter'
[09/Nov/2021 16:48:35] "POST /api/activities/40/chapter/1/upload HTTP/1.1" 500 86716
ReactJS page sends File to my Django API Server.
activityid and chapterid are Board and Post ID.
SO, I need these insert to DB.
How can I solve this?
Use the FileUploadParser, it's all in the request. Use a put method instead, you'll find an example in the docs :)
class FileUploadAPIView(views.APIView):
parser_classes = (FileUploadParser,)
def put(self, request, filename, format=None):
file_obj = request.FILES['file']
# Do some stuff with the uploaded file
return Response({'details': 'Your file uploaded successfully.'}, status=204)
my problem is that I can not establish a reverse match, most likely doing something wrong with my url definition (?). Ultimately what I am trying to do is the following:
User selects 2 location points which the 'new_pointview' view, saves into a DB. I also define a unique slug which contains location information and save it to the DB via the save() within the model. Then the user should be redirected to a url (pointview view) which uses the slug I created in the previous step i.e /pointview/slug. Here is the code:
models.py
class Points(models.Model):
starting_point_longitude = models.FloatField(null=True)
starting_point_latitude = models.FloatField(null=True)
ending_point_longitude = models.FloatField(null=True)
ending_point_latitude = models.FloatField(null=True)
url = models.SlugField(max_length=250, null=True, unique=True, blank=False)
def save(self, *args, **kwargs):
self.url = 'start_lon-{0}-start_lat-{1}-end_lon-{2}-end_lat-' \
'{3}'.format(self.starting_point_longitude,
self.starting_point_latitude,
self.ending_point_longitude,
self.ending_point_latitude)
super(Points, self).save(*args, **kwargs)
def get_absolute_url(self):
return reverse('pointview', kwargs={'url': self.url})
views.py
def pointview(request, url):
point = get_object_or_404(Points, url=url)
content = {'starting_point_longitude':
point.starting_point_longitude,
'starting_point_latitude':
point.starting_point_latitude,
'ending_point_longitude':
point.ending_point_longitude,
'ending_point_latitude':
point.ending_point_latitude}
return render(request, 'points.html', {'user_bundle': content})
def new_pointview(request):
Points.objects.create(
starting_point_longitude=request.POST['starting_point_longitude'],
starting_point_latitude=request.POST['starting_point_latitude'],
ending_point_longitude=request.POST['ending_point_longitude'],
ending_point_latitude=request.POST['ending_point_latitude'],
)
points_filtered = Points.objects.filter(
starting_point_longitude=request.POST[
'starting_point_longitude']).filter(
starting_point_latitude=request.POST[
'starting_point_latitude'])
unique_url = points_filtered.values()[0]['url']
return redirect('/pointview/{0}/'.format(unique_url))
urls.py
urlpatterns = [
path(r'^pointview/(?P<url>[-\w]+)/$', views.pointview, name='pointview'),
path('^new_pointview/', views.new_pointview, name='new_pointview'),
]
The error:
The current path, pointview/start_lon-738949.9146592747-start_lat--153698.8751025315-end_lon-759997.8063993475-end_lat--168467.65638300427/, didn't match any of URL patterns. Hope you can give me some feedback here..
For future reference, it was a regex problem, using the following non intuitive regex solved it:
url('^pointview/(?P<url>start_lon[--W]+start_lat[--W]+end_lon[--W]+end_lat[--W]+)/', views.pointview, name='pointview')
I would be very interesting though if someone can give a more elegant solution.
I have an app that lets people upload files, represented as UploadedFiles. However, I want to make sure that users only upload xml files. I know I can do this using magic, but I don't know where to put this check - I can't put it in the clean function since the file is not yet uploaded when clean runs, as far as I can tell.
Here's the UploadedFile model:
class UploadedFile(models.Model):
"""This represents a file that has been uploaded to the server."""
STATE_UPLOADED = 0
STATE_ANNOTATED = 1
STATE_PROCESSING = 2
STATE_PROCESSED = 4
STATES = (
(STATE_UPLOADED, "Uploaded"),
(STATE_ANNOTATED, "Annotated"),
(STATE_PROCESSING, "Processing"),
(STATE_PROCESSED, "Processed"),
)
status = models.SmallIntegerField(choices=STATES,
default=0, blank=True, null=True)
file = models.FileField(upload_to=settings.XML_ROOT)
project = models.ForeignKey(Project)
def __unicode__(self):
return self.file.name
def name(self):
return os.path.basename(self.file.name)
def save(self, *args, **kwargs):
if not self.status:
self.status = self.STATE_UPLOADED
super(UploadedFile, self).save(*args, **kwargs)
def delete(self, *args, **kwargs):
os.remove(self.file.path)
self.file.delete(False)
super(UploadedFile, self).delete(*args, **kwargs)
def get_absolute_url(self):
return u'/upload/projects/%d' % self.id
def clean(self):
if not "XML" in magic.from_file(self.file.url):
raise ValidationError(u'Not an xml file.')
class UploadedFileForm(forms.ModelForm):
class Meta:
model = UploadedFile
exclude = ('project',)
Validating files is a common challenge, so I would like to use a validator:
import magic
from django.utils.deconstruct import deconstructible
from django.template.defaultfilters import filesizeformat
#deconstructible
class FileValidator(object):
error_messages = {
'max_size': ("Ensure this file size is not greater than %(max_size)s."
" Your file size is %(size)s."),
'min_size': ("Ensure this file size is not less than %(min_size)s. "
"Your file size is %(size)s."),
'content_type': "Files of type %(content_type)s are not supported.",
}
def __init__(self, max_size=None, min_size=None, content_types=()):
self.max_size = max_size
self.min_size = min_size
self.content_types = content_types
def __call__(self, data):
if self.max_size is not None and data.size > self.max_size:
params = {
'max_size': filesizeformat(self.max_size),
'size': filesizeformat(data.size),
}
raise ValidationError(self.error_messages['max_size'],
'max_size', params)
if self.min_size is not None and data.size < self.min_size:
params = {
'min_size': filesizeformat(self.min_size),
'size': filesizeformat(data.size)
}
raise ValidationError(self.error_messages['min_size'],
'min_size', params)
if self.content_types:
content_type = magic.from_buffer(data.read(), mime=True)
data.seek(0)
if content_type not in self.content_types:
params = { 'content_type': content_type }
raise ValidationError(self.error_messages['content_type'],
'content_type', params)
def __eq__(self, other):
return (
isinstance(other, FileValidator) and
self.max_size == other.max_size and
self.min_size == other.min_size and
self.content_types == other.content_types
)
Then you can use FileValidator in your models.FileField or forms.FileField as follows:
validate_file = FileValidator(max_size=1024 * 100,
content_types=('application/xml',))
file = models.FileField(upload_to=settings.XML_ROOT,
validators=[validate_file])
From django 1.11, you can also use FileExtensionValidator.
from django.core.validators import FileExtensionValidator
class UploadedFile(models.Model):
file = models.FileField(upload_to=settings.XML_ROOT,
validators=[FileExtensionValidator(allowed_extensions=['xml'])])
Note this must be used on a FileField and won't work on a CharField (for example), since the validator validates on value.name.
ref: https://docs.djangoproject.com/en/dev/ref/validators/#fileextensionvalidator
For posterity: the solution is to use the read method and pass that to magic.from_buffer.
class UploadedFileForm(ModelForm):
def clean_file(self):
file = self.cleaned_data.get("file", False)
filetype = magic.from_buffer(file.read())
if not "XML" in filetype:
raise ValidationError("File is not XML.")
return file
class Meta:
model = models.UploadedFile
exclude = ('project',)
I think what you want to do is to clean the uploaded file in Django's Form.clean_your_field_name_here() methods - the data is available on your system by then if it was submitted as normal HTTP POST request.
Also if you consider this inefficient explore the options of different Django file upload backends and how to do streaming processing.
If you need to consider the security of the system when dealing with uploads
Make sure uploaded file has correct extension
Make sure the mimetype matches the file extension
In the case you are worried about user's uploading exploit files (for attacking against your site)
Rewrite all the file contents on save to get rid of possible extra (exploit) payload (so you cannot embed HTML in XML which the browser would interpret as a site-origin HTML file when downloading)
Make sure you use content-disposition header on download
Some more info here: http://opensourcehacker.com/2013/07/31/secure-user-uploads-and-exploiting-served-user-content/
Below is my example how I sanitize the uploaded images:
class Example(models.Model):
image = models.ImageField(upload_to=filename_gen("participant-images/"), blank=True, null=True)
class Example(forms.ModelForm):
def clean_image(self):
""" Clean the uploaded image attachemnt.
"""
image = self.cleaned_data.get('image', False)
utils.ensure_safe_user_image(image)
return image
def ensure_safe_user_image(image):
""" Perform various checks to sanitize user uploaded image data.
Checks that image was valid header, then
:param: InMemoryUploadedFile instance (Django form field value)
:raise: ValidationError in the case the image content has issues
"""
if not image:
return
assert isinstance(image, InMemoryUploadedFile), "Image rewrite has been only tested on in-memory upload backend"
# Make sure the image is not too big, so that PIL trashes the server
if image:
if image._size > 4*1024*1024:
raise ValidationError("Image file too large - the limit is 4 megabytes")
# Then do header peak what the image claims
image.file.seek(0)
mime = magic.from_buffer(image.file.getvalue(), mime=True)
if mime not in ("image/png", "image/jpeg"):
raise ValidationError("Image is not valid. Please upload a JPEG or PNG image.")
doc_type = mime.split("/")[-1].upper()
# Read data from cStringIO instance
image.file.seek(0)
pil_image = Image.open(image.file)
# Rewrite the image contents in the memory
# (bails out with exception on bad data)
buf = StringIO()
pil_image.thumbnail((2048, 2048), Image.ANTIALIAS)
pil_image.save(buf, doc_type)
image.file = buf
# Make sure the image has valid extension (can't upload .htm image)
extension = unicode(doc_type.lower())
if not image.name.endswith(u".%s" % extension):
image.name = image.name + u"." + extension
I found an interesting package who can do upload file validation recently. You can see the package here. the package approach is similar with sultan answer, thus we can just implement it right away.
from upload_validator import FileTypeValidator
validator = FileTypeValidator(
allowed_types=['application/msword'],
allowed_extensions=['.doc', '.docx']
)
file_resource = open('sample.doc')
# ValidationError will be raised in case of invalid type or extension
validator(file_resource)
I have a project in Django, and I'm using mongoengine to save images into a Mongo database using GridFSStorage.
All ok so far, but the problem is... when trying to retrieve the images via http request, with a REST API made with django-tastypie-mongoengine, I get back a json object like this:
{"file": "<GridFSProxy: 516ed7cf56ba7d01eb09f522>", "id": "516ed7cf56ba7d01eb09f524", "resource_uri": "/api/v1/pic/516ed7cf56ba7d01eb09f524/"}
Does anybody know how could I get the file from GridFS via http request?
Many thanks!
You'll need to write your own view, but you can make it seem like it's part of the API. First, the view:
def api_image(pk):
obj = get_object_or_404(Model, pk=pk)
image_file = obj.file
return Response(image_file.read(),
mime_type='image/png') # or whatever the MIME type is
Then, you can map it in your urls.py:
url('^/api/v1/pic/(?P<pk>\w+)/file/$', api_image)
And to make sure tastypie shows what you want in the output:
def dehydrate_file(self, bundle):
return '/api/v1/pic/%s/file/' % (bundle.obj.id)
Just make sure the fake API view appears ahead of your actual API definitions, and you should be all set!
Paul's hint was very useful. Here i have implemented this completely in tastypie manner for uploading and downloading images.
Here you go..
1. Overriding deseriazer to support 'multipart'.
class MultipartResource(object):
def deserialize(self, request, data, format=None):
if not format:
format = request.META.get('CONTENT_TYPE', 'application/json')
if format == 'application/x-www-form-urlencoded':
return request.POST
if format.startswith('multipart'):
data = request.POST.copy()
data.update(request.FILES)
return data
return super(MultipartResource, self).deserialize(request, data, format)
2. Model class
class Research(Document):
user = ReferenceField(User)
academic_year = StringField(max_length=20)
subject = StringField(max_length=150)
topic = StringField(max_length=50)
pub_date = DateTimeField()
authored = StringField(max_length=20)
research_level = StringField(max_length=20)
paper_presented = BooleanField()
thesis_written = BooleanField()
proof_image = ImageField()
3. Resource class
class ResearchResource(MultipartResource, MongoEngineResource):
class Meta:
queryset = Research.objects.all()
list_allowed_methods = ['get','post']
resource_name = 'research'
authentication = SessionAuthentication()
authorization = Authorization()
def prepend_urls(self):
return [
url(r"^(?P<resource_name>%s)/$" % self._meta.resource_name,
self.wrap_view('dispatch_list'), name="api_dispatch_list"),
#url to download image file.
url(r"^(?P<resource_name>%s)/(?P<pk>\w+)/file/$"% self._meta.resource_name,
self.wrap_view('get_image'), name="api_get_image"),
]
#Preparing image url dynamically
def dehydrate_proof_image(self, bundle):
return '/api/v1/%s/%s/file/' % (self._meta.resource_name,bundle.obj.id)
#view will call based on image url to download image.
def get_image(self, request, **kwargs):
obj = Research.objects.get(id=kwargs['pk'])
image_file = obj.proof_image
return HttpResponse(image_file.read(), content_type="image/jpeg"))
Hope this will be very useful for everyone in future. :)
I've been trying to get the tests running on upload forms. But, whenever I run the tests, it says that file is of wrong type.
Upload form saves the file with a randomly generated file name:
class Video(models.Model):
def get_generated_path(self):
# Generates random path for video file upload
original_file = models.CharField(null=True)
uploaded_file = models.FileField(storage=FileSystemStorage(location=
settings.MEDIA_ROOT), upload_to=get_generated_path)
video_name = models.TextField()
And form looks like:
class VideoForm(forms.Form):
video_file = forms.FileField()
video_name = forms.CharField()
def clean_video_file(forms.Form):
content = self.cleaned_data['video_file']
content_type = content.content_type.split('/')[0]
if content_type in settings.CONTENT_TYPES:
if content._size > settings.MAX_UPLOAD_SIZE:
raise forms.ValidationError(_('Please keep filesize under %s.
Current filesize %s') % (filesizeformat(
settings.MAX_UPLOAD_SIZE), filesizeformat(content._size)))
else:
raise forms.ValidationError(_('File type is not supported,
content type is: %s' % content_type))
return content
Most of the remaining logic is in views:
def upload_video(request):
try:
# Check if user is authenticated
if form.is_valid():
video_file = request.FILES['video_file']
video_name = form.cleaned_data['video_name']
save_video = Video.objects.create(
original_file = 'uploaded_videos' + user.username,
video_name = video_name)
return HTTPResponseRedirect('next-page')
except Exception, e:
...
The tests are written as:
def test_video_form(TestCase):
user = #Create a dummy user
test_video = SimpleUploadedFile('test_video.flv', open(
'path/to/test/video', 'rb'))
form = VideoForm(user, {'video_name': test_video}, )
self.assertTrue(form.is_valid())
The above test always fails since it says that the file type of 'test_video.flv' is plain/text. I've checked the 'test_video.flv' and its of correct type.
How to pass these files to the upload form and test it.
SimpleUploadedFile has the content_type text/plain by default. It is why your testing code goes fail in clean_video_file and this line:
raise forms.ValidationError(_('File type is not supported,
content type is: %s' % content_type))
is printing:
File type is not supported,
content type is: text/plain
Pass the content_type = 'video'in the SimpleUploadedFile line as shown below.
def test_video_form(TestCase):
user = #Create a dummy user
test_video = SimpleUploadedFile('test_video.flv', open(
'path/to/test/video', 'rb'), content_type='video') # Notice the change here.
form = VideoForm(user, {'video_name': test_video}, )
self.assertTrue(form.is_valid())