I'm trying to copy a file using a hardlink, where the file is stored as a Django FileField. I'd like to use a hardlink to save space and copy time (no changes are expected to be made to the original file or copy). However, I'm getting some odd errors when I try to call new_file.save() from the snippet below.
AttributeError: 'file' object has no attribute '_committed'
My thinking is that after making the hardlink, I can just open the linked file and store it to the Django new File instance's FileFile. Am I missing a step here or something?
models.py
class File(models.Model):
stored_file = models.FileField()
elsewhere.py
import os
original_file = File.objects.get(id=1)
original_file_path = original_file.file.path
new_file = File()
new_file_path = '/path/to/new/file'
os.makedirs(os.path.realpath(os.path.dirname(new_file_path)))
os.link(original_file_path, new_file_path)
new_file.stored_file = file(new_file_path)
new_file.save()
There is no need to create hardlink, just duplicate the file holder:
new_file = File(stored_file=original_file.stored_file)
new_file.save()
update
If you want to specify file to FileField or ImageField, you could simply
new_file = File(stored_file=new_file_path)
# or
new_file = File()
new_file.stored_file = new_file_path
# or
from django.core.files.base import File
# from django.core.files.images import ImageFile # for ImageField
new_file.stored_file = File(new_file_path)
the field accepts path in basestring or File() instance, the code in your question uses file() and hence is not accepted.
I think I solved this issue, but not sure why it works. I wrapped the file object in a "DjangoFile" class (I imported as DjangoFile to avoid clashing with my previously defined File model).
from django.core.files.base import File as DjangoFile
...
new_file.stored_file = DjangoFile(file(new_file_path))
new_file.save()
This approached seemed to save the file OK.
Related
I have a model that contains a FileField:
class Foo(models.Model):
fileobj = models.FileField(upload_to="bar/baz")
I am generating a file, and saving it in /tmp/ as part of the save method. This file then needs to be set as the "fileobj" of the model instance.
Currently, I'm trying this:
with open(
f"/tmp/{self.number}.pdf", "r"
) as h:
self.fileobj = File(h)
Unfortunately, this fails with: django.core.exceptions.SuspiciousFileOperation:, because the file exists outside of the django project.
I've tried reading the docs, but they didn't help much. Does django take a file, and upon assigning it as a FileField, move it to the media directory, or do I need to manually put it there myself, before attaching it to the model instance. If the second case, what is the point of "upload_to"?
You can use InMemoryUploadedFile object like this.
import os
import io
with open(path, 'rb') as h:
f = InMemoryUploadedFile(io.BytesIO(h.read()), 'fileobj',
'name.pdf', 'application/pdf',
os.path.getsize(path), None)
self.fileobj = f
I want to create a file and associate it with the FileField of my model. Here's my simplified attempt:
#instantiate my form with the POST data
form = CSSForm(request.POST)
#generate a css object from a ModelForm
css = form.save(commit=False)
#generate some css:
css_string = "body {color: #a9f;}"
#create a css file:
filename = "myfile.css"
#try to write the file and associate it with the model
with open(filename, 'wb') as f:
df = File(f) #create django File object
df.write(css_string)
css.css_file = df
css.save()
The call to save() throws a "seek of closed file" exception. If I move the save() to the with block, it produces an unsupported operation "read". At the moment, the files are being created in my media directory, but are empty. If I just render the css_string with the HttpResponse then I see the expected css.
The docs don't seem to have an example on how to link a generated file and a database field. How do I do this?
Django FileField would either be a django.core.files.File, which is a file instance or django.core.files.base.ContentFile, which takes a string as parameter and compose a ContentFile. Since you already had the file content as a string, sounds like ContentFile is the way to go(I couldn't test it but it should work):
from django.core.files.base import ContentFile
# create an in memory instance
css = form.save(commit=False)
# file content as string
css_string = "body {color: #a9f;}"
# create ContentFile instance
css_file = ContentFile(css_string)
# assign the file to the FileField
css.css_file.save('myfile.css', css_file)
css.save()
Check django doc about FileField details.
I have written an email form class in Django with a FileField. I want to check the uploaded file for its type via checking its mimetype. Subsequently, I want to limit file types to pdfs, word, and open office documents.
To this end, I have installed python-magic and would like to check file types as follows per the specs for python-magic:
mime = magic.Magic(mime=True)
file_mime_type = mime.from_file('address/of/file.txt')
However, recently uploaded files lack addresses on my server. I also do not know of any method of the mime object akin to "from_file_content" that checks for the mime type given the content of the file.
What is an effective way to use magic to verify file types of uploaded files in Django forms?
Stan described good variant with buffer. Unfortunately the weakness of this method is reading file to the memory. Another option is using temporary stored file:
import tempfile
import magic
with tempfile.NamedTemporaryFile() as tmp:
for chunk in form.cleaned_data['file'].chunks():
tmp.write(chunk)
print(magic.from_file(tmp.name, mime=True))
Also, you might want to check the file size:
if form.cleaned_data['file'].size < ...:
print(magic.from_buffer(form.cleaned_data['file'].read()))
else:
# store to disk (the code above)
Additionally:
Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).
So you might want to handle it like so:
import os
tmp = tempfile.NamedTemporaryFile(delete=False)
try:
for chunk in form.cleaned_data['file'].chunks():
tmp.write(chunk)
print(magic.from_file(tmp.name, mime=True))
finally:
os.unlink(tmp.name)
tmp.close()
Also, you might want to seek(0) after read():
if hasattr(f, 'seek') and callable(f.seek):
f.seek(0)
Where uploaded data is stored
Why no trying something like that in your view :
m = magic.Magic()
m.from_buffer(request.FILES['my_file_field'].read())
Or use request.FILES in place of form.cleaned_data if django.forms.Form is really not an option.
mime = magic.Magic(mime=True)
attachment = form.cleaned_data['attachment']
if hasattr(attachment, 'temporary_file_path'):
# file is temporary on the disk, so we can get full path of it.
mime_type = mime.from_file(attachment.temporary_file_path())
else:
# file is on the memory
mime_type = mime.from_buffer(attachment.read())
Also, you might want to seek(0) after read():
if hasattr(f, 'seek') and callable(f.seek):
f.seek(0)
Example from Django code. Performed for image fields during validation.
You can use django-safe-filefield package to validate that uploaded file extension match it MIME-type.
from safe_filefield.forms import SafeFileField
class MyForm(forms.Form):
attachment = SafeFileField(
allowed_extensions=('xls', 'xlsx', 'csv')
)
In case you're handling a file upload and concerned only about images,
Django will set content_type for you (or rather for itself?):
from django.forms import ModelForm
from django.core.files import File
from django.db import models
class MyPhoto(models.Model):
photo = models.ImageField(upload_to=photo_upload_to, max_length=1000)
class MyForm(ModelForm):
class Meta:
model = MyPhoto
fields = ['photo']
photo = MyPhoto.objects.first()
photo = File(open('1.jpeg', 'rb'))
form = MyForm(files={'photo': photo})
if form.is_valid():
print(form.instance.photo.file.content_type)
It doesn't rely on content type provided by the user. But
django.db.models.fields.files.FieldFile.file is an undocumented
property.
Actually, initially content_type is set from the request, but when
the form gets validated, the value is updated.
Regarding non-images, doing request.FILES['name'].read() seems okay to me.
First, that's what Django does. Second, files larger than 2.5 Mb by default
are stored on a disk. So let me point you at the other answer
here.
For the curious, here's the stack trace that leads to updating
content_type:
django.forms.forms.BaseForm.is_valid: self.errors
django.forms.forms.BaseForm.errors: self.full_clean()
django.forms.forms.BaseForm.full_clean: self._clean_fields()
django.forms.forms.BaseForm._clean_fiels: field.clean()
django.forms.fields.FileField.clean: super().clean()
django.forms.fields.Field.clean: self.to_python()
django.forms.fields.ImageField.to_python
I want to import the entire csv file in a model without reading row by row from the file.Please help me on this by providing a example model and a source code to import.
If you're opening the file from disk, you can wrap your file object in django.core.files.File and pass it to the save method of the model field you're saving it to:
from django.core.files import File
csv_file = open("sample.csv", "rb")
csv_file = File(csv_file)
my_model_instance.my_file_field.save("sample.csv", csv_file)
See https://docs.djangoproject.com/en/1.3/ref/files/file/#additional-methods-on-files-attached-to-objects
If you're dealing with an uploaded file from request.FILES, you can assign it directly to your model instance's FileField:
my_model_instance.my_file = request.FILES["csvfile"]
my_model_instance.save()
Don't forget enctype="multipart/form-data" on the form or request.FILES will be empty.
I have 2 simple models:
class UploadImage(models.Model):
Image = models.ImageField(upload_to="temp/")
class RealImage(models.Model):
Image = models.ImageField(upload_to="real/")
And one form
class RealImageForm(ModelForm):
class Meta:
model = RealImage
I need to save file from UploadImage into RealImage. How could i do this.
Below code doesn't work
realform.Image=UploadImage.objects.get(id=image_id).Image
realform.save()
Tnx for help.
Inspired by Gerard's solution I came up with the following code:
from django.core.files.base import ContentFile
#...
class Example(models.Model):
file = models.FileField()
def duplicate(self):
"""
Duplicating this object including copying the file
"""
new_example = Example()
new_file = ContentFile(self.file.read())
new_file.name = self.file.name
new_example.file = new_file
new_example.save()
This will actually go as far as renaming the file by adding a "_1" to the filename so that both the original file and this new copy of the file can exist on disk at the same time.
Although this is late, but I would tackle this problem thus,
class UploadImage(models.Model):
Image = models.ImageField(upload_to="temp/")
# i need to delete the temp uploaded file from the file system when i delete this model
# from the database
def delete(self, using=None):
name = self.Image.name
# i ensure that the database record is deleted first before deleting the uploaded
# file from the filesystem.
super(UploadImage, self).delete(using)
self.Image.storage.delete(name)
class RealImage(models.Model):
Image = models.ImageField(upload_to="real/")
# in my view or where ever I want to do the copying i'll do this
import os
from django.core.files import File
uploaded_image = UploadImage.objects.get(id=image_id).Image
real_image = RealImage()
real_image.Image = File(uploaded_image, uploaded_image.name)
real_image.save()
uploaded_image.close()
uploaded_image.delete()
If I were using a model form to handle the process, i'll just do
# django model forms provides a reference to the associated model via the instance property
form.instance.Image = File(uploaded_image, os.path.basename(uploaded_image.path))
form.save()
uploaded_image.close()
uploaded_image.delete()
note that I ensure the uploaded_image file is closed because calling real_image.save() will open the file and read its content. That is handled by what ever storage system is used by the ImageField instance
Try doing that without using a form. Without knowing the exact error that you are getting, I can only speculate that the form's clean() method is raising an error because of a mismatch in the upload_to parameter.
Which brings me to my next point, if you are trying to copy the image from 'temp/' to 'real/', you will have to do a some file handling to move the file yourself (easier if you have PIL):
import Image
from django.conf import settings
u = UploadImage.objects.get(id=image_id)
im = Image.open(settings.MEDIA_ROOT + str(u.Image))
newpath = 'real/' + str(u.Image).split('/', 1)[1]
im.save(settings.MEDIA_ROOT + newpath)
r = RealImage.objects.create(Image=newpath)
Hope that helped...
I had the same problem and solved it like this, hope it helps anybody:
# models.py
class A(models.Model):
# other fields...
attachment = FileField(upload_to='a')
class B(models.Model):
# other fields...
attachment = FileField(upload_to='b')
# views.py or any file you need the code in
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
from django.core.files.base import ContentFile
from main.models import A, B
obj1 = A.objects.get(pk=1)
# You and either copy the file to an existent object
obj2 = B.objects.get(pk=2)
# or create a new instance
obj2 = B(**some_params)
tmp_file = StringIO(obj1.attachment.read())
tmp_file = ContentFile(tmp_file.getvalue())
url = obj1.attachment.url.split('.')
ext = url.pop(-1)
name = url.pop(-1).split('/')[-1] # I have my files in a remote Storage, you can omit the split if it doesn't help you
tmp_file.name = '.'.join([name, ext])
obj2.attachment = tmp_file
# Remember to save you instance
obj2.save()
Update Gerard's Solution to handle it in a generic way:
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
from django.core.files.base import ContentFile
init_str = "src_obj." + src_field_name + ".read()"
file_name_str = "src_obj." + src_field_name + ".name"
try:
tmp_file = StringIO(eval(str(init_str)))
tmp_file = ContentFile(tmp_file.getvalue())
tmp_file.name = os.path.basename(eval(file_name_str))
except AttributeError:
tmp_file = None
if tmp_file:
try:
dest_obj.__dict__[dest_field_name] = tmp_file
dest_obj.save()
except KeyError:
pass
Variable's Used:
src_obj = source attachment object.
src_field_name = source attachment object's FileField Name.
dest_obj = destination attachment object.
dest_field_name = destination attachment object's FileField Name.