My application has a tastypie endpoint that received post requests with a file(not big...40-70kb) and is supposed to save the file and then process it. However, it seems the file never gets saved after upload and thus trying to retrieve the file after save fails. I have read in many places about it but it seems tastypie has no one standard way of doing it. Here are my models and resources:
models
class Tag(models.Model):
track = models.FileField(upload_to='tags/', max_length=250)
def __unicode__(self):
return self.track.url
api_resource
class TagResource(MultipartResource, ModelResource):
track = fields.FileField(attribute="track", null=True, blank=True)
class Meta:
queryset = Tag.objects.all()
resource_name = 'tag'
authorization = Authorization()
object_class = Tag
always_return_data = True
def dehydrate(self, bundle):
bundle.obj.save()
#the processing operation on the saved file
result = recognize(bundle.obj.track)
bundle.data['tag'] = result
return bundle
When I post a file with Curl:
curl -F "track=/path/to/track/track.mp3" http://127.0.0.1:8000/api/tag/
I get an error:
SuspiciousFileOperation: Attempted access to '/path/to/track/track.mp3'
and upon further investigation, i realized the error is being caused by the intended operation working on the url of the source file instead of the uploaded file's url.
What i'm I doing wrong??
I finally got it to work. Seems like I was not posting correctly using CURL. I now use this to upload the file:
curl -F "track=#/path/to/file.mp3" http://1**.**.***.**:8000/api/tag/
The '#' before the path was missing before. Adding that fixed it.
Related
I would like to test a file upload from my view with the following function:
def test_post(self):
with open("path/to/myfile/test_file.txt") as file:
post_data = {
'description': "Some important file",
'file': file,
}
response = self.client.post(self.test_url, post_data)
self.assertEqual(response.status_code, 302)
document = Document.objects.first()
self.assertEqual(document.description, "My File")
self.assertEqual(document.filename, 'test_file.txt')
When I test the file upload on the actual website, it works. But when I run this test, I get the following error:
django.core.exceptions.SuspiciousFileOperation: Storage can not find
an available filename for "WHJpMYuGGCMdSKFruieo/Documents/eWEjGvEojETTghSVCijsaCINZVxTvzpXNRBvmpjOrYvKAKCjYu/test_file_KTEMuQa.txt".
Please make sure that the corresponding file field allows sufficient
"max_length".
Here is my form save method:
def save(self, commit=True):
instance = super(DocumentForm, self).save(commit=False)
instance.filename = self.cleaned_data['file'].name
if commit:
instance.save() # error occurs here
return instance
Seeing as it works on the actual website, I suspect it has something to do with the way I setup the file in the test; probably something small.
For the sake of brevity, I had removed irrelevant model fields from my original question. But when Ahtisham requested to see the upload_to attribute (which had a custom function), I removed those irrelevant fields and it worked!
So this is my original code (which didn't work) with the irrelevant fields:
def documents_path(instance, filename):
grant = instance.grant
client = grant.client
return '{0}/Grant Documents/{1}/{2}'.format(client.folder_name, grant.name, filename)
....
file = models.FileField(upload_to=documents_path)
But this works:
def documents_path(instance, filename):
return 'Documents/{0}'.format(filename)
The reason why it worked on the actual website is because it wasn't using long characters from the test fixtures. Seems like those fields that I thought were irrelevant for the question were actually quite important!
TL;DR I decreased the length of the custom document path.
I had the same error with an ImageField. The solution was adding max_length=255 in models.py:
image = models.ImageField(upload_to=user_path, max_length=255)
Then running python manage.py makemigrations and python manage.py migrate.
I have two models,Foto and FotoMetadata. Foto just has one property called upload, that is an upload field. FotoMetadata has a few properties and should receive metadata from the foto uploaded at Foto. This can be done manually at the admin interface, but I want to do it automatically, i.e: when a photo is uploaded through admin interface, the FotoMetadata is automatically filled.
In my model.py I have a few classes, including Foto and FotoMetadata:
class Foto(models.Model):
upload = models.FileField(upload_to="fotos")
def __str__(self):
return '%s' %(self.upload)
class FotoMetadata(models.Model):
image_formats = (
('RAW', 'RAW'),
('JPG', 'JPG'),
)
date = models.DateTimeField()
camera = models.ForeignKey(Camera, on_delete=models.PROTECT)
format = models.CharField(max_length=8, choices=image_formats)
exposure = models.CharField(max_length=8)
fnumber = models.CharField(max_length=8)
iso = models.IntegerField()
foto = models.OneToOneField(
Foto,
on_delete=models.CASCADE,
primary_key=True,
)
When I login at the admin site, I have an upload form related to the Foto, and this is working fine. My problem is that I can't insert metadata at FotoMetadata on the go. I made a function that parse the photo and give me a dictionary with the info I need. This function is called GetExif is at a file called getexif.py. This will be a simplified version of it:
def GetExif(foto):
# Open image file for reading (binary mode)
f = open(foto, 'rb')
# Parse file
...
<parsing code>
...
f.close()
#create dictionary to receive data
meta={}
meta['date'] = str(tags['EXIF DateTimeOriginal'].values)
meta['fnumber'] = str(tags['EXIF FNumber'])
meta['exposure'] = str(tags['EXIF ExposureTime'])
meta['iso'] = str(tags['EXIF ISOSpeedRatings'])
meta['camera'] =str( tags['Image Model'].values)
return meta
So, basically, what I'm trying to do is use this function at admin.py to automatically populate the FotoMetadata when uploading a photo at Foto, but I really couldn't figure out how to make it. Does any one have a clue?
Edit 24/03/2016
Ok, after a lot more failures, I'm trying to use save_model in admin.py:
from django.contrib import admin
from .models import Autor, Camera, Lente, Foto, FotoMetadata
from fotomanager.local.getexif import GetExif
admin.site.register(Autor)
admin.site.register(Camera)
admin.site.register(Lente)
admin.site.register(FotoMetadata)
class FotoAdmin(admin.ModelAdmin):
def save_model(self, request, obj, form, change):
# populate the model
obj.save()
# get metadata
metadados = GetExif(obj.upload.url)
# Create instance of FotoMetadata
fotometa = FotoMetadata()
# FotoMetadata.id = Foto.id
fotometa.foto = obj.pk
# save exposure
fotometa.exposure = metadados['exposure']
admin.site.register(Foto, FotoAdmin)
I thought it would work, or that I will have problems saving data to the model, but actually I got stucked before this. I got this error:
Exception Type: FileNotFoundError
Exception Value:
[Errno 2] No such file or directory: 'http://127.0.0.1:8000/media/fotos/IMG_8628.CR2'
Exception Location: /home/ricardo/Desenvolvimento/fotosite/fotomanager/local/getexif.py in GetExif, line 24
My GetExif function can't read the file, however, the file path is right! If I copy and paste it to my browser, it downloads the file. I'm trying to figure out a way to correct the address, or to pass the internal path, or to pass the real file to the function instead of its path. I'm also thinking about a diferent way to access the file at GetExif() function too. Any idea of how to solve it?
Solution
I solved the problem above! By reading the FileField source, I've found a property called path, which solve the problem. I also made a few other modifications and the code is working. The class FotoAdmin, at admin.py is like this now:
class FotoAdmin(admin.ModelAdmin):
def save_model(self, request, obj, form, change):
# populate the model
obj.save()
# get metadata
metadados = GetExif(obj.upload.path)
# Create instance of FotoMetadata
fotometa = FotoMetadata()
# FotoMetadata.id = Foto.id
fotometa.foto = obj
# set and save exposure
fotometa.exposure = metadados['exposure']
fotometa.save()
I also had to set null=True at some properties in models.py and everything is working as it should.
I guess you want to enable post_save a signal
read : django signals
Activate the post_save signal - so after you save a FOTO you have a hook to do other stuff, in your case parse photometa and create a FotoMetadata instance.
More, if you want to save the foto only if fotometa succeed , or any other condition you may use , pre_save signal and save the foto only after meta foto was saved.
Full Disclosure: Cross posted to Tastypie Google Group
I have a situation where I have limited control over what is being sent to my api. Essentially there are two webservices that I need to be able to accept POST data from. Both use plain POST actions with urlencoded data (basic form submission essentially).
Thinking about it in "curl" terms it's like:
curl --data "id=1&foo=2" http://path/to/api
My problem is that I can't update records using POST. So I need to adjust the model resource (I believe) such that if an ID is specified, the POST acts as a PUT instead of a POST.
api.py
class urlencodeSerializer(Serializer):
formats = ['json', 'jsonp', 'xml', 'yaml', 'html', 'plist', 'urlencoded']
content_types = {
'json': 'application/json',
'jsonp': 'text/javascript',
'xml': 'application/xml',
'yaml': 'text/yaml',
'html': 'text/html',
'plist': 'application/x-plist',
'urlencoded': 'application/x-www-form-urlencoded',
}
# cheating
def to_urlencoded(self,content):
pass
# this comes from an old patch on github, it was never implemented
def from_urlencoded(self, data,options=None):
""" handles basic formencoded url posts """
qs = dict((k, v if len(v)>1 else v[0] )
for k, v in urlparse.parse_qs(data).iteritems())
return qs
class FooResource(ModelResource):
class Meta:
queryset = Foo.objects.all() # "id" = models.AutoField(primary_key=True)
resource_name = 'foo'
authorization = Authorization() # only temporary, I know.
serializer = urlencodeSerializer()
urls.py
foo_resource = FooResource
...
url(r'^api/',include(foo_resource.urls)),
)
In #tastypie on Freenode, Ghost[], suggested that I overwrite post_list() by creating a function in the model resource like so, however, I have not been successful in using this as yet.
def post_list(self, request, **kwargs):
if request.POST.get('id'):
return self.put_detail(request,**kwargs)
else:
return super(YourResource, self).post_list(request,**kwargs)
Unfortunately this method isn't working for me. I'm hoping the larger community could provide some guidance or a solution for this problem.
Note: I cannot overwrite the headers that come from the client (as per: http://django-tastypie.readthedocs.org/en/latest/resources.html#using-put-delete-patch-in-unsupported-places)
I had a similar problem on user creation where I wasn't able to check if the record already existed. I ended up creating a custom validation method which validated if the user didn't exist in which case post would work fine. If the user did exist I updated the record from the validation method. The api still returns a 400 response but the record is updated. It feels a bit hacky but...
from tastypie.validation import Validation
class MyValidation(Validation):
def is_valid(self, bundle, request=None):
errors = {}
#if this dict is empty validation passes.
my_foo = foo.objects.filter(id=1)
if not len(my_foo) == 0: #if object exists
foo[0].foo = 'bar' #so existing object updated
errors['status'] = 'object updated' #this will be returned in the api response
return errors
#so errors is empty if object does not exist and validation passes. Otherwise object
#updated and response notifies you of this
class FooResource(ModelResource):
class Meta:
queryset = Foo.objects.all() # "id" = models.AutoField(primary_key=True)
validation = MyValidation()
With Cathal's recommendation I was able to utilize a validation function to update the records I needed. While this does not return a valid code... it works.
from tastypie.validation import Validation
import string # wrapping in int() doesn't work
class Validator(Validation):
def __init__(self,**kwargs):
pass
def is_valid(self,bundle,request=None):
if string.atoi(bundle.data['id']) in Foo.objects.values_list('id',flat=True):
# ... update code here
else:
return {}
Make sure you specify the validation = Validator() in the ModelResource meta.
Hi all as i in learning stage of django so support me.
I have to generate pdf reports in django.I want that the details should be picked from the database and displayed in the pdf document.i am using report lab.
Now have a look at the code
def pdf_view(request):
response = HttpResponse(mimetype='application/pdf')
response['Content-Disposition'] = 'attachment; filename=hello.pdf'
p = canvas.Canvas(response)
details = Data.objects.all()
print details
p.drawString(20, 800, details)
p.drawString(30, 700, "I am a Python Django Professional.")
p.showPage()
p.save()
return response
now as a learning example i have made two fields in models
class Data(models.Model):
first_name = models.CharField(max_length =100,blank=True,null=True)
last_name = models.CharField(max_length =100,blank=True,null=True)
def __unicode__(self):
return self.first_name
and i want that in the pdf document it should display the name s whatever i fill through admin but it is giving me error
'Data' object has no attribute 'decode'
Request Method: GET
Request URL: http://localhost:8000/view_pdf/
Django Version: 1.3
Exception Type: AttributeError
Exception Value:
i want to pik the details from the database and display in the pdf document
'Data' object has no attribute 'decode'
It would have helped if you'd posted the actual traceback.
However I expect the issue is this line:
p.drawString(20, 800, details)
Details is a queryset, that is a list-like container of model instances. It's not a string, and neither does it contain a string. Maybe you want something like:
detail_string = u", ".join(unicode(obj) for obj in details)
which calls the __unicode__ method on every object in your queryset, and joins the resulting list with commas.
I've got a model like this:
class Talk(BaseModel):
title = models.CharField(max_length=200)
mp3 = models.FileField(upload_to = u'talks/', max_length=200)
seconds = models.IntegerField(blank = True, null = True)
I want to validate before saving that the uploaded file is an MP3, like this:
def is_mp3(path_to_file):
from mutagen.mp3 import MP3
audio = MP3(path_to_file)
return not audio.info.sketchy
Once I'm sure I've got an MP3, I want to save the length of the talk in the seconds attribute, like this:
audio = MP3(path_to_file)
self.seconds = audio.info.length
The problem is, before saving, the uploaded file doesn't have a path (see this ticket, closed as wontfix), so I can't process the MP3.
I'd like to raise a nice validation error so that ModelForms can display a helpful error ("You idiot, you didn't upload an MP3" or something).
Any idea how I can go about accessing the file before it's saved?
p.s. If anyone knows a better way of validating files are MP3s I'm all ears - I also want to be able to mess around with ID3 data (set the artist, album, title and probably album art, so I need it to be processable by mutagen).
You can access the file data in request.FILES while in your view.
I think that best way is to bind uploaded files to a form, override the forms clean method, get the UploadedFile object from cleaned_data, validate it anyway you like, then override the save method and populate your models instance with information about the file and then save it.
a cleaner way to get the file before be saved is like this:
from django.core.exceptions import ValidationError
#this go in your class Model
def clean(self):
try:
f = self.mp3.file #the file in Memory
except ValueError:
raise ValidationError("A File is needed")
f.__class__ #this prints <class 'django.core.files.uploadedfile.InMemoryUploadedFile'>
processfile(f)
and if we need a path, ther answer is in this other question
You could follow the technique used by ImageField where it validates the file header and then seeks back to the start of the file.
class ImageField(FileField):
# ...
def to_python(self, data):
f = super(ImageField, self).to_python(data)
# ...
# We need to get a file object for Pillow. We might have a path or we might
# have to read the data into memory.
if hasattr(data, 'temporary_file_path'):
file = data.temporary_file_path()
else:
if hasattr(data, 'read'):
file = BytesIO(data.read())
else:
file = BytesIO(data['content'])
try:
# ...
except Exception:
# Pillow doesn't recognize it as an image.
six.reraise(ValidationError, ValidationError(
self.error_messages['invalid_image'],
code='invalid_image',
), sys.exc_info()[2])
if hasattr(f, 'seek') and callable(f.seek):
f.seek(0)
return f