Use Django views for handling blobstore uploads - django

In stead of using the BlobstoreUploadHandler supplied in AppEngine, I'd prefer to use a Django view, so I can keep all the urls and view functions together. However, I can't find out how to get the blob-key of the uploaded file! (like get_uploads() does for the upload handler). I saw that the BlobstoreUploadHandler uses request.params, but I don't think that is available from Django's Request.
def upload_form(request):
upload_url = blobstore.create_upload_url(reverse(upload_blob))
output = '<html><body>'
output += '<form action="%s" method="POST" enctype="multipart/form-data">' % upload_url)
output += ('''Upload File: <input type="file" name="file"><br> <input type="submit"
name="submit" value="Submit"> </form></body></html>''')
def upload_blob(request):
print request
# How to get the 'blob-key' from request?!
When I examine the request object, all I get is
<WSGIRequest
GET:<QueryDict: {}>,
POST:<QueryDict: {u'submit': [u'Submit']}>
# And COOKIES, META, etcetera
EDIT: Request.FILES
I discovered that some info can be extracted using Request.FILES, which gives:
<MultiValueDict: {u'file': [<InMemoryUploadedFile: my_file (message/external-body)>]}>
However, I assume that the blobstore still handles the file content (is that why it says "content_type=message/external-body"?), so I still need the key somehow. Calling read() gives:
Content-Type: application/octet-stream
MIME-Version: 1.0
Content-Length: 17
Content-MD5: ZmQ3OTJhNjMzNGE0OTAzNGU4NjE5MDNmMGEwNjliMGE=
content-type: application/octet-stream
content-disposition: form-data; name="file"; filename="a1_blob"
X-AppEngine-Upload-Creation: 2012-02-12 22:11:49.643751
So it looks like AppEngine actually replaced the file content by this descriptor, but still, where does AppEngine put the key?
I'm starting to suspect that the blob-key is just lost when not using the webapp framework, since the UploadedFile object has no key() method.

It took me a long time to find, but the content_type: message/external-body requires extra parameters, to find the actual file, in AppEngine's case, this is the blob-key. However, Django doesn't support these extra content_type parameters, so they are indeed lost in the process. There seems to be a patch, but I don't think it's in the AppEngine Django version yet.
https://code.djangoproject.com/ticket/13721

I had the same problem yesterday. Thanks to your post I realizad that the problem was django and his class views. I finally use a code that I have since 2011 and it still works. It does not use BlobstoreUploadHandler, but it gets the blob_infos from the request after automatically upload it to blobstore.
You can use that function in the next way from your callback django function or class (I finally did not try it in a class view from Django but I think it will work. Currently I'm using it in a function view from Django with its request):
media_blobs = get_uploads(request, populate_post=True)
The function is the next:
import cgi
from google.appengine.ext import blobstore
def get_uploads(request, field_name=None, populate_post=False):
"""Get uploads sent to this handler.
Args:
field_name: Only select uploads that were sent as a specific field.
populate_post: Add the non blob fields to request.POST
Returns:
A list of BlobInfo records corresponding to each upload.
Empty list if there are no blob-info records for field_name.
"""
if hasattr(request,'__uploads') == False:
request.META['wsgi.input'].seek(0)
ja = request.META['wsgi.input']
fields = cgi.FieldStorage(request.META['wsgi.input'], environ=request.META)
request.__uploads = {}
if populate_post:
request.POST = {}
for key in fields.keys():
field = fields[key]
if isinstance(field, cgi.FieldStorage) and 'blob-key' in field.type_options:
request.__uploads.setdefault(key, []).append(blobstore.parse_blob_info(field))
elif populate_post:
if isinstance(field, list):
request.POST[key] = [val.value for val in field]
else:
request.POST[key] = field.value
if field_name:
try:
return list(request.__uploads[field_name])
except KeyError:
return []
else:
results = []
for uploads in request.__uploads.itervalues():
results += uploads
return results
The last function is not mine. I do not remember where I got it three or four years ago. But I think it will help someone.
UPDATE:
You also can use a view handler of webapp.WSGIApplication and at the same time use django. This way will allow you to use BlobstoreUploadHandler and BlobstoreDownloadHandler (for video stream as example). You only need to add the view class in main.py and create its handler:
class ServeVideoHandler(blobstore_handlers.BlobstoreDownloadHandler):
def get(self, resource):
...
downloader_handler = webapp.WSGIApplication([('/pathA/pathB/([A-Za-z0-9\-\=_]+)', ServeVideoHandler),], debug=True)
And in app.yaml add the handler before the script main.application that contains your django app.
- url: /pathA/pathB/(.+)
script: main.downloader_handler

The key info isn't directly in the file, it's in file.blobstore_info.key()
post your form containing your image to a url created using blobstore.create_upload_url():
from google.appengine.ext import blobstore
upload_url = blobstore.create_upload_url('/add_image/')
the created url will save the image in the blobstore and redirect the request (with modified file object) to /add_image/
define a url pattern and view for /add_image/ and handle the image:
def add_action_image(request):
image = request.data.get('image')
image_key = image.blobstore_info.key()
... addl' logic to save a record with the image_key...

As you noted, BlobstoreUploadHandler is open source, so you can see the logic they use to parse the key out of the request params. Note that request.params just includes variables from both the query string and the request body (for POST requests). So you might want to start with your djnago request's REQUEST object.

Related

Can I return a different Django Response based on whether or not the view is invoked from swagger?

I am using Django-Rest-Framework for my API. And I am documenting that API w/ swagger using drf_yasg. One of my views returns a Django FileResponse.
When I access the view directly at "http://localhost:8000/api/my_documents/1" it successfully displays the (PDF) file. But when I access it via swagger it successfully returns a 200 but gives the following message:
Unrecognized response type; displaying content as text.
This is b/c of this issue in swagger itself. As suggested in that ticket, the problem goes away if I change the "Content-Disposition" response header from "inline" to "attachment". However, I don't want to always download the file.
My question is: Can I determine whether the request was made by swagger in the view and conditionally change the headers? Something like:
class MyDocumentView(GenericAPIVIew):
def get(self, request, pk):
my_document = MyDocument.objects.get(pk=pk)
response = FileResponse(my_document.file) # (file is a FileField)
# WHAT DO I PUT HERE ?!?
if request.is_from_swagger:
response.headers["Content-Disposition"] = response.headers["Content-Disposition"].replace("inline", "attachment")
return response
Thanks.
You can do this by checking the request.headers.get('referer'). There should be your swagger url in there.

Issue with django rest framework while handling url-encoded data

I am building a REST API server that handles POST requests. The content type in the request is "application/x-www-form-urlencoded".In the request body, we are sending "data1" (some string) and "image" ( a file)
Here's the sample inputForm code I have:
from django import forms
class RequestForm(forms.Form):
data1= forms.CharField(label='data1',max_length=10000)
image = forms.ImageField()
I then validate the content in the form request:
if request.method == 'POST':
form = RequestForm(request.POST)
print("Form content: {0}".format(form))
if form.is_valid():
print("Works")
else:
print("Issue")
Now, when I send the above mentioned data, I always get an error. It prints "Issue". In addition, the line taht prints form content shows it as an error. Something like:
<ul class="errorlist"><li>This field is required.</li></ul><input type="text" name="data1" maxlength="10000"
One interesting point: if I remove "Content-type" from the request header, it works.
Any inputs on how I can read the form data correctly when we use content type as application/x-www-form-urlencoded.
thanks in advance...
As per Django Forms documentation:
By default, each Field class assumes the value is required, so if you pass an empty value – either None or the empty string ("") – then clean() will raise a ValidationError exception
You are on the right track, you should send the form as multipart/form-data as per this thread: Thread about form Content types
Found the solution. To begin with, I am sending a file in the input. So I should use content type as "multipart-formdata".
In addition, I am using Postman to pump in the REST API requests. In the body of the request, I set form-data which automatically sets the headers correctly based on what I am sending in the body. I was trying to override it with my own header which is not right.
When I resent my http POST request with no headers in Postman, it worked. (of course, I did verify the final http request itself and confirmed Postman is setting the header correctly)

Django: call REST API through a view instead of Postman

I have a working python code on my desktop that prints and makes PDFs perfectly. All I want to do is use that code and use Django to allow users to enter a value.
My code uses docusign API to call data. I use postman which needs a key and other parameters to use the API. The value entered by my user will determine what data they get.
What I think I have to do is rewrite my code, put it somewhere, then turn it into a view. The view will be sent to template.
Edit -
My code:
# Get Envelope Data- use account ID from above
# Get Todays Date, Daily Setting
day = datetime.datetime.today().strftime('%Y-%m-%d')
url = "https://demo.docusign.net/restapi/v2/accounts/" + accountId + "/envelopes"
# if Envelope is completed
querystring = {"from_date": Date, "status": "completed"}
headers = {
'X-DocuSign-Authentication': "{\"Username\":\""+ email +"\",\"Password\":\""+Password+"\",\"IntegratorKey\": \""+IntegratorKey+"\"}",
'Content-Type': "application/json",
'Cache-Control': "no-cache",
'Postman-Token': "e53ceaba-512d-467b-9f95-1b89f6f65211"
}
response = requests.request("GET", url, headers=headers, params=querystring)
envelopes = response.text
Sorry, let me try again. I currently have a python3 program on my desktop. I run it with idle and everything is how I want it.
What I want to do with Django is use this code to print its outputs on a webpage and have the user download it’s additional csv file output. I have managed to make a Django localhost and I am stuck at that point. I do not know how to use my python3 code to run to webpage.
The code is made up of API calls, I use postman to help me with sending the right parameters. I will add a picture of code. All I want is for user to enter value such as accountID so that the API can complete the request and give them data for their own request.
I'll try to give you a overview of how this could work with Django.
You could have a form to obtain the users account_id.
class AccountForm(forms.Form):
account_id = forms.IntegerField()
You could display this form through a generic FormView (see also this):
class AccountView(views.FormView):
form_class = AccountForm
template_name = 'account.html'
def form_valid(self, form):
# here you make your request to the external API
account_id = form.cleaned_data['account_id']
url = "https://demo.docusign.net/restapi/v2/accounts/" + account_id + "/envelopes"
headers = ...
querystring = ...
resp = requests.request("GET", url, headers=headers, params=querystring)
ctx = {
'result': resp.text,
}
return render(self.request, 'result.html', ctx)
I don't show the template account.html here. You will have to figure that one out yourself; the links I provided should point you in the right direction.
Now, what remains to be determined is what exactly the method form_valid should return. The code I showed renders a template with the API call response in the context, so in your template result.html you could display the result data any way you like.
You mentioned downloading a CSV file as well. That could be a different view, probably triggered by a link or button in result.html.

Sending form to another view django

I am building a website and I want various views that will ask the user to request a quote from our page. I want to keep the code as DRY as possible so I am writing a view quote which will receive the quote requests from various views and, if there is a validation error redirect back to the page that made the request. I managed to solve this using the super bad practice 'global variables'. I need a better solution, I would like redirecting to respective view with the current form so I can iterate through the form.errors. Here is my code:
def send_quote(request):
form = Quote(request.POST)
if form.is_valid():
# do stuff when valid
return redirect('Support:thanks', name=name or None)
quote_for = request.POST['for_what']
global session_form
session_form = form
return redirect('Main:' + quote_for) # Here I would like to send form instead of storing in global variable`
You can use the HttpResponseRedirect function, and pass as argument the page that made the request.
return HttpResponseRedirect(request.META.get('HTTP_REFERER'))
All the META data is store on a dictionary, if you want to learn more check the documentation.
https://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpRequest.META
If you redirect to the referrer, form.errors will be empty, as redirection is always a GET request.
I can think of two solutions to your problem:
Submit forms asynchronously using JavaScript and so populate the errors
Make all the views containing the form support POST - one way to do this would be to create a base class that inherits from FormView
The second option is a typical way of handling forms in Django - you process both POST and GET inside the same view.
After two days of searching I finally found the answer. Instead of saving form in request.session I just save request.POST and then redirect. Here is the code:
def send_quote(request):
form = Quote(request.POST)
if form.is_valid():
# do stuff when valid
return redirect('Support:thanks', name=name or None)
quote_for = request.POST['for_what']
request.session['invalid_form'] = request.POST
return redirect('Main:endview')
def endview(request):
session_form = request.session.pop('invalid_form', False)
if session_form:
form = Quote(session_form)
# render template again with invalid form ;)
Now I can repeat this with all the views I want and just change the what_for input of each form to match the respective view (Like I intended).

Getting Google App Engine blob info in Django view

This is a follow up question for Django on Google App Engine: cannot upload images
I got part of the upload of images to GAE Blobstore working. Here's what I did:
In models.py I created a model PhotoFeature:
class PhotoFeature(models.Model):
property = models.ForeignKey(
Property,
related_name = "photo_features"
)
caption = models.CharField(
max_length = 100
)
blob_key = models.CharField(
max_length = 100
)
In admin.py I created an admin entry with an override for the rendering of the change_form to allow for insert of the correct action to the Blobstore upload url:
class PhotoFeatureAdmin(admin.ModelAdmin):
list_display = ("property", "caption")
form = PhotoFeatureForm
def render_change_form(self, request, context, *args, **kwargs):
from google.appengine.ext import blobstore
if kwargs.has_key("add"):
context['blobstore_url'] = blobstore.create_upload_url('/admin/add-photo-feature')
else:
context['blobstore_url'] = blobstore.create_upload_url('/admin/update-photo-feature')
return super(PhotoFeatureAdmin, self).render_change_form(request, context, args, kwargs)
As I use standard Django, I want to use the Django views to process the result once GAE has updated the BlobStore in stead of BlobstoreUploadHandler. I created the following views (as per the render_change_form method) and updated urls.py:
def add_photo_feature(request):
def update_photo_feature(request):
This all works nicely but once I get into the view method I'm a bit lost. How do I get the Blob key from the request object so I can store it with PhotoFeature? I use standard Django, not Django non-rel. I found this related question but it appears not to contain a solution. I also inspected the request object which gets passed into the view but could not find anything relating to the blob key.
EDIT:
The Django request object contains a FILES dictionary which will give me an instance of InMemoryUploadedFile. I presume that somehow I should be able to retrieve the blob key from that...
EDIT 2:
Just to be clear: the uploaded photo appears in the Blobstore; that part works. It's just getting the key back from the Blobstore that's missing here.
EDIT 3:
As per Daniel's suggestion I added storage.py from the djangoappengine project which contains the suggested upload handler and added it to my SETTINGS.PY. This results in the following exception when trying to upload:
'BlobstoreFileUploadHandler' object has no attribute 'content_type_extra'
This is really tricky to fix. The best solution I have found is to use the file upload handler from the djangoappengine project (which is associated with django-nonrel, but does not depend on it). That should handle the required logic to put the blob key into request.FILES, as you'd expect in Django.
Edit
I'd forgotten that django-nonrel uses a patched version of Django, and one of the patches is here to add the content-type-extra field. You can replicate the functionality by subclassing the upload handler as follows:
from djangoappengine import storage
class BlobstoreFileUploadHandler(storage.BlobstoreFileUploadHandler):
"""Handler that adds blob key info to the file object."""
def new_file(self, field_name, *args, **kwargs):
# We need to re-process the POST data to get the blobkey info.
meta = self.request.META
meta['wsgi.input'].seek(0)
fields = cgi.FieldStorage(meta['wsgi.input'], environ=meta)
if field_name in fields:
current_field = fields[field_name]
self.content_type_extra = current_field.type_options
super(BlobstoreFileUploadHandler, self).new_file(field_name,
*args, **kwargs)
and reference this subclass in your settings.py rather than the original.