In Django, I want to stop any file uploads that don't end in my designated extension as soon as they're received. To do this, I define a custom Upload Handler and write new_file to look something like this:
def new_file(self, field_name, file_name, content_type, content_length, charset=None):
basename, extension = os.path.splitext(file_name)
if extension != ".txt":
raise StopUpload(True) # Error: Only .txt files are accepted
This works fine for stopping the upload but it also clears request.FILES. Now, when my view receives the request, it has no way to telling whether the upload handler caused the file to be missing and I can't display a useful message to the user.
Is there any way to propagate messages from the Upload Handler to the corresponding view, such that I can display the error to the user? I've tried using the request object, but it is immutable.
I've found a reason for why I couldn't get it working and a solution
as well.
It would appear that Django uses lazy evaluation for request.FILES to
determine when the upload handler is called. Therefore, the upload
handler is only evoked when and if you attempt to access
request.FILES. Additionally, the request object I am using
(WSGIRequest in my case) has made GET and POST immutable dictionaries,
so we can't pass information through there. However, META is still
available to add information to.
My combined solution has the line "request.FILES" in the view that
handles uploads, which forces the upload handler to begin. When the
error is captured in new_files, I set self.request.META['error'] to
the error message and raise StopUpload, which pushes us back into the
view without a file. Finally, I check for request.META['error'] in the
view and display that message when there is a problem.
I hope this helps!
Related
Today I learned Flask's flash() method.
What I understood was, it tells template to show message once.
But it can also be done by render_template() method like below:
#app.route('/')
def index():
if condition:
error = "Some error messages to flash"
else:
error = None
return render_template('index.html', error=error)
This makes flash() method look like an unnecessary duplicate.
What kind of advantages or difference in practice are there to use flash over this?
When you issue a redirect to, say, /profile, how would you know if the user visited that page himself, or he's redirected as a result of an action (like after a login)?
flash() method saves the message to the user session, so that they don't get lost due to navigation.
You can't use render_template(name, message="success") in this case, because when you redirect the user, the browser doesn't care about the response, when it sees a 30x response, it reads the Location header and loads that URL in a new request, so whatever context you've set, or whatever you render, is all discarded. That's why you need to flash messages to pass messages between two consequent requests.
Also, read the explanation in flask docs, it's quite clear.
https://flask.palletsprojects.com/en/2.0.x/patterns/flashing/
I have a class-based view which triggers the composition and downloading of a report for a user.
Normally in def get of the class I just compile the report, add response['Content-Disposition'] = 'attachment; filename="somefilename.pdf"' and return response to a user.
The problem is that some reports are large and while they are compiling the request timeout happens.
I know that the right way of dealing with this is to delegate it to a background process (like Celery). But the problem is that it means that instead of creating a temporary file which ceases to exist the moment the user downloads a report, I have to store these reports somewhere, and write a cronjob which will regularly clean the reports directory.
Is there any more elegant way in Django to deal with this issue?
One solution less fancy than using celery is to use is Django's StreamingHttpResponse:
(https://docs.djangoproject.com/en/2.0/ref/request-response/#django.http.StreamingHttpResponse
With this, you use a generator function, which is a python function that uses yield to return its results as an iterator. This allows you to return the data as you generate it, rather than all at once at after you're finished. You can yield after each line or section of the report.. thus keeping a flow of data back to the browser.
But.. this only works if you are building up the finished file bit by bit.. for example, a CSV file. If you're returning something that you need to format all at once, for example if you're using something like wkhtmltopdf to generate a pdf file after you're done, then it's not as easy.
But there's still a solution:
What you can do in that case is, use StreamingHttpReponse along with a generator function to generate your report into a temporary file, instead of back to the browser. But as you are doing this, yield HTML snippets back to the browser which lets the user know the progress, eg:
def get(self, request, **kwargs):
# first you need a tempfile name.. do that however you like
tempfile = "kfjkdsjfksjfks"
# then you need to create a view which will open that file and serve it
# but I won't show that here.
# For security reasons it has to serve only out of one directory
# that is dedicated to this.
fetchurl = reverse('reportgetter_url') + '?file=' + tempfile
def reportgen():
yield 'Starting report generation..<br>'
# do some stuff to generate your report into the tempfile
yield 'Doing this..<br>'
# do this
yield 'Doing that..<br>'
# do that
yield 'Finished.<br>'
# when the browser receives this script, it'll go to fetchurl where
# you will send them the finished report.
yield '<script>document.location="%s";</script>' % fetchurl
return http.StreamingHttpResponse(reportgen())
That's not a complete example obviously, but should give you the idea.
When your user fetches this view, they will see the progress of the report as it comes along. At the end, you're sending the javacript which redirect the browser to the other view you will have to write which returns the response containing the finished file. When the browser gets this javacript, if the view returning the tempfile is setting the response Content-Disposition as an attachment before returning it, eg:
response['Content-Disposition'] = 'attachment; filename="%s"' % filename
..then the browser will stay on the current page showing your progress.. and simply pop up a file save dialog for the user.
For cleanup, you'll need a cron job regardless.. because if people don't wait around, they'll never pick up the report. Sometimes things don't work out... So you could just clean up files older than let's say 1 hour. For a lot of systems this is acceptable.
But if you want to clean up right away, what you can do, if you are on unix/linux, is to use an old unix filesystem trick: Files which are deleted while they are open are not really gone until they are closed. So, open your tempfile.. then delete it. Then return your response. As soon as the response has finished sending, the space used by the file will be freed.
PS: I should add.. that if you take this second approach, you could use one view to do both jobs.. just:
if `file` in request.GET:
# file= was in the url.. they are trying to get an already generated report
with open(thepathname) as f:
os.unlink(f)
# file has been 'deleted' but f is still a valid open file
response = HttpResponse( etc etc etc)
response['Content-Disposition'] = 'attachment; filename="thereport"'
response.write(f)
return response
else:
# generate the report
# as above
This is not really a Django question but a general architecture question.
You can always increase your server time outs but it would still, IMO, give you a bad user experience if the user has to sit watching the browser just spin.
Doing this on a background task is the only way to do it right. I don’t know how large the reports are, but using email can be a good solution. The background task simply generates the report, sends it via email and deletes it.
If the files are too large to send via email, then you will have to store them. Maybe send an email with a link and a message indicating the link will not work after X days/hours. Once you have a background worker, creating a daily or hourly clean up task would be very easy.
Hope it helps
rest API:
someting/post expects 'token' as bytearray body data
something/delete expects 'token' as bytearray body data
Using Qt I can prepare the data in a QByteArray and send via deleteResource (that doesn't accepts a data parameter) and I can use sendCustomRequest that accepts a data parameter, but if I use the later with DELETE I have no data.
With POST, I do have the data.
Minimal code example, python server - just to exemplify. the Qt code is below.:
#route('/something/delete', "DELETE")
def somethingDelete(url, post):
print(post) # empty
#route('/something/delete2', "POST")
def somethingDelete2(url, post):
print(post) # correct output.
and the Qt code that triggers the server calls - This code is higly shortened to simplify, but the idea is that.
QNetworkRequest req;
req.setRawHeader("OCS-APIREQUEST", "true");
req.setUrl = Utility::concatUrlPath(account()->url(), path());
QByteArray bufferData("token=" + _token);
sendCustomRequest(req, "POST", bufferData);
as soon as I change the POST to DELETE, I don't get the token, but the correct python function is executed.
The DELETE HTTP verb does not have a request body so your buffer is probably simply dropped by Qt. To use DELETE you would need to encode your token in the URL.
As of Qt 5.9.2, it seems that Qt might ignore body data when performing a DELETE operation.
In Qt code in QNetworkReplyHttpImplPrivate::postRequest(), one can see that createUploadByteDevice() is not called when the operation is QNetworkAccessManager::DeleteOperation.
However, this is only valid when the DELETE request is sent by calling QNetworkAccessManager::deleteResource(), which is the only way to create a network request with the QNetworkAccessManager::DeleteOperation operation. Also note that this function does not allow you to send any body data.
If you use QNetworkAccessManager::sendCustomRequest() to send the request, then as far as Qt is concerned the operation is QNetworkAccessManager::CustomOperation. The custom verb you pass is not processed further, and Qt will behave exactly the same whatever the value of verb is. Even if verb is a known value like POST or DELETE.
This means that Qt does not discard the body data.
So if you used QNetworkAccessManager::sendCustomRequest(), as you claim, your body data is sent to the server (confirmed by Wireshark). So the issue is not on Qt side, but on the server side.
I have an application that enforces a strict page sequence. If the user clicks the Back button, the application detects an out-of-order page access and sends the user back to the start.
I'd like to make this a bit more friendly, by redirecting the user back to the correct page and displaying a pop-up javascript alert box telling them not to use the Back button.
I'm already using a function that does a lot of validity checking which returns None if the request is okay, or an HttpResponseRedirect to another page (generally the error page or login page) if the request is invalid. All of my views have this code at the top:
response = validate(request)
if response:
return response
So, since I have this validate() function already, it seems like a good place to add this extra code for detecting out-of-order access.
However, since the out-of-order detection flag has to survive across a redirect, I can't just set a view variable; I have to set the flag in the session data. But of course I don't want the flag to be set in the session data permanently; I want to remove the flag from the session data after processing the template.
I could add code like this to all of my render calls:
back_button = request.session.get('back_button', False)
response = render(request, 'foo.html', { 'back_button': back_button } )
if back_button:
del request.session['back_button']
return response
But this seems a bit messy. Is there some way to automatically remove the session key after processing the template? Perhaps a piece of middleware?
I'm using function-based views, not class-based, btw.
The session object uses the dictionary interface, so you can use pop instead of get to retrieve and delete the key at the same time:
back_button = request.session.pop('back_button', False)
(This is the first time I've done this actually.)
<mx:HTTPService id="post_update" method="POST" result="{Dumper.info('bye')}"/>
The result handler above is just for debugging purposes, but its never hit, even though what I'm uploading via POST...
post_update.url = getPath(parentDocument.url)+"update";
post_update.send(new_sel);
...is received and handled successfully by my Django view:
def wc_post(request) :
request.session['wc'] = request.POST
return http.HttpResponse("<ok/>", mimetype="text/xml")
As far as what I'm sending back from Django, I'm following the guidelines here:
Sending Images From Flex to a Server
I just don't want it to generate an error on the Flex side considering Django is actually receiving and processing the data. Any help appreciated. Can't remember the text of the error in Flex at the moment.
UPDATE: new_sel (what I'm posting from Flex) is just a Flex Object, with various text fields.
UPDATE: various error messages from event.message (in fault handler):
faultCode = "Server.Error.Request"
faultString = "HTTP request error"; DSStatusCode = 500; errorID = 2032; type = "ioError"
This is more grasping at straws than answers, but do I have to send a particular type of header back from Django- the default sent by Django includes a 200 success status code, and the response I was sending of "<ok/>" with mime type of "text/xml" was following the example exactly that I provided from that other source.
And also the url I'm sending the POST to is localhost:8000/wr_view1/wr_webcube/update, and I previously successfully did a GET to localhost:8000/wr_view1/wr_webcube/webcube.xml, and despite the .xml extension in the case of GET, it was still being handled by Django (and without errors in Flex). In the case of this POST, once again, the data is actually succesfully sent and handled by Django, but Flex is returning Error 2032, which I found out can mean numerous different things including cross domain issues, but don't see how that's the case here.
Just had to return HttpResponse("ok") Didn't like it being sent as xml for some reason. So much ado about nothing I guess.