I have a following situation. I created a simple backend in Flask that handles file uploads. With files received, Flask does something (uploads them), and returns the data to the caller. There are two scenarios with the app, to upload one image and multiple images. When uploading one image, I can simply get the response and voila, I'm all set.
However, I am stuck on handling multiple file uploads. I can use the same handler for the actual file upload, but the issue is that all of those files need to be stored into a list or something, then processed, and after doing that, a single link (album) containing all those images, needs to be delivered.
Here is my upload handling code:
#app.route('/uploadv3', methods=['POST'])
def upload():
if request.method == 'POST':
data_file = request.files["file"]
file_name = data_file.filename
path_to_save_to = os.path.join(app.config['UPLOAD_FOLDER'], file_name)
data_file.save(path_to_save_to)
file_url = upload_image_to_image_host(path_to_save_to)
return file_url
I was experimenting with session in flask, but I dont know can I create a list of items under one key, like session['links'], and then get all those, and clear it after doing the work. Or is there some other simpler solution?
I assume that I could probably do this via key for each image, like session["link1"], and so on, but that would impose a limit on the images (depending on how much of those I create), would make the code very ugly, make the iteration over each in order to generate a list that is passed to an album building method problematic, and session clearing would be tedious.
Some code that I wrote for getting the actual link at the end and clearing the session follows (this assume that session['link'] has a list of urls, which I can't really achieve with my knowledge of session management in Flask:
def create_album(images):
session.pop('link', None)
new_album = im.create_album(images)
return new_album.link
#app.route('/get_album_link')
def get_album_link():
return create_album(session['link'])
Thanks in advance for your time!
You can assign anything to a session including individual value or list/dictionary etc. If you know the links, you can store them in the session as follows:
session['links'] = ['link1','link2'...and so on]
This way, you have a list of all the links. You can now access a link by:
if 'links' in session:
for link in session['links']:
print link
Once you are done with them, you can clear the session as:
if 'links' in session:
del session['links']
To clarify what I have done to make this work. At the end, it appeared that the uploading images and adding them to the album anonymously had to be done "reversely", so not adding images to an album object, but uploading an image object to an album id.
I made a method that gets the album link and puts it in the session:
#app.route('/get_album_link')
def get_album_link():
im = pyimgur.Imgur(CLIENT_ID)
new_album = im.create_album()
session.clear()
session['album'] = new_album.deletehash
session['album_link'] = new_album.link
return new_album.link
Later on, when handling uploads, I just add the image to the album and voila, all set :)
uploaded_image = im.upload_image(path_of_saved_image, album=session['album'])
file_url = uploaded_image.link
return file_url
One caveat is that the image should be added to the "deleteahash" value passed as the album value, not the album ID (which is covered by the imgur api documentation).
Related
I have a Django app where users can upload images and can have a processed version of the images if they want. and the processing function returns the path, so my approach was
model2.processed_image = processingfunction( model1.uploaded_image.path)
and as the processing function returns path here's how it looks in my admin view
not like the normally uploaded images
In my machine it worked correctly and I always get a 404 error for the processed ones while the normally uploaded is shown correctly when I try to change the url of the processed from
myurl.com/media/home/ubuntu/Eyelizer/media/path/to/the/image
to
myurl.com/media/path/to/the/image
so how can I fix this ? is there a better approach to saving the images manually to the database ?
I have the same function but returns a Pil.image.image object and I've tried many methods to save it in a model but I didn't know how so I've made the function return a file path.
I think the problem is from nginx where I define the media path.
should/can I override the url attribute of the processedimage?
making something like
model.processed_image.url = media/somefolder/filename
Instead of using the PIL Image directly, create a django.core.files.File.
Example:
from io import BytesIO
from django.core.files import File
img_io = BytesIO() # create a BytesIO object to temporarily save the file in memory
img = processingfunction( model1.uploaded_image.path)
img.save(img_io, 'PNG') # save the PIL image to the BytesIO object
img_file = File(thumb_io, name='some-name.png') # create the File object
# you can use the `name` from `model1.uploaded_image` and use
# that above
# finally, pass the image file to your model field
model2.processed_image = img_file
To avoid repetition of this code, it would be a good idea to keep this code in processingfunction and return the File object directly from there.
My approach is a bit different from #Xyres's, I thought xyres's would make a duplicate of the existing image and create a new one and when I tried overriding the URL attribute it returned an error of
can't set the attribute
but when I saw this question and this ticket I tried making this and it worked
model2.processed_image = processingfunction(model1.uploaded_image.path)
full_path = model2.processed_image.path
model2.processed_image.name = full_path.split('media')[1]
so that explicitly making the URL media/path/to/image and cut out all of the unneeded parts like home/ubuntu and stuff
I have a huge list that will be generated dynamically from a csv file in a django view, but i had a requirement to use that list in the next view, so i thought to give a try on django sessions
def import_products(request):
if request.method == 'POST':
if request.FILES:
csv_file_data = ...........
total_records = [row for row in csv_file_data]
request.session['list_data'] = total_records
# total_records is `list of lists` of length more than 150
# do some processing with the list and render the page
return render_to_response('product/import_products.html')
def do_something_with_csv_data_from_above_view(request):
data = request.session['list_data']
# do some operations with list and render the page
return render_to_response('website/product/success.html',
So as mentioned in the above, i need to use the total_records huge list in the do_something_with_csv_data_from_above_view view and delete the session by storing it in another variable
So actually how to implement/use exactly the sessions concept(i have read the django docs but could n't able to get the exact concept)
In my case,
When a user tries to upload the csv file each time, i am reading the data and storing
the data as list to session
==> Is this the right way to do so ? also i want to store the session variable in
database concept
Because the list was huge, i need to delete it for sure in the next view when i
copied it in to another variable
Am i missing anything, can anyone please implement exact code for my above scenario ?
You have two options:
Client side RESTful - This is a RESTful solution but may require a bit more work. You can retrieve the data in the first request to the client and after selecting you can send the selected rows back to the server for processing, CSV etc.
Caching: In the first request you can cache your data on the server using a django file system or memcached. In the second request use the cache key (which would be the user session key + some timestamp + whatever else) to fetch the data and store in the db.
If it's a lot of data option 2 may be better.
I may have overcomplicated things.
I have two views. The first view generates a bunch of temporary data based on the user's input from the form. Each of the generated data contains a name and misc data. I want to pass only the names to the template to be rendered as a list of hyperlinks. If the user clicks on one of them, the second view should be given the specific name the user clicked on so that the view can manipulate it. The only problem is, I don't know how to get the misc data associated with the name.
The misc data generated could contain random characters that's not a standard character in URLs, so I can't turn misc into a hyperlink like I can with just the name.
I have something like this:
views:
# Displays the temp data names
def display(request):
return render_to_response('display.html',{},context_instance=RequestContext(request))
# User provides input, generate temp data to be displayed as hyperlinks
def search(request):
form = SearchForm(request.POST)
if form.is_valid():
usr_input = form.cleaned_data['input']
data = generate_data(usr_input) # generates a list of (name, misc) data.
request.session['hyperlinks'] = get_list_names(data) # returns only names in data
return HttpResponseRedirect('views.display')
else:
....
# User has clicked on a hyperlink, we must process specific data given its name.
def process_data(request, name):
# How to get associated misc data created from search()?
I haven't written the template yet, but the idea is:
template:
{% for name_link in request.session.hyperlinks %}
<a href={% url process name_link %}>
{% endfor %}
One solution could be creating a bunch of session variables:
for name in get_list_names(data):
request.session[name] = // associated misc data
But this seems like a waste. Plus I'd have to manage deleting the session variable later on since this is only temporary data generated based on user input. A new input from the user would create another huge horde of session variables!
Another solution could be to store it temporarily in the database, but that also seems like a bad idea.
EDIT - Trying out suggestion by christophe31:
I'm not quite sure if I understand your suggestion, but is it something like this?
data_dict = {name1:misc1, name2:misc2, etc...}
encoded = urllib.urlencode(data_dict) # encoded = 'name1=misc1&name2:misc2...etc'
request.session['hyperlinks'] = encoded
A few questions on this though:
1) Wouldn't encoding it using urllib defeat the purpose of having a dictionary? It returns a string rather than a dictionary
2) To expand on (1), what if the misc data had '&' and '=' in it? It would screw up parsing which is the key and value by the second view. Also, misc data may have unusual characters, so allowing that to be part of the url to be displayed may be bad.
3) Does Django protect from allowing the user to maliciously modify the session misc data so that the misc data generated from the first view may be different than the one passed to the second view? That would be a problem!
You may want to put a dictionary as a session variable, set a cookie, or pass as get argument throught the link your data.
For me you have to put all these data in a dictionary before export it as get parameters (with urllib2) or store it in your user's session.
Ask me if you want more info on a suggested way.
Edit:
They are 2 ways I see, by session:
data_dict = {name1:misc1, name2:misc2, etc...}
request.session['hyperlinks'] = data_dict
Or passing to the template the data if no session backend:
data_dict = {name1:misc1, name2:misc2, etc...}
encoded = urllib.urlencode(data_dict)
return render(request, "my_template.html", {"url_params":encoded,}
and
Go to results
How can I show a please wait loading message from a django view?
I have a Django view that takes significant time to perform calculations on a large dataset.
While the process loads, I would like to present the user with a feedback message e.g.: spinning loading animated gif or similar.
After trying the two different approaches suggested by Brandon and Murat, Brandon's suggestion proved the most successful.
Create a wrapper template that includes the javascript from http://djangosnippets.org/snippets/679/. The javascript has been modified: (i) to work without a form (ii) to hide the progress bar / display results when a 'done' flag is returned (iii) with the JSON update url pointing to the view described below
Move the slow loading function to a thread. This thread will be passed a cache key and will be responsible for updating the cache with progress status and then its results. The thread renders the original template as a string and saves it to the cache.
Create a view based on upload_progress from http://djangosnippets.org/snippets/678/ modified to (i) instead render the original wrapper template if progress_id='' (ii) generate the cache_key, check if a cache already exists and if not start a new thread (iii) monitor the progress of the thread and when done, pass the results to the wrapper template
The wrapper template displays the results via document.getElementById('main').innerHTML=data.result
(* looking at whether step 4 might be better implemented via a redirect as the rendered template contains javascript that is not currently run by document.getElementById('main').innerHTML=data.result)
Another thing you could do is add a javascript function that displays a loading image before it actually calls the Django View.
function showLoaderOnClick(url) {
showLoader();
window.location=url;
}
function showLoader(){
$('body').append('<div style="" id="loadingDiv"><div class="loader">Loading...</div></div>');
}
And then in your template you can do:
This will take some time...
Here's a quick default loadingDiv : https://stackoverflow.com/a/41730965/13476073
Note that this requires jQuery.
a more straightforward approach is to generate a wait page with your gif etc. and then use the javascript
window.location.href = 'insert results view here';
to switch to the results view which starts your lengthy calculation. The page wont change until the calculation is finished. When it finishes, then the results page will be rendered.
Here's an oldie, but might get you going in the right direction: http://djangosnippets.org/snippets/679/
A workaround that I chose was to use beforunload and unload events to show the loading image. This can be used with or without window.load. In my case, it's the view that is taking a great amount of time and not the page loading, hence I am not using window.load (because it's already a lot of time by the time window.load comes into picture, and at that point of time, I do not need the loading icon to be shown anymore).
The downside is that there is a false message that goes out to the user that the page is loading even when when the request has not even reached the server or it's taking much time. Also, it doesn't work for requests coming from outside my website. But I'm living with this for now.
Update: Sorry for not adding code snippet earlier, thanks #blockhead. The following is a quick and dirty mix of normal JS and JQuery that I have in the master template.
Update 2: I later moved to making my view(s) lightweight which send the crucial part of the page quickly, and then using ajax to get the remaining content while showing the loading icon. It needed quite some work, but the end result is worth it.
window.onload=function(){
$("#load-icon").hide(); // I needed the loading icon to hide once the page loads
}
var onBeforeUnLoadEvent = false;
window.onunload = window.onbeforeunload= function(){
if(!onBeforeUnLoadEvent){ // for avoiding dual calls in browsers that support both events
onBeforeUnLoadEvent = true;
$("#load-icon").show();
setTimeout(function(){
$("#load-icon").hide();},5000); // hiding the loading icon in any case after
// 5 seconds (remove if you do not want it)
}
};
P.S. I cannot comment yet hence posted this as an answer.
Iterating HttpResponse
https://stackoverflow.com/a/1371061/198062
Edit:
I found an example to sending big files with django: http://djangosnippets.org/snippets/365/ Then I look at FileWrapper class(django.core.servers.basehttp):
class FileWrapper(object):
"""Wrapper to convert file-like objects to iterables"""
def __init__(self, filelike, blksize=8192):
self.filelike = filelike
self.blksize = blksize
if hasattr(filelike,'close'):
self.close = filelike.close
def __getitem__(self,key):
data = self.filelike.read(self.blksize)
if data:
return data
raise IndexError
def __iter__(self):
return self
def next(self):
data = self.filelike.read(self.blksize)
if data:
return data
raise StopIteration
I think we can make a iterable class like this
class FlushContent(object):
def __init__(self):
# some initialization code
def __getitem__(self,key):
# send a part of html
def __iter__(self):
return self
def next(self):
# do some work
# return some html code
if finished:
raise StopIteration
then in views.py
def long_work(request):
flushcontent = FlushContent()
return HttpResponse(flushcontent)
Edit:
Example code, still not working:
class FlushContent(object):
def __init__(self):
self.stop_index=2
self.index=0
def __getitem__(self,key):
pass
def __iter__(self):
return self
def next(self):
if self.index==0:
html="loading"
elif self.index==1:
import time
time.sleep(5)
html="finished loading"
self.index+=1
if self.index>self.stop_index:
raise StopIteration
return html
Here is another explanation on how to get a loading message for long loading Django views
Views that do a lot of processing (e.g. complex queries with many objects, accessing 3rd party APIs) can take quite some time before the page is loaded and shown to the user in the browser. What happens is that all that processing is done on the server and Django is not able to serve the page before it is completed.
The only way to show a show a loading message (e.g. a spinner gif) during the processing is to break up the current view into two views:
First view renders the page with no processing and with the loading message
The page includes a AJAX call to the 2nd view that does the actual processing. The result of the processing is displayed on the page once its done with AJAX / JavaScript
We've got some clients sending a custom POST of a data blob to our django servers.
They do things in a rather funky way that I'd rather not get into - and we've since moved on from making that particular format the norm. To make further implementations of our upload protocol more streamlined, I was looking to roll a custom UploadHandler in django to make our data handling in the views a bit more streamlined.
So, moving forward, we want all code in the views to access our POSTs via the:
data = request.FILES['something']
So, for our new submissions, we're handling that dandily.
What I'd like to be able to do is get the upload handler we've made, affectionately called LegacyUploadHandler(), to populate the request.FILES dictionary with the right parts, so the code in our view can access the parts the same way.
So, my question:
How does a custom uploadhandler actually populate the request.FILES dictionary? The django documentation doesn't really give a descriptive example of doing that.
Our particular desire is that we have a singular blob of data coming in. We custom parse it and want it to appear as the request.FILES dictionary.
The current code as it stands right now does this:
def handle_raw_input(self, input_data, META, content_length, boundary, encoding=None):
files_dict = {}
files_dict = magic_parser(input_data.read())
#now what do I do?
I see examples of setting a files MultiValueDict in the http.MultiPartParser, but that seems to be outside the scope/control of where I am in my handlers.
Any ideas of how to actually do the return value? Or am I trying to populate the request.FILES object the completely wrong way?
From handle_raw_input you have to return a tuple of what will be POST and FILES on the requst. So in your case it's something like:
def handle_raw_input(self, input_data, META, content_length, boundary, encoding=None):
files_dict = magic_parser(input_data.read())
return QueryDict(), files_dict
The magic_parser should return a MultiValueDict of the form {'filename': fileobj}. A fileobj is an instance of some suitable django.core.files.File subclass (or may be that class itself).