Background: I'm starting off with Django, and have limited experience with Python, so please bear with me. I've written a Python script that runs periodically (in a cron job) to store data into a SQLite3 database, from which I'd like to read from and generate images with Matplotlib (more specifically, with Basemap). This started off as an interest in learning Python and building an "interesting" enough project. I'm picking the Django framework because it seems reasonably-well documented, although I was pleasantly surprised by web.py because of its "lightweightness" in its requirements (but web.py's sparse documentation makes it a bit harder to start off with); but at the moment, I'm not entirely dead-set on a framework.
The example in question 1874642 is almost what I'm looking for, with an image being generated on-the-fly without requiring having to write it to disk (and thusly having to deal with periodically cleaning up the generated files).
However, what is not clear to me is how the generated image can be incorporated in a template, instead of having the browser simply showing the image. From the tutorial material, I'm guessing that it should be possible to represent the variables incorporated in some django.template.Context into the django.http.HttpResponse, but the referenced example shortcuts it by responding directly with a Mime object instead of building it with a Context.
So what I'm asking is:
Is it necessary to invoke a print_png on the generated Matplotlib FigureCanvas object? Or is the FigureCanvas copyied "unprinted" to the Context, so that in the Django template I explicity write the HTML img tag and put by hand the tag's attributes?
I'm under the impression that I have to write the Canvas to disk (i.e. do a canvas.print_figure("image.png")), so that the HTML img tag sees it in the Django template. But I want to be sure that there isn't a "more manageable way" -- i.e. passing the image in the Context and having the template "magically" generate it. If it's really necessary to write to disk, I suppose I could use Django's filesystem caching facility to write the generated images in some way (checking whether an image was already written for a given input parameter set, of course). I welcome your suggestions on this regard, since it's not yet clear at this time the size and number of the images that will be generated, and thusly I'm looking to avoid spending disk space and instead prefer waiting for an image to be generated (even if it takes a few seconds).
Thank you in advance.
you can pass a StringIO object to pyplot.savefig(), and get the PNG file content by StringIO.getvalue().
This view would serve a PNG image. Just bind it to some URL like "img.png" and use that in an img tag.
def create_fig(request):
# MPL stuff
response = HttpResponse(content_type='image/png')
fig.savefig(response)
return response
Of course, that assumes that you can generate the image independent of the main view. You can pass arguments to the image, like (in urls.py):
url(r'^img(?P<nr>\d+).png$', create_fig),
which passes a (string repr. of) a number nr to create_fig.
Related
I'm new to clojurescript/reagent and am testing out ideas for a media display app.
At the moment I'm having problems with a few more specific elements of including html5 media components on my page and using their full features.
Example - including #t=10,10 at the end of a video source string(referenced here https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Using_HTML5_audio_and_video) will sometimes work but only take the end range value.
Same video element - using any attributes that aren't true/false will break compilation. e.g :preload auto doesn't work whereas :fullscreen false does.
Is there a clojurescript way to handle these elements or is this more js interop territory?
Since clojurescript will be compiled to JS eventually. Everything JS can do..clojurescript could do it equally well.
your "sometime work" issues are related to the nature of ReactJS ( Reagent is a wrapper of ReactJS ). Generally, you need to obtain the dom node of video tag to use most of the video tag's features.
Related issues
Video displayed in ReactJS component not updating
Example
Source Code Example of how to interact with video tag under ReactJS.
https://github.com/eisneim/react-html5-video/blob/master/src/Video.js
You should provide the minimum case to produce the problem, so others could help you.
I've got a Django project spinning up where it would be great to have the UI in lots of color schemes based on the users' school colors. I have this fantasy of having a base variables.less file along with a bunch of other .less files that compile into style.css.
But once a user sets their school colors it generates a blue.variables.less file (if they've selected the blue preset) or school123.variables.less file (if they got all fancy and used the color picker to make their own color scheme) and then compiles everything to blue.style.css or school123.style.css and then that's the .css file we load when we serve the page.
I can imagine lots of ways for this to fall down. Like how do I reprocess all those files when I push an update to forms.less or layout.less.
Is there a better way to do this? I Googled my fingers raw but haven't found anyone attempting this madness.
There are quite a few ways to accomplish your goal of being able to have user specific color schemes, but they each have their advantages / disadvantages.
I'm assuming you are using some framework like Bootstrap with the files that you name.
Option 1: Inline CSS for color-specific styles (Preferred)
This is my preferred option due to performance and simplicity. You can store each of the customized colors for each user, or even creating a model so you can reuse colors that represent a specific school. It's stored in the database and can scale to an very large number of color schemes without generating a lot of very similar files.
Create a snippet in your template code that has any style that uses the color variable.
base.html
{% include "color-snippet.css" with main-color="{{ user's main color }}" alt-color="{{ user's alt color }}" %}
color-snippet.css (note this file will be in your templates directory as it's being handled by your template engine
<style>
.some-style {
color: {{ main-color }} !important;
}
</style>
So the big downside to this is you'll need to customize Bootstrap beyond the variables.less. You'll need to grep through the less files to find all the classes that would be generated, and copy the style to your snippet in css and not less. It'll take some investment up front and work when you want to upgrade to a newer Bootstrap, but it'll allow you to separate the color part of the styles to be derived dynamically at runtime.
I prefer this approach since you don't have to deal with compilation of less outside your collectstatic step.
Option 2: Client side compilation of LESS
You can have Django serve a file that is dynamically created and returns the variables you want. Then you can have less.js compile it on the client.
This would involve adding to your base template a url path that is handled by Django that isn't part of your static path (e.g. /style/variables), creating a handler in views and then returning text content that would be your less file variables.
Option 3: Server side compilation of LESS
I use Django Pipeline to do my server side compilation of less to css. It takes some setup to get working with your Django application. In development mode, Django Pipeline will compile on every request the associated less files into CSS files. In production mode, it'll point to the appropriate file path to the compiled file. It hooks into collectstatic so your less gets compiled when you collectstatic.
The biggest problem with this approach is that the mapping for your static files (what less + css files compile to css) is defined in your settings file. This requires a server restart when you update this. You could base your own server side less compilation off how Django Pipeline works but have logic for the mapping instead of defining it in something that requires a server restart.
It's a lot of work and the less compilation of Bootstrap is non-trivial to have to do on every request.
If you created your own mapping that doesn't require you to restart your Django server process, you could always just run collectstatic to create the new css files. This would avoid compilation at every request.
While this last approach is closest to what you mentioned, it seems a lot more work and error prone than just separating the color-specific styles and using django templating to customize it.
The last approach as well works well if your number of schemes is rather low since you can just create all the mappings ahead of time and not let people generate their own at runtime. They can suggest them and you can just update them at some regular cadence.
I have the following class:
class VideoFile(models.Model):
media_file = models.FileField(upload_to=update_filename, null=True)
And when I try to upload large files to it (from 100mb up to 2Gb) using the following request, it can take quite a long time after the upload process, and during the VideoFile.save() process.
def upload(request):
video_file = VideoFile.objects.create(uploader=request.user.profile)
video_file.media_file = uploaded_file
video_file.save()
On my Macbook Pro Core i7, 8Gb RAM, a 300mb uploaded file can take around 20 seconds or so to run video_file.save()
I suspect this delay is relating to a disk copy operation from /tmp to the files permanent location? I've proven this by running watch ls -l on the target directory, and as soon as video_file.save() runs, I can see the file appear and grow throughout the delay.
Is there a way to eliminate this file transfer delay? Either by uploading the file directly to the target filename or just by moving the original file instead of copying? This is not the only upload operation across the site however so any solution needs to be localized to this model.
Thanks for any advice!
UPDATE:
Just further evidence to support a copy instead of a move, i can watch lsof during the upload and see a file within /private/var/folders/... written from python which maps exactly to upload progress. After upload is complete, another lsof entry appears for the ultimate file location which grows over time. After that's complete, both entries disappear.
Ok after a bit of digging I've come up with a solution. It turns out Django's default storage already attempts to move the file instead of copy, which it first tests:
hasattr(content, 'temporary_file_path')
This attribute exists for the class TemporaryUploadedFile which is the object returned to the Upload View, however the field itself is created as the class specified by FileField.attr_class
So instead I decided to subclass FieldFile and FileField and slot in the temporary_file_path attribute:
class VideoFieldFile(FieldFile):
_temporary_file_path = None
def temporary_file_path(self):
return self._temporary_file_path
class VideoFileField(FileField):
attr_class = VideoFieldFile
Finally in the view, before saving the model I manually assigned the temp path:
video_file.media_file._temporary_file_path = uploaded_file.temporary_file_path()
This now means my 1.1Gb test file becomes available in about 2-3 seconds rather than the 50 seconds or so I was seeing before. It also comes with the added benefit that if the files exist on different file systems, it appears to fall back to the copy operation.
As a side note however, my site is not utilizing MemoryFileUploadHandler which some sites may use to handle smaller file uploads, so I'm not sure how nice my solution might work with that, but I'm sure it would be simple enough to detect the uploaded file's class and act accordingly.
I would caution that there are quite a few reasons why uploading to /tmp and then cping over is best practice, and that uploading large files directly to their target is a potentially dangerous operation.
But, what you're asking is absolutely possible. Django defines upload handlers:
You can write custom handlers that customize how Django handles files. You could, for example, use custom handlers to enforce user-level quotas, compress data on the fly, render progress bars, and even send data to another storage location directly without storing it locally.
In my C++ app I'm embedding (via COM) a web browser (Internet Explorer) control (CLSID_WebBrowser).
I can display my own html in that control by using IHTMLDocument2::write() method but if the html has <img src="foo.png"> element, it's not displayed.
I assume there is a way for me to provide the data for foo.png somehow to the web control, but I can't find the right place to hook this functionality?
I need to be in full control of providing the content of foo.png, so work-arounds like using res:// protocol or saving to disk and using file:// protocol are not good enough. I just want to plug my code somehow so that when embedded CLSID_WebBrowser control sees <img src="foo.png"> in html data given with IHTMLDocument2::write() it will ask me to provide this data.
To answer my own question, the solution that finally worked for me is:
register custom IInternetProtocol/IInternetProtocolInfo/ via custom IClassFactory given to IInternetSession::RegisterNameSpace(). For reasons that seem like a bug to me, it has to be a protocol already known to IE (I've chosen "its") even though it would be much better if it was my own, unique namespace.
feed html data via custom IMoniker through IPersistentMoniker::Load() and make sure that IMoniker::GetDisplayName() (which is a base url according to which relative links in provided html will be resolved) starts with that protocol scheme (in my case "its://"). That way relative link "foo.png" in the html data will be its://foo.png to IE which will make urlmon call IInternetProtocol::Start() and IInternetProtocol::Read() to ask for the data for that url.
This is all rather complicated, you can look at the actual (BSD-licensed) code here:
http://code.google.com/p/sumatrapdf/source/browse/trunk/src/utils/HtmlWindow.cpp
You can embed a small webserver such as mongoose and reference those impage from there.
In mongoose, you can attach callback to specific path, thus returning images from C++ code.
We use this for our debugging tools, where each images is accessible from a web interface
The easiest solution would be a Data URI. You'd inline out the image directly with IHTMLDocument2::write().
I am doing a monitoring system using Django. In my views file, I have defined one class called showImage which collects the information necessary to plot a graph using matplotlib.
At the beginning, I just stored the image in a string buffer to represent it with HttpResponse:
buffer = StringIO.StringIO()
canvas = pylab.get_current_fig_manager().canvas
canvas.draw()
pilImage = PIL.Image.fromstring("RGB", canvas.get_width_height(), canvas.tostring_rgb())
pilImage.save(buffer, "PNG")
# Send buffer in a http response the the browser with the mime type image/png set
return HttpResponse(buffer.getvalue(), mimetype="image/png")
However, I need to implement some javaScript in the html file to add more applications. For that reason, I have decided to save the image in a variable and plot it in the html file:
# serialize to HTTP response
response = HttpResponse(buffer.getvalue(), mimetype="image/png")
return render_to_response('eQL/dev/showImage.html', {'response':response})
My question is that I don't really know how to represent it in the html file because I didn't find any example doing it. Any one knows the answer?
Thanks in advance!
Do you mean that in your first implementation, your response was a PNG file, but now you wish to make the response an HTML file instead, containing the image?
Well firstly, you need to change the response MIME type from image/png to text/html or similar.
Secondly, I'm not sure why you are passing a HttpResponse object (containing the PNG data) into the template. Can the template even read that? Surely you just want to be passing the raw PNG data, not a HttpResponse object.
Finally, how to do it. Well as you may know, HTML isn't so great at embedding images. As with normal websites, you can include text in the page, but if you want an image, you need a separate file and link to it using the <img src="..." /> element. This is tricky to do dynamically: it means you need to setup two separate URLs (one for the PNG and one for the HTML), which run independently of one another (you can't just have one piece of code; you need one handler for generating the PNG and the other for generating the HTML), and have the HTML link to the PNG URL.
If that is too hard, there is another way out, but it is a bit hacky: data URLs. They let you include image data in the HTML page itself, so you only need to produce one response. Unfortunately it is not well supported in Internet Explorer pre-9. IE8 supports images less than 32K, IE7 and below don't work. See the example on Wikipedia -- you are aiming to generate something like this:
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA
AAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO
9TXL0Y4OHwAAAABJRU5ErkJggg==" alt="Red dot" />
Basically, take the PNG data, and Base64-encode it (use Python's base64 library). Then just put "data:image/png;base64," in front of it, and set that as the URL for the img src. In other words, pass the Base64-encoded string to Django's template engine, and construct the URL as part of the img tag in the template.