I have the following UI, where the sonogram (freq+time sound representation) is shown. So the image is not loaded from somewhere, it is drawn by QPainter while reading WAV file.
My current implementation is a single huge QImage object, where the image is drawn. And on paintEvent(), I draw part of the large QImage on the widget:
QPainter painter(this);
// (int, int, QImage*, int, int)
painter.drawImage(0, 0, *m_sonogram, 0, m_offset);
But, as i know, the QPixmap is optimized for displaying pixmaps on the screen, so should I convert the QImage to a QPixmap after the drawing of the sonogram is done?
Also, is it worth to keep large image as some kind of a linked list of separate QPixmap objects of smaller size and make paintEvent() smarter to operate on a list of smaller objects to avoid Qt's auto-cutting procedures and so on?
When my QImage is large enough, each paintEvent() consuming a lot of CPU.
All kinds of advices are welcome :)
Yes, in my limited experience of Qt app development, if you have a static image (or an infrequently updated image) it's well worth (for performance purposes) creating a QPixmap from it and keeping it around to use via QPainter::drawPixmap in your paintEvent handler.
However, I've never tried doing this with anything larger than about 4Kx4K images, so whether it will work for your enormous image or fall over horribly when you start to stress your graphics memory I couldn't say. I'd certainly try it out before considering adding a complicated tiling system.
Related
Compositing multiple images into one using Qt's QPixmap as the storage:
QPainter painter(&destinationPixmap);
painter.drawPixmap(0, 0, sourcePixmap);
This seems to be quite slow (2-10ms for a maximised window on typical monitor) - any way to do it quicker without changing to completely different technology?
Qt documentation says:
QImage is designed and optimized for I/O, and for direct pixel access and manipulation, while QPixmap is designed and optimized for showing images on screen.
So the proper way is to complete all composition manipulations with QImages and then, if you are going to display/repaint the result multiple times, it would be better to convert a resulting QImage to a QPixmap before it rendered.
I do not understand what is the difference between QImage and QPixmap, they seem to offer the same functionality. When should I use a QImage and when should I use a QPixmap?
Easilly answered by reading the docs on QImage and QPixmap:
The QPixmap class is an off-screen image representation that can be used as a paint device.
The QImage class provides a hardware-independent image representation that allows direct access to the pixel data, and can be used as a paint device.
Edit: Also, from #Dave's answer:
You can't manipulate a QPixmap outside the GUI-thread, but QImage has no such restriction.
And from #Arnold:
Here's a short summary that usually (not always) applies:
If you plan to manipulate an image, modify it, change pixels on it,
etc., use a QImage.
If you plan to draw the same image more than once
on the screen, convert it to a QPixmap.
There is a nice series of articles at Qt Labs that explains a lot about the Qt graphics system. This article in particular has a section on QImage vs. QPixmap.
Here's a short summary that usually (not always) applies:
If you plan to manipulate an image, modify it, change pixels on it, etc., use a QImage.
If you plan to draw the same image more than once on the screen, convert it to a QPixmap.
One important difference is that you cannot create or manipulate a QPixmap on anything but the main GUI thread. You can, however, create and manipulate QImage instances on background threads and then convert them after passing them back to the GUI thread.
QPixmap
is an "image object" whose pixel representation are of no consequence in your code, Thus QPixmap is designed and optimized for rendering images on display screen, it is stored on the XServer when using X11, thus drawing QPixmap on XWindow is much faster than drawing QImages, as the data is already on the server, and ready to use.
When to use QPixmap: If you just want to draw an existing image (icon .. background .. etc) especially repeatedly, then use QPixmap.
QImage is an "array of pixels in memory" of the client code, QImage is designed and optimized for I/O, and for direct pixel access and manipulation.
When to use QImage: If you want to draw, with Qpaint, or manipulate an image pixels.
QBitmap is only a convenient QPixmap subclass ensuring a depth of 1, its a monochrome (1-bit depth) pixmap. Just like QPixmap , QBitmap is optimized for use of implicit data sharing.
QPicture is a paint device that records and replays QPainter commands -- your drawing --
Important in industrial environments:
The QPixmap is stored on the video card doing the display. Not the QImage.
So if you have a server running the application, and a client station doing the display, it is very significant in term of network usage.
With a Pixmap, a Redraw consists in sending only the order to redraw (a few bytes) over the network.
With a QImage, it consists in sending the whole image (around a few MB).
I'm creating a image visualizer that open large images(2gb+) in Qt.
I'm doing this by breaking the large image into several tiles of 512X512. I then load a QGraphicsScene of the original image size and use addPixmap to add each tile onto the QGraphic Scene. So ultimately it looks like a huge image to the end user when in fact it is a continuous array of smaller images stuck together on the scene.First of is this a good approach?
Trying to load all the tiles onto the scene takes up a lot of memory. So I'm thinking of only loading the tiles that are visible in the view. I've already managed to subclass QGraphicsScene and override its drag event thus enabling me to know which tiles need to be loaded next based on movement. My problem is tracking movement on the scrollbars. Is there any way I can create an event that get called every time the scrollbar moves. Subclassing QGraphicsView in not an option.
QGraphicsScene is smart enough not to render what isn't visible, so here's what you need to do:
Instead of loading and adding pixmaps, add classes that wrap the pixmap, and only load it when they are first rendered. (Computer scientists like to call this a "proxy pattern"). You could then unload the pixmap based on a timer. (They would be transparently re-loaded if unloaded too soon.) You could even notify this proxy path of the current zoom level, so that it loads lower resolution images when they will be rendered smaller.
Edit: here's some code to get you started. Note that everything that QGraphicsScene draws is a QGraphicsItem, (if you call ::addPixmap, it's converted to a ...GraphicsItem behind the scenes), so that's what you want to subclass:
(I haven't even compiled this, so "caveat lector", but it's doing the right thing ;)
class MyPixmap: public QGraphicsItem{
public:
// make sure to set `item` to nullptr in the constructor
MyPixmap()
: QGraphicsItem(...), item(nullptr){
}
// you will need to add a destructor
// (and probably a copy constructor and assignment operator)
QRectF boundingRect() const{
// return the size
return QRectF( ... );
}
void paint(QPainter *painter, const QStyleOptionGraphicsItem *option,
QWidget *widget){
if(nullptr == item){
// load item:
item = new QGraphicsPixmapItem( ... );
}
item->paint(painter, option, widget);
}
private:
// you'll probably want to store information about where you're
// going to load the pixmap from, too
QGraphicsPixmapItem *item;
};
then you can add your pixmaps to the QGraphicsScene using QGraphicsScene::addItem(...)
Although an answer has already been chosen, I'd like to express my opinion.
I don't like the selected answer, especially because of that usage of timers. A timer to unload the pixmaps? Say that the user actually wants to take a good look at the image, and after a couple of seconds - bam, the image is unloaded, he will have to do something in order the image to reappear. Or may be you will put another timer, that loads the pixmaps after another couple of seconds? Or you will check among your thousand of items if they are visible? Not only is this very very irritating and wrong, but that means that your program will be using resources all the time. Say the user minimizes you program and plays a movie, he will wonder why on earth my movie is freezing every couple of seconds...
Well, if I misunderstood the proposed idea of using timers, execuse me.
Actually the idea that mmutz suggested is better. It reminded me of the Mandelbrot example. Take a look at it. Instead of calculating what to draw you can rewrite this part to loading that part of the image that you need to show.
In conclusion I will propose another solution using QGraphicsView in a much simpler way:
1) check the size of the image without loading the image (use QImageReader)
2) make your scene's size equal to that of the image
3) instead of using pixmap items reimplement the DrawBackground() function. One of the parameters will give you the new exposed rectangle - meaning that if the user scrolls just a little bit, you will load and draw only this new part(to load only part of an image use setClipRect() and then read() methods of the QImageReader class). If there are some transformations you can get them from the other parameter(which is QPainter) and apply them to the image before you draw it.
In my opinion the best solution will be to combine my solution with the threading shown in the Mandelbrot example.
The only problem that I can think of now is if the user zooms out with a big scale factor. Then you will need a lot of resources for some time to load and scale a huge image. Well I see now that there is some function of the QImageReader that I haven't tried yet - setScaledSize(), which maybe do just what we need - if you set a scale size and then load the image maybe it won't load first the entire image – try it. Another way is just to limit the scale factor, a thing that you should do anyway if you stick to the method with the pixmap items.
Hope this helps.
Unless you absolutely need the view to be a QGraphicsView (e.g. because you place other objects on top of the large background pixmap), I'd really recommend just subclassing QAbstractScrollArea and reimplementing scrollContentsBy() and paintEvent().
Add in a LRU cache of pixmaps (see QPixmapCache for inspiration, though that one is global), and make the paintEvent() pull used pixmaps to the front, and be set.
If this sounds like more work than the QGraphicsItem, believe me, it's not :)
I have a QGraphicsView for a very wide QGraphicsScene. I need to draw the background in drawBackground() and the background is a bit complicated (long loop) although it doesn't need to be repainted constantly. I store it in a static QPixmap (I tried QImage too) inside the function drawBackground() and that pixmap is what I draw onto the painter of the view. Only when needed is the QPixmap painted on again.
If I didn't use a static pixmap, the complicated background would be generated every time I scroll sideways for example. The problem is that apparently there is a maximum width for pixmaps on Windows, on my computer it's 32770. I could store a list of pixmaps and draw them side by side but it would make the code uglier and I also don't know what the maximum width of a pixmap is for every Windows machine. Since this might be a well-known problem I was wondering if anyone has a better solution.
Thanks.
You can probably avoid the windows limit by using unaccelerated raster paint device, but 32770*1024 is 100MiB of pixmap; you probably don't want to do that even if Windows would let you.
You've already thought of the usual answer (tile it in more reasonably-sized chunks and load/generate them on demand). The other piece of the usual solution is to use something like QPixmapCache to keep the recently-used tiles so you don't regenerate them too often (only when the user scrolls a long way).
You didn't say how complex your complex background is, but you might also want to look at the Mandelbrot set example for how to do piecewise rendering of an (infinitely) large background pixmap on-demand, without blocking the UI.
This is the common use case for the tiling pattern. Basically you split the background into small images.
I'm not sure why you think "it would make the code uglier". It is certainly not a one-liner. Depending whether you have fixed size background image or not, the tiling code is usually pretty straightforward.
What is the fastest way to display images to a Qt widget? I have decoded the video using libavformat and libavcodec, so I already have raw RGB or YCbCr 4:2:0 frames. I am currently using a QGraphicsView with a QGraphicsScene object containing a QGraphicsPixmapItem. I am currently getting the frame data into a QPixmap by using the QImage constructor from a memory buffer and converting it to QPixmap using QPixmap::fromImage().
I like the results of this and it seems relatively fast, but I can't help but think that there must be a more efficient way. I've also heard that the QImage to QPixmap conversion is expensive. I have implemented a solution that uses an SDL overlay on a widget, but I'd like to stay with just Qt since I am able to easily capture clicks and other user interaction with the video display using the QGraphicsView.
I am doing any required video scaling or colorspace conversions with libswscale so I would just like to know if anyone has a more efficient way to display the image data after all processing has been performed.
Thanks.
Thanks for the answers, but I finally revisited this problem and came up with a rather simple solution that gives good performance. It involves deriving from QGLWidget and overriding the paintEvent() function. Inside the paintEvent() function, you can call QPainter::drawImage(...) and it will perform the scaling to a specified rectangle for you using hardware if available. So it looks something like this:
class QGLCanvas : public QGLWidget
{
public:
QGLCanvas(QWidget* parent = NULL);
void setImage(const QImage& image);
protected:
void paintEvent(QPaintEvent*);
private:
QImage img;
};
QGLCanvas::QGLCanvas(QWidget* parent)
: QGLWidget(parent)
{
}
void QGLCanvas::setImage(const QImage& image)
{
img = image;
}
void QGLCanvas::paintEvent(QPaintEvent*)
{
QPainter p(this);
//Set the painter to use a smooth scaling algorithm.
p.setRenderHint(QPainter::SmoothPixmapTransform, 1);
p.drawImage(this->rect(), img);
}
With this, I still have to convert the YUV 420P to RGB32, but ffmpeg has a very fast implementation of that conversion in libswscale. The major gains come from two things:
No need for software scaling. Scaling is done on the video card (if available)
Conversion from QImage to QPixmap, which is happening in the QPainter::drawImage() function is performed at the original image resolution as opposed to the upscaled fullscreen resolution.
I was pegging my processor on just the display (decoding was being done in another thread) with my previous method. Now my display thread only uses about 8-9% of a core for fullscreen 1920x1200 30fps playback. I'm sure it could probably get even better if I could send the YUV data straight to the video card, but this is plenty good enough for now.
I have the same problem with gtkmm (gtk+ C++ wrapping). The best solution besides using a SDL overlay was to update directly the image buffer of the widget then ask for a redraw. But I don't know if it is feasible with Qt ...
my 2 cents
Depending on your OpenGL/shading skills you could try to copy the videos frames to a texture, map the texture to a rectangle (or anything else..fun!) and display it in a OpenGL scene. Not the most straight approach, but fast, because you're writing directly into the graphics memory (like SDL). I would also recoomend to use YCbCR only since this format is compressed (color, Y=full Cb,Cr are 1/4 of the frame) so less memory + less copying is needed to display a frame. I'm not using Qts GL directly but indirectly using GL in Qt (vis OSG) and can display about 7-11 full HD (1440 x 1080) videos in realtime.