I'm trying to make video output (sequence of frames) to any qt visible widget. At beginning i thought that QLabel will be enough for this point... but i was wrong. Converting to pixmap is too overloading for processor at large images: 1080p for example.
Any other solution? (not QLabel?)
Example of code for one frame:
QImage m_outputFrameImage(width, height, QImage::Format_RGB888);
memcpy(m_outputFrameImage.bits(), m_frameRGB->data[0], height * width * 3);
QPixmap pixmap = QPixmap::fromImage(m_outputFrameImage); // BAD, slow and high load
/* Bad too (Same code?)
QPainter painter;
painter.begin(&pixmap);
painter.drawImage(0, 0, m_outputFrameImage);
painter.end();
*/
labelVideo->setPixmap(pixmap);
Yes, render the frames to a QGLWidget and let the video card handle it. That's how Qt MultimediaKit, Phonon and others do it.
Some time ago I shared some code that demonstrated how to accomplish this task: Image scaling (KeepAspectRatioByExpanding) through OpenGL
Related
I have a QPixmap and I set it to resize according to ratio as the window is resized. When the images is first loaded in, it is good and clear for that ratio, but as I resize the image, it distorts all of it.
It manages to go from this:
to this:
Relevant Code:
void MainWindow::resizeEvent(QResizeEvent *)
{
QPixmap pix = ui->labelImage->pixmap()->scaled(ui->labelImage->size(),
Qt::KeepAspectRatio);
ui->labelImage->setPixmap(pix);
}
You're reading the current pixmap from the widget, scaling it, and then writing the scaled pixmap to the widget. Consider what happens if the widget becomes very small and is then resized to something much larger -- you'll get a lot of artifacts due to the scaling transformation used.
I think a better approach would be to store the original full size image in a member of your MainWindow class, say...
QPixmap m_original_pixmap;
Then use a scaled version of that in your resizeEvent member...
void MainWindow::resizeEvent(QResizeEvent *)
{
QPixmap pix = m_original_pixmap.scaled(ui->labelImage->size(), Qt::KeepAspectRatio);
ui->labelImage->setPixmap(pix);
}
Not sure if that will clear everything up but should go some way to removing some of the artifacts.
As a side note, if you're concerned about image quality you might consider specifying Qt::SmoothTransformation as the pixmap transformation mode in the scaling operation.
My Qt application uses a QPainter to draw a vector graphic. I need this graphic output twice, once as a vector output in SVG format, where I'm using QSvgGenerator, and once as a pixel format, where I'm using QImage.
According to what I've found in the documentation I can either first paint to SVG and then convert the SVG output to a Qimage:
QPainter painter;
QSvgGenerator generator;
generator.setSize(QSize(width_, height_));
// more initializations here
painter.begin(&generator);
doPaintMyStuff(&painter);
painter.end();
generator.setOutputDevice(...) // pipe the SVG output to the server
QImage image(width_, height_, QImage::Format_ARGB32_Premultiplied);
QSvgRenderer renderer;
renderer.load(...) // get the svg output we just generated
painter.begin(&image);
renderer.render(&painter); // render the vector graphic to pixel
painter.end();
usePixelData(image.constBits()); // pipe the pixel output to the server
or draw twice using two different backends:
QPainter painter;
QSvgGenerator generator;
generator.setSize(QSize(width_, height_));
// more initializations here
QImage image(width_, height_, QImage::Format_ARGB32_Premultiplied);
painter.begin(&generator);
doPaintMyStuff(&painter);
painter.end();
painter.begin(&image);
doPaintMyStuff(&painter);
painter.end();
generator.setOutputDevice(...) // pipe the SVG output to the server
usePixelData(image.constBits()); // pipe the pixel output to the server
Both solutions work, but both seem terribly inefficient to me, since I'm always drawing the same scene twice. The latter calls all functions on QPainter twice, the former draws all operations again by re-tracing the SVG ouput I just generated.
Is there a way to attach multiple backends to one QPainter to paint the whole scene only once?
I don't think you can get what you want out of the box. You could dig into the private implementation of the painter and come up with a way to do everything just once - generate each vector painter component and rasterize it to another paint device and move onto the next, but it will probably not be simple and hardly be worth it.
Just profile the two solutions you have so far and stick to the faster one, looks like the first one might be a tad more efficient.
I do not understand what is the difference between QImage and QPixmap, they seem to offer the same functionality. When should I use a QImage and when should I use a QPixmap?
Easilly answered by reading the docs on QImage and QPixmap:
The QPixmap class is an off-screen image representation that can be used as a paint device.
The QImage class provides a hardware-independent image representation that allows direct access to the pixel data, and can be used as a paint device.
Edit: Also, from #Dave's answer:
You can't manipulate a QPixmap outside the GUI-thread, but QImage has no such restriction.
And from #Arnold:
Here's a short summary that usually (not always) applies:
If you plan to manipulate an image, modify it, change pixels on it,
etc., use a QImage.
If you plan to draw the same image more than once
on the screen, convert it to a QPixmap.
There is a nice series of articles at Qt Labs that explains a lot about the Qt graphics system. This article in particular has a section on QImage vs. QPixmap.
Here's a short summary that usually (not always) applies:
If you plan to manipulate an image, modify it, change pixels on it, etc., use a QImage.
If you plan to draw the same image more than once on the screen, convert it to a QPixmap.
One important difference is that you cannot create or manipulate a QPixmap on anything but the main GUI thread. You can, however, create and manipulate QImage instances on background threads and then convert them after passing them back to the GUI thread.
QPixmap
is an "image object" whose pixel representation are of no consequence in your code, Thus QPixmap is designed and optimized for rendering images on display screen, it is stored on the XServer when using X11, thus drawing QPixmap on XWindow is much faster than drawing QImages, as the data is already on the server, and ready to use.
When to use QPixmap: If you just want to draw an existing image (icon .. background .. etc) especially repeatedly, then use QPixmap.
QImage is an "array of pixels in memory" of the client code, QImage is designed and optimized for I/O, and for direct pixel access and manipulation.
When to use QImage: If you want to draw, with Qpaint, or manipulate an image pixels.
QBitmap is only a convenient QPixmap subclass ensuring a depth of 1, its a monochrome (1-bit depth) pixmap. Just like QPixmap , QBitmap is optimized for use of implicit data sharing.
QPicture is a paint device that records and replays QPainter commands -- your drawing --
Important in industrial environments:
The QPixmap is stored on the video card doing the display. Not the QImage.
So if you have a server running the application, and a client station doing the display, it is very significant in term of network usage.
With a Pixmap, a Redraw consists in sending only the order to redraw (a few bytes) over the network.
With a QImage, it consists in sending the whole image (around a few MB).
It's really not clear to me how to simply draw a 2d point in QT. I want it to overlay a QPixmap item, but every piece of documentation I find talks about drawing polygons with brushes.
Thanks in advance -
From Qt's documentation:
QImage is designed and optimized for
I/O, and for direct pixel access and
manipulation, while QPixmap is
designed and optimized for showing
images on screen.
So if you have a QPixmap, convert it to QImage and then use QImage::setPixel:
QImage image = pixmap->toImage();
image.setPixel(2, 4, 0x0000ff);
ui->label->setPixmap(QPixmap::fromImage(image)); // show the image in a label
What is the fastest way to display images to a Qt widget? I have decoded the video using libavformat and libavcodec, so I already have raw RGB or YCbCr 4:2:0 frames. I am currently using a QGraphicsView with a QGraphicsScene object containing a QGraphicsPixmapItem. I am currently getting the frame data into a QPixmap by using the QImage constructor from a memory buffer and converting it to QPixmap using QPixmap::fromImage().
I like the results of this and it seems relatively fast, but I can't help but think that there must be a more efficient way. I've also heard that the QImage to QPixmap conversion is expensive. I have implemented a solution that uses an SDL overlay on a widget, but I'd like to stay with just Qt since I am able to easily capture clicks and other user interaction with the video display using the QGraphicsView.
I am doing any required video scaling or colorspace conversions with libswscale so I would just like to know if anyone has a more efficient way to display the image data after all processing has been performed.
Thanks.
Thanks for the answers, but I finally revisited this problem and came up with a rather simple solution that gives good performance. It involves deriving from QGLWidget and overriding the paintEvent() function. Inside the paintEvent() function, you can call QPainter::drawImage(...) and it will perform the scaling to a specified rectangle for you using hardware if available. So it looks something like this:
class QGLCanvas : public QGLWidget
{
public:
QGLCanvas(QWidget* parent = NULL);
void setImage(const QImage& image);
protected:
void paintEvent(QPaintEvent*);
private:
QImage img;
};
QGLCanvas::QGLCanvas(QWidget* parent)
: QGLWidget(parent)
{
}
void QGLCanvas::setImage(const QImage& image)
{
img = image;
}
void QGLCanvas::paintEvent(QPaintEvent*)
{
QPainter p(this);
//Set the painter to use a smooth scaling algorithm.
p.setRenderHint(QPainter::SmoothPixmapTransform, 1);
p.drawImage(this->rect(), img);
}
With this, I still have to convert the YUV 420P to RGB32, but ffmpeg has a very fast implementation of that conversion in libswscale. The major gains come from two things:
No need for software scaling. Scaling is done on the video card (if available)
Conversion from QImage to QPixmap, which is happening in the QPainter::drawImage() function is performed at the original image resolution as opposed to the upscaled fullscreen resolution.
I was pegging my processor on just the display (decoding was being done in another thread) with my previous method. Now my display thread only uses about 8-9% of a core for fullscreen 1920x1200 30fps playback. I'm sure it could probably get even better if I could send the YUV data straight to the video card, but this is plenty good enough for now.
I have the same problem with gtkmm (gtk+ C++ wrapping). The best solution besides using a SDL overlay was to update directly the image buffer of the widget then ask for a redraw. But I don't know if it is feasible with Qt ...
my 2 cents
Depending on your OpenGL/shading skills you could try to copy the videos frames to a texture, map the texture to a rectangle (or anything else..fun!) and display it in a OpenGL scene. Not the most straight approach, but fast, because you're writing directly into the graphics memory (like SDL). I would also recoomend to use YCbCR only since this format is compressed (color, Y=full Cb,Cr are 1/4 of the frame) so less memory + less copying is needed to display a frame. I'm not using Qts GL directly but indirectly using GL in Qt (vis OSG) and can display about 7-11 full HD (1440 x 1080) videos in realtime.