How to access the raw widget pixels rendered on the screen? - c++

Question

I am trying to extract the pixel data from every frame, but when I try to get the pixmap it returns null. I thought that the pixmap would return the actual pixel data of what was on the screen, similar to glReadPixels but I think I am mistaken. I think it's meant to access the pixel map of the result of setPixmap.
Is there a way to access the raw pixels rendered on the screen? With the below example, "Hello World" is rendered on the screen and I want the actual pixel data of that label.
QWidget window;
window.resize(1280, 720);
window.show();
window.setWindowTitle(QApplication::translate("toplevel", "Top-Level Widget"));
QLabel *label = new QLabel(QApplication::translate("label", "Hello World"), &window);
label->move(500, 500);
label->raise();
label->show();
QPixmap pixmapVal = label->pixmap(Qt::ReturnByValue);

Answer 1

Cause
QPixmap:
The QPixmap class is an off-screen image representation that can be used as a paint device
In other words, it is a drawing canvas and, as any other canvas, if you have not drawn on it, it would be empty.
On the other hand QLabel serves as a view for the pixmap, not as its content, and when you try to access the pixmap of the label without having set one, it returns null.
Solution
There is of course a way to make a widget, the label in your case, content of the pixmap and access the pixel data. My approach to the this problem would be like this:
Create an empty pixmap with the size of the widget whos pixel content you want to access, e.g.:
QPixmap pixmap(widget->size());
pixmap.fill(Qt::transparent);
Use QWidget::render to render the content of the widget onto the pixmap:
widget->render(&pixmap);
Convert the pixmap to QImage and use QImage::pixel or QImage::pixelColor to access the rgb data of the pixel at (pixelX, pixelY) like this:
pixmap.toImage().pixelColor(pixelX, pixelY);
Example
Here is an example I have prepared for you to demonstrate how the proposed solution could be implemented:
#include "MainWindow.h"
#include <QApplication>
#include <QLabel>
#include <QDebug>
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
QWidget w;
auto *label = new QLabel(QObject::tr("Hello World"), &w);
label->move(500, 500);
label->raise();
w.setWindowTitle(QObject::tr("Top-Level Widget"));
w.resize(1280, 720);
w.show();
QPixmap pixmap(label->size());
pixmap.fill(Qt::transparent);
label->render(&pixmap);
int pixelX = 10;
int pixelY = 5;
// Access image pixels
qDebug() << pixmap.toImage().pixelColor(pixelX, pixelY);
return a.exec();
}
Result
For the pixel at (10, 5) the example produces the following result:
QColor(ARGB 1, 0, 0, 0.156863)