How to use PBO with Qt OpenGL - c++

Question

I'm trying to use QOpenGLBuffer as a Pixel Buffer Object. The goal would be to display a high resolution video stream (4K, 60FPS) so I need good performance.
Since I'm just starting with OpenGL, I'm trying at first to display a simple 2D texture the best way possible. I've already included VBO and VAO, the next step (As I read it) would be to use a PBO for better performance.
There are tutorial out there for PBO but with glGenBufferARB(), not with QOpenGLBuffer.
Here is my code :
glwidget.h
#ifndef GLWIDGET_H
#define GLWIDGET_H
#include <QOpenGLWidget>
#include <QOpenGLFunctions>
#include <QOpenGLBuffer>
#include <QDebug>
#include <QOpenGLTexture>
#include <QOpenGLShader>
#include <QOpenGLShaderProgram>
#include <QOpenGLVertexArrayObject>
class GLWidget : public QOpenGLWidget, protected QOpenGLFunctions
{
Q_OBJECT
public:
explicit GLWidget(QWidget *parent = 0);
~GLWidget();
void initializeGL();
void paintGL();
void resizeGL(int w, int h);
void LoadGLTextures();
private :
QOpenGLShaderProgram *program;
QOpenGLBuffer vbo;
QOpenGLVertexArrayObject vao;
GLuint tex;
GLint vertexLocation;
GLint texcoordLocation;
int tailleVerticesBytes;
int tailleCoordTexturesBytes;
float vertices[8];
float coordTexture[8];
public slots:
private slots:
};
#endif // GLWIDGET_H
glwidget.cpp
#ifndef BUFFER_OFFSET
#define BUFFER_OFFSET(offset) ((char*)NULL + (offset))
#include "glwidget.h"
#include <QElapsedTimer>
GLWidget::GLWidget(QWidget *parent) :
QOpenGLWidget(parent)
{
tailleVerticesBytes = 8*sizeof(float);
tailleCoordTexturesBytes = 8*sizeof(float);
}
GLWidget::~GLWidget(){
vao.destroy();
vbo.destroy();
delete program;
glDeleteTextures(1, &tex);
}
void GLWidget::LoadGLTextures(){
QImage img;
if(!img.load("C:\\Users\\Adrien\\Desktop\\open3.bmp")){
qDebug()<<"Image loading failed";
}
QImage t = (img.convertToFormat(QImage::Format_RGBA8888)).mirrored();
glGenTextures(1, &tex);
glBindTexture(GL_TEXTURE_2D, tex);
glTexImage2D(GL_TEXTURE_2D, 0, 3, t.width(), t.height(), 0, GL_RGBA, GL_UNSIGNED_BYTE, t.bits());
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glBindTexture( GL_TEXTURE_2D, 0);
}
void GLWidget::initializeGL(){
float verticesTmp[] = {-1.0,-1.0, 1.0,-1.0, 1.0,1.0, -1.0,1.0};
float coordTextureTmp[] = {0.0,0.0, 1.0,0.0, 1.0,1.0, 0.0,1.0};
for(int i = 0; i<8; i++){
vertices[i] = verticesTmp[i];
coordTexture[i] = coordTextureTmp[i];
}
initializeOpenGLFunctions();
glClear(GL_COLOR_BUFFER_BIT);
glEnable(GL_TEXTURE_2D);
LoadGLTextures();
//Shader setup
QOpenGLShader *vshader = new QOpenGLShader(QOpenGLShader::Vertex, this);
const char *vsrc =
"#version 150 core\n"
"in vec2 in_Vertex;\n"
"in vec2 vertTexCoord;\n"
"out vec2 fragTexCoord;\n"
"void main(void)\n"
"{\n"
" gl_Position = vec4(in_Vertex, 0.0, 1.0);\n"
" fragTexCoord = vertTexCoord;\n"
"}\n";
vshader->compileSourceCode(vsrc);
QOpenGLShader *fshader = new QOpenGLShader(QOpenGLShader::Fragment, this);
const char *fsrc =
"#version 150 core\n"
"uniform sampler2D tex;\n"
"in vec2 fragTexCoord;\n"
"void main(void)\n"
"{\n"
" gl_FragColor = texture2D(tex,fragTexCoord);\n"
"}\n";
fshader->compileSourceCode(fsrc);
program = new QOpenGLShaderProgram;
program->addShader(vshader);
program->addShader(fshader);
program->link();
program->bind();
glActiveTexture(GL_TEXTURE0);
program->setUniformValue("tex", 0);
vertexLocation = glGetAttribLocation(program->programId(), "in_Vertex");
texcoordLocation = glGetAttribLocation(program->programId(), "vertTexCoord");
//VAO setup
vao.create();
vao.bind();
//VBO setup
vbo.create();
vbo.setUsagePattern(QOpenGLBuffer::StaticDraw);
vbo.bind();
vbo.allocate(tailleVerticesBytes + tailleCoordTexturesBytes);
vbo.write(0, vertices, tailleVerticesBytes);
vbo.write(tailleVerticesBytes, coordTexture, tailleCoordTexturesBytes);
program->enableAttributeArray(vertexLocation);
program->setAttributeBuffer(vertexLocation, GL_FLOAT, 0, 2);
program->enableAttributeArray(texcoordLocation);
program->setAttributeBuffer(texcoordLocation, GL_FLOAT, tailleVerticesBytes, 2);
vbo.release();
vao.release();
program->release();
}
void GLWidget::paintGL(){
glClear(GL_COLOR_BUFFER_BIT);
program->bind();
{
vao.bind();
glBindTexture(GL_TEXTURE_2D, tex);
glDrawArrays(GL_QUADS, 0, 4);
glBindTexture(GL_TEXTURE_2D, 0);
vao.release();
}
program->release();
}
void GLWidget::resizeGL(int w, int h){
glViewport(0, 0, (GLint)w, (GLint)h);
}
#endif
So basically, how would I do to use PBO in this code ?
The first thing to do would be to create a QOpenGLBuffer object while specifying the type (QOpenglBuffer::PixelUnpackBuffer), then I guess I would need to upload the pixel on the buffer and finally use it instead of glTexImage2D ? This is just the global idea and I have no idea how to do it.
Thanks.

Answer 1

The goal would be to display a high resolution video stream (4K, 60FPS) so I need good performance.
The only and proper way to do this is using some accelerated presentation API (which have nothing to do with OpenGL).
If you want to stick with OpenGL, you'd want at least to have the GPU do the video decoding and uploading into a texture. How to do so depends on your OS and GPU. For instance, under Linux and using NVIDIA, you can use VDPAU for accelerated decoding and NV_VDPAU_interop for getting textures populated with decoded frames.
If, still, you want to use Pixel Unpack Buffer Objects for this (PUBO; you're uploading into GL => it's an unpack) there's very little magic going on. Create one:
QOpenGLBuffer *pubo = new QOpenGLBuffer(QOpenGLBuffer::PixelUnpackBuffer);
pubo->create();
then populate it with your frame's data:
pubo->bind();
pubo->allocate(...); // or, if already allocated, also write/map
Now the effect of a PUBO is that, if one is bound, certain calls will change semantics to read data not from user memory but from the PUBO. Notably, the calls that upload texture data. So if you have your texture around (and you should be using QOpenGLTexture, which uses immutable storage, not manual calls to glTexImage), you can do:
pubo->bind();
glBindTexture(GL_TEXTURE_2D, textureId);
glTexImage2D(GL_TEXTURE_2D,
level,
internalFormat,
width,
heigth,
border,
externalFormat,
type,
data);
Since there's a PUBO bound, the very last argument (data) changes semantics: it's no longer a pointer into client memory, but it's a byte offset into the currently bound pixel unpack buffer object. So, if your texture data starts at offset 0 into the buffer, you need to pass 0 there (or, actually, (const GLvoid *)0). Otherwise you'll need to adjust it accordingly.
Now you can release the pubo:
pubo->release();
and then use the texture as usual in your shaders and everything will be just fine.
Except that if you use the texture straight away you won't get any performance improvement! The whole point of this complicated setup is to allow the GL to transfer the image data asynchronously, while you render the data uploaded in the previous frame(s). If you use the image data immediately, GL needs to sync the entire pipeline in order to wait for the image data to be uploaded to the GPU.
So the typical setup in this scenario is having a number of PUBOs used in a round-robin fashion. For instance, if you have two (in ping-poing), you upload data in one PBO, and use the previous one to populate and draw the texture. This should buy "enough time" for the GL to actually transfer the current data across the bus, so at the next frame the texture upload and the draw will find the data immediately available.
Ideally, you could also perform the upload of data in PUBOs from another thread using a shared OpenGL context, and use fences to signal the render thread when the upload is complete so that the texture can be populated. And you can build further on by using orphaning, pinned maps, unsynchronized writes, and even more.
A great in depth explaination of all of this is available in chapters 28/29 of OpenGL Insights, which I can't reproduce integrally here, and comes with some code available here.
You can find also some more info about buffer object streaming on the OpenGL wiki here.