I am writing a video display software capable of displaying multiple video streams. For this I have a GridView holding VideoOutputs in QML connected to a QAbstractListModel derived class in c++ which provides instances of an object with a QAbstractVideoSurface Q_PROPERTY. It's working quite beautifully so far.
The video frames I am displaying come with metadata, however, containing data for axis aligned bounding boxes. I don't know beforehand how many boxes there are, the number could even change on a frame by frame basis, their position and size is also not set.
Ultimately, it should look something like this:
As I need to be able to display a few video streams at once, and preferably at 30+ fps, I need a fast method of drawing these boxes. Using QPainter on the QImage on which the QVideoFrame is based is rather slow so I was considering a few other approaches:
Using the QML object Rectangle in a Repeater with a c++ provided model (Was hoping to simply provide a QVariantList::fromVector() ): Could work, however I would need a lot of models which in turn I need to provide to QML with a model, and I would likely need to call begin/endResetModel every frame that the boxes change to cause QML to update - this is also very slow.
Using a Shader to draw the boxes: This is a rather difficult approach. I'm no stranger to shaders, but in Qt/Qml I don't know how to provide the shader with the information necessary.
Using OpenGL directly to draw the boxes: Again, I have no clue how to do this, but I think I could work it out if I googled.
My question: Which one, if any, of these approaches is the best? If none of these, which other approach could I use?
Thank you so much for taking the time to read my rather long question!
Related
What is the most efficient way of displaying real time plain text?
If I need to display large amount of changing text with high frequency.
What is the best solution?
I have tried QLabel, QPlaintextEdit and I also tried to rewrite a widget's paintEvent and do painting there. But they all do not work very well.
Due to the platform limitation, I cannot use QGLwidget or openGL support.
And the texts need to be grouped in related widgets where the user can perform some basic interaction(like selecting) with those widgets. The expected thing is something like this:
EDIT:
The data comes from many different vehicles' sensors, you could consider that I store the incoming data simply in several arrays. The method I have tried all resulted in a slow responding GUI thread.
I am building an editor using C++/Qt which has a click-and-drag feel to it. The behavour is similar to schematic editors (Eagle, KiCAD, etc), Microsoft Visio, or other programs where you drag objects from a toolbar into a central editing area.
My problem is that when the user clicks inside the custom widget I want to be able to select the instance of the box-like object and manipulate it. There will also be lines connecting the boxes together. However, I can't decide on an efficient method for selecting those objects.
I have two main thoughts on how to do the programming for this: The first is that the widget which is drawing the entire editor would simply encapsulate every one of the instances of the box. The other is to have each instance of the box (which is in my Model) carry with it an instance of a QWidget which would handle rendering the box (which would be in my View...but it would end up being strongly attached to the model). As for the lines connecting them, since they don't have a square bounding boxes they will have to be rendered by the containing widget.
So here is the summary of how I see this being done:
The editor widget turns into a container which holds the widgets and the widgets process their own click events. The potential issues here are that I don't know how to make the custom widget turn into a layout which lets click-and-drag functionality.
The editor widget takes care of all the rendering and processes the mouse clicks (the easier way in that I don't have to worry about layout...its just selecting the instances efficiently that I don't know what would be best).
So, now that there is a bit of background, for the 2nd method I plan on having each box-like instance having a bounding rectangle and the lines being represented by 3-4 pixel wide bounding rectangle segments (they are at 90 degree angles). I could iterate through every single box and line, but that seems really inefficient.
The big question: Is there some sort of data structure I can hold rectangles in and link them to widgets (or anything else for that matter) and then give it two coordinates (such as mouse coordinates) and have it spit me out the bounding box or linked object that those coordinates are inside of?
It sounds like your real question is about finding a good way to implement your editor, not the specifics of rectangle intersection performance.
You may be interested in Qt's "Diagram Scene" example project, which demonstrates the QGraphicsScene API. It sounds like a good fit for the scenario you describe. (The full source for the example ships with Qt.)
The best part is that you still don't have to implement hit testing yourself, because the API already provides what you are looking for (e.g., QGraphicsScene::itemAt()).
It's worth noting that internally, QGraphicsScene uses a simple iterative method to perform hit tests. As others have pointed out, this isn't going to be a serious bottleneck unless your scenes have a lot of individual items.
OpenCV recently upgraded its display window, when it is used in Qt.
It looks very good, however I did not find any possibility for it to be embedded into an existing Qt GUI window. The only possibility seems to be the creation of a cvNamedWindow or cv::namedWindow, but it creates a free-floating independent window.
Is there any possibility to create that OpenCV window inside an existing GUI? All I could find on the OpenCV forums is an unanswered question, somewhat similar to my own.
There is a straight-forward possibility to show an OpenCV image in Qt, but it has two major problems:
it involves copying the image pixel by pixel, and it's quite slow. It has function calls for every pixel! (in my test application, if I create a video out of the images, and display it in a cvNamedWindow, it runs very smoothly even for multiple videos the same time, but if I go through the IplImage --> QImage --> QPixmap --> QLabel route, it has severe lag even for one single video)
I can't use those nice new controls of the cvNamedWindow with it.
First of all, the image conversion is not as inefficient as you think. The 'function calls' per pixel at least in my code (one of the answers to the question you referenced) are inlined by optimized compilation.
Second, the code in highgui/imshow does the same. You have to get from the matrix to an ARGB image either way. The conversion QImage -> QPixmap is essentially nothing else than moving the data from main memory to GPU memory. That's also the reason why you cannot access the QPixmap data directly and have to go through QImage.
Third, it is several times faster if you use a QGLWidget to draw the image, and I assume you have QT_OPENGL enabled in your OpenCV build. I use QPainter to draw the QPixmap within a QGLWidget, and speed is no issue. Here is example code:
http://sourceforge.net/p/gerbil/svn/19/tree/gerbil-gui/scaledview.h
http://sourceforge.net/p/gerbil/svn/19/tree/gerbil-gui/scaledview.cpp
Now to your original question: Your current option is to take the code from OpenCV, include into your project under a different namespace, and alter it to fit your needs. Apart from that you have no alternative right now. OpenCV's highgui uses its own event loop, connection to the X server etc. and it is nothing you can intercept.
My first guess is to want to say this: I'm sure that if you dig into the code for namedWindow, you will find that they use some sort of standard, albeit not oft-referenced, object for painting said window (that's in the openCV code). If you were ambitious enough, you could extend this class yourself, to interface directly to a frame or custom widget in Qt. There might even be a way to take the entire window and embed it, using a similar method of a Qt frame, or an extension of the (general) widget class. This is a very interesting question and relates rather directly to work I've been doing of late, so I'll continue to think about and research it and see if I can't come up with something else more helpful.
[EDIT] What specific new controls are you so interested in? It might be more efficient on the part of the programmer to extend a Qt control to emulate that, as opposed to my first suggestion.[/EDIT]
simply check out the opencv highgui implementation. as i recall it uses qt.
I have to design a GUI using Qt. I would like to draw multiple lines depicting relationships between two objects. It's the same idea as matching a word with a definition by drawing a straight line (which might be a diagonal) between the two.
In my case it is an a label (with image inside of it) that needs to be matched with another label.
So we have something like this - http://dl.dropbox.com/u/46437808/DrawLines.png
And I want to add lines to make it look something like this http://dl.dropbox.com/u/46437808/DrawLines2.png
I need to do this in run time because the relationship will be changing based on different factors.
Thanks!
Do you need interaction or is this just an image that the user needs to see based on other information? If it's just a static image, I would simply draw it onto a QImage and show it. That way you have complete control over how things are drawn. So you can either cache the relationship diagrams you need ahead of time, or just draw them on the fly onto the QImage based on the relationship that needs to be displayed at the time. You can look at Qt's painting example for some ideas on how to accomplish what you need.
If you need interactivity, I would probably go with the Graphics View Framework. This way if you need push buttons, check boxes, etc. for any reason you can use the QGraphicsProxyWidget to get them, or you can just make your own from QGraphicsItem subclasses.
I need to implement a simple sound editor with a specific signal processing function.
I want to display the sound information vertically as a large picture - 1024 x [large height].
So I'm looking for Qt's best way to implement low-cpu-consuming vertical scrolling similar to web-browsers scrolling.
You need to create a custom widget by overriding it's paintEvent().
Have a look at this self-explanatory example on how to create a following widget.
A normal QScrollArea will give you what you want 99% of the time. But you're wanting to create a bitmap of potentially massive size, and you want to store it in RAM - you will quickly run out of space.
You are probably going to need a need a system where you save the image to file, and use the scroll area's bounds to intelligently load it's contents from the file (and a certain size around it to make it smooth) - Qt does not provide this. This is hardly groundbreaking stuff, so a web search for "scrolling very large images" or something similar should give a smart result.
If your intention is to plot STFT data, then you can easily adopt one of the widgets in the QWT library:
http://qwt.sourceforge.net/
Anyway, take a look at it, it is a great source of inspiration and the code is available...