I am trying different implementations of how to achieve further processing from the QML camera, I need to pass on the feed to the C++ end so it can then be converted to a cv::Mat image and passed to a function for processing. I have tried setting a QCamera from the C++ end and starting it on a button click but it seems Qt cannot create two instances of the same camera. I have also tried an open CV method but to no avail. What is the best way to do this?
This is a not a fix but a workaround. I used an OpenCv plugin for the camera and made it visible to my QML using qmlRegisterType. I could then easily send the frames from the backend to my other class for processing. For anyone looking to do this I used this plugin: https://github.com/rferrazz/CvCamView
Qt QML Camera to C++ QImage on Android
I just answered this question, I think you were having the same issue.
The basic Idea here is to get the instance of QML camera, access its QMediaObject point and probe it with QVideoProbe. There are other solutions, but AFAIK they arent really easy or fast in Android platforms, but if thats not your case, you probavly should try QAbstractVideoFilter alongside QVideoFilterRunnable classes which were developed specially for post processing qml video feed.
Note that QAbstractVideoFilter and QVideoFilterRunnable are Qt 5.5 classes only.
Related
I wonder if I can write my own native module, render something with using OpenGL in C++ and finally display rendered picture on react native side ( by simply using component).
If so, can I use that to render an animation in for example 60fps?
My case is that I've got the custom, let's say, game renderer written in OpenGL, and I looking for some fancy solution to create an editor detached from engine code.
I've already analyzed some react-native video libraries and I've discovered that frames are injecting as the texture of components, but I'm not sure is it the best solution (I can't find any documentation of those low-level mechanisms in react native).
Any advice? Thanks in advance!
I'm trying to adapt the C++ non QML camera example in QT 5.11 so that it can intercept and add a visual time stamp to the video frames before they are written to the video file.
I want to achieve this without the use of QML.
There is a way to intercept the frames with QVideoprobe but the frame is passed in by const ref and so can't be modified.
Any suggestions except to use qml would be appreciated
Update - the typical way of doing this is to use QAbstractVideoFilter but all the examples I've found only show the filter being applied with QML so I'm initially looking to see how the filter can be applied to the QCamera pipeline in C++.
I'm trying to write a simple AR app in ReactNative, it should simply see 4 predefined markers and draw a rectangle as a boundary on the live preview of the camera, the thing is I'm trying to do the processing in C++ using opencv so as to have the logic of the app in one place accessible to both Android & IOS.
here's what I've been thinking
write the OS dependent code to open the camera and get permissions in (java/ObjC) & the C++ part to do processing on each frame.
call the C++ code (from within the native code) on each frame, and that should return lets say coordinates for the markers.
draw the rect if 4 markers found on the preview in native code (No idea how to achieve this so far but I think it will be native code).
expose that preview (the live preview with the drawn view) to ReactNative (Not sure about that or how to achieve it)
I've looked at the react native camera component but it doesn't provide access to frames & if that's even possible, I'm not sure if it would be a good idea to send frames over the bridge between JS & java/ObjC.
the problem is that I'm not sure of the performance or if that is even possible.
if you know of any ReactNative library that would be great.
Your steps seem sound. After processing the frame in C++, you will need to set the application properties RCTRootView.appProperties in iOS, and emit an event using RCTDeviceEventEmitter on Android. So, you will need an Objective-C wrapper for your C++ code on iOS and a Java wrapper on Android. In either case, you should be able to use the same React Native code for actually drawing the rectangle on top of the camera preview. You're right that the React Native camera component does not have an API for getting individual frames from the camera, so you'll need to write that code natively for each platform.
Recently I want to write a gui application, and its appearance is shown below:
I used MFC before to write normal gui applications, but I have never used qt to write gui applications. But this time I want to use qt, so i'm wondering is there any way to implement such interface, integrating with opengl and opencv subwindows within one application.
Please give me some directions on:
1. Which kind of widget can I draw opengl and opencv subwindows in my application?
2. Is there any way to do event handling in those subwindows respectively?
3. How does qt support for opengl and opencv integration?
There should be no problem.
In fact, I've used openCV and OpenGL in different projects, but I don't see any problem.
You have to convert opencv cv::Mat to QImage (see various posts on the problem on StackOverflow) and then draw on a QLabel or a subclass of it.
For openGL there are special classes: http://qt-project.org/doc/qt-5/qtgui-index.html#opengl-and-opengl-es-integration
You should use event handling as usual in Qt (signal-slot, you know).
OpenCV recently upgraded its display window, when it is used in Qt.
It looks very good, however I did not find any possibility for it to be embedded into an existing Qt GUI window. The only possibility seems to be the creation of a cvNamedWindow or cv::namedWindow, but it creates a free-floating independent window.
Is there any possibility to create that OpenCV window inside an existing GUI? All I could find on the OpenCV forums is an unanswered question, somewhat similar to my own.
There is a straight-forward possibility to show an OpenCV image in Qt, but it has two major problems:
it involves copying the image pixel by pixel, and it's quite slow. It has function calls for every pixel! (in my test application, if I create a video out of the images, and display it in a cvNamedWindow, it runs very smoothly even for multiple videos the same time, but if I go through the IplImage --> QImage --> QPixmap --> QLabel route, it has severe lag even for one single video)
I can't use those nice new controls of the cvNamedWindow with it.
First of all, the image conversion is not as inefficient as you think. The 'function calls' per pixel at least in my code (one of the answers to the question you referenced) are inlined by optimized compilation.
Second, the code in highgui/imshow does the same. You have to get from the matrix to an ARGB image either way. The conversion QImage -> QPixmap is essentially nothing else than moving the data from main memory to GPU memory. That's also the reason why you cannot access the QPixmap data directly and have to go through QImage.
Third, it is several times faster if you use a QGLWidget to draw the image, and I assume you have QT_OPENGL enabled in your OpenCV build. I use QPainter to draw the QPixmap within a QGLWidget, and speed is no issue. Here is example code:
http://sourceforge.net/p/gerbil/svn/19/tree/gerbil-gui/scaledview.h
http://sourceforge.net/p/gerbil/svn/19/tree/gerbil-gui/scaledview.cpp
Now to your original question: Your current option is to take the code from OpenCV, include into your project under a different namespace, and alter it to fit your needs. Apart from that you have no alternative right now. OpenCV's highgui uses its own event loop, connection to the X server etc. and it is nothing you can intercept.
My first guess is to want to say this: I'm sure that if you dig into the code for namedWindow, you will find that they use some sort of standard, albeit not oft-referenced, object for painting said window (that's in the openCV code). If you were ambitious enough, you could extend this class yourself, to interface directly to a frame or custom widget in Qt. There might even be a way to take the entire window and embed it, using a similar method of a Qt frame, or an extension of the (general) widget class. This is a very interesting question and relates rather directly to work I've been doing of late, so I'll continue to think about and research it and see if I can't come up with something else more helpful.
[EDIT] What specific new controls are you so interested in? It might be more efficient on the part of the programmer to extend a Qt control to emulate that, as opposed to my first suggestion.[/EDIT]
simply check out the opencv highgui implementation. as i recall it uses qt.