I'm developing a small video editor to make quick edits to a multiple audio track video, using Qt. I'm a bit confused about whether it is possible or not to handle multiple audio tracks in payback and processing in Qt.
What I want to do with the video
list audio tracks
for each track, manage its volume, and choose to perform channel duplication on a mono track (to make it stereo).
play the video with the setting I chose for audio tracks, eventually being able to change settings on the fly.
extract a part of the video with the settings I chose.
I'm not sure if Qt can handle this by itself, or if I need to rely on a library specialized in video processing. Therefore, my question is double
If it is possible in Qt-only : how ?
I suppose I need to use QMediaPlayer class, but it doesn't look like it can handle multiple tracks at once.
Maybe by splitting the media into several sub-media, but then how to synchronize their playback?
Otherwise : external library. Are there any caveats to avoid?
I wonder what is the best way, if there is one, to display video frames (assuming audio will be handled by external lib)?
Should I draw frames directly on a QWidget, or should I use OpenGL directly? Or another method?
Note: I'm not forcibly looking for a detailed answer, I'm fine with short ones and/or external resources.
After a bit of searching around, I found that it is impossible to do in with Qt as for now.
I found several libraries able to fulfill such purpose (list is non exhaustive for sure) :
libvlc, which was able to render easily on a QWidget surface, but at the time I looked, was not able to play multiple channels at a time. Next version should be able to do it tho.
libffmpeg and libav, which are a bit too low-level for what I want, and which I found fairly complicated to use to render on a QWidget surface.
libopenshot, which has quite a difficult learning curve, and not that much documentation unfortunately, as it is oriented toward nonlinear video editing.
libgstreamer. The one I chose.
I found libgstreamer to be the best for my purpose, being fairly high level, and well documented. It also is flexible to use, as it allows loading from and dumping to any kind of file automatically.
It is able to render directly on a QWidget surface, which is super convenient.
On the other hand, it has an async and dynamic design which requires a bit more planning and error management, but documentation is there to help.
So far I didn't encountered any major problem so far, I'll update this post if something new comes up.
Related
For Imageprocessing I want to get all pixel information from a given process.
Concrete its for testing an image hashing algorithm for identifying hearthstone cards, so i need to get a screenshot of the given process.
How can I solve it in windows?
My idea so far:
Get the process name.
Get the process ID
Get Window Handle
I have no idea how to go further from this point.
I hope it understandable what I want to achieve.
Unfortunately, there is no general method for getting the pixels of a particular window that I would be aware of. Depending on how the target application draws itself, this task can be very simple or very complicated. If we were talking about an application that uses good old GDI, then you could just get yourself an HDC to the window via GetWindowDC() and BitBlt/StretchBlt the content over into a bitmap of your own.
Unfortunately, the target application in your case appears to be a game. Games typically use 3D graphics APIs like Direct3D or OpenGL for drawing. Assuming that you cannot simply modify the target application to just send the desired data over to you out of its own free will, the only way to specifically record output from such applications that I'm aware of is to hook into the graphics API and capture the data from underneath the API. This can be done. However, implementing such a system is quite involved. There might be existing libraries to aid with writing such applications, but I don't know any that I could recommend here. If you don't have to capture the game content in real-time, you could just use a screen recording application to, e.g., record a video and then use that video as input for your algorithm. There are also graphics debugging tools like NSight Graphics or RenderDoc that you could use. Be aware that games, particularly online games, these days often have cheat protection systems that are likely to get very angry at you if you attempt to hook into the game…
Apart from all that, one alternative approach might be to use DXGI Output Duplication to just capture the entire desktop. While you won't be able to target one specific application (as far as I know), this would potentially have several advantages: First of all, it's only moderately complex to set up compared to a fully-fledged API-hook-based approach. Second, it should work regardless of what API the target application uses and even if the application is in fullscreen mode. Third, since you will have the data delivered straight from the operating system, you shouldn't have any issues with cheat protection. You can use MonitorFromWindow() to get the monitor your target window appears on and then enumerate all outputs of all DXGI adapters to find the one that corresponds to that HMONITOR…
I am trying to make a simple QT android app, but basically my problem is that on my main screen I have about 250 little images that i want to scroll. But I really need the scrolling to be fluent and fast. First I tried it using QML but it wasnt really fast, then I tried to make the app in qt designer and use widgets but that was very slow. Then I tried using openGL but on android I can only use openGL ES and I cant find so much examples because every example that I find is much more advanced than I need.
But basically my main question is, what do you think is the best way to solve my problem and if its openGL which way of using it is the best that could solve it?
Thank you.
Neither approach should have problems when scrolling when compared to a native application on the same device. Check the following:
Make sure to measure performance only in release-builds, with QML debugging disabled and no debugger attached.
Maybe your device simply can't keep up with so many images in one view - then it's not a Qt problem. Compare with a 'native' java-App to see if this is the case.
Check if you implemented everything correctly; e.g. check if theres anything running in your main-loop or some events happenening repeatedly which consumes CPU time
And some more general advice:
Downscale your images to the appropriate view-size before giving them to the UI, as they might have to be re-scaled on every frame-update and/or consume graphics memory otherwise. E.g. dont set the source to a 1024x1024 image when it's going to show in a 64x64 view
Remove transparency from the images if they are going to display on solid-colored background anyway.
Dont overlay the images with other widgets/controls
If you're still getting a 'slow' UI, maybe try to merge all or multiple images and their surrounding UI/Controls into one or more bigger images
Very long views are not user-friendly. Maybe implement a pager or tab-view etc. to divide your list into multiple views. This way you can also decrease load-time
Dont try to implement an interface in openGL yourself. It's unlikely you'll make a better one than you already get with QtWidgets and QtQuick.
I would like to have a tool to debug 2D planar meshes. I would like to be able to display them and debug to be able to debug certain things.
Is there a widget (for any toolkit on Linux - QT, GTK+Cairo, ...) which would display the images, scroll them and zoom it. Is there any widget which would handle it (without need to implement zooming, scrolling etc. by hand)?
Side requirements:
Needs to work on CentOS 6
I need it for C++. Unfortunately changing the build system in my situation is harder then it sounds.
The Qt Graphics View is certainly a useful tool, it gives you scrolling, zooming, rotating easily. You probably want to learn the basics of Qt before. (and Qt is in C++).
Here's an off the wall suggestion.
This would be fairly easy to implement in a web browser. Web browser engines already have the base functionality for resizing and scrolling over images. You may need a little JavaScript to bind it together, of course.
So why not use WebKit? There are bindings for many of the leading toolkits (e.g. QWebView for Qt), so you could take your pick of which one you're most comfortable with.
Yes, it's overkill. But it's code you don't have to write, and time is money.
OpenCV recently upgraded its display window, when it is used in Qt.
It looks very good, however I did not find any possibility for it to be embedded into an existing Qt GUI window. The only possibility seems to be the creation of a cvNamedWindow or cv::namedWindow, but it creates a free-floating independent window.
Is there any possibility to create that OpenCV window inside an existing GUI? All I could find on the OpenCV forums is an unanswered question, somewhat similar to my own.
There is a straight-forward possibility to show an OpenCV image in Qt, but it has two major problems:
it involves copying the image pixel by pixel, and it's quite slow. It has function calls for every pixel! (in my test application, if I create a video out of the images, and display it in a cvNamedWindow, it runs very smoothly even for multiple videos the same time, but if I go through the IplImage --> QImage --> QPixmap --> QLabel route, it has severe lag even for one single video)
I can't use those nice new controls of the cvNamedWindow with it.
First of all, the image conversion is not as inefficient as you think. The 'function calls' per pixel at least in my code (one of the answers to the question you referenced) are inlined by optimized compilation.
Second, the code in highgui/imshow does the same. You have to get from the matrix to an ARGB image either way. The conversion QImage -> QPixmap is essentially nothing else than moving the data from main memory to GPU memory. That's also the reason why you cannot access the QPixmap data directly and have to go through QImage.
Third, it is several times faster if you use a QGLWidget to draw the image, and I assume you have QT_OPENGL enabled in your OpenCV build. I use QPainter to draw the QPixmap within a QGLWidget, and speed is no issue. Here is example code:
http://sourceforge.net/p/gerbil/svn/19/tree/gerbil-gui/scaledview.h
http://sourceforge.net/p/gerbil/svn/19/tree/gerbil-gui/scaledview.cpp
Now to your original question: Your current option is to take the code from OpenCV, include into your project under a different namespace, and alter it to fit your needs. Apart from that you have no alternative right now. OpenCV's highgui uses its own event loop, connection to the X server etc. and it is nothing you can intercept.
My first guess is to want to say this: I'm sure that if you dig into the code for namedWindow, you will find that they use some sort of standard, albeit not oft-referenced, object for painting said window (that's in the openCV code). If you were ambitious enough, you could extend this class yourself, to interface directly to a frame or custom widget in Qt. There might even be a way to take the entire window and embed it, using a similar method of a Qt frame, or an extension of the (general) widget class. This is a very interesting question and relates rather directly to work I've been doing of late, so I'll continue to think about and research it and see if I can't come up with something else more helpful.
[EDIT] What specific new controls are you so interested in? It might be more efficient on the part of the programmer to extend a Qt control to emulate that, as opposed to my first suggestion.[/EDIT]
simply check out the opencv highgui implementation. as i recall it uses qt.
I am exploring a possibility to write a kind of a notebook analogue that would reproduce the look and feel of using a traditional notebook, but with the added benefit of customizing the page in ways you can't do on paper - ask the program to lay ruled paper here, grid paper there, paste an image, insert a recording from the built-in camera, try to do handwriting recognition on the tablet input, insert some latex for neat formulas and so on. I'm pretty interested in developing it just to see if writing notes on computer can come anywhere close to the comfort plain paper + pencil offer (hard to do IMO) and can always turn it in as a university C++ project, so double gain there.
Coming from the type of project there are certain requirements for the user interface:
the user will be able to zoom, move and rotate the notebook as he wishes and I think it's pretty sensible delegate it to OpenGL, so the prospective GUI needs to work well with OGL (preferably being rendered in it)
the interface should be navigable with as little of keyboard input as user wishes (incorporating some sort of gestures maybe) up to limiting the keyboard keys as modifiers to the pen movements and taps; this includes tablet and possible multitouch support
the interface should keep out of the way where not needed and come up where needed and be easily layerable
the notebook sheet itself will be a container for objects representing the notebook blurbs, so it would be nice if the GUI would be able to overlay some frames over the exact parts of the OpenGL-drawn sheet to signify what can be done with given part (like moving, rotating, deleting, copying, editing etc.) and it's extents
In terms of interface it's probably going to end up similar to Alias' Sketch Book Pro:
picture.
As far as toolkits go I'm considering Qt and nui, but I'm not really aware how well would they match up the requirements and how well would they handle such an application.
As far as I know you can somehow coerce Qt into doing widget drawing with OpenGL, but on the other hand I heard voices it's slot-signal framework isn't exactly optimal and requires it's own preprocessor and I don't know how hard would be to do all the custom widgets I would need (say color-wheel, ruler, blurb frames, blurb selection, tablet-targeted pop-up menu etc.) in the constraints of Qt. Also quite a few Qt programs I've had on my machine seemed really sluggish, but it may be attributed to me having old PC or programmers using Qt suboptimally rather to the framework itself.
As for nui (http://www.libnui.net/) I know it's also cross-platform and all of the basic things you would require of a GUI toolkit and what is the biggest plus it is OpenGL-enabled from the start, but I don't know how it is with custom widgets and other facets and it certainly has smaller userbase and less elaborate documentation than Qt.
The question goes as this:
Does any of these toolkits fulfill (preferably all of) the requirements or there is a well fitting toolkit I haven't come across or maybe I should just roll up my sleeves, get SFML (or maybe Clutter would be more suited to this?) and something like FastDelegates or libsigc++ and program the GUI framework from the ground up myself?
I would be very glad if anyone had experience with a similar GUI project and can offer some comments on how well these toolkits hold up or is it worthwhile to pursue own GUI toolkit in this case.
Sorry for longwindedness, duh.
Have you tried FLTK? It is made with 3D graphics programming in mind and has interfaces to OpenGL. I wrote some FLTK->Scheme bindings and found the API to be real fun to work with.
OpenGL font support is terrible, in my experience. It sounds like you're going to have to develop all your own custom widgets anyway so don't even bother with a toolkit. You'll spend more time learning the toolkit, trying to figure out how to get that toolkit to work with OpenGL, and and trying to figure out how to make your special widgets in that toolkit than you will just rolling your own. I wouldn't give this advice in just any situation but it sounds like your application and your widget set is going to be very unique. Make a superclass for all widgets, define a draw method, even handler methods, etc., for override, and you've already done most of what those frameworks would do for you.
Also I'm sure you know this but this is an enormous project so you should initially narrow it down to a few simple objectives for a first iteration.