I'm using MLT framework to create a video player for my app in which users will be able to preform some small video editing for a specific task. I'm also using QT for this app. I have started with essentially the BuildOnMe example which can be found here
The problem is the player crashes on videos after a certain time (always different).
At one point I was printing the number of frames to see if it was on the same number (it isn't) and when it crashed it printed this: [mlt_pool] out of memory
Do I need to take care of memory management for mlt?
I'm using QT5.3
My code, in case it helps, can be found here (I didn't add the .h)
I found out the problem was from the Mlt::Frame created in the function on_frame_show
This frame needs to be deleted, in the example it is used on the mac openGL class. But on windows since it's never used it'll quickly build up on the memory.
Related
I'm trying to develop a little application in which you can load a mp3 file and play it in variable speeds! (I know it already exists :-) )
I'm using Qt and C++. I already have the basic player but I'm stuck with the rate thing, because I want to change the rate smoothly (like in Mixxx) without stopping the playback! The QMediaPlayer always stops if I change the value and creates a gap in the sound. Also I don't want the pitch to change!
I already found something called "SoundTouch" but now I'm completely clueless what to do with it, how to process my mp3 data and how to get it to the player! The "SoundTouch" Library is capable of doing what I want, i got that from the samples on the homepage.
How do I have to import the mp3 file, so I can process it with the SoundTouch functions
How can I play the output from the SoundTouch function? (Perhaps QMediaPlayer can do the job?)
How is that stuff done live? I have to do some kind of stream I guess? So I can change the speed during play and keep on playing without gaps. Graphicaly in my head it has to be something that sits between the data and the player, where all data has to go through live, with a small buffer (20-50 ms or so) behind to avoid gaps during processing future data.
Any help appreciated! I'm also open to any another solution then "SoundTouch" as long as I can stay with Qt/C++!
(Second thing: I want to view a waveform overview aswell as moving part of it (around actual position of the song), so I could also use hints on how to get the waveform data)
Thanks in advance!
As of now (Qt 5.5) this is impossible to do with QMediaPlayer only. You need to do the following:
Decode the audio using GStreamer, FFMpeg or (new) QAudioDecoder: http://doc.qt.io/qt-5/qaudiodecoder.html - this will give you raw PCM stream;
Apply SoundTouch or some other library to this raw data to change the pitch. If GPL is ok, take a look at http://nsound.sourceforge.net/examples/index.html, if you develop proprietary stuff, STK might be a better choice: https://ccrma.stanford.edu/software/stk/
Output the modified data into audio device by using QAudioOutput.
This strategy uses Qt as much as possible, and brings you the best platform coverage (you still lose Android though as it does not support QAudioOutput)
I need to do some basic video processing from the Webcam in Windows Phone 8.1.
I cannot find any examples of how to access the webcam preview buffer. The Microsoft examples (very few) all have a video preview frame activate, I can find none that show how to subscribe to a 'new frame ready' event, or where the buffer is.
The MediaCapture, and CaptureDevice appear to be the main ways of reaching the camera in this api.
Can anyone point to a specific example? For instance, a QR code reader, or maybe a program that adds video effects, like greyscale, would need to attach an event to every frame.
Thanks.
It sounds like you need to access the buffer from the preview stream of the phone cameras. If you absolutely need to target 8 and 8.1, then you should look into GetPreviewBufferArgb(out int[] pixels). See here: https://msdn.microsoft.com/en-us/library/windows.phone.media.capture.cameracaptureframe.getpreviewbufferargb.aspx
But if you can instead target Windows 10, you get the benefit of fully universal APIs, and you don't take a dependency on the likely soon to be deprecated (if not already) Windows.Phone.Media.Capture namespace. I would strongly recommend doing this instead, as the APIs will be easier to work with, and any 8.1 device can upgrade to 10 for free.
Here is a fully working Windows 10 sample: http://aka.ms/2015buildgetpreviewframesample, which was shown off at the last Build conference (video here: http://channel9.msdn.com/Events/Build/2015/2-730).
This is going to be my first question in StackOverflow after several days looking for an explanation. Please, be gentle with me for asking because I know my problem is a bit bizarre to be a general problem.
I made a MF capture video application, based in the Microsoft example 'CaptureToFile'. It did work on Windows 7 x64. I upgraded to Visual Studio 2013 without problems. Problems arose when I try to put all the development on a Windows 8.1 x64 machine.
The app compiles and executes without error, but it's UNABLE to capture samples by using m_pReader->ReadSample() in asynchronous mode; only the first two samples arrive to OnReadSample method; and there must be 'control' samples, because the IMFSample is null in all of them. After that, the app gets 'hanged' waiting for data.
I've tried the original MFCaptureToFile sample with the same sad results.
Of course, I think hardware and software are similar (the same capture card with the same driver version, both are desktop PC...)
Do you know any possible reason for this behaviour? in Win7 everything is working flawless! Or at least, if you could light me a bit about new paths for finding what's happening
Thanks in advance
UPDATE: There is another 'player' in the game. Looking into the threads, I see that a worker thread is in 'RTWorkQ.dll', the real-time work queue container, specific only for Windows 8. I will go on investigating. In the meantime, if you have any idea, something to share, I'll be glad to hear you.
UPDATE 2: I've modified the sample MFCaptureToFile to get the video samples synchronously, because I thought the problem could be due to the asynchronous behaviour; related with the queues. I've to say that the problem persist even with this change. The second time it tries to read a sample, the application gets 'hanged' waiting for a reading that doesn't never arrives.
UPDATE 3: I've tried with the CaptureEngine sample application that uses another MF way to capture video (MFCaptureEngine). It builds and runs flawlessly but doesn't show any images when starting the 'preview' and doesn't record any useful, only non-playable files.
UPDATE 4: I've installed Visual Studio 2010 Ultimate in Windows 8 PRO. The sample MFCaptureToFile fails again in the sample. It's unable to read a 2nd sample from the frame grabber. I'm starting to think that can be an incompatibility between the capture card (Datapath VisionRGB-E1S) and Windows 8 PRO despite the driver assures it works fine in this platform and the test program shows images. Tomorrow I'm going to try the test with an external USB webcam.
Finally, I have figured out the reason of this problem.
With Windows 8.1 release Microsoft has introduced New AVStream Interfaces for Windows 8.1
There is a small but very important change in KS_FRAME_INFO structure - the new FrameCompletionNumber member.
An identifying sequence number for the frame in the completed queue.
This number is used to verify proper frame order. When this value is
0, the frame was cancelled. This member is available starting with
Windows 8.1.
DirectShow doesn't care about this number. And MediaFoundation cares.
So, you cannot just fix that on your user-mode side. The manufacture developers must release an update. Btw, I have two webcams - Logitech C270 and Creative Live Socialize HD. Logitech supports Metro while Creative does not.
I have successfully updated my driver with only a few lines of code (to set up FrameCompletionNumber properly).
UPD. similar thread http://www.osronline.com/showthread.cfm?link=255004
It must be a problem of the frame grabber Datapath VisionRGB-E1S. I've tried with the brand-new USB webcam LifeCam Studio, and everything worked fine.
I will left for other future thread why this unpaired behaviour between Windows 8 and Windows 7, but it could be something related to the User-mode access...
I had the same kind of issue:
IMFSourceReader was obtained successfully
reader->SetCurrentMediaType() reported no error.
reader->ReadSample() was successful.
then OnReadSample() was called only once and the hrStatus argument 0x80070491
For me, the issue was that I modified the video subtype IMFMediaType, then applied to the reader as current media type.
I am developing game for iPhone using Cocos2d, the problem I am having now is the game crashes when it actually runs on the iPhone, but on the simulator is fine. Maybe the game has consume so much memory. Hmm I am thinking that maybe I create so many objects. Such as when the user play, if he makes mistakes, the game display a error subtitle on the screen, and the subtitles are different for each actions the player makes. I don't know the normal way developer use to create subtitles for game in Cocos2D. How do they switch the subtitle easily without consuming huge memory. I am trying the best way to reduce memory usage now. I switched Sprite to AtlasSprite, for timer and point counter I used LabelAtlas too. It still crashes, even though it uses less memory now.
Could anyone give me some good suggestions, like what standard people follow to create games in cocos.
First, investigate what the real cause is... don't just guess.
Run a debug build on the device, attached to the debugger. See where the crash is happening in the stack trace.
If you determine it is memory related, then run with insturmentation: object allocation and leaks are the ones you want.
The scenario is such: there's a program which loads my .dll/.so and calls a function from within it, possibly multiple times, each time expecting a different pointer to state. It uses the different states later in other calls into the dll. (It's a game AI, if you need context; each state is a AI player.)
What I want is a cross-platform way of creating a canvas window for each of those states (for visualization, debugging, etc.) I tried wx, but put it on hold, since it didn't appear to be easy at all. Are there any neat and small libraries that could do that or should I just go with WinAPI/X...?
Edit: Assume I cannot modify the host program.
Qt is simpler to set up and drive than Wx, in my experience. It's very cross platform too.
If you want to render some graphics from inside your DLL function without passing in any pointers to QImage or QWidget type things, probably the thing to do is use OpenGL. Your DLL should just render to the current OpenGL context, which is global state and can just be setup outside the DLL (maybe using QGLWidget).
Update: Ah, I just noticed your edit re not being able to modify the host code. This is a problem: any windows you create really need to be plugged into the host apps' event loop to work properly (e.g receive WM_PAINT when exposed/resized). It's certainly possible in win32 for any old code (e.g your library) to just CreateWindow and draw its contents with GDI whenever it gets the chance, but the general window behaviour may be pretty broken (it may not work at all with Vista's double buffering; I haven't tried). What I typically find easiest in this situation is simply to dump out image files and review then afterwards with image viewer of choice. IMHO this is actually more useful for debugging than a "live" window because you can step backwards and forwards, zoom in, apply image-enhancement to highlight various issues, compare against previous runs for regression testing etc etc. (If you really want the "live" views, write an image displayer which monitors a directory for new images, or streams them through a named pipe or something).
If you just want simple graphics, no widgets, SDL is very easy to use. If you do need complex controls, use Qt, as timday said.
You might check out IUP. It interfaces really well with Lua, and can be used entirely from an extension DLL there so it seems plausible that its C API could be used from a DLL plugged into something else.
IUP will get you a framework for opening a window containing the usual suspect kinds of controls, including a canvas. Its related library CD will give you the usual drawing operations in that canvas.
Current releases are portable between Windows and *nix. The next major release will support MacOSX too.