Thanks for reading my post.
I have a problem with multithreading an opencv application I was hoping you guys could help me out with.
My aim is to Save 400 frames (in jpeg) from the middle of a video sequence for further examination.
I have the code running fine single threaded, but the multithreading is causing quite a lot of issues so I’m wondering if I have got the philosophy all wrong.
In terms of a schematic of what I should do, would I be best to:
Option 1 : somehow simultaneously access the single video file (or make copies?), then with individual threads cycling through the video frame by frame, save each frame when it is between predetermined limits? E.g. thread 1 saves frames 50 to 100, thread 2 saves frames 101 to 150 etc.
Option 2 : open the file once, cycle through frame by frame then pass an individual frame to a series of unique threads to carry out a saving operation. E.g. frame 1 passed to thread 1 for saving, frame 2 to thread 2 for saving, frame 3 to thread 1, frame 4 to thread 2 etc etc.
Option 3 : some other buffer/thread arrangement which is a better idea than the above!
I'm using visual C++ with the standard libs.
Many thanks for your help on this,
Cheers, Kay
Option 1 is what i have tried to do this far, but because of the errors, i was wondering if it was even possible to do this! Can threads usually access the same file? how do I find out how many threads i can have?
Certainly, different threads can access the same file, but it's really a question if the supporting libraries support that. For reading a video stream, you can use either OpenCV or ffmpeg (you can use both in the same app, ffmpeg for reading and OpenCV for processing, for example). Haven't looked at the docs, so I'm guessing here: either lib should allow multiple readers on the same file.
To find out the number of cores:
SYSTEM_INFO sysinfo;
GetSystemInfo( &sysinfo );
numCPU = sysinfo.dwNumberOfProcessors;
from this post . You would create one thread / core as a starting point, then change the number based on your performance needs and on actual testing.
Related
I am writing a C++ program (Win64) using C++ Builder 11.1.5 that captures video from a web cam and stores the captured frames in a WMV file using the sink writer interface as described in the following tutorial:
https://learn.microsoft.com/en-gb/windows/win32/medfound/tutorial--using-the-sink-writer-to-encode-video?redirectedfrom=MSDN
The video doesn't need to be real time using 30 Frames per second as the process being recorded is a slow one so I have set the FPS to be 5 (which is fine.)
The recording needs to run for about 8-12 hours at a time and using the algorithms in the sink writer tutorial, I have seen the memory consumption of the program go up dramatically after 10 minutes of recording (in excess of 10 Gb of memory). I also have seen that the final WMV file only becomes populated when the Finalize routine is called. Because of the memory consumption, the program starts to slow down after a while.
First Question: Is it possible to flush the sink writer to free up ram while it is recording?
Second Question: Maybe it would be more efficient to save the video in pieces and finalize the recording every 10 minutes or so then start another recording using a different file name such that when 8 hours is done the program could combine all the saved WMV files? How would one go about combining numerous WMV files into one large file?
I am learner in Cpp and opencv,I am trying to access the same video from multiple thread and while doing it I am getting deadlock which is pretty expected
I am creating n threads and trying to process the video by dividing it in n part and simultaneously process it in different threads.this is my void function.I found some python solution of doing but didn't able to understand that.
void *finddensity(void *videoinfo)
{
VideoCapture cap(video.mp4);
//do some processing on each frame
}
and then I am creating thread using pthread_create
is there any ways to access the video avoid any deadlock and also there is struct for videoinfo ?
Thank you
So as the task was just a course assignment what I did is loaded the whole video frames in memory(which is not a good practice) and than used mutex lock to access the frames from every thread the video size was small (174MB) but I was able to store video at 5FPS in my memory and then completed the task.
But if there is any other general or better solution(which should be there) please respond here thanks BiOS for formatting code :-).
Forgive me for I am not an expert in multi-threading by any means and need some assistance. So just some given knowledge before I get to my question:
Pre-Knowledge
Developing C++ code on the Jetson TK1
Jetson has 4 CPU cores (quad-core CPU ARMv7 CPU)
From what I have researched, each core can utilize one thread ( 4 cores -> 4 threads)
I am running a computer vision application which uses OpenCV
Capturing frames from a webcam as well as grabbing frames from a video file
Psuedo-Code
I am trying to optimize my multi-threaded code such that I can gain the maximum amount of performance for my application. Currently this is basic layout of my code:
int HALT=0;
//Both func1 and func2 can be ran parallel for a short period of time
//but both must finish before moving to the next captured webcam frame
void func1(*STUFF){
//Processes some stuff
}
void func2(*STUFF){
//Processes similar stuff
}
void displayVideo(*STUFF){
while(PLAYBACK!=DONE){
*reads video from file and uses imshow to display the video*
*delay to match framerate*
}
HALT=1;
}
main{
//To open these I am using OpenCVs VideoCapture class
*OPEN VIDEO FILE*
*OPEN WEBCAM STREAM*
thread play(displayVideo, &STUFF);
play.detach();
while(HALT!=1){
*Grab frame from webcam*
//Process frame
thread A(func1,&STUFF);
thread B(func2,&STUFF);
A.join();
*Initialize some variables and do some other stuff*
B.join();
*Do some processing... more than what is between A.join and B.join*
*Possibly display webcam frame using imshow*
*Wait for user input to watch for terminating character*
}
//This while loop runs for about a minute or two so thread A and thread
//B are being constructed many times.
}
Question(s)
So what I would like to know is if there is a way to specify which core/thread I will use when I construct a new thread. I fear that when I am creating threads A and B over and over again, they are jump around to different threads and hampering the speed of my system and/or the reading of the video. Although this fear is not well justified, I see very bizarre behavior on the four cores when running the code. Typically I will always see one core running around 40-60% which I would assume is either the main thread or the play thread. But as for the other cores, the computational load is very jumpy. Also throughout the application playing, I see two cores go from around 60% all the way to 100% but these two cores don't remain constant. It could be the first, second, third, or even fourth core and then they will greatly decline usually to about 20->40%. Occasionally I will see only 1 core drop to 0% and remain that way for what appears to be another cycle through the while loop(i.e. grab frame, process, thread A, thread B, repeat). Then I will see all four of them active again which is the more expected behavior.
I am hoping that I have not been too vague in this post. I just see that I am getting slightly unexpected behavior and I would like to understand what I might be doing incorrectly or not accounting for. Thank you to whomever can help or point me in the right direction.
I would like to ask for advice. As I am not very experienced user of C++. I lack a bit knowledge about threads - but I've been doing a lot in Android so I know the general idea.
I need to write 8 *.wav files at once. I have a callback called very often with upcoming signal from 8 input channels. I need to save data to *.wav from each channel (recording). This require me to open file every time I get new data and write additional 256 samples of data at the end of it.
Doing so with 2 inputs is fine, but after 3 or more my input latency is starting to increase. Processor is lagging so probably I have to do it in some kind of thread.
I think it should be quite common problem but I haven't learn yet how to handle it. Can someone explain me right way to do it? Is it necessary to use it http://www.cplusplus.com/reference/thread/thread/ or are there any other simple/elegant patterns.
You need to record, or save, the data from 8 input channels.
I highly recommend at least 8 large buffers to contain the data.
When there is a pause in the input or in the background, you can write the data to the files. Wait for a large amount of data to collect before writing the buffers to the files. The file I/O loves to process large blocks of data rather than many small ones. You can always flush the output streams which tells the OS to write to the file.
If you want to play with threads, I recommend at least three.
Thread 1 -- read input channels and store in buffers.
Thread 2 -- GUI
Thread 3 -- Writes buffers to files.
Remember that thread 1 is the highest priority. When it sees a low amount of space remaining in the buffer, it should wake up thread 3 to write out the buffers to the files.
You should maintain at least 2 buffers for each input channel. This is called double buffering and allows thread 3 to write the buffer to file while thread 1 is reading the input channel data into another buffer.
Does anybody know any reasons why IMemAllocator:GetBuffer (Directshow) hangs, apart from all samples being in use?
I have a directshow application which uses GMFBridge by Geraint Davies to connect two graphs. The GMFBridge is used to be able to switch inputs, but I am not switching in this situation. The application captures audio and video, and should do that non-stop. But after about 10 hours it stops. I found out both audio and video got stuck in a call to IMemAllocator:GetBuffer:
/* _COM_SMARTPTR_TYPEDEF(IMemAllocator, IID_IMemAllocator); */
/* IMemAllocatorPtr m_pCopyAllocator; */
hr = m_pCopyAllocator->GetBuffer(&pOut, NULL, NULL, 0);
If all samples are in use, this function can block, but I am pretty sure this is not the case. There are two threads calling this function, one for the video and one for the audio samples. The Audio thread blocks first, and after the GetBuffer has returned a buffer for almost 60 video samples, the video thread blocks too. (this is about 2 seconds later)
After almost 8 hours both threads continue for a small period, first the audio thread, and after 45 buffers for audio samples have been returned, the video thread unblocks too.
So because both threads do not block at the same time, it looks to me there is not a problem with all samples being in use.
The stacktrace shows a function inside quartz.dll is being called at that moment.
UPDATE
It looks like there was a memoryleak caused by decoder filters already installed on the pc. The graph included decoding of mpeg, for example the audio decoding used a cyberlink decoder. After installing ffdshow, the ffdshow audio + video decoder was used instead, and the problem seems to be disappeared. Lesson learned, do not depend automatically on existing filters.
Not sure that I can debug this from the info given. Can you create a log file (create an empty file c:\gmfbridge.txt, run until it hangs, then zip the file and email it). Also, if you set up your symbols with _NT_SYMBOL_PATH, you could look at the stack trace to see where in quartz.dll the various threads are.
G