I am quite confused about the above topic and its question. I understand from the tutorial and webpage API that to delete an object,
you should use gst_object_unref(obj) which would decrement the reference value count by one.
When the reference count of object reaches zero, the object will be automatically deleted
However there seem to be functions that will increase the internal reference count of object, correct me if i am wrong
For example, I write a snippet of the function to sync a newly joined element with a pipeline. The reference count increase to 4 from 1
//// record -> mfsink is the splitmux
g_print("ref count (1) splitmuxsink is %d \n", GST_OBJECT_REFCOUNT(record->mfsink));
/// set to playing
if (!gst_element_sync_state_with_parent (GST_ELEMENT (record->mfsink)))
{
g_error ("Failed to go into PLAYING state");
return -4;
}
else
{
g_print("Splitmux set to playing \n);
}
g_print("thread Live: ref count (2) of splitmuxsink is %d \n", GST_OBJECT_REFCOUNT(record->mfsink));
The result:
ref count of (1) splitmuxsink is 1
Splitmux set to playing.
ref count of (2) splitmuxsink is 4
Where else, if we call gst_object_unref() function, it will only unreference the oreference counter by 1.
Similar with some other functions gst_element_set_state and gst_element_request_pad will increase reference count by 3 and 2 respectively.
I am trying to "recreate/refresh" an object in dynamic pipeline, so that each iteration of start/stop will create/renew an object (ok that would create overhead) but it seems that to delete an object is impossible.
Or it is possible? Thanks for your help
Regards
I think it will not be appropriate approach to that problem.
You can use GST_OBJECT_REFCOUNT_VALUE macro for iteration over "while loop".
while(GST_OBJECT_REFCOUNT_VALUE(obj))
{
gst_object_unref (obj);
}
Related
I am working on a data processing pipeline with some OpenCV code, after implementing my pipeline I found no speedup, also no slowdown. I am trying to investigate why this is so.
I came up with the following example:
int start = 0;
tbb::parallel_pipeline(16,
tbb::make_filter<void, int>(tbb::filter::serial_out_of_order, [&](tbb::flow_control& fc){
if(start < 1000) {
return start++;
}
fc.stop();
return start;
}) &
tbb::make_filter<int, int>(tbb::filter::parallel, [](int num){
std::cout << num << std::endl;
return num + 1;
}) &
tbb::make_filter<int, void>(tbb::filter::parallel, [](int num){
})
);
When this code executes, 1-1000 is printed sequentially. Is this correct behavior? Or do I have an issue with my environment?
Reordering is rather unlikely to be seen at the start of the second filter in practice.
The parallel_pipeline works in such a way that the same thread puts a given item through the pipeline for as long as possible (in your pipeline, all filters after the first are parallel, so the same thread will execute all three filters for an item). The overhead for a thread to move an item to the next filter is much less than what another thread needs to steal a task for the next item, process the first filter, and then also move to the second one. Reordering is still possible if e.g. the first thread is preempted by OS, but rather unlikely.
For better chances to observe out-of-order execution, move your print statements to the third filter and add some random amount of "work" to the second one, so that the time for it to process an item varies.
I have created using the C API of ffmpeg a C++ application that reads frames from a file and writes them to a new file. Everything works fine, as long as I write immediately the frames to the output. In other words, the following structure of the program outputs the correct result (I put only the pseudocode for now, if needed I can also post some real snippets but the classes that I have created for handling the ffmpeg functionalities are quite large):
AVFrame* frame = av_frame_alloc();
int got_frame;
// readFrame returns 0 if file is ended, got frame = 1 if
// a complete frame has been extracted
while(readFrame(inputfile,frame, &got_frame)) {
if (got_frame) {
// I actually do some processing here
writeFrame(outputfile,frame);
}
}
av_frame_free(&frame);
The next step has been to parallelize the application and, as a consequence, frames are not written immediately after they are read (I do not want to go into the details of the parallelization). In this case problems arise: there is some flickering in the output, as if some frames get repeated randomly. However, the number of frames and the duration of the output video remains correct.
What I am trying to do now is to separate completely the reading from writing in the serial implementation in order to understand what is going on. I am creating a queue of pointers to frames:
std::queue<AVFrame*> queue;
int ret = 1, got_frame;
while (ret) {
AVFrame* frame = av_frame_alloc();
ret = readFrame(inputfile,frame,&got_frame);
if (got_frame)
queue.push(frame);
}
To write frames to the output file I do:
while (!queue.empty()) {
frame = queue.front();
queue.pop();
writeFrame(outputFile,frame);
av_frame_free(&frame);
}
The result in this case is an output video with the correct duration and number of frames that is only a repetition of the last 3 (I think) frames of the video.
My guess is that something might go wrong because of the fact that in the first case I use always the same memory location for reading frames, while in the second case I allocate many different frames.
Any suggestions on what could be the problem?
Ah, so I'm assuming that readFrame() is a wrapper around libavformat's av_read_frame() and libavcodec's avcodec_decode_video2(), is that right?
From the documentation:
When AVCodecContext.refcounted_frames is set to 1, the frame is
reference counted and the returned reference belongs to the caller.
The caller must release the frame using av_frame_unref() when the
frame is no longer needed.
and:
When
AVCodecContext.refcounted_frames is set to 0, the returned reference
belongs to the decoder and is valid only until the next call to this
function or until closing or flushing the decoder.
Obviously, from this it follows from this that you need to set AVCodecContext.refcounted_frames to 1. The default is 0, so my gut feeling is you need to set it to 1 and that will fix your problem. Don't forget to use av_fame_unref() on the pictures after use to prevent memleaks, and also don't forget to free your AVFrame in this loop if got_frame = 0 - again to prevent memleaks:
while (ret) {
AVFrame* frame = av_frame_alloc();
ret = readFrame(inputfile,frame,&got_frame);
if (got_frame)
queue.push(frame);
else
av_frame_free(frame);
}
(Or alternatively you could implement some cache for frame so you only realloc it if the previous object was pushed in the queue.)
There's nothing obviously wrong with your pseudocode. The problem almost certainly lies in how you lock the queue between threads.
Your memory allocation seems same to me. Do you maybe do something else in between reading and writing the frames?
Is queue the same queue in the routines that read and write the frames?
I am trying to write C++ code which saves incoming video frames to disk. Asynchronously arriving frames are pushed onto queue by a producer thread. The frames are popped off the queue by a consumer thread. Mutual exclusion of producer and consumer is done using a mutex. However, I still notice frames being dropped. The dropped frames (likely) correspond to instances when producer tries to push the current frame onto queue but cannot do so since consumer holds the lock. Any suggestions ? I essentially do not want the producer to wait. A waiting consumer is okay for me.
EDIT-0 : Alternate idea which does not involve locking. Will this work ?
Producer initially enqueues n seconds worth of video. n can be some small multiple of frame-rate.
As long as queue contains >= n seconds worth of video, consumer dequeues on a frame by frame basis and saves to disk.
When the video is done, the queue is flushed to disk.
EDIT-1: The frames arrive at ~ 15 fps.
EDIT-2 : Outline of code :
Main driver code
// Main function
void LVD::DumpFrame(const IplImage *frame)
{
// Copies frame into internal buffer.
// buffer object is a wrapper around OpenCV's IplImage
Initialize(frame);
// (Producer thread) -- Pushes buffer onto queue
// Thread locks queue, pushes buffer onto queue, unlocks queue and dies
PushBufferOntoQueue();
// (Consumer thread) -- Pop off queue and save to disk
// Thread locks queue, pops it, unlocks queue,
// saves popped buffer to disk and dies
DumpQueue();
++m_frame_id;
}
void LVD::Initialize(const IplImage *frame)
{
if(NULL == m_buffer) // first iteration
m_buffer = new ImageBuffer(frame);
else
m_buffer->Copy(frame);
}
Producer
void LVD::PushBufferOntoQueue()
{
m_queingThread = ::CreateThread( NULL, 0, ThreadFuncPushImageBufferOntoQueue, this, 0, &m_dwThreadID);
}
DWORD WINAPI LVD::ThreadFuncPushImageBufferOntoQueue(void *arg)
{
LVD* videoDumper = reinterpret_cast<LVD*>(arg);
LocalLock ll( &videoDumper->m_que_lock, 60*1000 );
videoDumper->m_frameQue.push(*(videoDumper->m_buffer));
ll.Unlock();
return 0;
}
Consumer
void LVD::DumpQueue()
{
m_dumpingThread = ::CreateThread( NULL, 0, ThreadFuncDumpFrames, this, 0, &m_dwThreadID);
}
DWORD WINAPI LVD::ThreadFuncDumpFrames(void *arg)
{
LVD* videoDumper = reinterpret_cast<LVD*>(arg);
LocalLock ll( &videoDumper->m_que_lock, 60*1000 );
if(videoDumper->m_frameQue.size() > 0 )
{
videoDumper->m_save_frame=videoDumper->m_frameQue.front();
videoDumper->m_frameQue.pop();
}
ll.Unlock();
stringstream ss;
ss << videoDumper->m_saveDir.c_str() << "\\";
ss << videoDumper->m_startTime.c_str() << "\\";
ss << setfill('0') << setw(6) << videoDumper->m_frame_id;
ss << ".png";
videoDumper->m_save_frame.SaveImage(ss.str().c_str());
return 0;
}
Note:
(1) I cannot use C++11. Therefore, Herb Sutter's DDJ article is not an option.
(2) I found a reference to an unbounded single producer-consumer queue. However, the author(s) state that enqueue(adding frames) is probably not wait-free.
(3) I also found liblfds, a C-library but not sure if it will serve my purpose.
The queue cannot be the problem. Video frames arrive at 16 msec intervals, at worst. Your queue only needs to store a pointer to a frame. Adding/removing one in a thread-safe way can never take more than a microsecond.
You'll need to look for another explanation and solution. Video does forever present a fire-hose problem. Disk drives are not generally fast enough to keep up with an uncompressed video stream. So if your consumer cannot keep up with the producer then something is going go give. With a dropped frame the likely outcome when you (correctly) prevent the queue from growing without bound.
Be sure to consider encoding the video. Real-time MPEG and AVC encoders are available. After they compress the stream you should not have a problem keeping up with the disk.
Circular buffer is definitely a good alternative. If you make it use a 2^n size, you can also use this trick to update the pointers:
inline int update_index(int x)
{
return (x + 1) & (size-1);
}
That way, there is no need to use expensive compare (and consequential jumps) or divide (the single most expensive integer operation in any processor - not counting "fill/copy large chunks of memory" type operations).
When dealing with video (or graphics in general) it is essential to do "buffer management". Typically, this is a case of tracking state of the "framebuffer" and avoiding to copy content more than necessary.
The typical approach is to allocate 2 or 3 video-buffers (or frame buffers, or what you call it). A buffer can be owned by either the producer or the consumer. The transfer is ONLY the ownership. So when the video-driver signals that "this buffer is full", the ownership is now with the consumer, that will read the buffer and store it to disk [or whatever]. When the storing is finished, the buffer is given back ("freed") so that the producer can re-use it. Copying the data out of the buffer is expensive [takes time], so you don't want to do that unless it's ABSOLUTELY necessary.
I have done Nodejs: How to write high performance async loop for stormjs, you can check stormjs serial loop demo
but there is still have problem that parallel loop, e.g. we have a function requestContext(index, callback(err, context)) which remote get context 'http://host/post/{index}', and we need get the context of [0-99] and push context into an array in the order, [context0...context99]
but obviously this output cant work stormjs parallel loop
I still want to know how noders do this task, but you must make these requests parallel, not 1 by 1, it should be parallel request and serial push.
var counter = 0;
// create an array with numbers 0 to 99.
_.range(0,100).forEach(function(key, value, arr) {
// for each of them request context
requestContext(key, function(err, context) {
// add the context to the array under the correct key
if (!err) {
arr[key] = context;
}
// increment counter, if all have finished then fire finished.
if (++counter === 100) {
finished(arr);
}
});
});
function finished(results) {
// do stuff
}
No storm required. If you want an execution / flow control library I would recommend Futures because it doesn't compile your code and "hides the magic".
Previously you recursed through each one and executed them in a serial order pushing them into the array in order.
This time you execute them all in parallel and tell each one to assign the right ordered key in the array to their own value.
_.range Creates an array containing values 0 to 99.
i want to fill database table in wxListCtrl, i can do this , i m using wxThread for this. my problem is - my concept is working for small amount of data, when i increase the size, it shows a error like-
showingdatainwxlistctrl: ../../src/XlibInt.c:595: _XPrivSyncFunction: Assertion `dpy->synchandler == _XPrivSyncFunction' failed.
my code is given below:-
void *MyThread :: Entry()
{
int i=1,j,k=0 ;
while(i!=400)
{
long index=this->temp->data_list_control->InsertItem(i,wxT("amit"));
for(j=1;j<3;j++)
{
this->temp->data_list_control->SetItem(index,j,wxT("pathak"));
}
k++;
if(k==30)
{
this->Sleep(1000);
k=0;
}
i++;
}
}
if i used i =4, 10 100, it is working but i crossed the limit( i dont know at which point) it start showing error
if you have any suggestion then pls help me...
Instead of direct SetItem call from the worker thread, you need to post event to the main thread and add item in the event handler. Data for list control event should be placed to custom event class. See details in wxPostEvent function and here: http://wiki.wxwidgets.org/Custom_Events
You're accessing a non-threadsafe wxListCtrl from another thread, this will simply not work.
A better solution may be to skip the thread, use a wxTimer, then fill 400 more entries every time OnTimer is called.