I am having Gstreamer element which has request pad as sink pads. From application, I am requesting pad and linking as follows:
GstPadTemplate *sink_pad_template = gst_element_class_get_pad_template (GST_ELEMENT_GET_CLASS (mElement), "sink_%d");
GstPad *pad = gst_element_request_pad (mElement, sink_pad_template, NULL, NULL);
gst_pad_link(gst_element_get_static_pad (mQueue, "src"), pad);
Successfully pads are linked, but while releasing request pads gst_element_release_request_pad not returning.
Tried block the previous element's src pad (in pipeline), unlink and then release_request_pad but gst_element_release_request_pad not returning. Please let me know what are the right steps while releasing the request pads?
gst_pad_set_blocked(m_queueSrcPad, true);
gst_pad_unlink(m_queueSrcPad, pad);
gst_element_release_request_pad(m_Element, pad);
Tried setting source pad as blocked before calling above functions, still this did not help
gst_pad_set_blocked(SrcPad, true); // first element in pipeline
Related
I'm building a pipeline in Gstreamer, using a UriDecodeBin. This pipeline is meant to be build and released repeatedly, but I have a hard time to figure out how to properly delete the pipeline.
For now, I'm trying to figure out what to unref and what not to unref to get all resources of a pipeline to be released, with just a UriDecodeBin in the pipeline, but I can't get the full memory to be released somehow.
My sequence of actions is as follows :
Fist, I'm building the pipeline and UriDecodeBin :
_pipeline = (GstPipeline*)gst_pipeline_new("pipeline");
_bus = gst_pipeline_get_bus(_pipeline); //will be manually observed
_decoder = gst_element_factory_make("uridecodebin", "decoder");
gst_bin_add(GST_BIN(_pipeline), _decoder);
gst_element_sync_state_with_parent(_decoder);
std::string uri = std::string("file://some_video_file");
g_object_set(_decoder, "uri", uri.c_str(), nullptr);
g_signal_connect(_decoder, "no-more-pads", G_CALLBACK(GsDecoder::no_more_pads_cb), (gpointer)this);
changePipelineState(GST_STATE_PAUSED);
Then, I wait for the no-more-pads to trigger, and add a blocking probe on each pad :
GstIterator *it = gst_element_iterate_src_pads(_decoder);
GValue padV = G_VALUE_INIT;
while(gst_iterator_next(it, &padV) == GST_ITERATOR_OK){
GstPad* pad = (GstPad*)g_value_get_object(&padV);
GstCaps* caps = gst_pad_get_current_caps(pad);
if(caps_is_audio(caps)){
_decoderAudioSrcs.push_back(pad); //adding pad to a collection
gulong id = gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM, GsDecoder::pad_blocked_cb, (gpointer)this, NULL); //blocking probe
_decoderAudioProbeIds.push_back(id); //remembering id of the probe, for later
} else if (caps_is_video(caps)){
//same with video but file is guaranteed to have a single video track.
_decoderVideoSrc = pad;
_decoderVideoProbeId = gst_pad_add_probe(pad, GST_PAD_PROBE_TYPE_BLOCK_DOWNSTREAM, GsDecoder::pad_blocked_cb, (gpointer)this, NULL);
} else {
std::cout << "UNKNOWN PAD FOUND" << std::endl;
}
gst_caps_unref(caps); //releasing caps
g_value_unset(&padV);
}
At this point I wait that all probes block the pads, and then I unblock the pads :
for (unsigned i = 0; i < _decoderAudioSrcs.size(); i++){
gst_pad_remove_probe(_decoderAudioSrcs[i], _decoderAudioProbeIds[i]);
}
gst_pad_remove_probe(_decoderVideoSrc, _decoderVideoProbeId);
play(); //setting the pipeline to play
Finally, after a bit of time, I order the pipeline to delete itself, and there I am a bit confused about what shall be deleted or not (both in this pipeline and in general).
How shall the pipeline be deleted ? From what I understood, I have to set pipeline state to Null, and then unref everything (pads, decoder, bus, pipeline) using g_object_unref, however it still leads to pipeline having few refs pointing to it (in my case, 3), and not releasing its resources, would it be memory nor threads.
Is there a way to tell Gstreamer to fully delete a pipeline and everthing related to it, without having to unref everything by hand ?
Especially, I would be interested in a pointer to some documentation, since I couldn't found any on the subject (even if Gstreamer doc mentions unrefing time to time).
GStreamer objects are reference counted. Whenever the reference drops to 0 the object gets deleted. The GStreamer documentation usually tells you for a function about its transfer behavior. E.g.
gst_pad_get_current_caps()
https://gstreamer.freedesktop.org/documentation/gstreamer/gstpad.html#gst_pad_get_current_caps
Tells you the transfer is full and you are supposed to release it when you make no more use it (you do so in your sample code). If you don't do it, it will never be deleted.
So the typical use case is if-i-don't-need-it-anymore-i-release-it. That does not mean that it gets deleted. For example, if you put an element into a pipeline, the pipeline will add a reference to it. It will take care of the reference and release it once the pipeline's reference count drops to 0. But since you added it to the pipeline and your code itself does not need it anymore you would usually just release it.
So you need to the careful about the references. If you miss one tiny small object which may be part of a big pipeline it may result in many objects to not be released as they depend on each other.
My application requires gstreamer pipeline to be restarted multiple times.
But after setting the pipeline to GST_STATE_NULL and calling unref on the pipeline, memory appears to be not freed.
After every restart, the memory associated with the process keeps increasing.
I was able to reproduce the problem with just videotestsrc-fakesink elements as given below :
//g++ -Wall testpage_Simple.cpp -o testpage_Simple $(pkg-config --cflags --libs gstreamer-1.0)
#include <gst/gst.h>
GstElement *pipeline;
GstElement *src;
GstElement *sink;
void clearPipeline () {
// g_print ("clearPipeline ");
gst_element_set_state (pipeline, GST_STATE_NULL);
gst_object_unref (pipeline);
}
void createPipeline () {
pipeline = gst_pipeline_new ("pipelinePlay");
src = gst_element_factory_make ("videotestsrc", "source");
sink = gst_element_factory_make ("fakesink", "sink");
gst_bin_add_many (GST_BIN (pipeline), src, sink, NULL);
if (gst_element_link (src, sink)!= TRUE) {
g_printerr ("src, sink could not be linked.\n");
}
gst_element_set_state (pipeline, GST_STATE_PLAYING);
}
gint main (gint argc, gchar * argv[])
{
gst_init (NULL, NULL);
system("gst-launch-1.0 --gst-version");
g_print ("Start Test - ");
// for (int i=1; i<=10; i++) {
system ("top -b -n 1 | grep testpage | awk '{print $6}'");
createPipeline();
clearPipeline();
// }
g_print ("End of test !! ");
system ("top -b -n 1 | grep testpage | awk '{print $6}'");
gst_deinit();
return 0;
}
Sample output on Ubuntu 19.04 (showing only RES column value from top command for this process):
GStreamer Core Library version 1.16.1
Start Test - 7140
End of test !! 8504
We observed similar memory trend on Ubuntu 18.04 with gstreamer 1.12.4 as well.
Is this the expected behavior or did I miss something in cleaning up the pipeline ??
I checked the below link. It appears to be a similar problem, but this question is unanswered
GStreamer memory leak after the pipeline restart
I tried the disk cache suggestion in the below link, with a similar problem. But that did not help as well.
http://gstreamer-devel.966125.n4.nabble.com/Properly-freeing-resources-td4658631.html
Just a guess, maybe the problem is related to possible state transitions for elements
according
https://gstreamer.freedesktop.org/documentation/additional/design/states.html?gi-language=c#
the following state changes are possible:
NULL -> READY:
The element must check if the resources it needs are available. Device sinks and sources typically try to probe the device to constrain their caps.
The element opens the device, this is needed if the previous step requires the device to be opened.
READY -> PAUSED:
The element pads are activated in order to receive data in PAUSED. Streaming threads are started.
Some elements might need to return ASYNC and complete the state change when they have enough information. It is a requirement for sinks to return ASYNC and complete the state change when they receive the first buffer or EOS event (preroll). Sinks also block the dataflow when in PAUSED.
A pipeline resets the running_time to 0.
Live sources return NO_PREROLL and don't generate data.
PAUSED -> PLAYING:
Most elements ignore this state change.
The pipeline selects a clock and distributes this to all the children before setting them to PLAYING. This means that it is only allowed to synchronize on the clock in the PLAYING state.
The pipeline uses the clock and the running_time to calculate the base_time. This base_time is distributed to all children when performing the state change.
Sink elements stop blocking on the preroll buffer or event and start rendering the data.
Sinks can post the EOS message in the PLAYING state. It is not allowed to post EOS when not in the PLAYING state.
While streaming in PAUSED or PLAYING elements can create and remove sometimes pads.
Live sources start generating data and return SUCCESS.
PLAYING -> PAUSED:
Most elements ignore this state change.
The pipeline calculates the running_time based on the last selected clock and the base_time. It stores this information to continue playback when going back to the PLAYING state.
Sinks unblock any clock wait calls.
When a sink does not have a pending buffer to play, it returns ASYNC from this state change and completes the state change when it receives a new buffer or an EOS event.
Any queued EOS messages are removed since they will be reposted when going back to the PLAYING state. The EOS messages are queued in GstBins.
Live sources stop generating data and return NO_PREROLL.
PAUSED -> READY:
Sinks unblock any waits in the preroll.
Elements unblock any waits on devices
Chain or get_range() functions return FLUSHING.
The element pads are deactivated so that streaming becomes impossible and all streaming threads are stopped.
The sink forgets all negotiated formats
Elements remove all sometimes pads
READY -> NULL:
Elements close devices
Elements reset any internal state.
if the current state is playing then st. like
gst_element_set_state (pipeline, GST_STATE_PAUSED);
gst_element_set_state (pipeline, GST_STATE_READY);
gst_element_set_state (pipeline, GST_STATE_NULL);
might help
Checking the return value of gst_element_set_state(...) might not be a bad idea either :)
I added an start-offset option to my C++ application using the GStreamer library. It basically tells the application to start mid-video instead of playing from the beginning.
The pipeline is created like this: filesrc location="file_path" ! qtdemux ! h264parse ! mppvideodec ! waylandsink. This is what gives me best results on my hardware, and I'm running under Weston (wayland)
I do the seeking with a simple gst_element_seek on the pipeline. First I tried waiting until the pipeline changes state to PLAYING or PAUSED in the bus, and then seeking once. This worked, but on some larger videos it produced an effect that after launching the app, the first video frame is displayed for a glimpse of a second and then the video actually jumps to the desired position. I don't want that, I'd like to jump right to where I want without displaying anything else earlier.
So I changed my seeking method to happen before I even set the pipeline to PLAYING state like this:
// seek if needed
if(startTime > 0) {
// First set pipeline to PAUSED and wait for async state change
gst_element_set_state(m_pipeline, GST_STATE_PAUSED);
gst_element_get_state(m_pipeline, nullptr, nullptr, GST_CLOCK_TIME_NONE);
if (!gst_element_seek (m_pipeline, 1.0, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH, GST_SEEK_TYPE_SET, startTime, GST_SEEK_TYPE_NONE, -1)) {
g_print ("Seek failed!\n");
}
gst_element_get_state(m_pipeline, nullptr, nullptr, GST_CLOCK_TIME_NONE); // wait for seek to finish and then set to PLAYING
}
m_ret = gst_element_set_state(m_pipeline, GST_STATE_PLAYING);
if (m_ret == GST_STATE_CHANGE_FAILURE) {
g_printerr("Unable to set the pipeline to the playing state.\n");
gst_object_unref(m_pipeline);
exit(0);
}
And while this also works, it unfortunately produces the exact same effect as before...the first frame is still being displayed even though the seek happens before the pipeline is started.
How can I ensure that nothing is displayed until the seek is actually done?
Update
I couldn't get any method of seeking to work like I wanted, but I achieved the effect by setting the show-preroll-frame on the GstVideoSink before constructing the pipeline, like that:
m_sink = gst_element_factory_make("waylandsink", "sink");
g_object_set(m_sink, "show-preroll-frame", FALSE, NULL); // prevent preroll frame when seeking
And it works! I'll use it for now if there's no better way. The seek is still done in PAUSED state, but video is displayed directly from the seeking point.
As the documentation of the gst_element_seek_simple method says:
Some elements allow for seeking in the READY state...
This is true if the pipeline is completely prerolled before the seek.
Then you can get the result you expect, showing only the frames you expect.
gst_element_set_state(m_pipeline, GST_STATE_READY);
//make sure all the pipeline is prerolled (dynamic pads linked and other stuff)
if (!gst_element_seek (m_pipeline, 1.0, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH, GST_SEEK_TYPE_SET, startTime, GST_SEEK_TYPE_NONE, -1)) {
g_print ("Seek failed!\n");
}
gst_element_get_state(m_pipeline, nullptr, nullptr, GST_CLOCK_TIME_NONE);
I would like to play a ts file, which has one or two videos.
I created a pipeline, and dynamically linked the elements with demux, whenever a pad arises in callback of demux function.
As shown below.
if(g_str_has_prefix(pad_name, "video"))
{
UU_PRINT("Player :: In dynamic ADDING DL PAD %s DLLinkFlag %d, LinkFlag %d", pad_name, pObjPlayer->mDlLinkFlag, linkFlag);
if(!linkFlag)
{
GstPad *dlsink = gst_element_get_static_pad(pObjPlayer->mpDlQueue, "sink");
}
else
{
GstPad *dlsink = gst_element_get_static_pad(pObjPlayer->mpIrQueue, "sink");
}
if(GST_IS_PAD(DeMuxPad) && GST_IS_PAD(dlsink))
{
if(gst_pad_link(DeMuxPad, dlsink) != GST_PAD_LINK_OK)
{
UU_PRINT( "Player :: Failed to Link Demux with DL Video Queue !!");
}
else
{
linkFlag = 1;
}
}
Initially, i added two videosinks to Pipeline BIN, if the ts file has two videos then it is working fine.
But if the ts file has only one video, then in ximagesink window gets paused.
In order to play that ts file, i have to remove another sink element, which is not receiving data should be removed from bin.
And in that case i am not able to receive the END OF THE STREAM MESSAGE.
Why is is happening? Is there any other way to play those two ts files, by using same pipeline, without removing the sink element.??
All the sinks in a pipeline must receive data in order for the pipeline to preroll and then move to the playing state. If one sink isn't receiving data all the others will wait for it to receive.
Details on prerolling: https://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-preroll.txt
Basically if one sink would just start rendering before other receive data it would mean that the data that reaches the 'late' sink would already be discarded because it would be too late to play it in sync with the data on the first sink (it has already been played).
In short, you should only add sinks to the pipeline if you intend to use them.
I am interleaving 2 mono files to playout a stereo output file. one file is shorter than the other and I want to capture the EOS when the shorter file reaches end-of-streamer (EOS).
Unfortunately, when I used a gst_bus_add_watch() on the pipeline (see following code), my_callback() get called with EOS message ONLY when the resulted interleaved stream reaches end-of-file. In my case, that happens only when the longest file reaches EOS:
*loop = g_main_loop_new (NULL, FALSE);
pipeline = gst_pipeline_new ("pipeline");
bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline));
gst_bus_add_watch(bus, my_callback, loop);
Is there a way to capture the EOS on one of the 2 streams (the shorted stream for example) (from my_callback()) when this stream reaches EOS?
(gboolean my_bus_callback (GstBus *bus, GstMessage message,gpointer data))
Thank you
It might be easier to implement a mode-flag on the interleave element.
What you can do now is to use a pad-probe on the sink pads of the interleave element and listen for eos-events there. You should be able to post them from the to the bus, but I haven't tried that.