Memory not freed after gstreamer pipeline set to GST_STATE_NULL - c++

My application requires gstreamer pipeline to be restarted multiple times.
But after setting the pipeline to GST_STATE_NULL and calling unref on the pipeline, memory appears to be not freed.
After every restart, the memory associated with the process keeps increasing.
I was able to reproduce the problem with just videotestsrc-fakesink elements as given below :
//g++ -Wall testpage_Simple.cpp -o testpage_Simple $(pkg-config --cflags --libs gstreamer-1.0)
#include <gst/gst.h>
GstElement *pipeline;
GstElement *src;
GstElement *sink;
void clearPipeline () {
// g_print ("clearPipeline ");
gst_element_set_state (pipeline, GST_STATE_NULL);
gst_object_unref (pipeline);
}
void createPipeline () {
pipeline = gst_pipeline_new ("pipelinePlay");
src = gst_element_factory_make ("videotestsrc", "source");
sink = gst_element_factory_make ("fakesink", "sink");
gst_bin_add_many (GST_BIN (pipeline), src, sink, NULL);
if (gst_element_link (src, sink)!= TRUE) {
g_printerr ("src, sink could not be linked.\n");
}
gst_element_set_state (pipeline, GST_STATE_PLAYING);
}
gint main (gint argc, gchar * argv[])
{
gst_init (NULL, NULL);
system("gst-launch-1.0 --gst-version");
g_print ("Start Test - ");
// for (int i=1; i<=10; i++) {
system ("top -b -n 1 | grep testpage | awk '{print $6}'");
createPipeline();
clearPipeline();
// }
g_print ("End of test !! ");
system ("top -b -n 1 | grep testpage | awk '{print $6}'");
gst_deinit();
return 0;
}
Sample output on Ubuntu 19.04 (showing only RES column value from top command for this process):
GStreamer Core Library version 1.16.1
Start Test - 7140
End of test !! 8504
We observed similar memory trend on Ubuntu 18.04 with gstreamer 1.12.4 as well.
Is this the expected behavior or did I miss something in cleaning up the pipeline ??
I checked the below link. It appears to be a similar problem, but this question is unanswered
GStreamer memory leak after the pipeline restart
I tried the disk cache suggestion in the below link, with a similar problem. But that did not help as well.
http://gstreamer-devel.966125.n4.nabble.com/Properly-freeing-resources-td4658631.html

Just a guess, maybe the problem is related to possible state transitions for elements
according
https://gstreamer.freedesktop.org/documentation/additional/design/states.html?gi-language=c#
the following state changes are possible:
NULL -> READY:
The element must check if the resources it needs are available. Device sinks and sources typically try to probe the device to constrain their caps.
The element opens the device, this is needed if the previous step requires the device to be opened.
READY -> PAUSED:
The element pads are activated in order to receive data in PAUSED. Streaming threads are started.
Some elements might need to return ASYNC and complete the state change when they have enough information. It is a requirement for sinks to return ASYNC and complete the state change when they receive the first buffer or EOS event (preroll). Sinks also block the dataflow when in PAUSED.
A pipeline resets the running_time to 0.
Live sources return NO_PREROLL and don't generate data.
PAUSED -> PLAYING:
Most elements ignore this state change.
The pipeline selects a clock and distributes this to all the children before setting them to PLAYING. This means that it is only allowed to synchronize on the clock in the PLAYING state.
The pipeline uses the clock and the running_time to calculate the base_time. This base_time is distributed to all children when performing the state change.
Sink elements stop blocking on the preroll buffer or event and start rendering the data.
Sinks can post the EOS message in the PLAYING state. It is not allowed to post EOS when not in the PLAYING state.
While streaming in PAUSED or PLAYING elements can create and remove sometimes pads.
Live sources start generating data and return SUCCESS.
PLAYING -> PAUSED:
Most elements ignore this state change.
The pipeline calculates the running_time based on the last selected clock and the base_time. It stores this information to continue playback when going back to the PLAYING state.
Sinks unblock any clock wait calls.
When a sink does not have a pending buffer to play, it returns ASYNC from this state change and completes the state change when it receives a new buffer or an EOS event.
Any queued EOS messages are removed since they will be reposted when going back to the PLAYING state. The EOS messages are queued in GstBins.
Live sources stop generating data and return NO_PREROLL.
PAUSED -> READY:
Sinks unblock any waits in the preroll.
Elements unblock any waits on devices
Chain or get_range() functions return FLUSHING.
The element pads are deactivated so that streaming becomes impossible and all streaming threads are stopped.
The sink forgets all negotiated formats
Elements remove all sometimes pads
READY -> NULL:
Elements close devices
Elements reset any internal state.
if the current state is playing then st. like
gst_element_set_state (pipeline, GST_STATE_PAUSED);
gst_element_set_state (pipeline, GST_STATE_READY);
gst_element_set_state (pipeline, GST_STATE_NULL);
might help
Checking the return value of gst_element_set_state(...) might not be a bad idea either :)

Related

GStreamer c++ - seeking before starting any playback

I added an start-offset option to my C++ application using the GStreamer library. It basically tells the application to start mid-video instead of playing from the beginning.
The pipeline is created like this: filesrc location="file_path" ! qtdemux ! h264parse ! mppvideodec ! waylandsink. This is what gives me best results on my hardware, and I'm running under Weston (wayland)
I do the seeking with a simple gst_element_seek on the pipeline. First I tried waiting until the pipeline changes state to PLAYING or PAUSED in the bus, and then seeking once. This worked, but on some larger videos it produced an effect that after launching the app, the first video frame is displayed for a glimpse of a second and then the video actually jumps to the desired position. I don't want that, I'd like to jump right to where I want without displaying anything else earlier.
So I changed my seeking method to happen before I even set the pipeline to PLAYING state like this:
// seek if needed
if(startTime > 0) {
// First set pipeline to PAUSED and wait for async state change
gst_element_set_state(m_pipeline, GST_STATE_PAUSED);
gst_element_get_state(m_pipeline, nullptr, nullptr, GST_CLOCK_TIME_NONE);
if (!gst_element_seek (m_pipeline, 1.0, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH, GST_SEEK_TYPE_SET, startTime, GST_SEEK_TYPE_NONE, -1)) {
g_print ("Seek failed!\n");
}
gst_element_get_state(m_pipeline, nullptr, nullptr, GST_CLOCK_TIME_NONE); // wait for seek to finish and then set to PLAYING
}
m_ret = gst_element_set_state(m_pipeline, GST_STATE_PLAYING);
if (m_ret == GST_STATE_CHANGE_FAILURE) {
g_printerr("Unable to set the pipeline to the playing state.\n");
gst_object_unref(m_pipeline);
exit(0);
}
And while this also works, it unfortunately produces the exact same effect as before...the first frame is still being displayed even though the seek happens before the pipeline is started.
How can I ensure that nothing is displayed until the seek is actually done?
Update
I couldn't get any method of seeking to work like I wanted, but I achieved the effect by setting the show-preroll-frame on the GstVideoSink before constructing the pipeline, like that:
m_sink = gst_element_factory_make("waylandsink", "sink");
g_object_set(m_sink, "show-preroll-frame", FALSE, NULL); // prevent preroll frame when seeking
And it works! I'll use it for now if there's no better way. The seek is still done in PAUSED state, but video is displayed directly from the seeking point.
As the documentation of the gst_element_seek_simple method says:
Some elements allow for seeking in the READY state...
This is true if the pipeline is completely prerolled before the seek.
Then you can get the result you expect, showing only the frames you expect.
gst_element_set_state(m_pipeline, GST_STATE_READY);
//make sure all the pipeline is prerolled (dynamic pads linked and other stuff)
if (!gst_element_seek (m_pipeline, 1.0, GST_FORMAT_TIME, GST_SEEK_FLAG_FLUSH, GST_SEEK_TYPE_SET, startTime, GST_SEEK_TYPE_NONE, -1)) {
g_print ("Seek failed!\n");
}
gst_element_get_state(m_pipeline, nullptr, nullptr, GST_CLOCK_TIME_NONE);

Unable to play ts file, when pipeline bin has more number of sinks

I would like to play a ts file, which has one or two videos.
I created a pipeline, and dynamically linked the elements with demux, whenever a pad arises in callback of demux function.
As shown below.
if(g_str_has_prefix(pad_name, "video"))
{
UU_PRINT("Player :: In dynamic ADDING DL PAD %s DLLinkFlag %d, LinkFlag %d", pad_name, pObjPlayer->mDlLinkFlag, linkFlag);
if(!linkFlag)
{
GstPad *dlsink = gst_element_get_static_pad(pObjPlayer->mpDlQueue, "sink");
}
else
{
GstPad *dlsink = gst_element_get_static_pad(pObjPlayer->mpIrQueue, "sink");
}
if(GST_IS_PAD(DeMuxPad) && GST_IS_PAD(dlsink))
{
if(gst_pad_link(DeMuxPad, dlsink) != GST_PAD_LINK_OK)
{
UU_PRINT( "Player :: Failed to Link Demux with DL Video Queue !!");
}
else
{
linkFlag = 1;
}
}
Initially, i added two videosinks to Pipeline BIN, if the ts file has two videos then it is working fine.
But if the ts file has only one video, then in ximagesink window gets paused.
In order to play that ts file, i have to remove another sink element, which is not receiving data should be removed from bin.
And in that case i am not able to receive the END OF THE STREAM MESSAGE.
Why is is happening? Is there any other way to play those two ts files, by using same pipeline, without removing the sink element.??
All the sinks in a pipeline must receive data in order for the pipeline to preroll and then move to the playing state. If one sink isn't receiving data all the others will wait for it to receive.
Details on prerolling: https://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-preroll.txt
Basically if one sink would just start rendering before other receive data it would mean that the data that reaches the 'late' sink would already be discarded because it would be too late to play it in sync with the data on the first sink (it has already been played).
In short, you should only add sinks to the pipeline if you intend to use them.

How to restart a pipeline when it is in playing state

I have a pipeline, which is in playing state,
GST_STATE_PLAYING
Now, after a while, when some certain condition met, i would like to restart the same pipeline.
How can i achieve this?
It depends on the situation as stated here
You can put it into NULL and then PLAYING:
gst_element_set_state(pipeline, GST_STATE_NULL);
//do your stuff for example, change some elements, remove some elements etc:
gst_element_set_state(pipeline, GST_STATE_PLAYING);
Or you can just put it into READY(replace NULL with READY in above code) and set some parameters which should work I guess..
Regarding sync/async etc of the gst_element_set_state check docs here

How to flush Gstreamer pipeline queue and elements?

I want to unkink this two queue from pipeline and exchange some element of this queue.
Like that :
gst_element_set_state (pipeline, GST_STATE_PAUSED);
gst_element_set_state (sink, GST_STATE_NULL);
gst_element_unlink_many(screen_queue2, **filter2**, **videobox2**, videoBox2_queue, (char* )0);
gst_element_unlink_many(screen_queue4, **filter4**, **videobox4**, videoBox4_queue, (char* )0);
gst_element_link_many(screen_queue2, **filter2**, **videobox4**, videoBox4_queue, (char* )0);
gst_element_link_many(screen_queue4, **filter4**, **videobox2**, videoBox2_queue, (char* )0);
but my code crashes, kindly help me out to resolved this crash issue. It works properly when pipeline is in NULL state, but I want to handle this when pipeline is in only PAUSED state.

Gstreamer-Interleave 2 files: how to capture End-Of-Stream on one stream from the pipeline?

I am interleaving 2 mono files to playout a stereo output file. one file is shorter than the other and I want to capture the EOS when the shorter file reaches end-of-streamer (EOS).
Unfortunately, when I used a gst_bus_add_watch() on the pipeline (see following code), my_callback() get called with EOS message ONLY when the resulted interleaved stream reaches end-of-file. In my case, that happens only when the longest file reaches EOS:
*loop = g_main_loop_new (NULL, FALSE);
pipeline = gst_pipeline_new ("pipeline");
bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline));
gst_bus_add_watch(bus, my_callback, loop);
Is there a way to capture the EOS on one of the 2 streams (the shorted stream for example) (from my_callback()) when this stream reaches EOS?
(gboolean my_bus_callback (GstBus *bus, GstMessage message,gpointer data))
Thank you
It might be easier to implement a mode-flag on the interleave element.
What you can do now is to use a pad-probe on the sink pads of the interleave element and listen for eos-events there. You should be able to post them from the to the bus, but I haven't tried that.