What is the correct way of using splitmuxsink in a dynamic pipeline?
Previously I have used filesink to record (no problem what so ever) but there is requirement to save the file in segments so I have tried to use splitmuxsink in dynamic pipeline(there is async time in recording). In doing so I have faced two problems
when I tried to stop the recording, I use a idle pad to block the recording queue and launch a callback function to do steps to delink the recording branch (send eos, set elements in recording bin to NULL, then dequeue the bin). I have set a downstream data probe to notify me that the eos has reached the splitmuxsink sink before I tried to do step 2..(set elemets to null)
However, the end result is that i still have an empty last file (o bytes). It seem that the pipe is not yet closed or having some problem. I did a workaround to split the video immediately when the record stop (though I lost a few frames)
How should one stop in a dynamic branch?
When I tried to create the recording bin when i start the recording(utilizing the pad-added signal when a pad is created to connect the recording bin). Previously I have created the recording bin in normal sequence (not creating them during the glib loop that I have created). The previous step work ok but the present step has the splitmuxsink's filesink in a locked state
How should I workaround this? What causes the lock state?
Here is my code
/// create record bin
static void
pad-added(GstElement * self,
GstPad * new_pad,
gpointer user_data)
{
char* pad_name = gst_pad_get_name(new_pad);
if(g_str_equal(pad_name,"src"))
{
//RECORD records;
records.recording = gst_bin_new("recording");
records.queue = gst_element_factory_make("queue","queue");
records.enc = gst_element_factory_make("vpuenc_h264","enc");
records.parser = gst_element_factory_make("h264parse","parser");
records.sink = gst_element_factory_make("splitmuxsink","sink");
// Add it to the main pipeline
gst_bin_add_many(GST_BIN(records.recording),
records.queue,
records.enc,
records.parser,
records.sink,NULL);
// link up the recording elements queue
gst_element_link_many(records.queue,
records.enc,
records.parser,
records.sink,NULL)
g_object_set(G_OBJECT(records.fsink),
//"location","video_%d.mp4",
"max-size-time", (guint64) 10L * GST_SECOND,
"async-handling", TRUE,
"async-finalize", TRUE,
NULL);
records.queue_sink_pad = gst_element_get_static_pad (records.queue, "sink");
records.ghost_pad = gst_ghost_pad_new ("sink", records.queue_sink_pad);
gst_pad_set_active(records.ghost_pad, TRUE);
gst_element_add_pad(GST_ELEMENT(records.recording),records.ghost_pad);
g_signal_connect (records.sink, "format-location",
(GCallback)format_location_callback,
&records);
}
}
gboolean cmd_loop()
{
// other cmd not shown here
if(RECORD)
{
//create tee sink pad
// this step will trigger the pad-added function
tee_sink_pad = gst_element_get_request_pad (tee,"src");
// ....other function
}
}
int main()
{
// add the pad-added signal response
g_signal_connect(tee, "pad-added", G_CALLBACK(pad-added), NULL);
// use to construct the loop (cycle every 1s)
GSource* source = g_timeout_source_new(1000);
// set function to watch for command
g_source_set_callback(source,
(GSourceFunc)cmd_loop,
NULL,
NULL);
}
Related
I`m a beginner at using Gstreamer to handle some input videos. I have already built the pipeline using GStreamer to transcode the videos but the last part I cannot do is How I can get those batches of frames and do some custom image processing techniques to handle the purpose of my task.
Input Videos -----> Gstreamer Pipeline -----> Task: Apply some Image Processing Techniques
I`ve been searching about this problem on the Internet but cannot find any solution and the more I search, the more I am confused.
AppSink is the good element for you. You can enable "emit-signal" property and listen the event "new-sample". Then you can get an access to the buffer.
Here the entire documentation :
https://gstreamer.freedesktop.org/documentation/tutorials/basic/short-cutting-the-pipeline.html?gi-language=c
You have to create appsink element, enable "emit-signals" then register "new-sample" callback like this :
g_signal_connect (data.app_sink, "new-sample", G_CALLBACK (new_sample), &data)
static GstFlowReturn new_sample (GstElement *sink, CustomData *data) {
GstSample *sample;
/* Retrieve the buffer */
g_signal_emit_by_name (sink, "pull-sample", &sample);
if (sample) {
/* The only thing we do in this example is print a * to indicate a received buffer */
g_print ("*");
gst_sample_unref (sample);
return GST_FLOW_OK;
}
return GST_FLOW_ERROR;
}
Now you can retrieve buffer from sample instead of g_print ... (gst_sample_get_buffer)
https://gstreamer.freedesktop.org/documentation/gstreamer/gstsample.html?gi-language=c
Then read data inside the buffer :
GstMapInfo info;
gst_buffer_map (buf, &info, GST_MAP_READ);
gst_buffer_unmap (buf, &info);
gst_buffer_unref (buf);
info.data ==> buffer content.
Best regards.
I have following very basic code using GStreamer library (GStreamer v1.8.1 on Xubuntu 16.04 if it important)
#include <gst/gst.h>
int main(int argc, char *argv[])
{
gst_init(&argc, &argv);
const gchar* pd =
"filesrc location=some.mp4 ! qtdemux name=d "
"d.video_0 ! fakesink "
"d.audio_0 ! fakesink ";
GError* error = nullptr;
GstElement *pipeline = gst_parse_launch(pd, &error);
GstState state; GstState pending;
switch(gst_element_set_state(pipeline, GST_STATE_PAUSED)) {
case GST_STATE_CHANGE_FAILURE:
case GST_STATE_CHANGE_NO_PREROLL:
return -1;
case GST_STATE_CHANGE_ASYNC: {
gst_element_get_state(pipeline, &state, &pending, GST_CLOCK_TIME_NONE);
}
case GST_STATE_CHANGE_SUCCESS:
break;
}
GMainLoop* loop = g_main_loop_new(nullptr, false);
g_main_loop_run(loop);
gst_object_unref(pipeline);
return 0;
}
The problem is when I try run this code it hangs on
gst_element_get_state(pipeline, &state, &pending, GST_CLOCK_TIME_NONE);
The question is - why it hangs? Especially if take into account, if I remove d.audio_0 ! fakesink from pipeline description it doesn't hang.
It is good practice to always add queues (or a multiqueue) after elements that produces multiple output branches in the pipeline e.g. demuxers.
The reason is that sinks will block waiting for other sinks to receive the first buffer (preroll). With a single thread, as your code, it will block the only thread available to push data into the sinks. A single thread is going from the demuxers to both sinks, once 1 blocks the there is no way for data to arrive on the second sink.
Using queues will spawn new threads and each sink will have a dedicated one.
That's quite an old thread but it probably hangs because you have an infinite timeout (GST_CLOCK_TIME_NONE).
Given a video file, I am trying to play the video from a given start time to a given end time.
I am trying to achieve this using Segment Event as follows:
Firstly, I am creating and initialising a segment.
Created a segment event using gst_event_new_segment()
Send the segment event to the pipeline using gst_element_send_event()
But, The video just plays from the beginning and not from the start time.
The segment event does not have an effect on the playback eventhough the event is reaching the sink element of the pipeline (verified by gst debug messages).
Following is the code-snippet:
GstSegment *gstSegment;
GstEvent *event_seg;
gstSegment = gst_segment_new();
gst_segment_init( gstSegment, GST_FORMAT_BYTES );
gstSegment->rate = 2.0;
gstSegment->start = 10 * 1920 * 1080 * 2;
gstSegment->stop = 20 * 1920 * 1080 * 2;
event_seg = gst_event_new_segment(gstSegment);
ret = gst_element_set_state (data.videoPipeline, GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE) {
printf("Unable to set the pipeline to the playing state\n");
gst_object_unref (data.videoPipeline);
return 1;
}
if ( !gst_element_send_event( GST_ELEMENT (data.videoPipeline), event_seg))
printf("Error");
I am able to achieve this using Seek Event.
But unable to do with Segment Event.
Can anyone please tell me how to achieve this using Segment Event?
I have written gstreamer code for camera live-streaming and recording at the same time.
My pipeline looks like that:
/ [ queue1 | videosink ]
v4l2src | tee
\ [ queue2 | filesink ]
Currently both live streaming and file recording is working together.
Now I need to start the pipeline with only one queue i.e. queue1 (live streaming queue), After a while I need to add the recording queue and remove it dynamically too.
My working code are given below :
pipeline = gst_pipeline_new ("rv_camera");
/*Create source element. We use mfw_v4lsrc from Freescale as source */
source= gst_element_factory_make (GSTREAMER_SOURCE,"camera-source");
g_object_set(G_OBJECT(source),"device",camDeviceName, (char *)0);
/*Set default properties of mfw_v4lsrc */
g_object_set(G_OBJECT(source),"capture-width", CAMERA_CAPTURE_WIDTH,
"capture-height", CAMERA_CAPTURE_HEIGHT,
"sensor-width", CAMERA_SENSOR_WIDTH,
"sensor-height", CAMERA_SENSOR_HEIGHT,
"preview", CAMERA_PREVIEW_DISPLAY,
"preview-width",CAMERA_PREVIEW_WIDTH,
"preview-height",CAMERA_PREVIEW_HEIGHT,
"fps-n",CAMERA_FRAMERATE,
"rotate",mirror_effect,
(char *)0);
/* Tee that copies the stream to multiple outputs */
tee = gst_element_factory_make("tee", "tee");
/* Queue creates new thread for the stream */
screen_queue = gst_element_factory_make("queue", "screen_queue");
/*Create sink element. We use mfw_v4lsink from Freescale as sink. fbdevsink is not used as
it directly writes into framebuffer which is not desired*/
sink= gst_element_factory_make (GSTREAMER_SINK,"video-output");
capture_queue = gst_element_factory_make("queue", "capture_queue");
encoder = gst_element_factory_make("mfw_vpuencoder", "encoder");
g_object_set(G_OBJECT(encoder),"codec-type",0,
"mirror-direction",0,
(char *)0);
clockoverlay = gst_element_factory_make("clockoverlay", "Timestamp");
g_object_set(G_OBJECT(clockoverlay),"time-format","%R %d-%b-%Y", (char *)0);
avimux = gst_element_factory_make("avimux", "avimux");
filesink = gst_element_factory_make("filesink", "file-output");
g_object_set(G_OBJECT(filesink),"location","/KPIT/OBITS/Blackbox/OBITS-SCNLog.avi", (char *)0);
/* Check if all elements are created or not*/
if (!pipeline || !source || !tee || !screen_queue || !sink || !capture_queue || !clockoverlay || !encoder || !avimux || !filesink) {
LOGERR((TEXT("GstreamerStream :: camInit: 1 One or more element(s) could not be created .... logerr\n")));
return CAM_STATUS_INIT_FAIL;
}
/* we add all elements into the pipeline */
gst_bin_add_many (GST_BIN (pipeline),source,tee,screen_queue, sink, capture_queue,clockoverlay,encoder,avimux,filesink, (char *)0);
/* we link the elements together */
if( gst_element_link_many( source, tee, NULL ) && gst_element_link_many( tee,screen_queue,sink, NULL ) &&
gst_element_link_many( tee,capture_queue,clockoverlay,encoder,avimux,filesink, NULL ))
{
bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline));
/*Add watch to look for error events */
gst_bus_add_watch(bus, process_events, this);
gst_object_unref(bus);
}
gst_element_set_state (pipeline, GST_STATE_PLAYING);
Kindly let me know the way, I can add or remove any queue dynamically.
I'd appreciate your help if someone can provide sample code related to this.
Keep the tee in the pipeline and you can request/release pads from the tee at any time during playback. Request the pad and add your new elements to the pipeline, link them, and set them to playing, too. When you are done, unlink this branch and remember to send EOS to it to have the recording properly finished. After you receive the EOS message from the filesink you can shutdown, remove and unref the branch you unlinked.
If you are using 0.10 (don't use it, move to 1.0), then you might need to send a segment event to the new branch once you add it.
I'm implementing gstreamer media player with my own source of data using appsrc. Everything works fine except one thing:
When stream reaches it's end, callback emits "end-of-stream" signal. Signals sending fucntion g_signal_emit_by_name(appsrc, "end-of-stream", &ret) returns GstFlowReturn value GST_FLOW_OK. But then it calls need-data my callback again, so it returns "end-of-stream" signal again. And this time GstFlowReturn value is (-3) which is GST_FLOW UNEXPECTED. I assume that it does not expect "end-of-stream" signal when it already recieved one, but why it requests more data than? Maybe it is because I didn't set size value iof the steam?
Gstreamer version is 0.10.
Callback function code (appsrc type is seekable btw):
static void cb_need_data (GstElement *appsrc, guint size, gpointer user_data)
{
GstBuffer *buffer;
GstFlowReturn ret;
AppsrcData* data = static_cast<AppsrcData*>(user_data);
buffer = gst_buffer_new_and_alloc(size);
int read = fread(GST_BUFFER_DATA(buffer), 1, size, data->file);
GST_BUFFER_SIZE(buffer) = read;
g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret);
if (ret != GST_FLOW_OK) {
/* something wrong, stop pushing */
g_printerr("GST_FLOW != OK, return value is %d\n", ret);
g_main_loop_quit (data->loop);
}
if(feof(data->file) || read == 0)
{
g_signal_emit_by_name(appsrc, "end-of-stream", &ret);
if (ret != GST_FLOW_OK) {
g_printerr("EOF reached, GST_FLOW != OK, return value is %d\nAborting...", ret);
g_main_loop_quit (data->loop);
}
}
}
You should provide some corrections to your code(if they are not there already) that should alleviate your issue and help the overall application:
Never try and send a buffer without first checking if it actually has data. So, simply check the buffer data and length to make sure that the data is not NULL and that the length is >0
You can flag that a stream is ended in your user_data. When you send your EOS, set an item in your userdata to indicate that it has been sent and if the appsrc requests more data, simply check if it has been sent and then do not send anything else to the buffer.
Listen for the EOS on your pipeline bus so that it can destroy the stream and close the loop when the EOS message is handled so that you can be sure that your mediasink has received the EOS and you can safely dispose of the pipeline and loop without losing any data.
Have you tried the method gst_app_src_end_of_stream()? I'm not sure what return code you should use after invoking it, but it should be either GST_FLOW_OK or GST_FLOW_UNEXPECTED.
In GStreamer 1.x you return GST_FLOW_EOS.