I'm using splitmuxsink element to save videos based on size, I can use format-location signal to set the next video name to be used for dumping video.
static gchararray
format_location_callback (GstElement * splitmux,
guint fragment_id,
gpointer udata)
{
static int i =0;
gchararray myarray = g_strdup_printf("myvid%d.mp4", i);
i += 1;
return myarray;
}
# add a callback signal
g_signal_connect (G_OBJECT (bin->sink), "format-location",
G_CALLBACK (format_location_callback), bin);
How do I get the current video name that's being dumped by the splitmuxsink? I think that might be possible using the GstMessages, but I'm not sure how to get the message related to particular plugin.
In fact, when I use the DEBUG_MODE=4, I can see the message that video name changed when the video split happens in splitmuxsink.
0:00:06.238114046 31488 0x55928d253d90 INFO splitmuxsink gstsplitmuxsink.c:2389:set_next_filename:<sink_sub_bin_sink1> Setting file to myvid0.mp4
0:00:06.238149341 31488 0x55928d253d90 INFO filesink gstfilesink.c:294:gst_file_sink_set_location:<sink> filename : myvid0.mp4
0:00:06.238160223 31488 0x55928d253d90 INFO filesink gstfilesink.c:295:gst_file_sink_set_location:<sink> uri
I'm able to get video location using GstMessage of type GST_MESSAGE_ELEMENT by listening on the GstBus. The splitmuxsink has message with name splitmuxsink-fragment-opened when video split takes place.
const gchar * location;
const GstStructure* s = gst_message_get_structure(message);
if (gst_structure_has_name(s, "splitmuxsink-fragment-opened"))
{
g_message ("get message: %s",
gst_structure_to_string (gst_message_get_structure(message)));
location = gst_structure_get_string(s, "location");
cout << location << endl;
}
break;
}
Output
** Message: 12:00:27.618: get message: splitmuxsink-fragment-opened, location=(string)myvid0.mp4, running-time=(guint64)1199530439;
myvid0.mp4
Related
I`m a beginner at using Gstreamer to handle some input videos. I have already built the pipeline using GStreamer to transcode the videos but the last part I cannot do is How I can get those batches of frames and do some custom image processing techniques to handle the purpose of my task.
Input Videos -----> Gstreamer Pipeline -----> Task: Apply some Image Processing Techniques
I`ve been searching about this problem on the Internet but cannot find any solution and the more I search, the more I am confused.
AppSink is the good element for you. You can enable "emit-signal" property and listen the event "new-sample". Then you can get an access to the buffer.
Here the entire documentation :
https://gstreamer.freedesktop.org/documentation/tutorials/basic/short-cutting-the-pipeline.html?gi-language=c
You have to create appsink element, enable "emit-signals" then register "new-sample" callback like this :
g_signal_connect (data.app_sink, "new-sample", G_CALLBACK (new_sample), &data)
static GstFlowReturn new_sample (GstElement *sink, CustomData *data) {
GstSample *sample;
/* Retrieve the buffer */
g_signal_emit_by_name (sink, "pull-sample", &sample);
if (sample) {
/* The only thing we do in this example is print a * to indicate a received buffer */
g_print ("*");
gst_sample_unref (sample);
return GST_FLOW_OK;
}
return GST_FLOW_ERROR;
}
Now you can retrieve buffer from sample instead of g_print ... (gst_sample_get_buffer)
https://gstreamer.freedesktop.org/documentation/gstreamer/gstsample.html?gi-language=c
Then read data inside the buffer :
GstMapInfo info;
gst_buffer_map (buf, &info, GST_MAP_READ);
gst_buffer_unmap (buf, &info);
gst_buffer_unref (buf);
info.data ==> buffer content.
Best regards.
What is the correct way of using splitmuxsink in a dynamic pipeline?
Previously I have used filesink to record (no problem what so ever) but there is requirement to save the file in segments so I have tried to use splitmuxsink in dynamic pipeline(there is async time in recording). In doing so I have faced two problems
when I tried to stop the recording, I use a idle pad to block the recording queue and launch a callback function to do steps to delink the recording branch (send eos, set elements in recording bin to NULL, then dequeue the bin). I have set a downstream data probe to notify me that the eos has reached the splitmuxsink sink before I tried to do step 2..(set elemets to null)
However, the end result is that i still have an empty last file (o bytes). It seem that the pipe is not yet closed or having some problem. I did a workaround to split the video immediately when the record stop (though I lost a few frames)
How should one stop in a dynamic branch?
When I tried to create the recording bin when i start the recording(utilizing the pad-added signal when a pad is created to connect the recording bin). Previously I have created the recording bin in normal sequence (not creating them during the glib loop that I have created). The previous step work ok but the present step has the splitmuxsink's filesink in a locked state
How should I workaround this? What causes the lock state?
Here is my code
/// create record bin
static void
pad-added(GstElement * self,
GstPad * new_pad,
gpointer user_data)
{
char* pad_name = gst_pad_get_name(new_pad);
if(g_str_equal(pad_name,"src"))
{
//RECORD records;
records.recording = gst_bin_new("recording");
records.queue = gst_element_factory_make("queue","queue");
records.enc = gst_element_factory_make("vpuenc_h264","enc");
records.parser = gst_element_factory_make("h264parse","parser");
records.sink = gst_element_factory_make("splitmuxsink","sink");
// Add it to the main pipeline
gst_bin_add_many(GST_BIN(records.recording),
records.queue,
records.enc,
records.parser,
records.sink,NULL);
// link up the recording elements queue
gst_element_link_many(records.queue,
records.enc,
records.parser,
records.sink,NULL)
g_object_set(G_OBJECT(records.fsink),
//"location","video_%d.mp4",
"max-size-time", (guint64) 10L * GST_SECOND,
"async-handling", TRUE,
"async-finalize", TRUE,
NULL);
records.queue_sink_pad = gst_element_get_static_pad (records.queue, "sink");
records.ghost_pad = gst_ghost_pad_new ("sink", records.queue_sink_pad);
gst_pad_set_active(records.ghost_pad, TRUE);
gst_element_add_pad(GST_ELEMENT(records.recording),records.ghost_pad);
g_signal_connect (records.sink, "format-location",
(GCallback)format_location_callback,
&records);
}
}
gboolean cmd_loop()
{
// other cmd not shown here
if(RECORD)
{
//create tee sink pad
// this step will trigger the pad-added function
tee_sink_pad = gst_element_get_request_pad (tee,"src");
// ....other function
}
}
int main()
{
// add the pad-added signal response
g_signal_connect(tee, "pad-added", G_CALLBACK(pad-added), NULL);
// use to construct the loop (cycle every 1s)
GSource* source = g_timeout_source_new(1000);
// set function to watch for command
g_source_set_callback(source,
(GSourceFunc)cmd_loop,
NULL,
NULL);
}
I've been trying to setup a simple sendonly WebRTC client with GStreamer but I'm having issues with getting the actual video to display on the WebRTC receiver side. I am new to both GStreamer and WebRTC.
I'm using the examples from https://gitlab.freedesktop.org/gstreamer/gst-examples/-/tree/master/webrtc to try and come up with a combination of certain parts. I've had 1:1 communication working but I wanted to introduce the rooms so I can have more clients viewing the "view-only" stream from GStreamer.
My current code is based on the multiparty-sendrecv example where I swapped out the audio for video. Furthermore, I'm using a modified version of the signalling server and a modified version of the javascript webrtc client. If necessary I could provide code for all of the above, but to keep things simple I won't. This is because I don't think the problem lies in either the signalling server or webrtc client, because the ICE candidates have been successfully negotiated along with the SDP offer & answer according to chrome://webrtc-internals/. See the image below.
In order to figure out what's going on I've exported a graph that shows the GStreamer pipeline after a user has joined the room and was added to the pipeline. See graph below.
As far as I can tell I should be receiving video data on my frontend, but I'm not. I've had a single weird case where the videotestsrc did show up, but I haven't been able to reproduce it. But because of this, it makes me think that the pipeline itself isn't neccesarily wrong, but perhaps we're dealing with some kind of race condition.
I've added the modified example of multiparty-sendrecv below, please take a look at it. Most of the methods have purposely been left out due to Stackoverflow's character limit.
Main functions
static void
handle_media_stream(GstPad* pad, GstElement* pipe, const char* convert_name,
const char* sink_name)
{
GstPad* qpad;
GstElement* q, * conv, * sink;
GstPadLinkReturn ret;
q = gst_element_factory_make("queue", NULL);
g_assert_nonnull(q);
conv = gst_element_factory_make(convert_name, NULL);
g_assert_nonnull(conv);
sink = gst_element_factory_make(sink_name, NULL);
g_assert_nonnull(sink);
gst_bin_add_many(GST_BIN(pipe), q, conv, sink, NULL);
gst_element_sync_state_with_parent(q);
gst_element_sync_state_with_parent(conv);
gst_element_sync_state_with_parent(sink);
gst_element_link_many(q, conv, sink, NULL);
qpad = gst_element_get_static_pad(q, "sink");
ret = gst_pad_link(pad, qpad);
g_assert_cmpint(ret, == , GST_PAD_LINK_OK);
}
static void
on_incoming_decodebin_stream(GstElement* decodebin, GstPad* pad,
GstElement* pipe)
{
GstCaps* caps;
const gchar* name;
if (!gst_pad_has_current_caps(pad)) {
g_printerr("Pad '%s' has no caps, can't do anything, ignoring\n",
GST_PAD_NAME(pad));
return;
}
caps = gst_pad_get_current_caps(pad);
name = gst_structure_get_name(gst_caps_get_structure(caps, 0));
if (g_str_has_prefix(name, "video")) {
handle_media_stream(pad, pipe, "videoconvert", "autovideosink");
}
else if (g_str_has_prefix(name, "audio")) {
handle_media_stream(pad, pipe, "audioconvert", "autoaudiosink");
}
else {
g_printerr("Unknown pad %s, ignoring", GST_PAD_NAME(pad));
}
}
static void
on_incoming_stream(GstElement* webrtc, GstPad* pad, GstElement* pipe)
{
GstElement* decodebin;
GstPad* sinkpad;
if (GST_PAD_DIRECTION(pad) != GST_PAD_SRC)
return;
decodebin = gst_element_factory_make("decodebin", NULL);
g_signal_connect(decodebin, "pad-added",
G_CALLBACK(on_incoming_decodebin_stream), pipe);
gst_bin_add(GST_BIN(pipe), decodebin);
gst_element_sync_state_with_parent(decodebin);
sinkpad = gst_element_get_static_pad(decodebin, "sink");
gst_pad_link(pad, sinkpad);
gst_object_unref(sinkpad);
}
static void
add_peer_to_pipeline(const gchar* peer_id, gboolean offer)
{
int ret;
gchar* tmp;
GstElement* tee, * webrtc, * q;
GstPad* srcpad, * sinkpad;
tmp = g_strdup_printf("queue-%s", peer_id);
q = gst_element_factory_make("queue", tmp);
g_free(tmp);
webrtc = gst_element_factory_make("webrtcbin", peer_id);
g_object_set(webrtc, "bundle-policy", GST_WEBRTC_BUNDLE_POLICY_MAX_BUNDLE, NULL);
gst_bin_add_many(GST_BIN(pipeline), q, webrtc, NULL);
srcpad = gst_element_get_static_pad(q, "src");
g_assert_nonnull(srcpad);
sinkpad = gst_element_get_request_pad(webrtc, "sink_%u");
g_assert_nonnull(sinkpad);
ret = gst_pad_link(srcpad, sinkpad);
g_assert_cmpint(ret, == , GST_PAD_LINK_OK);
gst_object_unref(srcpad);
gst_object_unref(sinkpad);
tee = gst_bin_get_by_name(GST_BIN(pipeline), "videotee");
g_assert_nonnull(tee);
srcpad = gst_element_get_request_pad(tee, "src_%u");
g_assert_nonnull(srcpad);
gst_object_unref(tee);
sinkpad = gst_element_get_static_pad(q, "sink");
g_assert_nonnull(sinkpad);
ret = gst_pad_link(srcpad, sinkpad);
g_assert_cmpint(ret, == , GST_PAD_LINK_OK);
gst_object_unref(srcpad);
gst_object_unref(sinkpad);
/* This is the gstwebrtc entry point where we create the offer and so on. It
* will be called when the pipeline goes to PLAYING.
* XXX: We must connect this after webrtcbin has been linked to a source via
* get_request_pad() and before we go from NULL->READY otherwise webrtcbin
* will create an SDP offer with no media lines in it. */
if (offer)
g_signal_connect(webrtc, "on-negotiation-needed",
G_CALLBACK(on_negotiation_needed), (gpointer)peer_id);
/* We need to transmit this ICE candidate to the browser via the websockets
* signalling server. Incoming ice candidates from the browser need to be
* added by us too, see on_server_message() */
g_signal_connect(webrtc, "on-ice-candidate",
G_CALLBACK(send_ice_candidate_message), (gpointer)peer_id);
/* Incoming streams will be exposed via this signal */
g_signal_connect(webrtc, "pad-added", G_CALLBACK(on_incoming_stream),
pipeline);
/* Set to pipeline branch to PLAYING */
ret = gst_element_sync_state_with_parent(q);
g_assert_true(ret);
ret = gst_element_sync_state_with_parent(webrtc);
g_assert_true(ret);
GST_DEBUG_BIN_TO_DOT_FILE(GST_BIN(pipeline), GST_DEBUG_GRAPH_SHOW_ALL, "pipeline");
}
static gboolean
start_pipeline(void)
{
GstStateChangeReturn ret;
GError* error = NULL;
/* NOTE: webrtcbin currently does not support dynamic addition/removal of
* streams, so we use a separate webrtcbin for each peer, but all of them are
* inside the same pipeline. We start by connecting it to a fakesink so that
* we can preroll early. */
/*pipeline = gst_parse_launch("tee name=videotee ! queue ! fakesink "
"videotestsrc is-live=true pattern=ball ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! "
"queue ! " RTP_CAPS_VP8 "96 ! videotee. ", &error);*/
pipeline = gst_parse_launch("tee name=videotee ! queue ! fakesink "
"videotestsrc is-live=true pattern=ball ! videoconvert ! queue ! vp8enc deadline=1 ! rtpvp8pay ! "
"queue ! " RTP_CAPS_VP8 "96 ! videotee. ", &error);
if (error) {
g_printerr("Failed to parse launch: %s\n", error->message);
g_error_free(error);
goto err;
}
g_print("Starting pipeline, not transmitting yet\n");
ret = gst_element_set_state(GST_ELEMENT(pipeline), GST_STATE_PLAYING);
if (ret == GST_STATE_CHANGE_FAILURE)
goto err;
return TRUE;
err:
g_print("State change failure\n");
if (pipeline)
g_clear_object(&pipeline);
return FALSE;
}
/*
* When we join a room, we are responsible for calling by starting negotiation
* with each peer in it by sending an SDP offer and ICE candidates.
*/
static void
do_join_room(const gchar* text)
{
gint ii, len;
gchar** peer_ids;
if (app_state != ROOM_JOINING) {
cleanup_and_quit_loop("ERROR: Received ROOM_OK when not calling",
ROOM_JOIN_ERROR);
return;
}
app_state = ROOM_JOINED;
g_print("Room joined\n");
/* Start recording, but not transmitting */
if (!start_pipeline()) {
cleanup_and_quit_loop("ERROR: Failed to start pipeline", ROOM_CALL_ERROR);
return;
}
peer_ids = g_strsplit(text, " ", -1);
g_assert_cmpstr(peer_ids[0], == , "ROOM_OK");
len = g_strv_length(peer_ids);
/* There are peers in the room already. We need to start negotiation
* (exchange SDP and ICE candidates) and transmission of media. */
if (len > 1 && strlen(peer_ids[1]) > 0) {
g_print("Found %i peers already in room\n", len - 1);
app_state = ROOM_CALL_OFFERING;
for (ii = 1; ii < len; ii++) {
gchar* peer_id = g_strdup(peer_ids[ii]);
g_print("Negotiating with peer %s\n", peer_id);
/* This might fail asynchronously */
call_peer(peer_id);
peers = g_list_prepend(peers, peer_id);
}
}
g_strfreev(peer_ids);
return;
}
int
main(int argc, char* argv[])
{
GOptionContext* context;
GError* error = NULL;
context = g_option_context_new("- gstreamer webrtc sendrecv demo");
g_option_context_add_main_entries(context, entries, NULL);
g_option_context_add_group(context, gst_init_get_option_group());
if (!g_option_context_parse(context, &argc, &argv, &error)) {
g_printerr("Error initializing: %s\n", error->message);
return -1;
}
if (!check_plugins())
return -1;
if (!room_id) {
g_printerr("--room-id is a required argument\n");
return -1;
}
if (!local_id)
local_id = g_strdup_printf("%s-%i", g_get_user_name(),
g_random_int_range(10, 10000));
/* Sanitize by removing whitespace, modifies string in-place */
g_strdelimit(local_id, " \t\n\r", '-');
g_print("Our local id is %s\n", local_id);
if (!server_url)
server_url = g_strdup(default_server_url);
/* Don't use strict ssl when running a localhost server, because
* it's probably a test server with a self-signed certificate */
{
GstUri* uri = gst_uri_from_string(server_url);
if (g_strcmp0("localhost", gst_uri_get_host(uri)) == 0 ||
g_strcmp0("127.0.0.1", gst_uri_get_host(uri)) == 0)
strict_ssl = FALSE;
gst_uri_unref(uri);
}
loop = g_main_loop_new(NULL, FALSE);
connect_to_websocket_server_async();
g_main_loop_run(loop);
gst_element_set_state(GST_ELEMENT(pipeline), GST_STATE_NULL);
g_print("Pipeline stopped\n");
gst_object_unref(pipeline);
g_free(server_url);
g_free(local_id);
g_free(room_id);
return 0;
}
I just downloaded FFMPEG and now I'm trying to use it in Qt with MSVC2013 compiler.
To understand how it works, I started reading the documentation and the API.
According to this figure, I was trying to make a little test with libavformat.
I did all they said in the demuxing module, then the muxing module. But, my program crashes when I call the avformat_write_header() function.
I would like to know what I did wrong and if you could help me to understand that.
In the main:
av_register_all();
if(!decode())
return;
The decode() methode:
bool MainWindow::decode()
{
AVFormatContext *formatContext = NULL;
AVPacket packet;
/**************** muxing varaiables ******************/
AVFormatContext *muxingContext = avformat_alloc_context();
AVOutputFormat *outputFormat = NULL;
AVIOContext *contextIO = NULL;
AVCodec *codecEncode = avcodec_find_encoder(AV_CODEC_ID_WMAV2);
AVStream *avStream = NULL;
AVCodecContext *codecContext = NULL;
/******************* demuxing **************************/
//open a media file
if(avformat_open_input(&formatContext,"h.mp3",NULL,NULL)!=0)
{
qDebug() << "paka ouve fichier";
return false;
}
//function which tries to read and decode a few frames to find missing
information.
if(avformat_find_stream_info(formatContext,NULL)<0)
{
qDebug()<<"paka find stream";
return false;
}
/**************** muxing *************************/
//The oformat field must be set to select the muxer that will be used.
muxingContext->oformat = outputFormat;
//Unless the format is of the AVFMT_NOFILE type, the pb field must be set to
//an opened IO context, either returned from avio_open2() or a custom one.
if(avio_open2(&contextIO,"out.wma",AVIO_FLAG_WRITE,NULL,NULL)<0)
{
qDebug() <<"paka kreye fichier soti";
return false;
}
muxingContext->pb = contextIO;
//Unless the format is of the AVFMT_NOSTREAMS type, at least
//one stream must be created with the avformat_new_stream() function.
avStream = avformat_new_stream(muxingContext,codecEncode);
//The caller should fill the stream codec context information,
//such as the codec type, id and other parameters
//(e.g. width / height, the pixel or sample format, etc.) as known
codecContext = avStream->codec;
codecContext->codec_type = AVMEDIA_TYPE_AUDIO;
codecContext->codec_id = AV_CODEC_ID_WMAV2;
codecContext->sample_fmt = codecEncode->sample_fmts[0];
codecContext->bit_rate = 128000;
codecContext->sample_rate = 44000;
codecContext->channels = 2;
//The stream timebase should be set to the timebase that the caller desires
//to use for this stream (note that the timebase actually used by the muxer
//can be different, as will be described later).
avStream->time_base = formatContext->streams[0]->time_base;
qDebug()<<formatContext->streams[0]->time_base.num <<"/"
<<formatContext- >streams[0]->time_base.den;
//When the muxing context is fully set up, the caller must call
//avformat_write_header()
//to initialize the muxer internals and write the file header
qDebug() << "does not crash yet";
if(avformat_write_header(muxingContext,NULL) <0)
{
qDebug()<<"cannot write header";
return false;
}
qDebug() << "OOps you can't see me (John Cena)";
///////////////////// Reading from an opened file //////////////////////////
while(av_read_frame(formatContext,&packet)==0)
{
//The data is then sent to the muxer by repeatedly calling
//av_write_frame() or av_interleaved_write_frame()
if(av_write_frame(muxingContext,&packet)<0)
qDebug()<<"paka write frame";
else
qDebug()<<"writing";
}
//Once all the data has been written, the caller must call
//av_write_trailer() to flush any buffered packets and finalize
//the output file, then close the IO context (if any) and finally
//free the muxing context with avformat_free_context().
if(av_write_trailer(muxingContext)!=0)
{
qDebug()<<"paka ekri trailer";
return false;
}
return true;
}
The program shows the message does not crash yet. But not OOps you can't see me (John Cena)
And there is no error. I used an MP3 file as input and I would like to ouput it in WMA.
Instead of avformat_alloc_context(), use avformat_alloc_output_context2(). This will set muxingContext->oformat.
I'm implementing gstreamer media player with my own source of data using appsrc. Everything works fine except one thing:
When stream reaches it's end, callback emits "end-of-stream" signal. Signals sending fucntion g_signal_emit_by_name(appsrc, "end-of-stream", &ret) returns GstFlowReturn value GST_FLOW_OK. But then it calls need-data my callback again, so it returns "end-of-stream" signal again. And this time GstFlowReturn value is (-3) which is GST_FLOW UNEXPECTED. I assume that it does not expect "end-of-stream" signal when it already recieved one, but why it requests more data than? Maybe it is because I didn't set size value iof the steam?
Gstreamer version is 0.10.
Callback function code (appsrc type is seekable btw):
static void cb_need_data (GstElement *appsrc, guint size, gpointer user_data)
{
GstBuffer *buffer;
GstFlowReturn ret;
AppsrcData* data = static_cast<AppsrcData*>(user_data);
buffer = gst_buffer_new_and_alloc(size);
int read = fread(GST_BUFFER_DATA(buffer), 1, size, data->file);
GST_BUFFER_SIZE(buffer) = read;
g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret);
if (ret != GST_FLOW_OK) {
/* something wrong, stop pushing */
g_printerr("GST_FLOW != OK, return value is %d\n", ret);
g_main_loop_quit (data->loop);
}
if(feof(data->file) || read == 0)
{
g_signal_emit_by_name(appsrc, "end-of-stream", &ret);
if (ret != GST_FLOW_OK) {
g_printerr("EOF reached, GST_FLOW != OK, return value is %d\nAborting...", ret);
g_main_loop_quit (data->loop);
}
}
}
You should provide some corrections to your code(if they are not there already) that should alleviate your issue and help the overall application:
Never try and send a buffer without first checking if it actually has data. So, simply check the buffer data and length to make sure that the data is not NULL and that the length is >0
You can flag that a stream is ended in your user_data. When you send your EOS, set an item in your userdata to indicate that it has been sent and if the appsrc requests more data, simply check if it has been sent and then do not send anything else to the buffer.
Listen for the EOS on your pipeline bus so that it can destroy the stream and close the loop when the EOS message is handled so that you can be sure that your mediasink has received the EOS and you can safely dispose of the pipeline and loop without losing any data.
Have you tried the method gst_app_src_end_of_stream()? I'm not sure what return code you should use after invoking it, but it should be either GST_FLOW_OK or GST_FLOW_UNEXPECTED.
In GStreamer 1.x you return GST_FLOW_EOS.