I have a gstreamer pipeline that works in the command line and I am trying to convert it to C++ code. I have most of it, except I need to be able to write the -e flag in C++ but I'm not sure how to add it to the pipeline. Here is the command line
gst-launch-1.0 -e udpsrc port=8000 ! application/x-rtp, encoding-name=H264, payload=109 ! tee name=t t. ! rtph264depay ! h264parse ! queue ! avdec_h264 ! videoconvert ! autovideosink t. ! rtph264depay ! h264parse ! queue ! mp4mux ! filesink location=!/camera.mp4"
Here is the C++ code I have. This works to display a live stream from a camera and write a mp4 file, however it is not readable. The -e flag makes the file able to be played.
// [1] Create Elements
pipeline = gst_pipeline_new("xvoverlay");
src = gst_element_factory_make("udpsrc", NULL);
caps = gst_element_factory_make("capsfilter", NULL);
tee = gst_element_factory_make("tee", "tee");
// Display
rtpDepay = gst_element_factory_make("rtph264depay", NULL);
h264Parse = gst_element_factory_make("h264parse", NULL);
displayQueue = gst_element_factory_make("queue", NULL);
decoder = gst_element_factory_make("avdec_h264", NULL);
videoConvert = gst_element_factory_make("videoconvert", NULL);
upload = gst_element_factory_make("d3d11upload", NULL);
sink = gst_element_factory_make("d3d11videosink", NULL);
// Record
recordRtpDepay = gst_element_factory_make("rtph264depay", NULL);
recordH264Parse = gst_element_factory_make("h264parse", NULL);
recordQueue = gst_element_factory_make("queue", "save_queue");
mux = gst_element_factory_make("mp4mux", NULL);
filesink = gst_element_factory_make("filesink", NULL);
// [2] Set element properties
g_object_set(src, "port", port, NULL);
g_object_set(caps, "caps", gst_caps_from_string("application/x-rtp, encoding-name=H264, payload=109"), NULL);
g_object_set(filesink, "location", "camera.mp4", NULL);
//g_object_set(mux, "faststart", true, NULL);
// [3] Add elements to pipeline and link together
//gst_bin_add_many(GST_BIN(pipeline), src, caps, rtpDepay, h264Parse, displayQueue, decoder, videoConvert, upload, sink, NULL);
//gst_element_link_many(src, caps, rtpDepay, h264Parse, displayQueue, decoder, videoConvert, upload, sink, NULL);
gst_bin_add_many(GST_BIN(pipeline), src, caps, tee, rtpDepay, h264Parse, displayQueue, decoder, videoConvert, upload, sink, recordRtpDepay, recordH264Parse, recordQueue, mux, filesink, NULL);
if (!gst_element_link_many(src, caps, tee, NULL)
|| !gst_element_link_many(tee, rtpDepay, h264Parse, displayQueue, decoder, videoConvert, upload, sink, NULL)
|| !gst_element_link_many(tee, recordRtpDepay, recordH264Parse, recordQueue, mux, filesink, NULL))
{
qDebug() << "Failed to link elements";
}
How do I add a -e flag as a GstElement? I've searched online and I can't find anyone trying to do this programatically with that flag.
The -e flag sends and EOS at the end of the stream. While processing the EOS message, the video writer will write the header information needed for the video to be playable.
The solution is to change the way you stop your pipeline. Instead of however you currently do it (you did not include that code), you should get access to the srcpad of your udpsrc object, and then send a GST_EVENT_EOS. This will signal to the application that you have ended the stream. Each element will process what it needs to, and then forward that event further down the pipeline. It will reach the video writer, which will then write the needed header information to your videofile, before entering a paused state.
On shutdown, call
gst_element_send_event(src, gst_event_new_eos ())
This will send EOS event downstream and write the required meta data.
it's easier to use gst_parse_launch and then gst_bin_get_by_name on elements you want to do fancy things with.
The -e flags can be given in gst_init
Related
I have an RTP streaming app which implements the following pipeline using the C API.
gst-launch-1.0 -v rtpbin name=rtpbin \
videotestsrc ! x264enc ! rtph264pay! rtpbin.send_rtp_sink_0 \
rtpbin.send_rtp_src_0 ! udpsink port=5002 host=127.0.0.1 \
rtpbin.send_rtcp_src_0 ! udpsink port=5003 host=127.0.0.1 sync=false async=false \
udpsrc port=5007 ! rtpbin.recv_rtcp_sink_0
I want to add header extensions to the RTP packet; therefore I created an extension using the new GstRTPHeaderExtension class introduced in GStreamer v1.20. I want to set the attributes of the extension (e.g. color space properties for the example below). AFAIU this should be done by providing those as caps to the payloader element. However, I can't figure out how I should provide these caps exactly. Do I need to use a capsfilter here or what is the right way? In the current state, I can send the RTP packets and see that the extension is added but can't set the attributes.
Related parts of the code are below:
#define URN_COLORSPACE "http://www.webrtc.org/experiments/rtp-hdrext/color-space"
const GstVideoColorimetry colorimetry = {
GST_VIDEO_COLOR_RANGE_0_255,
GST_VIDEO_COLOR_MATRIX_BT601,
GST_VIDEO_TRANSFER_BT2020_10,
GST_VIDEO_COLOR_PRIMARIES_BT2020};
const GstVideoChromaSite chroma_site = GST_VIDEO_CHROMA_SITE_MPEG2;
ext = gst_rtp_header_extension_create_from_uri(URN_COLORSPACE);
gst_rtp_header_extension_set_id(ext, 1);
g_signal_emit_by_name(videopay, "add-extension", ext);
// other element definitions, links..
videopay = gst_element_factory_make("rtph264pay", "videopay");
colorimetry_str = gst_video_colorimetry_to_string(&colorimetry);
// How to provide these caps to the payloader set the extension properties?
caps = gst_caps_new_simple("application/x-rtp",
"media", G_TYPE_STRING, "video",
"clock-rate", G_TYPE_INT, 90000,
"encoding-name", G_TYPE_STRING, "H264",
"colorimetry", G_TYPE_STRING, colorimetry_str,
"chroma-site", G_TYPE_STRING,
gst_video_chroma_to_string(chroma_site), NULL);
The caps should be provided to the sink of the RTP payloader element using a capsfilter element:
GstElement *capsfilt;
capsfilt = gst_element_factory_make("capsfilter", "capsfilter");
g_object_set(capsfilt, "caps", caps, NULL);
gst_element_link_many(videosrc, videoenc, capsfilt, videopay, NULL)
where videosrc, videoenc, videopay are the source, encoder and payloader elements, respectively.
Also, the caps should have a media type matching to the encoder element. E.g. video/x-h264 if the encoder element is an instance of x264enc.
The payloader will try to automatically enable the extension with the attributes set in the caps by passing the caps to the extension, if auto-header-extension is enabled (set to true by default).
In a gst-launch pipeline, the caps are passed automatically when the header extension is inserted after the payloader.
I'm new for gstreamer. I want to encode the video of my MacbookPro. built-in cam to h264 and then play. in command line, I tried "
gst-launch-1.0 autovideosrc ! queue ! x264enc ! avdec_h264 ! queue ! autovideosink " and it works.but when I run the c++ code, it failed, only show a green screen.
video_src = gst_element_factory_make("autovideosrc", "video_source");
video_enc = gst_element_factory_make("x264enc", "videoEncoder");
video_dec = gst_element_factory_make("avdec_h264", "videodecoder");
video_sink = gst_element_factory_make("osxvideosink", nullptr);
gst_bin_add_many...
gst_element_link_many (video_src, screen_queue, video_enc, video_dec, video_sink, NULL);
not sure how to correct it. thanks!
I am sending an encoded H264 stream using gstreamer and decoding it on an Intel hardware.
My sender is actually an application and its pipeline elements look something like this:
caps2 = gst_caps_new_simple("video/x-raw",
"format", G_TYPE_STRING, "I420",
"width", G_TYPE_INT, 640,
"height", G_TYPE_INT, 480,
"framerate",GST_TYPE_FRACTION, 15 ,1,
"pixxel-aspect-ratio",GST_TYPE_FRACTION,1,1, NULL);
gst_app_src_set_caps(GST_APP_SRC(app->videosrc),caps2);
gst_bin_add_many(GST_BIN(app->pipeline), app->videosrc, app->x264enc, app->rtppay, app->udpsink, NULL );
So basically an appsrc element getting data from a live camera, encoded using h264 encoder and sent through udp sink .
The receiver pipeline looks like this :
gst-launch-1.0 -v udpsrc port=5002 ! application/x-rtp, media=video, clock-rate=90000, encoding-name=H264, payload=96 ! rtph264pdepay ! vaapih264dec ! videoconvert ! vaapisink sync=FALSE
The output looks like the one shown in the figure:output of the receiver pipeline on my hardware
I want to know where am I going wrong?is it latency or color format?
I am trying to create a pipeline and then add on a videosink after I've created it. I need to do this so I can set the video overlay window id of the videosink before I commit it to the pipeline.
so I have this code
pipeline = gst_parse_launch( "filesrc location=file.svg ! rsvgdec ! imagefreeze", &err );
sink = gst_element_factory_make( "glimagesink" );
gst_video_overlay_set_window_handle( GST_VIDEO_OVERLAY( sink ), this->winId() );
gst_bin_add_many( GST_BIN( pipeline ), sink, nullptr );
if ( !gst_element_link_many( pipeline, sink, nullptr ) )
{
qCritical() << "Unable to link elements";
}
When I run it it fails to link the elements.
Any idea why this is happening. I'm assuming it's because I'm trying to link an element to a "bin" rather than another element. However, I can't see any examples of where someone adds an element to a pipeline which was created through gst_parse_launch.
You can't connect it to a bin. You need to specify a pad - or an element from where it tries to pick a pad. So you would need iterate through the bin and pick the imagefreeze element from the list.
Alternative approach - add the sink and get it from the pipeline:
pipeline = gst_parse_launch( "filesrc location=file.svg ! rsvgdec ! imagefreeze ! glimagesink name=mysink", &err );
GstElement *sink = gst_bin_get_by_name( GST_BIN( pipeline ), "mysink" );
gst_video_overlay_set_window_handle( GST_VIDEO_OVERLAY( sink ), this->winId() );
gst_object_unref( sink );
You may have issues here as well tough as you may require a videoconvert to satisfy the sink's format requirements.
filesrc location=file.svg ! rsvgdec ! imagefreeze ! videoconvert ! glimagesink
or maybe
filesrc location=file.svg ! rsvgdec ! imagefreeze ! glupload ! glcolorconvert ! glimagesink
I'd like to use pipeline below to play content with sound and without sound. Problem is that content without sound PREROLLING pipeline, but doesn't play
gst-launch-1.0.exe uridecodebin uri=file:///home/mymediafile.ogv name=d1 ! tee name=t1 ! queue max-size-buffers=2 ! jpegenc ! appsink name=myappsink t1. ! queue ! autovideosink d1. ! queue ! audioconvert ! audioresample ! autoaudiosink
How can I solve such issue?
I found no way to get your pipeline going on the command line. If I put in the audio portion of the pipeline, the files with no audio hang.
In your application however, you'll be able to add a signal for the pad_added events, and only added the audio portion of the pipeline when needed. Some pseudo code:
void decodebin_pad_added(GstElement *decodebin, GstPad *new_pad, gpointer user_data) {
GstElement* pipeline = (GstElement*)user_data;
GstCaps* audio_caps = gst_caps_from_string("audio/x-raw");
GstCaps* pad_caps = gst_pad_get_current_caps(new_pad);
if(! gst_caps_can_intersect(pad_caps, audio_caps)) {
return;
}
GstElement* audio_pipeline = gst_parse_launch("queue ! audioconvert ! audioresample ! autoaudiosink", NULL);
gst_bin_add(GST_BIN(pipeline), audio_pipeline);
GstElement* decodebin = gst_bin_get_by_name(GST_BIN(pipeline), "d1");
gst_element_link(decodebin, audio_pipeline);
gst_object_unref(decodebin);
}
void decodebin_no_more_pads(GstElement *decodebin, gpointer user_data) {
GstElement* pipeline = (GstElement*)user_data;
gst_element_set_state(pipeline, GST_PLAYING);
}
GstElement* pipeline = gst_parse_launch("uridecodebin uri=file:///home/mymediafile.ogv name=d1 ! tee name=t1 ! queue max-size-buffers=2 ! jpegenc ! appsink name=myappsink t1. ! queue ! autovideosink", NULL);
GstElement* decodebin = gst_bin_get_by_name(GST_BIN(pipeline), "d1");
g_signal_connect(decodebin, "pad-added", G_CALLBACK(decodebin_pad_added), pipeline);
g_signal_connect(decodebin, "no-more-pads", G_CALLBACK(decodebin_no_more_pads), pipeline);
gst_element_set_state(pipeline, GST_STATE_PAUSED); //pause to make demuxer and decoders get setup and find out what's in the file
Add async-handling=true to the autoaudiosink.
gst-launch-1.0.exe uridecodebin uri=file:///home/mymediafile.ogv
name=d1 ! tee name=t1 ! queue max-size-buffers=2 ! jpegenc ! appsink
name=myappsink t1. ! queue ! autovideosink d1. ! queue ! audioconvert
! audioresample ! autoaudiosink async-handling=true