This isn't documented so maybe someone has an answer.
In android, I'm wondering if you can use androidmedia in a custom appsrc pipeline in place of the x264 plugin which is currently broken on Android. Does anyone know androidmedia's src/sink availability?
The pipeline is: appsrc->androidmedia->h264parse
AndroidMedia registers codecs that are suffixed by the hardware you have. As an exemple if you use a qualcom based hardware encoder you can use this pipeline :
videotestsrc is-live=true ! amcvidenc-omxqcomvideoencoderavc bitrate=6000000 i-frame-interval=2 ! h264parse ...
To know which hardware encoder is present on your system you can use this kotlin sample code :
var mediaCodecList = MediaCodecList(MediaCodecList.REGULAR_CODECS)
var codecName = "amcvidenc-"+mediaCodecList.findEncoderForFormat(MediaFormat.createVideoFormat("video/avc", 1920, 1080)).replace(".","").replace("-","").lowercase()
I think most of androids codecs are register with higher rank in the registry, so maybe you can use encodebin.
You can get more information here:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/blob/master/sys/androidmedia/gstamc.c
Sink and Src are pad templates for amcvidenc-...
templ =
gst_pad_template_new ("sink", GST_PAD_SINK, GST_PAD_ALWAYS, sink_caps);
gst_element_class_add_pad_template (element_class, templ);
gst_caps_unref (sink_caps);
templ = gst_pad_template_new ("src", GST_PAD_SRC, GST_PAD_ALWAYS, src_caps);
gst_element_class_add_pad_template (element_class, templ);
gst_caps_unref (src_caps);
Best regards
Related
I have a simple mjpeg pipeline and I want to access the buffer on the sink to get the pts to calculate the latency.
Pipeline:
souphttpsrc -> jpegparse -> imxvpudec -> imxipusink
What is the best way to do this? Some code examples would be great.
The time things in gstreamer confusing me a little bit.
I'd add an identity element in your pipeline where you want to analyze the PTS:
souphttpsrc ! jpegparse ! identity ! imxvpudec ! imxipusink
Then connect to the "handoff" signal:
static void pts_analysis_cb(GstElement *identity,
GstBuffer *buffer,
gpointer user_data) {
GstClockTime pts = GST_BUFFER_PTS(buffer);
//analysis
}
g_signal_connect_data(identity, "handoff",
G_CALLBACK(pts_analysis_cb),
NULL, NULL, GConnectFlags());
If you're seeing MJPEG related latency though you may just need to have sync=false on your tail element or set flags to drop buffers if it's falling behind.
UPDATE::
I want to stream video data (H264) through RTSP in Gstreamer.
gst_rtsp_media_factory_set_launch (factory, "videotestsrc ! x264enc ! rtph264pay name=pay0 pt=96 ");
I want "videotestsrc ! x264enc ! rtph264pay name=pay0 pt=96" this pipeline would also be in C programming in place of direct command.
Actually I have custom pipeline, i want to pass this pipeline to GstRTSPMediaFactory.
With launch i am not able to pass my pipline.
source = gst_element_factory_make("videotestsrc", "test-source");
parse = gst_element_factory_make("x264enc", "parse");
sink = gst_element_factory_make("rtph264pay", "sink");
gst_bin_add_many(GST_BIN(pipeline), source, parse, sink, NULL);
gst_element_link_many(source, parse, sink, NULL);
Now, I want to stream this pipeline using RTSP. I can stream with gst_rtsp_media_factory_set_launch,
But i want to pass only pipeline variable, and has to stream the video.
Can it possible, if so How?
I Modified the rtsp-media-factory.c as follows,
Added GstElement *pipeline in struct _GstRTSPMediaFactoryPrivate.
And the Added two more functions get_pipeline & set pipeline
void
gst_rtsp_media_factory_set_launch_pipeline (GstRTSPMediaFactory * factory, GstElement *pipeline)
{
g_print("PRASANTH :: SET LAUNCH PIPELINE\n");
GstRTSPMediaFactoryPrivate *priv;
g_return_if_fail (GST_IS_RTSP_MEDIA_FACTORY (factory));
g_return_if_fail (pipeline != NULL);
priv = factory->priv;
GST_RTSP_MEDIA_FACTORY_LOCK (factory);
// g_free (priv->launch);
priv->pipeline = pipeline;
Bin = priv->pipeline;
GST_RTSP_MEDIA_FACTORY_UNLOCK (factory);
}
In the Same way get also.
And at last in place of gst_parse_launch in function default_create_element,
added this line
element = priv->pipeline; // priv is of type GstRTSPMediaFactoryPrivate
return element;
but I am not able to receive the data.
When i put pay0 for rtpmp2pay it is working.
But it is working for once only. If Client stops and again starts its not working. To work it, again i am restarting the server.
What is the problem?
** (rtsp_server:4292): CRITICAL **: gst_rtsp_media_new: assertion 'GST_IS_ELEMENT (element)' failed
To have some answer here.
It solves the main problem according to comments discussion, but there is still problem with requesting another stream (when stopping and starting client).
The solution was to add proper name for payloader element as stated in docs:
The pipeline description should contain elements named payN, one for each
stream (ex. pay0, pay1, ...). Also, for increased compatibility each stream
should have a different payload type which can be configured on the payloader.
So this has to be changed to:
sink = gst_element_factory_make("rtph264pay", "pay0");
notice the change in name of element from sink -> pay0.
For the stopping client issue I would check if this works for parse version.
If yes then check if the parse pipeline string (in original source code of rtsp server) is saved anywhere and reused after restart.. you need to debug this.
I am designing a pipeline to Encode a video frame from a opencv application (got from a web cam) to video/x-h264 format, send it via network and decode it on another device of different type (probably a raspberry pi ) to a proper RGB stream for my project.
For this I am supposed to use a hardware accelerated Encoder and Decoder.
Since , the whole scenario is huge , the current development is performed on a Intel machine using the gstreamer VAAPI plugins(vaapiencode_h264 & vaapidecode ) . Ánd also, the fact that we need to NOT use any of the networking plugins like TCPServer or UDPServer
For this I have used the below pipeline for my purpose :
On the Encoder End:
appsrc name=applicationSource ! videoconvert ! video/x-raw, format=I420, width=640, height=480,framerate=30/1, pixel-aspect-ratio=1/1,interlace-mode=progressive ! vaapiencode_h264 bitrate=600 tune=high-compression ! h264parse config-interval=1 ! appsink name=applicationSink sync=false
The Appsrc part works perfectly well while the appsink part is having some issue with it.
The appsink part of this pipeline has been set with the below caps:
"video/x-h264, format=(string){avc,avc3,byte-stream },alignment=(string){au,nal};video/mpeg, mpegversion=(int)2, profile=(string)simple"
The code for the data extraction of my appsink is
bool HWEncoder::grabData()
{
// initial checks..
if (!cameraPipeline)
{
GST_ERROR("ERROR AS TO NO PIPE FOUND ... Stopping FRAME GRAB HERE !! ");
return false;
}
if (gst_app_sink_is_eos (GST_APP_SINK(applicationSink)))
{
GST_WARNING("APP SINK GAVE US AN EOS! BAILING OUT ");
return false;
}
if (sample)
{
cout << "sample available ... unrefing it ! "<< endl;
gst_sample_unref(sample);
}
sample = gst_app_sink_pull_sample (GST_APP_SINK(applicationSink));
if (!sample)
{
GST_WARNING("No valid sample");
return false; // no valid sample pulled !
}
sink_buffer = gst_sample_get_buffer(sample);
if (!sink_buffer)
{
GST_ERROR("No Valid Buffer ");return false;
}
return true;
}
After bringing up the pipeline and checking for the buffer filling up in my appsink, I am getting stuck at the below said lines ofmy code indefinitely:
sample = gst_app_sink_pull_sample (GST_APP_SINK(applicationSink));
I have the following questions :
1) Is my Caps for appsink correct ? If not How can I determine the caps for them ?
2) Is there something wrong in my pipeline above ?
How can I fix this issue with Appsink ??
Any kind of help would be useful!
Thanks !!
Just a guess (I had similar problems) the problem having appsink and appsrc in same pipeline may be that when you fill/empty one of them it will block the other(more on that below).
appsink and appsrc would block when they are full/empty - this is normal desired behaviour. There is option drop for appsink or for appsrc there is option block - but using these it may be just workaround and you will get glitches in your stream. Proper solution is to handle the synchronisation between appsrc and appsink in a better way.
You can react on appsrc signals enough-data and need-data - this is our way. Also we fiddled with properties of appsrc: is-live, do-timestamp and buffer size (this may or may not help you):
g_object_set(src->appsrc,
"stream-type", GST_APP_STREAM_TYPE_STREAM,
"format", GST_FORMAT_TIME,
"do-timestamp", TRUE,
"is-live", TRUE,
"block", TRUE,
NULL);
Why do they block each other?
Because (I guess) you process appsink and at the same time appsrc in main application thread. When one of the appsink/appsrc block the thread there is no one that would handle the processing for the other one. So when appsink is blocked because it does not have any data there is noone that can feed appsrc with new data - thus endless deadlock.
We also implemented noblock version of appsink *pull_sample method but it was just a workaround and resulted in more problems than solutions.
If you want to debug what is happening you can add GST_DEBUG entry for appsrc/appsink (I do not remember what they were), you can add callback on mentioned enough-data and need-data signals or you may add queues and enable GST_DEBUG=queue_dataflow:5 to see which queue is filled first etc.. this is always helpful when debugging the "data-deadlock".
My pipeline scheme(dynamic link):
videotestsrc OR audiotestsrc ! decodebin ! queue ! autovideosink OR
autoaudiosink
I trying to use this advice to check which type of data I got (video/audio), but if I use decodebin like demuxer, then I get just "src_0" instead of "audio" or "video". How I can check my pad type for linking right element for playback? May be I can use one universal element for audio playback and video playback, like playsink(but it does not work for video)?
You can get the caps of the newly added pad and check if it contains audio or video caps (or something else).
Try with:
gst_pad_get_current_caps (pad);
or:
gst_pad_get_allowed_caps (pad);
If you are using gstreamer 0.10 (which is 3+ years obsolete an unmantained), you have:
gst_pad_get_caps_reffed (pad);
Then just check the returned caps if it is audio or video by getting the structure from the caps and checking if its name starts with video or audio.
/* There might be multiple structures depending on how you do it,
* but usually checking one in this case is enough */
structure = gst_caps_get_structure (caps, 0);
name = gst_structure_get_name (structure);
if (g_str_has_prefix (name, "video/")) {
...
} else if (g_str_has_prefix (name, "audio/")) {
...
}
I'm having difficulties in retrieving rtsp stream from a specific camera, because the rtp payload type the camera is providing is 35 (unassigned) and payload types accepted by the rtph264depay plugin are in range [96-127]. The result is that gstreamer displays ann error like:
<udpsrc0> error: Internal data flow error.
<udpsrc0> error: streaming task paused, reason not-linked (-1)
Other cameras that I have tested are working because they define a good payload type.
FFmpeg, MPlayer and other tools play the stream, although they may display a warning for the unknown type, for instance in Mplayer:
rtsp_session: unsupported RTSP server. Server type is 'unknown'
Is there any way in gstreamer to fake the payload type, ignore the mismatching property, force linking between the plugins or otherwise create a workaroud to my problem?
Pipeline I am using is:
gst-launcg-0.10 rtspsrc location="..." ! rtph264depay ! capsfilter caps="video/x-h264,width=1920,height=1080,framerate=(fraction)25/1" ! h264parse ! matroskamux ! filesink location="test.mkv"
I figured it out and got it working. Posting an answer here in hope that it might benefit someone. There are multiple similar questions out there, but they lack proper answers.
Following does the trick:
GstElement* depay = gst_element_factory_make("rtph264depay", "video_demux");
assert(depay);
GstPad* depay_sink = gst_element_get_static_pad(depay, "sink");
GstCaps* depay_sink_caps = gst_caps_new_simple("application/x-rtp",
"media", G_TYPE_STRING, "video",
"encoding-name", G_TYPE_STRING, "H264",
NULL);
gst_pad_use_fixed_caps(depay_sink);
gst_pad_set_caps(depay_sink, depay_sink_caps);
gst_object_unref(depay_sink);
it overrides the rtph264depay plugin's sink pad caps to be less restrictive, now it accepts any payload type (and any clock-rate) as long as it is rtp and has H.264 encoding.
I don't think this is possible with gst-launch.
There is a select-stream signal in rtspsrc module documented here http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good-plugins/html/gst-plugins-good-plugins-rtspsrc.html#GstRTSPSrc-select-stream
it's a callback where you check the stream and if you return true, gstreamer will SETUP and PLAY the stream, if you return false it will ignore it, this should let you ignore the unsupported stream, in my case I'm having trouble with ONVIF metadata stream, it always tries to play it and there is no parser for it, I really wish gstreamer will just ignore the streams that can't play and work with what it has or at least a flag to toggle that behaviour.