I am trying to convert a webcam on a raspberry pi to x264, but keep running into an error about an " Unsupported profile constrained-baseline".
GST_DEBUG=3 /home/pi/gst-rtsp-server/examples/test-launch "( v4l2src device=/dev/video0 ! videoconvert ! omxh264enc ! h264parse ! rtph264pay name=pay0 )"
stream ready at rtsp://127.0.0.1:8554/test
0:00:03.043939441 10314 0x75c08350 WARN v4l2src gstv4l2src.c:692:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:03.044207251 10314 0x7491de30 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<appsrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:03.044211053 10314 0x7491de60 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<appsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:03.087901354 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:03.087992083 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:03.088080988 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:03.092800068 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:03.092867411 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:03.092942775 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:03.092983764 10314 0x7491de90 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:<videoscale0:src> could not send sticky events
0:00:04.001816134 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:04.001956914 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:04.002098111 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:04.037642275 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:04.037781284 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:04.037959303 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:04.038154562 10314 0x7491de90 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:04.038222947 10314 0x7491de90 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4)
0:00:04.038854870 10314 0x75c08350 WARN rtspmedia rtsp-media.c:1834:default_handle_message: 0x75c4e120: got error Internal data stream error. (gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:media-pipeline/GstBin:bin0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4))
0:00:04.039143357 10314 0x20b2490 WARN rtspmedia rtsp-media.c:2127:wait_preroll: failed to preroll pipeline
0:00:04.039219346 10314 0x20b2490 WARN rtspmedia rtsp-media.c:2384:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:04.053306794 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:04.053442678 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:04.053582781 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:04.079457589 10314 0x20b2490 ERROR rtspclient rtsp-client.c:678:find_media: client 0x2010610: can't prepare media
0:00:04.081144660 10314 0x20b2490 ERROR rtspclient rtsp-client.c:2210:handle_describe_request: client 0x2010610: no media
There isn't anything in the docs for omxh264enc about profiles that I can find:
gst-inspect-1.0 omxh264enc
Factory Details:
Rank primary + 1 (257)
Long-name OpenMAX H.264 Video Encoder
Klass Codec/Encoder/Video
Description Encode H.264 video streams
Author Sebastian Dröge <sebastian.droege#collabora.co.uk>
Plugin Details:
Name omx
Description GStreamer OpenMAX Plug-ins
Filename /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstomx.so
Version 1.14.4
License LGPL
Source module gst-omx
Source release date 2018-10-02
Binary package GStreamer OpenMAX Plug-ins source release
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoEncoder
+----GstOMXVideoEnc
+----GstOMXH264Enc
+----GstOMXH264Enc-omxh264enc
Implemented Interfaces:
GstPreset
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
video/x-h264
width: [ 16, 4096 ]
height: [ 16, 4096 ]
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
name : The name of the object
flags: readable, writable
String. Default: "omxh264enc-omxh264enc0"
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: false
control-rate : Bitrate control method
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncControlRate" Default: -1, "default"
(0): disable - Disable
(1): variable - Variable
(2): constant - Constant
(3): variable-skip-frames - Variable Skip Frames
(4): constant-skip-frames - Constant Skip Frames
(-1): default - Component Default
target-bitrate : Target bitrate in bits per second (0xffffffff=component default)
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-i-frames : Quantization parameter for I-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-p-frames : Quantization parameter for P-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-b-frames : Quantization parameter for B-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
inline-header : Inline SPS/PPS header before IDR
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: true
periodicity-idr : Periodicity of IDR frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
periodicty-idr : Periodicity of IDR frames (0xffffffff=component default) DEPRECATED - only for backwards compat
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
interval-intraframes: Interval of coding Intra frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
b-frames : Number of B-frames between two consecutive I-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
entropy-mode : Entropy mode for encoding process
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXH264EncEntropyMode" Default: -1, "default"
(0): CAVLC - CAVLC entropy mode
(1): CABAC - CABAC entropy mode
(-1): default - Component Default
constrained-intra-prediction: If enabled, prediction only uses residual data and decoded samples from neighbouring coding blocks coded using intra prediction modes
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
loop-filter-mode : Enable or disable the deblocking filter (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXH264EncLoopFilter" Default: -1, "default"
(0): enable - Enable deblocking filter
(1): disable - Disable deblocking filter
(2): disable-slice-boundary - Disables deblocking filter on slice boundary
(-1): default - Component Default
I am new to gstreamer so I could be reading this all wrong, would appreciate any ideas or feedback! Thanks!
You can change that through the SRC cap, try e.g.:
v4l2src device=/dev/video0 ! videoconvert ! omxh264enc ! video/x-h264,profile=baseline ! h264parse ! rtph264pay name=pay0
Related
I have an RTP streaming app which implements the following pipeline using the C API.
gst-launch-1.0 -v rtpbin name=rtpbin \
videotestsrc ! x264enc ! rtph264pay! rtpbin.send_rtp_sink_0 \
rtpbin.send_rtp_src_0 ! udpsink port=5002 host=127.0.0.1 \
rtpbin.send_rtcp_src_0 ! udpsink port=5003 host=127.0.0.1 sync=false async=false \
udpsrc port=5007 ! rtpbin.recv_rtcp_sink_0
I want to add header extensions to the RTP packet; therefore I created an extension using the new GstRTPHeaderExtension class introduced in GStreamer v1.20. I want to set the attributes of the extension (e.g. color space properties for the example below). AFAIU this should be done by providing those as caps to the payloader element. However, I can't figure out how I should provide these caps exactly. Do I need to use a capsfilter here or what is the right way? In the current state, I can send the RTP packets and see that the extension is added but can't set the attributes.
Related parts of the code are below:
#define URN_COLORSPACE "http://www.webrtc.org/experiments/rtp-hdrext/color-space"
const GstVideoColorimetry colorimetry = {
GST_VIDEO_COLOR_RANGE_0_255,
GST_VIDEO_COLOR_MATRIX_BT601,
GST_VIDEO_TRANSFER_BT2020_10,
GST_VIDEO_COLOR_PRIMARIES_BT2020};
const GstVideoChromaSite chroma_site = GST_VIDEO_CHROMA_SITE_MPEG2;
ext = gst_rtp_header_extension_create_from_uri(URN_COLORSPACE);
gst_rtp_header_extension_set_id(ext, 1);
g_signal_emit_by_name(videopay, "add-extension", ext);
// other element definitions, links..
videopay = gst_element_factory_make("rtph264pay", "videopay");
colorimetry_str = gst_video_colorimetry_to_string(&colorimetry);
// How to provide these caps to the payloader set the extension properties?
caps = gst_caps_new_simple("application/x-rtp",
"media", G_TYPE_STRING, "video",
"clock-rate", G_TYPE_INT, 90000,
"encoding-name", G_TYPE_STRING, "H264",
"colorimetry", G_TYPE_STRING, colorimetry_str,
"chroma-site", G_TYPE_STRING,
gst_video_chroma_to_string(chroma_site), NULL);
The caps should be provided to the sink of the RTP payloader element using a capsfilter element:
GstElement *capsfilt;
capsfilt = gst_element_factory_make("capsfilter", "capsfilter");
g_object_set(capsfilt, "caps", caps, NULL);
gst_element_link_many(videosrc, videoenc, capsfilt, videopay, NULL)
where videosrc, videoenc, videopay are the source, encoder and payloader elements, respectively.
Also, the caps should have a media type matching to the encoder element. E.g. video/x-h264 if the encoder element is an instance of x264enc.
The payloader will try to automatically enable the extension with the attributes set in the caps by passing the caps to the extension, if auto-header-extension is enabled (set to true by default).
In a gst-launch pipeline, the caps are passed automatically when the header extension is inserted after the payloader.
gstreamer raises an error when trying to stream side-by-side video from a stereoscopic UVC camera.
I have a stereoscopic camera attached via USB to an ARM board, but on the highest resolution setting that the camera allows gstreamer is raising an Invalid Dimension 0x0 error.
v4l2-ctl --list-formats-ext -d /dev/video2
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'MJPG' (compressed)
Name : Motion-JPEG
Size: Discrete 2560x960
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 2560x720
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x480
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x240
Interval: Discrete 0.017s (60.000 fps)
Interval: Discrete 0.033s (30.000 fps)
$ gst-launch-1.0 -v v4l2src device=/dev/video2 ! "image/jpeg, width=2560, height=960, framerate=60/1" ! progressreport ! rtpjpegpay ! udpsink host=127.0.0.1 port=5000
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = image/jpeg, width=(int)2560, height=(int)960, framerate=(fraction)60/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = image/jpeg, width=(int)2560, height=(int)960, framerate=(fraction)60/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstProgressReport:progressreport0.GstPad:src: caps = image/jpeg, width=(int)2560, height=(int)960, framerate=(fraction)60/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstRtpJPEGPay:rtpjpegpay0.GstPad:src: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)JPEG, a-framerate=(string)60.000000, x-dimensions=(string)"2560\,960", payload=(int)26, ssrc=(uint)1656850644, timestamp-offset=(uint)2590317031, seqnum-offset=(uint)18356
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)JPEG, a-framerate=(string)60.000000, x-dimensions=(string)"2560\,960", payload=(int)26, ssrc=(uint)1656850644, timestamp-offset=(uint)2590317031, seqnum-offset=(uint)18356
/GstPipeline:pipeline0/GstRtpJPEGPay:rtpjpegpay0.GstPad:sink: caps = image/jpeg, width=(int)2560, height=(int)960, framerate=(fraction)60/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstProgressReport:progressreport0.GstPad:sink: caps = image/jpeg, width=(int)2560, height=(int)960, framerate=(fraction)60/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = image/jpeg, width=(int)2560, height=(int)960, framerate=(fraction)60/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
0:00:00.163107430 7652 0x55d47a920a30 WARN v4l2bufferpool gstv4l2bufferpool.c:790:gst_v4l2_buffer_pool_start:<v4l2src0:pool:src> Uncertain or not enough buffers, enabling copy threshold
/GstPipeline:pipeline0/GstRtpJPEGPay:rtpjpegpay0: timestamp = 2590320450
/GstPipeline:pipeline0/GstRtpJPEGPay:rtpjpegpay0: seqnum = 18356
progressreport0 (00:00:05): 4 seconds
progressreport0 (00:00:10): 9 seconds
progressreport0 (00:00:15): 14 seconds
0:00:15.770955137 7652 0x55d47a920a30 WARN v4l2src gstv4l2src.c:968:gst_v4l2src_create:<v4l2src0> lost frames detected: count = 2 - ts: 0:00:15.622372937
progressreport0 (00:00:20): 19 seconds
progressreport0 (00:00:25): 24 seconds
progressreport0 (00:00:30): 29 seconds
progressreport0 (00:00:35): 34 seconds
Then on the viewing machine (currently just using localhost on the same laptop):
$ gst-launch-1.0 -e -v udpsrc port=5000 ! application/x-rtp, encoding-name=JPEG, payload=26 ! rtpjpegdepay ! jpegdec ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = application/x-rtp, encoding-name=(string)JPEG, payload=(int)26, media=(string)video, clock-rate=(int)90000
/GstPipeline:pipeline0/GstRtpJPEGDepay:rtpjpegdepay0.GstPad:sink: caps = application/x-rtp, encoding-name=(string)JPEG, payload=(int)26, media=(string)video, clock-rate=(int)90000
WARNING: from element /GstPipeline:pipeline0/GstRtpJPEGDepay:rtpjpegdepay0: Invalid Dimension 0x0.
Additional debug info:
gstrtpjpegdepay.c(741): gst_rtp_jpeg_depay_process (): /GstPipeline:pipeline0/GstRtpJPEGDepay:rtpjpegdepay0
WARNING: from element /GstPipeline:pipeline0/GstRtpJPEGDepay:rtpjpegdepay0: Invalid Dimension 0x0.
The two lowest resolution modes work with this configuration, but the 720p side-by-side mode throws the foregoing errors.
What am I doing wrong here? And does this have anything to do with gst-launch-1.0 not supporting full screen mode perhaps?
Thanks in advance
See RFC 2435 https://www.rfc-editor.org/rfc/rfc2435 on width and height:
3.1.5. Width: 8 bits
This field encodes the width of the image in 8-pixel multiples (e.g.,
a width of 40 denotes an image 320 pixels wide). The maximum width
is 2040 pixels.
3.1.6. Height: 8 bits
This field encodes the height of the image in 8-pixel multiples
(e.g., a height of 30 denotes an image 240 pixels tall). When
encoding interlaced video, this is the height of a video field, since
fields are individually JPEG encoded. The maximum height is 2040
pixels.
Here you can see that the limit is 2040 pixels in both dimensions. This is a protocol limitation.
Check the GStreamer source code https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/blob/master/gst/rtp/gstrtpjpegdepay.c:
/* allow frame dimensions > 2040, passed in SDP session or media attributes
* from gstrtspsrc.c (gst_rtspsrc_sdp_attributes_to_caps), or in caps */
if (!width)
width = rtpjpegdepay->media_width;
if (!height)
height = rtpjpegdepay->media_height;
Here you can see the GStreamer devs offer you a solution for this protocol limitation. When receiving images over 2040 pixels in either dimension you probably will have to add width and height information to the caps.
I know this question has been posted several times. But none of the solutions worked for me.
I am trying to play .wav files using gstreamer apis.
The below command plays .wav files (with or without the audioconvert):
gst-launch-1.0 filesrc location=sound.wav ! wavparse ! audioconvert ! alsasink
I have written a simple c++ code for the above command referring the GStreamer Hello world example. But it ends with "Internal data flow error". This is my code:
{
gst_init(NULL, NULL);
m_pipeline = gst_pipeline_new("audio-player");
m_fileSource = gst_element_factory_make("filesrc", "file-source");
m_parser = gst_element_factory_make("wavparse", "wav-parser");
m_sink = gst_element_factory_make("alsasink", "audio-output");
if (!m_pipeline || !m_fileSource || !m_parser || !m_sink) {
g_printerr ("One or more elements could not be created !! \n");
return false;
}
/* Set up the pipeline */
else {
/* set the input filename to the source element */
g_object_set (G_OBJECT (m_fileSource), "location", path.toLatin1().data(), NULL);
/* set a message handler on a bus */
GstBus *bus = gst_pipeline_get_bus (GST_PIPELINE (m_pipeline));
gst_bus_add_watch(bus, bus_call, this);
gst_object_unref(bus);
/* add all elements into the pipeline */
gst_bin_add_many (GST_BIN (m_pipeline), m_fileSource, m_parser, m_sink, NULL);
/* link the elements together */
gst_element_link (m_fileSource, m_parser);
g_signal_connect(m_parser, "pad-added", G_CALLBACK(on_pad_added), m_sink);
}
gst_element_set_state(m_pipeline, GST_STATE_READY);
gst_element_set_state(m_pipeline, GST_STATE_PAUSED);
}
on_pad_added(GstElement *src_element, GstPad *src_pad, gpointer data)
{
GstElement *sink_element = (GstElement *)data;
GstPad *sink_pad = gst_element_get_static_pad(sink_element, "sink");
gst_pad_link(src_pad, sink_pad);
gst_object_unref(sink_pad);
src_element = NULL;
}
I even tried the solutions suggested in this link; replaced "oggdemux" with "wavparse" and "vorbisdec" with "identity" from the Gstreamer Hello World example as suggested in this link. Error I received:
0:00:00.289443936 1624 0x1234e00 WARN basesrc gstbasesrc.c:3470:gst_base_src_start_complete:<file-source> pad not activated yet
0:00:00.290993573 1624 0x1740030 FIXME default gstutils.c:3643:gst_pad_create_stream_id_internal:<wav-parser:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:00.291883886 1624 0x1740030 WARN wavparse gstwavparse.c:1564:gst_wavparse_stream_headers:<wav-parser> Ignoring chunk bext
0:00:00.292198262 1624 0x1740030 WARN wavparse gstwavparse.c:1564:gst_wavparse_stream_headers:<wav-parser> Ignoring chunk junk
0:00:00.305086920 1624 0x1234e00 WARN basesrc gstbasesrc.c:3470:gst_base_src_start_complete:<file-source> pad not activated yet
0:00:00.306444838 1624 0x17400c0 FIXME default gstutils.c:3643:gst_pad_create_stream_id_internal:<wav-parser:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:00.307224214 1624 0x17400c0 WARN wavparse gstwavparse.c:1564:gst_wavparse_stream_headers:<wav-parser> Ignoring chunk bext
0:00:00.307636819 1624 0x17400c0 WARN wavparse gstwavparse.c:1564:gst_wavparse_stream_headers:<wav-parser> Ignoring chunk junk
0:00:00.526277506 1624 0x17400c0 WARN wavparse gstwavparse.c:2186:gst_wavparse_loop:<wav-parser> error: Internal data flow error.
0:00:00.526495475 1624 0x17400c0 WARN wavparse gstwavparse.c:2186:gst_wavparse_loop:<wav-parser> error: streaming task paused, reason not-linked (-1)
0:00:00.527296570 1624 0x1740030 WARN wavparse gstwavparse.c:2186:gst_wavparse_loop:<wav-parser> error: Internal data flow error.
0:00:00.527439278 1624 0x1740030 WARN wavparse gstwavparse.c:2186:gst_wavparse_loop:<wav-parser> error: streaming task paused, reason not-linked (-1)
ERROR: Internal data flow error.
ERROR: Internal data flow error.
What am I missing in the code?
You get a not-linked error, some element was left unlinked.
The element is wavparse. Your code is trying to wait for it to add a pad to link it, but wavparse has an 'always' source pad, meaning that it is there since the creation of the element so you can just link it directly just like you did with your filesrc.
I want to send raw audio data(PCM) to VLC player on RTP for playing the PCM using gstreamer.
Here is the command to send the PCM
gst-launch-1.0 -v filesrc location=/home/webos/pcm_data_dump ! audio/x-raw, rate=44100, channels=2, endianness=1234, format=S16LE, layout=interleaved, clock-rate=44100 ! audioconvert ! audioresample ! audio/x-raw, rate=44100, channels=2, format=S32LE, layout=interleaved ! audioconvert ! rtpL16pay pt=10 ! application/x-rtp, pt=10, encoding-name=L16, payload=10, clock-rate=44100, channels=2 ! udpsink host=192.168.0.2 port=5555
Here is the VLC option to receive the PCM
rtp://192.168.0.2:5555
VLC player can get the PCM from gstreamer, but it cannot play.
VLC shows the debug message like below.
Lastly "core debug: Buffering 0%" message is shown repeatedly in VLC debug message.
core debug: output 'f32l' 44100 Hz Stereo frame=1 samples/8 bytes
core debug: looking for audio volume module matching "any": 2 candidates
core debug: using audio volume module "float_mixer"
core debug: input 's16l' 44100 Hz Stereo frame=1 samples/4 bytes
core debug: looking for audio filter module matching "scaletempo": 14 candidates
scaletempo debug: format: 44100 rate, 2 nch, 4 bps, fl32
scaletempo debug: params: 30 stride, 0.200 overlap, 14 search
scaletempo debug: 1.000 scale, 1323.000 stride_in, 1323 stride_out, 1059
standing, 264 overlap, 617 search, 2204 queue, fl32 mode
core debug: using audio filter module "scaletempo"
core debug: conversion: 's16l'->'f32l' 44100 Hz->44100 Hz Stereo->Stereo
core debug: looking for audio converter module matching "any": 12 candidates
audio_format debug: s16l->f32l, bits per sample: 16->32
core debug: using audio converter module "audio_format"
core debug: conversion pipeline complete
core debug: conversion: 'f32l'->'f32l' 44100 Hz->44100 Hz Stereo->Stereo
core debug: Buffering 0%
core debug: conversion pipeline complete
core debug: looking for audio resampler module matching "any": 3 candidates
core debug: Buffering 0%
core debug: Buffering 0%
core debug: Buffering 0%
core debug: Buffering 0%
core debug: Buffering 0%
.......
And, the log below is shown once the gstreamer command to send PCM starts.
Normally, gstreamer is blocked with this message"New clock: GstSystemClock" when command starts.
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = audio/x- raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels= (int)2
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels= (int)2
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16BE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16BE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16BE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstRtpL16Pay:rtpl16pay0.GstPad:src: caps = application/x-rtp, media=(string)audio, clock-rate=(int)44100, encoding-name=(string)L16, encoding-params=(string)2, channels=(int)2, payload=(int)10, ssrc=(uint)2226113402, timestamp-offset=(uint)1744959080, seqnum-offset=(uint)62815
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)audio, clock-rate=(int)44100, encoding-name=(string)L16, encoding-params=(string)2, channels=(int)2, payload=(int)10, ssrc=(uint)2226113402, timestamp-offset=(uint)1744959080, seqnum-offset=(uint)62815
/GstPipeline:pipeline0/GstRtpL16Pay:rtpl16pay0.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16BE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstRtpL16Pay:rtpl16pay0: timestamp = 1744959080
/GstPipeline:pipeline0/GstRtpL16Pay:rtpl16pay0: seqnum = 62815
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.622147167
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
gst-launch-0.1 has no problem, only 1.0 has problem.
Is there any problem?
If I replace your filesrc with audiotestsrc, the example works for me. Still let me point out some room for improvement.
use audioparse instead of the first capsfilter
don't audioconvert twice.
Here is a simplified pipeline that works for me:
gst-launch-1.0 -v audiotestsrc ! audioresample ! audioconvert ! rtpL16pay pt=10 ! application/x-rtp, pt=10, encoding-name=L16, payload=10, clock-rate=44100, channels=2 ! udpsink host=localhost port=5555
I have seen this kind of pipeline-running commands in gstreamer:
e.g.,
gst-launch-1.0 videotestsrc ! video/x-raw, format=I420, framerate=25/1, width=640, height=360 ! xvimagesink
And I have read in some pages that video/x-raw, format=I420, framerate=25/1, width=640, height=360 specifies the media-type. But I am not able to understand what effect would it make - is it transforming the input to the specified framerate/format/width/height etc.. or is it just like specifying the input is in this framerate/width/ht already? And what effect it would have on the pipeline if it is just specifying the input is in this framerate etc... instead of transforming.
And is it really necessary or we can ignore it?
They call this video/x-raw, format=I420, framerate=25/1, width=640, height=360 thing "capabilities" or "caps" for short.
Sometimes it is required to specify explicitly what type of data flows between elements in a gstreamer pipe. And caps is the way to do this.
Why this is required sometimes? Because some gstreamer elements can accept (or produce)
several types of media. You can see this with gst-inspect command:
$ gst-inspect videotestsrc
...
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw-yuv
format: YUY2
color-matrix: { sdtv, hdtv }
chroma-site: { mpeg2, jpeg }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw-yuv
format: UYVY
color-matrix: { sdtv, hdtv }
chroma-site: { mpeg2, jpeg }
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
video/x-raw-yuv
format: YVYU
color-matrix: { sdtv, hdtv }
...
This means that videotestsrc has pad src which can produce outputs in various formats (YUY2, UYVY, YVYU, etc), sizes, framerates, etc.
The same is for xvimagesink:
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw-rgb
framerate: [ 0/1, 2147483647/1 ]
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
video/x-raw-yuv
framerate: [ 0/1, 2147483647/1 ]
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
It is able to view streams of data in various formats.
GStreamer uses the process called caps negotiation to decide which concrete data format to use between two elements. It tries to select "best fitting" format if no capabilities are provided by the user. That is why it is possible to drop capabilities from your pipeline and it will still work:
$ gst-launch-1.0 -v videotestsrc ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstXvImageSink:xvimagesink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
I added -v flag to see what caps gstreamser actually decided to use.
But sometimes there are cases when gstreamser fails to negotiate data format or you have different preferences.
E.g. in the case of pipeline that reads stream from socket it is impossible guess data format for certain and you need to provide correct capabilities.
You can see that specifying caps makes difference by executing these two pipelines:
$ gst-launch -v videotestsrc ! 'video/x-raw-yuv, width=600, height=600' ! xvimagesink
$ gst-launch -v videotestsrc ! 'video/x-raw-yuv, width=60, height=60' ! xvimagesink
It is important to understand that capabilities does not convert data but specify which format elements will produce or consume.