Related
I have an RTP streaming app which implements the following pipeline using the C API.
gst-launch-1.0 -v rtpbin name=rtpbin \
videotestsrc ! x264enc ! rtph264pay! rtpbin.send_rtp_sink_0 \
rtpbin.send_rtp_src_0 ! udpsink port=5002 host=127.0.0.1 \
rtpbin.send_rtcp_src_0 ! udpsink port=5003 host=127.0.0.1 sync=false async=false \
udpsrc port=5007 ! rtpbin.recv_rtcp_sink_0
I want to add header extensions to the RTP packet; therefore I created an extension using the new GstRTPHeaderExtension class introduced in GStreamer v1.20. I want to set the attributes of the extension (e.g. color space properties for the example below). AFAIU this should be done by providing those as caps to the payloader element. However, I can't figure out how I should provide these caps exactly. Do I need to use a capsfilter here or what is the right way? In the current state, I can send the RTP packets and see that the extension is added but can't set the attributes.
Related parts of the code are below:
#define URN_COLORSPACE "http://www.webrtc.org/experiments/rtp-hdrext/color-space"
const GstVideoColorimetry colorimetry = {
GST_VIDEO_COLOR_RANGE_0_255,
GST_VIDEO_COLOR_MATRIX_BT601,
GST_VIDEO_TRANSFER_BT2020_10,
GST_VIDEO_COLOR_PRIMARIES_BT2020};
const GstVideoChromaSite chroma_site = GST_VIDEO_CHROMA_SITE_MPEG2;
ext = gst_rtp_header_extension_create_from_uri(URN_COLORSPACE);
gst_rtp_header_extension_set_id(ext, 1);
g_signal_emit_by_name(videopay, "add-extension", ext);
// other element definitions, links..
videopay = gst_element_factory_make("rtph264pay", "videopay");
colorimetry_str = gst_video_colorimetry_to_string(&colorimetry);
// How to provide these caps to the payloader set the extension properties?
caps = gst_caps_new_simple("application/x-rtp",
"media", G_TYPE_STRING, "video",
"clock-rate", G_TYPE_INT, 90000,
"encoding-name", G_TYPE_STRING, "H264",
"colorimetry", G_TYPE_STRING, colorimetry_str,
"chroma-site", G_TYPE_STRING,
gst_video_chroma_to_string(chroma_site), NULL);
The caps should be provided to the sink of the RTP payloader element using a capsfilter element:
GstElement *capsfilt;
capsfilt = gst_element_factory_make("capsfilter", "capsfilter");
g_object_set(capsfilt, "caps", caps, NULL);
gst_element_link_many(videosrc, videoenc, capsfilt, videopay, NULL)
where videosrc, videoenc, videopay are the source, encoder and payloader elements, respectively.
Also, the caps should have a media type matching to the encoder element. E.g. video/x-h264 if the encoder element is an instance of x264enc.
The payloader will try to automatically enable the extension with the attributes set in the caps by passing the caps to the extension, if auto-header-extension is enabled (set to true by default).
In a gst-launch pipeline, the caps are passed automatically when the header extension is inserted after the payloader.
I am trying to convert a webcam on a raspberry pi to x264, but keep running into an error about an " Unsupported profile constrained-baseline".
GST_DEBUG=3 /home/pi/gst-rtsp-server/examples/test-launch "( v4l2src device=/dev/video0 ! videoconvert ! omxh264enc ! h264parse ! rtph264pay name=pay0 )"
stream ready at rtsp://127.0.0.1:8554/test
0:00:03.043939441 10314 0x75c08350 WARN v4l2src gstv4l2src.c:692:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:03.044207251 10314 0x7491de30 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<appsrc1:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:03.044211053 10314 0x7491de60 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<appsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:03.087901354 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:03.087992083 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:03.088080988 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:03.092800068 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:03.092867411 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:03.092942775 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:03.092983764 10314 0x7491de90 WARN GST_PADS gstpad.c:4226:gst_pad_peer_query:<videoscale0:src> could not send sticky events
0:00:04.001816134 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:04.001956914 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:04.002098111 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:04.037642275 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:04.037781284 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:04.037959303 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:04.038154562 10314 0x7491de90 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:04.038222947 10314 0x7491de90 WARN basesrc gstbasesrc.c:3055:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4)
0:00:04.038854870 10314 0x75c08350 WARN rtspmedia rtsp-media.c:1834:default_handle_message: 0x75c4e120: got error Internal data stream error. (gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:media-pipeline/GstBin:bin0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4))
0:00:04.039143357 10314 0x20b2490 WARN rtspmedia rtsp-media.c:2127:wait_preroll: failed to preroll pipeline
0:00:04.039219346 10314 0x20b2490 WARN rtspmedia rtsp-media.c:2384:gst_rtsp_media_prepare: failed to preroll pipeline
0:00:04.053306794 10314 0x7491de90 ERROR omxh264enc gstomxh264enc.c:706:gst_omx_h264_enc_set_format:<omxh264enc-omxh264enc0> Unsupported profile constrained-baseline
0:00:04.053442678 10314 0x7491de90 ERROR omxvideoenc gstomxvideoenc.c:2239:gst_omx_video_enc_set_format:<omxh264enc-omxh264enc0> Subclass failed to set the new format
0:00:04.053582781 10314 0x7491de90 WARN videoencoder gstvideoencoder.c:678:gst_video_encoder_setcaps:<omxh264enc-omxh264enc0> rejected caps video/x-raw, width=(int)1920, height=(int)1080, framerate=(fraction)5/1, pixel-aspect-ratio=(fraction)1/1, format=(string)I420, interlace-mode=(string)progressive
0:00:04.079457589 10314 0x20b2490 ERROR rtspclient rtsp-client.c:678:find_media: client 0x2010610: can't prepare media
0:00:04.081144660 10314 0x20b2490 ERROR rtspclient rtsp-client.c:2210:handle_describe_request: client 0x2010610: no media
There isn't anything in the docs for omxh264enc about profiles that I can find:
gst-inspect-1.0 omxh264enc
Factory Details:
Rank primary + 1 (257)
Long-name OpenMAX H.264 Video Encoder
Klass Codec/Encoder/Video
Description Encode H.264 video streams
Author Sebastian Dröge <sebastian.droege#collabora.co.uk>
Plugin Details:
Name omx
Description GStreamer OpenMAX Plug-ins
Filename /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstomx.so
Version 1.14.4
License LGPL
Source module gst-omx
Source release date 2018-10-02
Binary package GStreamer OpenMAX Plug-ins source release
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoEncoder
+----GstOMXVideoEnc
+----GstOMXH264Enc
+----GstOMXH264Enc-omxh264enc
Implemented Interfaces:
GstPreset
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
video/x-h264
width: [ 16, 4096 ]
height: [ 16, 4096 ]
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
width: [ 1, 2147483647 ]
height: [ 1, 2147483647 ]
framerate: [ 0/1, 2147483647/1 ]
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
name : The name of the object
flags: readable, writable
String. Default: "omxh264enc-omxh264enc0"
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: false
control-rate : Bitrate control method
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXVideoEncControlRate" Default: -1, "default"
(0): disable - Disable
(1): variable - Variable
(2): constant - Constant
(3): variable-skip-frames - Variable Skip Frames
(4): constant-skip-frames - Constant Skip Frames
(-1): default - Component Default
target-bitrate : Target bitrate in bits per second (0xffffffff=component default)
flags: readable, writable, changeable in NULL, READY, PAUSED or PLAYING state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-i-frames : Quantization parameter for I-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-p-frames : Quantization parameter for P-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
quant-b-frames : Quantization parameter for B-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
inline-header : Inline SPS/PPS header before IDR
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: true
periodicity-idr : Periodicity of IDR frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
periodicty-idr : Periodicity of IDR frames (0xffffffff=component default) DEPRECATED - only for backwards compat
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
interval-intraframes: Interval of coding Intra frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
b-frames : Number of B-frames between two consecutive I-frames (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
entropy-mode : Entropy mode for encoding process
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXH264EncEntropyMode" Default: -1, "default"
(0): CAVLC - CAVLC entropy mode
(1): CABAC - CABAC entropy mode
(-1): default - Component Default
constrained-intra-prediction: If enabled, prediction only uses residual data and decoded samples from neighbouring coding blocks coded using intra prediction modes
flags: readable, writable, changeable only in NULL or READY state
Boolean. Default: false
loop-filter-mode : Enable or disable the deblocking filter (0xffffffff=component default)
flags: readable, writable, changeable only in NULL or READY state
Enum "GstOMXH264EncLoopFilter" Default: -1, "default"
(0): enable - Enable deblocking filter
(1): disable - Disable deblocking filter
(2): disable-slice-boundary - Disables deblocking filter on slice boundary
(-1): default - Component Default
I am new to gstreamer so I could be reading this all wrong, would appreciate any ideas or feedback! Thanks!
You can change that through the SRC cap, try e.g.:
v4l2src device=/dev/video0 ! videoconvert ! omxh264enc ! video/x-h264,profile=baseline ! h264parse ! rtph264pay name=pay0
I have a simple GStreamer pipeline that I instantiate through OpenCV VideoWriter
LOG_INFO("Opening VP8 GStreamer writer object at "
<< host << ":" << port << " with " << horizontal_resolution
<< "x" << vertical_resolution << "p#" << fps << "fps"
<< "(color=" << is_color << ")");
const std::string filename{
" appsrc is-live=true ! videoconvert ! "
" vp8enc cpu-used=2 threads=2 min_quantizer=5 max_quantizer=5 deadline=100000 ! "
" rtpvp8pay ! application/x-rtp,media=video,encoding-name=VP8,payload=96 ! "
" udpsink host= " + host + " port= " + port
};
writer.open(
filename,
cv::CAP_GSTREAMER,
fps,
cv::Size{horizontal_resolution, vertical_resolution},
is_color
);
On my host machine it runs smoothly but when I cross-compile and run on the target I get the following:
Opening VP8 GStreamer writer object at 127.0.0.1:8004 with 160x120p#30fps(color=true)
0:00:00.003785000 3484 0xe7e0ac0 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element factory "appsrc"!
0:00:00.003901750 3484 0xe7e0ac0 ERROR GST_PIPELINE grammar.y:816:priv_gst_parse_yyparse: no element "appsrc"
0:00:00.003951125 3484 0xe7e0ac0 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element factory "videoconvert"!
0:00:00.003972875 3484 0xe7e0ac0 ERROR GST_PIPELINE grammar.y:816:priv_gst_parse_yyparse: no element "videoconvert"
0:00:00.003998375 3484 0xe7e0ac0 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no source [sink=#(nil)]
0:00:00.004042250 3484 0xe7e0ac0 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element factory "vp8enc"!
0:00:00.004070875 3484 0xe7e0ac0 ERROR GST_PIPELINE grammar.y:816:priv_gst_parse_yyparse: no element "vp8enc"
0:00:00.004105500 3484 0xe7e0ac0 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no source [sink=#(nil)]
0:00:00.004135750 3484 0xe7e0ac0 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element factory "rtpvp8pay"!
0:00:00.004160250 3484 0xe7e0ac0 ERROR GST_PIPELINE grammar.y:816:priv_gst_parse_yyparse: no element "rtpvp8pay"
0:00:00.004186875 3484 0xe7e0ac0 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no source [sink=#(nil)]
0:00:00.004309875 3484 0xe7e0ac0 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element factory "udpsink"!
0:00:00.004336375 3484 0xe7e0ac0 ERROR GST_PIPELINE grammar.y:816:priv_gst_parse_yyparse: no element "udpsink"
0:00:00.004364125 3484 0xe7e0ac0 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no source [sink=#(nil)]
(EXE:3484): GStreamer-CRITICAL **: gst_debug_log_valist: assertion 'category != NULL' failed
** (EXE:3484): CRITICAL **: gst_app_src_end_of_stream: assertion 'GST_IS_APP_SRC (appsrc)' failed
My Yocto configuration file is the following, target platform is a i.MX8M Mini.
require conf/distro/include/yocto-uninative.inc
MACHINE ??= "e0219"
UBOOT_CONFIG ??= "sd"
UBOOT_CONFIG[sd] ??= "e0219_config"
#MACHINE ??= 'imx8mmevk'
DISTRO ?= 'fsl-imx-xwayland'
PACKAGE_CLASSES ?= "package_rpm"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks"
USER_CLASSES ?= "buildstats image-mklibs image-prelink"
PATCHRESOLVE = "noop"
BB_DISKMON_DIRS ??= "\
STOPTASKS,${TMPDIR},1G,100K \
STOPTASKS,${DL_DIR},1G,100K \
STOPTASKS,${SSTATE_DIR},1G,100K \
STOPTASKS,/tmp,100M,100K \
ABORT,${TMPDIR},100M,1K \
ABORT,${DL_DIR},100M,1K \
ABORT,${SSTATE_DIR},100M,1K \
ABORT,/tmp,10M,1K"
PACKAGECONFIG_append_pn-qemu-native = " sdl"
PACKAGECONFIG_append_pn-nativesdk-qemu = " sdl"
CONF_VERSION = "1"
DL_DIR ?= "${BSPDIR}/downloads/"
CORE_IMAGE_EXTRA_INSTALL += " net-tools iputils dhcpcd"
CORE_IMAGE_EXTRA_INSTALL += " openssh nano python3 ethtool i2c-tools strace spitools"
CORE_IMAGE_EXTRA_INSTALL += " tcpdump socat opencv libmicrohttpd libsrtp curl boost"
CORE_IMAGE_EXTRA_INSTALL += " nmap python-djangorestframework net-snmp"
CORE_IMAGE_EXTRA_INSTALL += " openmp llvm"
ACCEPT_FSL_EULA = "1"
If I understand this correctly the opencv recipe should require gstreamer and some of its plugins but it seems to me that, after the build, GStreamer can't find the needed plugins to run the pipeline. Here's the output of gst-inspect:
$ gst-inspect-1.0
coreelements: capsfilter: CapsFilter
coreelements: concat: Concat
coreelements: dataurisrc: data: URI source element
coreelements: downloadbuffer: DownloadBuffer
coreelements: fakesrc: Fake Source
coreelements: fakesink: Fake Sink
coreelements: fdsrc: Filedescriptor Source
coreelements: fdsink: Filedescriptor Sink
coreelements: filesrc: File Source
coreelements: funnel: Funnel pipe fitting
coreelements: identity: Identity
coreelements: input-selector: Input selector
coreelements: output-selector: Output selector
coreelements: queue: Queue
coreelements: queue2: Queue 2
coreelements: filesink: File Sink
coreelements: tee: Tee pipe fitting
coreelements: typefind: TypeFind
coreelements: multiqueue: MultiQueue
coreelements: valve: Valve element
coreelements: streamiddemux: Streamid Demux
staticelements: bin: Generic bin
staticelements: pipeline: Pipeline object
Total count: 2 plugins, 23 features
How can I solve this?
Update 2 using #Subrata suggestion
I added the packages suggested by #Subrata to my local.conf like so:
CORE_IMAGE_EXTRA_INSTALL += " gstreamer1.0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav"
And now can see 179 plugins through gst-inspect.
Launching my application I'm still missing an element, but it's a step in the right direction.
Opening VP8 GStreamer writer object at 127.0.0.1:8004 with 160x120p#30fps(color=true)
0:00:00.053317750 3520 0x16290b00 WARN GST_ELEMENT_FACTORY gstelementfactory.c:456:gst_element_factory_make: no such element factory "vp8enc"!
0:00:00.053537875 3520 0x16290b00 ERROR GST_PIPELINE grammar.y:816:priv_gst_parse_yyparse: no element "vp8enc"
0:00:00.053606125 3520 0x16290b00 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no sink [source=#0x16293290]
0:00:00.057937500 3520 0x16290b00 ERROR GST_PIPELINE grammar.y:901:priv_gst_parse_yyparse: link has no source [sink=#0x162a0250]
0:00:00.081852625 3520 0x1629f720 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<appsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Try reinstalling the gstreamer packages like gstreamer1.0-plugins-good, gstreamer1.0-plugins-bad, gstreamer1.0-plugins-ugly and gstreamer1.0-libav.
Following packages will provide: gstreamer1.0-audiosink, gstreamer1.0-audiosource, gstreamer1.0-videosink, gstreamer1.0-videosource, gstreamer1.0-visualization
libav have elements for encoding 40+ formats (MPEG, DivX, MPEG4, AC3, DV, ...), decoding elements for decoding 90+ formats (AVI, MPEG, OGG, Matroska, ASF, ...).
To complete the answer of #Subrata:
You need to install the package corresponding to the elements requiered by gstreamer.
I don't know which Yocto version you are using but you can see which gstreamer elements are installed by default on gstreamer. In the meta/recipes-multimedia/gstreamer/gstreamer1.0-plugins-good.inc file:
PACKAGECONFIG ??= " \
${GSTREAMER_ORC} \
${#bb.utils.filter('DISTRO_FEATURES', 'pulseaudio x11', d)} \
cairo flac gdk-pixbuf gudev jpeg libpng soup speex taglib v4l2 \
"
[...]
PACKAGECONFIG[vpx] = "--enable-vpx,--disable-vpx,libvpx"
Now you are required vp8enc, included in vpx library, but, as you can see, the PACKAGECONFIG variable does not contains this option.
You need to add a .bbappend file with this option added to use this plugin, in your meta :your-meta/repices-multimedia/gstreamer/gstreamer1.0-plugins-good_%.bbappend
PACKAGECONFIG_append=" vpx"
If you plan to write a recipe which needs this vpx element, you can add it in the RDEPENDS:
DEPENDS=" gstreamer1.0 gstreamer1.0-plugins-good"
RDEPENDS_${PN}=" gstreamer1.0 \
gstreamer1.0-plugins-good-vpx "
How can i write a pipeline that streams videotestsrc h265 encoded on RTSP and another that playback the former?
As far as i understand, this should be a valid server
gst-launch-1.0 -v videotestsrc ! video/x-raw,width=1280,height=720 ! x265enc ! rtph265pay ! udpsink host=127.0.0.1 port=5000
The output is
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive"
/GstPipeline:pipeline0/GstX265Enc:x265enc0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive"
Redistribute latency...
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive"
/GstPipeline:pipeline0/GstX265Enc:x265enc0.GstPad:src: caps = "video/x-h265\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ level\=\(string\)3.1\,\ tier\=\(string\)main\,\ profile\=\(string\)main\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstRtpH265Pay:rtph265pay0.GstPad:sink: caps = "video/x-h265\,\ stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\ level\=\(string\)3.1\,\ tier\=\(string\)main\,\ profile\=\(string\)main\,\ width\=\(int\)1280\,\ height\=\(int\)720\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstRtpH265Pay:rtph265pay0.GstPad:src: caps = "application/x-rtp\,\ media\=\(string\)video\,\ payload\=\(int\)96\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)H265\,\ ssrc\=\(uint\)2573237941\,\ timestamp-offset\=\(uint\)1713951204\,\ seqnum-offset\=\(uint\)27727\,\ a-framerate\=\(string\)30"
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = "application/x-rtp\,\ media\=\(string\)video\,\ payload\=\(int\)96\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)H265\,\ ssrc\=\(uint\)2573237941\,\ timestamp-offset\=\(uint\)1713951204\,\ seqnum-offset\=\(uint\)27727\,\ a-framerate\=\(string\)30"
/GstPipeline:pipeline0/GstRtpH265Pay:rtph265pay0.GstPad:src: caps = "application/x-rtp\,\ media\=\(string\)video\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)H265\,\ sprop-parameter-sets\=\(string\)\"QAEMAf//AWAAAAMAkAAAAwAAAwBdlZgJAA\\\=\\\=\\\,QgEBAWAAAAMAkAAAAwAAAwBdoAKAgC0WWVmkkyuAQAAAAwBAAAAHggA\\\=\\\,RAHBcrRiQAA\\\=\"\,\ payload\=\(int\)96\,\ seqnum-offset\=\(uint\)27727\,\ timestamp-offset\=\(uint\)1713951204\,\ ssrc\=\(uint\)2573237941\,\ a-framerate\=\(string\)30"
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = "application/x-rtp\,\ media\=\(string\)video\,\ clock-rate\=\(int\)90000\,\ encoding-name\=\(string\)H265\,\ sprop-parameter-sets\=\(string\)\"QAEMAf//AWAAAAMAkAAAAwAAAwBdlZgJAA\\\=\\\=\\\,QgEBAWAAAAMAkAAAAwAAAwBdoAKAgC0WWVmkkyuAQAAAAwBAAAAHggA\\\=\\\,RAHBcrRiQAA\\\=\"\,\ payload\=\(int\)96\,\ seqnum-offset\=\(uint\)27727\,\ timestamp-offset\=\(uint\)1713951204\,\ ssrc\=\(uint\)2573237941\,\ a-framerate\=\(string\)30"
/GstPipeline:pipeline0/GstRtpH265Pay:rtph265pay0: timestamp = 1713951204
/GstPipeline:pipeline0/GstRtpH265Pay:rtph265pay0: seqnum = 27727
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
So i tried
gst-launch-1.0 udpsrc uri=udp://127.0.0.1:5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H265, payload=(int)96, seqnum-offset=(uint)27727, timestamp-offset=(uint)1713951204, ssrc=(uint)2573237941, a-framerate=(string)30" ! rtph265depay ! vaapidecode ! vaapipostproc ! vaapisink
but with no luck
libva info: VA-API version 0.39.3
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'pipeline0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)NULL;
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstVaapiDecode:vaapidecode0: No valid frames decoded before end of stream
Additional debug info:
gstvideodecoder.c(1167): gst_video_decoder_sink_event_default (): /GstPipeline:pipeline0/GstVaapiDecode:vaapidecode0:
no valid frames found
Execution ended after 0:00:00.025823038
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Try adding the parser before the decode like: gst-launch-1.0 udpsrc uri=udp://127.0.0.1:5000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H265, payload=(int)96, seqnum-offset=(uint)27727, timestamp-offset=(uint)1713951204, ssrc=(uint)2573237941, a-framerate=(string)30" ! rtph265depay ! h265parse ! vaapidecode ! vaapipostproc ! vaapisink
Notice the h265parse element between the depay and decode
I want to send raw audio data(PCM) to VLC player on RTP for playing the PCM using gstreamer.
Here is the command to send the PCM
gst-launch-1.0 -v filesrc location=/home/webos/pcm_data_dump ! audio/x-raw, rate=44100, channels=2, endianness=1234, format=S16LE, layout=interleaved, clock-rate=44100 ! audioconvert ! audioresample ! audio/x-raw, rate=44100, channels=2, format=S32LE, layout=interleaved ! audioconvert ! rtpL16pay pt=10 ! application/x-rtp, pt=10, encoding-name=L16, payload=10, clock-rate=44100, channels=2 ! udpsink host=192.168.0.2 port=5555
Here is the VLC option to receive the PCM
rtp://192.168.0.2:5555
VLC player can get the PCM from gstreamer, but it cannot play.
VLC shows the debug message like below.
Lastly "core debug: Buffering 0%" message is shown repeatedly in VLC debug message.
core debug: output 'f32l' 44100 Hz Stereo frame=1 samples/8 bytes
core debug: looking for audio volume module matching "any": 2 candidates
core debug: using audio volume module "float_mixer"
core debug: input 's16l' 44100 Hz Stereo frame=1 samples/4 bytes
core debug: looking for audio filter module matching "scaletempo": 14 candidates
scaletempo debug: format: 44100 rate, 2 nch, 4 bps, fl32
scaletempo debug: params: 30 stride, 0.200 overlap, 14 search
scaletempo debug: 1.000 scale, 1323.000 stride_in, 1323 stride_out, 1059
standing, 264 overlap, 617 search, 2204 queue, fl32 mode
core debug: using audio filter module "scaletempo"
core debug: conversion: 's16l'->'f32l' 44100 Hz->44100 Hz Stereo->Stereo
core debug: looking for audio converter module matching "any": 12 candidates
audio_format debug: s16l->f32l, bits per sample: 16->32
core debug: using audio converter module "audio_format"
core debug: conversion pipeline complete
core debug: conversion: 'f32l'->'f32l' 44100 Hz->44100 Hz Stereo->Stereo
core debug: Buffering 0%
core debug: conversion pipeline complete
core debug: looking for audio resampler module matching "any": 3 candidates
core debug: Buffering 0%
core debug: Buffering 0%
core debug: Buffering 0%
core debug: Buffering 0%
core debug: Buffering 0%
.......
And, the log below is shown once the gstreamer command to send PCM starts.
Normally, gstreamer is blocked with this message"New clock: GstSystemClock" when command starts.
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = audio/x- raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels= (int)2
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels= (int)2
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:src: caps = audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16BE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16BE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16BE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstRtpL16Pay:rtpl16pay0.GstPad:src: caps = application/x-rtp, media=(string)audio, clock-rate=(int)44100, encoding-name=(string)L16, encoding-params=(string)2, channels=(int)2, payload=(int)10, ssrc=(uint)2226113402, timestamp-offset=(uint)1744959080, seqnum-offset=(uint)62815
/GstPipeline:pipeline0/GstUDPSink:udpsink0.GstPad:sink: caps = application/x-rtp, media=(string)audio, clock-rate=(int)44100, encoding-name=(string)L16, encoding-params=(string)2, channels=(int)2, payload=(int)10, ssrc=(uint)2226113402, timestamp-offset=(uint)1744959080, seqnum-offset=(uint)62815
/GstPipeline:pipeline0/GstRtpL16Pay:rtpl16pay0.GstPad:sink: caps = audio/x-raw, layout=(string)interleaved, rate=(int)44100, format=(string)S16BE, channels=(int)2, channel-mask=(bitmask)0x0000000000000003
/GstPipeline:pipeline0/GstAudioConvert:audioconvert0.GstPad:sink: caps = audio/x-raw, format=(string)S32LE, layout=(string)interleaved, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstRtpL16Pay:rtpl16pay0: timestamp = 1744959080
/GstPipeline:pipeline0/GstRtpL16Pay:rtpl16pay0: seqnum = 62815
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.622147167
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
gst-launch-0.1 has no problem, only 1.0 has problem.
Is there any problem?
If I replace your filesrc with audiotestsrc, the example works for me. Still let me point out some room for improvement.
use audioparse instead of the first capsfilter
don't audioconvert twice.
Here is a simplified pipeline that works for me:
gst-launch-1.0 -v audiotestsrc ! audioresample ! audioconvert ! rtpL16pay pt=10 ! application/x-rtp, pt=10, encoding-name=L16, payload=10, clock-rate=44100, channels=2 ! udpsink host=localhost port=5555