Gstreamer H264 UDP -> WebRTC Restreaming - c++

I have a Ricoh THETA Z1 360 degrees camera that outputs a 4K 360 stream. I'm using their own libuvc-theta-sample for retrieving the video stream and getting it into Gstreamer. I've used the following pipeline to sink the video stream to a different machine on my network that runs a Gstreamer application that restreams udpsrc into a webrtcbin:
appsrc name=ap ! queue max-size-buffers=1 leaky=downstream ! h264parse ! rtph264pay ! udpsink host=192.168.1.137 port=1935 qos=false sync=false
On my receiving end this works perfectly fine:
As you can see I'm using a WebRTC bin that sinks the video to a WebRTC browser client.
This is all working as expected as long as I'm on localhost. If I try to reach the video stream from a different machine in my network, it does the ICE and SDP offers perfectly, the WebRTC connection is established, but no video is shown. If I replace my pipeline with a pipeline that depays, decodes, and re-encodes the H264 stream, it works perfectly fine. Obviously, this is not what I want because I'm losing (significant) video quality and it takes time to re-encode.
Something that might help discover the issue is the following log that gets spammed in my console. This log doesn't appear when I'm viewing the stream from localhost, but it does appear when trying to view the stream from a different machine within my network.
0:00:07.248383000 22700 00000139F5F3E340 INFO h264parse gsth264parse.c:3741:gst_h264_parse_src_event: received upstream force-key-unit event, seqnum 253 running_time 99:99:99.999999999 all_headers 0 count 0
0:00:07.559580000 22700 00000139F5F3E340 INFO h264parse gsth264parse.c:3741:gst_h264_parse_src_event: received upstream force-key-unit event, seqnum 254 running_time 99:99:99.999999999 all_headers 0 count 0
0:00:07.854883000 22700 00000139F5F3E340 INFO h264parse gsth264parse.c:3741:gst_h264_parse_src_event: received upstream force-key-unit event, seqnum 255 running_time 99:99:99.999999999 all_headers 0 count 0
0:00:08.160170000 22700 00000139F5F3E340 INFO h264parse gsth264parse.c:3741:gst_h264_parse_src_event: received upstream force-key-unit event, seqnum 256 running_time 99:99:99.999999999 all_headers 0 count 0
0:00:08.414529000 22700 00000139F5F3E340 INFO h264parse gsth264parse.c:3741:gst_h264_parse_src_event: received upstream force-key-unit event, seqnum 257 running_time 99:99:99.999999999 all_headers 0 count 0
0:00:08.715480000 22700 00000139F5F3E340 INFO h264parse gsth264parse.c:3741:gst_h264_parse_src_event: received upstream force-key-unit event, seqnum 258 running_time 99:99:99.999999999 all_headers 0 count 0
0:00:09.013550000 22700 00000139F5F3E340 INFO h264parse gsth264parse.c:3741:gst_h264_parse_src_event: received upstream force-key-unit event, seqnum 259 running_time 99:99:99.999999999 all_headers 0 count 0
I've thought about NAT traversal issues with STUN and TURN, but it wouldn't make sense as my Google Chrome WebRTC internals page shows that the WebRTC connection is successfully established. Besides, it does work when re-encoding the stream.
I'm failing to understand why it is working on localhost, but not on other machines within my network. Perhaps someone can shed some light on why it's not working.

Disable firewall on streaming server and client machine then test streaming works or not. If works then you can add your firewall rules for WebRTC and UDP ports .
2)Try streaming with creating direct tunnel using ngrok or other free service with direct IP addresses. Then go with STUN and TURN setup.
Also try following pipeline:
webrtcbin name=sendrecv bundle-policy=max-bundle
udpsrc port=7001 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay !
h264parse ! rtph264pay config-interval=-1 !
queue ! application/x-rtp,media=video,encoding-name=H264,payload=96 ! rtpjitterbuffer ! sendrecv

Not sure at all for your case, but you may try adding rtpjitterbuffer with a few frames latency in the remote case:
... ! udpsrc port=1935 ! application/x-rtp,encoding-name=H264 ! rtpjitterbuffer latency=500 ! rtph264depay ! ...

Related

How can I choose which stream to display using gst-launch-1.0?

I have a parrot drone that has 4 cameras being streamed with RTSP. The 4 cameras are being streamed by the same location (rtsp://192.168.53.1:554/live). Is it possible to choose which stream I want to be displayed using gst-launch-1.0?
When I run:
gst-launch-1.0 rtspsrc location=rtsp://192.168.53.1:554/live latency=0 ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,width=1920,height=1080 ! autovideosink
I can see one of the streams, the problem is that the stream displayed is a random one from the four.
The output from the gst-launch-1.0 is the following:
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.53.1:554/live
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (request) SETUP stream 2
Progress: (request) SETUP stream 3
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
The following images are frames that I extracted from 2 of the 4 streams where, one time the displayed stream was the one from the main camera and another time the one from stereo.
Frame of the main camera stream:
Frame of one stereo camera stream:

GStreamer and RTSP stream

I'm trying to view an rtsp stream trough gstreamer but when i run this command:
gst-launch-1.0 rtspsrc location=rtsp://admin:#192.168.1.27:554/ch0_0.264 ! videoconvert ! video/x-raw ! autovideosink
it blocks here and doesn't display anything.
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Pipeline is PREROLLED ...
Prerolled, waiting for progress to finish...
Progress: (connect) Connecting to rtsp://admin:#192.168.1.27:554/ch0_0.264
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Redistribute latency...
I'm new to gstreamer. Can someone hepl me with that?
Thanks in advice.
Using decodebin I found out that the H264 decoder was missing so I solved it by installing the package gst-libav

Gstreamer rtspsrc+decodebin vs uridecodebin

Consider the following gstreamer commands (I'm trying them in Windows):
gst-launch-1.0 rtspsrc location=rtsp://<path-to-stream> ! decodebin ! autovideosink
gst-launch-1.0 uridecodebin uri=rtsp://<path-to-stream> ! autovideosink
Both commands should play video on auto created window. But in my case first one can start playing only about in 1 of 5 attempts. Whereas second one works every time.
In case of failure first command prints out the following messages:
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://<path-to-stream-here>
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0: Delayed linking failed.
Additional debug info:
./grammar.y(506): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0:
failed delayed linking some pad of GstDecodeBin named decodebin0 to some pad of
GstAutoVideoSink named autovideosink0
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstRTSPSrc:rtsp
src0/GstUDPSrc:udpsrc0:
streaming task paused, reason not-linked (-1)
Execution ended after 0:00:02.289918784
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
What's the main difference between those commands from user's point of view?
Is it possible to make first command more reliable (i.e. works without errors like second one)?
Has you set other properoties, such as caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264".
If the parameters don't match, it will report an error!

How to stream a video file from a amazon server using gstreamer to the internet

I have found the way to stream the videotestsrc element to the internet by calling these functions in the test-video.c file of gst-rtsp-server examples directory:
gst_rtsp_server_set_address(server,"10.xxx.xx.xxx");
gst_rtsp_server_set_service(server,"8555");
and unblocking the ports in the aws console.
But now replacing videotestsrc with the following in test-video.c, like below:
gst_rtsp_media_factory_set_launch (factory, "( "
"filesrc location=/home/ubuntu/DELTA.mpg ! mpeg2dec ! x264enc ! rtph264pay name=pay0 pt=96 "
")");
I am not able to get the stream visible on the vlc player.
Trying the below command produces following output:
ubuntu#ip-xx-xxx-xx-xxx:~/gst-rtsp-server-1.2.3/examples$ /usr/local/bin/gst-launch-1.0 playbin uri=rtsp://10.xxx.xx.xxx:8555/test
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://10.185.10.118:8555/test
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
(lt-test-video1:9027): GLib-CRITICAL **: unblock_source: assertion `!SOURCE_DEST
ROYED (source)' failed
WARNING: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstRTSP
Src:source: Could not read from resource.
Additional debug info:
gstrtspsrc.c(4367): gst_rtspsrc_reconnect (): /GstPlayBin:playbin0/GstURIDecodeB
in:uridecodebin0/GstRTSPSrc:source:
Could not receive any UDP packets for 5.0000 seconds, maybe your firewall is blo
cking it. Retrying using a TCP connection.
♥handling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:22.473442001
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
ubuntu#ip-10-xxx-xx-xxx:~/gst-rtsp-server-1.2.3/examples$
What am I doing wrong here ?

Gstreamer Pipeline Jackaudiosrc output input

I have the following problem, jackaudiosrc connects automatically to the first jack ports, with my capture_1 and capture_2.
I set the option connect=O, but this is not that what i think I want. What I want is that when I start script jackaudiosrc automaticly connect to another port, original script here:
gst-launch v4l2src device=/dev/video0 ! video/x-raw-yuv,width=320,height=240 ! queue ! videorate ! ffmpegcolorspace ! tee name=tscreen ! queue ! autovideosink tscreen. ! queue ! theoraenc quality=16 ! queue ! oggmux name=mux jackaudiosrc connect=0 ! audio/x-raw-float,channels=2 ! queue ! audioconvert ! vorbisenc quality=0.2 ! queue ! mux. mux. ! queue ! shout2send ip=xxx port=xxx mount=test.ogg password=xxxxx name= description= genre= url=
I have a program aj-snapshot, that makes an xml file, in this file is the connect I use, here is:
jack
client name=idjc_default
port name=str_out_l
connection port=idjc_default:output_in_l
connection port=camstream1.py:in_jackaudiosrc0_1
port
port name=str_out_r
connection port=idjc_default:output_in_r
connection port=camstream1.py:in_jackaudiosrc0_2
port
client
jack
My Question is, how can I add connect to this ports in cmd gst-launch jackaudiosrc to automaticly connect to this ports when i start my script.
I don't think it is possible to do this from gst-launch. You could write a small application that use gst_parse_launch and talks to jack to setup the connections.