How to add audio to a h264 video stream using gstreamer - gstreamer

I can successfully stream HD video using following pipelines:
streame server:
gst-launch-1.0 filesrc location="Gravity.2013.720p.BluRay.x264.YIFY.mp4" ! decodebin ! x264enc ! rtph264pay pt=96 ssrc=0 timestamp-offset=0 seqnum-offset=0 pt=96 ! gdppay ! tcpclientsink host=192.168.1.93 port=5000
client:
gst-launch-1.0 tcpserversrc host=192.168.1.93 port=5000 ! gdpdepay ! rtph264depay ! decodebin ! autovideosink
I want to add the audio stream too.
I guess it is possible to use a different port and tcpserver/tcpclient combination to stream audio parallel to video. But I am not certain, how gstreamer would synchronize two streams properly to play the movie in the client end. Apart from this method, are there any other methods? such as muxing two streams before and demuxing it in client end?

Related

How can I fix the missing duration of a video file from a RTSP stream on Gstreamer?

I'm using the following pipeline to listen to a RTSP stream and save a video file:
gst-launch-1.0 -q rtspsrc location=rtsp://<ip>:<port>/video ! decodebin ! autovideoconvert ! x264enc pass=5 quantizer=25 speed-preset=6 ! h264parse ! matroskamux ! filesink location=<filename>
But even though I can see the files generated, they lack the duration of the video when playing on VLC.
I can fix it by passing it through ffmpeg later, but I want to generate the video from gstreamer already completely valid. How can I fix this pipeline?
gst-launch-1.0 -e rtspsrc location=rtsp://<ip>:<port>/video ! decodebin ! videoconvert ! x264enc ! mp4mux ! filesink location="filename.mp4"
This will create a video with duration shown correctly

Transform RTP into RTSP with gstreamer

I have a third party application that reads data from a thermal camera and generates a RTP stream to a given UDP source. I am trying to wrap this RTP into a RTSP stream but I am running into problems...
The third party application basically runs gstreamer with this command
appsrc format=GST_FORMAT_TIME is-live=true block=true caps=video/x-raw,width=640,height=480,format=GRAY8,clock-rate=90000,framerate=10/1 ! openjpegenc ! rtpj2kpay ! udpsink host=127.0.0.1 port=3000
Using the command below I can visualize the stream on my machine
gst-launch-1.0 udpsrc port=3000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)JPEG2000, sampling=(string)GRAYSCALE, width=(int)640, height=(int)480, payload=(int)96" ! queue ! rtpj2kdepay ! openjpegdec ! videoconvert ! xvimagesink
However when trying to use the default RTP to RTSP application example using https://github.com/freedesktop/gstreamer-gst-rtsp-server/blob/master/examples/test-launch.c to just forward it with a RTSP container the connection fails with VLC. Command below:
./rtp-src-to-rtsp '( udpsrc port=3000 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)JPEG2000, sampling=(string)GRAYSCALE, width=(int)640, height=(int)480, payload=(int)96" ! queue ! rtpj2kdepay ! rtpj2kpay )'
Any light on what I am doing wrong? VLC gives only a non-descriptive error
live555 error: Nothing to play for rtsp://{IP}:{PORT}/test
It might be a lack of support of J2K in VLC (I'm using revision 3.0.8-0).
Simulating your source with:
gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,framerate=10/1,format=GRAY8 ! openjpegenc ! rtpj2kpay ! udpsink host=127.0.0.1 port=3000
and relaying as RSTP with:
./test-launch "udpsrc port=3000 auto-multicast=0 ! application/x-rtp,encoding-name=JPEG2000,sampling=GRAYSCALE ! queue ! rtpj2kdepay ! image/x-jpc ! jpeg2000parse ! rtpj2kpay name=pay0 "
works on Linux with X using:
gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/test ! application/x-rtp, encoding-name=JPEG2000,sampling=GRAYSCALE ! rtpj2kdepay ! jpeg2000parse ! openjpegdec ! videoconvert ! xvimagesink -v
Though, I haven't been able to receive with VLC, nor able to make a correct J2K/RTP SDP for VLC nor ffmpeg. Someone better skilled may further advise.

GStreamer: get video from RTP, setting the format automatically

Let's say I've got a simple RTP video server:
ximagesrc ! video/x-raw,framerate=30/1 ! videoscale ! videoconvert ! x264enc tune=zerolatency bitrate=500 speed-preset=slow ! rtph264pay ! udpsink host=127.0.0.1 port=5000
And I'm receiving it just fine:
udpsrc port=5000 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! decodebin ! videoconvert ! autovideosink
But if my server was sending H265 (or any other format), I'd have to use another pipeline. Considering there are a ton of possible formats, that is definitely something I'd like to avoid. Is there any way to get my video decoded from any format?
No. RTP streams are supposed to be initiated by SDP and RTSP control protocols which will tell you what kind of stream you are dealing with. If you want self describing media, perhaps try looking at MPEG-TS instead.

How to publish RTSP stream from ip camera with gstreamer?

I have a rtsp-simple-server running on Debian and
I try to publish RTSP from my ip camera (h264 + pcm ulaw) to RTSP server with gstreamer.
I tried this command:
gst-launch-1.0 rtspsrc location=rtsp://192.168.1.1:554/11 !
rtph264depay ! rtspclientsink location=rtsp://server.com:8554/camstream1 protocols=tcp
It works, but it streams video only. The following command streams audio only:
gst-launch-1.0 rtspsrc location=rtsp://192.168.1.1:554/11 !
rtppcmudepay ! rtspclientsink location=rtsp://server.com:8554/camstream1 protocols=tcp
How can I stream video + audio?
Try this (or something like this) :
gst-launch-1.0 rtspsrc location=rtsp://192.168.1.1:554/11 name=mysrc !
rtph264depay ! rtspclientsink location=rtsp://server.com:8554/camstream1 protocols=tcp mysrc. ! rtppcmudepay ! rtspclientsink location=rtsp://server.com:8554/camstream1 protocols=tcp
OR you might have to MUX the audio and video first, before sending to rtspclientsink (this part is beyond me)
Try this first on a PC to see if the first part works
gst-launch-1.0 rtspsrc location=rtsp://192.168.1.1:554/11 name=mysrc !
rtph264depay ! autovideosink mysrc. ! rtppcmudepay ! autoaudiosink

Why is encoding required for video transfer? (gstreamer)

The video I transferred is already encoded. Why do I encode again when transferring?
example: gst-launch-1.0 -v filesrc location=123.mp4 ! decodebin ! x264enc ! rtph264pay ! udpsink host=192.168.10.186 port=9001
Just send the video without encoding. Can I view it on the other side?
for example:
server: gst-launch-1.0 -v filesrc location =123.mp4 ! udpsink host=192.168.10.186 port=9001
123.mp4 encoded h265
client: gst-launch-1.0 udpsrc port=9001 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H265, payload=(int)96" ! rtph265depay ! h265parse ! nvh265dec ! autovideosink
best regards
Okay, with the clarification that your input is assumed to be an MPEG4 file which contains an H.265: yes, then it's possible (if this is assumption doesn't hold, then this will not work).
The following should do the trick:
gst-launch-1.0 filesrc location=123.mp4 ! qtdemux ! h265parse config-interval=-1 ! rtph265pay ! udpsink host=192.168.10.186 port=9001
Explanation:
the qtdemux will demux the MPEG4 container into the contained video/audio/subtitile streams (if there's more than one stream inside that container, you'll need to link to it multple times, or GStreamer will error out)
the h265parse config-interval=1 will make sure that your contains the correct SPS/PPS parameter sets. If the stream inside your original input file is not an H265 video stream, this will fail to link.
the rth265pay will translate this into RTP packets.
... which the udpsink can then send over the specified socket
P.S.: you might also be interested in the rtpsink (which used to live out-of-tree, but is now included in the latest master of GStreamer)
P.P.S.: you should use an even port to send an RTP stream