I am trying to stream an mkv file over udpsink as an rtp payload, but when the I receive the packets it drops almost every other frame. The stream just freezes for a couple of seconds and then shows the time synced frame. I have a timestamp on the video file and it jumps a the same amount of time it freezes for. The setup is gstreamer is run on a Raspberry Pi 4 and I view the stream using vlc on another computer.
I have the payload working so it sends over the udpsink successfully using:
gst-launch-1.0 filesrc location=file.mkv ! matroskademux ! rtph264pay ! udpsink host=127.0.0.1 port=8004
I tried changing the buffer size of the udpsink and it had little effect (it might have increased the shown frames a bit).
Related
I am trying to stream video from a raspberry pi 4 employing a raspberry pi camera module 2 to a laptop and I am encountering difficulties depending on the interface (ethernet or WiFi) but also the router I am using.
On the raspberry pi 4 (bullseye, gstreamer 1.18.4) I am running following command
gst-launch-1.0 libcamerasrc ! video/x-raw, width=1280, height=720 ! videoconvert ! jpegenc ! rtpjpegpay ! udpsink host=192.168.0.100 port=5200
On the laptop (bullseye, gstreamer 1.18.4) I am accessing the stream as follows
gst-launch-1.0 -v udpsrc port=5200 ! application/x-rtp, media=video, clock-rate=90000, payload=96 ! rtpjpegdepay ! jpegdec ! videoconvert ! queue ! fpsdisplaysink
When both devices are connected over WiFi to an access point, which supports WiFi 5 (TP-Link Archer MR600), I get very smooth video on the laptop with 30 FPS and almost no dropped frames. However, when connecting both devices over WiFi 6 (TP-Link Archer AX50) there are still no dropped frames but the frame rate is 1-3 FPS. Manually lowering the WiFi standard on this router doesn't solve the problem. Also, when connecting both devices over Ethernet the frame rate is again in the range 1-3 FPS, but only connecting the laptop over Ethernet and the raspberry over WiFi seems to work with 30 FPS.
When streaming I have collected the incoming packets with tcpdump
sudo tcpdump -i wlp0s20f3 -s 0 -w dump.pcap host 192.168.0.201 and udp
and then played them back with
gst-launch-1.0 filesrc location=dump.pcap ! pcapparse ! application/x-rtp, media=video, clock-rate=90000, payload=96 ! rtpjpegdepay ! jpegdec ! videoconvert ! queue ! fpsdisplaysink
showing the original stream in either of the above described cases perfectly as expected with 30 FPS. I simply noticed that for the WiFi 6 case and Ethernet case there are slightly more rendered frames for the same time period compared to the WiFi 5 case.
To me it looks like that the video stream arrives correctly, but for some reason cannot be rendered by gstreamer live but only after capturing it.
i am trying to stream live video feed from a camera connected to a Jetson NX, to a computer connected to the same network, the network works as wireless ethernet, meaning the jetson sees it as wired connection but in reality its wireless and is limited by bitrate.
On the jetson side, I am using OpenCV VideoWrite to send frames over the network using this pipeline:
cv::VideoWriter video_write("appsrc ! video/x-raw,format=BGR ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv\
! video/x-raw(memory:NVMM),format=NV12,width=640,height=360,framerate=30/1 ! nvv4l2h264enc insert-sps-pps=1 insert-vui=1 idrinterval=30 \
bitrate=1800000 EnableTwopassCBR=1 ! h264parse ! rtph264pay ! udpsink host=169.254.84.2 port=5004 auto-multicast=0",\
cv::CAP_GSTREAMER,30,cv::Size(640,360));
on the receiving computer my video capture is :
cv::VideoCapture video("udpsrc port=5004 auto_multicast=0 !
application/x-rtp,media=video,encoding-name=H264 ! rtpjitterbuffer latency=0 !
rtph264depay ! decodebin ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1 "
, cv::CAP_GSTREAMER);
Problem is, if the camera jitters a lot or moves a lot to any direction, the video stream either freezes or completely pixelates. I was hoping I could get suggestions for either a better encoder(im not limited to nvv4l2h264enc or h264 in general) or a solution for the pixaltion and freeze, or maybe even a better way to stream the video other than VideoWrite.
I am trying to stream 360p video at 30fps, my bitrate is limited to either 6mbps or 2.5mbps depending on the distance i want to limit myself at. it does not seem like a network problem simply because If i change parameters like Codec(MJPG instead of gstreamer for example) it changes the behaviour of the video feed, in my case it lowers the amount of freezing but makes the pixelating worse.
I'm working on a video streaming wearable device. During the tests, it came up that the pipeline clock and stream stop while fast walking or running. It's bizarre behaviour because in debug messages there are no errors about the broken pipeline, besides lost frames. It's frizzed and only restarting help. May you guys guess what causes the problem?
The pipelines I use:
streaming device:
gst-launch-1.0 -vem --gst-debug=3 v4l2src device=/dev/video0 ! video/x-raw,width=640,height=480,framerate=\(fraction\)30/1 ! v4l2h264enc extra-controls=s,video_bitrate=250000 capture-io-mode=4 output-io-mode=4 ! "video/x-h264,level=(string)4" ! rtph264pay config-interval=1 ! multiudpsink clients="127.0.0.1:5008,10.123.0.2:5008"
client:
udpsrc port=5008 do-timestamp=true ! application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96 ! rtpjitterbuffer latency=100 drop-on-latency=true drop-messages-interval=100000000 ! queue max-size-buffers=20000 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! glupload ! qmlglsink name=qmlglsink sync=false
The hardware I use is a PS3 Eye cam, and LTE modem to transmit video with a pretty low uplink of 1-2 Mbit/s, and everything running on RaspberryPi 3b+ 1GB.
For more debug info there are also pictures of the log file after last registered dropped frame and every next "cycle" sends a new query, loops over GST Element from sink to the source which is my camera and ends with max query duration(highlighted query to v4l2src)
Do you know how to overcome this problem?
The problem has been resolved. The issue was not variable encoder bitrate.
A more detailed inspection and pipeline that works for me is in this GStreamer issue page
0
The objective I am trying to achieve is streaming 1080p video from Raspberry pi camera and record the video simultaneously.
I tried recording the http streaming as source but didn't work on 30fps. A lot of frames were missing and almost got 8fps only.
As a second approach, I am trying to record the file directly from camera and then streaming the "recording in progress/buffer" file. For the same I am trying to use GStreamer. Please suggest if this is good option or should I try any other?
For Recording using GStreamer I used
gst-launch-1.0 -v v4l2src device=/dev/video0 ! capsfilter caps="video/x-raw,width=1920,height=1080,framerate=30/1" !
videoflip method=clockwise ! videoflip method=clockwise ! videoconvert ! videorate ! x264enc! avimux ! filesink location=test_video.h264
Result : recorded video shows 1080p and 30fps but frames are dropping heavily.
For Streaming the video buffer I have used UDP in Gstreamer as,
gst-launch-1.0 -v v4l2src device=/dev/video0 ! capsfilter caps="video/x-raw,width=640,height=480,framerate=30/1" ! x264enc ! queue ! rtph264pay ! udpsink host=192.168.5.1 port=8080
Result : No specific errors on terminal but can't get stream on vlc.
Please suggest the best method here.
I stream h264 contents as rtsp from a filesrc to a udpsink
gst-launch-0.10 filesrc location=./test.movie ! h264parse ! rtph264pay ! udpsink host=127.0.0.1 port=9004
This pipeline works well but gives a quite high CPU load and my question is if there is any way of minimizing the CPU load, limiting the processing rate?
Feels like since there is a filesrc and udpsink there is really nothing stopping gstreamer from just working as quick as it can. A 35sec movie takes <0,5 seconds for gstreamer to stream with the given pipeline.