duplicate webcam stream in a virtual device created with v4l2loopback - gstreamer

Question

hello gods of videos streams
I'm trying to duplicate a webcam stream in order to send it and still be able to use it in a web browser for example
goal is to create a virtual video device with v4l2Loopback use gstreamer to get the source from real web cam make a tee stream it where i need (rtps server) and on the other tee branch use a v4l2sink to forward to the virtual device created with v4l2loopback
I'm testing under ubuntu 20.04 but planning to use it on raspian buster in the end
I've
sudo apt install v4l2loopback-dkms v4l2loopback-utils
and
sudo modprobe v4l2loopback video_nr=100
and it looks ok
$ v4l2-ctl --all -d /dev/video100
Driver Info:
Driver name : v4l2 loopback
Card type : Dummy video device (0x0000)
Bus info : platform:v4l2loopback-000
Driver version : 5.4.65
Capabilities : 0x85208003
Video Capture
Video Output
Video Memory-to-Memory
Read/Write
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x85208000
Video Memory-to-Memory
Read/Write
Streaming
Extended Pix Format
Device Capabilities
Priority: 0
Video input : 0 (loopback: ok)
Video output: 0 (loopback in)
Format Video Output:
Width/Height : 1920/1080
Pixel Format : 'YU12' (Planar YUV 4:2:0)
Field : None
Bytes per Line : 1920
Size Image : 3110400
Colorspace : sRGB
Transfer Function : Default (maps to sRGB)
YCbCr/HSV Encoding: Default (maps to ITU-R 601)
Quantization : Default (maps to Limited Range)
Flags :
Streaming Parameters Video Capture:
Frames per second: 30.000 (30/1)
Read buffers : 8
Streaming Parameters Video Output:
Frames per second: 30.000 (30/1)
Write buffers : 8
User Controls
keep_format 0x0098f900 (bool) : default=0 value=0
sustain_framerate 0x0098f901 (bool) : default=0 value=0
timeout 0x0098f902 (int) : min=0 max=100000 step=1 default=0 value=0
timeout_image_io 0x0098f903 (bool) : default=0 value=0
here
before streaming my real webcam I just try to stream videotestsrc like explained in gstreamer v4l2sink doc
gst-launch-1.0 videotestsrc ! v4l2sink device=/dev/video100
strangely it says it's not an output device (GST-DEBUG=3)
$ gst-launch-1.0 videotestsrc ! v4l2sink device=/dev/video100
Setting pipeline to PAUSED ...
0:00:00.137162087 12355 0x5601d5516010 WARN v4l2 v4l2_calls.c:636:gst_v4l2_open:<v4l2sink0> error: Device '/dev/video100' is not a output device.
0:00:00.137209656 12355 0x5601d5516010 WARN v4l2 v4l2_calls.c:636:gst_v4l2_open:<v4l2sink0> error: Capabilities: 0x85208000
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0: Device '/dev/video100' is not a output device.
Additional debug info:
v4l2_calls.c(636): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0:
Capabilities: 0x85208000
Setting pipeline to NULL ...
Freeing pipeline ...
I' ve look to find other issue like mine but maybe I've not well searched I didnt find any relevant infos
If any one can help I will be so greatfull
thank's so mutch for reading until here and thank's so mutch more for any help info idea etc
edit :
I also tried on raspian buster raspberry pi 4
I got few issue with v4l2loopback at install but I followed that topic https://github.com/RPi-Distro/repo/issues/188 and be able to install it
i've do the same (modprobe and then gstream) and gstreamer v4l2Sink looks to work when using workaround to avoid not negociated-4 error
gst-launch-1.0 videotestsrc ! identity drop-allocation=1 ! v4l2sink device=/dev/video100
it look to work but I got lots of warnings :
$ gst-launch-1.0 -v videotestsrc ! identity drop-allocation=1 ! v4l2sink device=/dev/video100
Définition du pipeline à PAUSED...
Le pipeline est en phase de PREROLL…
0:00:00.128889803 11076 0xe230f0 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<videotestsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:00.129497619 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at YUYV#48x32: 0
0:00:00.129647893 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at UYVY#48x32: 0
0:00:00.129766111 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at 422P#48x32: 0
0:00:00.129872201 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at YU12#48x32: 0
0:00:00.129975513 11076 0xe230f0 WARN v4l2 gstv4l2o$ gst-launch-1.0 -v videotestsrc ! identity drop-allocation=1 ! v4l2sink device=/dev/video100
Définition du pipeline à PAUSED...
Le pipeline est en phase de PREROLL…
0:00:00.128889803 11076 0xe230f0 FIXME default gstutils.c:3981:gst_pad_create_stream_id_internal:<videotestsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
0:00:00.129497619 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at YUYV#48x32: 0
0:00:00.129647893 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at UYVY#48x32: 0
0:00:00.129766111 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at 422P#48x32: 0
0:00:00.129872201 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at YU12#48x32: 0
0:00:00.129975513 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at YV12#48x32: 0
0:00:00.130079232 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at 411P#48x32: 0
0:00:00.130190210 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at YVU9#48x32: 0
0:00:00.130291689 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at YUV9#48x32: 0
0:00:00.130395612 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at JPEG#48x32: 0
0:00:00.130483220 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at MJPG#48x32: 0
0:00:00.130570810 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at dvsd#48x32: 0
0:00:00.130658326 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at RGB4#48x32: 0
0:00:00.130753046 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at BGR4#48x32: 0
0:00:00.130847173 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at RGB3#48x32: 0
0:00:00.130959966 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at BGR3#48x32: 0
0:00:00.131055000 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at NV12#48x32: 0
0:00:00.131170441 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at RGBQ#48x32: 0
0:00:00.131266012 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at RGBP#48x32: 0
0:00:00.131359676 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at RGBO#48x32: 0
0:00:00.131454285 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at BA81#48x32: 0
0:00:00.131539338 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at GBRG#48x32: 0
0:00:00.131623317 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at GRBG#48x32: 0
0:00:00.131707907 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at RGGB#48x32: 0
0:00:00.131792701 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at GREY#48x32: 0
0:00:00.131889365 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at VP90#48x32: 0
0:00:00.131974881 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at VP80#48x32: 0
0:00:00.132062378 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at VC1L#48x32: 0
0:00:00.132151839 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at VC1G#48x32: 0
0:00:00.132250595 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at XVID#48x32: 0
0:00:00.132340129 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at MPG4#48x32: 0
0:00:00.132427053 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at MPG2#48x32: 0
0:00:00.132513495 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at MPG1#48x32: 0
0:00:00.132601677 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at H263#48x32: 0
0:00:00.132691304 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at AVC1#48x32: 0
0:00:00.132781080 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at H264#48x32: 0
0:00:00.132869910 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at Y16 #48x32: 0
0:00:00.132966408 11076 0xe230f0 WARN v4l2 gstv4l2object.c:2514:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2sink0:sink> Unknown frame interval type at YVYU#48x32: 0
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)2:4:7:1, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)2:4:7:1, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)2:4:7:1, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)2:4:7:1, pixel-aspect-ratio=(fraction)1/1
Le pipeline a terminé la phase PREROLL…
Passage du pipeline à la phase PLAYING…
New clock: GstSystemClock
and finaly when trying to get and image from this video device
$ ffmpeg -f video4linux2 -s 640x480 -i /dev/video100 -ss 0:0:2 -frames 1 /tmp/out2.jpg
ffmpeg version 4.1.6-1~deb10u1+rpt1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
...[truncated because of stack overflow body size limit]...
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler # 0x136f230] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to '/tmp/out2.jpg':
Metadata:
encoder : Lavf58.20.100
Stream #0:0: Video: mjpeg, yuvj422p(pc), 320x240, q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc
Metadata:
encoder : Lavc58.35.100 mjpeg
Side data:
cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
[video4linux2,v4l2 # 0x1363200] Dequeued v4l2 buffer contains 155648 bytes, but 153600 were expected. Flags: 0x00000001.
/dev/video100: Invalid data found when processing input
frame= 0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed= 0x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

Answer 1

I just went thru this and will explain the big stumbling block with v4l2 video devices. The basic issue is real devices can only be accessed once, while loopback devices are not limited.
First start up your usb webcam. You should now have /dev/video0 and /dev/video1. I have no idea why video1 gets created, but ignore it.
Next start v4l2loopback with "modprobe v4l2loopback devices=4" Now you should have video0 thru video6.
Now comes the tricky part, if you try to duplicate /dev/video0 to /dev/video3, video3 will work, but video0 will now report "device or resource busy", so you are still stuck with 1 video output. This occurs because REAL devices can only be accessed by 1 program at a time because they have real v4l2 controls. You can't have 2 programs accessing the same controls.
The solution is to duplicate /dev/video0 twice, like this:
ffmpeg -f video4linux2 -i /dev/video0 -codec copy -f v4l2 /dev/video3 -codec copy -f v4l2 /dev/video4
Now, you can view the normal video0 with "ffplay -i /dev/video3"
and, you can do something else with /dev/video4, like this
gst-launch-1.0 -v v4l2src device=/dev/video4 ! autovideoconvert ! videoconvert ! video/x-raw, width=640, height=480, framerate=30/1 ! vertigotv ! v4l2sink device=/dev/video5
Now you can see the original video with "ffplay -i /dev/video3" and your manipulated video with "ffplay -i /dev/video5"
I hope it helps, it took me a few hours of googling to figure that out. :-)