Access Camera using OpenCV (Via GStreamer) - c++

I'm trying to develop an application which should analyse a video stream from a MIPI camera(5MP). So I'm using gstreamer to get the video feed access it using OpenCV. I tried the following pipeline and it's working:
imxv4l2videosrc device="/dev/video0" ! autovideosink
But when I try to use it with OpenCV, it gives some errors.
VideoCapture cap("imxv4l2videosrc device=\"/dev/video0\" ! autovideosink");
OpenCV Error: Unspecified error (GStreamer: cannot find appsink in manual pipeline
) in cvCaptureFromCAM_GStreamer, file /root/OpenCV/opencv/modules/videoio/src/cap_gstreamer.cpp, line 759
VIDEOIO(cvCreateCapture_GStreamer (CV_CAP_GSTREAMER_FILE, filename)): raised OpenCV exception:
/root/OpenCV/opencv/modules/videoio/src/cap_gstreamer.cpp:759: error: (-2) GStreamer: cannot find appsink in manual pipeline
in function cvCaptureFromCAM_GStreamer
Then I tried to use the following pipeline, and it's not working as well:
VideoCapture cap("imxv4l2videosrc device=\"/dev/video0\" ! appsink");
ERROR: unrecognized std! 0 (PAL=ff, NTSC=b000
ERROR: v4l2 capture: unsupported ioctrl!
GStreamer Plugin: Embedded video playback halted; module imxv4l2videosrc0 reported: Internal data flow error.
ERROR: v4l2 capture: unsupported ioctrl!
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /root/OpenCV/opencv/modules/videoio/src/cap_gstreamer.cpp, line 832
VIDEOIO(cvCreateCapture_GStreamer (CV_CAP_GSTREAMER_FILE, filename)): raised OpenCV exception:
/root/OpenCV/opencv/modules/videoio/src/cap_gstreamer.cpp:832: error: (-2) GStreamer: unable to start pipeline
in function cvCaptureFromCAM_GStreamer
GStreamer version: 1.0
OpenCV version: 3.2
What is the piece i'm missing here?
Or is my approach is wrong?

Here is the answer to my question(with #Alper Kucukkomurler's help)
You can access the MIPI camera through OpenCV (with GStreamer) with
VideoCapture cap("imxv4l2videosrc device=\"/dev/video0\" ! videoconvert ! appsink");
Also If you want to change the resolution of the input, imx-capture-mode parameter can be used, which is of imxv4l2videosrc element.
For example,
imxv4l2videosrc imx-capture-mode=5 ! <other elements>

Related

GStreamer warning: Error opening bin: syntax error in C++ but same syntax works on terminal

I am creating a video stream from my headless raspberry pi using usb camera and opencv.
Ubuntu 18.04 LTS
Raspi OS: Bullseye
USB Camera
gst syntax test on terminal:
gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert! videoscale ! video/x-raw, width=640, height=480 ! autovideosink -v
In OpenCV C++:
string pipeline;
pipeline = "v4l2src device=/dev/video0 ! videoconvert! videoscale ! video/x-raw, width=640, height=480 ! autovideosink -v";
cv::VideoCapture cap(pipeline, cv::CAP_GSTREAMER);
When I launch the pipeline from terminal it creates a window with by usb webcam feed. However when I try to use the same pipeline from c++ it throws this error:
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (734) open OpenCV | GStreamer warning: Error opening bin: syntax error
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (501) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
terminate called after throwing an instance of 'std::invalid_argument'
what(): cannot predict with an empty OpenCV image
Aborted
What am i missing in the syntax in the code?
Edit:
After some trials I could get the pipeline generated, with this:
pipeline = "v4l2src device=/dev/video0 ! videoconvert! video/x-raw, width=640, height=480 ! appsink";
and warning:
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (961) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
.
.
.
some stuff for that frame
.
.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (509) isPipelinePlaying OpenCV | GStreamer warning: unable to query pipeline state
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1824) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Could not read from resource.
When searched for it, I found this post. Which led to a bug report here and here.
I am using OpenCV 4.5, and I assume that bug would have been fixed by later version as it was reported for OpenCV 3.4.

How to fix gstreamer v4l2src error when streaming video over usb?

I am using this command: gst-launch-1.0 v4l2src ! xvimagesink
to stream video over usb on my nvidia jetson nano and I am getting this output:
Setting pipeline to PAUSED...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: Could not initialise Xv output
Additional debug info:
xvimagesink.c(1773): gst_xv_image_sink_open (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0:
Could not display (null)
Setting pipeline to NULL..
Freeing pipeline...

Getting error while opening file with gstreamer in OpenCV

I am getting error while opening file with gstreamer in opencv below is my pipeline. I have searched on internet any try all possible things but nothing is working for me.
ERROR
[ WARN:0] OpenCV | GStreamer warning: unable to query duration of stream
[ WARN:0] OpenCV | GStreamer warning: Cannot query video position: status=1, value=1, duration=-1
cv::VideoCapture capture("filesrc location=a.mp4 ! qtdemux ! queue ! h264parse ! v4l2h264dec ! videoconvert ! video/x-raw,format=GRAY8 ,width=1280,height=720 ! appsink",CAP_GSTREAMER);

Cannot play wav audio on armv7 with Qemu with gst-launch-1.0

I'm trying to play a WAV audio file using Qt with C++, it fails then I tried to play it using gst-launch-1.0, it also fails. This is all using an armv7 toolchain and the running under QEMU.
After failing in C++ using code
QAudioDecoder decoder;
QFileInfo fileInfo(WAV_FILE_NAME);
decoder.setSourceFilename(fileInfo.absoluteFilePath());
decoder.start();
if (decoder.state() == QAudioDecoder::DecodingState)
qDebug() << "Decoding";
It fails with error
0:00:00.167441195 29283 0xf375dc00 WARN basesrc gstbasesrc.c:3583:gst_base_src_start_complete:<source> pad not activated yet
0:00:00.176093619 29283 0xf375dc00 WARN basesrc gstbasesrc.c:3583:gst_base_src_start_complete:<source> pad not activated yet
0:00:00.237290158 29283 0xf37fc660 WARN uridecodebin gsturidecodebin.c:921:unknown_type_cb:<uridecodebin0> warning: No decoder available for type 'audio/x-wav'.
0:00:00.238382698 29283 0xf37fc660 WARN decodebin gstdecodebin2.c:4640:gst_decode_bin_expose:<decodebin0> error: no suitable plugins found:
Missing decoder: WAV (audio/x-wav)
0:00:00.239083745 29283 0xf37fc660 WARN uridecodebin gsturidecodebin.c:988:no_more_pads_full:<uridecodebin0> error: no suitable plugins found:
../../../gst-plugins-base-1.14.4/gst/playback/gstdecodebin2.c(4640): gst_decode_bin_expose (): /GstPlayBin:playbin1/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0:
no suitable plugins found:
Missing decoder: WAV (audio/x-wav)
0:00:00.240739264 29283 0xf37fc660 WARN typefind gsttypefindelement.c:1231:gst_type_find_element_loop:<typefind> error: Internal data stream error.
0:00:00.240853974 29283 0xf37fc660 WARN typefind gsttypefindelement.c:1231:gst_type_find_element_loop:<typefind> error: streaming stopped, reason not-linked (-1)
That's why I tried to use
./gst-launch-1.0 -v uridecodebin uri=file:///home/vbox/myaudio.wav
Then I got this error as well
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: source = "\(GstFileSrc\)\ source"
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = audio/x-wav
Missing element: WAV demuxer
WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: No decoder available for type 'audio/x-wav'.
Additional debug info:
../../../gst-plugins-base-1.14.4/gst/playback/gsturidecodebin.c(921): unknown_type_cb (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: Your GStreamer installation is missing a plug-in.
Additional debug info:
../../../gst-plugins-base-1.14.4/gst/playback/gsturidecodebin.c(988): no_more_pads_full (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0:
no suitable plugins found:
../../../gst-plugins-base-1.14.4/gst/playback/gstdecodebin2.c(4640): gst_decode_bin_expose (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0:
no suitable plugins found:
Missing decoder: WAV (audio/x-wav)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
I'm not sure what's really missing. Also, this is the output of gst-inspect-1.0
$> ./gst-inspect-1.0 | grep wav
audiovisualizers: wavescope: Waveform oscilloscope
typefindfunctions: application/x-shockwave-flash: swf, swfl
typefindfunctions: audio/x-wav: wav
typefindfunctions: audio/x-wavpack: wv, wvp
typefindfunctions: audio/x-wavpack-correction: wvc
What could be missing here?
The "wavparse" plugin should be available.

Access an IP camera with Raspberry Pi2, OpenCV and GStreamer: Gstremaer unable to start pipeline

I'm trying to get frames from an IP camera. I can get frames with FFMPEG or GStreamer from my Ubuntu 14.04 PC and omxplayer or gst-launch-1.0 from Raspberry Pi. But when I want to access camera with c++ OpenCV from Raspberry Pi, it is giving me an error about GStreamer pipeline;
(ocvip:2870): GStreamer-WARNING **: Failed to load plugin '/usr/local/lib/gstreamer-1.0/libgstrtsp.so': /usr/local/lib/gstreamer-1.0/libgstrtsp.so: undefined symbol: gst_rtsp_connection_set_tls_validation_flags
GStreamer Plugin: Embedded video playback halted; module u-uridecodebin, reported: URI was not accepted by any element
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline) in icvStartPipeline, file /opt/opencv-2.4.10/modules/highgui/src/cap_gstreamer.cpp, line 383
what(): /opt/opencv-2.4.10/modules/highgui/src/cap_gstreamer.cpp:383: error: (-2) GStreamer unable to start pipeline in function icvStartPipeline
Aborted
code:
const string videoAddress = "rtsp://user:password#xxx.xxx.x.xxx:xxx/cam/realmonitor?channel=1&subtype=1";
int main() {
cv::VideoCapture video;
cv::Mat im;
cv::namedWindow("IP", 1);
video.open(videoAddress);
if (video.isOpened()) {
video >> im;
if (!im.data) break;
cv::imshow("IP", im);
if (cv::waitKey(20) >= 0) break;
}
}
I can access camera with same rtsp address, omxplayer is showing normal gst-launch is showing frames rotated by 180 degrees. I've built GStreamer-1.0 using this page because prebuilt libraries hasn't found by cmake. I found this issue. There are two fixes int this bug but I'm not sure are these the correct solutions and solve my problem because I'm not accessing camera with video.open(0) so I couldn't do like video.open(IPAddress + CV_CAP_PVAPI)?
Thanks.