Opening a GStreamer pipeline from OpenCV with VideoWriter - c++

Question

I am capturing and processing video frames with OpenCV, and I would like to write them as a h265 video file. I am struggling to get a proper Gstreamer pipeline to work from OpenCV.
Gstreamer works fine by itself. In particular, I am able to run this command, which encodes video very quickly (thanks to GPU acceleration) and saves it to a mkv file:
gst-launch-1.0 videotestsrc num-buffers=90 ! 'video/x-raw, format=(string)I420, width=(int)640, height=(int)480' ! omxh265enc ! matroskamux ! filesink location=test.mkv
Now I would like to do the same thing from within my OpenCV application. My code is something like:
Mat img_vid = Mat(1024, 1024, CV_8UC3);
VideoWriter video;
video.open("appsrc ! autovideoconvert ! omxh265enc ! matroskamux ! filesink location=test.mkv", 0, (double)25, cv::Size(1024, 1024), true);
if (!video.isOpened()) {
printf("can't create writer\n");
return -1;
}
while ( ... ) {
// Capture frame into img_vid => That works fine
video.write(img_vid);
...
}
At first sight, this seems to work, but what it does is it creates file named "appsrc ! autovideoconvert ! omxh265enc ! matroskamux ! filesink location=test.mkv" and fills it with uncompressed video frames, completely ignoring the fact that this is a Gstreamer pipeline.
I have tried other pipelines, but they result in a variety of errors:
video.open("appsrc ! autovideoconvert ! omxh264enc ! 'video/x-h264, streamformat=(string)byte-stream' ! h264parse ! qtmux ! filesink location=test.mp4 -e", 0, (double)25, cv::Size(1024, 1024), true);
Which results in:
(Test:5533): GStreamer-CRITICAL **: gst_element_make_from_uri:
assertion 'gst_uri_is_valid (uri)' failed OpenCV Error: Unspecified
error (GStreamer: cannot find appsrc in manual pipeline ) in
CvVideoWriter_GStreamer::open, file
/home/ubuntu/opencv/modules/videoio/src/cap_gstreamer.cpp, line 1363
VIDEOIO(cvCreateVideoWriter_GStreamer(filename, fourcc, fps,
frameSize, is_color)): raised OpenCV exception:
/home/ubuntu/opencv/modules/videoio/src/cap_gstreamer.cpp:1363: error:
(-2) GStreamer: cannot find appsrc in manual pipeline in function
CvVideoWriter_GStreamer::open
I also tried the simple:
video.open("appsrc ! autovideosink", 0, (double)25, cv::Size(1024, 1024), true);
which yields:
GStreamer Plugin: Embedded video playback halted; module appsrc0
reported: Internal data flow error.
I am using OpenCV 3.1 with Gstreamer support. The hardware is a Jetson TX1 with L4T 24.2.1.

Answer 1

I encountered a similar problem before. Since the pipe/file name ends with .mkv, OpenCV interprets it as a video file instead of a pipe.
You can try ending it with a dummy spacing like after mkv
video.open("appsrc ! autovideoconvert ! omxh265enc ! matroskamux ! filesink location=test.mkv ", 0, (double)25, cv::Size(1024, 1024), true);
or with a dummy property like
video.open("appsrc ! autovideoconvert ! omxh265enc ! matroskamux ! filesink location=test.mkv sync=false", 0, (double)25, cv::Size(1024, 1024), true);