ALREADY WORKING:
I get video from embedded video source (just device) through LAN and I can get video from it and save it to file ".h264" (append to file every next "encodedPacket", C++) (it is worked fine, I can play file using VLC).
TASK:
How can I save image files periodically (5 in 1 second for example) (any format, but I want jpg)?
File info:
Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : Baseline#L3.1
Format settings : 1 Ref Frames
Format settings, CABAC : No
Format settings, RefFrames : 1 frame
Width : 640 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
ffmpeg is your friend:
https://trac.ffmpeg.org/wiki/Create%20a%20thumbnail%20image%20every%20X%20seconds%20of%20the%20video
I'd try something like: ffmpeg -i input.h264 -vf fps=5 out%d.jpg
If your input is a network stream you can do something like ffmpeg -i tcp://local_hostname:port?listen
https://trac.ffmpeg.org/wiki/StreamingGuide
Related
I have a simple script to read mp4 file like this.
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/videoio/videoio.hpp>
int main()
{
cv::VideoCapture file("test.mp4");
cv::Mat frame;
while (true)
{
file.read(frame);
cv::imshow("Preview", frame);
cv::waitKey(42);
}
file.release();
return 0;
}
This works fine. But when I integrate this script into another project I'm working on. The the image frame shows in wrong aspect ratio.
Correct (Side by Side):
Wrong (Only show one side and aspect ratio is wrong):
I'm running on Windows VS2019. I have remove all other script in my exists project, just leave the above. The only different I can think of is the includes and linker setting. I use ceres, glog, d3d11, realsense2, VTK, pcl, eigen3, OpenXR in the project. Does any of that effects what OpenCV behave? Or what might be the problem?
I've already try setting the frame width and height for VideoCapture and it's not working.
I've test both OpenCV 4.1 and 4.6.
When accessing frame.cols, frame.rows, I got the correcy resolution.
UPDATE
The metadata of the file I'm trying to read is as below. This is a side by side 3D video. It also can display correctly in player such as VLC.
mediainfo test.mp4
General
Complete name : test.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 181 MiB
Duration : 31 s 339 ms
Overall bit rate : 48.4 Mb/s
Writing application : Lavf58.20.100
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High#L5.1
Format settings : CABAC / 1 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 1 frame
Format settings, GOP : M=1, N=30
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 31 s 317 ms
Bit rate : 48.0 Mb/s
Width : 3 840 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Original display aspect ratio : 3.556
Frame rate mode : Variable
Frame rate : 60.000 FPS
Minimum frame rate : 59.920 FPS
Maximum frame rate : 60.080 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.193
Stream size : 179 MiB (99%)
Title : SStar Video
Codec configuration box : avcC
Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 31 s 339 ms
Bit rate mode : Constant
Bit rate : 140 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 534 KiB (0%)
Title : SStar Audio
Default : Yes
Alternate group : 1
What I hope will be a quick question.. Does anybody know how to specify the output format of a v4l2Convert Plugin in GStreamer? Slightly more specifically the stride alignment or Bytes per line I don't mind which?
To give the full details. I'm playing around with an embedded video processing platform and wish to simultaneously connect multiple outputs to a single input using a GStreamer Tee object. The problem is that different outputs need different stride alignments / bytes per line.
I can set the stride alignment on the v4l2src plugin I'm using to read the input device and can get a combination that "works" for all outputs. But under the hood GStremaer is being helpful and instantiating a buffer copy, to perform re-alignment, using memcpy and thus my CPU utilisation goes through the roof.
My proposed solution is to use a hardware DMA loopback device (v4l2 Mem2Mem) which is controlled by a v4l2Convert plugin, to provide a simple, low CPU load way of realigning the data.
I've tired this system with several pipleines and been monitoring it using v4l2-ctl and it appears to be able to do what I want. If I change the stride-align of the initial v4l2src plugin I can see GStreamer change the format of the data written into the Mem2Mem device to match this. However the capture / read format always remains at N BytesPerPixel x NumberofPixelsPerLine bytes per line.
Format Video Capture Multiplanar:
Width/Height : 1920/1080
Pixel Format : 'NV16' (Y/CbCr 4:2:2)
Field : None
Number of planes : 1
Flags :
Colorspace : SMPTE 170M
Transfer Function : Rec. 709
YCbCr/HSV Encoding: ITU-R 601
Quantization : Limited Range
Plane 0 :
Bytes per Line : 1920
Size Image : 4147200
Format Video Output Multiplanar:
Width/Height : 1920/1080
Pixel Format : 'NV16' (Y/CbCr 4:2:2)
Field : None
Number of planes : 1
Flags :
Colorspace : SMPTE 170M
Transfer Function : Rec. 709
YCbCr/HSV Encoding: ITU-R 601
Quantization : Limited Range
Plane 0 :
Bytes per Line : 2048
Size Image : 4423680
Vs
Format Video Capture Multiplanar:
Width/Height : 1920/1080
Pixel Format : 'NV16' (Y/CbCr 4:2:2)
Field : None
Number of planes : 1
Flags :
Colorspace : SMPTE 170M
Transfer Function : Rec. 709
YCbCr/HSV Encoding: ITU-R 601
Quantization : Limited Range
Plane 0 :
Bytes per Line : 1920
Size Image : 4147200
Format Video Output Multiplanar:
Width/Height : 1920/1080
Pixel Format : 'NV16' (Y/CbCr 4:2:2)
Field : None
Number of planes : 1
Flags :
Colorspace : SMPTE 170M
Transfer Function : Rec. 709
YCbCr/HSV Encoding: ITU-R 601
Quantization : Limited Range
Plane 0 :
Bytes per Line : 1920
Size Image : 4147200
Is there a way for me to change the v4l2Convert's capture / source format properties from within my GStreamer pipelines declaration? GST-Inpect-1.0 doesn't show any equivalent caps to the stride-align cap of v4l2src for v4l2Convert. And caps filters like video/x-raw don't appear to to be able to provide what I need (I accept if this is wrong as I'm very much a noob in this respect)
The best I've found is the extra-controls cap but I can find very little documentation on this, and what I have found appears to suggest it's used for setting the V4L2 devices "physical" controls rather than things like the format information, so I'm probably barking up the wrong tree there any way.
my test pipeline is: -
v4l2src name=videosrc device=/dev/video0 ! video/x-raw, width=1920, height=1080, format=NV16, framerate=30/1 ! queue ! v4l2convert device=/dev/video2 disable-passthrough=true capture-io-mode=4 output-io-mode=4 import-buffer-alignment=true ! queue ! kmssink sync=false fullscreen-overlay=true
If it's helps what I want to be able to do is provide video at 1920x1080, 1920 bytes per line (GStreamer will does this quite happily for me). But set the V2l2Converts capture / source to be 1920x1080, 2048 bytes per line, as the problem sink device needs a stride align of 256.
Thanks
I would like to improve decoding H.264 video stream with MPC-HC using LAV video decoder.
The stream I will play back is always in format yuvj444p (Planar YCbCr 4:4:4 in TV level [0-255]), encoded with x264.
I'm using MPC-HC version 1.7.10, and LAV video decoder 0.68.1
I have a nVidia Quadro K5200, and I know how to write GLSL shader to run YUV to RGB conversion.
I'm wondering if someone here could give me a hint if it's worth doing such job, and possibly on where to start.
Should I customize the EVR (Enhanced Video Renderer - Custom Presenter) ?
Should I just write an internal shader?
...
Yes, LAV Video Decoder filter does support yuvj444p to RGB32 color conversion.
In order to prove it, I have tried the following test:
Create uncompressed AVI file in RGB color format (using MATLAB).
Convert the AVI file to x264 compressed MKV file in yuvj444p color format (using FFMPEG).
Build a filter graph in Graph Studio Next, with LAV Video Decoder (DirectShow) filter.
Inspect the output pin of Decoder filter.
Play the graph, and compare the output frame to original input frame.
Input AVI file name: RGB_INPUT.avi
MKV file name: OUTPUT.mkv
I used ffmpeg with the following parameters (in command line):
ffmpeg -i RGB_INPUT.avi -pix_fmt yuvj444p -vf scale=w=0:h=0:out_color_matrix=bt709 -c:v libx264 -crf 18 -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709 -an OUTPUT.mkv
I took the example from here: http://forum.doom9.org/showthread.php?p=1671195
Filter Graph:
Inspecting the output pin of LAV Video Decoder, shows that output color format is RGB32 (media sub-type is: MEDIASUBTYPE_RGB32):
Comparing uncompressed input frame, to decoded output frame:
Source frame (uncompressed image):
Video Renderer output (screenshot):
Absolute difference image (scaled by 10):
Conclusion: LAV Video Decoder correctly convert yuvj444 to RGB32.
So. I have been trying to get my Raspberry Pi 2 to capture H264 stream with OpenCV from my Logitech C920 for quite some time now. I have been scavenging the internet for info, but with no luck.
A short system description:
Raspberry Pi 2, running Raspbian, Kernel 3.18
Logitech HD Pro Webcam c920
OpenCV 2.4.11
boneCV - Credits to Derek Molloy (https://github.com/derekmolloy/boneCV)
libx264 and FFMPEG (built with x264 support)
libv4l-dev, v4l-utils, qv4l2, v4l2ucp
I know OpenCV forces format to BGR24 (MJPG). This is specified in cap_libv4l.cpp. It looks like this(line 692->):
/* libv4l will convert from any format to V4L2_PIX_FMT_BGR24 */
CLEAR (capture->form);
capture->form.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
capture->form.fmt.pix.pixelformat = V4L2_PIX_FMT_BGR24;
capture->form.fmt.pix.field = V4L2_FIELD_ANY;
capture->form.fmt.pix.width = capture->width;
capture->form.fmt.pix.height = capture->height;
I can set the pixelformat manualy with v4l2-ctl --set-fmt-video
pi#raspberrypi ~/boneCV$ v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=H264
pi#raspberrypi ~/boneCV$ v4l2-ctl --get-fmt-video
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : 'H264'
Field : None
Bytes per Line: 3840
Size Image : 4147200
Colotspace : SRGB
And if I now run "./boneCV" - A very simple capture program that captures a picture and does a canny edge detection. (I'll add the code in the end). I get this:
pi#raspberrypi ~/boneCV$ ./boneCV
pi#raspberrypi ~/boneCV$ v4l2-ctl --get-fmt-video
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : 'MJPG'
Field : None
Bytes per Line: 0
Size Image : 4147200
Colorspace : SRGB
As you can se the "Pixelformat" and the "Bytes per Line" changes. The "Field" stays at None and the "Colourspace" stays at SRGB.
Then I tried to replace every "V4L2_PIX_FMT_BGR24" with "V4L2_PIX_FMT_H264" in cap_lib4vl.cpp and rebuilded OpenCV. When I then ran the "./boneCV" my two .png images are only black with one or two stripes of white color.
To find out if it is libv4l or OpenCV I ran "./capture" script that follow Derek Molloys boneCV. It uses libv4l directly and captures an H264 video stream with no problems at all. I then have to use "./raw2mpg4" to be able to watch it. The .mp4 file is 1920x1080 at 30 fps with no glitches. And after this I checked "v4l2-ctl --get-fmt-video" again and got this:
pi#raspberrypi ~/boneCV$ v4l2-ctl --get-fmt-video
Format Video Capture:
Width/Height : 1920/1080
Pixel Format : 'H264'
Field : None
Bytes per Line: 3840
Size Image : 4147200
Colotspace : SRGB
Exactly the same as when I did set everything manualy.
I have come to the conclusion that if I want OpenCV to be able to capture raw H264 streams I'll have to change the cap_libv4l.cpp, but I have no idea how. I think it may be because the difference in bits per frame and/or colorspace.
Do anybody know how to do this or how to make an workaround so that I stil can use OpenCVs "VideoCapture" function?
I know alot of Raspberry Pi and BeagleboneBlack users would be ever so gratefull if there was any solution to this problem.
I have tried to cover everything that I think is relevant, if there is anything more I could provide to paint the picture better, please say so.
Her some links to the mentioned scripts and programs:
(edit. I tried to post the links to each of the programs, but I didn't have enough reputation. Go to Derek Molloys github page and you'll find boneCV there.)
And no I can not use the "CV_FOURCC('H','2','6','4');" because this function is not implemented for linux yet.
I have an array of rgb data generated from glReadPixels().
Note that RGB data is pixel packed (r1,g1,b1,r2,g2,b2,...).
How can I quickly write a YUV video frame using OpenCV or another C++ library, so that I can stream them to FFMPEG? Converting RGB pixel to YUV pixel is not a problem, as there are many conversion formula available online. However writing the YUV frame is the main problem for me. I have been trying to write the YUV video frame since the last few days and were not successful in doing that.
This is one of my other question about writing YUV frame and the issues that I encountered: Issue with writing YUV image frame in C/C++
I don't know what is wrong with my current approach in writing the YUV frame to a file.
So right now I may want to use existing library (if any), that accepts an RGB data, and convert them to YUV and write the YUV frame directly to a file or to a pipe. Of course it would be much better if I can fix my existing program to write the YUV frame, but you know, there is also a deadline in every software development project, so time is also a priority for me and my project team members.
FFmpeg will happily receive RGB data in. You can see what pixel formats FFmpeg supports by running:
ffmpeg -pix_fmts
Any entry with an I in the first column can be used as an input.
Since you haven't specified the pixel bit depth, I am going to assume it's 8-bit and use the rgb8 pixel format. So to get FFmpeg to read rgb8 data from stdin you would use the following command (I am cating data in but you would be supplying via your pipe):
cat data.rgb | ffmpeg -f rawvideo -pix_fmt rgb8 -s WIDTHxHEIGHT -i pipe:0 output.mov
Since it is a raw pixel format with no framing, you need to subsitite WIDTH and HEIGHT for the appropriate values of your image dimensions so that FFmpeg knows how to frame the data.
I have specifed the output as a MOV file but you would need to configure your FFmpeg/Red5 output accordingly.
OpenCV does not support the YUV format directly, as you know, so it's really up to you to find a way to do RGB <-> YUV conversions.
This is a very interesting post as it shows how to load and create YUV frames on the disk, while storing the data as IplImage.
ffmpeg will write an AVI file with YUV but as karl says there isn't direct support for it in openCV.
Alternatively (and possibly simpler) you can just write the raw UYVY values to a file and then use ffmpeg to convert it to an AVI/MP4 in any format you want. It's also possible to write directly to a pipe and call ffmpeg directly from your app avoiding the temporary yuv file
eg. to convert an HD yuv422 stream to a h264 MP4 file at 30fps
ffmpeg -pix_fmt yuyv422 -s 1920x1080 -i input.yuv -vcodec libx264 -x264opts -r 30 output.mp4