I am developing a program that is capturing raspicam and streaming with GStreamer. The first course, capturing raspicam doesn't have problem. But, the next course is have a big problem. I was created a total of 2 of sources code (server, client). Streaming data is very slow. Can I have a way to improve it?
Please, help me.
Thank you.
----------- Server.cpp (Raspberry Pi, Raspbian) -----------
cap.set(CAP_PROP_FPS, 30);
cap.open(0);
// Movie Frame Setup
fps = cap.get(CAP_PROP_FPS);
width = cap.get(CAP_PROP_FRAME_WIDTH);
height = cap.get(CAP_PROP_FRAME_HEIGHT);
cout << "Capture camera with " << fps << " fps, " << width << "x" << height << " px" <<
writer.open("appsrc ! gdppay ! tcpserversink host=192.168.0.29 port=5000", 0, fps, cv::Size(width, height), true);
while(1){
printf("AA");
cap >> frame;
writer << frame;
}
----------- Client.cpp (PC, Ubuntu) -----------
Mat test;
String captureString = "tcpclientsrc host=192.168.0.29 port=5000 ! gdpdepay ! appsink";
VideoCapture cap(captureString);//0);
namedWindow("t");
while(1)
{
cap >> test;
imshow("t", test);
if( waitKey(10) > 0)
break;
}
}
You might benefit from using a udp stream instead of tcp. Check out this link for an example where a video was streamed from rpi to pc with only 100 ms lag.
Related
I am trying to run openCV in C++ and capture the camera input.
The program looks like this:
#include <iostream>
#include <sstream>
#include <new>
#include <string>
#include <sstream>
#include <opencv2/opencv.hpp>
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#define INPUT_WIDTH 3264
#define INPUT_HEIGHT 2464
#define DISPLAY_WIDTH 640
#define DISPLAY_HEIGHT 480
#define CAMERA_FRAMERATE 21/1
#define FLIP 2
void DisplayVersion()
{
std::cout << "OpenCV version: " << cv::getVersionMajor() << "." << cv::getVersionMinor() << "." << cv::getVersionRevision() << std::endl;
}
int main(int argc, const char** argv)
{
DisplayVersion();
std::stringstream ss;
ss << "nvarguscamerasrc ! video/x-raw(memory:NVMM), width=3264, height=2464, format=NV12, framerate=21/1 ! nvvidconv flip-method=2 ! video/x-raw, width=480, height=680, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink";
//ss << "nvarguscamerasrc ! video/x-raw(memory:NVMM), width=" << INPUT_WIDTH <<
//", height=" << INPUT_HEIGHT <<
//", format=NV12, framerate=" << CAMERA_FRAMERATE <<
//" ! nvvidconv flip-method=" << FLIP <<
//" ! video/x-raw, width=" << DISPLAY_WIDTH <<
//", height=" << DISPLAY_HEIGHT <<
//", format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink";
cv::VideoCapture video;
video.open(ss.str());
if (!video.isOpened())
{
std::cout << "Unable to get video from the camera!" << std::endl;
return -1;
}
std::cout << "Got here!" << std::endl;
cv::Mat frame;
while (video.read(frame))
{
cv::imshow("Video feed", frame);
if (cv::waitKey(25) >= 0)
{
break;
}
}
std::cout << "Finished!" << std::endl;
return 0;
}
When running this code I get the following outout:
OpenCV version: 4.6.0
nvbuf_utils: Could not get EGL display connection
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:751 Failed to create CaptureSession
[ WARN:0#0.269] global /tmp/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (1405) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
Got here!
Finished!
If I run the other commented command to video.open() I get this output:
OpenCV version: 4.6.0
nvbuf_utils: Could not get EGL display connection
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:751 Failed to create CaptureSession
I'm currently running this from headless mode on a jetson nano.
I also know that OpenCV and xlaunch works because I can use mjpeg streamer from my laptop and successfully stream my laptop camera output to my jetson nano by using video.open(http://laptop-ip:laptop-port/); and that works correctly (OpenCV is able to display a live video feed using xlaunch just fine).
I think this command is telling me my camera is successfully installed:
$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'RG10'
Name : 10-bit Bayer RGRG/GBGB
Size: Discrete 3264x2464
Interval: Discrete 0.048s (21.000 fps)
Size: Discrete 3264x1848
Interval: Discrete 0.036s (28.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1640x1232
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)
Any help would be much appreciated
The warning seems to be stating that you cannot use egl, ie OpenGL when in headless mode (because there is no screen)
If you run in headless mode. Would in not make more sense to not try to open a window do display?
cv::imshow("Video feed", frame);
if (cv::waitKey(25) >= 0)
{
break;
}
Remove this code and instead use cv::imwrite to write to a file, or whatever you want to do with the data.
Or if you run ssh. Run ssh with -X option to show the windows on your client computer instead. Could be slow, but if you really want to use cv::imshow it could be a option.
Well I fixed it by rebooting. I already did do a reboot but I also now have some errors whenever I run the program. I did recompile the dlib library but so I do think that when you update the gstreamer library you need to reboot your machine to successfully use it.
I would like to capture images with a webcam (i.e. logitech C922) with C++. Does anyone succeed in capturing images with the webcam at 60fps and 720p? I read the code in the following thread and add "cap.set(CAP_PROP_FPS, 60)", but the frame rate was maintained at about 30fps.
How to set camera fps in opencv?
Then I posted the same question, but the forum is under maintenance.
http://answers.opencv.org/question/172992/streaming-logitec-c922-at-60fps-with-c/
I added the both proposed codes to mine.
As the result, the value of fps was 33.3... and FPS was 60.0 in the case that I used cap.set(CAP_PROP_EXPOSURE, -5) because I'm in office and at night here.
I tried to use lower value for CAP_PROP_EXPOSURE (e.g. -10), but the fps didn't change.
The image shown with imshow wasn't updated obviously at 60fps.
Is there anything I can do?
This is the code I used.
VideoCapture cap(0); //capture the video from web cam
if (!cap.isOpened()) // if not success, exit program
{
cout << "Cannot open the web cam" << endl;
return -1;
}
cap.set(CV_CAP_PROP_FOURCC, CV_FOURCC('M','J','P','G'));
cap.set(CAP_PROP_FPS, 60);
cap.set(CAP_PROP_EXPOSURE, -5);
cap.set(CV_CAP_PROP_FRAME_WIDTH, 1280);
cap.set(CV_CAP_PROP_FRAME_HEIGHT, 720);
cout << cap.get(CAP_PROP_FPS) << endl;
cvNamedWindow("img");
time_t cap_start, cap_end;
Mat frame;
double MAX_FRAME_NUM = 100;
time(&cap_start);
for (int n = 0; n < MAX_FRAME_NUM; n++) {
cap >> frame;
}
time(&cap_end);
double fps = MAX_FRAME_NUM / difftime(cap_end, cap_start);
cout << "fps:" << fps << endl;
cv::waitKey(0);
Environment Information OpenCv: 3.3.0 OS: Windows 10 Pro IDE: Visual Studio 2017 CPU: i7-7560U RAM 16GB USB: 3.0
Best regards, gellpro
I stumbled upon the same issue with this camera.
My environment is Ubuntu 18.04, python 3.6.5 and OpenCV 3.4.
I found this solution from your first link to be working:
cap.set(CV_CAP_PROP_FOURCC, CV_FOURCC('M', 'J', 'P', 'G'));
For python, the code I use is:
cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc('M','J','P','G'))
cap.set(cv2.CAP_PROP_FPS, 60)
I'm using OpenCV and v4l2loopback library to emulate video devices:
modprobe v4l2loopback devices=2
Then I check what devices I have:
root#blah:~$ v4l2-ctl --list-devices
Dummy video device (0x0000) (platform:v4l2loopback-000):
/dev/video1
Dummy video device (0x0001) (platform:v4l2loopback-001):
/dev/video2
XI100DUSB-SDI (usb-0000:00:14.0-9):
/dev/video0
video0 is my actual camera where I grab frames from, then I plan to process them via OpenCV and write it to video2 (which is a sink I believe).
Here is how I attempt to do so:
int width = 320;
int height = 240;
Mat frame(height, width, CVX_8UC3, Scalar(0, 0, 255));
cvtColor(frame, frame, CVX_BGR2YUV);
int fourcc = CVX_FOURCC('Y', 'U', 'Y', '2');
cout << "Trying to open video for write: " << FLAGS_out_video << endl;
VideoWriter outputVideo = VideoWriter(
FLAGS_out_video, fourcc, 30, frame.size());
if (!outputVideo.isOpened()) {
cerr << "Could not open the output video for write: " << FLAGS_out_video
<< endl;
}
As far as I know video output format should be YUYV (which is equal to YUY2 in OpenCV). Please correct me if I'm wrong. In my code I'm not writing into outputVideo anything yet, just trying to open it for write, but I keep getting outputVideo.isOpened()==false for some reason, no additional errors/info in the output:
root#blah:~$ main --uid='' --in_video='0' --out_video='/dev/video2'
Trying to open video for write: /dev/video2
Could not open the output video for write: /dev/video2
I'd appreciate any advice or help on how to debug/resolve this issue. Thank you in advance!
I am using the following code for capturing video frames from a USB webcam using openCV3 in MS VC++ 2012. But the problem is that sometimes I am able to display the captured frames # 30 fps but sometimes I get black frames with a very low fps (or with a high delay). In other words, the program works randomly. Do you know how I can solve this problem? I tried different solutions suggested in stackoverflow or some other places but none of them solved the problem.
VideoCapture v(1);
v.set(CV_CAP_PROP_FRAME_WIDTH, 720);
v.set(CV_CAP_PROP_FRAME_HEIGHT, 480);
if(!v.isOpened()){
cout << "Error opening video stream or file" << endl;
return;
}
Mat Image;
namedWindow("win",1);
while(1){
v >> Image;
imshow("win", Image);
}
try this:
while(1){
v >> Image;
imshow("win", Image);
char c=waitKey(10);//add a 10ms delay per frame to sync with cam fps
if(c=='b')
{
break;//break when b is pressed
}
}
I have multiple recorded video samples, when I run these through my program it returns the FPS among other things. It is accurate enough for all of my video samples (see table below) but when I run a video sample taken through my smartphone it is returning the FPS at 90000, this happens with every video that captured through my smartphone so it is not just a problem with a single video file.
File Actual FPS OpenCV FPS ffmpeg FPS
action-60fps 60 59 60
action-24fps 24 24 24
phone_panning 29 90000 29
What is causing this problem?
EDIT: Managed to forget to add my code...
VideoCapture capture(argv[1]);
Mat frame;
if(capture.isOpened()) {
int fps = capture.get(CV_CAP_PROP_FPS);
int width = capture.get(CV_CAP_PROP_FRAME_WIDTH);
int height = capture.get(CV_CAP_PROP_FRAME_HEIGHT);
cout << "FPS: " << fps << ", width: " << width << ", height: " << height << endl;
VideoWriter writer("output.mpg",
CV_FOURCC('P','I','M','1'), fps, cvSize(width, height), 0); // 0 means gray, 1 means color
if(writer.isOpened()) {
while(true) {
capture >> frame;
if(!frame.empty()) {
imshow("Video", frame);
}
Mat frame_gray = frame.clone();
cvtColor(frame, frame_gray, CV_RGB2GRAY);
writer << frame_gray;
int key = waitKey(25);
if((char)key == 'q') {
break;
}
}
}
}
I had the same problem with opencv in calculating the FPS and number of frames in a video. (It was returning 90,000 for FPS and 5,758,245 for frame count for a 64-second video!!)
According to this answer:
OpenCV captures only a fraction of the frames from a video file
it's an opencv issue and they are working on it.
Another reason could be a problem with file header, mine was caused by converting video format. I solved it by using original video format mp4, instead of avi.