I am trying to make a feature matching algorithm with OpenCV on CUDA.
I am using the ORB features.
The main problem I have is that I am able to well compute the ORB keypoints but the descriptors are always empty. I tried all the ways but without success. I tried with the last version of opencv (3.4.0) and the 3.2.0 version.
This is the simplified version of my code:
cv::Ptr<cv::cuda::ORB> orb_gpu = cv::cuda::ORB::create(5000);
std::vector<cv::KeyPoint> keypoints;
cv::cuda::GpuMat descriptors;
cv::cuda::GpuMat img_gpu(curr_frame);
orb_gpu->detect(img_gpu, keypoints);
orb_gpu->compute(img_gpu, keypoints, descriptors);
cv::Mat desc_cpu;
descriptors.upload(desc_cpu);
The problem is with the direction of memory copy. You are calling upload instead of download function to copy descriptors from device to host. The following line is the culprit:
descriptors.upload(desc_cpu);
Replace it with:
descriptors.download(desc_cpu);
upload copies from host to device whereas download copies from device to host.
I have tested and verified the example with OpenCV 3.4 compiled with CUDA 8 on Ubuntu 14.04.
Related
I want to call cv::findCountours on some cv::UMat (OpenCL activated) and get the results into std::vector<std::vector<cv::Point>>.
std::vector<std::vector<cv::Point>> contours;
cv::findContours(frame_umat, contours, cv::RETR_LIST, cv::ContourApproximationModes::CHAIN_APPROX_SIMPLE);
Will OpenCV still able to optimise it using OpenCL even if I am using std::vector? Is there any advantage of using a special container like for example cv::UMat as a container(not an image)?
By tracing OpenCV cv::findContours function, I realized that it is not optimised using OpenCL at all (not CUDA also). The only implementation it has is SSE2 as far as I found in OpenCV 3.1
I use the following code for feature detection:
auto detector = cv::xfeatures2d::SURF::create();
std::vector<cv::KeyPoint> keypoints;
for (const cv::Mat& image : images) {
detector->detect(image, keypoints);
process(keypoints);
}
Now I try to use OpenCL version of SURF. I modified my code to work with cv::UMat. Unfortunately, execution time does not change. HAS_OPENCL is set, cv::ocl::useOpenCL() is true. What could go wrong?
I know that I'm a little bit late for this, but looking at surf.cpp,
SURF_Impl::detectAndCompute starts with a #ifdef HAVE_OPENCL, so I think that if you built OpenCV with OpenCL enabled, OpenCL is used by default
There are several OpenCV CPU functions which have a direct CUDA counterpart like cv::cvtColor & cv::cuda::cvtColor.
But I found no direct or indirect (GPU) Cuda counterpart for cv::findContours CPU.
Isn't there a OpenCV Cuda function similar to findContours? Or does findContours work on both cv::Mat and cv::cuda::GpuMat?
Unfortunately, not. Not even in the latest OpenCV 3.2.0 version. But they have this update, as shown here: https://github.com/opencv/opencv/wiki/ChangeLog
findContours can now find contours on a 32-bit integer image of labels (not only on a black-and-white 8-bit image). This is a step towards more convenient connected component analysis.
No. OpenCV 4.6.0 does not have it.
Nobody has dared to implement this with CUDA for years.
I am trying to calculate and display opticalFlowBM but having serious troubles. I am using Cuda 7.5 and OpenCV 3.0.0.
I was trying to rewrite it to 3.0.0.
Sizes of input arguments do not match in cvcalcopticalflowbm opencv 2.4.7
My code (imgA and imgB are Mats read from VideoCapture // cap.read(ImgA)):
cvtColor(imgA,imgA,COLOR_BGR2GRAY);
cvtColor(imgB,imgB,COLOR_BGR2GRAY);
CvSize size = imgA.size();
CvSize velsize = CvSize(size.width,size.height);
size.width/=BS;
size.height/=BS;
//result=Mat(size,IPL_DEPTH_8U);
//result=Scalar(255,255,255);
Mat velx,vely;
velx=Mat(velsize,IPL_DEPTH_32F);
vely=Mat(velsize,IPL_DEPTH_32F);
GpuMat imgAA;
imgAA.upload(imgA);
GpuMat imgBB;
cvtColor(imgAA,imgAA,COLOR_BGR2GRAY);
cvtColor(imgBB,imgBB,COLOR_BGR2GRAY);
imgBB.upload(imgB);
GpuMat velxx;
velxx.upload(velx);
GpuMat velyy;
velyy.upload(vely);
GpuMat buffor;
calcOpticalFlowBM(imgBB,imgAA,cvSize(BS,BS),cvSize(1,1),cvSize(1,1),0,velxx,velyy,buffor);
Mat abc(imgAA);
namedWindow("abc");
imshow("abc",imgAA);
But keep getting error:
OpenCV Error: The function/feature is not implemented (You
should explicitly call download method for cuda::GpuMat
object) in getMat,file
/home/pentaquark/Pulpit/opencv/opencv-3.0.0-
alpha/modules/core/src/matrix.cpp, line 1193
terminate called after throwing an instance of
'cv::Exception'what():
/home/pentaquark/Pulpit/opencv/opencv-3.0.0-
alpha/modules/core/src/matrix.cpp:1193: error: (-213) You
should explicitly call download method for cuda::GpuMat
object in function getMat
Could anyone help me?
Has anyone managed to get the Brox dense optical flow algorithm in OpenCV working?
Here is some code:
{
// Load images
cv::Mat PreviousFrameGrayFloat; // Has an image in format CV_32FC1
cv::Mat CurrentFrameGrayFloat; // Has an image in format CV_32FC1
// Upload images to GPU
cv::gpu::GpuMat PreviousFrameGPU(PreviousFrameGrayFloat);
cv::gpu::GpuMat CurrentFrameGPU(CurrentFrameGrayFloat);
// Prepare receiving variables
cv::gpu::GpuMat FlowXGPU;
cv::gpu::GpuMat FlowYGPU;
// Create optical flow object
cv::gpu::BroxOpticalFlow OpticalFlowGPU = cv::gpu::BroxOpticalFlow(0.197f, 0.8f, 50.0f, 10, 77, 10);
// Perform optical flow
OpticalFlowGPU(PreviousFrameGPU, CurrentFrameGPU, FlowXGPU, FlowYGPU); // EXCEPTION
// Exception in opencv_core244d!cv::GlBuffer::unbind
// Download flow from GPU
cv::Mat FlowX;
cv::Mat FlowY;
FlowXGPU.download(FlowX);
FlowYGPU.download(FlowY);
}
I get an exception, as commented above, when I try and use the cv::gpu::BroxOpticalFlow object, I have the same problem with cv::gpu::PyrLKOpticalFlow but only the dense version not the sparse one, and cv::gpu::FarnebackOpticalFlow works fine...
Weird.
Entire exception is:
Exception at 0x7c812fd3, code: 0xe06d7363: C++ exception, flags=0x1 (execution cannot be continued) in opencv_core244d!cv::GlBuffer::unbind
I'm using a debug build with debug libraries, using OpenCV 2.4.4 however the code also raises an exception in OpenCV 2.4.3.
When I use OpenCV 2.4.3 I get this exception:
Exception at 0x7c812fd3, code: 0xe06d7363: C++ exception, flags=0x1 (execution cannot be continued) in opencv_core243d!cv::SparseMat::erase
OpenGL is required, but there may also be a problem with your scale parameter (50.0f). It seems too large. As I understand, this should be less than 1. If it is a large number, the algorithm fills up GPU memory quickly. Plus, it may not make sense to use an expanding (upside-down?) image pyramid. Not perfectly sure though.
After experimenting with other versions of OpenCV and following the information contained here: http://stuartjames.info/Journal/opencv-brox-optical-flow-sample-possible-fix.aspx
It looks like I need to recompile OpenCV with OpenGL.
It seems that you can test whether your install of OpenCV has OpenGL by running the command: cv::gpu::setGlDevice(0); if this fails with a weird exception like the one above then you need to recompile.
OpenGL is required as the newer optical flow algorithms in OpenCV map data to OpenGL textures to (I assume) speed up operations, or maybe to just make the code simpler.
So, after all this, the solution is to recompile OpenCV with OpenGL. This can be achieved by ticking the WITH_OPENGL box when using CMake to configure the build.
If you want to use the GPU version of the video reader make sure you also tick WITH_NVCUVID this is included with CUDA but not included unless you request it...
-- EDIT --
After the comments from paul I've corrected the scale factor in my code from the question.
Here is the complete code I'm testing, for nkint:
{
// Load images
cv::Mat PreviousFrameGray = cv::imread("Input1.png", 0);
cv::Mat CurrentFrameGray = cv::imread("Input2.png", 0);
cv::Mat PreviousFrameGrayFloat; // Has an image in format CV_32FC1
cv::Mat CurrentFrameGrayFloat; // Has an image in format CV_32FC1
PreviousFrameGray.convertTo(PreviousFrameGrayFloat, CV_32FC1, 1.0/255.0);
CurrentFrameGray.convertTo(CurrentFrameGrayFloat, CV_32FC1, 1.0/255.0);
// Upload images to GPU
cv::gpu::GpuMat PreviousFrameGPU(PreviousFrameGrayFloat);
cv::gpu::GpuMat CurrentFrameGPU(CurrentFrameGrayFloat);
// Prepare receiving variables
cv::gpu::GpuMat FlowXGPU;
cv::gpu::GpuMat FlowYGPU;
// Create optical flow object
cv::gpu::BroxOpticalFlow OpticalFlowGPU = cv::gpu::BroxOpticalFlow(0.197f, 50.0f, 0.8f, 10, 77, 10);
// Perform optical flow
OpticalFlowGPU(PreviousFrameGPU, CurrentFrameGPU, FlowXGPU, FlowYGPU);
// Download flow from GPU
cv::Mat FlowX;
cv::Mat FlowY;
FlowXGPU.download(FlowX);
FlowYGPU.download(FlowY);
// Use FlowX and FlowY in further processing
//...
}