I use the following code for feature detection:
auto detector = cv::xfeatures2d::SURF::create();
std::vector<cv::KeyPoint> keypoints;
for (const cv::Mat& image : images) {
detector->detect(image, keypoints);
process(keypoints);
}
Now I try to use OpenCL version of SURF. I modified my code to work with cv::UMat. Unfortunately, execution time does not change. HAS_OPENCL is set, cv::ocl::useOpenCL() is true. What could go wrong?
I know that I'm a little bit late for this, but looking at surf.cpp,
SURF_Impl::detectAndCompute starts with a #ifdef HAVE_OPENCL, so I think that if you built OpenCV with OpenCL enabled, OpenCL is used by default
Related
I am trying to setup a planar image stitching app, but if I give the stitcher below a PlaneWarper, the app crashes with a bad access exception. I also learned that ORB feature finding is best for planar stitching, but using an OrbFeatureFinder also causes the app to crash within the stitch function. I know I am not fully aware of how the stitching pipeline works, so if someone could help me understand the issue here, I would be grateful.
vector<Mat> imgs;
cv::Mat stitch (vector<Mat>& images)
{
imgs = images;
Mat pano;
Ptr<Stitcher> stitcher = Stitcher::create(Stitcher::SCANS, true);
stitcher->setPanoConfidenceThresh(0.8f);
stitcher->setFeaturesMatcher(makePtr<cv::detail::AffineBestOf2NearestMatcher>(true, true, 0.8f));
Stitcher::Status status = stitcher->stitch(imgs, pano);
if (status != Stitcher::OK)
{
cout << "Can't stitch images, error code = " << int(status) << endl;
//return 0;
}
return pano;
}
I have tested on my Mac the stitching_detailed program with Orb feature finding and Planar warping, and it gave me great results, so I attempted to run stitching_detailed.cpp in the iOS app interface, but that cause all types of crashes, so I am trying this way now.
The stitching works well, but there is some distortion here and there and using the ORB feature finding with the Planar warping eliminated it on my Mac.
I only did a cursory look, but I suspect your issue lies with how OpenCV is structured. When running on a Mac, it can utilize the GPU via OpenCL. However, when running on an iOS device, it cannot use OpenCL since it is unsupported. Because of this, it must use the CPU based implementation found here.
https://github.com/opencv/opencv/blob/808ba552c532408bddd5fe51784cf4209296448a/modules/stitching/src/stitcher.cpp
You will see the variable try_use_gpu used extensively, and based on the way it configures and runs, this is likely the culprit. While I cannot say for certain in your case, I have found previously that there is iOS specific functionality that is broken, or simply even non-existant. With that said, you may want to file an issue with the project in the hope that someone can pick it up and fix it.
Use open cv 2.4.9 version of stitching for iOS app. Also, use this code it will work great for iOS App
https://github.com/foundry/OpenCVSwiftStitch
I already spend too much time to fixed the crash but now it got fixed.
I am trying to make a feature matching algorithm with OpenCV on CUDA.
I am using the ORB features.
The main problem I have is that I am able to well compute the ORB keypoints but the descriptors are always empty. I tried all the ways but without success. I tried with the last version of opencv (3.4.0) and the 3.2.0 version.
This is the simplified version of my code:
cv::Ptr<cv::cuda::ORB> orb_gpu = cv::cuda::ORB::create(5000);
std::vector<cv::KeyPoint> keypoints;
cv::cuda::GpuMat descriptors;
cv::cuda::GpuMat img_gpu(curr_frame);
orb_gpu->detect(img_gpu, keypoints);
orb_gpu->compute(img_gpu, keypoints, descriptors);
cv::Mat desc_cpu;
descriptors.upload(desc_cpu);
The problem is with the direction of memory copy. You are calling upload instead of download function to copy descriptors from device to host. The following line is the culprit:
descriptors.upload(desc_cpu);
Replace it with:
descriptors.download(desc_cpu);
upload copies from host to device whereas download copies from device to host.
I have tested and verified the example with OpenCV 3.4 compiled with CUDA 8 on Ubuntu 14.04.
I am beginner using opencv, and was using the FAST feature detector in opencv to simply get the keypoints of an image.
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/features2d.hpp"
std::vector<cv::KeyPoint> kp;
int main()
{
cv::Mat img = cv::imread("im.jpg", 0);
cv::FAST(img, kp, 100, true);
cv::Mat img2;
cv::drawKeypoints(img, kp, img2 , CV_RGB(255, 0, 0));
cv::imshow("Display", img2);
cv::waitKey(0);
cv::destroyWindow("Display");
return 0;
}
But when debugging the code (Debug mode):
I found out that the vector kp (passed as the keypoints vector) in to FAST() has a large size ( kp = { size=658812287755660302 }) which is not possible. But the image denoting correct keypoints is generated. The above mentioned access violation error message comes at the end of debugging.
In Release mode:
The code runs fine at first showing the keypoints in the image. At the end when image window is closed, the program is crashed. When debugged, kp has a reasonable size (kp = { size=427 }).
When the code is run in debug mode without debugging, an application error pops out stating that memory at 0xffffffff could not be read.
The call stack is as shown here, which doesn't make any sense to me.
What am I doing wrong? Is it related to the way I'm initializing the vector or something else?
I was unable to reproduce this issue either using my own image or using the image you provided. I therefore have to conclude that there's a problem with your build environment.
Make sure that you've set up your project includes and libraries correctly, including the shared libraries that OpenCV uses by default.
Make sure that you're using Visual Studio 2015 (or the binary-compatible VS2017) to compile your code.
I want to call cv::findCountours on some cv::UMat (OpenCL activated) and get the results into std::vector<std::vector<cv::Point>>.
std::vector<std::vector<cv::Point>> contours;
cv::findContours(frame_umat, contours, cv::RETR_LIST, cv::ContourApproximationModes::CHAIN_APPROX_SIMPLE);
Will OpenCV still able to optimise it using OpenCL even if I am using std::vector? Is there any advantage of using a special container like for example cv::UMat as a container(not an image)?
By tracing OpenCV cv::findContours function, I realized that it is not optimised using OpenCL at all (not CUDA also). The only implementation it has is SSE2 as far as I found in OpenCV 3.1
Has anyone managed to get the Brox dense optical flow algorithm in OpenCV working?
Here is some code:
{
// Load images
cv::Mat PreviousFrameGrayFloat; // Has an image in format CV_32FC1
cv::Mat CurrentFrameGrayFloat; // Has an image in format CV_32FC1
// Upload images to GPU
cv::gpu::GpuMat PreviousFrameGPU(PreviousFrameGrayFloat);
cv::gpu::GpuMat CurrentFrameGPU(CurrentFrameGrayFloat);
// Prepare receiving variables
cv::gpu::GpuMat FlowXGPU;
cv::gpu::GpuMat FlowYGPU;
// Create optical flow object
cv::gpu::BroxOpticalFlow OpticalFlowGPU = cv::gpu::BroxOpticalFlow(0.197f, 0.8f, 50.0f, 10, 77, 10);
// Perform optical flow
OpticalFlowGPU(PreviousFrameGPU, CurrentFrameGPU, FlowXGPU, FlowYGPU); // EXCEPTION
// Exception in opencv_core244d!cv::GlBuffer::unbind
// Download flow from GPU
cv::Mat FlowX;
cv::Mat FlowY;
FlowXGPU.download(FlowX);
FlowYGPU.download(FlowY);
}
I get an exception, as commented above, when I try and use the cv::gpu::BroxOpticalFlow object, I have the same problem with cv::gpu::PyrLKOpticalFlow but only the dense version not the sparse one, and cv::gpu::FarnebackOpticalFlow works fine...
Weird.
Entire exception is:
Exception at 0x7c812fd3, code: 0xe06d7363: C++ exception, flags=0x1 (execution cannot be continued) in opencv_core244d!cv::GlBuffer::unbind
I'm using a debug build with debug libraries, using OpenCV 2.4.4 however the code also raises an exception in OpenCV 2.4.3.
When I use OpenCV 2.4.3 I get this exception:
Exception at 0x7c812fd3, code: 0xe06d7363: C++ exception, flags=0x1 (execution cannot be continued) in opencv_core243d!cv::SparseMat::erase
OpenGL is required, but there may also be a problem with your scale parameter (50.0f). It seems too large. As I understand, this should be less than 1. If it is a large number, the algorithm fills up GPU memory quickly. Plus, it may not make sense to use an expanding (upside-down?) image pyramid. Not perfectly sure though.
After experimenting with other versions of OpenCV and following the information contained here: http://stuartjames.info/Journal/opencv-brox-optical-flow-sample-possible-fix.aspx
It looks like I need to recompile OpenCV with OpenGL.
It seems that you can test whether your install of OpenCV has OpenGL by running the command: cv::gpu::setGlDevice(0); if this fails with a weird exception like the one above then you need to recompile.
OpenGL is required as the newer optical flow algorithms in OpenCV map data to OpenGL textures to (I assume) speed up operations, or maybe to just make the code simpler.
So, after all this, the solution is to recompile OpenCV with OpenGL. This can be achieved by ticking the WITH_OPENGL box when using CMake to configure the build.
If you want to use the GPU version of the video reader make sure you also tick WITH_NVCUVID this is included with CUDA but not included unless you request it...
-- EDIT --
After the comments from paul I've corrected the scale factor in my code from the question.
Here is the complete code I'm testing, for nkint:
{
// Load images
cv::Mat PreviousFrameGray = cv::imread("Input1.png", 0);
cv::Mat CurrentFrameGray = cv::imread("Input2.png", 0);
cv::Mat PreviousFrameGrayFloat; // Has an image in format CV_32FC1
cv::Mat CurrentFrameGrayFloat; // Has an image in format CV_32FC1
PreviousFrameGray.convertTo(PreviousFrameGrayFloat, CV_32FC1, 1.0/255.0);
CurrentFrameGray.convertTo(CurrentFrameGrayFloat, CV_32FC1, 1.0/255.0);
// Upload images to GPU
cv::gpu::GpuMat PreviousFrameGPU(PreviousFrameGrayFloat);
cv::gpu::GpuMat CurrentFrameGPU(CurrentFrameGrayFloat);
// Prepare receiving variables
cv::gpu::GpuMat FlowXGPU;
cv::gpu::GpuMat FlowYGPU;
// Create optical flow object
cv::gpu::BroxOpticalFlow OpticalFlowGPU = cv::gpu::BroxOpticalFlow(0.197f, 50.0f, 0.8f, 10, 77, 10);
// Perform optical flow
OpticalFlowGPU(PreviousFrameGPU, CurrentFrameGPU, FlowXGPU, FlowYGPU);
// Download flow from GPU
cv::Mat FlowX;
cv::Mat FlowY;
FlowXGPU.download(FlowX);
FlowYGPU.download(FlowY);
// Use FlowX and FlowY in further processing
//...
}