I am using KCF tracker which works well for sometime but breaks when I re-create a new project doing something else. For example, I use the tracker to track an object while meantime extract some feature from the object but it really does not affect the whole processing of the tracker, and this error comes as well when the code of feature extraction is commented. The error goes here
cv::Mat real(cv::Mat img)
{
std::vector<cv::Mat> planes(3);
//std::cout << img.size() << std::endl;
//std::cout << img.channels() << std::endl;
//planes.resize(img.channels()+1);
cv::split(img, planes); // memory writing problem? Why?
return planes[0];
}
KCFD.exe!FFTTools::real(cv::Mat img={...}) Line 135 C++
BTW, the program is built in debug mode of VS2013. OpenCV is 2.4.13.
I have also modified it to give a 3 or 2-element vector of planes in advance but it does not change.
Related
EDIT: I use PCL one in all installer 1.12.0 on Windows 11. Used cmake to build a project with librealsense and pcl. My IDE is Visual Studio 2019
I get multiple point clouds out of multiple depth frames from 2 cameras. I want to use some filters on each point cloud, so I loop through the vector of point clouds with a for loop. Before the first iteration of the loop can finish I get a Debug Assertion Failed: _CrtIsValidHeapPointer(block) error. This only happens in debug mode but not in release mode.
std::vector<pcl::PointCloud<pcl::PointXYZRGB>::Ptr, Eigen::aligned_allocator <pcl::PointCloud<pcl::PointXYZRGB>::Ptr >> filtered_clouds1;
std::vector<pcl::PointCloud<pcl::PointXYZRGB>::Ptr, Eigen::aligned_allocator <pcl::PointCloud<pcl::PointXYZRGB>::Ptr >> filtered_clouds2;
//Jede aufgenommene Punktwolke filtern und in vector speichern
for (int i = 0; i < pcl_points1_vector.size(); i++) {
pcl::PointCloud<pcl::PointXYZRGB>::Ptr cloud_filtered(new pcl::PointCloud<pcl::PointXYZRGB>);
pcl::PointCloud<pcl::PointXYZRGB>::Ptr cloud_filtered2(new pcl::PointCloud<pcl::PointXYZRGB>);
complete_filter(pcl_points1_vector[i], cloud_filtered);
complete_filter(pcl_points2_vector[i], cloud_filtered2);
filtered_clouds1.push_back(cloud_filtered);
filtered_clouds2.push_back(cloud_filtered2); //After this the error occurs
}
complete_filter is made out of 2 filters:
void complete_filter(const pcl::PointCloud<pcl::PointXYZRGB>::Ptr& cloud, pcl::PointCloud<pcl::PointXYZRGB>::Ptr& cloud_filtered) {
pcl::PointCloud<pcl::PointXYZRGB>::Ptr planes_filtered(new pcl::PointCloud<pcl::PointXYZRGB>);
remove_planes(cloud, planes_filtered);
downsample_PC(planes_filtered, cloud_filtered);
}
void downsample_PC(const pcl::PointCloud<pcl::PointXYZRGB>::Ptr& cloud, pcl::PointCloud<pcl::PointXYZRGB>::Ptr& cloud_filtered) {
// Downsampling
cout << "Downsampling each pointcloud... \n";
pcl::VoxelGrid<pcl::PointXYZRGB> downsample;
downsample.setInputCloud(cloud);
downsample.setLeafSize(0.001f, 0.001f, 0.001f);
downsample.setFilterFieldName("z");
downsample.setFilterLimits(0, 1.3);
downsample.filter(*cloud_filtered);
}
and
void filter_radius(const pcl::PointCloud<pcl::PointXYZRGB>::Ptr& cloud, pcl::PointCloud<pcl::PointXYZRGB>::Ptr& cloud_filtered){
pcl::RadiusOutlierRemoval<pcl::PointXYZRGB> outrem;
outrem.setInputCloud(cloud);
outrem.setRadiusSearch(0.01);
outrem.setMinNeighborsInRadius(13);
outrem.filter(*cloud_filtered);
std::cout << "Cloud size after removing with RadiusOutlierRemoval: " << cloud_filtered->points.size() << std::endl;}
If I put the point clouds created outside the for loop the error vanishes but the vector is filled with the same pointer over and over. I have 0 clue on how to tackle this problem and am too inexperienced with C++. Does anyone have a clue how I can solve this problem?
I will just copy paste my own answer in the comments here:
ALRIGHT, some things I changed: I upgraded to pcl 1.12.1 which didnt change anything, removed the allocator because it doesn't make much difference on the surface (issue is somewhere else) and instead of passing references I copy the pointclouds into the vector which fixed the issue. After the for loop the pointers just point to an invalid address which I guess makes sense since the pointclouds are created inside the for. I am sure there is a better solution (and copying is probably the worst case here) but it works for me I guess...
I ran the code below to check for the performance difference between GPU and CPU usage. I am calculating the Average time for cv::cvtColor() function. I make four function calls:
Just_mat()(Without using OpenCL for Mat object)
Just_UMat()(Without using OpenCL for Umat object)
OpenCL_Mat()(using OpenCL for Mat object)
OpenCL_UMat() (using OpenCL for UMat object)
for both CPU and GPU.
I did not find a huge performance difference between GPU and CPU usage.
int main(int argc, char* argv[])
{
loc = argv[1];
just_mat(loc);// Calling function Without OpenCL
just_umat(loc);//Calling function Without OpenCL
cv::ocl::Context context;
std::vector<cv::ocl::PlatformInfo> platforms;
cv::ocl::getPlatfomsInfo(platforms);
for (size_t i = 0; i < platforms.size(); i++)
{
//Access to Platform
const cv::ocl::PlatformInfo* platform = &platforms[i];
//Platform Name
std::cout << "Platform Name: " << platform->name().c_str() << "\n" << endl;
//Access Device within Platform
cv::ocl::Device current_device;
for (int j = 0; j < platform->deviceNumber(); j++)
{
//Access Device
platform->getDevice(current_device, j);
int deviceType = current_device.type();
cout << "Device name: " << current_device.name() << endl;
if (deviceType == 2)
cout << context.ndevices() << " CPU devices are detected." << std::endl;
if (deviceType == 4)
cout << context.ndevices() << " GPU devices are detected." << std::endl;
cout << "===============================================" << endl << endl;
switch (deviceType)
{
case (1 << 1):
cout << "CPU device\n";
if (context.create(deviceType))
opencl_mat(loc);//With OpenCL Mat
break;
case (1 << 2):
cout << "GPU device\n";
if (context.create(deviceType))
opencl_mat(loc);//With OpenCL UMat
break;
}
cin.ignore(1);
}
}
return 0;
}
int just_mat(string loc);// I check for the average time taken for cvtColor() without using OpenCl
int just_umat(string loc);// I check for the average time taken for cvtColor() without using OpenCl
int opencl_mat(string loc);//ocl::setUseOpenCL(true); and check for time difference for cvtColor function
int opencl_umat(string loc);//ocl::setUseOpenCL(true); and check for time difference for cvtColor function
The output(in miliseconds) for the above code is
__________________________________________
|GPU Name|With OpenCL Mat | With OpenCl UMat|
|_________________________________________|
|--Carrizo---|------7.69052 ------ |------0.247069-------|
|_________________________________________|
|---Island--- |-------7.12455------ |------0.233345-------|
|_________________________________________|
__________________________________________
|----CPU---|With OpenCL Mat | With OpenCl UMat |
|_________________________________________|
|---AMD---|------6.76169 ------ |--------0.231103--------|
|_________________________________________|
________________________________________________
|----CPU---| WithOut OpenCL Mat | WithOut OpenCl UMat |
|_______________________________________________|
|----AMD---|------7.15959------ |------------0.246138------------ |
|_______________________________________________|
In code, using Mat Object always runs on CPU & using UMat Object always runs on GPU, irrespective of the code ocl::setUseOpenCL(true/false);
Can anybody explain the reason for all output time variation?
One more question, i didn't use any OpenCL specific .dll with .exe file and yet GPU was used without any error, while building OpenCV with Cmake i checked With_OpenCL did this built all OpenCL required function within opencv_World310.dll ?
In code, using Mat Object always runs on CPU & using UMat Object always runs on GPU, irrespective of the code ocl::setUseOpenCL(true/false);
I'm sorry, because I'm not sure if this is a question or a statement... in either case it's partially true. In 3.0, for the UMat, if you don't have a dedicated GPU then OpenCV just runs everything on the CPU. If you specifically ask for Mat you get it on the CPU. And in your case you have directed both to run on each of your GPUs/CPU by selecting each specifically (more on "choosing a CPU below)... read this:
Few design choices support the new architecture:
A unified abstraction cv::UMat that enables the same APIs to be implemented using CPU or OpenCL code, without a requirement to call
OpenCL accelerated version explicitly. These functions use an OpenCL
-enabled GPU if exists in the system, and automatically switch to CPU
operation otherwise.
The UMat abstraction enables functions to be called asynchronously.
Unlike the cv::Mat of the OpenCV version 2.x, access to the underlyi
ng data for the cv::UMat is performed through a method of class, and not though its data member. Such an approach enables the
implementation to explicitly wait for GPU completion only when CPU code absolutely needs the result.
The UMat implementation makes use of CPU-GPU shared physical memory available on Intel SoCs, including allocations that come from pointers passed into OpenCV.
I think there also might be a misunderstanding about "using OpenCL". When you use an UMat, you are specifically trying to use the GPU. And, I'll plead some ignorance here, as a result I believe that CV is using some of the CL library to make that happen automatically... as a side in 2.X we had cv::ocl to specifically/manually do this, so be careful if you are using that 2.X legacy code in 3.X. There are reasons to do it, but they are not always straightforward. But, back on topic, when you say,
with OpenCL UMat
you are potentially being redundant. The CL code you have in your snippet is basically finding out what equipment is installed, how many there are, what their names are, and choosing which to use... I'd have to dig through the way it is instantiated, but perhaps when you make it UMat it automatically sets OpenCL to True? (link) That would definitely support the data you presented. You could probably test that idea by checking what the state of ocl::setUseOpenCL after you set it to false and then use an UMat.
Finally, I'm guessing your CPU has a built in GPU. So it is running parallel processing with OpenCL and not paying a time penalty to travel to the seperate/dedicated GPU and back, hence your perceived performance increase over the GPUs (since it is not technically the CPU running it)... only when you are specifically using the Mat is the CPU only being used.
Your last question, I'm not sure... this is my speculation: OpenCL architexture exists on the GPU, when you install CV with CL you are installing the link between the two libraries and associated header files. I'm not sure which dll files you need to make that magic happen.
I'm wondering what the best way to detect a high DPI display is. Currently I'm trying to use SDL_GetDisplayDPI (int, *float, *float, *float), however this has only returned errors on the two different computers I tested with (MacBook Pro running OS X 10.11.5 and iMac running macOS 10.12 Beta (16A238m)). For reference, my code is bellow.
float diagDPI = -1;
float horiDPI = -1;
float vertDPI = -1;
int dpiReturn = SDL_GetDisplayDPI (0, &diagDPI, &horiDPI, &vertDPI);
std::cout << "GetDisplayDPI() returned " << dpiReturn << std::endl;
if (dpiReturn != 0)
{
std::cout << "Error: " << SDL_GetError () << std::endl;
}
std::cout << "DDPI: " << diagDPI << std::endl << "HDPI: " << horiDPI << std::endl << "VDPI: " << vertDPI << std::endl;
Unfortunately, this is only giving me something like this:
/* Output */
GetDisplayDPI() returned -1
Error:
DDPI: -1
HDPI: -1
VDPI: -1
Not Retina
I also tried comparing the OpenGL drawable size with the SDL window size, but SDL_GetWindowSize (SDL_Window, *int, *int) is returning 0s, too. That code is bellow, followed by the output.
int gl_w;
int gl_h;
SDL_GL_GetDrawableSize (window, &gl_w, &gl_h);
std::cout << "GL_W: " << gl_w << std::endl << "GL_H: " << gl_h << std::endl;
int sdl_w;
int sdl_h;
SDL_GetWindowSize (window, &sdl_w, &sdl_h);
std::cout << "SDL_W: " << sdl_w << std::endl << "SDL_H: " << sdl_h << std::endl;
/* Output */
GL_W: 1280
GL_H: 720
SDL_W: 0
SDL_H: 0
It's entirely possible that I'm doing something wrong here, or making these calls in the wrong place, but I think more likely is that I'm on the wrong track entirely. There's a hint to disallow high-dpi canvases, so there's probably a simple bool somewhere, or something that I'm missing. I have certainly looked through the wiki, and checked Google, but I can't really find any help for this. Any suggestions or feedback are welcome!
Thank you for your time!
I know I'm not answering your question directly, and want to reiterate one thing you tried.
On a Macbook pro, when an SDL window is on an external display, SDL_GetWindowSize and SDL_GL_GetDrawableSize return the same values. If the window is on a Retina screen, they're different. Specifically, the drawable size is 2x larger in each dimension.
I was using a .framework installation of SDL when I encountered this issue. For an unrelated reason, I trashed the .framework SDL files (image and ttf as well), and built SDL from source (thus transitioning to a "unix-style" SDL-installation). To my surprise, SDL_GetDisplayDPI () is now returning 0, setting the values of DDPI, HDPI, and VDPI, to 109 on a non-retina iMac, and 113.5 on a retina MacBook Pro. I'm not certain that these are correct/accurate, but it is consistent between launches, so I'll work with it.
At this point, I'm not sure if it was a bug, which has been fixed in the repo, or was an issue with my .framework files. On a somewhat unrelated note, SDL_GetBasePath () and SDL_GetPrefPath (), which also weren't working, now return the expected values. If you're also experiencing any of these problems on macOS, try compiling and installing SDL from source (https://hg.libsdl.org/SDL).
Thanks for your input, everyone!
This might have been asked several times before. But I feel mine is a bit different and since I lack complete understanding of concepts, I am posting it again.
I am working on opencv code written in C++ on Ubuntu that can match vein patterns. I have captured 4 vein images. In my program, I would like to capture a new image from the IR camera and compare it with the images in the images directory. I am planning to use fuzzy C clustering algorithm for my matching. I have created a user menu in which one option is comparing my algorithm with FLANN, SIFT, etc. This comparison is based on the time taken. How do you calculate the time taken for computation?
I am completely new to Fuzzy clustering and any tutorials/Sample codes that might help is greatly appreciated.
Also, can you please suggest how to go about comparing a file captured from camera to a file in directory in linux?
Edit 1: Have uploaded two sample vein patterns with their Canny Edge Detectors.
Vein Pattern 1
Vein Pattern 2
www.i.imgur.com/mvt3kIy.jpg (Canny Edge 1)
www.i.imgur.com/8GwaLTu.jpg (Canny Edge 2)
Please suggest some methods to compare the same.
To calculate the time elapsed between a set of instructions,
#include <time>
int main()
{
// whatever code
clock_t tstart = clock();
/// more code and implementations
cout << "Processing time = " << (double)(clock() - tstart)/(CLOCKS_PER_SEC) << " second(s)" << endl;
}
There are many ways in which you can compare 2 files; if you post some images, I might be able to guide you further. You might try and read some of OpenCV documentation and related papers. This link will give you a head start to feature description..
I use this function for timings:
#include <sys/time.h>
#include <iostream>
inline long getMilliSecs()
{
timeval t;
gettimeofday(&t, NULL);
return t.tv_sec*1000 + t.tv_usec/1000;
}
int main()
{
long start_time = getMilliSecs();
///
//do stuff;
///
long end_time = getMilliSecs();
std::cout << ((double)(end_time - start_time))/1000 << " seconds" << std::endl;
}
I'm writing a code in which I read and image and process it and get a Mat of double/float. I'm saving it to a file and later on, I'm reading it from that file.
When I use double, the space it requires is 8MB for 1Kx1K image, when I use float it is 4MB. So I want to use float.
Here is my code and output:
Mat data = readFloatFile("file_location");
cout << data.at<float>(0,0) << " " << data.at<double>(0,0);
When I run this code in the DEBUG mode, the print out for float is -0 and double gives exception namely assertion failed. But when I use RELEASE mode the print out for float is -0 and 0.832 for double which is true value.
My question is why I cant get output when I use data.at<float>(0,0) and why I don't get exception when I use data.at<double>(0,0) in RELEASE mode which is supposed to be the case?
EDIT: Here is my code which writes and reads
void writeNoiseFloat(string imageName,string fingerprintname) throw(){
Mat noise = getNoise(imageName);
FILE* fp = fopen(fingerprintname.c_str(),"wb");
if (!fp){
cout << "not found ";
perror("fopen");
}
float *buffer = new float[noise.cols];
for(int i=0;i<noise.rows;++i){
for(int j=0;j<noise.cols;++j)
buffer[j]=noise.at<float>(i,j);
fwrite(buffer,sizeof(float),noise.cols,fp);
}
fclose(fp);
free(buffer);
}
void readNoiseFloat(string fpath,Mat& data){
clock_t start = clock();
cout << fpath << endl;
FILE* fp = fopen(fpath.c_str(),"rb");
if (!fp)perror("fopen");
int size = 1024;
data.create(size,size,CV_32F);
float* buffer= new float[size];
for(int i=0;i<size;++i) {
fread(buffer,sizeof(float),size,fp);
for(int j=0;j<size;++j){
data.at<float>(i,j)=buffer[j];
cout << data.at<float>(i,j) << " " ;
cout << data.at<double>(i,j);
}
}
fclose(fp);
}
Thanks in advance,
The first of all, you can not use the float and double in one cv::Mat as storage itself is only array of bytes. Size of this array will be different for matrix of float and matrix of double.
So, you have to decide what you are using.
Essentially, data.at<type>(x,y) is equivalent to (type*)data_ptr[x][y] (note this is not exact code, its purpose is to show what is happening)
EDIT:
On the basis of code you added you are creating matrix of CV_32F this means that you must use float to write and read and element. Using of double causes reinterpretation of value and will definitely give you an incorrect result.
Regarding to assertion, I am sure that inside the cv::MAT::at<class T> there is a kind of following code:
assert(sizeof<T>==this.getDepth());
Usually asserts are compiled only in DEBUG mode, so that's why you do not give this error in RELEASE.
EDIT2:
Not regarding to issue, but never use free() with new or delete with malloc(). The result can be a hardly debugging issue.
So please use delete[] for buffer.
Difference between debug & release:
There's a bug in your code. It just doesn't show up in Release mode. That is what the debugger is for. Debugger tells you if there's any bug/issue with the code, Release just runs through it...
Also the compiler optimizes your code to run faster and is therefore smaller, the debugger uses more size on your HD because you can actually DEBUG it.
Release initializes your un-initialized variables to 0. This may vary on different compilers.