I am still pretty new to OpenCV and I've just recently come across the HoughLinesP function. First and foremost, my goal is to write code that will detect rectangles in a webcam. Currently, the code I have below only detect lines in general. However, I still have problems while I am debugging the program. Here is my code:
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>
using namespace cv;
using namespace std;
int main() {
int erosion_size = 0;
VideoCapture cam(0);
if (!cam.isOpened()) {
cout << "cannot open camera";
}
while (true) {
Mat frame;
cam.read(frame);
Mat gray, edge, draw, die;
cvtColor(frame, gray, CV_BGR2GRAY);
Canny(gray, edge, 100, 150, 3);
edge.convertTo(draw, CV_8U);
dilate(draw, die, Mat(), Point(-1, -1), 2, 1, 1);
erode(die, die, Mat(), Point(-1, -1), 1, 1, 1);
#if 0
vector<Vec2f> lines;
HoughLines(die, lines, 1, CV_PI / 180, 100, 0, 0);
for (size_t i = 0; i < lines.size(); i++)
{
float rho = lines[i][0], theta = lines[i][1];
Point pt1, pt2;
double a = cos(theta), b = sin(theta);
double x0 = a*rho, y0 = b*rho;
pt1.x = cvRound(x0 + 1000 * (-b));
pt1.y = cvRound(y0 + 1000 * (a));
pt2.x = cvRound(x0 - 1000 * (-b));
pt2.y = cvRound(y0 - 1000 * (a));
line(frame, pt1, pt2, Scalar(0, 0, 255), 3, CV_AA);
}
#else
vector<Vec4i> lines;
HoughLinesP(die, lines, 1, CV_PI / 180, 200, 50, 10);
for (size_t i = 0; i < lines.size(); i++)
{
Vec4i l = lines[i];
line(frame, Point(l[0], l[1]), Point(l[2], l[3]), Scalar(0, 0, 255), 3, CV_AA);
}
#endif
imshow("Canny", die);
imshow("original", frame);
if (waitKey(30) >= 0)
break;
}
return 0;
}
When I debug the program, the webcam pops up okay, but when I show a rectangular object with lines(piece of paper) it stops the program with a break point error. I concluded that my program stopped every time it found a line. When I choose to continue instead of breaking, it gives me this error:
Debug Assertion failed!
Program: ....Studio
2013/Projects/TrialRectangle/Debug/TrialRectangle.exe
File: f:/dd/vctools/crt/crtw32/misc/dbgheap.c
Line: 1332
Expression: _CrtIsValidHeapPointer(pUserData)
I played around with the HoughLinesP function and found that a high threshold parameter(ex. 500) seems to make my program run fine BUT it does not show any HoughLines at all in my webcam. If someone could explain why that is, that would be helpful as well!
Does anyone have any ideas as to how to solve this breakpoint error?
I got into this Debug Assertion too.
In my case, It was because that my project was statically compiled, Yet I used OpenCV dynamically with it's dll. So I change my project into dynamically compile, Solved the Problem.
It's Because that OpenCV object is allocated in different heap. And When this object is destructed, current run-time can's find that heap, This is why the Assertion is hit.
Related
I am trying to port this response to c++ but I am not able to get past this cryptic exception (see image below). Not sure what is the limiting factor. I imagine it is the image color format or the corners parameter but nothing seems to be working. If it is related to converting color format please provide a small code snippet.
The python code provided by Anubhav Singh is working great however I would like to develop in c++. Any help would be greatly appreciated.
I am using OpenCV04.2.0
void CornerDetection(){
std::string image_path = samples::findFile("../wing.png");
Mat img = imread(image_path);
Mat greyMat;
Mat dst;
cv::cvtColor(img, greyMat, COLOR_BGR2GRAY);
threshold(greyMat, greyMat, 0, 255, THRESH_BINARY | THRESH_OTSU);
cornerHarris(greyMat, dst, 9, 5, 0.04);
dilate(dst, dst,NULL);
Mat img_thresh;
threshold(dst, img_thresh, 0.32 * 255, 255, 0);
img_thresh.convertTo(img_thresh, CV_8UC1);
Mat labels = Mat();
Mat stats = Mat();
Mat centroids = Mat();
cv::connectedComponentsWithStats(img_thresh, labels, stats, centroids, 8, CV_32S);
TermCriteria criteria = TermCriteria(TermCriteria::EPS + TermCriteria::MAX_ITER, 30, 0.001);
std::vector<Point2f> corners = std::vector<Point2f>();
Size winSize = Size(5, 5);
Size zeroZone = Size(-1, -1);
cornerSubPix(greyMat, corners, winSize, zeroZone, criteria);
for (int i = 0; i < corners.size(); i++)
{
circle(img, Point(corners[i].x, corners[i].y), 5, Scalar(0, 255, 0), 2);
}
imshow("img", img);
waitKey();
destroyAllWindows();
}
The solution was to iterate over the centroids to build the corners vector before passing the corners variable to the cornerSubPix(...) function.
std::vector<Point2f> corners = std::vector<Point2f>();
for (int i = 0; i < centroids.rows; i++)
{
double x = centroids.at<double>(i, 0);
double y = centroids.at<double>(i, 1);
corners.push_back(Point2f(x, y));
}
The output of the solution is still not exactly what the python output is, regardless it fixed this question in case anyone else ran across this issue.
For some background, I am compiling in Visual Studio 2019 and running the code inside LabVIEW 2017. The reason I am doing it in LabVIEW is for research to control a robotic gantry. This is the vision system and it is supposed to detect rectangles (wirebond pads for silicon detectors).
I need it to atleast show me a picture or something but when I run it in LabVIEW, it just says it is not responding and makes me hard close the program. So frustrating! If theres no huge errors in my C++ code then I know I have to dig deeper into my LabVIEW code.
The following code is my problem. I am fairly new to C++ and programming in general. I have done the step each line inside LabVIEW and when it stops responding is when it starts to grab the nominalHeight, xfov etc... or just when it goes into the WBPdetection function in general.
Any help is much appreciated. Or if someone could just point me in the right direction.
#include "stdafx.h"
#include "utils.h"
#include "WBPdetection.h"
#include "opencv2/opencv.hpp"
#include "opencv2/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include <vector>
using namespace std;
void show3(cv::Mat img)
{
cv::namedWindow("MyWindow", cv::WINDOW_AUTOSIZE);
cv::imshow("MyWindow", img);
cv::waitKey(0);
cv::destroyWindow("MyWindow");
}
__declspec(dllexport) int __cdecl WBPdetection(
char* imgPtr,
int imgLineWidth,
int imgWidth,
int imgHeight,
double percent_size,
double nominalWidth,
double nominalHeight,
double tolerance,
double xfov,
double yfov)
{
cv::Mat img(imgHeight, imgWidth, CV_8U, (void*)imgPtr, imgLineWidth);
cv::resize(img, img, cv::Size(img.cols * percent_size, img.rows * percent_size), 0, 0);
//PREPPING IMAGE FOR DETECTION ALGORITHIM
cv::threshold(img, img, 125, 255, cv::THRESH_OTSU);
cv::GaussianBlur(img, img, cv::Size(5, 5), 0);
cv::erode(img, img, cv::Mat(), cv::Point(-1, -1), 2, 1, 1);
cv::dilate(img, img, cv::Mat(), cv::Point(-1, -1), 1, 1, 1);
//USE FIND CONTOURS ALGORITHIM
vector<vector<cv::Point>> contours;
vector<cv::Vec4i> hierarchy;
cv::findContours(img, contours, hierarchy, cv::RETR_TREE, cv::CHAIN_APPROX_SIMPLE);
for( int i = 0; i < contours.size(); i++ )
{ approxPolyDP( cv::Mat(contours[i]), contours_poly[i], 3, true );
boundRect[i] = cv::boundingRect( cv::Mat(contours_poly[i]) );
}
vector<vector<double>> dimRects; //ex [ [w1,h1], [w2,h2], [w3,h3], ...]
vector<cv::Point> centerRects; //ex [ [c1], [c2], [c3], ... ]
//PUTTING DIMENSIONS OF ALL RECTANGLES IN VECTORS
for (int i = 0; i < contours.size(); i++)
{
cv::Point center = ((boundRect[i].tl().x + boundRect[i].br().x) / 2, (boundRect[i].tl().y + boundRect[i].br().y) / 2); //what about even pixels
double rectWidth = (boundRect[i].br().x - boundRect[i].tl().x) * (xfov / img.cols); //might not matter tbh
double rectHeight = (boundRect[i].tl().y - boundRect[i].br().y) * (yfov / img.rows);
dimRects[i].push_back(rectWidth);
dimRects[i].push_back(rectHeight);
centerRects.push_back(center);
}
//DEFINING minWidth, etc... FROM tolerance AND nominalWidth
double minWidth = nominalWidth * (1 - tolerance);
double maxWidth = nominalWidth * (1 + tolerance);
double minHeight = nominalHeight * (1 - tolerance);
double maxHeight = nominalHeight * (1 + tolerance);
// DRAWING CONTOURS AND BOUNDING RECTANGLE + CENTER
for( int i = 0; i< dimRects.size(); i++ )
{
cv::Scalar color = cv::Scalar(255,255,255); //creates color
if ((dimRects[i][0] > minWidth && dimRects[i][0] < maxWidth) && (dimRects[i][1] > minHeight && dimRects[i][1] < maxHeight))
{
drawContours(img, contours_poly, i, color, 1, 8, vector<cv::Vec4i>(), 0, cv::Point());
rectangle(img, boundRect[i].tl(), boundRect[i].br(), color, 2, 8, 0);
circle(img, centerRects[i], 1, cv::Scalar(0, 0, 255), 1, cv::LINE_8);
}
}
show3(img);
return 0;
}
Well there is one error here
vector<vector<double>> dimRects;
...
for (int i = 0; i < contours.size(); i++)
{
...
dimRects[i].push_back(rectWidth);
dimRects[i].push_back(rectHeight);
dimRects has zero size but your code treats it as if it has the same size as contours.
I'm supposed to detect the two white lines of the road with the function HoughLines. I use three trackbars in order to find the best parameters to detect ONLY the two white lines of the road. I have tried this: (the problem is that it looks like even if I change the values of the trackbars it doesn't updates the images, it is still at the first values). I'm using opencv with c++.
Without trackbars it works, but it's almost impossibile finding good values without it because I don't know how to tune the parameters and the image is pretty complex.
#include <iostream>
#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/calib3d.hpp>
using namespace cv;
using namespace std;
const int kernel_size = 3;
Mat src, src_gray;
Mat dst, detected_edges;
Mat cdst;
int slider_value_one;
int slider_value_two;
int slider_value_three;
vector<Vec2f> lines; // will hold the results of the detection
static void Hough_transform(int, void*)
{
// Standard Hough Line Transform
HoughLines(detected_edges, lines, 1, CV_PI/180, 130,slider_value_one,slider_value_two); // runs the actual detectio
// Draw the lines
for( size_t i = 0; i < lines.size(); i++ )
{
float rho = lines[i][0], theta = lines[i][1];
Point pt1, pt2;
double a = cos(theta), b = sin(theta);
double x0 = a*rho, y0 = b*rho;
pt1.x = cvRound(x0 + 1000*(-b));
pt1.y = cvRound(y0 + 1000*(a));
pt2.x = cvRound(x0 - 1000*(-b));
pt2.y = cvRound(y0 - 1000*(a));
line( cdst, pt1, pt2, Scalar(0,0,255), 3, CV_AA);
}
//printing
imshow("standard Hough Line Transform", cdst);
}//HoughTransform
int main(int argc, const char * argv[]) {
//-----Loads an image
src = imread("/Users/massimilianolorenzin/Documents/Progetti\ XCode/lab4/lab4/lab4/images/road2.png");
/// ---- CANNY DETECTOR
/// Convert the image to grayscale
cvtColor( src, src_gray, CV_BGR2GRAY);
/// Reduce noise with a kernel 3x3
blur( src_gray, detected_edges, Size(3,3) );
/// Canny detector
Canny( detected_edges, detected_edges, 150, 450, kernel_size );
// Copy edges to the images that will display the results in BGR
cvtColor(detected_edges, cdst, COLOR_GRAY2BGR);
/// ---- HOUGH LINE TRANSFORM
namedWindow("standard Hough Line Transform"); // Create Window
//first TrackBar
createTrackbar( "First Par", "standard Hough Line Transform", &slider_value_one, 200, Hough_transform);
Hough_transform(slider_value_one,0 );
//second TrackBar
createTrackbar( "Second Par", "standard Hough Line Transform", &slider_value_two, 100, Hough_transform);
Hough_transform(slider_value_two, 0 );
//third TrackBar
createTrackbar( "Third Par", "standard Hough Line Transform", &slider_value_three, 100, Hough_transform);
Hough_transform( slider_value_three, 0 );
//printing
imshow("Input Image",src);
imshow( "edges", detected_edges );
waitKey(0);
return 0;
HoughLines() looks doesn't answer at the values set with the trackbars. The window appear normally, slider_value_one, slider_value_two, slider_value_three have the right values because I printed and i saw them, so I don't understand why HoughLines() doesn't take the passed values.
enter image description here
Above there is the input image, while below there is the final ouput. I am asked, in this step, just to create the two lines to the side of the road, coloring the area between the two lines( like in the photo) is requested in the next step.
If I don't do anything (that is, don't change the color detection HSV via a Controls Window), the app runs fine. However if I change the HSV values when the app is running, I get the following errors. I have tested the code without Hough, it runs fine.
The CPU Error -
Unhandled exception at 0x00007FF9ECA64388 (ucrtbase.dll) in HoughFinder.exe: An invalid parameter was passed to a function that considers invalid parameters fatal.
This is my code -
VideoCapture capture(0); // 0 is my webcam
...
capture.read(displayOriginal))
...(Code to detect colors for extra accuracy)
cudaCanny->detect(imgThresholded, imgCanny);
vector<Vec2f> lines;
//Ptr<HoughLinesDetector> hough = createHoughLinesDetector(1, CV_PI / 180, 100); CUDA code...
//hough->detect(imgCanny, lines); CUDA code...
HoughLines(displayCanny, lines, 1, CV_PI / 180, 100, 0, 0); // CPU code...
for (size_t i = 0; i < lines.size(); i++)
{
float rho = lines[i][0], theta = lines[i][1];
Point pt1, pt2;
double a = cos(theta), b = sin(theta);
double x0 = a*rho, y0 = b*rho;
pt1.x = cvRound(x0 + 1000 * (-b));
pt1.y = cvRound(y0 + 1000 * (a));
pt2.x = cvRound(x0 - 1000 * (-b));
pt2.y = cvRound(y0 - 1000 * (a));
line(displayHough, pt1, pt2, Scalar(0, 0, 255), 3, CV_AA);
}
imshow("Hough", displayHough);
imshow("Live Video", displayOriginal);
Extra Info -
If I use CUDA code to use Hough, I get this error -
Unhandled exception at 0x00007FF9F561A1C8 in HoughFinder.exe: Microsoft C++ exception: cv::Exception at memory location 0x000000A75E81EB70.
App Error (Don't get this error while using CPU code) -
OpenCV Error: Assertion failed (d == 2 && (sizes[0] == 1 || sizes[1] == 1 || sizes[0]*sizes[1] == 0)) in cv::_OutputArray::create, file OPENCV_DIR\opencv-sources\modules\core\src\matrix.cpp, line 2363
Can anyone help? If either the CPU or CUDA code is fixed, its fine, but I would more prefer the CUDA error to be fixed (As CUDA has extra speed).
After lots of trials and errors, I finally found the solution. Actually the output in detect should be a GpuMat not a vect2d. I would have figured this out earlier, but the documentation of OpenCV is very confusing. Here's the edited code -
Ptr<HoughLinesDetector> houghLines = createHoughLinesDetector(1, CV_PI / 180, 120);
GpuMat tmpLines; // This should be GpuMat...
vector<Vec2d> lines;
GpuMat imgCanny;
...
while(true) {
...
houghLines->detect(imgCanny, tmpLines);
houghLines->downloadResults(tmpLines, lines);
...
}
OpenCV GPU Error (Function Not Implemented) in Hough Transform
I had a similar problem. My code is something like this:
cv::Mat src, src_gray, src_blured, detected_edges, hough_lines;
std::vector<cv::Vec4i> lines;
src = cv::cvarrToMat(img, false);
opencv_run_ = true;
while (opencv_run_) {
cv::cvtColor(src, src_gray, CV_BGR2GRAY);
cv::medianBlur(src_gray, src_blured, blur_size_);
cv::Canny(src_blured, detected_edges, canny_low_threshold_, canny_max_threshold_, canny_aperture_size_);
cv::HoughLinesP(detected_edges, lines, 2, CV_PI / 2, hough_votes_, hough_min_line_length_, hough_max_line_gap_);
hough_lines = cv::Mat::zeros(size_horizontal_, size_vertical_, CV_8UC4);
for (size_t i = 0; i < lines.size(); i++) {
cv::line(hough_lines, cv::Point(lines[i][0], lines[i][1]), cv::Point(lines[i][2], lines[i][3]), cv::Scalar(0, 0, 255), 3, 8);
}
if (lines.size() > 0) lines.clear();
cv::imshow("Gray scale", src_gray);
cv::imshow("Edges", detected_edges);
cv::imshow("Hough", hough_lines);
//Some logic in waitKey and sleeptime
cv::waitKey(sleep_time);
}
cvDestroyAllWindows();
cvReleaseImageHeader(&img);
where img is a pointer to an IplImage in which i manually configure the values. (My image data are coming from a camera whose API gives me void * i.e. raw data)
This particular piece of code is running inside a boost::thread. While everything was running fine while inside the loop, when i did
opencv_run = false;
this_boost_thread.join();
in order to stop the thread, i was getting the invalid parameter exception. What baffled me is that the exception was thrown after the thread return which alarmed that this a classic case of stack corruption.
After hours of search i came across some post in some forum that said this is probably a problem with linked libraries. So i checked my opencv installation and saw that my libs are in a vc12 folder which means Visual Studio 2014 (I like to install pre built because i am an idiot), different from VS 2015 that i use.
So i searched for Visual Studio 2015 opencv libs, found some in the opencv 3.1 version https://sourceforge.net/projects/opencvlibrary/files/opencv-win/ , while i was using opencv 2.4.13. I decided not use them and build opencv from scratch. So i cloned from here https://github.com/opencv/opencv, followed the instructions given here http://docs.opencv.org/2.4/doc/tutorials/introduction/windows_install/windows_install.html and builded vc14 x86 opencv3.1 libraries which seem to work.
I am trying to detect some lines using Hough Transform on a cv::gpu::GpuMat structure. I have tried using both gpu::HoughLines and gpu::HoughLinesP but even with extremely low thresholds, I am not getting any results at all. During debugging, I see that the container which should contain the results (houghLines) has only zeros stored inside it. The code I have written is given below,
static cv::Mat drawHoughLinesOnMat (cv::gpu::GpuMat hough_Mat, cv::gpu::GpuMat houghLines)
{
cv::Mat output_Mat;
cv::cvtColor(cv::Mat(hough_Mat), output_Mat, CV_GRAY2BGR);
std::vector<cv::Vec4i> lines_vector;
if (!houghLines.empty())
{
lines_vector.resize(houghLines.cols);
cv::Mat temp_Mat (1, houghLines.cols, CV_8UC3, &lines_vector[0]);
houghLines.download (temp_Mat);
}
for (size_t i=0; i<lines_vector.size(); ++i)
{
cv::Vec4i l = lines_vector[i];
cv::line(output_Mat, cv::Point(l[0], l[1]), cv::Point(l[2], l[3]), cv::Scalar(0, 0, 255), 1, 8);
}
return output_Mat;
}
int main()
{
cv::Mat input = cv::imread(INPUT_DATA_1->c_str(), CV_LOAD_IMAGE_GRAYSCALE);
std::string imageType = getImgType(input.type());
cv::gpu::GpuMat mat_input(input), bil_out, mat_thresh, hough_lines;
cv::gpu::HoughLinesBuf hough_buffer;
int bilateral_thresh = 15; // 5 == 0.085s; 15 == 0.467s at run-time
cv::gpu::bilateralFilter(mat_input, bil_out, bilateral_thresh, bilateral_thresh*2, bilateral_thresh/2);
//cv::gpu::threshold(bil_out, mat_thresh, 10, 255, CV_THRESH_BINARY);
cv::gpu::Canny(bil_out, mat_thresh, 10, 60, 5);
cv::gpu::HoughLinesP(mat_thresh, hough_lines, hough_buffer, 1.0f, (float)(CV_PI/180.0f), 5, 1);
//cv::Mat test_hough(hough_lines);
cv::Mat hough_Mat = drawHoughLinesOnMat(mat_input, hough_lines);
cv::gpu::HoughLines(mat_thresh, hough_lines, 1.0f, (float)(CV_PI/180.0f), 1, true);
/*cv::Mat */hough_Mat = drawHoughLinesOnMat(mat_input, hough_lines);
return EXIT_SUCCESS
}
The image I am using is,
Could someone tell me what it is that I am doing wrong..? Thanks in advance.!
The output of the Canny filter is,
EDIT:
I have tested on the CPU version of HoughLines and it seems to work just fine.
EDIT_2:
The solution posted by #jet47 works perfectly.
You use incorrect code for downloading results from GPU back to CPU:
lines_vector.resize(houghLines.cols);
cv::Mat temp_Mat (1, houghLines.cols, CV_8UC3, &lines_vector[0]);
houghLines.download (temp_Mat);
You use incorrect type for temp_Mat - CV_8UC3, it must be CV_32SC4.
The correct code is:
lines_vector.resize(houghLines.cols);
cv::Mat temp_Mat(1, houghLines.cols, CV_32SC4, &lines_vector[0]);
houghLines.download(temp_Mat);
My guess is that the Method you are using is outdated (but im not entirely sure).
This is how i would do it(as demonstrated in this Example Code):
//d_src filled with your image somewhere
GpuMat d_lines;
{
Ptr<cuda::HoughSegmentDetector> hough = cuda::createHoughSegmentDetector(1.0f, (float) (CV_PI / 180.0f), 50, 5);
hough->detect(d_src, d_lines);
}
vector<Vec4i> lines_gpu;
if (!d_lines.empty())
{
lines_gpu.resize(d_lines.cols);
Mat h_lines(1, d_lines.cols, CV_32SC4, &lines_gpu[0]);
d_lines.download(h_lines);
}
for (size_t i = 0; i < lines_gpu.size(); ++i)
{
Vec4i l = lines_gpu[i];
line(dst_gpu, Point(l[0], l[1]), Point(l[2], l[3]), Scalar(0, 0, 255), 3, LINE_AA);
}
EDIT The above uses the OpenCv 3.0 Interface