HSV color detection with OpenCV - c++

The "red" color-detection is not working yet. The following code is supposed to detect a red bar from an input-image and return a mask-image showing a white bar at the corresponding location.
The corresponding HSV-values of the "red" bar in the inputRGBimage are : H = 177, S = 252, V = 244
cv::Mat findColor(cv::Mat inputRGBimage) {
cv::Mat imageHSV(inputRGBimage.rows, inputRGBimage.cols, CV_8UC3);
cv::Mat imgThreshold(inputRGBimage.rows, inputRGBimage.cols, CV_8UC1);
// convert input-image to HSV-image
cvtColor(inputRGBimage, imageHSV, cv::COLOR_BGR2HSV);
// for red: (H < 14)
// cv::inRange(imageHSV, cv::Scalar(0, 53, 185, 0), cv::Scalar(14, 255, 255, 0), imgThreshold);
// or (H > 165) (...closing HSV-circle)
cv::inRange(imageHSV, cv::Scalar(165, 53, 185, 0), cv::Scalar(180, 255, 255, 0), imgThreshold);
return imgThreshold;
}
The two images below show the inputRGBimage (top) and the returned imgThreshold (bottom). As you can see, the mask is not showing the white bar at the expected color "red" but shows it for some unknown reason at the "blue" bar. Why ????
The following change of the cv::inRange line of code (i.e. H > 120) and its result again illustrates that the color detection is not actually acting as expected :
// or (H > 120) (...closing HSV-circle)
cv::inRange(imageHSV, cv::Scalar(120, 53, 185, 0), cv::Scalar(180, 255, 255, 0), imgThreshold);
As a third example: (H > 100):
// or (H > 100) (...closing HSV-circle)
cv::inRange(imageHSV, cv::Scalar(100, 53, 185, 0), cv::Scalar(180, 255, 255, 0), imgThreshold);
Why the unexpected order of colors in my 3 code-examples (decreasing the H-value from 165 to 100) showing mask orders of "blue->violet->red->orange" instead of the actually expected HSV-wheel rough order of "red->violet->blue->green->yellow->orange" ?????
HSV in OpenCV has ranges:
0 <= H <= 180,
0 <= S <= 255,
0 <= V <= 255, (not quite like in the illustrating graphic above - but the order of colors should be the same for OpenCV HSV-colors - or not ???)

Make sure that the image uses the channel order B, G, R. Also, for the color red you need check two ranges of values, one around H=0 and the other around H=180. You could try this function:
cv::Mat findColor(const cv::Mat & inputBGRimage, int rng=15)
{
// Make sure that your input image uses the channel order B, G, R (check not implemented).
cv::Mat input = inputBGRimage.clone();
cv::Mat imageHSV;//(input.rows, input.cols, CV_8UC3);
cv::Mat imgThreshold, imgThreshold0, imgThreshold1;//(input.rows, input.cols, CV_8UC1);
assert( ! input.empty() );
// convert input-image to HSV-image
cv::cvtColor( input, imageHSV, cv::COLOR_BGR2HSV );
// In the HSV-color space the color 'red' is located around the H-value 0 and also around the
// H-value 180. That is why you need to threshold your image twice and the combine the results.
cv::inRange(imageHSV, cv::Scalar( 0, 53, 185, 0), cv::Scalar(rng, 255, 255, 0), imgThreshold0);
if ( rng > 0 )
{
cv::inRange(imageHSV, cv::Scalar(180-rng, 53, 185, 0), cv::Scalar(180, 255, 255, 0), imgThreshold1);
cv::bitwise_or( imgThreshold0, imgThreshold1, imgThreshold );
}
else
{
imgThreshold = imgThreshold0;
}
return imgThreshold;
}
Good luck! :)

Related

how to get a mask of an image so that i can use it in the inpainting function

i want my mask to be black obviously and the red line which is my region of interest to be white so that i can use it inside the inpainting function...! am having this code but it not working
How to concentrate on a particular part of the image... because my mask image is showing all the image... this is the image and code
cv::inRange(img, cv::Scalar(0, 100, 220), cv::Scalar(10, 255, 255), lower);
cv::inRange(img, cv::Scalar(0, 10, 100), cv::Scalar(255, 255, 255), upper);
threshold(mask, mask,10,255, CV_THRESH_BINARY);
inpaint(img, mask, inpainted,3,CV_INPAINT_TELEA);
Mat img = imread("Lennared.jpg");
Mat mask, inpainted;
cvtcolor(img,mask,CV_BRG2GRAY);
inrange(img, Scalar(10,10,200), Scalar(40,40,255),mask); // make sure your
targeted color is between the range you stated
inpaint(img,mask, inpainted,3,CV_INPAINT_TELEA);
for( int key =0 ; 23 !-key; key=waitKey())
{
switch (key)
{
case 'm' : imshow("maskimage", mask)
break;
case 'i': imshow("inpainted image", inpainted)
break;
default : imshow("original" img);
}
}
return 0;
since am i trying to detect the red color in the image, i have to pass the scalar value of the red color, that from a lower range to a higher range all inclusive... That should give you the perfect mask image for use in the inpaint function, hope this help everyone else ..

how to find coordinates after using Houghlinep function in opencv

Hi guys i'm new to OpenCV.
I'm using opencv to remove lines and get removed coordinates from image.
I'm using HoughlineP like this
Mat src = imread("F:/003-00.jpg", IMREAD_GRAYSCALE);
Mat bw;
blur(src, bw, Size(3, 3));
pyrDown(bw, bw);
//threshold(bw, bw, 170, 255, THRESH_BINARY_INV);
Canny(bw, bw, 100, 200, 3);
Mat color_dst = Mat::zeros(bw.size(), CV_8UC1);
vector<Vec4i> lines;
HoughLinesP(bw, lines, 1, CV_PI / 180, 400, 300, 20);
for (size_t i = 0; i < lines.size(); i++)
{
line(color_dst, Point(lines[i][0], lines[i][1]),
Point(lines[i][2], lines[i][3]), Scalar(255, 255, 255), 3);
}
imwrite("F:/result.jpg", color_dst);
and i got a result image like this
HoughlinesP Output
but one line actually contains many of child lines and group to one bold line
Now I want to erode,group, and normally it to one line with two Point begin , end for every line
Here is input image
Input image

How to multiply 2 OpenCV mats using a GPU

In OpenCV, I can multiply an RGB 1920 x 1080 mat by a 3 x 3 Mat to change the color composition of my source Mat. Once my source mat is properly shaped, I can use the '*' operator to perform the multiplication. This operator is not available when using a cv::gpu::GpuMat.
My question is how would I format my input source Mat to use cv::gpu::gemm?Can I even use cv::gpu::gemm?
This is the only call that performs matrix multiplication in the OpenCV library from what I can tell. cv::gpu::gemm wants to see a CV_32FC1 , CV_64FC1 type Mat. The type I normally use with the CPU is CV_32FC3.
//sourceMat is CV_32FC3 1920 x 1080 Mat
Mat sourceMat = matFromBuffer(data->bufferA, data->widthA, data->heightA);
//This is the color Matrix
float matrix[3][3] = {{1.057311, -0.204043, 0.055648},
{ 0.041556, 1.875992, -0.969256},
{-0.498535,-1.537150, 3.240479}};
Mat colorMatrixMat = Mat(3, 3, CV_32FC1, matrix).t();
//Color Correct the Mat
Mat linearSourceMat = sourceMat.reshape(1, 1080*1920);
Mat multipliedMatrix = linearSourceMat * colorMatrixMat;
Mat recoloredMat = multipliedMatrix.reshape(3, 1080);
Update:
As a test, I created the test routine:
static int gpuTest(){
float matrix[9] = {1.057311, -0.204043, 0.055648, 0.041556, 1.875992, -0.969256, -0.498535,-1.537150, 3.240479};
Mat matrixMat = Mat(1, 9, CV_32FC1, matrix).t();
cv::gpu::GpuMat gpuMatrixMat;
gpuMatrixMat.upload(matrixMat);
float matrixDest[9] = {1,1,1,1,1,1,1,1,1};
Mat matrixDestMat = Mat(1, 9, CV_32FC1, matrixDest).t();
cv::gpu::GpuMat destMatrixMat;
destMatrixMat.upload(matrixDestMat);
cv::gpu::GpuMat nextMat;
cv::gpu::gemm(gpuMatrixMat, destMatrixMat, 1, cv::gpu::GpuMat(), 0, nextMat);
return 0;
};
and the error I receive is:
OpenCV Error: Assertion failed (src1Size.width == src2Size.height) in gemm, file /Users/myuser/opencv-2.4.12/modules/gpu/src/arithm.cpp, line 109
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /Users/myuser/opencv-2.4.12/modules/gpu/src/arithm.cpp:109: error: (-215) src1Size.width == src2Size.height in function gemm
Now how can the src1Size.width be equal to src2Size.height? The width and height are different.
Here's a minimum working example using OpenCV 3.1.
#include <opencv2/opencv.hpp>
#include <opencv2/cudaarithm.hpp>
int main()
{
cv::Mat sourceMat = cv::Mat::ones(1080, 1920, CV_32FC3);
//This is the color Matrix
float matrix[3][3] = {
{ 1.057311, -0.204043, 0.055648 }
, { 0.041556, 1.875992, -0.969256 }
, { -0.498535, -1.537150, 3.240479 }
};
cv::Mat colorMatrixMat = cv::Mat(3, 3, CV_32FC1, matrix).t();
cv::Mat linearSourceMat = sourceMat.reshape(1, 1080 * 1920);
cv::Mat multipliedMatrix = linearSourceMat * colorMatrixMat;
try {
cv::Mat dummy, gpuMultipliedMatrix;
// Regular gemm
cv::gemm(linearSourceMat, colorMatrixMat, 1.0, dummy, 0.0, gpuMultipliedMatrix);
// CUDA gemm
// cv::cuda::gemm(linearSourceMat, colorMatrixMat, 1.0, dummy, 0.0, gpuMultipliedMatrix);
std::cout << (cv::countNonZero(multipliedMatrix != gpuMultipliedMatrix) == 0);
} catch (cv::Exception& e) {
std::cerr << e.what();
return -1;
}
}
Note that when the beta parameter to gemm(...) is zero, the third input matrix is ignored (based on the code).
Unfortunately I don't have a build of OpenCV compiled with CUBLAS available to try it, but it should work.
Following is somewhat speculative...
To make this work with OpenCV 2.4, you will need to add a little bit more. Before calling gemm(...), you need to create GpuMat objects and upload the data.
cv::gpu::GpuMat gpuLinSrc, gpuColorMat, dummy, gpuResult;
gpuLinSrc.upload(linearSourceMat);
gpuColorMat.upload(colorMatrixMat);
Then...
cv::gpu::gemm(gpuLinSrc, gpuColorMat, 1.0, cv::gpu::GpuMat(), 0.0, gpuResult);
and finally download the data back from the GPU.
cv::Mat resultFromGPU;
gpuResult.download(resultFromGPU);
Update
Here's a more detailed example to show you what's happening:
#include <opencv2/opencv.hpp>
#include <iostream>
#include <numeric>
#include <vector>
// ============================================================================
// Make a 3 channel test image with 5 rows and 4 columns
cv::Mat make_image()
{
std::vector<float> v(5 * 4);
std::iota(std::begin(v), std::end(v), 1.0f); // Fill with 1..20
cv::Mat seq(5, 4, CV_32FC1, v.data()); // 5 rows, 4 columns, 1 channel
// Create 3 channels, each with different offset, so we can tell them apart
cv::Mat chans[3] = {
seq, seq + 100, seq + 200
};
cv::Mat merged;
cv::merge(chans, 3, merged); // 5 rows, 4 columns, 3 channels
return merged;
}
// Make a transposed color correction matrix.
cv::Mat make_color_mat()
{
float color_in[3][3] = {
{ 0.1f, 0.2f, 0.3f } // Coefficients for channel 0
, { 0.4f, 0.5f, 0.6f } // Coefficients for channel 1
, { 0.7f, 0.8f, 0.9f } // Coefficients for channel 2
};
return cv::Mat(3, 3, CV_32FC1, color_in).t();
}
void print_mat(cv::Mat m, std::string const& label)
{
std::cout << label << ":\n size=" << m.size()
<< "\n channels=" << m.channels()
<< "\n" << m << "\n" << std::endl;
}
// Perform matrix multiplication to obtain result point (r,c)
float mm_at(cv::Mat a, cv::Mat b, int r, int c)
{
return a.at<float>(r, 0) * b.at<float>(0, c)
+ a.at<float>(r, 1) * b.at<float>(1, c)
+ a.at<float>(r, 2) * b.at<float>(2, c);
}
// Perform matrix multiplication to obtain result row r
cv::Vec3f mm_test(cv::Mat a, cv::Mat b, int r)
{
return cv::Vec3f(
mm_at(a, b, r, 0)
, mm_at(a, b, r, 1)
, mm_at(a, b, r, 2)
);
}
// ============================================================================
int main()
{
try {
// Step 1
cv::Mat source_image(make_image());
print_mat(source_image, "source_image");
std::cout << "source pixel at (0,0): " << source_image.at<cv::Vec3f>(0, 0) << "\n\n";
// Step 2
cv::Mat color_mat(make_color_mat());
print_mat(color_mat, "color_mat");
// Step 3
// Reshape the source matrix to obtain a matrix:
// * with only one channel (CV_32FC1)
// * where each row corresponds to a single pixel from source
// * where each column corresponds to a single channel from source
cv::Mat reshaped_image(source_image.reshape(1, source_image.rows * source_image.cols));
print_mat(reshaped_image, "reshaped_image");
// Step 4
cv::Mat corrected_image;
// corrected_image = 1.0 * reshaped_image * color_mat
cv::gemm(reshaped_image, color_mat, 1.0, cv::Mat(), 0.0, corrected_image);
print_mat(corrected_image, "corrected_image");
// Step 5
// Reshape back to the original format
cv::Mat result_image(corrected_image.reshape(3, source_image.rows));
print_mat(result_image, "result_image");
std::cout << "result pixel at (0,0): " << result_image.at<cv::Vec3f>(0, 0) << "\n\n";
// Step 6
// Calculate one pixel manually...
std::cout << "check pixel (0,0): " << mm_test(reshaped_image, color_mat, 0) << "\n\n";
} catch (cv::Exception& e) {
std::cerr << e.what();
return -1;
}
}
// ============================================================================
Step 1
First we create a small test input image:
The image contains 3 channels of float values, i.e. the data type is CV_32FC3. Let's treat the channels as red, green, blue in that order.
The image contains 5 rows of pixels.
The image contains 4 columns of pixels.
Values in each channel are sequential, green = red + 100 and blue = red + 200.
source_image:
size=[4 x 5]
channels=3
[1, 101, 201, 2, 102, 202, 3, 103, 203, 4, 104, 204;
5, 105, 205, 6, 106, 206, 7, 107, 207, 8, 108, 208;
9, 109, 209, 10, 110, 210, 11, 111, 211, 12, 112, 212;
13, 113, 213, 14, 114, 214, 15, 115, 215, 16, 116, 216;
17, 117, 217, 18, 118, 218, 19, 119, 219, 20, 120, 220]
We can print out a single pixel, to make the structure clearer:
source pixel at (0,0): [1, 101, 201]
Step 2
Create a sample colour correction matrix (transposed) such that:
First column contains coefficients used to determine the red value
Second column contains coefficients used to determine the green value
Third column contains coefficients used to determine the blue value
color_mat:
size=[3 x 3]
channels=1
[0.1, 0.40000001, 0.69999999;
0.2, 0.5, 0.80000001;
0.30000001, 0.60000002, 0.89999998]
Sidenote: Color Correction Algorithm
We want to transform source pixel S to pixel T using coefficients C
S = [ sr, sg, sb ]
T = [ tr, tg, tb ]
C = [ cr1, cr2, cr3;
cg1, cg2, cg3;
cb1, cb2, cb3]
Such that
Tr = cr1 * sr + cr2 * sg + cr3 * sb
Tg = cg1 * sr + cg2 * sg + cg3 * sb
Tb = cb1 * sr + cb2 * sg + cb3 * sb
Which can be represented by the following matrix expression
T = S * C_transpose
Step 3
In order to be able to use the above algorithm, we first need to reshape our image into a matrix that:
Contains a single channel, so that value at each point is just a float
Has one pixel per row.
Has 3 columns representing red, green, blue
In this shape, matrix multiplication will mean that each pixel/row from input gets multiplied by the coefficient matrix to determine one pixel/row in the output.
The reshaped matrix looks as follows:
reshaped_image:
size=[3 x 20]
channels=1
[1, 101, 201;
2, 102, 202;
3, 103, 203;
4, 104, 204;
5, 105, 205;
6, 106, 206;
7, 107, 207;
8, 108, 208;
9, 109, 209;
10, 110, 210;
11, 111, 211;
12, 112, 212;
13, 113, 213;
14, 114, 214;
15, 115, 215;
16, 116, 216;
17, 117, 217;
18, 118, 218;
19, 119, 219;
20, 120, 220]
Step 4
We perform the multiplication, for example using gemm, to get the following matrix:
corrected_image:
size=[3 x 20]
channels=1
[80.600006, 171.5, 262.39999;
81.200005, 173, 264.79999;
81.800003, 174.5, 267.20001;
82.400002, 176, 269.60001;
83, 177.5, 272;
83.600006, 179, 274.39999;
84.200005, 180.5, 276.79999;
84.800003, 182, 279.20001;
85.400002, 183.5, 281.60001;
86, 185, 284;
86.600006, 186.5, 286.39999;
87.200005, 188, 288.79999;
87.800003, 189.5, 291.20001;
88.400009, 191, 293.60001;
89, 192.5, 296;
89.600006, 194, 298.39999;
90.200005, 195.50002, 300.79999;
90.800003, 197, 303.20001;
91.400009, 198.5, 305.60001;
92, 200, 308]
Step 5
Now we can reshape the image back to the original shape. The result is
result_image:
size=[4 x 5]
channels=3
[80.600006, 171.5, 262.39999, 81.200005, 173, 264.79999, 81.800003, 174.5, 267.20001, 82.400002, 176, 269.60001;
83, 177.5, 272, 83.600006, 179, 274.39999, 84.200005, 180.5, 276.79999, 84.800003, 182, 279.20001;
85.400002, 183.5, 281.60001, 86, 185, 284, 86.600006, 186.5, 286.39999, 87.200005, 188, 288.79999;
87.800003, 189.5, 291.20001, 88.400009, 191, 293.60001, 89, 192.5, 296, 89.600006, 194, 298.39999;
90.200005, 195.50002, 300.79999, 90.800003, 197, 303.20001, 91.400009, 198.5, 305.60001, 92, 200, 308]
Let's have a look at one pixel from the result:
result pixel at (0,0): [80.6, 171.5, 262.4]
Step 6
Now we can double check our result by performing the appropriate calculations manually (functions mm_test and mm_at).
check pixel (0,0): [80.6, 171.5, 262.4]

Edge Extraction Suggections OpenCV

Im looking for suggestions to improve my algorithm to search for parts in the following image
so far I have the following
GaussianBlur(canny, canny, Size(5, 5), 2, 2);
Canny(canny, canny, 100, 200, 5);
HoughCircles(canny, Part_Centroids, CV_HOUGH_GRADIENT, 2, 30, 100, 50, 50, 60);
My edge detect output looks like this
and Im using a HoughCircle to try to find the parts. I havent been having great success though because the HoughCircle seems very fussy and often returns a circle that isnt really the best match for a part.
Any suggestions on improving this search algorithm
EDIT:
I have tried the suggestions in the comments below. The normalization made some improvements but removing the canny before hough circles altered the required settings but not the stability.
I think now that I need to do something like the hough circles with very open thresholds and then find a way to score the results. Are there any good methods to score the results of hough circle or correlate the results with the canny output for percentage of match
I thought I would post my solution as someone may find my lessons learned valuable.
I started by taking several frames and averaging them out. This solved some of the noise issues I was having while preserving the strong edges. Next I did a basic filter and canny edge to extract a decent edge map.
Scalar cannyThreshold = mean(filter);
// Canny Edge Detection
Canny(filter, canny, cannyThreshold[0]*(2/3), cannyThreshold[0]*(1+(1/3)), 3);
Next I use a cross correlation with increasing diametered templates and store matches that score over a threshold
// Iterate through diameter ranges
for (int r = 40; r < 70; r++)
{
Mat _mask, _template(Size((r * 2) + 4, (r * 2) + 4), CV_8U);
_template = Scalar(0, 0, 0);
_mask = _template.clone();
_mask = Scalar(0, 0, 0);
circle(_template, Point(r + 4, r + 4), r, Scalar(255, 255, 255), 2, CV_AA);
circle(_template, Point(r + 4, r + 4), r / 3.592, Scalar(255, 255, 255), 2, CV_AA);
circle(_mask, Point(r + 4, r + 4), r + 4, Scalar(255, 255, 255), -1);
Mat res_32f(canny.rows, canny.cols, CV_32FC1);
matchTemplate(canny, _template, res_32f, CV_TM_CCORR_NORMED, _mask);
Mat resize(canny.rows, canny.cols, CV_32FC1);
resize = Scalar(0, 0, 0);
res_32f.copyTo(resize(Rect((resize.cols - res_32f.cols) / 2, (resize.rows - res_32f.rows) / 2, res_32f.cols, res_32f.rows)));
// Strore Well Scoring Results
double minVal, maxVal;
double threshold = .25;
do
{
Point minLoc, maxLoc;
minMaxLoc(resize, &minVal, &maxVal, &minLoc, &maxLoc);
if (maxVal > threshold)
{
matches.push_back(CircleScore(maxLoc.x, maxLoc.y, r, maxVal,1));
circle(resize, maxLoc, 30, Scalar(0, 0, 0), -1);
}
} while (maxVal > threshold);
}
I filter out circles for the best match in each zone
// Sort Matches For Best Match
for (size_t i = 0; i < matches.size(); i++)
{
size_t j = i + 1;
while (j < matches.size())
{
if (norm(Point2f(matches[i].X, matches[i].Y) - Point2f(matches[j].X, matches[j].Y)) - abs(matches[i].Radius - matches[j].Radius) < 15)
{
if (matches[j].Score > matches[i].Score)
{
matches[i] = matches[j];
}
matches[j] = matches[matches.size() - 1];
matches.pop_back();
j = i + 1;
}
else j++;
}
}
Next was the tricky one. I wanted to see which part was likely to be on top. I did this by examining every set of parts that are closer then the sum of there radii, then seeing if the edges in the overlap zone are a stronger match for one over the other. Any covered circle should have little strong edges in the overlap zone.
// Layer Sort On Intersection
for (size_t i = 0; i < matches.size(); i++)
{
size_t j = i + 1;
while (j < matches.size())
{
double distance = norm(Point2f(matches[i].X, matches[i].Y) - Point2f(matches[j].X, matches[j].Y));
// Potential Overlapping Part
if (distance < ((matches[i].Radius+matches[j].Radius) - 10))
{
int score_i = 0, score_j = 0;
Mat intersect_a(canny.rows, canny.cols, CV_8UC1);
Mat intersect_b(canny.rows, canny.cols, CV_8UC1);
intersect_a = Scalar(0, 0, 0);
intersect_b = Scalar(0, 0, 0);
circle(intersect_a, Point(cvRound(matches[i].X), cvRound(matches[i].Y)), cvRound(matches[i].Radius) +4, Scalar(255, 255, 255), -1);
circle(intersect_a, Point(cvRound(matches[i].X), cvRound(matches[i].Y)), cvRound(matches[i].Radius / 3.592-4), Scalar(0, 0, 0), -1);
circle(intersect_b, Point(cvRound(matches[j].X), cvRound(matches[j].Y)), cvRound(matches[j].Radius) + 4, Scalar(255, 255, 255), -1);
circle(intersect_b, Point(cvRound(matches[j].X), cvRound(matches[j].Y)), cvRound(matches[j].Radius / 3.592-4), Scalar(0, 0, 0), -1);
bitwise_and(intersect_a, intersect_b, intersect_a);
double a, h;
a = (matches[i].Radius*matches[i].Radius - matches[j].Radius*matches[j].Radius + distance*distance) / (2 * distance);
h = sqrt(matches[i].Radius*matches[i].Radius - a*a);
Point2f p0((matches[j].X - matches[i].X)*(a / distance) + matches[i].X, (matches[j].Y - matches[i].Y)*(a / distance) + matches[i].Y);
circle(intersect_a, Point2f(p0.x + h*(matches[j].Y - matches[i].Y) / distance, p0.y - h*(matches[j].X - matches[i].X) / distance), 6, Scalar(0, 0, 0), -1);
circle(intersect_a, Point2f(p0.x - h*(matches[j].Y - matches[i].Y) / distance, p0.y + h*(matches[j].X - matches[i].X) / distance), 6, Scalar(0, 0, 0), -1);
bitwise_and(intersect_a, canny, intersect_a);
intersect_b = Scalar(0, 0, 0);
circle(intersect_b, Point(cvRound(matches[i].X), cvRound(matches[i].Y)), cvRound(matches[i].Radius), Scalar(255, 255, 255), 6);
bitwise_and(intersect_a, intersect_b, intersect_b);
score_i = countNonZero(intersect_b);
intersect_b = Scalar(0, 0, 0);
circle(intersect_b, Point(cvRound(matches[j].X), cvRound(matches[j].Y)), cvRound(matches[j].Radius), Scalar(255, 255, 255), 6);
bitwise_and(intersect_a, intersect_b, intersect_b);
score_j = countNonZero(intersect_b);
if (score_i < score_j)matches[i].Layer = matches[j].Layer + 1;
if (score_j < score_i)matches[j].Layer = matches[i].Layer + 1;
}
j++;
}
}
After that it was easy to extract the best part to pick(Im correlating to depth data as well
The blue circles are parts, the green circle is the tallest stack and red circles are part that are under other parts.
I hope this may help someone else working on similar problems

cv::Scalar not displaying expected color

On an image frame, I use
void ellipse(Mat& img, Point center, Size axes, double angle, double startAngle, double endAngle, const Scalar& color, int thickness=1, int lineType=8, int shift=0)
to draw an ellipse and I want to set the ellipse color to green [ RGB value : (165, 206, 94) ].
So I set the parameter const Scalar& color to
cv::Scalar(94.0, 206.0, 165.0, 0.0); // as BGR order, suppose the value is 0.0 - 255.0
cv::Scalar(94.0/255.0, 206.0/255.0, 165.0/255.0, 0.0); // suppose the value is 0.0 - 1.0
I also tried RGB alternative.
CV_RGB(165.0, 206.0, 94.0); // as RGB order, suppose the value is 0.0 - 255.0
CV_RGB(165.0/255.0, 206.0/255.0, 94.0/255.0); // suppose the value is 0.0 - 1.0
But the color being displayed is white [ RGB value (255, 255, 255) ] , not the desired green one.
What I missed at this point? Any suggestion please. Thank you.
EDIT:
Let me put whole related code here. According to OpenCV iOS - Video Processing, this is the CvVideoCamera config in - (void)viewDidLoad;:
self.videoCamera = [[CvVideoCamera alloc] initWithParentView:imgView];
[self.videoCamera setDelegate:self];
self.videoCamera.defaultAVCaptureDevicePosition = AVCaptureDevicePositionFront;
self.videoCamera.defaultAVCaptureSessionPreset = AVCaptureSessionPreset352x288;
self.videoCamera.defaultAVCaptureVideoOrientation = AVCaptureVideoOrientationPortrait;
self.videoCamera.defaultFPS = 30;
self.videoCamera.grayscaleMode = NO;
[self.videoCamera adjustLayoutToInterfaceOrientation:UIInterfaceOrientationPortrait];
Then after [self.videoCamera start]; called, the (Mat&)image would be captured and can be processed in the CvVideoCameraDelegate method - (void)processImage:(Mat&)image; and here are the code to draw an ellipse:
- (void)processImage:(Mat&)image {
NSLog(#"image.type(): %d", image.type()); // got 24
// image.convertTo(image, CV_8UC3); // try to convert image type, but with or without this line result the same
NSLog(#"image.type(): %d", image.type()); // also 24
cv::Scalar colorScalar = cv::Scalar( 94, 206, 165 );
cv::Point center( image.size().width*0.5, image.size().height*0.5 );
cv::Size size( 100, 100 );
cv::ellipse( image, center, size, 0, 0, 360, colorScalar, 4, 8, 0 );
}
Eventually, the ellipse is still in white, not the desired green one.
Set alpha to 255 can fix this problem.
Scalar(94,206,165,255)
As mrgloom points correctly in the comment, it might be because of type of your image [ the Mat object where you want to draw, i.e Mat &img in ellipse() function].
cv::Scalar(94, 206, 165) is the desired green color for 8UC3 type images. Setting these values in 32FC3 image will result in white color.
you can use
src.convertTo(src, CV_8UC3);
Where CV_8UC3 means that you use 8 bits unsigned char and 3 color image representation.
More information you can find here OpenCV docs
after that your ellipse should be green, if it doesn't help post the whole code.
I was having similar problem and I have managed to fix it by first converting image to BGR. So in your case processImage function would look like as:
-(void)processImage:(Mat&)image
{
cvtColor(image, image, CV_RGBA2BGR);
cv::Scalar colorScalar = cv::Scalar( 94, 206, 165 );
cv::Point center( image.size().width*0.5, image.size().height*0.5 );
cv::Size size( 100, 100 );
cv::ellipse( image, center, size, 0, 0, 360, colorScalar, 4, 8, 0 );
}
The only line which I have included in your code is:
cvtColor(image, image, CV_RGBA2BGR);
If you also log channel, depth and type information in the above function as follows:
NSLog(#"Before conversion");
NSLog(#"channels %d", image.channels());
NSLog(#"depth %d", image.depth());
NSLog(#"type %d", image.type());
NSLog(#"element size %lu", image.elemSize());
cvtColor(image, image, CV_RGBA2BGR);
NSLog(#"After conversion");
NSLog(#"channels %d", image.channels());
NSLog(#"depth %d", image.depth());
NSLog(#"type %d", image.type());
NSLog(#"element size %lu", image.elemSize());
you will see before conversion:
channels 4
depth 0
type 24
element size 4
which I think is CV_8UC4 and after conversion it becomes:
channels 3
depth 0
type 16
element size 3
which is CV_8UC3.
I guess one of the reason why it does not work without cvtColor is that the opencv drawing functions don't support alpha transparency when the target image is 4-channel as mentioned in opencv documentation. So by converting CV_RGBA2BGR we take out alpha channel. However having said that I do not managed to get it work if I do:
cvtColor(image, image, CV_RGBA2RGB);
In this Red and Blue colors are inverted in the image. So although it seems to work but I am not sure if it is the actual reason.