OpenCV GrabCut Mask - c++

I have utilised the OpenCV GrabCut functionality to perform an image segmentation. When viewing the segmented image as per the code below, the segmentation is reasonable/correct. However, when looking at(at attempting to use) the segmrntation mask values, I am getting some very large numbers, and not the enumerated values one would expect from the cv::GrabCutClasses enum.
void doGrabCut(){
Vector2i imgDims = getImageDims();
//Wite image to OpenCV Mat.
const Vector4u *rgb = getRGB();
cv::Mat rgbMat(imgDims.height, imgDims.width, CV_8UC3);
for (int i = 0; i < imgDims.height; i++) {
for (int j = 0; j < imgDims.width; j++) {
int idx = i * imgDims.width + j;
rgbMat.ptr<cv::Vec3b>(i)[j][2] = rgb[idx].x;
rgbMat.ptr<cv::Vec3b>(i)[j][1] = rgb[idx].y;
rgbMat.ptr<cv::Vec3b>(i)[j][0] = rgb[idx].z;
}
}
//Do graph cut.
cv::Mat res, fgModel, bgModel;
cv::Rect bb(bb_begin.x, bb_begin.y, bb_end.x - bb_begin.x, bb_end.y - bb_begin.y);
cv::grabCut(rgbMat, res, bb, bgModel, fgModel, 10, cv::GC_INIT_WITH_RECT);
cv::compare(res, cv::GC_PR_FGD, res, cv::CMP_EQ);
//Write mask.
Vector4u *maskPtr = getMask();//uchar
for (int i = 0; i < imgDims.height; i++) {
for (int j = 0; j < imgDims.width; j++) {
cv::GrabCutClasses classification = res.at<cv::GrabCutClasses>(i, j);
int idx = i * imgDims.width + j;
std::cout << classification << std::endl;//Strange numbers here.
maskPtr[idx].x = (classification == cv::GC_PR_FGD) ? 255 : 0;//This always evaluates to 0.
}
}
cv::Mat foreground(rgbMat.size(), CV_8UC3, cv::Scalar(255, 255, 255));
rgbMat.copyTo(foreground, res);
cv::imshow("GC Output", foreground);
}
Why would one get numbers outside the enumeration when the segmentation is qualitatively correct?

I doubt on your //Write mask. step, why do you re-iterate the res and modify maskPtr as maskPtr[idx].x = (classification == cv::GC_PR_FGD) ? 255 : 0;, Basically you already have a single channel Binary image stored in the res variable, the cv::compare() returns a binary image
However if you still want to debug the values by iteration then you should use the standard technique for iterating a single channel image as:
for (int i = 0; i < m.rows; i++) {
for (int j = 0; j < m.cols; j++) {
uchar classification = res.at<uchar>(i, j);
std::cout << int(classification) << ", ";
}
}
As you are iterating a single channel mat you must use res.at<uchar>(i, j) and not res.at<cv::GrabCutClasses>.

Related

Change RGB to GrayScale in c++

I am trying to convert RGB image to gray scale using average method. But the output that is get is different from the desired output. I'm taking the image and getting the rgb values. I perform average operation and store the averaged and another array of same size of the image. Finally i'm converting the array to Mat and displaying the image.
Input image:
Desired output:
My output:
int main()
{
Mat image;
image =imread("<image_path>");
int rows=image.rows;
int cols=image.cols;
int myArray[rows][cols];
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
myArray[i][j] = 0;
}
}
uint8_t* pixelPtr = (uint8_t*)image.data;
int cn = image.channels();
Scalar_<uint8_t> bgrPixel;
for(int i = 0; i < rows; i++)
{
for(int j = 0; j < cols; j++)
{
bgrPixel.val[0] = pixelPtr[i*image.cols*cn + j*cn + 0]; // B
bgrPixel.val[1] = pixelPtr[i*image.cols*cn + j*cn + 1]; // G
bgrPixel.val[2] = pixelPtr[i*image.cols*cn + j*cn + 2]; // R
int average = (bgrPixel.val[0]+bgrPixel.val[1]+bgrPixel.val[2])/3;
myArray[i][j]=average;
}
}
Mat averaged_image(Size(rows, cols), CV_8UC3, myArray, Mat::AUTO_STEP);
imwrite("<path to save the image>",averaged_image);
imshow("averaged_image",averaged_image);
waitKey(0);
return 0;
}
When creating Mat averaged_image,
Mat averaged_image(Size(rows, cols), CV_8UC3, myArray, Mat::AUTO_STEP);
you need to use CV_32S not CV_8UC3 because your array element is not three chars, it's one 32-bit int.
You can also use the function cvtColor:
cv::Mat gray;
cv::cvtColor(image, gray, CV_BGR2GRAY);
Bonus: this function does correct weighting of the channels, because simple averaging may not be the right thing to do.

Opencv only process the parts of image

I want to make a negative transformation for the image which is a very simple program.
But when I run the program. I want to transform all of the pixels in the image, but only 1/3 parts of that are processed. I don't make sure where is wrong. all the code I followed the book. But the result is different.
I think there is something wrong about the columns, but when I change the value of I.cols in negativeImage function with the actual value of image. the output still keep the same. only 1/3 parts of image are processed. If I 3 times the I.cols all of the pixels in the iamge could be processed.
vector<uchar> getNegativeLUT() {
vector<uchar> LUT(256, 0);
for (int i = 0; i < 256; ++i)
LUT[i] = (uchar)(255 - i);
return LUT;
}
void negativeImage(Mat& I) {
vector<uchar> LUT = getNegativeLUT();
for (int i = 0; i < I.rows; ++i) {
for (int j = 0; j < I.cols; ++j) {
I.at<uchar>(i, j) = LUT[I.at<uchar>(i, j)];
//stack overflow
}
}
}
int main() {
Mat image = imread("1.png");
Mat processed_image2 = image.clone();
negativeImage(processed_image2);
printf("%d", image.cols);
imshow("Input Image", image);
imshow("Negative Image", processed_image2);
waitKey(0);
return 0;
}
Output Image
You need to put correct type with at<> operator. Your PNG image has to be converted to 8UC1 to then use uchar type to access each pixel. I suppose your image has 3 channels, so you only iterate over 1/3 of the image. Also, I suggest you to use ptr<> operator in rows loop and then access to pixel as an array.
Mat M;
cvtColor(I, M, CV_BGR2GRAY);
// M is CV_8UC1 type
for(int i = 0; i < M.rows; i++)
{
uchar* p = M.ptr<uchar>(i);
for(int j = 0; j < I.cols; j++)
{
p[j] = LUT[p[j]];
}
}
EDIT: you should use cv::LUT instead of doing it yourself.
cv::Mat lut(1, 256, CV_8UC1);
for( int i = 0; i < 256; ++i)
{
lut.at<uchar>(0,i) = uchar(255-i);
}
cv::LUT(M, lut, result);

How to set pixel value of a cv::Mat1b?

I have copied a grayscale image into a cv::Mat1b, and I want to loop through each pixel and read and change its value. How can I do that?
My code looks like this :
cv::Mat1b newImg;
grayImg.copyTo(newImg);
for (int i = 0; i < grayImg.rows; i++) {
for (int j = 0; i < grayImg.cols; j++) {
int pixelValue = static_cast<int>(newImg.at<uchar>(i, j));
if(pixelValue > thresh)
newImg.at<int>(i,j) = 0;
else
newImg.at<int>(i, j) = 255;
}
}
But in the assignments (inside of if and else), I get the error Access violation writing location.
How do I read and write specific pixels correctly?
Thanks !
Edit
Thanks to #Miki and #Micka, this is how I solved it :
for (int i = 0; i < newImg.rows; i++) {
for (int j = 0; j < newImg.cols; j++) {
// read :
cv::Scalar intensity1 = newImg.at<uchar>(i,j);
int intensity = intensity1.val[0];
// write :
newImg(i, j) = 255;
}
}
newImg.at<int>(i,j)
should be
newImg.at<uchar>(i,j)
Because cv::Mat1b is of uchar type
i suggest :
cv::Mat1b newImg;
newImg = grayImg > thresh ;
or
cv::Mat1b newImg;
newImg = grayImg < thresh ;
also look at the OpenCV Tutorials to know how to go through each and every pixel of an image

C++: OpenCV: fast pixel iteration

I'm trying to get BGR values from a streaming webcam image. I'm getting a memory access violation because I'm not using the pointer correctly in the nested for loop but I don't know what the syntax should be. I can't find documentation that is specific enough to the seemingly basic task I'm trying to do.
In addition to solving he memory access violation, I want to also be able to edit each pixel on the fly without having to do a deep copy but don't know what he syntax should be for that also.
This is the code I have so far:
int main(int argc, char** argv)
{
int c;
Mat img;
VideoCapture capture(0);
namedWindow("mainWin", CV_WINDOW_AUTOSIZE);
bool readOk = true;
while (capture.isOpened()) {
readOk = capture.read(img);
// make sure we grabbed the frame successfully
if (!readOk) {
std::cout << "No frame" << std::endl;
break;
}
int nChannels = img.channels();
int nRows = img.rows;
int nCols = img.cols * nChannels;
if (img.isContinuous())
{
nCols *= nRows;
nRows = 1;
}
int i, j;
uchar r, g, b;
for (i = 0; i < nRows; ++i)
{
for (j = 0; j < nCols; ++j)
{
r = img.ptr<uchar>(i)[nChannels*j + 2];
g = img.ptr<uchar>(i)[nChannels*j + 1];
b = img.ptr<uchar>(i)[nChannels*j + 0];
}
}
if (!img.empty()) imshow("mainWin", img);
c = waitKey(10);
if (c == 27)
break;
}
}
Your scanning loop is not correct. You should be only getting a pointer to the row once per row.
Since pixels are 3 byte quantities, it is easiest to treat them as a Vec3b.
You should have something like
uchar r, g, b;
for (int i = 0; i < img.rows; ++i)
{
cv::Vec3b* pixel = img.ptr<cv::Vec3b>(i); // point to first pixel in row
for (int j = 0; j < img.cols; ++j)
{
r = pixel[j][2];
g = pixel[j][1];
b = pixel[j][0];
}
}
OR
uchar r, g, b;
for (int i = 0; i < img.rows; ++i)
{
uchar* pixel = img.ptr<uchar>(i); // point to first color in row
for (int j = 0; j < img.cols; ++j)
{
b = *pixel++;
g = *pixel++;
r = *pixel++;
}
}
NOTE
It is fairly common to see Mat::at() used to access pixels sequentially like:
// DON'T DO THIS!
uchar r, g, b;
for (int i = 0; i < img.rows; ++i)
{
for (int j = 0; j < img.cols; ++j)
{
cv::Vec3b pixel = img.at<cv::Vec3b>(i, j);
r = pixel[2];
g = pixel[1];
b = pixel[0];
}
}
However such uses are inappropriate.
For every pixel access, at() needs to calculate an index by multiplying the row number and row length - and over a whole image that calculation can result in processing times considerably slower than with the code above (where ptr() does an equivalent calculation once per row.
Furthermore, in debug mode at() has an assertion that makes it much slower again.
If you are sure there is no padding between rows, it is possible to go faster by eliminating the call to ptr(). In this case the pixel pointer in the second loop above will after the end of each line be pointing at the start of the next line. But that wont work if your Mat is for example some region of interest of some other Mat.
On the other hand, if you were accessing pixels in a random fashion, rather than scanning sequentially like above, at() is then very appropriate.

using clustered indexes from point cloud in rgb image

I'm working with depth map acquired from two images (i took it from opencv StereoBM) and now i need to find clusters in them
I decided to use pcl region growing segmentation http://www.pointclouds.org/documentation/tutorials/region_growing_segmentation.php. I converted cv::Mat to point cloud after reading this article http://blog.martinperis.com/2012/01/3d-reconstruction-with-opencv-and-point.html and now i have cluster indexes
This to functions here https://gist.github.com/Daiver/5586252
Now i want, to use these indexes to display clusters on depth map from StereoBM (cv::Mat)
I'm trying this but I'm not satisfied with the results
pcl::PointCloud<pcl::PointXYZRGB>::Ptr cloud; //cloud from depth map and rgb image
std::vector <pcl::PointIndices> clusters;// clusters inices, extracted before
for(int j = 0; j < clusters.size(); j++)
{
cv::Mat to_show = cv::Mat::zeros(288, 384, cv::DataType<uchar>::type);//image has size that equal size of rgb image and depth map
for(int i = 0; i < clusters[j].indices.size(); i++)
{
to_show.data[clusters[j].indices[i]] = 200;// regions in this Mat must be equal with regions from depth map
}
cv::imshow("", to_show);
cv::waitKey();
}
Result
Some cluster
Another cluster
Visualized cloud
How i can project clusters to cv::Mat?
PS sorry for my writing mistakes. English in not my native language
UPD
I've tryed to "restore" depth map by using loops like loops in mat_to_cloud function
int counter = 0;
cv::Mat to_show = cv::Mat::zeros(288, 384, cv::DataType<uchar>::type);
for(int i = 0; i < cloud->height; i++)
{
for(int j = 0; j < cloud->width; j++)
{
to_show.at<uchar>(i, j) = cloud->at(counter).z;
counter++;
}
}
And another order of loops
int counter = 0;
cv::Mat to_show = cv::Mat::zeros(288, 384, cv::DataType::type);
for(int j = 0; j < cloud->width; j++)
{
for(int i = 0; i < cloud->height; i++)
{
to_show.at(i, j) = cloud->at(counter).z;
counter++;
}
}
I don't know why these image is similar
I have not worked with PCL before but it looks like this line might be wrong:
// regions in this Mat must be equal with regions from depth map
to_show.data[clusters[j].indices[i]] = 200;
to_show is an opencv matrix but you use the indices from the point cloud. You need to convert the indices to pixel coordinates first.
I solved my problem, but my solution is a little dirty and silly.
I Made my own simple reprojection function
void straight_reproject_cloud(cv::Mat& img_rgb, cv::Mat& img_disparity, pcl::PointCloud<pcl::PointXYZRGB>::Ptr& point_cloud_ptr)
{
uchar pr, pg, pb;
for (int i = 0; i < img_rgb.rows; i++)
{
uchar* rgb_ptr = img_rgb.ptr<uchar>(i);
uchar* disp_ptr = img_disparity.ptr<uchar>(i);
for (int j = 0; j < img_rgb.cols; j++)
{
uchar d = disp_ptr[j];
if ( d == 0 ) continue; //Discard bad pixels
pb = rgb_ptr[3*j];
pg = rgb_ptr[3*j+1];
pr = rgb_ptr[3*j+2];
//Insert info into point cloud structure
pcl::PointXYZRGB point;
point.x = j;
point.y = i;
point.z = d;
uint32_t rgb = (static_cast<uint32_t>(pr) << 16 |
static_cast<uint32_t>(pg) << 8 | static_cast<uint32_t>(pb));
point.rgb = *reinterpret_cast<float*>(&rgb);
point_cloud_ptr->push_back (point);
}
}
}
This function adds points to cloud directly from images, without changes
New cloud is not coincided with old reprojection but i can work with it
And now points coordinates in cloud is concided to coordinates in images
I can show all clusters from cloud in image:
for(int k = 0; k < clusters.size(); k++)
{
cv::Mat res = cv::Mat::zeros(img_rgb.rows, img_rgb.cols, CV_8U);
//for(int i =0 ; i < point_cloud_ptr->points.size(); i++)
for(int j =0 ; j < clusters[k].indices.size(); j++)
{
int i = clusters[k].indices[j];
int x = point_cloud_ptr->at(i).x;
int y = point_cloud_ptr->at(i).y;
res.at<uchar>(y, x) = (int)(point_cloud_ptr->at(i).z);
}
cv::imshow("rec2", res);
cv::waitKey();
}
New cloud