using clustered indexes from point cloud in rgb image - c++

Question

I'm working with depth map acquired from two images (i took it from opencv StereoBM) and now i need to find clusters in them
I decided to use pcl region growing segmentation http://www.pointclouds.org/documentation/tutorials/region_growing_segmentation.php. I converted cv::Mat to point cloud after reading this article http://blog.martinperis.com/2012/01/3d-reconstruction-with-opencv-and-point.html and now i have cluster indexes
This to functions here https://gist.github.com/Daiver/5586252
Now i want, to use these indexes to display clusters on depth map from StereoBM (cv::Mat)
I'm trying this but I'm not satisfied with the results
pcl::PointCloud<pcl::PointXYZRGB>::Ptr cloud; //cloud from depth map and rgb image
std::vector <pcl::PointIndices> clusters;// clusters inices, extracted before
for(int j = 0; j < clusters.size(); j++)
{
cv::Mat to_show = cv::Mat::zeros(288, 384, cv::DataType<uchar>::type);//image has size that equal size of rgb image and depth map
for(int i = 0; i < clusters[j].indices.size(); i++)
{
to_show.data[clusters[j].indices[i]] = 200;// regions in this Mat must be equal with regions from depth map
}
cv::imshow("", to_show);
cv::waitKey();
}
Result
Some cluster
Another cluster
Visualized cloud
How i can project clusters to cv::Mat?
PS sorry for my writing mistakes. English in not my native language
UPD
I've tryed to "restore" depth map by using loops like loops in mat_to_cloud function
int counter = 0;
cv::Mat to_show = cv::Mat::zeros(288, 384, cv::DataType<uchar>::type);
for(int i = 0; i < cloud->height; i++)
{
for(int j = 0; j < cloud->width; j++)
{
to_show.at<uchar>(i, j) = cloud->at(counter).z;
counter++;
}
}
And another order of loops
int counter = 0;
cv::Mat to_show = cv::Mat::zeros(288, 384, cv::DataType::type);
for(int j = 0; j < cloud->width; j++)
{
for(int i = 0; i < cloud->height; i++)
{
to_show.at(i, j) = cloud->at(counter).z;
counter++;
}
}
I don't know why these image is similar

Answer 1

I have not worked with PCL before but it looks like this line might be wrong:
// regions in this Mat must be equal with regions from depth map
to_show.data[clusters[j].indices[i]] = 200;
to_show is an opencv matrix but you use the indices from the point cloud. You need to convert the indices to pixel coordinates first.

Answer 2

I solved my problem, but my solution is a little dirty and silly.
I Made my own simple reprojection function
void straight_reproject_cloud(cv::Mat& img_rgb, cv::Mat& img_disparity, pcl::PointCloud<pcl::PointXYZRGB>::Ptr& point_cloud_ptr)
{
uchar pr, pg, pb;
for (int i = 0; i < img_rgb.rows; i++)
{
uchar* rgb_ptr = img_rgb.ptr<uchar>(i);
uchar* disp_ptr = img_disparity.ptr<uchar>(i);
for (int j = 0; j < img_rgb.cols; j++)
{
uchar d = disp_ptr[j];
if ( d == 0 ) continue; //Discard bad pixels
pb = rgb_ptr[3*j];
pg = rgb_ptr[3*j+1];
pr = rgb_ptr[3*j+2];
//Insert info into point cloud structure
pcl::PointXYZRGB point;
point.x = j;
point.y = i;
point.z = d;
uint32_t rgb = (static_cast<uint32_t>(pr) << 16 |
static_cast<uint32_t>(pg) << 8 | static_cast<uint32_t>(pb));
point.rgb = *reinterpret_cast<float*>(&rgb);
point_cloud_ptr->push_back (point);
}
}
}
This function adds points to cloud directly from images, without changes
New cloud is not coincided with old reprojection but i can work with it
And now points coordinates in cloud is concided to coordinates in images
I can show all clusters from cloud in image:
for(int k = 0; k < clusters.size(); k++)
{
cv::Mat res = cv::Mat::zeros(img_rgb.rows, img_rgb.cols, CV_8U);
//for(int i =0 ; i < point_cloud_ptr->points.size(); i++)
for(int j =0 ; j < clusters[k].indices.size(); j++)
{
int i = clusters[k].indices[j];
int x = point_cloud_ptr->at(i).x;
int y = point_cloud_ptr->at(i).y;
res.at<uchar>(y, x) = (int)(point_cloud_ptr->at(i).z);
}
cv::imshow("rec2", res);
cv::waitKey();
}
New cloud