OpenCV: How to visualize a depth image - c++

I am using a dataset in which it has images where each pixel is a 16 bit unsigned int storing the depth value of that pixel in mm. I am trying to visualize this as a greyscale depth image by doing the following:
cv::Mat depthImage;
depthImage = cv::imread("coffee_mug_1_1_1_depthcrop.png", CV_LOAD_IMAGE_ANYDEPTH | CV_LOAD_IMAGE_ANYCOLOR ); // Read the file
depthImage.convertTo(depthImage, CV_32F); // convert the image data to float type
namedWindow("window");
float max = 0;
for(int i = 0; i < depthImage.rows; i++){
for(int j = 0; j < depthImage.cols; j++){
if(depthImage.at<float>(i,j) > max){
max = depthImage.at<float>(i,j);
}
}
}
cout << max << endl;
float divisor = max / 255.0;
cout << divisor << endl;
for(int i = 0; i < depthImage.rows; i++){
for(int j = 0; j < depthImage.cols; j++){
cout << depthImage.at<float>(i,j) << ", ";
max = depthImage.at<float>(i,j) /= divisor;
cout << depthImage.at<float>(i,j) << endl;
}
}
imshow("window", depthImage);
waitKey(0);
However, it is only showing two colours this is because all of the values are close together i.e. in the range of 150-175 + the small values which show up black (see below).
Is there a way to normalize this data such that it will show various grey levels to highlight these small depth differences?

According to the documentation, the function imshow can be used with a variety of image types. It support 16-bit unsigned images, so you can display your image using
cv::Mat map = cv::imread("image", CV_LOAD_IMAGE_ANYCOLOR | CV_LOAD_IMAGE_ANYDEPTH);
cv::imshow("window", map);
In this case, the image value range is mapped from the range [0, 255*256] to the range [0, 255].
If your image only contains values on the low part of this range, you will observe an obscure image. If you want to use the full display range (from black to white), you should adjust the image to cover the expected dynamic range, one way to do it is
double min;
double max;
cv::minMaxIdx(map, &min, &max);
cv::Mat adjMap;
cv::convertScaleAbs(map, adjMap, 255 / max);
cv::imshow("Out", adjMap);

Adding to samg' answer, you can expand even more the range of your displayed image.
double min;
double max;
cv::minMaxIdx(map, &min, &max);
cv::Mat adjMap;
// expand your range to 0..255. Similar to histEq();
map.convertTo(adjMap,CV_8UC1, 255 / (max-min), -min);
// this is great. It converts your grayscale image into a tone-mapped one,
// much more pleasing for the eye
// function is found in contrib module, so include contrib.hpp
// and link accordingly
cv::Mat falseColorsMap;
applyColorMap(adjMap, falseColorsMap, cv::COLORMAP_AUTUMN);
cv::imshow("Out", falseColorsMap);
The result should be something like the one below

Ifimshow input has floating point data type then the function assumes that pixel values are in [0; 1] range. As result all values higher than 1 are displayed white.
So you need not divide your divisor by 255.

Adding to Sammy answer, if the original range color is [-min,max] and you want to perform histogram equalization and display the Depth color, the code should be like below:
double min;
double max;
cv::minMaxIdx(map, &min, &max);
cv::Mat adjMap;
// Histogram Equalization
float scale = 255 / (max-min);
map.convertTo(adjMap,CV_8UC1, scale, -min*scale);
// this is great. It converts your grayscale image into a tone-mapped one,
// much more pleasing for the eye
// function is found in contrib module, so include contrib.hpp
// and link accordingly
cv::Mat falseColorsMap;
applyColorMap(adjMap, falseColorsMap, cv::COLORMAP_AUTUMN);
cv::imshow("Out", falseColorsMap);

Related

Opencv Cumulative = true difference to false

i have this part of code. im beginner and want to understand this code. can someone explain me what happens if i set cumulative to true. Where is the difference to false. Would be nice if someone could explain me the difference.
i just see that the output is difference but i dont know why
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
#include <opencv2/imgproc.hpp>
using namespace std;
using namespace cv;
cv::Mat plotHistogram(cv::Mat &image, bool cumulative = false, int histSize = 256);
int main()
{
cv::Mat img = cv::imread("\\schrott.png"); // Read the file
if (img.empty()) // Check for invalid input
{
std::cout << "Could not open or find the frame" << std::endl;
return -1;
}
cv::Mat img_gray;
cv::cvtColor(img, img_gray, cv::COLOR_BGR2GRAY); // In case img is colored
cv::namedWindow("Input Image", cv::WINDOW_AUTOSIZE); // Create a window for display.
cv::imshow("Input Image", img);
cv::Mat hist;
hist = plotHistogram(img_gray);
cv::namedWindow("Histogram", cv::WINDOW_NORMAL); // Create a window for display.
cv::imshow("Histogram", hist);
cv::waitKey(0);
}
cv::Mat plotHistogram(cv::Mat &image, bool cumulative, int histSize) {
// Create Image for Histogram
int hist_w = 1024; int hist_h = 800;
int bin_w = cvRound((double)hist_w / histSize);
cv::Mat histImage(hist_h, hist_w, CV_8UC1, Scalar(255, 255, 255));
if (image.channels() > 1) {
cerr << "plotHistogram: Please insert only gray images." << endl;
return histImage;
}
// Calculate Histogram
float range[] = { 0, 256 };
const float* histRange = { range };
cv::Mat hist;
calcHist(&image, 1, 0, Mat(), hist, 1, &histSize, &histRange);
if (cumulative) {
cv::Mat accumulatedHist = hist.clone();
for (int i = 1; i < histSize; i++) {
accumulatedHist.at<float>(i) += accumulatedHist.at<float>(i - 1);
}
hist = accumulatedHist;
}
// Normalize the result to [ 0, histImage.rows ]
normalize(hist, hist, 0, histImage.rows, NORM_MINMAX, -1, Mat());
// Draw bars
for (int i = 1; i < histSize; i++) {
cv::rectangle(histImage, Point(bin_w * (i - 1), hist_h),
Point(bin_w * (i), hist_h - cvRound(hist.at<float>(i))),
Scalar(50, 50, 50), 1);
}
return histImage; // Not really call by value, as cv::Mat only saves a pointer to the image data
}
``
Without looking at the code: the difference between a histogram and a cumulative histogram is that the cumulative histogram at index i has the value of the normal histogram at index i, plus the value of the cumulative histogram at index i - 1. There is a c++ stl algorithm that does the same, and it's called std::partial_sum.
In other words, in image processing a cumulative histogram tells you how many pixels have at most a given color value c
For example, given an array [0, 1, 2, 1, 2, 3, 2, 1, 2, 3, 4] we can plot the histogram and cumulative histogram like so:
The X axis here is the value of the array element while the Y axis is the number of times this element occurs in the array. This is a typical pattern in image processing: in a color histogram the X axis is usually the value of your color. In a 8bpp grayscale image, for example, the X axis has values in the range 0..255. The Y axis then is the number of pixels that have that specific color value.
One important property of a cumulative histogram (in contrast to a normal histogram) is that it's monotonically increasing, i.e. h[i] >= h[i - 1] where i = 1..length(h) and h is the cumulative histogram. This allows operations like histogram equalization. Since a monotonically increasing sequence is by definition also a sorted sequence, you can also perform operation on it that are only allowed on sorted sequences, like binary search.
The next step is usually to calculate a normalized cumulative histogram, which is done by dividing each value in the cumulative histogram by the number of values in your original array. For example
a = [1, 2, 3, 4]
h = hist(a) // histogram of a
ch = partial_sum(h) // cumulative histogram of a
nch = ch / length(a) // normalized, cumulative histogram of a
Another example, given an array [1, 2, 3, 4, 3, 2, 1] we can plot the histogram and cumulative histogram like so:
The X axis here is the 1 based index in the array and the Y axis is the value of that element.
Here is another figure:
And another one, explaining the same:
Starting with what the cumulative histogram is might be good. A histogram is a distribution of the number of pixels according to their intensities but if the thing in question is a cumulative histogram; we don't find counts for a single bin in the vertical axis, but rather we map that counts the cumulative number of pixel intensity values in all of the bins up to the current bin. And linear cumulative distribution or cumulative histogram is essential for some image processing algorithms e.g image equalization.
Histogram (H):
For each pixel of the image
value = Intensity(pixel)
H(value)++
end
The cumulative histogram of the H:
When you set cumulative to true; you are now calculating the cumulative histogram therefore, it is normal for the outputs to be different. In each step of the iteration, you add the previous histogram value to the cumulative histogram.
if (cumulative) {
cv::Mat accumulatedHist = hist.clone();
for (int i = 1; i < histSize; i++) {
accumulatedHist.at<float>(i) += accumulatedHist.at<float>(i - 1);
}
hist = accumulatedHist;
}
You can think of this as a trick when switching from a normal histogram to a cumulative histogram.
accumulatedHist.at<float>(i) += accumulatedHist.at<float>(i - 1);
These references might be useful to understand the basic structure
Histograms
Histogram Equalization
Cumulative Distribution Function

Accessing certain pixel's intensity value(grayscale image) in openCV

I just realised that there is nothing on the web, after much searching about how to access a pixel's intensity value in OpenCv. A grayscale image.
Most online searches are about how to access BGR values of a colour image, like this one: Accessing certain pixel RGB value in openCV
image.at<> is basically for 3 channels, namely the BGR, out of curiousity, is there another similar method from OpenCV of accessing a certain pixel value of a grayscale image?
You can use image.at<uchar>(j,i) to acces a pixel value of a grayscale image.
cv::Mat::at<>() function is for every type of image, whether it is a single channel image or multi-channel image. The type of value returned just depends on the template argument provided to the function.
The value of grayscale image can be accessed like this:
//For 8-bit grayscale image.
unsigned char value = image.at<unsigned char>(row, column);
Make sure to return the correct data type depending on the image type (8u, 16u, 32f etc.).
For IplImage* image, you can use
uchar intensity = CV_IMAGE_ELEM(image, uchar, y, x);
For Mat image, you can use
uchar intensity = image.at<uchar>(y, x);
at(y,x)]++;
for(int i = 0; i < 256; i++)
cout<<histogram[i]<<" ";
// draw the histograms
int hist_w = 512; int hist_h = 400;
int bin_w = cvRound((double) hist_w/256);
Mat histImage(hist_h, hist_w, CV_8UC1, Scalar(255, 255, 255));
// find the maximum intensity element from histogram
int max = histogram[0];
for(int i = 1; i < 256; i++){
if(max < histogram[i]){
max = histogram[i];
}
}
// normalize the histogram between 0 and histImage.rows
for(int i = 0; i < 255; i++){
histogram[i] = ((double)histogram[i]/max)*histImage.rows;
}
// draw the intensity line for histogram
for(int i = 0; i < 255; i++)
{
line(histImage, Point(bin_w*(i), hist_h),
Point(bin_w*(i), hist_h - histogram[i]),
Scalar(0,0,0), 1, 8, 0);
}
// display histogram
namedWindow("Intensity Histogram", CV_WINDOW_AUTOSIZE);
imshow("Intensity Histogram", histImage);
namedWindow("Image", CV_WINDOW_AUTOSIZE);
imshow("Image", image);
waitKey();
return 0;
}

Converting to Floating Point Image from .tif

I am relatively new to C++ and coding in general and have run into a problem when attempting to convert an image to a floating point image. I am attempting to do this to eliminate round off errors with calculating the mean and standard deviation of pixel intensity for images as it starts to effect data quite substantially. My code is below.
Mat img = imread("Cells2.tif");
cv::namedWindow("stuff", CV_WINDOW_NORMAL);
cv::imshow("stuff",img);
CvMat cvmat = img;
Mat dst = cvCreateImage(cvGetSize(&cvmat),IPL_DEPTH_32F,1);
cvConvertScale(&cvmat,&dst);
cvScale(&dst,&dst,1.0/255);
cvNamedWindow("Test",CV_WINDOW_NORMAL);
cvShowImage("Test",&dst);
And I am running into this error
OpenCV Error: Bad argument (Array should be CvMat or IplImage) in an unknown function, file ......\modules\core\src\array.cpp, line 1238
I've looked everywhere and everyone was saying to convert img to CvMat which I attempted above.
When I did that as above code shows I get
OpenCV Error: Bad argument (Unknown array type) in unknown function, file ......\modules\core\src\matrix.cpp line 697
Thanks for your help in advance.
Just use C++ OpenCV interface instead of C interface and use convertTo function to convert between data types.
Mat img = imread("Cells2.tif");
cv::imshow("source",img);
Mat dst; // destination image
// check if we have RGB or grayscale image
if (img.channels() == 3) {
// convert 3-channel (RGB) 8-bit uchar image to 32 bit float
src.convertTo(dst, CV_32FC3);
}
else if (img.channels() == 1) {
// convert 1-chanel (grayscale) 8-bit uchar image to 32 bit float
img1.convertTo(dst, CV_32FC1);
}
// display output, note that to display dst image correctly
// we have to divide each element of dst by 255 to keep
// the pixel values in the range [0,1].
cv::imshow("output",dst/255);
waitKey();
Second part of the question To calculate the mean of all elements in dst
cv::Salar avg_pixel;
double avg;
// note that Scalar is a vector.
// If your image is RGB, Scalar will contain 3 values,
// representing color values for each channel.
avg_pixel = cv::mean(dst);
if (dst.channels() == 3) {
//if 3 channels
avg = (avg_pixel[0] + avg_pixel[1] + avg_pixel[2]) / 3;
}
if(dst.channels() == 1) {
avg = avg_pixel[0];
}
cout << "average element of m: " << avg << endl;
Here is my code for calculating the average in C++ OpenCV.
int NumPixels = img.total();
double avg;
double c;
for(int y = 0; y <= img.cols; y++)
for(int x = 0; x <= dst.rows; x++)
c+=img.at<uchar>(x,y);
avg = c/NumPixels;
cout << "Avg Value\n" << 255*avg;
For MATLAB I just load the image and take Q = mean(img(:)); which returns 1776.23
And for the return of 1612.36 I used cv:Scalar z = mean(dst);

Copying two images pixel by pixel

I am trying to work with each pixel from depth map. (I am implementing image segmentation.) I don't know how to work with pixels from image with depth higher than 1.
This sample code copies depth map to another cv::Mat pixel by pixel. It works fine, if I normalize it (depth of normalized image = 1). But it doesn't work with depth = 3, because .at<uchar> is wrong operation for this depth.
cv::Mat res;
cv::StereoBM bm(CV_STEREO_BM_NORMALIZED_RESPONSE);
bm(left, right, res);
std::cout<<"type "<<res.type()<<" depth "<<res.depth()<<" channels "<<res.channels()<<"\n";// type 3 depth 3 channels 1
cv::Mat tmp = cv::Mat::zeros(res.rows, res.cols, res.type());
for(int i = 0; i < res.rows; i++)
{
for(int j = 0; j < res.cols; j++)
{
tmp.at<uchar>(i, j) = res.at<uchar>(i, j);
//std::cout << (int)res.at<uchar>(i, j) << " ";
}
//std::cout << std::endl;
}
cv::imshow("tmp", normalize(tmp));
cv::imshow("res", normalize(res));
normilize function
cv::Mat normalize(cv::Mat const &depth_map)
{
double min;
double max;
cv::minMaxIdx(depth_map, &min, &max);
cv::Mat adjMap;
cv::convertScaleAbs(depth_map, adjMap, 255 / max);
return adjMap;
}
left image - tmp, right image - res
How can I get the pixel from image with depth equal to 3?
Mat::depth() returns value equal to a constant symbolising bit depth of the image. If You get depth equal to for example CV_32F, to get to the pixels You would need to use float instead of uchar.
CV_8S -> char
CV_8U -> uchar
CV_16U -> unsigned int
CV_16S -> int
CV_32F -> float
CV_64F -> double
Mat::channels() tells You how many values of that type are assigned to a coordinate. These multiple values can be extracted as cv::Vec. So if You have a two channel Mat with depth CV_8U, instead using Mat.at<uchar> You would need to go with Mat.at<Vec2b>, or Mat.at<Vec2f> for CV_32F one.
When your images are of depth 3, do this for copying pixel by pixel:
tmp.at<Vec3b>(i,j) = res.at<Vec3b>(i,j);
However, if you are copying the whole image , I do not understand the point of copying each pixel individually, unless you want to do different processing with different pixels.
You can just copy the whole image res to tmp by this:
res.copyTo(tmp);

color depth reduction with opencv and LUT

I'd like to perform a color reduction via color depth scaling.
Like this example:
the first image is CGA resolution, the second is EGA, the third is HAM.
I'd like to do it with cv::LUT because i think it is the betterway to do it.
I can do with greyscale with this code:
Mat img = imread("test1.jpg", 0);
uchar* p;
Mat lookUpTable(1, 256, CV_8U);
p = lookUpTable.data;
for( int i = 0; i < 256; ++i)
p[i] = 16 * (i/16)
LUT(img, lookUpTable, reduced);
original:
color reduced:
but if i try to do it with color I get strange result..
with this code:
imgColor = imread("test1.jpg");
Mat reducedColor;
int n = 16;
for (int i=0; i<256; i++) {
uchar value = floor(i/n) * n;
cout << (int)value << endl;
lut.at<Vec3b>(i)[2]= (value >> 16) & 0xff;
lut.at<Vec3b>(i)[1]= (value >> 8) & 0xff;
lut.at<Vec3b>(i)[0]= value & 0xff;
}
LUT(imgColor, lut, reducedColor);
You'll probably have moved on by now, but the root of the problem is that you are doing a 16-bit shift to uchar value, which is just 8-bits long. Even an 8-bit shift in this case is too much, as you'll erase all the bits in the uchar. Then there is the fact that the cv::LUT documentation explicitly states that src must be an "input array of 8-bit elements", which clearly isn't the case in your code. The net result is that only the first channel of the color image (the Blue channel) is transformed by cv::LUT.
The best way to work around these limitations is to split color images across channels, transform each channel separately, and then merge the transformed channels into a new color image. See the code below:
/*
Calculates a table of 256 assignments with the given number of distinct values.
Values are taken at equal intervals from the ranges [0, 128) and [128, 256),
such that both 0 and 255 are always included in the range.
*/
cv::Mat lookupTable(int levels) {
int factor = 256 / levels;
cv::Mat table(1, 256, CV_8U);
uchar *p = table.data;
for(int i = 0; i < 128; ++i) {
p[i] = factor * (i / factor);
}
for(int i = 128; i < 256; ++i) {
p[i] = factor * (1 + (i / factor)) - 1;
}
return table;
}
/*
Truncates channel levels in the given image to the given number of
equally-spaced values.
Arguments:
image
Input multi-channel image. The specific color space is not
important, as long as all channels are encoded from 0 to 255.
levels
The number of distinct values for the channels of the output
image. Output values are drawn from the range [0, 255] from
the extremes inwards, resulting in a nearly equally-spaced scale
where the smallest and largest values are always 0 and 255.
Returns:
Multi-channel images with values truncated to the specified number of
distinct levels.
*/
cv::Mat colorReduce(const cv::Mat &image, int levels) {
cv::Mat table = lookupTable(levels);
std::vector<cv::Mat> c;
cv::split(image, c);
for (std::vector<cv::Mat>::iterator i = c.begin(), n = c.end(); i != n; ++i) {
cv::Mat &channel = *i;
cv::LUT(channel.clone(), table, channel);
}
cv::Mat reduced;
cv::merge(c, reduced);
return reduced;
}
Both i and n are integers, therefore i/n is an integer. Perhaps you want it converted to double ((double)i/n) before taking the floor and multiplying by n?