OpenNI depth Image swap depth display - c++

I've been able to find/create some code that allows me to open the depth and color stream from the OpenNI enabled camera (It is an Orbbec Astra S to be specific). Except unlike with the standard OpenNI Viewer, My stream displays the closest points as darkest and further points as the lighter colors.
How would I be able to change this around so that the points closest to the cameras are shown as lighter (whites) and further away is shown as dark?
#include "stdafx.h"
#include "OpenNI.h"
#include <iostream>
#include <iomanip>
#include <fstream>
#include <string>
#include <array>
// OpenCV Header
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/calib3d/calib3d.hpp>
using namespace std;
using namespace cv;
using namespace openni;
//Recorder
int main(int argc, char** argv)
{
Device device;
VideoStream DepthStream,ColorStream;
VideoFrameRef DepthFrameRead,ColorFrameRead;
const char* deviceURI = openni::ANY_DEVICE;
if (argc > 1)
{
deviceURI = argv[1];
}
Status result = STATUS_OK;
result = OpenNI::initialize();
result = device.open(deviceURI);
result = DepthStream.create(device, openni::SENSOR_DEPTH);
result = DepthStream.start();
result = ColorStream.create(device, openni::SENSOR_COLOR);
result = ColorStream.start();
device.setImageRegistrationMode(ImageRegistrationMode::IMAGE_REGISTRATION_DEPTH_TO_COLOR);
int framenum = 0;
Mat frame;
while (true)
{
if (DepthStream.readFrame(&DepthFrameRead) == STATUS_OK)
{
cv::Mat cDepthImg(DepthFrameRead.getHeight(), DepthFrameRead.getWidth(),
CV_16UC1, (void*)DepthFrameRead.getData());
cv::Mat c8BitDepth;
cDepthImg.convertTo(c8BitDepth, CV_8U, 255.0 / (8000));
cv::imshow("Orbbec", c8BitDepth);
}
if (ColorStream.readFrame(&ColorFrameRead) == STATUS_OK)
{
ColorStream.readFrame(&ColorFrameRead);
const openni::RGB888Pixel* imageBuffer = (const openni::RGB888Pixel*)ColorFrameRead.getData();
frame.create(ColorFrameRead.getHeight(), ColorFrameRead.getWidth(), CV_8UC3);
memcpy(frame.data, imageBuffer, 3 * ColorFrameRead.getHeight()*ColorFrameRead.getWidth() * sizeof(uint8_t));
cv::cvtColor(frame, frame, CV_BGR2RGB); //this will put colors right
cv::imshow("frame", frame);
framenum++;
}
if (cvWaitKey(30) >= 0)
{
break;
}
}
DepthStream.destroy();
ColorStream.destroy();
device.close();
OpenNI::shutdown();
return 0;
}
-------------------EDIT-------------------
These Images are originally read in as 16bit images, which look like this (note how dark it is):
But after converting to an 8bit image, they look as follows:

The image you attached shows that the sensor is capturing the data with directly encoding the distance (in mm) of the object in the depth. This is quite normal for such depth cameras. What we want instead for displaying is higher values for objects closer to the sensor (this is totally opposite to the depth image encoding but useful for displaying).
One can devise a simple depth adjustment function if the operating range of the sensor is known. For Astra S, the operating range is from 0.35m to 2.5m. So what we want now is a function that converts 0.35m -> 2.5m and 2.5m -> 0.35m.
This is pretty straightforward, the only caveat is that you have to take care of the invalid depth pixel (depth == 0) yourself. Here is the code for doing this:
#include "include\opencv\cv.h"
#include "include\opencv\highgui.h"
cv::Mat adjustDepth(const cv::Mat& inImage)
{
// from https://orbbec3d.com/product-astra/
// Astra S has a depth in the range 0.35m to 2.5m
int maxDepth = 2500;
int minDepth = 350; // in mm
cv::Mat retImage = inImage;
for(int j = 0; j < retImage.rows; j++)
for(int i = 0; i < retImage.cols; i++)
{
if(retImage.at<ushort>(j, i))
retImage.at<ushort>(j, i) = maxDepth - (retImage.at<ushort>(j, i) - minDepth);
}
return retImage;
}
int main ()
{
cv::Mat inImage;
inImage = cv::imread("testImage.png", CV_LOAD_IMAGE_UNCHANGED);
cv::Mat adjustedDepth = adjustDepth(inImage);
cv::Mat dispImage;
adjustedDepth.convertTo(dispImage, CV_8UC1, 255.0f/2500.0f);
cv::imshow(" ", dispImage);
//cv::imwrite("testImageAdjusted.png", adjustedDepth);
//cv::imwrite("savedImage.png", dispImage);
cv::waitKey(0);
return 0;
}
Here is the output renormalized depth image:
If one wants to further explore what happens in such readjustment function, one can have a look at the histogram for image both before and after applying the adjustment.
Histogram for input depth image (D):
Histogram for negative input depth image (-D):
Histogram for (maxVal-(D-minVal)):
Hope this answers your question.

Related

How to prepare image data for kmeans opencv function input? [duplicate]

I am making a function using C++ and OpenCV that will detect the color of a pixel in an image, determine what color range it is in, and replace it with a generic color. For example, green could range from dark green to light green, the program would determine that its still green and replace it with a simple green, making the output image very simple looking. everything is set up but I'm having trouble defining the characteristics of each range and was curious if anyone knows or a formula that, given BGR values, could determine the overall color of a pixel. If not I'll have to do much experimentation and make it myself, but if something already exists that'd save time. I've done plenty of research and haven't found anything so far.
If you want to make your image simpler (i.e. with less colors), but good looking, you have a few options:
A simple approach would be to divide (integer division) by a factor N the image, and then multiply by a factor N.
Or you can divide your image into K colors, using some clustering algorithm such as kmeans showed here, or median-cut algorithm.
Original image:
Reduced colors (quantized, N = 64):
Reduced colors (clustered, K = 8):
Code Quantization:
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat3b img = imread("path_to_image");
imshow("Original", img);
uchar N = 64;
img /= N;
img *= N;
imshow("Reduced", img);
waitKey();
return 0;
}
Code kmeans:
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat3b img = imread("path_to_image");
imshow("Original", img);
// Cluster
int K = 8;
int n = img.rows * img.cols;
Mat data = img.reshape(1, n);
data.convertTo(data, CV_32F);
vector<int> labels;
Mat1f colors;
kmeans(data, K, labels, cv::TermCriteria(), 1, cv::KMEANS_PP_CENTERS, colors);
for (int i = 0; i < n; ++i)
{
data.at<float>(i, 0) = colors(labels[i], 0);
data.at<float>(i, 1) = colors(labels[i], 1);
data.at<float>(i, 2) = colors(labels[i], 2);
}
Mat reduced = data.reshape(3, img.rows);
reduced.convertTo(reduced, CV_8U);
imshow("Reduced", reduced);
waitKey();
return 0;
}
Yes, what you probably mean by "Overall color of a pixel" is either the "Hue" or "Saturation" of the color.
So you want a formula that transform RGB to HSV (Hue, Saturation, Value), and then you would only be interested by the Hue or Saturation values.
See: Algorithm to convert RGB to HSV and HSV to RGB in range 0-255 for both
EDIT: You might need to max out the saturation, and then convert it back to RGB, and inspect which value is the highest (for instance (255,0,0), or (255,0,255), etc.
If you want to access RGB value of all pixels , then below is code,
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat image = imread("image_path");
for(int row = 1; row < image.rows; row++)
{
for(int col = 1; col < image.cols; col++)
{
Vec3b rgb = image.at<Vec3b>(row, col);
}
}
}

Why Png Compression doesn't change destination size (C++)? OpenCV VS2010

I have tested this code with various values from compression_params.push_back(1); to compression_params.push_back(9); but the PNG image always has same size. 1950x1080 (contains screenshot of Google map - not the satellite photo) has 2,36 MB (2 477 230 bytes. Is this normal is takes so much. I thought png images are small size if they do not contain photos.
vector<int> compression_params;
compression_params.push_back(CV_IMWRITE_PNG_COMPRESSION);
compression_params.push_back(1);
try {
imwrite("RGB_1.png", source, compression_params);
}
catch (runtime_error& ex) {
fprintf(stderr, "Exception converting image to PNG format: %s\n", ex.what());
return 1;
}
Why is it? Also I cannot find out how to create the PNG object in memory (to keep the encode data in buffer). This means, I would like to save more images into one file (e.g database) so I need to convert into buffer and then save add buffer to file. Is it possible to do it usin OpenCV? Your tips welcome.
I think PNG should support some feature where the algorithm auto-selects background color, so if you see some cv::Scallar(200,200,200) takes too many place on the image, the algorithm could set it as background color and it is removed from the image so the image should take small place. So when it takes same size as regular PNG or even more, that doesn't give any sense.
i am not an expert on this subject but tried to test some compression_params maybe you will find the answer by testing the code below.
probably adding the following lines will do the trick.
compression_params.push_back(IMWRITE_PNG_STRATEGY);
compression_params.push_back(IMWRITE_PNG_STRATEGY_DEFAULT);
or you can try other alternatives accordind to documentation
also i opened an issue
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <iostream>
#include <string>
using namespace cv;
using namespace std;
void createAlphaMat(Mat &mat)
{
CV_Assert(mat.channels() == 4);
for (int i = 0; i < mat.rows; ++i) {
for (int j = 0; j < mat.cols; ++j) {
Vec4b& bgra = mat.at<Vec4b>(i, j);
bgra[0] = UCHAR_MAX; // Blue
bgra[1] = saturate_cast<uchar>((float(mat.cols - j)) / ((float)mat.cols) * UCHAR_MAX); // Green
bgra[2] = saturate_cast<uchar>((float(mat.rows - i)) / ((float)mat.rows) * UCHAR_MAX); // Red
bgra[3] = saturate_cast<uchar>(0.5 * (bgra[1] + bgra[2])); // Alpha
}
}
}
int main( int argc, char** argv )
{
// Create mat with alpha channel
Mat mat(480, 640, CV_8UC4);
createAlphaMat(mat);
vector<int> compression_params;
compression_params.push_back(IMWRITE_PNG_COMPRESSION);
compression_params.push_back(0);
compression_params.push_back(IMWRITE_PNG_STRATEGY);
compression_params.push_back(IMWRITE_PNG_STRATEGY_DEFAULT);
for (int i = 0; i < 10; i++)
{
compression_params[1] = i;
imwrite(format("alpha%d.png",i), mat, compression_params);
}
return 0;
}

Is there a formula to determine overall color given BGR values? (OpenCV and C++)

I am making a function using C++ and OpenCV that will detect the color of a pixel in an image, determine what color range it is in, and replace it with a generic color. For example, green could range from dark green to light green, the program would determine that its still green and replace it with a simple green, making the output image very simple looking. everything is set up but I'm having trouble defining the characteristics of each range and was curious if anyone knows or a formula that, given BGR values, could determine the overall color of a pixel. If not I'll have to do much experimentation and make it myself, but if something already exists that'd save time. I've done plenty of research and haven't found anything so far.
If you want to make your image simpler (i.e. with less colors), but good looking, you have a few options:
A simple approach would be to divide (integer division) by a factor N the image, and then multiply by a factor N.
Or you can divide your image into K colors, using some clustering algorithm such as kmeans showed here, or median-cut algorithm.
Original image:
Reduced colors (quantized, N = 64):
Reduced colors (clustered, K = 8):
Code Quantization:
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat3b img = imread("path_to_image");
imshow("Original", img);
uchar N = 64;
img /= N;
img *= N;
imshow("Reduced", img);
waitKey();
return 0;
}
Code kmeans:
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat3b img = imread("path_to_image");
imshow("Original", img);
// Cluster
int K = 8;
int n = img.rows * img.cols;
Mat data = img.reshape(1, n);
data.convertTo(data, CV_32F);
vector<int> labels;
Mat1f colors;
kmeans(data, K, labels, cv::TermCriteria(), 1, cv::KMEANS_PP_CENTERS, colors);
for (int i = 0; i < n; ++i)
{
data.at<float>(i, 0) = colors(labels[i], 0);
data.at<float>(i, 1) = colors(labels[i], 1);
data.at<float>(i, 2) = colors(labels[i], 2);
}
Mat reduced = data.reshape(3, img.rows);
reduced.convertTo(reduced, CV_8U);
imshow("Reduced", reduced);
waitKey();
return 0;
}
Yes, what you probably mean by "Overall color of a pixel" is either the "Hue" or "Saturation" of the color.
So you want a formula that transform RGB to HSV (Hue, Saturation, Value), and then you would only be interested by the Hue or Saturation values.
See: Algorithm to convert RGB to HSV and HSV to RGB in range 0-255 for both
EDIT: You might need to max out the saturation, and then convert it back to RGB, and inspect which value is the highest (for instance (255,0,0), or (255,0,255), etc.
If you want to access RGB value of all pixels , then below is code,
#include <opencv2/opencv.hpp>
using namespace std;
using namespace cv;
int main()
{
Mat image = imread("image_path");
for(int row = 1; row < image.rows; row++)
{
for(int col = 1; col < image.cols; col++)
{
Vec3b rgb = image.at<Vec3b>(row, col);
}
}
}

Knowing a pixel value after making an RGBtoHSV conversion OpenCv

Im trying to get the H,S and V Values of an image, so i convert an RGB image to HSV, and then just ask for the desired values, and then print them.. Im not quite sure im making this right, because when printing the Value (V of hsV) i get values of 100+ and i understand that the V just goes to 0-100, maybe im not using a correct method, here's the code:
#include "opencv/highgui.h"
#include "opencv/cv.h"
#include <cstdlib>
#include <iostream>
#include <stdio.h>
using namespace std;
int main(int argc, char** argv) {
int i=0,total=0;
IplImage* img = cvLoadImage( argv[1] );
IplImage* hsv;
CvSize size;
int key = 0, depth;
size = cvGetSize(img);
depth = img->depth;
hsv = cvCreateImage(size, depth, 3);
cvCvtColor( img, hsv, CV_BGR2HSV );
for(i=0;i<480;i++){ //asking for the values in \ form (1,1)(2,2),...(480,480)
CvScalar s;
s = cvGet2D(hsv,i,i);
printf("s=%f\n,s.val[2]); //s.val[2] equals to hs**V** right?
}
cvReleaseImage(&img);
cvReleaseImage(&val);
return 0;
}
The other answer here is correct but here is a code snippet that I have to calculate the V channel in opencv. I get the value from the Gimp app and this function gives me the opencv value.
//Max values: App HSV H=360 S=100 V=100 OpenCV H=180 S=255 V=255
double newHSV(double value)
{
//new_val = value * opencv_max_range / other_app_max_range
double newValue = value * 255 / 100;
return newValue;
}
To check your opencv HSV values in another application like Gimp, just calculate the formula to:
gimp_value = opencv_value * other_app_max_range / opencv_max_range
The way you're doing it is correct. Just that values are a little different.
H should ideally go from 0-360. But because a byte can only hold 0-255, H values are halved. So the range is 0-180.
V and S use the full range of 0-255 to specify value and saturation.
You can read more about it here: http://opencv.willowgarage.com/documentation/python/miscellaneous_image_transformations.html#cvtcolor

OpenCV Grid Area

I want to find the non-white area of an image from a camera using OpenCV. I can already find circles using images from my web cam. I want to make a grid or something so I can determine the percent of the image is not white. Any ideas?
If you want to find the percentage of pixels in your image which is not white, why don't you just count all the pixels which are not white and divide it by the total number of pixels in the image?
Code in C
#include <stdio.h>
#include <cv.h>
#include <cxcore.h>
#include <highgui.h>
int main()
{
// Acquire the image (I'm reading it from a file);
IplImage* img = cvLoadImage("image.bmp",1);
int i,j,k;
// Variables to store image properties
int height,width,step,channels;
uchar *data;
// Variables to store the number of white pixels and a flag
int WhiteCount,bWhite;
// Acquire image unfo
height = img->height;
width = img->width;
step = img->widthStep;
channels = img->nChannels;
data = (uchar *)img->imageData;
// Begin
WhiteCount = 0;
for(i=0;i<height;i++)
{
for(j=0;j<width;j++)
{ // Go through each channel of the image (R,G, and B) to see if it's equal to 255
bWhite = 0;
for(k=0;k<channels;k++)
{ // This checks if the pixel's kth channel is 255 - it can be faster.
if (data[i*step+j*channels+k]==255) bWhite = 1;
else
{
bWhite = 0;
break;
}
}
if(bWhite == 1) WhiteCount++;
}
}
printf("Percentage: %f%%",100.0*WhiteCount/(height*width));
return 0;
}
You can use cv::countNonZero and subtract if your image is only black and white.