ITK - Calculate texture features for segmented 3D brain MRI - c++

I'm trying to calculate texture features for a segmented 3D brain MRI using ITK library with C++. So I followed this example. The example takes a 3D image, and extracts 3 different features for all 13 possible spatial directions. In my program, I just want for a given 3D image to get :
Energy
Correlation
Inertia
Haralick Correlation
Inverse Difference Moment
Cluster Prominence
Cluster Shade
Here is what I have so far :
//definitions of used types
typedef itk::Image<float, 3> InternalImageType;
typedef itk::Image<unsigned char, 3> VisualizingImageType;
typedef itk::Neighborhood<float, 3> NeighborhoodType;
typedef itk::Statistics::ScalarImageToCooccurrenceMatrixFilter<InternalImageType>
Image2CoOccuranceType;
typedef Image2CoOccuranceType::HistogramType HistogramType;
typedef itk::Statistics::HistogramToTextureFeaturesFilter<HistogramType> Hist2FeaturesType;
typedef InternalImageType::OffsetType OffsetType;
typedef itk::AddImageFilter <InternalImageType> AddImageFilterType;
typedef itk::MultiplyImageFilter<InternalImageType> MultiplyImageFilterType;
void calcTextureFeatureImage (OffsetType offset, InternalImageType::Pointer inputImage)
{
// principal variables
//Gray Level Co-occurance Matrix Generator
Image2CoOccuranceType::Pointer glcmGenerator=Image2CoOccuranceType::New();
glcmGenerator->SetOffset(offset);
glcmGenerator->SetNumberOfBinsPerAxis(16); //reasonable number of bins
glcmGenerator->SetPixelValueMinMax(0, 255); //for input UCHAR pixel type
Hist2FeaturesType::Pointer featureCalc=Hist2FeaturesType::New();
//Region Of Interest
typedef itk::RegionOfInterestImageFilter<InternalImageType,InternalImageType> roiType;
roiType::Pointer roi=roiType::New();
roi->SetInput(inputImage);
InternalImageType::RegionType window;
InternalImageType::RegionType::SizeType size;
size.Fill(50);
window.SetSize(size);
window.SetIndex(0,0);
window.SetIndex(1,0);
window.SetIndex(2,0);
roi->SetRegionOfInterest(window);
roi->Update();
glcmGenerator->SetInput(roi->GetOutput());
glcmGenerator->Update();
featureCalc->SetInput(glcmGenerator->GetOutput());
featureCalc->Update();
std::cout<<"\n Entropy : ";
std::cout<<featureCalc->GetEntropy()<<"\n Energy";
std::cout<<featureCalc->GetEnergy()<<"\n Correlation";
std::cout<<featureCalc->GetCorrelation()<<"\n Inertia";
std::cout<<featureCalc->GetInertia()<<"\n HaralickCorrelation";
std::cout<<featureCalc->GetHaralickCorrelation()<<"\n InverseDifferenceMoment";
std::cout<<featureCalc->GetInverseDifferenceMoment()<<"\nClusterProminence";
std::cout<<featureCalc->GetClusterProminence()<<"\nClusterShade";
std::cout<<featureCalc->GetClusterShade();
}
The program works. However I have this problem : it gives the same results for different 3D images, even when I change the window size.
Does any one used ITK to do this ? If there is any other method to achieve that, could anyone point me to a solution please ?
Any help will be much apreciated.

I think that your images have only one gray scale level. For example, if you segment your images using itk-snap tool, when you save the result of the segmentation, itk-snap save it with one gray scale level. So, if you try to calculate texture features for images segmented with itk-snap you'll always have the same results even if you change the images or the window size because you have only one gray scale level in the co-occurrence matrix. Try to run your program using unsegmented images, you'll certainly have different results.
EDIT :
To calculate texture features for segmented images, try another segmentation method which saves the original gray scale levels of the unsegmented image.

Something strange in your code is size.Fill(50), while in the example they show it should hold the image dimension:
size.Fill(3); //window size=3x3x3

Related

How to do this specific tensor transformation in Eigen?

I am looking for an idiomatic and efficient solution for this problem:
Let's say I have 3D Tensor where I want to represent an image with 100*100 pixels on 3 color channels,
Eigen::Tensor<int, 3> input(3,100,100);
The output I would like to get could be stored in
Eigen::Tensor<int, 4> output(3,3,100,100);
I would like to project the 3D input into the 4D output in a way that each color channel in the original tensor would have its own individual 3D tensor in the output, where each channel would contain the same values, that is
tensor(0,0,42,42) = tensor(0,1,42,42) = tensor(0,2,42,42)
tensor(0,0,12,12) = tensor(0,1,12,12) = tensor(0,2,12,12)
Illustrated on a picture:
Originally I wanted to solve this method:
Chip the individual color channels.
Broadcast the individual color channels into the size I need,
Reshape the broadcasted result into the desirable format(this is just a 3D Tensor at this point)
Concatenate the individual 3D Tensors into a big 4d one.
I have two problems with this approach.
Firstly, I just can not get the reshaping right, it always gives back a reshaped tensor with the dimensionality I want, but the coefficients get shuffled. I started to experiment with the layout of the Tensors, but it did not seem to help.
Secondly, this seems to be very tedious, I just feel like there should be a more convenient way to achieve this but I could not find any cue about that in the documentation.

Incomplete Delaunay triangulation

I'm using C++ and OpenCV to create a Delaunay triangle mesh from user-specified sample points on an image (which will then be extrapolated across the domain using the FEM for the relevant ODE).
Since the 4 corners of the (rectangular) image are in the list of vertices supplied to Subdiv2D, I expect the outer convex hull of the triangulation to trace the perimeter of the image. However, very frequently, there are missing elements around the outside.
Sometimes I can get the expected result by nudging the coordinates of certain points to avoid high aspect ratio triangles. But this is not a solution as in general the user most be able to specify any valid coordinates.
An example output is like this: CV Output. Elements are in white with black edges. At the bottom and right edges, no triangles have been added, and you can see through to the black background.
How can I make the outer convex hull of the triangulation trace the image perimeter with no gaps please?
Here is a MWE (with a plotting function included):
#include <opencv2/opencv.hpp>
#include <vector>
void DrawDelaunay(cv::Mat& image,cv::Subdiv2D& subdiv);
int main(int argc,char** argv)
{
// image dim
int width=3440;
int height=2293;
// sample coords
std::vector<int> x={0,width-1,width-1,0,589,1015,1674,2239,2432,3324,2125,2110,3106,3295,1298,1223,277,208,54,54,1749,3245,431,1283,1397,3166};
std::vector<int> y={0,0,height-1,height-1,2125,1739,1154,817,331,143,1377,2006,1952,1501,872,545,812,310,2180,54,2244,2234,1387,1412,118,1040};
// add delaunay nodes
cv::Rect rect(0,0,width,height);
cv::Subdiv2D subdiv(rect);
for(size_t i=0;i<x.size();++i)
{
cv::Point2f p(x[i],y[i]);
subdiv.insert(p);
}
// draw elements
cv::Mat image(height,width,CV_8U);
DrawDelaunay(image,subdiv);
cv::resize(image,image,cv::Size(),0.3,0.3);
cv::imshow("Delaunay",image);
cv::waitKey(0);
return 0;
}
void DrawDelaunay(cv::Mat& image,cv::Subdiv2D& subdiv)
{
std::vector<cv::Vec6f> elements;
subdiv.getTriangleList(elements);
std::vector<cv::Point> pt(3);
for(size_t i=0;i<elements.size();++i)
{
// node coords
cv::Vec6f t=elements[i];
pt[0]=cv::Point(cvRound(t[0]),cvRound(t[1]));
pt[1]=cv::Point(cvRound(t[2]),cvRound(t[3]));
pt[2]=cv::Point(cvRound(t[4]),cvRound(t[5]));
// element edges
cv::Scalar black(0,0,0);
cv::line(image,pt[0],pt[1],black,3);
cv::line(image,pt[1],pt[2],black,3);
cv::line(image,pt[2],pt[0],black,3);
// element fill
int nump=3;
const cv::Point* pp[1]={&pt[0]};
cv::fillPoly(image,pp,&nump,1,cv::Scalar(255,0,0));
}
}
If relevant, I coded this in Matlab first where the Delaunay triangulation worked exactly as I expected.
My solution was to add a border around the 'cv::Rect rect' provided to cv::Subdiv2D, making it larger in width and height than the image (20% larger seems to work well).
Then instead of adding nodes to the corners of the image, I added 4 corner nodes and 4 edge nodes to the perimiter of this enlarged 'cv::Rect rect' variable which holds the Delaunay points.
This seems to solve the problem. I think what was happening was that if the user placed any samples near the edge of the image, it resulted in high aspect ratio triangles at the edges. This ticket suggests there is a bug around this in the OpenCV implementation of the Delaunay algorithm.
My solution hopefully means that corner and edge nodes are never too close to user samples, side-stepping the issue.
I haven't tested this extensively yet. I'm not sure how robust the solution will turn out to be. It has worked so far.
I'm still interested to know of other solutions.
I ran your data points through the Tinfour project's demo application and got the results shown below. It looks like your data is fine. Unfortunately, the Tinfour project is written in Java and you're working in C++, so it will have limited value to you.
Since you plan on using Finite Element Methods, you might want to see whether there is any way you can run a Delaunay Refinement operation over your data to improve the geometry. The skinny triangles sometimes lead to numerical issues when using FEM software.

How to project IR image on a 2D plane using OpenCV and PCL

I have a Kinect and I'm using OpenCV and point cloud library. I would like to project the IR Image onto a 2D plane for forklift pallet detection. How would I do that?
I'm trying to detect the pallet in the forklift here is an image:
Where are the RGB data? You can use it to help with the detection. You do not need to project the image onto any plane to detect a pellet. There are basically 2 ways used for detection
non-deterministic based on neural network, fuzzy logic, machine learning, etc
This approach need a training dataset to recognize the object. Much experience is needed for proper training set and classifier architecture/topology selection. But other then that you do not need to program it... as usually some readily available lib/tool is used just configure and pass the data.
deterministic based on distance or correlation coefficients
I would start with detecting specific features like:
pallet has specific size
pallet has sharp edges and specific geometry shape in depth data
pallet has specific range of colors (yellowish wood +/- lighting and dirt)
Wood has specific texture patterns
So compute some coefficient for each feature how close the object is to real pallet. And then just treshold the distance of all coefficients combined (possibly weighted as some features are more robust).
I do not use the #1 approach so I would go for #2. So combine the RGB and depth data (they have to be matched exactly). Then segmentate the image (based on depth and color). After that for each found object classify if it is pallet ...
[Edit1]
Your colored image does not correspond to depth data. The aligned gray-scale has poor quality and the depth data image is also very poor. Is the depth data processed somehow (loosing precision)? If you look at your data from different sides:
You can see how poor it is so I doubt you can use depth data for detection at all...
PS. I used my Align already captured rgb and depth images for the visualization.
The only thing left is the colored image and detect areas with matching color only. Then detect the features and classify. The color of your pallet in the image is almost white. Here HSV reduced colors to basic 16 colors (too lazy to segmentate)
You should obtain range of colors of the pallets possible by your setup to ease up the detection. Then check those objects for the features like size, shape,area,circumference...
[Edit2]
So I would start with Image preprocessing:
convert to HSV
treshold only pixels close to pallet color
I chose (H=40,S=18,V>100) as a pallet color. My HSV ranges are <0,255> per channel so Hue angle difference can be only <-180deg,+180deg> max which corresponds to <-128,+128> in my ranges.
remove too thin areas
Just scan all Horizontal an Vertical lines count consequent set pixels and if too small size recolor them to black...
This is the result:
On the left the original image (downsized so it fits to this page), In the middle is the color treshold result and last is the filtering out of small areas. You can play with tresholds and pallet color to change behavior to suite your needs.
Here C++ code:
int tr_d=10; // min size of pallet [pixels[
int h,s,v,x,y,xx;
color c;
pic1=pic0;
pic1.pf=_pf_rgba;
pic2.resize(pic1.xs*3,pic1.ys); xx=0;
pic2.bmp->Canvas->Draw(xx,0,pic0.bmp); xx+=pic1.xs;
// [color selection]
for (y=0;y<pic1.ys;y++)
for (x=0;x<pic1.xs;x++)
{
// get color from image
c=pic0.p[y][x];
rgb2hsv(c);
// distance to white-yellowish color in HSV (H=40,S=18,V>100)
h=c.db[picture::_h]-40;
s=c.db[picture::_s]-18;
v=c.db[picture::_v];
// hue is cyclic angular so use only shorter angle
if (h<-128) h+=256;
if (h>+128) h-=256;
// abs value
if (h< 0) h=-h;
if (s< 0) s=-s;
// treshold close colors
c.dd=0;
if (h<25)
if (s<25)
if (v>100)
c.dd=0x00FFFFFF;
pic1.p[y][x]=c;
}
pic2.bmp->Canvas->Draw(xx,0,pic1.bmp); xx+=pic1.xs;
// [remove too thin areas]
for (y=0;y<pic1.ys;y++)
for (x=0;x<pic1.xs;)
{
for ( ;x<pic1.xs;x++) if ( pic1.p[y][x].dd) break; // find set pixel
for (h=x;x<pic1.xs;x++) if (!pic1.p[y][x].dd) break; // find unset pixel
if (x-h<tr_d) for (;h<x;h++) pic1.p[y][h].dd=0; // if too small size recolor to zero
}
for (x=0;x<pic1.xs;x++)
for (y=0;y<pic1.ys;)
{
for ( ;y<pic1.ys;y++) if ( pic1.p[y][x].dd) break; // find set pixel
for (h=y;y<pic1.ys;y++) if (!pic1.p[y][x].dd) break; // find unset pixel
if (y-h<tr_d) for (;h<y;h++) pic1.p[h][x].dd=0; // if too small size recolor to zero
}
pic2.bmp->Canvas->Draw(xx,0,pic1.bmp); xx+=pic1.xs;
See how to extract the borders of an image (OCT/retinal scan image) for the description of picture and color. Or look at any of my DIP/CV tagged answers. Now the code is well commented and straightforward but just need to add:
You can ignore pic2 stuff it is just the image posted above so I do not need to manually print screen and merge the subresult in paint... To improve robustness you should add enhancing of dynamic range (so the tresholds have the same conditions for any input images). Also you should compare to more then just single color (if more wood types of pallet is present).
Now you should segmentate or label the areas
loop through entire image
find first pixel set with the pallet color
flood fill the area with some distinct ID color different from set pallet color
I use black 0x00000000 space and white 0x00FFFFFF as pallete pixel color. So use ID={1,2,3,4,5...}. Also remember number of filled pixels (that is your area) so you do not need to compute it again. You can also compute bounding box directly while filling.
compute and compare features
You need to experiment with more then one image. To find out what properties are good for detection. I would go for circumference length vs area ratio. and or bounding box size... The circumference can be extracted by simply selecting all pixels with proper ID color neighboring black pixel.
See also similar Fracture detection in hand using image proccessing
Good luck and have fun ...

Armadillo porting imagesc to save image bitmap from matrix

I have this matlab code to display image object after do super spectrogram (stft, couple plca...)
t = z2 *stft_options.hop/stft_options.sr;
f = stft_options.sr*[0:size(spec_t,1)-1]/stft_options.N/1000;
max_val = max(max(db(abs(spec_t))));
imagesc(t, f, db(abs(spec_t)),[max_val-60 max_val]);
And get this result:
I was porting to C++ successfully by using Armadillo lib and get the mat results:
mat f,t,spec_t;
The problem is that I don't have any idea for converting bitmap like imagesc in matlab.
I searched and found this answer, but seems it doesn't work in my case because:
I use a double matrix instead of integer matrix, which can't be mark as bitmap color
The imagesc method take 4 parameters, which has the bounds with vectors x and y
The imagesc method also support scale ( I actually don't know how it work)
Does anyone have any suggestion?
Update: Here is the result of save method in Armadillo. It doesn't look like spectrogram image above. Do I miss something?
spec_t.save("spec_t.png", pgm_binary);
Update 2: save spectrogram with db and abs
mat spec_t_mag = db(abs(spec_t)); // where db method: m = 10 * log10(m);
mag_spec_t.save("mag_spec_t.png", pgm_binary);
And the result:
Armadillo is a linear algebra package, AFAIK it does not provide graphics routines. If you use something like opencv for those then it is really simple.
See this link about opencv's imshow(), and this link on how to use it in a program.
Note that opencv (like most other libraries) uses row-major indexing (x,y) and Armadillo uses column-major (row,column) indexing, as explained here.
For scaling, it's safest to convert to unsigned char yourself. In Armadillo that would be something like:
arma::Mat<unsigned char> mat2=255*(mat-mat.min())/(mat.max()-mat.min());
The t and f variables are for setting the axes, they are not part of the bitmap.
For just writing an image you can use Armadillo. Here is a description on how to write portable grey map (PGM) and portable pixel map (PPM) images. PGM export is only possible for 2D matrices, PPM export only for 3D matrices, where the 3rd dimension (size 3) are the channels for red, green and blue.
The reason your matlab figure looks prettier is because it has a colour map: a mapping of every value 0..255 to a vector [R, G, B] specifying the relative intensity of red, green and blue. A photo has an RGB value at every point:
colormap(gray);
x=imread('onion.png');
imagesc(x);
size(x)
That's the 3rd dimension of the image.
Your matrix is a 2d image, so the most natural way to show it is as grey levels (as happened for your spectrum).
x=mean(x,3);
imagesc(x);
This means that the R, G and B intensities jointly increase with the values in mat. You can put a colour map of different R,G,B combinations in a variable and use that instead, i.e. y=colormap('hot');colormap(y);. The variable y shows the R,G,B combinations for the (rescaled) image values.
It's also possible to make your own colour map (in matlab you can specify 64 R, G, and B combinations with values between 0 and 1):
z[63:-1:0; 1:2:63 63:-2:0; 0:63]'/63
colormap(z);
Now for increasing image values, red intensities decrease (starting from the maximum level), green intensities quickly increase then decrease, and blue values increase from minuimum to maximum.
Because PPM appears (I don't know the format) not to support colour maps, you need to specify the R,G,B values in a 3D array. For a colour order similar to z you would neet to make a Cube<unsigned char> c(ysize, xsize, 3) and then for every pixel y, x in mat2, do:
c(y,x,0) = 255-mat2(y,x);
c(y,x,1) = 255-abs(255-2*mat2(y,x));
x(y,x,2) = mat2(y,x)
or something very similar.
You may use SigPack, a signal processing library on top of Armadillo. It has spectrogram support and you may save the plot to a lot of different formats (png, ps, eps, tex, pdf, svg, emf, gif). SigPack uses Gnuplot for the plotting.

C++ - Image Conversion

I am new to C++ and would like to know how to read in a .jpg image and then convert it to binary (black and white/bi-level/two-level)?
Thank you.
Your better choice is probably boost Gil.
Boost libraries are not especially for beginner, but they are often well designed.
#include <boost/gil/image.hpp>
#include <boost/gil/typedefs.hpp>
#include <boost/gil/extension/io/jpeg_io.hpp>
int main() {
using namespace boost::gil;
rgb8_image_t img;
jpeg_read_image("test.jpg",img);
gray8s_view_t view(img.dimensions());
color_converted_view<gray8_pixel_t>(const_view(img), view);
jpeg_write_view("grey.jpg", view);
}
You can use DevIL to read the image. It supports a lot of different formats.
To convert it to pure black and white, you then go through the whole image data and compute the intensity or light contribution of each pixel and if it falls below a certain threshold you'll output a black pixel otherwise a white pixel.
You could do it as simply as check the RGB-values of each pixel against a threshold of RGB(0.5, 0.5, 0.5). But you might get better results if you convert the image to HSI and use the intensity value for each pixel, but that's more work.
There is the option for libpng, which as been used on many projects. For additional reading on how to write a grayscale image, take a look at this chapter from their website.