How to Compute the Structure Tensor of an Image using OpenCV - c++

Question

I am trying to implement an application that uses images to search for similar images in a large image database. I am developing an image descriptor to use for this search and I would like to combine color information with some gradient information. I have seen structure tensors used in this domain to find the main gradient direction in images or sub-images.
I would like to take an image, divide it into grid of sub-images, for example, 4x4 grid (in total 16 sub-images) and then find the leading gradient direction of each cell. To find the leading gradient direction I want to see if computing the structure tensor for each cell can give good representation of the image gradient and lead to improved image matching. Is this a good idea or a bad idea? The idea was to get a feature vector similar to the idea in section 3.2 in this paper http://cybertron.cg.tu-berlin.de/eitz/pdf/2009_sbim.pdf
Dividing the image into sub-images(cells) is trivial and with opencv I can compute the partial derivatives using the Sobel function.
Mat dx, dy;
Sobel(im, dx, CV_32F, 1, 0, 3, 1, 0, BORDER_DEFAULT);
Sobel(im, dy, CV_32F, 0, 1, 3, 1, 0, BORDER_DEFAULT);
Computing dx^2, dy^2 and dxy should not be a problem, but I am not sure how I can compute the structure tensor matrix and use the tensor matrix to find the main gradient direction for an image or sub-image. How can I implement this with OpenCV?
EDIT
Okay, this is what I have done.
Mat _im; // Image to compute main gradient direction for.
cvtColor(im, _im, CV_BGR2GRAY);
GaussianBlur(_im, _im, Size(3, 3), 0, 0, BORDER_DEFAULT); //Blur the image to remove unnecessary details.
GaussianBlur(_im, _im, Size(5, 5), 0, 0, BORDER_DEFAULT);
GaussianBlur(_im, _im, Size(7, 7), 0, 0, BORDER_DEFAULT);
// Calculate image derivatives
Mat dx2, dy2, dxy;
Sobel(_im, dx2, CV_32F, 2, 0, 3, 1, 0, BORDER_DEFAULT);
Sobel(_im, dy2, CV_32F, 0, 2, 3, 1, 0, BORDER_DEFAULT);
Sobel(_im, dxy, CV_32F, 1, 1, 3, 1, 0, BORDER_DEFAULT);
Mat t(2, 2, CV_32F); // tensor matrix
// Insert values to the tensor matrix.
t.at<float>(0, 0) = sum(dx2)[0];
t.at<float>(0, 1) = sum(dxy)[0];
t.at<float>(1, 0) = sum(dxy)[0];
t.at<float>(1, 1) = sum(dy2)[0];
// eigen decomposition to get the main gradient direction.
Mat eigVal, eigVec;
eigen(t, eigVal, eigVec);
// This should compute the angle of the gradient direction based on the first eigenvector.
float* eVec1 = eigVec.ptr<float>(0);
float* eVec2 = eigVec.ptr<float>(1);
cout << fastAtan2(eVec1[0], eVec1[1]) << endl;
cout << fastAtan2(eVec2[0], eVec2[1]) << endl;
Is this approach correct?
Using this image the application outputs 44.9905, 135.01.
This gives 0, 90.
When I use a part of a real image I get 342.743, 72.7425, which I find odd. I expected to get an angle along the color change (90ish).
After testing I am not sure if my implementation is correct, so any feedback or comments on this are welcomed.

Answer 1

I believe your problem is that you are computing second order derivatives instead of squaring first order derivatives. It should be something like this instead:
// Calculate image derivatives
Mat dx, dy;
Mat dx2, dy2, dxy;
Sobel(_im, dx, CV_32F, 1, 0);
Sobel(_im, dy, CV_32F, 0, 1);
multiply(dx, dx, dx2);
multiply(dy, dy, dy2);
multiply(dx, dy, dxy);
P.S.
Oh, by the way, there is no need to do Gaussian blurring over and over again. Just use a bigger kernel and blur once.
D.S.