I'm trying to use cv::FindFundamentalMat but when I try to get the 4th argument (that should be :
Output array of N elements, every element of which is set to 0 for outliers and to 1 for the other points. The array is computed only in the RANSAC and LMedS methods. For other methods, it is set to all 1’s.
)
It only gives me 0's.
I'm using siftGPU to generate the keypoints (x,y) that are used in the function.
My code :
/*
... Use siftgpu
*/
std::vector<int(*)[2]> match_bufs; //Contain (x,y) from the 2 images that are paired
SiftGPU::SiftKeypoint & key1 = keys[match_bufs[i][0]];
SiftGPU::SiftKeypoint & key2 = keys[match_bufs[i][1]];
float x_l, y_l, x_r, y_r; //(x,y of left and right images)
x_l = key1.x; y_l = key1.y;
x_r = key2.x; y_r = key2.y;
vec1.push_back(x_l); vec1.push_back(y_l);
vec2.push_back(x_r); vec2.push_back(y_r);
std::vector<uchar> results;
int size = vec1.size();
results.resize(size);
std::vector<cv::Point2f> points1;
std::vector<cv::Point2f> points2;
for (int i = 0; i < size; i+=2) {
points1.push_back(cv::Point2f(vec1[i], vec1[i + 1]));
points2.push_back(cv::Point2f(vec2[i], vec2[i + 1]));
}
cv::Mat fund = cv::findFundamentalMat(points1, points2, CV_FM_RANSAC, 3, 0.99, results);
then,
std::cout << std::endl << fund << std::endl;
for (int j = 0; j < results.size(); ++j) {
std::cout << (int)results[j];
}
fund is :
0, -0.001, 0.6
0, 0, -0.3
-0.4, 0.2, 0
and results is composed with only 0's.
I'm maybe fooling myself because findFundamentalMat says :
Array of N points from the first image. The point coordinates should be floating-point (single or double precision).
Since i'm not native speaker english, there is maybe something that I'm missing... My (x,y) are like (350.0, 560.0) (that are floating points). But do I have to normalize them between [0,1] and that's what floating-point means?
Or do I am missing something else?
Thanks!
(EDIT : I tried to normalize my points (divide by height and width of respective images, but results are still 0's)
The answer is quite easy : I have to use the good format for the template and cast it well.
So :
((int)results.at<uchar>(i, 0) == 1)
works :)
If it can help someone.
Related
I want to see the distribution of a color with respect to image width. That is, if a (black and white) image has width of 720 px, then I want to conclude that a specific range (e.g. pixels [500,720]) has more white color in compared to rest of the image. What I thought is, I need a slice of the image of 720x1 px, then I need to check the values and distribute them w.r.t. width of 720 px. But I don't know the way I can apply this in a suitable way?
edit: I use OpenCV 4.0.0 with C++.
Example Case: In the first image, it is obvious that right hand side pixels are white. I want to get estimate coordinates of this dense line or zone. The light pink zone is where I am interested in and the red borders are the range where I want to find it.
If you want to get minimum continious range of image columns which contain more white than the rest of the image, than you need first to calculate number of white pixels in each column. Lets assume we have an image 720x500 (500 pixels high and 720 pixels wide). Than you will get an array Arr of 720 elements that equal number of white pixels in each column (1x500) respectively.
const int Width = img.cols;
int* Arr = new int[Width];
for( int x = 0; x < Width; x++ ) {
Arr[x] = 0;
for( int y = 0; y < img.rows; y++ ) {
if ( img.at<cv::Vec3b>(y,x) == cv::Vec3b(255,255,255) ) {
Arr[x]++;
}
}
}
You need to find a minimum range [A;B] in this array that satisfies condition Sum(Arr[0 to A-1]) + Sum(Arr[B+1 to Width-1]) < Sum(Arr[A to B]).
// minimum range width is guaranteed to be less or equal to (Width/2 + 1)
int bestA = 0, minimumWidth = Width/2 + 1;
int total = RangeSum(Arr, 0, Width-1);
for (int i = 0; i < Width; i++) {
for (int j = i; j < Width && j < i + minimumWidth; j++) {
int rangeSum = RangeSum(Arr, i, j);
if (rangeSum > total - rangeSum) {
bestA = i;
minimumWidth = j - i + 1;
break;
}
}
}
std::cout << "Most white minimum range - [" << bestA << ";" << bestA + minimumWidth - 1 << "]\n";
You can optimize the code if you precalculate sums for all [0; i] ranges, i from 0 to Width - 1. Than you can calculate RangeSum(Arr, A, B) as PrecalculatedSums[B] - PrecalculatedSums[A] (in O(1) complexity).
I am trying to manually implement a fundamental matrix estimation function for corresponding points (based on similarities between two images). The corresponding points are obtained after performing ORB feature detection, extraction, matching and ratio test.
There is a lot of literature available on good sources about this topic. However none of them appear to give a good pseudo-code for doing the process. I went through various Chapters on Multiple View Geometry book; and also many online sources.
This source appears to give a formula for doing the normalization and I followed the formula mentioned on page 13 of this source.
Based on this formula, I created the following algorithm. I am not sure if I am doing it the right way though !
Normalization.hpp
class Normalization {
typedef std::vector <cv::Point2f> intercepts;
typedef std::vector<cv::Mat> matVec;
public:
Normalization () {}
~Normalization () {}
void makeAverage(intercepts pointsVec);
std::tuple <cv::Mat, cv::Mat> normalize(intercepts pointsVec);
matVec getNormalizedPoints(intercepts pointsVec);
private:
double xAvg = 0;
double yAvg = 0;
double count = 0;
matVec normalizedPts;
double distance = 0;
matVec matVecData;
cv::Mat forwardTransform;
cv::Mat reverseTransform;
};
Normalization.cpp
#include "Normalization.hpp"
typedef std::vector <cv::Point2f> intercepts;
typedef std::vector<cv::Mat> matVec;
/*******
*#brief : The makeAverage function receives the input 2D coordinates (x, y)
* and creates the average of x and y
*#params : The input parameter is a set of all matches (x, y pairs) in image A
************/
void Normalization::makeAverage(intercepts pointsVec) {
count = pointsVec.size();
for (auto& member : pointsVec) {
xAvg = xAvg + member.x;
yAvg = yAvg + member.y;
}
xAvg = xAvg / count;
yAvg = yAvg / count;
}
/*******
*#brief : The normalize function accesses the average distance calculated
* in the previous step and calculates the forward and inverse transformation
* matrices
*#params : The input to this function is a vector of corresponding points in given image
*#return : The returned data is a tuple of forward and inverse transformation matrices
*************/
std::tuple <cv::Mat, cv::Mat> Normalization::normalize(intercepts pointsVec) {
for (auto& member : pointsVec) {
// Accumulate the distance for every point
distance += ((1 / (count * std::sqrt(2))) *\
(std::sqrt(std::pow((member.x - xAvg), 2)\
+ std::pow((member.y - yAvg), 2))));
}
forwardTransform = (cv::Mat_<double>(3, 3) << (1 / distance), \
0, -(xAvg / distance), 0, (1 / distance), \
-(yAvg / distance), 0, 0, 1);
reverseTransform = (cv::Mat_<double>(3, 3) << distance, 0, xAvg, \
0, distance, yAvg, 0, 0, 1);
return std::make_tuple(forwardTransform, reverseTransform);
}
/*******
*#brief : The getNormalizedPoints function trannsforms the raw image coordinates into
* transformed coordinates using the forwardTransform matrix estimated in previous step
*#params : The input to this function is a vector of corresponding points in given image
*#return : The returned data is vector of transformed coordinates
*************/
matVec Normalization::getNormalizedPoints(intercepts pointsVec) {
cv::Mat triplet;
for (auto& member : pointsVec) {
triplet = (cv::Mat_<double>(3, 1) << member.x, member.y, 1);
matVecData.emplace_back(forwardTransform * triplet);
}
return matVecData;
}
Is this the right way ? Are there other ways of Normalization ?
I think you can do it your way but in "Multiple View Geometry in Computer Vision" Hartley and Zisserman recommend isotropic scaling (p. 107):
Isotropic scaling. As a first step of normalization, the coordinates in each image are
translated (by a different translation for each image) so as to bring the centroid of the
set of all points to the origin. The coordinates are also scaled so that on the average a
point x is of the form x = (x, y,w)T, with each of x, y and w having the same average
magnitude. Rather than choose different scale factors for each coordinate direction, an
isotropic scaling factor is chosen so that the x and y-coordinates of a point are scaled
equally. To this end, we choose to scale the coordinates so that the average distance of
a point x from the origin is equal to
√
2. This means that the “average” point is equal
to (1, 1, 1)T. In summary the transformation is as follows:
(i) The points are translated so that their centroid is at the origin.
(ii) The points are then scaled so that the average distance from the origin is equal
to √2.
(iii) This transformation is applied to each of the two images independently.
They state that it is important for the direct linear transformation (DLT) but even more important for the calculation of a Fundamental Matrix like you want to do.
The algorithm you chose, normalized the point coordinates to (1, 1, 1) but did not apply a scaling so that the average distance from the origin is equal to √2.
Here is some code for this type of normalization. The averaging step stayed the same:
std::vector<cv::Mat> normalize(std::vector<cv::Point2d> pointsVec) {
// Averaging
double count = (double) pointsVec.size();
double xAvg = 0;
double yAvg = 0;
for (auto& member : pointsVec) {
xAvg = xAvg + member.x;
yAvg = yAvg + member.y;
}
xAvg = xAvg / count;
yAvg = yAvg / count;
// Normalization
std::vector<cv::Mat> points3d;
std::vector<double> distances;
for (auto& member : pointsVec) {
double distance = (std::sqrt(std::pow((member.x - xAvg), 2) + std::pow((member.y - yAvg), 2)));
distances.push_back(distance);
}
double xy_norm = std::accumulate(distances.begin(), distances.end(), 0.0) / distances.size();
// Create a matrix transforming the points into having mean (0,0) and mean distance to the center equal to sqrt(2)
cv::Mat_<double> Normalization_matrix(3, 3);
double diagonal_element = sqrt(2) / xy_norm;
double element_13 = -sqrt(2) * xAvg / xy_norm;
double element_23 = -sqrt(2)* yAvg/ xy_norm;
Normalization_matrix << diagonal_element, 0, element_13,
0, diagonal_element, element_23,
0, 0, 1;
// Multiply the original points with the normalization matrix
for (auto& member : pointsVec) {
cv::Mat triplet = (cv::Mat_<double>(3, 1) << member.x, member.y, 1);
points3d.emplace_back(Normalization_matrix * triplet);
}
return points3d;
}
I need help figuring out how OpenCV handles setting a matrix equal to something.
I have an 8-Bit Mat called Radiance that I want to tone map. Here is working code that accomplishes this for me, with K being the constant 450.
cv::cvtColor(radiance, radiance, CV_BGR2XYZ);
radiance = (K * radiance)/(1 + (K * radiance));
cv::cvtColor(radiance, radiance, CV_XYZ2BGR);`
This does not seem like it should work, but it does. It will create a fully tone mapped image that looks great. However, if you try to do this method on the individual pixels, they become a decimal that is between 0 and 1, which truncates to 0. Here is an example of this -
cv::cvtColor(radiance, radiance, CV_BGR2XYZ);
int x = radiance.at<cv::Vec3b>(500, 500)[0];
x = (K * x)/(1 + (K * x));
std::cout << x << "\n";
The output of this is exactly what I would expect
0
I understand why the second snippet of code prints out a zero, but what is going on in the first part that allows it to tone map the image properly, and how can I recreate this on the individual pixel level?
Can't you just define radiance as float matrix?
Mat radiance(m, n, DataType<float>::type);
So you can get a float
cv::cvtColor(radiance, radiance, CV_BGR2XYZ);
float x = radiance.at<cv::Vec3b>(500, 500)[0];
x = (K*x)/(1 + (K*x));
std::cout << x << "\n";
I'm using this article: nonlingr as a font to understand non linear transformations, in the section GLYPHS ALONG A PATH he explains how to use a parametric curve to transform an image, i'm trying to apply a cubic bezier to an image, however i have been unsuccessfull, this is my code:
OUT.aloc(IN.width(), IN.height());
//get the control points...
wVector p0(values[vindex], values[vindex+1], 1);
wVector p1(values[vindex+2], values[vindex+3], 1);
wVector p2(values[vindex+4], values[vindex+5], 1);
wVector p3(values[vindex+6], values[vindex+7], 1);
//this is to calculate t based on x
double trange = 1 / (OUT.width()-1);
//curve coefficients
double A = (-p0[0] + 3*p1[0] - 3*p2[0] + p3[0]);
double B = (3*p0[0] - 6*p1[0] + 3*p2[0]);
double C = (-3*p0[0] + 3*p1[0]);
double D = p0[0];
double E = (-p0[1] + 3*p1[1] - 3*p2[1] + p3[1]);
double F = (3*p0[1] - 6*p1[1] + 3*p2[1]);
double G = (-3*p0[1] + 3*p1[1]);
double H = p0[1];
//apply the transformation
for(long i = 0; i < OUT.height(); i++){
for(long j = 0; j < OUT.width(); j++){
//t = x / width
double t = trange * j;
//apply the article given formulas
double x_path_d = 3*t*t*A + 2*t*B + C;
double y_path_d = 3*t*t*E + 2*t*F + G;
double angle = 3.14159265/2.0 + std::atan(y_path_d / x_path_d);
mapped_point.Set((t*t*t)*A + (t*t)*B + t*C + D + i*std::cos(angle),
(t*t*t)*E + (t*t)*F + t*G + H + i*std::sin(angle),
1);
//test if the point is inside the image
if(mapped_point[0] < 0 ||
mapped_point[0] >= OUT.width() ||
mapped_point[1] < 0 ||
mapped_point[1] >= IN.height())
continue;
OUT.setPixel(
long(mapped_point[0]),
long(mapped_point[1]),
IN.getPixel(j, i));
}
}
Applying this code in a 300x196 rgb image all i get is a black screen no matter what control points i use, is hard to find information about this kind of transformation, searching for parametric curves all i find is how to draw them, not apply to images. Can someone help me on how to transform an image with a bezier curve?
IMHO applying a curve to an image sound like using a LUT. So you will need to check for the value of the curve for different image values and then switch the image value with the one on the curve, so, create a Look-Up-Table for each possible value in the image (e.g : 0, 1, ..., 255, for a gray value 8 bit image), that is a 2x256 matrix, first column has the values from 0 to 255 and the second one having the value of the curve.
I want to use an array as an input to k means algorithm .That array has the values of displacement in x and y direction and is a result of Lucas Kanade optical flow estimation. The code is following :
EDITED :
int number_of_features=150;
// Lucas Kanade optical flow
cvCalcOpticalFlowPyrLK(frame1_1C,frame2_1C,pyramid1,pyramid2,frame1_features,frame2_features,number_of_features,optical_flow_window, 5,optical_flow_found_feature, optical_flow_feature_error,optical_flow_termination_criteria, 0 );
float Dx[150],Dy[150]; // displacement matrices
float Dis[150][2]; // total displacement matrix
int K=2; // clusters selected
Mat bestLabels, centers;
for(int i = 0; i < number_of_features; i++)
{
CvPoint p,q;
p.x = (int) frame1_features[i].x;
p.y = (int) frame1_features[i].y;
q.x = (int) frame2_features[i].x;
q.y = (int) frame2_features[i].y;
//displacements
Dx[i]=p.x-q.x;
Dy[i]=p.y-q.y;
Dis[i][0] = Dx[i];
Dis[i][1] = Dy[i];
}
// k means algorithm
// Creating Mat for Input data
cv::Mat flt_Dis(150, 2, CV_32F, Dis);
cv::kmeans(flt_Dis, K, bestLabels,TermCriteria(CV_TERMCRIT_EPS+CV_TERMCRIT_ITER, 10, 1.0),3, KMEANS_PP_CENTERS, centers);
I have solved my previous problem , now i want to show the clustered image. I guess bestLabels store the indices for each element , e.g. if it is categorized to 0th or 1st category. Am I right ?? How can I show the clustered image ?
K-means can be implemented to work on integers. You just have to do it.
The result will not be integer though.