I am working on a c++ codebase right now which uses a matrix library to calculate various things. One of those things is calculating the inverse of a matrix. It uses gauss elimation to achieve that. But the result is very inaccurate. So much so that multiplying the inverse matrix with the original matrix isn't even close the the identity matrix.
Here is the code that is used to calculate the inverse, the matrix is templated on a numerical type and the rows and columns:
/// \brief Take the inverse of the matrix.
/// \return A new matrix which is the inverse of the current one.
matrix<T, M, M> inverse() const
{
static_assert(M == N, "Inverse matrix is only defined for square matrices.");
// augmented the current matrix with the identiy matrix.
auto augmented = this->augment(matrix<T, M, M>::get_identity());
for (std::size_t i = 0; i < M; i++)
{
// divide the current row by the diagonal element.
auto divisor = augmented[i][i];
for (std::size_t j = 0; j < 2 * M; j++)
{
augmented[i][j] /= divisor;
}
// For each element in the column of the diagonal element that is currently selected
// set all element in that column to 0 except the diagonal element by using the currently selected row diagonal element.
for (std::size_t j = 0; j < M; j++)
{
if (i == j)
{
continue;
}
auto multiplier = augmented[j][i];
for (std::size_t k = 0; k < 2 * M; k++)
{
augmented[j][k] -= multiplier * augmented[i][k];
}
}
}
// Slice of the the new identity matrix on the left side.
return augmented.template slice<0, M, M, M>();
}
Now I have made a unit test which test if the inverse is correct using pre computed values. I try two matrices one 3x3 and one 4x4. I used this website to compute the inverse: https://matrix.reshish.com/ and they do match to a certain degree. since the unit test does succeed. But once I calculate the original matrix * the inverse nothing even resembling an identity matrix is achieved. See the comment in the code below.
BOOST_AUTO_TEST_CASE(matrix_inverse)
{
auto m1 = matrix<double, 3, 3>({
{7, 8, 9},
{10, 11, 12},
{13, 14, 15}
});
auto inverse_result1 = matrix<double,3, 3>({
{264917625139441.28, -529835250278885.3, 264917625139443.47},
{-529835250278883.75, 1059670500557768, -529835250278884.1},
{264917625139442.4, -529835250278882.94, 264917625139440.94}
});
auto m2 = matrix<double, 4, 4>({
{7, 8, 9, 23},
{10, 11, 12, 81},
{13, 14, 15, 11},
{1, 73, 42, 65}
});
auto inverse_result2 = matrix<double, 4, 4>({
{-0.928094660194201, 0.21541262135922956, 0.4117111650485529, -0.009708737864078209},
{-0.9641231796116679, 0.20979975728155775, 0.3562651699029188, 0.019417475728154842},
{1.7099261731391882, -0.39396237864078376, -0.6169346682848 , -0.009708737864076772 },
{-0.007812499999999244, 0.01562499999999983, -0.007812500000000278, 0}
});
// std::cout << (m1.inverse() * m1) << std::endl;
// results in
// 0.500000000 1.000000000 -0.500000000
// 1.000000000 0.000000000 0.500000000
// 0.500000000 -1.000000000 1.000000000
// std::cout << (m2.inverse() * m2) << std::endl;
// results in
// 0.396541262 -0.646237864 -0.689016990 -2.162317961
// 1.206917476 2.292475728 1.378033981 3.324635922
// -0.884708738 -0.958737864 -0.032766990 -3.756067961
// -0.000000000 -0.000000000 -0.000000000 1.000000000
BOOST_REQUIRE_MESSAGE(
m1.inverse().fuzzy_equal(inverse_result1, 0.1) == true,
"3x3 inverse is not the expected result."
);
BOOST_REQUIRE_MESSAGE(
m2.inverse().fuzzy_equal(inverse_result2, 0.1) == true,
"4x4 inverse is not the expected result."
);
}
I am at my wits end. I am by no means a specialist on matrix math since I had to learn it all on the job but this really is stumping me.
The complete code matrix class is available at:
https://codeshare.io/johnsmith
Line 404 is where the inverse function is located.
Any help is appreciated.
As already established in the comments the matrix of interest is singular and thus there is no inverse.
Great, your testing found already the first issue in the code - this case isn't handled properly, no error is raised.
The bigger problem is, that this is not easy to detect: If there where no errors due to rounding errors, it would be a cake of piece - just test that divisor isn't 0! But there are rounding errors in floating operations, so divisor will be a very small nonzero number.
And there is no way to tell, whether this nonzero value due to rounding errors or to the fact that the matrix is near singular (but not singular). However, if matrix is near singular it has a poor condition and thus the results cannot be trusted anyway.
So ideally, the algorithm should not only calculate the inverse, but also (estimate) the condition of the original matrix, so the caller can react upon a bad condition.
Probably it is wise to use well-known and well-tested libraries for this kind of calculation - there is a lot to be considered and what can be done wrong.
Related
I have a recursive function to calculate the inverse of an upper triangular matrix. I have divided the matrix into Top, Bottom and Corner sections and then followed the methodology as laid down in https://math.stackexchange.com/a/2333418. Here is a pseudocode form:
//A diagram structure of the Matrix
Matrix = [Top Corner]
[0 Bottom]
Matrix multiply_matrix(Matrix A, Matrix B){
Simple Code to multiply two matrices and return a Matrix
}
Matrix simple_inverse(Matrix A){
Simple Code to get inverse of a 2x2 Matrix
}
Matrix inverse_matrix(Matrix A){
//Creating an empty A_inv matrix of dimension equal to A
Matrix A_inv;
if(A.dimension == 2){
A_inv = simple_inverse(A)
}
else{
Top_inv = inverse_matrix(Top);
(Code to check Top*Top_inv == Identity Matrix)
Bottom_inv = inverse_matrix(Bottom);
(Code to check Bottom*Bottom_inv == Identity Matrix)
Corner_inv = multiply_matrix(Top_inv, Corner);
Corner_inv = multiply_matrix(Corner_inv, Bottom_inv);
Corner_inv = negate(Corner_inv); //Just a function for negation of the matrix elements
//Code to copy Top_inv, Bottom_inv and Corner_inv to A_inv
...
...
}
return A_inv;
}
int main(){
matrix A = {An upper triangular matrix with random integers between 1 and 9};
A_inv = inverse_matrix(A);
test_matrix = multiply_matrix(A, A_inv);
(Code to Raise error if test_matrix != Identity matrix)
}
For simplicity I have implemented the code such that only power of 2 dimension matrices are supported.
My problem is that I have tested this code for matrix dimensions of 2, 4, 8, 16, 32 and 64. All of these pass all of the assertion checks as shown in code.
But for matrix dimension of 128 I get failure is the assertion in main(). And when I check, I observer that the test_matrix is not Identity matrix. Some non-diagonal elements are not equal to 0.
I am wondering what could be the reason for this:-
I am using C++ std::vector<std::vector<double>> for Matrix representation.
Since the data type is double the non-diagonal elements of test_matrix for cases 2, 4, 8, ..., 64 do have some value but very small. For example, -9.58122e-14
All my matrices at any recursion stage are square matrix
I am performing checks that Top*Top_inv = Identity and Bottom*Bottom_inv = Identity.
Finally, for dimensions 2, 4, ..., 64 I generated random numbers(b/w 1 and 10) to create my upper triangular matrix. Since, these cases passed, I guess my mathematical implementation is correct.
I feel like there is some aspect of C++ datatypes about double which I am unaware of that could be causing the error. Otherwise the sudden error from 64->128 doesn't make sense.
Could you please elaborate on how the matrix == identity operation is implemented?
My guess is that the problem might be resumed to the floating point comparison.
The matrix inversion can be O(n^3) in the worst case. This means that, as the matrix size increases, the amount of computations involved also increase. Real numbers cannot be perfectly represented even when using 64 bit floating point, they are always an approximation.
For operations such as matrix inversion this can cause problems of numerical error propagation, due to the loss of precision on the accumulated multiply adds operations.
For this, there has been discussions already in the StackOverflow: How should I do floating point comparison?
EDIT: Other thing to consider if the full matrix is actually invertible.
Perhaps the Top and/or Bottom matrices are invertible, but the full matrix (when composing with the Corner matrix) is not.
I have an Nx3 Eigen matrix.
I have an Nx1 Egein marix.
I'm trying to get the coefficient multiplication of each row in the Nx3 by the corresponding scal in the Nx1 so I can scale a bunch of 3d vectors.
I'm sure I'm overlooking something obvious but I can't get it to work.
#include <Eigen/Dense>
MatrixXf m(4, 3);
m << 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12;
MatrixXf dots(4, 1)
dots << 2,2,2,2;
I want to resulting matrix to be Nx3 like so:
2,4,6
8,10,12,
14,16,18,
20,22,24
You can use broadcasting:
m = m.colwise().cwiseProduct(dots);
or observe that all you want to do is to apply a non uniform scaling:
m = dots.asDiagonal() * m;
Both expressions will generate similar code.
Okay, so I got something working. I'm probably doing something wrong but this worked for me so I thought I would share. I wrote my first line of c++ a week ago so I figure I deserve some grace. Anyone with a better solution is encouraged to post.
// scalar/coefficient multiplication (not matrix) on Nx3 x N. For multiplying dot products by vectors
void N3xNcoefIP(MatrixXf &A, MatrixXf &B) {
A.array() *= B.replicate(1, A.size()).array();
}
I'm having trouble solving what I think should be a fairly simple problem. The basic problem is I want to modify an Eigen PermutationMatrix but I don't know how.
I'm doing a QR decomposition of some matrix X using the C++ Eigen library. I'm doing this on rank-deficient matrices and I need a particular output. Specifically, I need
R^{-1} * t(R^{-1})
The problem is that using Eigen::ColPivHouseholderQR returns a permuted version of R. This is easy enough to fix when X is full rank, but I'd like the fastest solution for when it is rank-deficient. Let me demonstrate:
using namespace Eigen;
// Do QR
ColPivHouseholderQR<MatrixXd> PQR(X);
// Get permutation matrix
ColPivHouseholderQR<MatrixXd>::PermutationType Pmat(PQR.colsPermutation());
int r(PQR.rank());
int p(X.cols());
// Notice I'm only getting r elements, so R_inv is r by r
MatrixXd R_inv = PQR.matrixQR().topLeftCorner(r, r).triangularView<Upper>().solve(MatrixXd::Identity(r, r));
// This only works if r = p and X is full-rank
R_inv = Pmat * R_inv * Pmat;
XtX_inv = R_inv * R_inv.transpose();
So the basic problem is that I would like to modify Pmat so that it only permutes the r columns of R_inv that I've extracted from PQR.matrixQR(). My basic problem is that I have no idea how to modify work with an Eigen PermutationMatrix, as it doesn't seem to have any of the methods or properties of a normal matrix.
One possible solution is the following: when I multiply Pmat * MatrixXd::Identity(p, p), I get a useful matrix.
For example, I get something like:
[0, 1, 0, 0,
1, 0, 0, 0,
0, 0, 0, 1,
0, 0, 1, 0]
If p = 4 and r = 3, then I would just like this sub-view, where I drop all columns right of the first r columns, and then remove all rows that are all 0:
[0, 1, 0,
1, 0, 0,
0, 0, 1]
So I could do the following:
P = Pmat * MatrixXd::Identity(p, p)
P.leftCols(p);
MatrixXd P = Pmat * Eigen::MatrixXd::Identity(p, p);
// https://stackoverflow.com/questions/41305178/removing-zero-columns-or-rows-using-eigen
// find non-zero columns:
Matrix<bool, 1, Dynamic> non_zeros = P.leftCols(r).cast<bool>().rowwise().any();
// allocate result matrix:
MatrixXd res(non_zeros.count(), r);
// fill result matrix:
Index j=0;
for(Index i=0; i<P.rows(); ++i)
{
if(non_zeros(i))
res.row(j++) = P.row(i).leftCols(r);
}
R_inv = res * R_inv * res;
XtX_inv = R_inv * R_inv.transpose();
but this seems expensive and doesn't take advantage of the fact that Pmat already knows which rows of Pmat should be dropped. I'm guessing there is an easier way to work with Pmat.
Is there any way to easily modify an Eigen PermutationMatrix to only consider columns that weren't placed beyond the first r positions?
Any help or tips would be greatly appreciated.
I've come up with another solution, which probably requires less computation.
// Get all column indices
ArrayXi Pmat_indices = Pmat.indices();
// Get the order for the columns you are keeping
ArrayXi Pmat_keep = Pmat_indices.head(r);
// Get the indices for columns you are discarding
ArrayXi Pmat_toss = Pmat_indices.tail(p - r);
// this code takes the indices you are keeping, and, while preserving order, keeps them in the range [0, r-1]
// For each one, see how many dropped indices are smaller, and subtract that difference
// Ex: p = 4, r = 2
// Pmat_indices = {3, 1, 0, 2}
// Pmat_keep = {3, 1}
// Pmat_toss = {0, 2}
// Now we go through each keeper, count how many in toss are smaller, and then modify accordingly
// 3 - 2 and 1 - 1
// Pmat_keep = {1, 0}
for(Index i=0; i<r; ++i)
{
Pmat_keep(i) = Pmat_keep(i) - (Pmat_toss < Pmat_keep(i)).count();
}
// Now this will order just the first few columns in the right order
PermutationMatrix<Dynamic, Dynamic> P = PermutationWrapper<ArrayXi>(Pmat_keep);
R_inv = P * R_inv * P;
I am using OpenCV for some time and now I hit the point where I need a multiplication of this type:
Define a matrix T, which contains elements of the type Vec3d1 . Matrix T has the size: M X N. Matrix T has to be multiplied with a Vector Phi, which has the size: N X 1, containing doubles as values. Each element of the result has to be the result of a matrix multiplication of both matrices.
I don't want to do a component-wise multiplication, but a "real" matrix multiplication, e.g. multiplying the first element of T2 with the first element of matrix J, then multiplying the second element of matrix T3 with the second element of matrix J. Do this until you completed the first row of T and then sum up the results. The result is a M X 1.
For example, if T would be a 3 X 2 matrix and Phi a 2 X 1 matrix, then the calculation should be T_11 * phi_11 + T_12 * phi_21 for the first value of the result. Currently I'm using two for loops which are slow:
for (int i = 0; i<M; ++i){
cv::Mat summedResult = cv::Mat(3, 1, CV_64F, double(0));
for (uint32 j = 0; j<N; ++j){
summedResult = summedResult +
(cv::Mat(mMatrixT.at<cv::Vec3d>(i, j)) * mMatrixPhi.at<double>(j));
}
// The result matrix contains values of type Vec3d again
mResultMatrix.at<cv::Vec3d>(i) = cv::Vec3d(summedResult);
}
More generally: Is it possible to efficiently multiply matrices containing Vec3ds and doubles in OpenCV?
1. three dimensional vector containing doubles.
2. coordinate: 1,1
3. coordinate: 1,2
I still don't know what kind of result you expect, but maybe try this:
Assuming that you have a MxN matrix of Vec3d and a Nx1 Matrix of type double, your result will be a Mx1 matrix of type Vec3d:
for (int i = 0; i<M; ++i)
{
cv::Vec3d summedResult; // here this must be a Vec3d instead of a matrix, if I assume the right result expection
for (uint32 j = 0; j<N; ++j)
{
summedResult = summedResult + (mMatrixT.at<cv::Vec3d>(i, j) * mMatrixPhi.at<double>(j));
}
// The result matrix contains values of type Vec3d again
mResultMatrix.at<cv::Vec3d>(i) = summedResult;
}
EDIT:
ah sorry, didnt read to the end that your provided code works but is too slow... well, I expect that there is no optimization for that because mathematically this isnt defined. What you can do is to convert your Vec3d mat to a Mx(3*N) matrix and convert your Nx1 mat to a (3*N)x1 mat (3 times the same value before the next value) and use the OpenCV matrix product directly. But probably that's not faster because of the 3* bigger size of both matrices ;)
EDIT: will be a different result, since each element will be the sum of the Vec3d elements...
I have a Polynomial class that has a get_vect member function which stores integers in a vector that is to be the representation of the coefficients of the polynomial. Now, I am trying to multiply two polynomials together using a Multiply non-member function, but I get stuck when it comes to the actual multiplication of the vectors. So far, what I have is what is shown below:
Polynomial Multiply(const Polynomial & poly1, const Polynomial & poly2)
{
vector<int> Poly1 = poly1.get_vect();
vector<int> Poly2 = poly2.get_vect();
vector<int> Poly3;
if( Poly1.size() < Poly2.size() )
{
for(size_t i = 0 ; Poly2.size()-Poly1.size() ; ++i )
{
Poly2.push_back(0);
}
}
else if( Poly1.size() > Poly2.size() )
{
for(size_t i = 0 ; Poly1.size()-Poly2.size() ; ++i )
{
Poly1.push_back(0);
}
}
return Poly3;
}
I see that it some how has to follow the below pattern:
Ok, so if I understand the problem correctly, you want Poly3 to be a vector<int> that holds the coefficients that result from a polynomial multiplication between the polynomials represented by Poly1 and Poly2.
Tacit in this request is that all three polynomials are polynomials in a single variable, with each coefficient representing the coefficient in front of an increasing power of that variable. ie. that { 4, 5, 6, 7 } corresponds to 4 + 5x + 6x2 + 7x3.
If so, then the actual multiplication shouldn't be that difficult at all, as long as your polynomials aren't terribly huge. You need code that looks approximately like this:
Poly3.resize(Poly1.size() + Poly2.size() - 1, 0); // Make the output big enough; init to 0
for (size_t i = 0; i != Poly1.size(); i++)
for (size_t j = 0; j != Poly2.size(); j++)
Poly3[i+j] += Poly1[i] * Poly2[j];
Now the result in Poly3 should be the product of Poly1 and Poly2.
It's entirely possible I forgot an edge condition; I'll watch for comments here to point out where I did. In the meantime, though, I did a few tests and it appears this gives the correct output.
If you have rather large polynomials, then you might want to look into math libraries to handle the multiplication. But for anything under about 20 - 30 terms? Unless your code leans very hard on this polynomial evaluation, I suspect this won't be your bottleneck.