Orthogonalization in QR Factorization outputting slightly innaccurate orthogonalized matrix - c++

I am writing code for QR Factorization and for some reason my orthogonal method does not work as intended. Basically, my proj() method is outputting random projections. Here is the code:
apmatrix<double> proj(apmatrix<double> v, apmatrix<double> u)
//Projection of u onto v
{
//proj(v,u) = [(u dot v)/(v dot v)]*v
double a = mult(transpose(u,u),v)[0][0], b = mult(transpose(v,v),v)[0][0], c = (a/b);
apmatrix<double>k;
k.resize(v.numrows(),v.numcols());
for(int i = 0; i<v.numrows(); i++)
{
for(int j = 0; j<v.numcols(); j++)
{
k[i][j]=v[i][j]*c;
}
}
return k;
}
I tested the method by itself with manual matrix inputs, and it seems to work fine. Here is my orthogonal method:
apmatrix<double> orthogonal(apmatrix<double> A) //Orthogonal
{
/*
n = (number of columns of A)-1
x = columns of A
v0 = x0
v1 = x1 - proj(v0,x1)
vn = xn - proj(v0,xn) - proj(v1,xn) - ... - proj(v(n-1),xn)
V = {v1, v2, ..., vn} or [v0 v1 ... vn]
*/
apmatrix<double> V, x, v;
int n = A.numcols();
V.resize(A.numrows(),n);
x.resize(A.numrows(), 1);
v.resize(A.numrows(),1);
for(int i = 0; i<A.numrows(); i++)
{
x[i][0]=A[i][1];
v[i][0]=A[i][0];
V[i][0]=A[i][0];
}
for (int c = 1; c<n; c++) //Iterates through each col of A as if each was its own matrix
{
apmatrix<double>vn,vc; //vn = Orthogonalized v (avoiding matrix overwriting of v); vc = previously orthogonalized v
vn=x;
vc.resize(v.numrows(), 1);
for(int i=0; i<c; i++) //Vn = an-(sigma(t=1, n-1, proj(vt, xn))
{
for(int k = 0; k<V.numrows(); k++)
vc[k][0] = V[k][i]; //Sets vc to designated v matrix
apmatrix<double>temp = proj(vc, x);
for(int j = 0; j<A.numrows(); j++)
{
vn[j][0]-=temp[j][0]; //orthogonalize matrix
}
}
for(int k = 0; k<V.numrows(); k++)
{
V[k][c]=vn[k][0]; //Subtracts orthogonalized col to V
v[k][0]=V[k][c]; //v is redundant. more of a placeholder
}
if((c+1)<A.numcols()) //Matrix Out of Bounds Checker
{
for(int k = 0; k<A.numrows(); k++)
{
vn[k][0]=0;
vc[k][0]=0;
x[k][0]=A[k][c+1]; //Moves x onto next v
}
}
}
system("PAUSE");
return V;
}
For testing purposes, I have been using the 2D Array: [[1,1,4],[1,4,2],[1,4,2],[1,1,0]]. Each column is its own 4x1 matrix. The matrices should be outputted as: [1,1,1,1]T, [-1.5,1.5,1.5,-1.5]T, and [2,0,0,-2]T respectively. What's happening now is that the first column comes out correctly (it's the same matrix), but the second and third come out to something that is potentially similar but not equal to their intended values.
Again, each time I call on the orthogonal method, it outputs something different. I think it's due to the numbers inputted in the proj() method, but I am not fully sure.
The apmatrix is from the AP college board, back when they taught cpp. It is similar to vectors or ArrayLists in Java.
Here is a link to apmatrix.cpp and to the documentation or conditions (probably more useful), apmatrix.h.
Here is a link to the full code (I added visual markers to see what the computer is doing).
It's fair to assume that all custom methods work as intended (except maybe Matrix Regressions, but that's irrelevant). And be sure to enter the matrix using the enter method before trying to factorize. The code might be inefficient partly because I self-taught myself cpp not too long ago and I've been trying different ways to fix my code. Thank you for the help!

As said in comments:
#AhmedFasih After doing more tests today, I have found that it is in-fact some >memory issue. I found that for some reason, if a variable or an apmatrix object >is declared within a loop, initialized, then that loop is reiterated, the >memory does not entirely wipe the value stored in that variable or object. This >is noted in two places in my code. For whatever reason, I had to set the >doubles a,b, and c to 0 in the proj method and apmatrixdh to 0 in the >mult method or they would store some value in the next iteration. Thank you so >much for you help!

Related

Having a hard time figuring out logic behind array manipulation

I am given a filled array of size WxH and need to create a new array by scaling both the width and the height by a power of 2. For example, 2x3 becomes 8x12 when scaled by 4, 2^2. My goal is to make sure all the old values in the array are placed in the new array such that 1 value in the old array fills up multiple new corresponding parts in the scaled array. For example:
old_array = [[1,2],
[3,4]]
becomes
new_array = [[1,1,2,2],
[1,1,2,2],
[3,3,4,4],
[3,3,4,4]]
when scaled by a factor of 2. Could someone explain to me the logic on how I would go about programming this?
It's actually very simple. I use a vector of vectors for simplicity noting that 2D matrixes are not efficient. However, any 2D matrix class using [] indexing syntax can, and should be for efficiency, substituted.
#include <vector>
using std::vector;
int main()
{
vector<vector<int>> vin{ {1,2},{3,4},{5,6} };
size_t scaleW = 2;
size_t scaleH = 3;
vector<vector<int>> vout(scaleH * vin.size(), vector<int>(scaleW * vin[0].size()));
for (size_t i = 0; i < vout.size(); i++)
for (size_t ii = 0; ii < vout[0].size(); ii++)
vout[i][ii] = vin[i / scaleH][ii / scaleW];
auto x = vout[8][3]; // last element s/b 6
}
Here is my take. It is very similar to #Tudor's but I figure between our two, you can pick what you like or understand best.
First, let's define a suitable 2D array type because C++'s standard library is very lacking in this regard. I've limited myself to a rather simple struct, in case you don't feel comfortable with object oriented programming.
#include <vector>
// using std::vector
struct Array2d
{
unsigned rows, cols;
std::vector<int> data;
};
This print function should give you an idea how the indexing works:
#include <cstdio>
// using std::putchar, std::printf, std::fputs
void print(const Array2d& arr)
{
std::putchar('[');
for(std::size_t row = 0; row < arr.rows; ++row) {
std::putchar('[');
for(std::size_t col = 0; col < arr.cols; ++col)
std::printf("%d, ", arr.data[row * arr.cols + col]);
std::fputs("]\n ", stdout);
}
std::fputs("]\n", stdout);
}
Now to the heart, the array scaling. The amount of nesting is … bothersome.
Array2d scale(const Array2d& in, unsigned rowfactor, unsigned colfactor)
{
Array2d out;
out.rows = in.rows * rowfactor;
out.cols = in.cols * colfactor;
out.data.resize(std::size_t(out.rows) * out.cols);
for(std::size_t inrow = 0; inrow < in.rows; ++inrow) {
for(unsigned rowoff = 0; rowoff < rowfactor; ++rowoff) {
std::size_t outrow = inrow * rowfactor + rowoff;
for(std::size_t incol = 0; incol < in.cols; ++incol) {
std::size_t in_idx = inrow * in.cols + incol;
int inval = in.data[in_idx];
for(unsigned coloff = 0; coloff < colfactor; ++coloff) {
std::size_t outcol = incol * colfactor + coloff;
std::size_t out_idx = outrow * out.cols + outcol;
out.data[out_idx] = inval;
}
}
}
}
return out;
}
Let's pull it all together for a little demonstration:
int main()
{
Array2d in;
in.rows = 2;
in.cols = 3;
in.data.resize(in.rows * in.cols);
for(std::size_t i = 0; i < in.rows * in.cols; ++i)
in.data[i] = static_cast<int>(i);
print(in);
print(scale(in, 3, 2));
}
This prints
[[0, 1, 2, ]
[3, 4, 5, ]
]
[[0, 0, 1, 1, 2, 2, ]
[0, 0, 1, 1, 2, 2, ]
[0, 0, 1, 1, 2, 2, ]
[3, 3, 4, 4, 5, 5, ]
[3, 3, 4, 4, 5, 5, ]
[3, 3, 4, 4, 5, 5, ]
]
To be honest, i'm incredibly bad at algorithms but i gave it a shot.
I am not sure if this can be done using only one matrix, or if it can be done in less time complexity.
Edit: You can estimate the number of operations this will make with W*H*S*S where Sis the scale factor, W is width and H is height of input matrix.
I used 2 matrixes m and r, where m is your input and r is your result/output. All that needs to be done is to copy each element from m at positions [i][j] and turn it into a square of elements with the same value of size scale_factor inside r.
Simply put:
int main()
{
Matrix<int> m(2, 2);
// initial values in your example
m[0][0] = 1;
m[0][1] = 2;
m[1][0] = 3;
m[1][1] = 4;
m.Print();
// pick some scale factor and create the new matrix
unsigned long scale = 2;
Matrix<int> r(m.rows*scale, m.columns*scale);
// i know this is bad but it is the most
// straightforward way of doing this
// it is also the only way i can think of :(
for(unsigned long i1 = 0; i1 < m.rows; i1++)
for(unsigned long j1 = 0; j1 < m.columns; j1++)
for(unsigned long i2 = i1*scale; i2 < (i1+1)*scale; i2++)
for(unsigned long j2 = j1*scale; j2 < (j1+1)*scale; j2++)
r[i2][j2] = m[i1][j1];
// the output in your example
std::cout << "\n\n";
r.Print();
return 0;
}
I do not think it is relevant for the question, but i used a class Matrix to store all the elements of the extended matrix. I know it is a distraction but this is still C++ and we have to manage memory. And what you are trying to achieve with this algorithm needs a lot of memory if the scale_factor is big so i wrapped it up using this:
template <typename type_t>
class Matrix
{
private:
type_t** Data;
public:
// should be private and have Getters but
// that would make the code larger...
unsigned long rows;
unsigned long columns;
// 2d Arrays get big pretty fast with what you are
// trying to do.
Matrix(unsigned long rows, unsigned long columns)
{
this->rows = rows;
this->columns = columns;
Data = new type_t*[rows];
for(unsigned long i = 0; i < rows; i++)
Data[i] = new type_t[columns];
}
// It is true, a copy constructor is needed
// as HolyBlackCat pointed out
Matrix(const Matrix& m)
{
rows = m.rows;
columns = m.columns;
Data = new type_t*[rows];
for(unsigned long i = 0; i < rows; i++)
{
Data[i] = new type_t[columns];
for(unsigned long j = 0; j < columns; j++)
Data[i][j] = m[i][j];
}
}
~Matrix()
{
for(unsigned long i = 0; i < rows; i++)
delete [] Data[i];
delete [] Data;
}
void Print()
{
for(unsigned long i = 0; i < rows; i++)
{
for(unsigned long j = 0; j < columns; j++)
std::cout << Data[i][j] << " ";
std::cout << "\n";
}
}
type_t* operator [] (unsigned long row)
{
return Data[row];
}
};
First of all, having a suitable 2D matrix class is presumed but not the question. But I don't know the API of yours, so I'll illustrate with something typical:
struct coord {
size_t x; // x position or column count
size_t y; // y position or row count
};
template <typename T>
class Matrix2D {
⋮ // implementation details
public:
⋮ // all needed special members (ctors dtor, assignment)
Matrix2D (coord dimensions);
coord dimensions() const; // return height and width
const T& cell (coord position) const; // read-only access
T& cell (coord position); // read-write access
// handy synonym:
const T& operator[](coord position) const { return cell(position); }
T& operator[](coord position) { return cell(position); }
};
I just showed the public members I need: create a matrix with a given size, query the size, and indexed access to the individual elements.
So, given that, your problem description is:
template<typename T>
Matrix2D<T> scale_pow2 (const Matrix2D& input, size_t pow)
{
const auto scale_factor= 1 << pow;
const auto size_in = input.dimensions();
Matrix2D<T> result ({size_in.x*scale_factor,size_in.y*scale_factor});
⋮
⋮ // fill up result
⋮
return result;
}
OK, so now the problem is precisely defined: what code goes in the big blank immediately above?
Each cell in the input gets put into a bunch of cells in the output. So you can either iterate over the input and write a clump of cells in the output all having the same value, or you can iterate over the output and each cell you need the value for is looked up in the input.
The latter is simpler since you don't need a nested loop (or pair of loops) to write a clump.
for (coord outpos : /* ?? every cell of the output ?? */) {
coord frompos {
outpos.x >> pow,
outpos.y >> pow };
result[outpos] = input[frompos];
}
Now that's simple!
Calculating the from position for a given output must match the way the scale was defined: you will have pow bits giving the position relative to this clump, and the higher bits will be the index of where that clump came from
Now, we want to set outpos to every legal position in the output matrix indexes. That's what I need. How to actually do that is another sub-problem and can be pushed off with top-down decomposition.
a bit more advanced
Maybe nested loops is the easiest way to get that done, but I won't put those directly into this code, pushing my nesting level even deeper. And looping 0..max is not the simplest thing to write in bare C++ without libraries, so that would just be distracting. And, if you're working with matrices, this is something you'll have a general need for, including (say) printing out the answer!
So here's the double-loop, put into its own code:
struct all_positions {
coord current {0,0};
coord end;
all_positions (coord end) : end{end} {}
bool next() {
if (++current.x < end.x) return true; // not reached the end yet
current.x = 0; // reset to the start of the row
if (++current.y < end.y) return true;
return false; // I don't have a valid position now.
}
};
This does not follow the iterator/collection API that you could use in a range-based for loop. For information on how to do that, see my article on Code Project or use the Ranges stuff in the C++20 standard library.
Given this "old fashioned" iteration helper, I can write the loop as:
all_positions scanner {output.dimensions}; // starts at {0,0}
const auto& outpos= scanner.current;
do {
⋮
} while (scanner.next());
Because of the simple implementation, it starts at {0,0} and advancing it also tests at the same time, and it returns false when it can't advance any more. Thus, you have to declare it (gives the first cell), use it, then advance&test. That is, a test-at-the-end loop. A for loop in C++ checks the condition before each use, and advances at the end, using different functions. So, making it compatible with the for loop is more work, and surprisingly making it work with the ranged-for is not much more work. Separating out the test and advance the right way is the real work; the rest is just naming conventions.
As long as this is "custom", you can further modify it for your needs. For example, add a flag inside to tell you when the row changed, or that it's the first or last of a row, to make it handy for pretty-printing.
summary
You need a bunch of things working in addition to the little piece of code you actually want to write. Here, it's a usable Matrix class. Very often, it's prompting for input, opening files, handling command-line options, and that kind of stuff. It distracts from the real problem, so get that out of the way first.
Write your code (the real code you came for) in its own function, separate from any other stuff you also need in order to house it. Get it elsewhere if you can; it's not part of the lesson and just serves as a distraction. Worse, it may be "hard" in ways you are not prepared for (or to do well) as it's unrelated to the actual lesson being worked on.
Figure out the algorithm (flowchart, pseudocode, whatever) in a general way before translating that to legal syntax and API on the objects you are using. If you're just learning C++, don't get bogged down in the formal syntax when you are trying to figure out the logic. Until you naturally start to think in C++ when doing that kind of planning, don't force it. Use whiteboard doodles, tinkertoys, whatever works for you.
Get feedback and review of the idea, the logic of how to make it happen, from your peers and mentors if available, before you spend time coding. Why write up an idea that doesn't work? Fix the logic, not the code.
Finally, sketch the needed control flow, functions and data structures you need. Use pseudocode and placeholder notes.
Then fill in the placeholders and replace the pseudo with the legal syntax. You already planned it out, so now you can concentrate on learning the syntax and library details of the programming language. You can concentrate on "how do I express (some tiny detail) in C++" rather than keeping the entire program in your head. More generally, isolate a part that you will be learning; be learning/practicing one thing without worrying about the entire edifice.
To a large extent, some of those ideas translate to the code as well. Top-Down Design means you state things at a high level and then implement that elsewhere, separately. It makes code readable and maintainable, as well as easier to write in the first place. Functions should be written this way: the function explains how to do (what it does) as a list of details that are just one level of detail further down. Each of those steps then becomes a new function. Functions should be short and expressed at one semantic level of abstraction. Don't dive down into the most primitive details inside the function that explains the task as a set of simpler steps.
Good luck, and keep it up!

How to write Multiplicative Update Rules for Matrix Factorization when one doesn't have access to the whole matrix?

So we want to approximate the matrix A with m rows and n columns with the product of two matrices P and Q that have dimension mxk and kxn respectively. Here is an implementation of the multiplicative update rule due to Lee in C++ using the Eigen library.
void multiplicative_update()
{
Q = Q.cwiseProduct((P.transpose()*matrix).cwiseQuotient(P.transpose()*P*Q));
P = P.cwiseProduct((matrix*Q.transpose()).cwiseQuotient(P*Q*Q.transpose()));
}
where P, Q, and the matrix (matrix = A) are global variables in the class mat_fac. Thus I train them using the following method,
void train_2(){
double error_trial = 0;
for (int count = 0;count < num_iterations; count ++)
{
multiplicative_update();
error_trial = (matrix-P*Q).squaredNorm();
if (error_trial < 0.001)
{
break;
}
}
}
where num_iterations is also a global variable in the class mat_fac.
The problem is that I am working with very large matrices and in particular I do not have access to the entire matrix. Given a triple (i,j,matrix[i][j]), I have access to the row vector P[i][:] and the column vector Q[:][j]. So my goal is to write rewrite the multiplicative update rule in such a way that I update these two vectors every time, I see a non-zero matrix value.
In code, I want to have something like this:
void multiplicative_update(int i, int j, double mat_value)
{
Eigen::MatrixXd q_vect = get_vector(1, j); // get_vector returns Q[:][j] as a column vector
Eigen::MatrixXd p_vect = get_vector(0, i); // get_vector returns P[i][:] as a column vector
// Somehow compute coeff_AQ_t, coeff_PQQ_t, coeff_P_tA and coeff_P_tA.
for(int i = 0; i< k; i++):
p_vect[i] = p_vect[i]* (coeff_AQ_t)/(coeff_PQQ_t)
q_vect[i] = q_vect[i]* (coeff_P_tA)/(coeff_P_tA)
}
Thus the problem boils down to computing the required coefficients given the two vectors. Is this a possible thing to do? If not, what more data do I need for the multiplicative update to work in this manner?

how to elminate the "doubled" elements of a vector in c++

I'm using the HoughLinesto detect line in a frame, the lines information are saved in a cv::vector<cv::Vec2f> which I handle as two dimensional array, I'm interested in the second one , it the angle of the line, I want to keep only the lines that have a angle difference greater than 1.5 rad for that here I what I did :
.............................
cv::vector<cv::Vec2f> lineQ;
..............................
// ordring the vector based on the angle value in rad
for ( int i = 0 ; i< lineQ.size()-1; i++){
for(int j= i+1;j<lineQ.size();j++){
if(lineQ[i][1] > lineQ[j][1]){
tmp = lineQ[i];
lineQ[i] = lineQ[j];
lineQ[j] = tmp;
}
}
}
now I want to compare the vector elements between each other based on the angle
cv::vector<cv::Vec2f> line;
for ( int i = 0 ; i< lineQ.size()-1; i++){
for ( int j= i+1; j<lineQ.size(); j++){
if(fabs(lineQ[i][1] - lineQ[j][1])>1.5){
line.push_back(lineQ[i]);
}
}
}
this works for 2 lines but when I got 3 whit let's say 1.3rad as an angle the size of line
is than 2. I though to use erase but this change the size of my vector !
One option is to supply a soft "equals" to std::unique_copy:
std::unique_copy(lineQ.begin(), lineQ.end(), std::back_inserter(line),
[](const cv::Vec2f & a, const cv::Vec2f & b) {
return b[1] - a[1] <= 1.5;
});
Sidenote: You can also avoid the effort of writing your own sort (Bubble sort is just about the worst choice.) and use the standard library. Something like this ought to work:
std::sort(lineQ.begin(), lineQ.end(),
[](const cv::Vec2f & a, const cv::Vec2f & b) {
return a[1] < b[1];
})).
(The above code assumes C++11, which most of us have by now. If you're stuck on an earlier version, you can write a couple of functor classes instead.)

Matrix multiplication in a cpp file for Matlab

How would I do a matrix multiplication in cpp format that would after be compiled into a mex file?
My normal matrix multiplication in a Matlab script is as follow:
cMatrix = (1 / r) * pfMatrix * wcMatrix; %here pfMatrix is 2x3 and wcMatrix is 3x8
% Hence cMatrix is 2x8
% r is a scalar
The pfMatrix, wcMatrix and r are declared correctly in the cpp file and they have the same values as in the script. However cMatrix doesn't give me the same results. Here the implementation of the Matrix multiplication in the cpp :
int i, n, j;
for (i = 0; i<1; i++)
{
for (n = 0; n<7; n++)
{
for (j = 0; j<2; j++)
{
d->cMatrix[i][n] += (d->pfMatrix[i][j]) * (d->wcMatrix[j][n]);
}
d->cMatrix[i][n] = (1 / d->r) * d->cMatrix[i][n];
}
}
Edit:
I modified the loop following Ben Voigt answer. The results in cMatrix are still not identical to the one calculated from the Matlab script.
For example :
pfMatrix = [7937.91049469652,0,512;0,7933.81033431703,384];
wcMatrix = [-0.880633810389421,-1.04063381038942,-1.04063381038942,-0.880633810389421,-0.815633810389421,-1.10563381038942,-1.10563381038942,-0.815633810389421;-0.125,-0.125,0.125,0.125,-0.29,-0.29,0.29,0.29;100,100,100,100,100,100,100,100];
r = 100;
In this case, cMatrix(1,1) is :
(pfMatrix(1,1)*wcMatrix(1,1) + pfMatrix(1,2)*wcMatrix(2,1) + pfMatrix(1,3)*wcMatrix(3,1)) / r = 442.09
However, with the mex file the equivalent result is 959.
Edit #2:
I found the error in an element of pfMatrix that was not declared correctly (missing a division by 2). So the answer of Ben Voigt is working correctly. However, there is still a slight difference between the two results (Matlab script gives 442 and the mex gives 447, could it be a results of different data type?).
Edit #3:
Found the error and it was not related with the matrix multiplication loop.
Using your result matrix as scratch space is not a great idea. The compiler has to worry about aliasing, which means it can't optimize.
Try an explicit working variable, which also provides a convenient place to zero it:
for (int i = 0; i < 2; ++i) {
for (int n = 0; n < 8; ++n) {
double accum = 0.0;
for (int j = 0; j < 3; ++j) {
accum += (d->pfMatrix[i][j]) * (d->wcMatrix[j][n]);
}
d->cMatrix[i][n] = accum / d->r;
}
}
Your ranges were also wrong, which I've fixed.
(Also note that good performance on large matrices requires banding to get good cache behavior, however that shouldn't be an issue on a product of this size.)
A multiplication between matrices must be in this way: A[m][n] * B[n][p] = R[m][p].
The conditions that you wrote in the for loops are not correct and doesn't respect the matrix dimensions.
Look also at the Eigen libraries, which are open-source and provide a simple way to do the matrix multiplications.

opencv : create matrix or vector of matrices

I have the following code, which is a part of the algorithm that I am following. as you see I need to do some calculation for 10 different bands. and will end up with a matrix for each band that I need to recreate an image from it, the problem is that I dont know how to create/hold the 10 different matrix on the while loop, then after the while loop I can construct the images one by one. if you have any idea please let me know thank you
cv::Mat _reconstructionMatrix(height,width,CV_8UC1);
_reconsPointer = _reconstructionMatrix.ptr<uchar>(0);
while(_bandIteration<_bandsNumber){
if(_mainMatrix.isContinuous())
{
nCols *= nRows;
nRows = 1;
}
//for all the pixels
for(int i = 0; i < nRows; i++)
{
p = _mainMatrix.ptr<uchar>(i);
//in the images
for (int j = 0; j < nCols; j++)
{
if(_pCounter<_totalImgNO){
....
}else{
...
_reconsPointer[_resultFlag]=_summation;
_resultFlag++;
...
}
}
}
_bandIteration++;
}
Your question is a bit vague. But if you are asking simply how to create/hold the 10 different matrix on the while loop? then you can use STL vectors as normal.
#include<vector>
...
std::vector<cv::Mat> listOfMatrices;
...
cv::Mat M = SomehowGetMatrix();
listOfMatrices.push_back(M);
If this is not what you are looking for, then please provide more detail to your question.