I ve got two matrices W2 and hiddenLayer and i want to proceed the multiplication of those. W2 size's 12x50 and hiddenLayer size's 50x1. The proper code for the above calculation:
for(int h=0; h<50; h++){
for(int k=0; k<12; k++){
outputLayer += W2[k][h]*HiddenLayer[h];
}
}
or i ve got to put at first k-for??
Matrix multiplication is defined as:
C = AB ⇔ Ci,j = Σk=1..n Ai,k Bk,j for i,j = 1...n (in case of square matrices).
Thus outputLayer is a vector. Since HiddenLayer is a vector too, this isn't really a matrix multiplication but a matrix vector multiplication, which simplifies the formula above:
b = Ax ⇔ bi = Σk=1..m Ai,k xk for i = 1...n (A is an n x m matrix).
So all in all your code should be something like
for(int row = 0; row < 12; row++){
outputLayer[row] = 0;
for(int column = 0; column < 50; column++){
outputLayer[row] += W2[row][column]*HiddenLayer[column];
}
}
Related
Let's say I have a 4x4 matrix M and a 3x3 matrix R.
How can write the 3x3 submatrix of M with R?
The naive way would be using two loops:
for(int i = 0; i < 3; i++) {
for(int j = 0; j < 3; j++) {
M[i][j] = R[i][j];
}
}
But I would like to know if there is a shorthand for this in GLM.
EDIT:
Someone pointed out there is a similar question: Simplest way to convert mat3 to mat4
But it's not the same because, in that question, the 4th row&col are filled with the pattern of the identity matrix. But I need to preserve the original contents of the 4th row&col of M
I came up with this algorithm for matrix multiplication. I read somewhere that matrix multiplication has a time complexity of o(n^2).
But I think my this algorithm will give o(n^3).
I don't know how to calculate time complexity of nested loops. So please correct me.
for i=1 to n
for j=1 to n
c[i][j]=0
for k=1 to n
c[i][j] = c[i][j]+a[i][k]*b[k][j]
Using linear algebra, there exist algorithms that achieve better complexity than the naive O(n3). Solvay Strassen algorithm achieves a complexity of O(n2.807) by reducing the number of multiplications required for each 2x2 sub-matrix from 8 to 7.
The fastest known matrix multiplication algorithm is Coppersmith-Winograd algorithm with a complexity of O(n2.3737). Unless the matrix is huge, these algorithms do not result in a vast difference in computation time. In practice, it is easier and faster to use parallel algorithms for matrix multiplication.
The naive algorithm, which is what you've got once you correct it as noted in comments, is O(n^3).
There do exist algorithms that reduce this somewhat, but you're not likely to find an O(n^2) implementation. I believe the question of the most efficient implementation is still open.
See this wikipedia article on Matrix Multiplication for more information.
The standard way of multiplying an m-by-n matrix by an n-by-p matrix has complexity O(mnp). If all of those are "n" to you, it's O(n^3), not O(n^2). EDIT: it will not be O(n^2) in the general case. But there are faster algorithms for particular types of matrices -- if you know more you may be able to do better.
In matrix multiplication there are 3 for loop, we are using since execution of each for loop requires time complexity O(n). So for three loops it becomes O(n^3)
I recently had a matrix multiplication problem in my college assignment, this is how I solved it in O(n^2).
import java.util.Scanner;
public class q10 {
public static int[][] multiplyMatrices(int[][] A, int[][] B) {
int ra = A.length; // rows in A
int ca = A[0].length; // columns in A
int rb = B.length; // rows in B
int cb = B[0].length; // columns in B
// if columns of A is not equal to rows of B, then the two matrices,
// cannot be multiplied.
if (ca != rb) {
System.out.println("Incorrect order, multiplication cannot be performed");
return A;
} else {
// AB is the product of A and B, and it will have rows,
// equal to rown in A and columns equal to columns in B
int[][] AB = new int[ra][cb];
int k = 0; // column number of matrix B, while multiplying
int entry; // = Aij, value in ith row and at jth index
for (int i = 0; i < A.length; i++) {
entry = 0;
k = 0;
for (int j = 0; j < A[i].length; j++) {
// to evaluate a new Aij, clear the earlier entry
if (j == 0) {
entry = 0;
}
int currA = A[i][j]; // number selected in matrix A
int currB = B[j][k]; // number selected in matrix B
entry += currA * currB; // adding to the current entry
// if we are done with all the columns for this entry,
// reset the loop for next one.
if (j + 1 == ca) {
j = -1;
// put the evaluated value at its position
AB[i][k] = entry;
// increase the column number of matrix B as we are done with this one
k++;
}
// if this row is done break this loop,
// move to next row.
if (k == cb) {
j = A[i].length;
}
}
}
return AB;
}
}
#SuppressWarnings({ "resource" })
public static void main(String[] args) {
Scanner ip = new Scanner(System.in);
System.out.println("Input order of first matrix (r x c):");
int ra = ip.nextInt();
int ca = ip.nextInt();
System.out.println("Input order of second matrix (r x c):");
int rb = ip.nextInt();
int cb = ip.nextInt();
int[][] A = new int[ra][ca];
int[][] B = new int[rb][cb];
System.out.println("Enter values in first matrix:");
for (int i = 0; i < ra; i++) {
for (int j = 0; j < ca; j++) {
A[i][j] = ip.nextInt();
}
}
System.out.println("Enter values in second matrix:");
for (int i = 0; i < rb; i++) {
for (int j = 0; j < cb; j++) {
B[i][j] = ip.nextInt();
}
}
int[][] AB = multiplyMatrices(A, B);
System.out.println("The product of first and second matrix is:");
for (int i = 0; i < AB.length; i++) {
for (int j = 0; j < AB[i].length; j++) {
System.out.print(AB[i][j] + " ");
}
System.out.println();
}
}
}
Given a rank-4 tensor (each rank with dimension K), for example T(p,q,r,s), we can 1-to-1 map all the tensor elements into a matrix of dimension K^2 x K^2, for example M(i,j) in which the two first tensor indices p,q and the last two indices r,s are combined in a column major way:
i = p + K * q
j = r + K * s
Exploiting some (anti-)symmetries of the given tensor, for example T(p,q,r,s) = -T(q,p,r,s) = -T(p,q,s,r) = T(q,p,s,r) and T(p,q,r,s) = T(r,s,p,q), we would like to be able to construct a matrix H(m,n) that only contains the unique elements (i.e. those not related by the previously defined symmetries), such that p>q and r>s into the matrix H(m,n), which would then be of dimension K(K-1)/2 x K(K-1)/2.
How could we find an algorithm (or even better: how can we use the C++ library Eigen) to accomplish these index transformations? Furthermore, can we write down m and n algebraically in terms of p,q and r,s, like we can do in the case where we would want to extract the strict lower triangular matrix (no diagonal) into a vector?
For reference, given a square matrix Eigen::MatrixXd M (K,K), here is an algorithm that extracts the strict lower triangle of a given square matrix M into a vector m of size, using the C++ Eigen library:
Eigen::VectorXd strictLowerTriangle(const Eigen::MatrixXd& M) {
auto K = static_cast<size_t>(M.cols()); // the dimension of the matrix
Eigen::VectorXd m = Eigen::VectorXd::Zero((K*(K-1)/2)); // strictly lower triangle has K(K-1)/2 parameters
size_t vector_index = 0;
for (size_t q = 0; q < K; q++) { // "column major" ordering for, so we do p first, then q
for (size_t p = q+1; p < K; p++) {
m(vector_index) = M(p,q);
vector_index++;
}
}
return m;
}
We are able to extend this algorithm to the requested general case:
Eigen::MatrixXd strictLowerTriangle(const Eigen::Tensor<double, 4>& T) {
auto K = static_cast<size_t> (dims[0]);
Eigen::MatrixXd M (K*(K-1)/2, K*(K-1)/2);
size_t row_index = 0;
for (size_t j = 0; j < K; j++) { // "column major" ordering for row_index<-i,j so we do j first, then i
for (size_t i = j+1; i < K; i++) { // in column major indices, columns are contiguous, so the first of two indices changes more rapidly
// require i > j
size_t column_index = 0;
for (size_t l = 0; l < K; l++) { // "column major" ordering for column_index<-k,l so we do l first, then k
for (size_t k = l+1; k < K; k++) { // in column major indices, columns are contiguous, so the first of two indices changes more rapidly
// require l > k
M(row_index,column_index) = T(i,j,k,l);
column_index++;
}
}
row_index++;
}
}
return M;
}
I want to compute the error in linear least squares method.
I have matrices A,B and X. (AX=B).
Sizes are : A(NxN) , B(NxNRHS) , X(N,NRHS) ,where NRHS is number of right hand side.
The error is computed as sqrt(sum(B-AX)).
But I must take into account every column of B and X in order to make the substraction.
I must substract B[i]-A[..]X[i] -> where i is every column of B and X.
I can't figure how to do it ,hence how to extract every column.I can't find the right indices for B and X matrices (I think) ,because I must go beyond whole A matrix and only beyond every column of B and X.
I am doing something like this (using column major order):
int N=128;
int NRHS =1;
int Asize=N*N;
int Bsize=N*NRHS;
int Xsize=N*NRHS;
A=(double*)malloc(Asize*sizeof(double));
B=(double*)malloc(Bsize*sizeof(double));
X=(double*)malloc(Xsize*sizeof(double));
...
for(int i = 0; i < N; i++)
{
for (int j=0;j<NRHS; j++){
diff[i+j*N] = fabs(B[i+j*N] - A[i+j*N]*X[i+j*N]);
abs_error=sqrt(sums(diff,N));
}
}
I thought of adding some statement using the modulo operator but I couldn't figure.
sums is just a function which gives the sum of an array where the second argument is the number of elements.
You could first do a matrix multiplication of A and X using loops.
Then you could write another 2 loops to compute the difference (B - AX). This would simply your problem.
Edit
After you compute the product of A and X, assuming that you store the product in a variable named AX,the following code will give you the difference between corresponding elements.
differenceMatrix = (double*)malloc(Bsize*sizeof(double));
for(int i = 0; i < N; i++)
{
for (int j = 0; j < NRHS; j++){
differenceMatrix[i+j*N] = fabs(B[i+j*N] - AX[i+j*N]);
}
}
Each column of the differenceMatrix contains the difference between corresponding elements.
Edit
To obtain the sum of difference of each column
double sumOfDifferencePerColumn;
for(int i = 0; i < N; i++)
{
sumOfDifferencePerColumn = 0.0;
for (int j = 0; j < NRHS; j++){
sumOfDifferencePerColumn += ( fabs(B[i+j*N] - AX[i+j*N]) );
}
// add code to take square root or use the sum of difference of each column
}
I want to create a function that calculates neural network output. The elements of my NN
is a 19D input vector and a 19D output vector. I choose one hidden layer with 50 neurons. My code is the following but i am not quite sure if it works properly.
double *BuildPlanner::neural_tactics(){
norm(); //normalize input vector
ReadFromFile(); // load weights W1 W2 b1
double hiddenLayer [50][1];
for(int h=0; h<50; h++){
hiddenLayer[h][0] =0;
for(int f = 0; f < 19; f++){
hiddenLayer[h][0] = hiddenLayer[h][0] + W1[h][f]*input1[f][0];
}
}
double HiddenLayer[50][1];
for(int h=0; h<50; h++){
HiddenLayer[h][0] = tanh(hiddenLayer[h][0] + b1[h][0]);
}
double outputLayer[50][1];
for(int h=0; h<19; h++){
for(int k=0; k<50; k++){
outputLayer[h][0] = outputLayer[h][0] + W2[h][k]*HiddenLayer[k][0];
}
}
double Output[19];
for(int h=0; h<19; h++){
Output[h] = tanh(outputLayer[h][0]);
}
return Output;
}
Actually I not quite sure about the matrices multiplication. W1*input+b1 where the size
of the matrices are 50x19 * 19x1 + 50x1 and W2*outHiddenLayer 19x50*50x1!
Your matrix multiplication looks ok to me, but there are other problems--`outputLayer is 50x1 but a) you only iterate through the first 19 elements, and b) you have it on the RHS of your equation
outputLayer[h][0] = outputLayer[h][0] + W2[h][k]...
before that element has ever been defined. That could be causing all your problems. Also, although I assume you're making outputLayer 2-dimensional to make them look matrix-like, it's completely gratuitous and slows things down when the second dimension has size 1--just declare it and the others as
double outputLayer[50];
since it's a vector and those are always one dimensional so it will actually make your code clearer.