I was given some code to paralellize using OpenMP and, among the various function calls, I noticed this for loop takes some good guilt on the computation time.
double U[n][n];
double L[n][n];
double Aprime[n][n];
for(i=0; i<n; i++) {
for(j=0; j<n; j++) {
if (j <= i) {
double s;
s=0;
for(k=0; k<j; k++) {
s += L[j][k] * U[k][i];
}
U[j][i] = Aprime[j][i] - s;
} else if (j >= i) {
double s;
s=0;
for(k=0; k<i; k++) {
s += L[j][k] * U[k][i];
}
L[j][i] = (Aprime[j][i] - s) / U[i][i];
}
}
However, after trying to parallelize it and applying some semaphores here and there (with no luck), I came to the realization that the else if condition has a strong dependency on the early if (L[j][i] being a processed number with U[i][i], which may be set on the early if), making it, in my oppinion, non-parallelizable due to race conditions.
Is it possible to parallelize this code in such a manner to make the else if only be executed if the earlier if has already completed?
Before trying to parallelize things, try simplification first.
For example, the if can be completely eliminated.
Also, the code is accessing the matrixes in a way that causes worst cache performance. That may be the real bottleneck.
Note: In update #3 below, I did benchmarks and the cache friendly version fix5, from update #2, outperforms the original by 3.9x.
I've cleaned things up in stages, so you can see the code transformations.
With this, it should be possible to add omp directives successfully. As I mentioned in my top comment, the global vs. function scope of the variables affects the type of update that may be required (e.g. omp atomic update, etc.)
For reference, here is your original code:
double U[n][n];
double L[n][n];
double Aprime[n][n];
for (i = 0; i < n; i++) {
for (j = 0; j < n; j++) {
if (j <= i) {
double s;
s = 0;
for (k = 0; k < j; k++) {
s += L[j][k] * U[k][i];
}
U[j][i] = Aprime[j][i] - s;
}
else if (j >= i) {
double s;
s = 0;
for (k = 0; k < i; k++) {
s += L[j][k] * U[k][i];
}
L[j][i] = (Aprime[j][i] - s) / U[i][i];
}
}
}
The else if (j >= i) was unnecessary and could be replaced with just else. But, we can split the j loop into two loops so that neither needs an if/else:
// fix2.c -- split up j's loop to eliminate if/else inside
double U[n][n];
double L[n][n];
double Aprime[n][n];
for (i = 0; i < n; i++) {
for (j = 0; j <= i; j++) {
double s = 0;
for (k = 0; k < j; k++)
s += L[j][k] * U[k][i];
U[j][i] = Aprime[j][i] - s;
}
for (; j < n; j++) {
double s = 0;
for (k = 0; k < i; k++)
s += L[j][k] * U[k][i];
L[j][i] = (Aprime[j][i] - s) / U[i][i];
}
}
U[i][i] is invariant in the second j loop, so we can presave it:
// fix3.c -- save off value of U[i][i]
double U[n][n];
double L[n][n];
double Aprime[n][n];
for (i = 0; i < n; i++) {
for (j = 0; j <= i; j++) {
double s = 0;
for (k = 0; k < j; k++)
s += L[j][k] * U[k][i];
U[j][i] = Aprime[j][i] - s;
}
double Uii = U[i][i];
for (; j < n; j++) {
double s = 0;
for (k = 0; k < i; k++)
s += L[j][k] * U[k][i];
L[j][i] = (Aprime[j][i] - s) / Uii;
}
}
The access to the matrixes is done in probably the worst way for cache performance. So, if the assignment of dimensions can be flipped, a substantial savings in memory access can be achieved:
// fix4.c -- transpose matrix coordinates to get _much_ better memory/cache
// performance
double U[n][n];
double L[n][n];
double Aprime[n][n];
for (i = 0; i < n; i++) {
for (j = 0; j <= i; j++) {
double s = 0;
for (k = 0; k < j; k++)
s += L[k][j] * U[i][k];
U[i][j] = Aprime[i][j] - s;
}
double Uii = U[i][i];
for (; j < n; j++) {
double s = 0;
for (k = 0; k < i; k++)
s += L[k][j] * U[i][k];
L[i][j] = (Aprime[i][j] - s) / Uii;
}
}
UPDATE:
In the Op's first k-loop its k<j and in the 2nd k<i don't you have to fix that?
Yes, I've fixed it. It was too ugly a change for fix1.c, so I removed that and applied the changes to fix2-fix4 where it was easy to do.
UPDATE #2:
These variables are all local to the function.
If you mean they are function scoped [without static], this says that the matrixes can't be too large because, unless the code increases the stack size, they're limited to the stack size limit (e.g. 8 MB)
Although the matrixes appeared to be VLAs [because n was lowercase], I ignored that. You may want to try a test case using fixed dimension arrays as I believe they may be faster.
Also, if the matrixes are function scope, and want to parallelize things, you'd probably need to do (e.g.) #pragma omp shared(Aprime) shared(U) shared(L).
The biggest drag on cache were the loops to calculate s. In fix4, I was able to make access to U cache friendly, but L access was poor.
I'd need to post a whole lot more if I did include the external context
I guessed as much, so I did the matrix dimension swap speculatively, not knowing how much other code would need changing.
I've created a new version that changes the dimensions on L back to the original way, but keeping the swapped versions on the other ones. This provides the best cache performance for all matrixes. That is, the inner loop for most matrix access is such that each iteration is incrementing along the cache lines.
In fact, give it a try. It may improve things to the point where parallel isn't needed. I suspect the code is memory bound anyway, so parallel might not help as much.
// fix5.c -- further transpose to fix poor performance on s calc loops
//
// flip the U dimensions back to original
double U[n][n];
double L[n][n];
double Aprime[n][n];
double *Up;
double *Lp;
double *Ap;
for (i = 0; i < n; i++) {
Ap = Aprime[i];
Up = U[i];
for (j = 0; j <= i; j++) {
double s = 0;
Lp = L[j];
for (k = 0; k < j; k++)
s += Lp[k] * Up[k];
Up[j] = Ap[j] - s;
}
double Uii = Up[i];
for (; j < n; j++) {
double s = 0;
Lp = L[j];
for (k = 0; k < i; k++)
s += Lp[k] * Up[k];
Lp[i] = (Ap[j] - s) / Uii;
}
}
Even if you really need the original dimensions, depending upon the other code, you might be able to transpose going in and transpose back going out. This would keep things the same for other code, but, if this code is truly a bottleneck, the extra transpose operations might be small enough to merit this.
UPDATE #3:
I've run benchmarks on all the versions. Here are the elapsed times and ratios relative to original for n equal to 1037:
orig: 1.780916929 1.000x
fix1: 3.730602026 0.477x
fix2: 1.743769884 1.021x
fix3: 1.765769482 1.009x
fix4: 1.762100697 1.011x
fix5: 0.452481270 3.936x
Higher ratios are better.
Anyway, this is the limit of what I can do. So, good luck ...
Related
I'm trying to implement parallel algorithm that will compute Levenshtein distance between each of sequences in a list and store them in matrix (2d vector). In other words, I'm given 2d vector with numbers (thousands of number sequences of up to 30 numbers) and I need to compute Levenshtein distance between each vector of integers. I implemented serial algorithm that works, but when I tried to convert it to parallel, it is much slower (the more threads, the slower it is). The parallel version is implemented with c++11 threads (I also tried OpenMP, but with the same results).
Here is the function that distributes work:
vector<vector<int>> getGraphParallel(vector<vector<int>>& records){
int V = records.size();
auto threadCount = std::thread::hardware_concurrency();
if(threadCount == 0){
threadCount = 1;
}
vector<future<vector<vector<int>>>> futures;
int rowCount = V / threadCount;
vector<vector<int>>::const_iterator first = records.begin();
vector<vector<int>>::const_iterator last = records.begin() + V;
for(int i = 0; i < threadCount; i++){
int start = i * rowCount;
if(i == threadCount - 1){
rowCount += V % threadCount;
}
futures.push_back(std::async(getRows, std::ref(records), start, rowCount, V));
}
vector<vector<int>> graph;
for(int i = 0; i < futures.size(); i++){
auto result = futures[i].get();
for(const auto &row : result){
graph.push_back(row);
}
}
for(int i = 0; i < V; i++)
{
for(int j = i + 1; j < V; j++){
graph[j][i] = graph[i][j];
}
}
return graph;
}
Here is the function that computes rows of final matrix:
vector<vector<int>> getRows(vector<vector<int>>& records, int from, int count, int size){
vector<vector<int>> result(count, vector<int>(size, 0));
for(int i = 0; i < count; i++){
for(int j = i + from + 1; j < size; j++){
result[i][j] = levenshteinDistance(records[i + from], records[j]);
}
}
return result;
}
And finally function that computes Levenshtein distance:
int levenshteinDistance(const vector<int>& first, const vector<int>& second){
const int sizeFirst = first.size();
const int sizeSecond = second.size();
if(sizeFirst == 0) return sizeSecond;
if(sizeSecond == 0) return sizeFirst;
vector<vector<int>> distances(sizeFirst + 1, vector<int>(sizeSecond + 1, 0));
for(int i = 0; i <= sizeFirst; i++){
distances[i][0] = i;
}
for(int j = 0; j <= sizeSecond; j++){
distances[0][j] = j;
}
for (int j = 1; j <= sizeSecond; j++)
for (int i = 1; i <= sizeFirst; i++)
if (first[i - 1] == second[j - 1])
distances[i][j] = distances[i - 1][j - 1];
else
distances[i][j] = min(min(
distances[i - 1][j] + 1,
distances[i][j - 1] + 1),
distances[i - 1][j - 1] + 1
);
return distances[sizeFirst][sizeSecond];
}
One thing that came to my mind is that this slow down is caused by false sharing, but I could not check it with perf, because I'm working with Ubuntu in Oracle VirtualBox - cache misses are not avalaible there. If I'm right and the slow down is caused by false sharing, what should I do to fix it? If not, what is the reason of this slow down?
One problem I can see is that you are using std::async without declaring how it should run. It can either be run async or deferred. In the case of deferred it would all run in one thread, it would just be lazily evaluated. The default behavior is implementation defined. For your case it'd make sense that it would run slower with more deferred evaluations. You can try std::async(std::launch::async, ...).
Make sure your VM is also set up to use more than 1 core. Ideally when doing optimizations such as these it'd be best to try and eliminate as many variables as possible. If you can, run the program locally without a VM. Profiling tools are your best bet and will show you exactly where time is being spent.
Say I have the following code for backsubstitution.
for (int i = n - 1; i >= 0; i--) {
double temp = b[i];
if (i != n - 1) {
for (int j = i + 1; j < n; j++)
temp -= A[i][j] * b[j];
}
b[i] = temp/A[i][i];
}
But for me as a newbie to programming, the following seems simpler:
for (int i = n - 1; i >= 0; i--) {
if (i != n - 1) {
for (int j = i + 1; j < n; j++)
b[i] -= A[i][j] * b[j];
}
b[i] /= A[i][i];
}
which requires the indexing of b[i] every time it iterates from j = i+1 to j = n - 1. But this b[i] is a fixed quantity since this iteration does not depend on the value of i.
But I'm not sure which one the complier more prefers. Any help?
The definition double temp = b[i]; encourages the compiler to keep the tally temp in one of the CPU's registers. With your other code, the compiler might leave the tally in main memory, registers being unsuited to store arrays.
The array would probably be kept in cache unless it were very large, so it would still be fairly fast; but registers are faster.
I'm trying to implement a quick program to solve a system of linear equations. The program reads the input from a file and then writes the upper-triangular system and solutions to a file. It is working with no pivoting, but when I try to implement the pivoting it produces incorrect results.
As example input, here is the following system of equations:
w+2x-3y+4z=12
2w+2x-2y+3z=10
x+y=-1
w-x+y-2z=-4
I expect the results to be w=1, x=0, y=-1 and z=2. When I don't pivot, I get this answer (with some rounding error on x). When I add in the pivoting, I get the same numbers but in the wrong order: w=2,x=1,y=-1 and z=0.
What do I need to do to get these in the correct order? Am I missing a step somewhere? I need to do column swapping instead of rows because I need to adapt this to a parallel algorithm later that requires that. Here is the code that does the elimination and back substitution:
void gaussian_elimination(double** A, double* b, double* x, int n)
{
int maxIndex;
double temp;
int i;
for (int k = 0; k < n; k++)
{
i = k;
for (int j = k+1; j < n; j++)
{
if (abs(A[k][j]) > abs(A[k][i]))
{
i = j;
}
}
if (i != k)
{
for (int j = 0; j < n; j++)
{
temp = A[j][k];
A[j][k] = A[j][i];
A[j][i] = temp;
}
}
for (int j = k + 1; j < n; j++)
{
A[k][j] = A[k][j] / A[k][k];
}
b[k] = b[k] / A[k][k];
A[k][k] = 1;
for (i = k + 1; i < n; i++)
{
for (int j = k + 1; j < n; j++)
{
A[i][j] = A[i][j] - A[i][k] * A[k][j];
}
b[i] = b[i] - A[i][k] * b[k];
A[i][k] = 0;
}
}
}
void back_substitution(double**U, double*x, double*y, int n)
{
for (int k = n - 1; k >= 0; k--)
{
x[k] = y[k];
for (int i = k - 1; i >= 0; i--)
{
y[i] = y[i] - x[k]*U[i][k];
}
}
}
I believe what you implemented is actually complete pivoting.
With complete pivoting, you must keep track of the permutation of columns, and apply the same permutation to your answer.
You can do this with an array {0, 1, ..., n}, where you swap the i'th and k'th values in the second loop. Then, rearange the solution using this array.
If what you were trying to do is partial pivoting, you need to look for the maximum in the respective row, and swap the rows and the values of 'b' accordingly.
I have to use a nested for-loop to compute the entries of a Eigen::MatrixXd type matrix output columnwise. Here input[0], input[1] and input[2] are defined as Eigen::ArrayXXd in order to use the elementwise oprerations. This part seems to be the bottleneck for my code. Can anyone help me to accelerate this loop? Thanks!
for (int i = 0; i < r; i++) {
for (int j = 0; j < r; j++) {
for (int k = 0; k < r; k++) {
output.col(i * (r * r) + j * r + k) =
input[0].col(i) * input[1].col(j) * input[2].col(k);
}
}
}
When thinking about optimizing code of a for loop, it helps to think, "Are there redundant calculations that I can eliminate?"
Notice how in the inner most loop, only k is changing. You should move all possible calculations that don't involve k out of that loop:
for (int i = 0; i < r; i++) {
int temp1 = i * (r * r);
for (int j = 0; j < r; j++) {
int temp2 = j * r;
for (int k = 0; k < r; k++) {
output.col(temp1 + temp2 + k) =
input[0].col(i) * input[1].col(j) * input[2].col(k);
}
}
}
Notice how i * (r * r) is being calculated over and over, but the answer is always the same! You only need to recalculate this when i increments. The same goes for j * r.
Hopefully this helps!
To reduce the number of flops, you should cache the result of input[0]*input[1]:
ArrayXd tmp(input[0].rows());
for (int i = 0; i < r; i++) {
for (int j = 0; j < r; j++) {
tmp = input[0].col(i) * input[1].col(j);
for (int k = 0; k < r; k++) {
output.col(i * (r * r) + j * r + k) = tmp * input[2].col(k);
}
}
}
Then, to fully use your CPU, enable AVX/FMA with -march=native and of course compiler optimizations (-O3).
Then, to get an idea of what you could gain more, measure accurately the time taken by this part, count the number of multiplications (r^2*(n+r*n)), and then compute the number of floating point operations per second you achieve. Then compare it to the capacity of your CPU. If you're good, then the only option is to multithread one of the for loop using, e.g., OpenMP. The choice of which for loop depends on the size of your inputs, but you can try with the outer one, making sure each thread has its own tmp array.
First, I understand that the are many algorithms for matrix multiplication. For this question, I will be considering the following:
Algorithm A:
for(i = 0; i < N; i++)
{
for(j = 0; j < N; j++)
{
prod[i][j] = 0;
for(k = 0; k < N; k++)
prod[i][j] += mat1[i][k] * mat2[k][j]
}
}
Algorithm B:
for(i = 0; i < N; i++)
{
for(j = 0; j < N; j++)
{
temp = 0;
for(k = 0; k < N; k++)
temp += mat1[i][k] * mat2[k][j]
prod[i][j] = temp;
}
}
A coworker and I both agree that algorithm B outperforms (faster run times than) algorithm A. However, we both have different reasoning for why B performs better.
The first theory is that because we replace C[i][j] inside the inner most loop, we perform N3 less pointer arithmetic operations, thus causing our improvement.
The second theory is that because we replace C[i][j] inside the inner most loop, we perform N3 less memory accesses, thus causing our improvement.
The question is:
How do I set up an experiment to isolate each of these factors to decide empirically which, if either, have the larger affect on run time?