I'm running the following code for matrix multiplication the performance of which I'm supposed to measure:
for (int j = 0; j < COLUMNS; j++)
#pragma omp for schedule(dynamic, 10)
for (int k = 0; k < COLUMNS; k++)
for (int i = 0; i < ROWS; i++)
matrix_r[i][j] += matrix_a[i][k] * matrix_b[k][j];
Yes, I know it's really slow, but that's not the point - it's purely for performance measuring purposes. I'm running 3 versions of the code depending on where I put the #pragma omp directive, and therefore depending on where the parallelization happens. The code is run in Microsoft Visual Studio 2012 in release mode and profiled in CodeXL.
One thing I've noticed from the measurements is that the option in the code snippet (with parallelization before the k loop) is the slowest, then the version with the directive before the j loop, then the one with it before the i loop. The presented version is also the one which calculates a wrong result because of race conditions - multiple threads accessing the same cell of the result matrix at the same time. I understand why the i loop version is the fastest - all the particular threads process only part of the range of the i variable, increasing the temporal locality. However, I don't understand what causes the k loop version to be the slowest - does it have something to do with the fact that it produces the wrong result?
Of course race conditions can slow the code down. When two or more threads access the same part of memory (same cache line), that part must be loaded into the cache of the given cores over and over again as the the other thread invalidates the content of the cache by writing into it. They compete for a shared resource.
When two variables located too close in memory are written and read by more threads, it also results in a slowdown. This is known as false sharing. In your case it is even worse, they are not just too close, they even coincide.
Your assumption is correct. But if we are talking about performance, and not just validating your assumption, there is more to the story.
The order of your indexes is a big issue, multi-threaded or not. Given than the distance between mat[x][y] and mat[x][y+1] is one, while the distance between mat[x][y] and mat[x+1][y] is dim(mat[x]) You want x to be the outer index and y the inner to have the minimal distance between iteration. Given __[i][j] += __[i][k] * __[k][j]; you see that the proper order for spacial locality is i -> k -> j.
Whatever the order, there is one value which can be saved for later. Given your snippet
for (int j = 0; j < COLUMNS; j++)
for (int k = 0; k < COLUMNS; k++)
for (int i = 0; i < ROWS; i++)
matrix_r[i][j] += matrix_a[i][k] * matrix_b[k][j];
matrix_b[k][j] value will be fetched from memory i times. You could have started from
for (int j = 0; j < COLUMNS; j++)
for (int k = 0; k < COLUMNS; k++)
int temp = matrix_b[k][j];
for (int i = 0; i < ROWS; i++)
matrix_r[i][j] += matrix_a[i][k] * temp;
But given that you are writing to matrix_r[i][j], the best access to optimize is matrix_r[i][j], given that writing is slower than reading
Unnecessary write accesses to memory
for (int i = 0; i < ROWS; i++)
matrix_r[i][j] += matrix_a[i][k] * matrix_b[k][j];
will write to the memory of matrix_r[i][j] ROWS times. Using a temporary variable would reduce the accesses to one.
for (int i = 0; i < ...; j++)
for (int j = 0; j < ...; k++)
int temp = 0;
for (int k = 0; k < ...; i++)
temp += matrix_a[i][k] * matrix_b[k][j];
matrix_r[i][j] = temp;
This decreases write accesses from n^3 to n^2.
Now you are using threads. To maximize the efficiency of multithreading you should isolate as much a thread memory access from the others. One way to do it would be to give each thread a column, and prefect that column once. One simple way would be to have the transpose of matrix_b such that
matrix_r[i][j] += matrix_a[i][k] * matrix_b[k][j]; becomes
matrix_r[i][j] += matrix_a[i][k] * matrix_b_trans[j][k];
such that the most inner loop on k always deal with contiguous memory respective to matrix_a and matrix_b_trans
for (int i = 0; i < ROWS; j++)
for (int j = 0; j < COLS; k++)
int temp = 0;
for (int k = 0; k < SAMEDIM; i++)
temp += matrix_a[i][k] * matrix_b_trans[j][k];
matrix_r[i][j] = temp;
Related
I aim to compute a simple N-body program on C++ and I am using OpenMP to speed things up with the computations. At some point, I have nested loops that look like that:
int N;
double* S = new double[N];
double* Weight = new double[N];
double* Coordinate = new double[N];
...
#pragma omp parallel for
for (int i = 0; i < N; ++i)
{
for (int j = 0; j < i; ++j)
{
double K = Coordinate[i] - Coordinate[j];
S[i] += K*Weight[j];
S[j] -= K*Weight[i];
}
}
The issue here is that I do not obtain exactly the same result when removing the #pragma ... I am guessing it has to do with the fact that the second loop is dependent on the integer i, but I don't see how to get past that issue
The problem is that there is a data race during updating S[i] and S[j]. Different threads may read from/write to the same element of the array at the same time, therefore it should be an atomic operation (you have to add #pragma omp atomic) to avoid data race and to ensure memory consistency:
for (int j = 0; j < i; ++j)
{
double K = Coordinate[i] - Coordinate[j];
#pragma omp atomic
S[i] += K*Weight[j];
#pragma omp atomic
S[j] -= K*Weight[i];
}
I am new to C++ and programming so I think I am making inefficient codes.
I was wondering whether there is any way I can speed up the matrix calculation process.
For example, this is the sample code I write which finds the maximum differences(in absolute value) between 3d array 'V' and 'Vnew'.
First, I take subtraction.
And then, I put the value of tempdiff[0][0][0] to 'dif'
Then, I compare 'dif' and tempdiff[i][j][k] and replace if the latter is larger than the former.
This is just a part of my code and there are lots of matrix calculations inside so that I have too many 'for' statements.
So I was wondering whether there is any way I could avoid using 'for' in the matrix calculations.
Thanks in advance.
for (int i = 0; i < Na; i++) {
for (int j = 0; j < Nd; j++) {
for (int k = 0; k < Ny; k++) {
tempdiff[i][j][k] = abs(V[i][j][k] - Vnew[i][j][k]);
}
}
}
dif = tempdiff[0][0][0];
for (int i = 0; i < Na; i++) {
for (int j = 0; j < Nd; j++) {
for (int k = 0; k < Ny; k++) {
if (tempdiff[i][j][k] > dif) {
dif = tempdiff[i][j][k];
}
else {
dif = dif;
}
}
}
}
There's not much you can do with the for loops, as the maximum difference can locate at all possible places. You have already succeeded in iterating the array in the correct, linear, order.
Compilers are generally quite efficient in optimising, but they apparently fail to flatten a contiguous array, such as float V[Na][Nd][Ny];. After you flatten it manually to float V[Na*Nd*Ny], at least clang can auto-vectorise and produce SIMD code for x64 and arm.
A further optimisation is to avoid making this in two steps, as the total memory throughput is exactly doubled with the temporary array compared to a one-pass solution.
I was assuming your matrices are of type float -- if you can select int, gcc can auto-vectorise this as well (relates to NaN handling); furthermore int16_t or int8_t types are even quicker to evaluate, as more operations can be packed to a single SIMD instruction.
I'm writing a program that should run both in serial and parallel versions. Once I get it to actually do what it is supposed to do I started trying to parallelize it with OpenMP (compulsory).
The thing is I can't find documentation or references on when to use what #pragma. So I am trying my best at guessing and testing. But testing is not going fine with nested loops.
How would you parallelize a series of nested loops like these:
for(int i = 0; i < 3; ++i){
for(int j = 0; j < HEIGHT; ++j){
for(int k = 0; k < WIDTH; ++k){
switch(i){
case 0:
matrix[j][k].a = matrix[j][k] * someValue1;
break;
case 1:
matrix[j][k].b = matrix[j][k] * someValue2;
break;
case 2:
matrix[j][k].c = matrix[j][k] * someValue3;
break;
}
}
}
}
HEIGHT and WIDTH are usually the same size in the tests I have to run. Some test examples are 32x32 and 4096x4096.
matrix is an array of custom structs with attributes a, b and c
someValue is a double
I know that OpenMP is not always good for nested loops but any help is welcome.
[UPDATE]:
So far I've tried unrolling the loops. It boosts performance but am I adding unnecesary overhead here? Am I reusing threads? I tried getting the id of the threads used in each for but didn't get it right.
#pragma omp parallel
{
#pragma omp for collapse(2)
for (int j = 0; j < HEIGHT; ++j) {
for (int k = 0; k < WIDTH; ++k) {
//my previous code here
}
}
#pragma omp for collapse(2)
for (int j = 0; j < HEIGHT; ++j) {
for (int k = 0; k < WIDTH; ++k) {
//my previous code here
}
}
#pragma omp for collapse(2)
for (int j = 0; j < HEIGHT; ++j) {
for (int k = 0; k < WIDTH; ++k) {
//my previous code here
}
}
}
[UPDATE 2]
Apart from unrolling the loop I have tried parallelizing the outer loop (worst performance boost than unrolling) and collapsing the two inner loops (more or less same performance boost as unrolling). This are the times I am getting.
Serial: ~130 milliseconds
Loop unrolling: ~49 ms
Collapsing two innermost loops: ~55 ms
Parallel outermost loop: ~83 ms
What do you think is the safest option? I mean, which should be generally the best for most systems, not only my computer?
The problem with OpenMP is that it's very high-level, meaning that you can't access low-level functionality, such as spawning the thread, and then reusing it. So let me make it clear what you can and what you can't do:
Assuming you don't need any mutex to protect against race conditions, here are your options:
You parallelize your outer-most loop, and that will use 3 threads, and that's the most peaceful solution you're gonna have
You parallelize the first inner loop with, and then you'll have a performance boost only if the overhead of spawning a new thread for every WIDTH element is much smaller the efforts required to perform the most inner loop.
Parallelizing the most inner loop, but this is the worst solution in the world, because you'll respawn the threads 3*HEIGHT times. Never do that!
Not use OpenMP, and use something low-level, such as std::thread, where you can create your own Thread Pool, and push all the operations you want to do in a queue.
Hope this helps to put things in perspective.
Here's another option, one which recognises that distributing the iterations of the outermost loop when there are only 3 of them might lead to very poor load balancing,
i=0
#pragma omp parallel for
for(int j = 0; j < HEIGHT; ++j){
for(int k = 0; k < WIDTH; ++k){
...
}
i=1
#pragma omp parallel for
for(int j = 0; j < HEIGHT; ++j){
for(int k = 0; k < WIDTH; ++k){
...
}
i=2
#pragma omp parallel for
for(int j = 0; j < HEIGHT; ++j){
for(int k = 0; k < WIDTH; ++k){
...
}
Warning -- check the syntax yourself, this is no more than a sketch of manual loop unrolling.
Try combining this and collapsing the j and k loops.
Oh, and don't complain about code duplication, you've told us you're being scored partly on performance improvements.
You probably want to parallelize this example for simd so the compiler can vectorize, collapse the loops because you use j and k only in the expression matrix[j][k], and because there are no dependencies on any other element of the matrix. If nothing modifies somevalue1, etc., they should be uniform. Time your loop to be sure those really do improve your speed.
in my previous question
Shared vectors in OpenMP
it was stated that one can let diferent threads read and write on a shared vector as long as
the different threads access different elements of the vector.
What if different threads have to read all the (so sometimes the same) elements of a vector, like in the following case ?
#include <vector>
int main(){
vector<double> numbers;
vector<double> results(10);
double x;
//write 25 values in vector numbers
for (int i =0; i<25; i++){
numbers.push_back(cos(i));
}
#pragma omp parallel for default(none) \
shared(numbers, results) \
private(x)
for (int j = 0; j < 10; j++){
for(int k = 0; k < 25; k++){
x += 2 * numbers[j] * numbers[k] + 5 * numbers[j * k / 25];
}
results[j] = x;
}
return 0;
}
Will this parallelization be slow because only one thread at a time can read any element of the vector or is this not the case? Could I resolve the problem with the clause firstprivate(numbers)?
Would it make sense to create an array of vectors so every thread gets his own vector ?
For instance:
vector<double> numbersx[**-number of threads-**];
Reading elements of the same vector from multiple threads is not a problem. There is no synchronization in your code, so they will be accessed concurrently.
With the size of vectors that you are working with, you will not have any cache problems either, although for bigger vectors you may get some slow-downs due to the cache access pattern. In that case, separate copies of the numbers data would improve performance.
better approach:
#include <vector>
int main(){
vector<double> numbers;
vector<double> results(10);
//write 25 values in vector numbers
for (int i =0; i<25; i++){
numbers.push_back(cos(i));
}
#pragma omp parallel for
for (int j = 0; j < 10; j++){
double x = 0; // make x local var
for(int k = 0; k < 25; k++){
x += 2 * numbers[j] * numbers[k] + 5 * numbers[j * k / 25];
}
results[j] = x; // no race here
}
return 0;
}
it will be slow kinda due to fact that there isn't much work to share
I wrote a programm that multiplies a vector by a matrix. The matrix has periodically repeated cells, so I use a temporary variable to sum vector elements before multiplication. The period is the same for adjacent rows. I create a separate temp variable for each thread. sizeof(InnerVector) == 400 and I don't want to allocate memory for it on every iterration (= 600 times).
Code looks something like this:
tempsSize = omp_get_max_threads();
InnerVector temps = new InnerVector[tempsSize];
for(int k = 0; k < tempsSize; k++)
InnerVector_init(temps[k]);
for(int jmin = 1, jmax = 2; jmax < matrixSize/2; jmin *= 2, jmax *= 2)
{
int period = getPeriod(jmax);
#pragma omp parallel
{
int threadNum = omp_get_thread_num();
// printf("\n threadNum = %i", threadNum);
#pragma omp for
for(int j = jmin; j < jmax; j++)
{
InnerVector_reset(temps[threadNum]);
for(int i = jmin; i < jmax; i++)
{
InnerMatrix cell = getCell(i, j);
if(temps[threadNum].IsZero)
for(int k = j; k < matrixSize; k += period)
InnerVector_add(temps[threadNum], temps[threadNum], v[k]);
InnerVector_add_mul(v_res[i], cell, temps[threadNum]);
}
}
}
}
The code looks to be correct but I get wrong result. In fact, I get different results for different runs... sometimes result is correct.
When I compile in debug mode the result is always correct.
When I uncomment the row with "printf" the result is always correct.
p.s. I use Visual Studio 2010.
I suspect there might be a data race in
InnerVector_add_mul(v_res[i], cell, temps[threadNum]);
Since v_res appears to be a resulting vector, and i changes from jmin to jmax in each iteration of the parallelized loop, it can happen that multiple threads write to v_res[i] for the same value of i, with unpredictable result.