OpenMP: why this application scale sometimes? - c++

Question

I'm trying to speedup the OpenCV SIFT algorithm with OpenMP on a Intel® Core™ i5-6500 CPU # 3.20GHz × 4. You can find the code in sift.cpp.
The most expensive part is the descriptor computaton, in particular:
static void calcDescriptors(const std::vector<Mat>& gpyr, const std::vector<KeyPoint>& keypoints,
Mat& descriptors, int nOctaveLayers, int firstOctave )
{
int d = SIFT_DESCR_WIDTH, n = SIFT_DESCR_HIST_BINS;
for( size_t i = 0; i < keypoints.size(); i++ )
{
KeyPoint kpt = keypoints[i];
int octave, layer;
float scale;
unpackOctave(kpt, octave, layer, scale);
CV_Assert(octave >= firstOctave && layer <= nOctaveLayers+2);
float size=kpt.size*scale;
Point2f ptf(kpt.pt.x*scale, kpt.pt.y*scale);
const Mat& img = gpyr[(octave - firstOctave)*(nOctaveLayers + 3) + layer];
float angle = 360.f - kpt.angle;
if(std::abs(angle - 360.f) < FLT_EPSILON)
angle = 0.f;
calcSIFTDescriptor(img, ptf, angle, size*0.5f, d, n, descriptors.ptr<float>((int)i));
}
}
The serial version of this function take 52 ms on average.
This for has an high granulatiy: it's executed 604 times (which is keypoints.size() ). The main time consuming component inside the for is calcSIFTDescriptor which takes most of the cycle time computation and it takes on 105 us on average, but it often happens that it can take 200usor 50us.
However, we are incredibly lucky: there is no dependency between each for cycle, so we can just add:
#pragma omp parallel for schedule(dynamic,8)
and obtain an initial speedup. The dynamic option is introduced since it seems it give little better performances than static (don't know why).
The problem is that it's really unstable and doesn't scale. This is the time needed to compute the function in parallel mode:
25ms 43ms 32ms 15ms 27ms 53ms 21ms 24ms
As you can see only once the optimal speedup in a quad-core system is reached (15ms). Most of the times we reach half of the optimal speedup: 25ms in a quadcore system is only half of the theoretical optimal speedup.
Why this happens? How can we improve this?
UPDATE:
As suggested in the comments, I tried to use a bigger dataset. Using an huge image, the serial version takes 13574ms to compute the descriptors, while the parallel version 3704ms with the same quad-core of before. Much better: even if it's not the best theoretical result, it actually scales well. But actually the problem remain, since the previous results are obtained from a typical image.
UPDATE 1: as suggested by the comment, I tried to benchmark without any interval between the execution in an "hot mode" (see comment for more details). Better results are achieved more frequently, but still there is a lot of variations. This are the times (in ms) for 100 runs in hot mode:
43 42 14 26 14 43 13 26 15 51 15 20 14 40 34 15 15 31 15 22 14 21 17 15 14 27 14 16 14 22 14 22 15 15 14 43 16 16 15 28 14 24 14 36 15 32 13 21 14 23 14 15 13 26 15 35 13 32 14 36 14 34 15 40 28 14 14 15 15 35 15 22 14 17 15 23 14 24 17 16 14 35 14 29 14 25 14 32 14 28 14 34 14 30 22 14 15 24 14 31
You can see a lot of good results (14ms, 15ms) but a lot of horrible results also (>40ms). The average is 22ms Notice that there is no at most 4ms of variation in the sequential mode:
52 54 52 52 51 52 52 53 53 52 53 51 52 53 53 54 53 53 53 53 54 53 54 54 53 53 53 52 53 52 51 52 52 53 54 54 54 55 55 55 54 54 54 53 53 52 52 52 51 52 54 53 54 54 54 55 54 54 52 55 52 52 52 51 52 51 52 52 51 51 52 52 53 53 53 53 55 54 55 54 54 54 55 52 52 52 51 51 52 51 51 51 52 53 53 54 53 54 53 55
UPDATE 2:
I've noticed that each CPU utilization during the "hot mode" benchmarking is quite random and also it never reach more than 80%, as shown in the image below:
Instead the image below shows the CPUs utilization while I compile OpenCV through make -j4. As you can see it more stable and used almost 100% of it:
I think that this is variation in the first image are normal since we execute the same short program many times, which is more unstable than one big program. What I don't understand is why we never reach more than 80% of CPU utilization.

Answer 1

I strongly suggest you to use some performance tools such as Paraver (http://www.bsc.es/paraver), TAU (http://www.cs.uoregon.edu/research/tau/home.php) Vampir (https://tu-dresden.de/die_tu_dresden/zentrale_einrichtungen/zih/forschung/projekte/vampir) or even Intel's Vtune (https://software.intel.com/en-us/intel-vtune-amplifier-xe).
These tools will help you understand where threads spends their cycles. With them, you can find whether the application is unbalanced (either by IPC or instructions), whether there is any limitation due to memory bandwidth or false sharing problems, among many other issues.