No speedup with many slow cores? - c++

Question

my C++ program calculates lots of features for 3D points. This takes a while, so I do the calculations in different threads. At the end all features (of all threads) have to be stored in one file.
On my local machine the multithread-implementation was a great success (4 threads -> runtime reduced by 73%).
However, on my server (40 slow 2GHz cores, 80 threads) it's even slower than on my local 4 threads.
Runtimes:
Local 1 Core: 7.5 minutes
Local 4 Core: 2 minutes
Server 80 Threads: 3.1 minutes (slower than my local 4 cores)
Server 20 Threads: 6.2 minutes
Server 4 Threads: 4.75 minutes (Interesting - it's faster with less threads)
My code is appended.
I tried:
Making the critical part smaller/faster by building a string within each thread and only writing it to file within critical part: No improvement
Only writing the results to disk at the very end: No improvement (thought it could be Disk I/O)
Using schedule(guided) for OpenMP-Loop for bigger chunk sizes: No improvement
...
std::vector<double*> points;
for(unsigned int j = 0; j < xyz.size(); j++) {
points.push_back(new double[3]{xyz[j][0], xyz[j][1], xyz[j][2]});
}
ofstream featuresOut;
featuresOut.open(...);
...
KDtree t(&points[0], points.size()); // Build tree
float batchSize = ((float)points.size())/jobs;
unsigned int first = job * batchSize;
unsigned int last = ((job+1) * batchSize) - 1;
// Generate features
#ifdef _OPENMP
omp_set_num_threads(OPENMP_NUM_THREADS);
#pragma omp parallel for schedule(dynamic)
#endif
for(unsigned int r = first; r <= last; r++) {
if (r % 100000 == 0) {
cout << "Calculating features for point nr. " << r << endl;
}
#ifdef _OPENMP
int thread_num = omp_get_thread_num();
#else
int thread_num = 0;
#endif
double features[FEATURE_VECTOR_SIZE];
if (!(ignoreClass0 && type.valid() && type[r]==0)) {
double reference[3] = {xyz[r][0], xyz[r][1], xyz[r][2]};
vector<Point> neighbors = t.kNearestNeighbors(reference, kMax, thread_num); // here we have a ordered set of kMax neighbors (at maximum - could be less for small scans)
//std::vector<double> features = generateNeighborhoodFeatures(t, reference, kMin, kMax, kDelta, cylRadius);
unsigned int kOpt = determineKOpt(t, reference, neighbors, kMin, kMax, kDelta);
generateNeighborhoodFeatures(features, t, reference, neighbors, kOpt, cylRadius, false, thread_num);
#pragma omp critical
{
featuresOut << xyz[r][0] << "," << xyz[r][1] << "," << xyz[r][2] << ",";
featuresOut << kOpt;
for(unsigned int j = 0; j < FEATURE_VECTOR_SIZE; j++) {
if (isfinite(features[j])) {
featuresOut << "," << features[j];
}
else {
cout << "Attention! Feature " << j << " (+5) was not finite. Replaced it with very big number" << endl;
featuresOut << "," << DBL_MAX;
}
}
featuresOut << ",";
if (type.valid()) {
featuresOut << type[r];
} else {
featuresOut << 0;
}
featuresOut << endl;
}
}
}
Only writing to disk at the very end (aggregated results of threads) does not solve the problem (see answer of #J.Svejda). Also keeping one KDtree for each thread results in no speedup.
Thanks for your help.

Answer 1

I believe it's because of your critical section. The time to write to disk usually takes much more than to compute something on the CPU. I don't know what complexity the work you do on the KDTree has, but writing to disk can take milliseconds, whereas instructions on CPU are in the order of nanoseconds. Though the data probably does not get flushed to featuresOut until you send an endl, it would explain your poor scaling. The critical section just takes too long and threads have to wait for each other.
Maybe increase the number of work per thread, say a thread does 5% of the points.
And then it would output to the file aggregated data from more points. See if it improves the speedup.