Why is the C++ thread/future overhead so big - c++

Question

I have a worker routine (code below), which is running slower when I run it in a separate thread. As far as I can tell, the worker code and data is completely independent of other threads. All the worker does is to append nodes to a tree. The goal is having multiple workers growing trees in parallel.
Can someone help me understand why there is (significant) overhead when running the worker in a separate thread?
Edit:
Initially I was testing WorkerFuture twice, I corrected that and I now get the same (better) performance in the no thread and defer async cases, and considerable overhead when an extra thread is involved.
The command to compile (linux): g++ -std=c++11 main.cpp -o main -O3 -pthread
Here is the output (time in milliseconds):
Thread : 4000001 size in 1861 ms
Async : 4000001 size in 1836 ms
Defer async: 4000001 size in 1423 ms
No thread : 4000001 size in 1455 ms
Code:
#include <iostream>
#include <vector>
#include <random>
#include <chrono>
#include <thread>
#include <future>
struct Data
{
int data;
};
struct Tree
{
Data data;
long long total;
std::vector<Tree *> children;
long long Size()
{
long long size = 1;
for (auto c : children)
size += c->Size();
return size;
}
~Tree()
{
for (auto c : children)
delete c;
}
};
int
GetRandom(long long size)
{
static long long counter = 0;
return counter++ % size;
}
void
Worker_(Tree *root)
{
std::vector<Tree *> nodes = {root};
Tree *it = root;
while (!it->children.empty())
{
it = it->children[GetRandom(it->children.size())];
nodes.push_back(it);
}
for (int i = 0; i < 100; ++i)
nodes.back()->children.push_back(new Tree{{10}, 1, {}});
for (auto t : nodes)
++t->total;
}
long long
Worker(long long iterations)
{
Tree root = {};
for (long long i = 0; i < iterations; ++i)
Worker_(&root);
return root.Size();
}
void ThreadFn(long long iterations, long long &result)
{
result = Worker(iterations);
}
long long
WorkerThread(long long iterations)
{
long long result = 0;
std::thread t(ThreadFn, iterations, std::ref(result));
t.join();
return result;
}
long long
WorkerFuture(long long iterations)
{
std::future<long long> f = std::async(std::launch::async, [iterations] {
return Worker(iterations);
});
return f.get();
}
long long
WorkerFutureSameThread(long long iterations)
{
std::future<long long> f = std::async(std::launch::deferred, [iterations] {
return Worker(iterations);
});
return f.get();
}
int main()
{
long long iterations = 40000;
auto t1 = std::chrono::high_resolution_clock::now();
auto total = WorkerThread(iterations);
auto t2 = std::chrono::high_resolution_clock::now();
std::cout << "Thread : " << total << " size in " << std::chrono::duration_cast<std::chrono::milliseconds>(t2 - t1).count() << " ms\n";
t1 = std::chrono::high_resolution_clock::now();
total = WorkerFuture(iterations);
t2 = std::chrono::high_resolution_clock::now();
std::cout << "Async : " << total << " size in " << std::chrono::duration_cast<std::chrono::milliseconds>(t2 - t1).count() << " ms\n";
t1 = std::chrono::high_resolution_clock::now();
total = WorkerFutureSameThread(iterations);
t2 = std::chrono::high_resolution_clock::now();
std::cout << "Defer async: " << total << " size in " << std::chrono::duration_cast<std::chrono::milliseconds>(t2 - t1).count() << " ms\n";
t1 = std::chrono::high_resolution_clock::now();
total = Worker(iterations);
t2 = std::chrono::high_resolution_clock::now();
std::cout << "No thread : " << total << " size in " << std::chrono::duration_cast<std::chrono::milliseconds>(t2 - t1).count() << " ms\n";
}

Answer 1

It seems that the problem is caused by dynamic memory management. When multiple threads are involved (even if the main thread does nothing), C++ runtime must synchronize access to dynamic memory (heap), which generates some overhead. I did some experiments with GCC and the solution of your problem is to use some scalable memory allocator library. For instance, when I used tbbmalloc, e.g.,
export LD_LIBRARY_PATH=$TBB_ROOT/lib/intel64/gcc4.7:$LD_LIBRARY_PATH
export LD_PRELOAD=libtbbmalloc_proxy.so.2
the whole problem disappeared.

Answer 2

The reason is simple. You do not do anything in parallel manner.
When extra thread is doing something main thread does nothing (waits for thread job to complete).
In case of thread you have extra thing to do (handle thread and synchronization) so you have a trade off.
To see any gain you have to do at least two things at the same time.