How to use `boost::thread_specific_ptr` with `for_each` - c++

Question

In the code below, Bar is supposed to model a thread-unsafe object that is moderately expensive to create. Foo contains a Bar and is multi-threaded, so it uses a thread_specific_ptr<Bar> to make a per-thread Bar that can be re-used across multiple calls to loop for the same Foo (therefore amortizing the cost of creating a Bar for each thread). Foo always creates a Bar with the same num, so the sanity check is supposed to always pass, yet it fails.
The reason for this is (I think) explained in the requirement for the thread_specific_ptr destructor:
All the thread specific instances associated to this thread_specific_ptr (except maybe the one associated to this thread) must be null.
So the problem is caused by a combination of three things:
Bar objects created in worker threads are not cleaned up when Foos thread_specific_ptr is cleaned up, and are therefore persisted across iterations of the loop in main (essentially, a memory leak)
The C++ runtime is re-using threads in for_each between iterations of the loop in main
The C++ runtime is re-allocating each Foo in the main loop to the same memory address
The way that thread_specific_ptrs are indexed (by the thread_specific_ptr's memory address and the thread ID) results in old Bars being accidentally reused. I understand the problem; what I don't understand is what to do about it. Note the remark from the docs:
The requirement is due to the fact that in order to delete all these instances, the implementation should be forced to maintain a list of all the threads having an associated specific ptr, which is against the goal of thread specific data.
I'd like to avoid this complexity as well.
How can I use for_each for simple thread management, but also avoid the memory leak? Solution requirements:
It should only create one Bar per thread per Foo (i.e., don't create a new Bar inside the for_each)
Assume Bar is not thread-safe.
If possible, use for_each to make the parallel loop as simple as possible
The loop should actually run in parallel (i.e., no mutex around a single Bar)
Bar objects created by loop should be available for use until the Foo object that created them is destructed, at which point all Bar objects should also be destructed.
The following code compiles and should exit with return code 1 with high probability on a machine with sufficient cores.
#include <boost/thread/tss.hpp>
#include <execution>
#include <iostream>
#include <vector>
using namespace std;
class Bar {
public:
// models a thread-unsafe object
explicit Bar(int i) : num(i) { }
int num;
};
class Foo {
public:
explicit Foo(int i) : num(i) { }
void loop() {
vector<int> idxs(32);
iota(begin(idxs), end(idxs), 0);
for_each(__pstl::execution::par, begin(idxs), end(idxs), [&](int) {
if (ptr.get() == nullptr) {
// no `Bar` exists for this thread yet, so create one
Bar *tmp = new Bar(num);
ptr.reset(tmp);
}
// Get the thread-local Bar
Bar &b = *ptr;
// Sanity check: we ALWAYS create a `Bar` with the same num as `Foo`;
// see the `if` block above.
// Therefore, this condition shouldn't ever be true (but it is!)
if (b.num != num) {
cout << "NOT THREAD SAFE: Foo index is " << num << ", but Bar index is " << b.num << endl;
exit(1);
}
});
}
boost::thread_specific_ptr<Bar> ptr;
int num;
};
int main() {
for(int i = 0; i < 100; i++) {
Foo f(i);
f.loop();
}
return 0;
}

Answer 1

According to the documentation
~thread_specific_ptr();
Requires:
All the thread specific instances associated to this thread_specific_ptr
(except maybe the one associated to this thread) must be null.
This means you are not allowed to destroy Foo until all of its Bar have been destroyed. This is a problem because execution_policy::par does not have to operate on a fresh thread pool, nor does it have to terminate the threads once the for_each() is done.
This gives us enough to answer the question as asked: You can only use thread_specific_ptr alongside execution::par to share data between various iterations on the same thread if:
The thread_specific_ptr is never destroyed. This is required because there is no way to know whether a given iteration of the for_each will be the last one for its assigned thread, and that thread might never get scheduled again.
You are comfortable leaking one instance of the pointed object per thread until the end of the program.
What's going on in your code
We are already in Undefined Behavior land, but the behavior you are seeing can still be explained a bit further. Considering that:
Boost.Thread uses the address of the thread_specific_ptr instance as key of the thread specific pointers. This avoids to create/destroy a key which will need a lock to protect from race conditions. This has a little performance liability, as the access must be done using an associative container.
... and that all 100 instances of Foo will most likely be at the same place in memory, you end up seeing instances of Bar from the previous Foo when the worker threads are recycled, leading to your (innacurate, see below) check to hit.
Solution: What I think you should do
I would suggest you just drop thread_specific_ptr altogether and manually manage the pool of per-thread/per-Foo Bar instances with an associative container, this makes managing the lifetime of the Bar objects a lot more straightforward:
class per_thread_bar_pool {
std::map<std::thread::id, Bar> bars_;
// alternatively:
// std::map<std::thread::id, std::unique_ptr<Bar>> bars_;
std::mutex mtx_;
public:
Bar& get(int num) {
auto tid = std::this_thread::get_id();
std::unique_lock l{mtx_};
auto found = bars_.find(tid);
if(found == bars_.end()) {
l.unlock(); // Let other threads access the map while `Bar` is being built.
Bar new_bar(num);
// auto new_bar = std::make_unique<Bar>(num);
l.lock();
assert(bars_.find(tid) == bars_.end());
found = bars_.emplace(tid, std::move(new_bar)).first;
}
return found->second;
// return *found->second;
}
};
void loop() {
per_thread_bar_pool bars;
vector<int> idxs(32);
iota(begin(idxs), end(idxs), 0);
for_each(__pstl::execution::par, begin(idxs), end(idxs), [&](int) {
Bar& current_bar = bars.get(num);
// ...
}
}
thread_specific_ptr already uses std::map<> under the hood (it maintains one per thread). So introducing one here is not that big of a deal.
We do introduce a mutex, but it only comes into play for a simple lookup/insertion into a map, and since constructing Bar is supposed to be so expensive, it will most likely have very little impact. It also has the benefit that multiple instances of Foo do not interact with each other anymore, so you avoid surprising bugs that could occur if you ever end up calling foo::loop() from multiple threads.
N.B.: if (b.num != num) { is not a valid test since all instances of Bar from a given Foo share the same num. That should only cause false-negatives though.
Solution: Making your code work (almost)
All this being said, if you are absolutely gung-ho about using thread_specific_pointer and execution::par at the same time you'll have to do the following:
void loop() {
static boost::thread_specific_ptr<Bar> ptr; // lives till the end of the program
vector<int> idxs(32);
iota(begin(idxs), end(idxs), 0);
for_each(__pstl::execution::par, begin(idxs), end(idxs), [&](int) {
if (ptr.get() == nullptr || ptr->num != num) {
// no `Bar` exists for this thread yet, or it's from a previous run
Bar *tmp = new Bar(num);
ptr.reset(tmp);
}
// Get the thread-local Bar
Bar &b = *ptr;
});
However, this will leak up to 1 Bar per thread, as cleanup only ever happens when we try to reuse a Bar from a previous run. There are no ways around this.