C++ concurrency: Variable visibility outside of mutexes [duplicate] - c++

Question

This question already exists:
C++ concurrency: conditional atomic operations?
Closed 6 years ago.
I'm having some trouble understanding when variables are forced to be written to memory, even outside of mutex blocks. I apologize for the convoluted code below, because I have stripped away logic that deals with whether reader decides if some data is stale. The important thing to note is that 99.9% of the time, readers will take the fast path and synchronization must be very fast, which is why I use an atomic int32 to communicate both staleness and whether the slow path is now necessary.
I have the following setup, which I am "fairly" certain is race-free:
#define NUM_READERS 10
BigObject mSharedObject;
std::atomic_int32_t mStamp = 1;
std::mutex mMutex;
std::condition_variable mCondition;
int32_t mWaitingReaders = 0;
void reader() {
for (;;) { // thread loop
for (;;) { // spin until stamp is acceptible
int32_t stamp = mStamp.load();
if (stamp > 0) { // fast path
if (stampIsAcceptible(stamp) &&
mStamp.compare_exchange_weak(stamp, stamp + 1)) {
break;
}
} else { // slow path
// tell the loader (writer) that we're halted
std::unique_lock<mutex> lk(mMutex);
mWaitingReaders++;
mCondition.notify_all();
while (mWaitingReaders != 0) {
mCondition.wait(lk);
} // ###
lk.unlock();
// *** THIS IS WHERE loader's CHANGES TO mSharedObject
// *** MUST BE VISIBLE TO THIS THREAD!
}
}
// stamp acceptible; mSharedObject guaranteed not written to
mSharedObject.accessAndDoFunStuff();
mStamp.fetch_sub(1); // part of hidden staleness logic
}
}
void loader() {
for (;;) { // thread loop
// spin until we somehow decide we want to change mSharedObject!
while (meIsHappySleeping()) {}
// we want to modify mSharedObject, so set mStamp to 0 and wait
// for readers to see this and report that they are now waiting
int32_t oldStamp = mStamp.exchange(0);
unique_lock<mutex> lk(mMutex);
while (mWaitingReaders != NUM_READERS) {
mCondition.wait(lk);
}
// all readers are waiting. start writing to mSharedObject
mSharedObject.loadFromFile("example.foo");
mStamp.store(oldStamp);
mWaitingReaders = 0; // report completion
lk.unlock();
mCondition.notify_all();
// *** NOW loader's CHANGES TO mSharedObject
// *** MUST BE VISIBLE TO THE READER THREADS!
}
}
void setup() {
for (int i = 0; i < NUM_READERS; i++) {
std::thread t(reader); t.detach();
}
std::thead t(loader); t.detach();
}
The parts marked in stars *** are what concerns me. This is because while my code excludes races (as far as I can see), mSharedObject is only protected by a mutex while being written to by loader(). Because reader() needs to be extremely fast (as noted above), I do not want its read-only accesses to mSharedObject to have to be protected by a mutex.
One "guaranteed" solution is to introduce a thread-local variable const BigObject *latestObject at line ###, which is set to &mSharedObject and then use that for access. But is this bad practice? And is it really necessary? Will the atomic operations / mutex release operations guarantee that readers see the changes?
Thanks!

Answer 1

Lock-free code, and even locking code using just atomics is far from simple. The first thing to do would be to just add a mutex and profile how much of the performance is actually lost in the synchronisation. Note that current implementations of mutex may just do a quick spin-lock, which is roughly an atomic operation when uncontended.
If you want to attempt lock-free programming you will need to look into the memory ordering arguments to atomic operations. The writer will need to ..._release to synchronise with a reader doing ..._acquire (or use sequential consistency in both sides). Otherwise the reads/writes to any other variables may not be visible.