Multiple-readers, single-writer locks in Boost - c++

I'm trying to implement the following code in a multithreading scenario:
Get shared access to mutex
Read data structure
If necessary:
Get exclusive access to mutex
Update data structure
Release exclusive lock
Release shared lock
Boost threads has a shared_mutex class which was designed for a multiple-readers, single-writer model. There are several stackoverflow questions regarding this class. However, I'm not sure it fits the scenario above where any reader may become a writer. The documentation states:
The UpgradeLockable concept is a
refinement of the SharedLockable
concept that allows for upgradable
ownership as well as shared ownership
and exclusive ownership. This is an
extension to the multiple-reader /
single-write model provided by the
SharedLockable concept: a single
thread may have upgradable ownership
at the same time as others have shared
ownership.
From the word "single" I suspect that only one thread may hold an upgradable lock. The others only hold a shared lock which can't be upgraded to an exclusive lock.
Do you know if boost::shared_lock is useful in this situation (any reader may become a writer), or if there's any other way to achieve this?

Yes, you can do exactly what you want as shown in the accepted answer here. A call to upgrade to exclusive access will block until all readers are done.
boost::shared_mutex _access;
void reader()
{
// get shared access
boost::shared_lock<boost::shared_mutex> lock(_access);
// now we have shared access
}
void writer()
{
// get upgradable access
boost::upgrade_lock<boost::shared_mutex> lock(_access);
// get exclusive access
boost::upgrade_to_unique_lock<boost::shared_mutex> uniqueLock(lock);
// now we have exclusive access
}

boost::shared_lock doesn't help in this situation (multiple readers that can become writers), since only a single thread may own an upgradable lock. This is both implied by the quote from the documentation in the question, and by looking at the code (thread\win32\shared_mutex.hpp). If a thread tries to acquire an upgradable lock while another thread holds one, it will wait for the other thread.
I settled on using a regular lock for all reader/writers, which is OK in my case since the critical section is short.

You know LightweightLock
or same at LightweightLock_zip
does exactly what you want.
I have used it a long time.
[EDIT]
here's the source:
/////////////////////////////////////////////////////////////////////////////
//
// Copyright (C) 1995-2002 Brad Wilson
//
// This material is provided "as is", with absolutely no warranty
// expressed or implied. Any use is at your own risk. Permission to
// use or copy this software for any purpose is hereby granted without
// fee, provided the above notices are retained on all copies.
// Permission to modify the code and to distribute modified code is
// granted, provided the above notices are retained, and a notice that
// the code was modified is included with the above copyright notice.
//
/////////////////////////////////////////////////////////////////////////////
//
// This lightweight lock class was adapted from samples and ideas that
// were put across the ATL mailing list. It is a non-starving, kernel-
// free lock that does not order writer requests. It is optimized for
// use with resources that can take multiple simultaneous reads,
// particularly when writing is only an occasional task.
//
// Multiple readers may acquire the lock without any interference with
// one another. As soon as a writer requests the lock, additional
// readers will spin. When the pre-writer readers have all given up
// control of the lock, the writer will obtain it. After the writer
// has rescinded control, the additional readers will gain access
// to the locked resource.
//
// This class is very lightweight. It does not use any kernel objects.
// It is designed for rapid access to resources without requiring
// code to undergo process and ring changes. Because the "spin"
// method for this lock is "Sleep(0)", it is a good idea to keep
// the lock only long enough for short operations; otherwise, CPU
// will be wasted spinning for the lock. You can change the spin
// mechanism by #define'ing __LW_LOCK_SPIN before including this
// header file.
//
// VERY VERY IMPORTANT: If you have a lock open with read access and
// attempt to get write access as well, you will deadlock! Always
// rescind your read access before requesting write access (and,
// of course, don't rely on any read information across this).
//
// This lock works in a single process only. It cannot be used, as is,
// for cross-process synchronization. To do that, you should convert
// this lock to using a semaphore and mutex, or use shared memory to
// avoid kernel objects.
//
// POTENTIAL FUTURE UPGRADES:
//
// You may consider writing a completely different "debug" version of
// this class that sacrifices performance for safety, by catching
// potential deadlock situations, potential "unlock from the wrong
// thread" situations, etc. Also, of course, it's virtually mandatory
// that you should consider testing on an SMP box.
//
///////////////////////////////////////////////////////////////////////////
#pragma once
#ifndef _INC_CRTDBG
#include
#endif
#ifndef _WINDOWS_
#include
#endif
#ifndef __LW_LOCK_SPIN
#define __LW_LOCK_SPIN Sleep(0)
#endif
class LightweightLock
{
// Interface
public:
// Constructor
LightweightLock()
{
m_ReaderCount = 0;
m_WriterCount = 0;
}
// Destructor
~LightweightLock()
{
_ASSERTE( m_ReaderCount == 0 );
_ASSERTE( m_WriterCount == 0 );
}
// Reader lock acquisition and release
void LockForReading()
{
while( 1 )
{
// If there's a writer already, spin without unnecessarily
// interlocking the CPUs
if( m_WriterCount != 0 )
{
__LW_LOCK_SPIN;
continue;
}
// Add to the readers list
InterlockedIncrement((long*) &m_ReaderCount );
// Check for writers again (we may have been pre-empted). If
// there are no writers writing or waiting, then we're done.
if( m_WriterCount == 0 )
break;
// Remove from the readers list, spin, try again
InterlockedDecrement((long*) &m_ReaderCount );
__LW_LOCK_SPIN;
}
}
void UnlockForReading()
{
InterlockedDecrement((long*) &m_ReaderCount );
}
// Writer lock acquisition and release
void LockForWriting()
{
// See if we can become the writer (expensive, because it inter-
// locks the CPUs, so writing should be an infrequent process)
while( InterlockedExchange((long*) &m_WriterCount, 1 ) == 1 )
{
__LW_LOCK_SPIN;
}
// Now we're the writer, but there may be outstanding readers.
// Spin until there aren't any more; new readers will wait now
// that we're the writer.
while( m_ReaderCount != 0 )
{
__LW_LOCK_SPIN;
}
}
void UnlockForWriting()
{
m_WriterCount = 0;
}
long GetReaderCount() { return m_ReaderCount; };
long GetWriterConut() { return m_WriterCount; };
// Implementation
private:
long volatile m_ReaderCount;
long volatile m_WriterCount;
};

Related

Swapping mutex locks

I'm having trouble with properly "swapping" locks. Consider this situation:
bool HidDevice::wait(const std::function<bool(const Info&)>& predicate)
{
/* A method scoped lock. */
std::unique_lock waitLock(this->waitMutex, std::defer_lock);
/* A scoped, general access, lock. */
{
std::lock_guard lock(this->mutex);
bool exitEarly = false;
/* do some checks... */
if (exitEarly)
return false;
/* Only one thread at a time can execute this method, however
other threads can execute other methods or abort this one. Thus,
general access mutex "this->mutex" should be unlocked (to allow threads
to call other methods) while at the same time, "this->waitMutex" should
be locked to prevent multiple executions of code below. */
waitLock.lock(); // How do I release "this->mutex" here?
}
/* do some stuff... */
/* The main problem is with this event based OS function. It can
only be called once with the data I provide, therefore I need to
have a 2 locks - one blocks multiple method calls (the usual stuff)
and "waitLock" makes sure that only one instance of "osBlockingFunction"
is ruinning at the time. Since this is a thread blocking function,
"this->mutex" must be unlocked at this point. */
bool result = osBlockingFunction(...);
/* In methods, such as "close", "this->waitMutex" and others are then used
to make sure that thread blocking methods have returned and I can safely
modify related data. */
/* do some more stuff... */
return result;
}
How could I solve this "swapping" problem without overly complicating code? I could unlock this->mutex before locking another, however I'm afraid that in that nanosecond, a race condition might occur.
Edit:
Imagine that 3 threads are calling wait method. The first one will lock this->mutex, then this->waitMutex and then will unlock this->mutex. The second one will lock this->mutex and will have to wait for this->waitMutex to be available. It will not unlock this->mutex. The third one will get stuck on locking this->mutex.
I would like to get the last 2 threads to wait for this->waitMutex to be available.
Edit 2:
Expanded example with osBlockingFunction.
It smells like that the design/implementation should be a bit different with std::condition_variable cv on the HidDevice::wait and only one mutex. And as you write "other threads can execute other methods or abort this one" will call cv.notify_one to "abort" this wait. The cv.wait {enter wait & unlocks the mutex} atomically and on cv.notify {exits wait and locks the mutex} atomically. Like that HidDevice::wait is more simple:
bool HidDevice::wait(const std::function<bool(const Info&)>& predicate)
{
std::unique_lock<std::mutex> lock(this->m_Mutex); // Only one mutex.
m_bEarlyExit = false;
this->cv.wait(lock, spurious wake-up check);
if (m_bEarlyExit) // A bool data-member for abort.
return;
/* do some stuff... */
}
My assumption is (according to the name of the function) that on /* do some checks... */ the thread waits until some logic comes true.
"Abort" the wait, will be in the responsibility of other HidDevice function, called by the other thread:
void HidDevice::do_some_checks() /* do some checks... */
{
if ( some checks )
{
if ( other checks )
m_bEarlyExit = true;
this->cv.notify_one();
}
}
Something similar to that.
I recommend creating a little "unlocker" facility. This is a mutex wrapper with inverted semantics. On lock it unlocks and vice-versa:
template <class Lock>
class unlocker
{
Lock& locked_;
public:
unlocker(Lock& lk) : locked_{lk} {}
void lock() {locked_.unlock();}
bool try_lock() {locked_.unlock(); return true;}
void unlock() {locked_.lock();}
};
Now in place of:
waitLock.lock(); // How do I release "this->mutex" here?
You can instead say:
unlocker temp{lock};
std::lock(waitLock, temp);
where lock is a unique_lock instead of a lock_guard holding mutex.
This will lock waitLock and unlock mutex as if by one uninterruptible instruction.
And now, after coding all of that, I can reason that it can be transformed into:
waitLock.lock();
lock.unlock(); // lock must be a unique_lock to do this
Whether the first version is more or less readable is a matter of opinion. The first version is easier to reason about (once one knows what std::lock does). But the second one is simpler. But with the second, the reader has to think more carefully about the correctness.
Update
Just read the edit in the question. This solution does not fix the problem in the edit: The second thread will block the third (and following threads) from making progress in any code that requires mutex but not waitMutex, until the first thread releases waitMutex.
So in this sense, my answer is technically correct, but does not satisfy the desired performance characteristics. I'll leave it up for informational purposes.

C++ concurrency: Variable visibility outside of mutexes [duplicate]

This question already exists:
C++ concurrency: conditional atomic operations?
Closed 6 years ago.
I'm having some trouble understanding when variables are forced to be written to memory, even outside of mutex blocks. I apologize for the convoluted code below, because I have stripped away logic that deals with whether reader decides if some data is stale. The important thing to note is that 99.9% of the time, readers will take the fast path and synchronization must be very fast, which is why I use an atomic int32 to communicate both staleness and whether the slow path is now necessary.
I have the following setup, which I am "fairly" certain is race-free:
#define NUM_READERS 10
BigObject mSharedObject;
std::atomic_int32_t mStamp = 1;
std::mutex mMutex;
std::condition_variable mCondition;
int32_t mWaitingReaders = 0;
void reader() {
for (;;) { // thread loop
for (;;) { // spin until stamp is acceptible
int32_t stamp = mStamp.load();
if (stamp > 0) { // fast path
if (stampIsAcceptible(stamp) &&
mStamp.compare_exchange_weak(stamp, stamp + 1)) {
break;
}
} else { // slow path
// tell the loader (writer) that we're halted
std::unique_lock<mutex> lk(mMutex);
mWaitingReaders++;
mCondition.notify_all();
while (mWaitingReaders != 0) {
mCondition.wait(lk);
} // ###
lk.unlock();
// *** THIS IS WHERE loader's CHANGES TO mSharedObject
// *** MUST BE VISIBLE TO THIS THREAD!
}
}
// stamp acceptible; mSharedObject guaranteed not written to
mSharedObject.accessAndDoFunStuff();
mStamp.fetch_sub(1); // part of hidden staleness logic
}
}
void loader() {
for (;;) { // thread loop
// spin until we somehow decide we want to change mSharedObject!
while (meIsHappySleeping()) {}
// we want to modify mSharedObject, so set mStamp to 0 and wait
// for readers to see this and report that they are now waiting
int32_t oldStamp = mStamp.exchange(0);
unique_lock<mutex> lk(mMutex);
while (mWaitingReaders != NUM_READERS) {
mCondition.wait(lk);
}
// all readers are waiting. start writing to mSharedObject
mSharedObject.loadFromFile("example.foo");
mStamp.store(oldStamp);
mWaitingReaders = 0; // report completion
lk.unlock();
mCondition.notify_all();
// *** NOW loader's CHANGES TO mSharedObject
// *** MUST BE VISIBLE TO THE READER THREADS!
}
}
void setup() {
for (int i = 0; i < NUM_READERS; i++) {
std::thread t(reader); t.detach();
}
std::thead t(loader); t.detach();
}
The parts marked in stars *** are what concerns me. This is because while my code excludes races (as far as I can see), mSharedObject is only protected by a mutex while being written to by loader(). Because reader() needs to be extremely fast (as noted above), I do not want its read-only accesses to mSharedObject to have to be protected by a mutex.
One "guaranteed" solution is to introduce a thread-local variable const BigObject *latestObject at line ###, which is set to &mSharedObject and then use that for access. But is this bad practice? And is it really necessary? Will the atomic operations / mutex release operations guarantee that readers see the changes?
Thanks!
Lock-free code, and even locking code using just atomics is far from simple. The first thing to do would be to just add a mutex and profile how much of the performance is actually lost in the synchronisation. Note that current implementations of mutex may just do a quick spin-lock, which is roughly an atomic operation when uncontended.
If you want to attempt lock-free programming you will need to look into the memory ordering arguments to atomic operations. The writer will need to ..._release to synchronise with a reader doing ..._acquire (or use sequential consistency in both sides). Otherwise the reads/writes to any other variables may not be visible.

Using std::condition_variable with atomic<bool>

There are several questions on SO dealing with atomic, and other that deal with std::condition_variable. But my question if my use below is correct?
Three threads, one ctrl thread that does preparation work before unpausing the two other threads. The ctrl thread also is able to pause the worker threads (sender/receiver) while they are in their tight send/receive loops.
The idea with using the atomic is to make the tight loops faster in case the boolean for pausing is not set.
class SomeClass
{
public:
//...
// Disregard that data is public...
std::condition_variable cv; // UDP threads will wait on this cv until allowed
// to run by ctrl thread.
std::mutex cv_m;
std::atomic<bool> pause_test_threads;
};
void do_pause_test_threads(SomeClass *someclass)
{
if (!someclass->pause_test_threads)
{
// Even though we use an atomic, mutex must be held during
// modification. See documentation of condition variable
// notify_all/wait. Mutex does not need to be held for the actual
// notify call.
std::lock_guard<std::mutex> lk(someclass->cv_m);
someclass->pause_test_threads = true;
}
}
void unpause_test_threads(SomeClass *someclass)
{
if (someclass->pause_test_threads)
{
{
// Even though we use an atomic, mutex must be held during
// modification. See documentation of condition variable
// notify_all/wait. Mutex does not need to be held for the actual
// notify call.
std::lock_guard<std::mutex> lk(someclass->cv_m);
someclass->pause_test_threads = false;
}
someclass->cv.notify_all(); // Allow send/receive threads to run.
}
}
void wait_to_start(SomeClass *someclass)
{
std::unique_lock<std::mutex> lk(someclass->cv_m); // RAII, no need for unlock.
auto not_paused = [someclass](){return someclass->pause_test_threads == false;};
someclass->cv.wait(lk, not_paused);
}
void ctrl_thread(SomeClass *someclass)
{
// Do startup work
// ...
unpause_test_threads(someclass);
for (;;)
{
// ... check for end-program etc, if so, break;
if (lost ctrl connection to other endpoint)
{
pause_test_threads();
}
else
{
unpause_test_threads();
}
sleep(SLEEP_INTERVAL);
}
unpause_test_threads(someclass);
}
void sender_thread(SomeClass *someclass)
{
wait_to_start(someclass);
...
for (;;)
{
// ... check for end-program etc, if so, break;
if (someclass->pause_test_threads) wait_to_start(someclass);
...
}
}
void receiver_thread(SomeClass *someclass)
{
wait_to_start(someclass);
...
for (;;)
{
// ... check for end-program etc, if so, break;
if (someclass->pause_test_threads) wait_to_start(someclass);
...
}
I looked through your code manipulating conditional variable and atomic, and it seems that it is correct and won't cause problems.
Why you should protect writes to shared variable even if it is atomic:
There could be problems if write to shared variable happens between checking it in predicate and waiting on condition. Consider following:
Waiting thread wakes spuriously, aquires mutex, checks predicate and evaluates it to false, so it must wait on cv again.
Controlling thread sets shared variable to true.
Controlling thread sends notification, which is not received by anybody, because there is no threads waiting on conditional variable.
Waiting thread waits on conditional variable. Since notification was already sent, it would wait until next spurious wakeup, or next time when controlling thread sends notification. Potentially waiting indefinetly.
Reads from shared atomic variables without locking is generally safe, unless it introduces TOCTOU problems.
In your case you are reading shared variable to avoid unnecessary locking and then checking it again after lock (in conditional wait call). It is a valid optimisation, called double-checked locking and I do not see any potential problems here.
You might want to check if atomic<bool> is lock-free. Otherwise you will have even more locks you would have without it.
In general, you want to treat the fact that variable is atomic independently of how it works with a condition variable.
If all code that interacts with the condition variable follows the usual pattern of locking the mutex before query/modification, and the code interacting with the condition variable does not rely on code that does not interact with the condition variable, it will continue to be correct even if it wraps an atomic mutex.
From a quick read of your pseudo-code, this appears to be correct. However, pseudo-code is often a poor substitute for real code for multi-threaded code.
The "optimization" of only waiting on the condition variable (and locking the mutex) when an atomic read says you might want to may or may not be an optimization. You need to profile throughput.
atomic data doesn't need another synchronization, it's basis of lock-free algorithms and data structures.
void do_pause_test_threads(SomeClass *someclass)
{
if (!someclass->pause_test_threads)
{
/// your pause_test_threads might be changed here by other thread
/// so you have to acquire mutex before checking and changing
/// or use atomic methods - compare_exchange_weak/strong,
/// but not all together
std::lock_guard<std::mutex> lk(someclass->cv_m);
someclass->pause_test_threads = true;
}
}

Cross-platform up- and downgradable read/write lock

I try to turn some central data structure of a large codebase multithreaded.
The access interfaces were changed to represent read/write locks, which may be up- and downgraded:
Before:
Container& container = state.getContainer();
auto value = container.find( "foo" )->bar;
container.clear();
Now:
ReadContainerLock container = state.getContainer();
auto value = container.find( "foo" )->bar;
{
// Upgrade read lock to write lock
WriteContainerLock write = state.upgrade( container );
write.clear();
} // Downgrades write lock to read lock
Using an actual std::mutex for the locking (instead of r/w implementation) works fine but brings no performance benefit (actually degrades runtime).
Actual changing data is relatively rare, so it seems very desirable to go with the read/write concept. The big issue now is that I cannot seem to find any library, which implements the read/write concept and supports upgrade and downgrade and works on Windows, OSX and Linux alike.
Boost has BOOST_THREAD_PROVIDES_SHARED_MUTEX_UPWARDS_CONVERSIONS but does not seem to support downgrading (blocking) atomic upgrading from shared to unique.
Is there any library out there, that supports the desired feature set?
EDIT:
Sorry for being unclear. Of course I mean multiple-readers/single-writer lock semantic.
The question has changed since I answered. As the previous answer is still useful, I will leave it up.
The new question seems to be "I want a (general purpose) reader writer lock where any reader can be upgraded to a writer atomically".
This cannot be done without deadlocks, or the ability to roll back operations (transactional reads), which is far from general-purpose.
Suppose you have Alice and Bob. Both want to read for a while, then they both want to write.
Alice and Bob both get a read lock. They then upgrade to a write lock. Neither can progress, because a write lock cannot be acquired while a read lock is acquired. You cannot unlock the read lock, because then the state Alice read while read locked may not be consistent with the state after the write lock is acquired.
This can only be solved with the possibility the read->write upgrade can fail, or the ability to rollback all operations in a read (so Alice can "unread", Bob can advance, then Alice can re-read and try to get the write lock).
Writing type-safe transactional code isn't really supported in C++. You can do it manually, but beyond simple cases it is error prone. Other forms of transactional rollbacks can also be used. None of them are general purpose reader-writer locks.
You can roll your own. If the states are R, U, W and {} (read, upgradable, write and no lock), these are transitions you can easily support:
{} -> R|U|W
R|U|W -> {}
U->W
W->U
U->R
and implied by the above:
W->R
which I think satisifies your requirements.
The "missing" transition is R->U, which is what lets us have multiple-readers safely. At most one reader (the upgrade reader) has the right to upgrade to write without releasing their read lock. While they are in that upgrade state they do not block other threads from reading (but they do block other threads from writing).
Here is a sketch. There is a shared_mutex A; and a mutex B;.
B represents the right to upgrade to write and the right to read while you hold it. All writers also hold a B, so you cannot both have the right to upgrade to write while someone else has the right to write.
Transitions look like:
{}->R = read(A)
{}->W = lock(B) then write(A)
{}->U = lock(B)
U->W = write(A)
W->U = unwrite(A)
U->R = read(A) then unlock(B)
W->R = W->U->R
R->{} = unread(A)
W->{} = unwrite(A) then unlock(B)
U->{} = unlock(B)
This simply requires std::shared_mutex and std::mutex, and a bit of boilerplate to write up the locks and the transitions.
If you want to be able to spawn a write lock while the upgrade lock "remains in scope" extra work needs to be done to "pass the upgrade lock back to the read lock".
Here are some bonus try transitions, inspired by #HowardHinnat below:
R->try U = return try_lock(B) && unread(A)
R->try W = return R->try U->W
Here is an upgradable_mutex with no try operations:
struct upgradeable_mutex {
std::mutex u;
std::shared_timed_mutex s;
enum class state {
unlocked,
shared,
aspiring,
unique
};
// one step at a time:
template<state start, state finish>
void transition_up() {
transition_up<start, (state)((int)finish-1)>();
transition_up<(state)((int)finish-1), finish>();
}
// one step at a time:
template<state start, state finish>
void transition_down() {
transition_down<start, (state)((int)start-1)>();
transition_down<(state)((int)start-1), finish>();
}
void lock();
void unlock();
void lock_shared();
void unlock_shared();
void lock_aspiring();
void unlock_aspiring();
void aspiring_to_unique();
void unique_to_aspiring();
void aspiring_to_shared();
void unique_to_shared();
};
template<>
void upgradeable_mutex::transition_up<
upgradeable_mutex::state::unlocked, upgradeable_mutex::state::shared
>
() {
s.lock_shared();
}
template<>
void upgradeable_mutex::transition_down<
upgradeable_mutex::state::shared, upgradeable_mutex::state::unlocked
>
() {
s.unlock_shared();
}
template<>
void upgradeable_mutex::transition_up<
upgradeable_mutex::state::unlocked, upgradeable_mutex::state::aspiring
>
() {
u.lock();
}
template<>
void upgradeable_mutex::transition_down<
upgradeable_mutex::state::aspiring, upgradeable_mutex::state::unlocked
>
() {
u.unlock();
}
template<>
void upgradeable_mutex::transition_up<
upgradeable_mutex::state::aspiring, upgradeable_mutex::state::unique
>
() {
s.lock();
}
template<>
void upgradeable_mutex::transition_down<
upgradeable_mutex::state::unique, upgradeable_mutex::state::aspiring
>
() {
s.unlock();
}
template<>
void upgradeable_mutex::transition_down<
upgradeable_mutex::state::aspiring, upgradeable_mutex::state::shared
>
() {
s.lock();
u.unlock();
}
void upgradeable_mutex::lock() {
transition_up<state::unlocked, state::unique>();
}
void upgradeable_mutex::unlock() {
transition_down<state::unique, state::unlocked>();
}
void upgradeable_mutex::lock_shared() {
transition_up<state::unlocked, state::shared>();
}
void upgradeable_mutex::unlock_shared() {
transition_down<state::shared, state::unlocked>();
}
void upgradeable_mutex::lock_aspiring() {
transition_up<state::unlocked, state::aspiring>();
}
void upgradeable_mutex::unlock_aspiring() {
transition_down<state::aspiring, state::unlocked>();
}
void upgradeable_mutex::aspiring_to_unique() {
transition_up<state::aspiring, state::unique>();
}
void upgradeable_mutex::unique_to_aspiring() {
transition_down<state::unique, state::aspiring>();
}
void upgradeable_mutex::aspiring_to_shared() {
transition_down<state::aspiring, state::shared>();
}
void upgradeable_mutex::unique_to_shared() {
transition_down<state::unique, state::shared>();
}
I attempt to get the compiler to work out some of the above transitions "for me" with the transition_up and transition_down trick. I think I can do better, and it did increase code bulk significantly.
Having it 'auto-write' the unlocked-to-unique, and unique-to-(unlocked|shared) was all I got out of it. So probably not worth it.
Creating smart RAII objects that use the above is a bit tricky, as they have to support some transitions that the default unique_lock and shared_lock do not support.
You could just write aspiring_lock and then do conversions in there (either as operator unique_lock, or as methods that return said, etc), but the ability to convert from unique_lock&& down to shared_lock is exclusive to upgradeable_mutex and is a bit tricky to work with implicit conversions...
live example.
Here's my usual suggestion: Seqlock
You can have a single writer and many readers concurrently. Writers compete using a spinlock. A single writer doesn't need to compete so is cheaper.
Readers are truly only reading. They're not writing any state variables, counters, etc. This means you don't really know how many readers are there. But also, there no cache line ping pong so you get the best performance possible in terms of latency and throughput.
What's the catch? the data almost has to be POD. It doesn't really have to POD, but it can not be invalidated (no deleting std::map nodes) as readers may read it while it's being written.
It's only after the fact that readers discover the data is possibly bad and they have to re-read.
Yes, writers don't wait for readers so there's no concept of upgrade/downgrade. You can unlock one and lock the other. You pay less than with any sort of mutex but the data may have changed in the process.
I can go into more detail if you like.
The std::shared_mutex (as implemented in boost if not available on your platform(s)) provides some alternative for the problem.
For atomic upgrade lock semantics, the boost upgrade lock may be the best cross platform alternative.
It does not have an upgrade and downgrade locking mechanism you are looking for, but to get an exclusive lock, the shared access can be relinquished first, then exclusive access sought.
// assumes shared_lock with shared access has been obtained
ReadContainerLock container = state.getContainer();
auto value = container.find( "foo" )->bar;
{
container.shared_mutex().unlock();
// Upgrade read lock to write lock
std::unique_lock<std::shared_mutex> write(container.shared_mutex());
// container work...
write.unlock();
container.shared_mutex().lock_shared();
} // Downgrades write lock to read lock
A utility class can be used to cause the re-locking of the shared_mutex at the end of the scope;
struct re_locker {
re_locker(std::shared_mutex& m) : m_(m) { m_.unlock(); }
~re_locker() { m_.shared_lock(); }
// delete the copy and move constructors and assignment operator (redacted for simplicity)
};
// ...
auto value = container.find( "foo" )->bar;
{
re_locker re_lock(container.shared_mutex());
// Upgrade read lock to write lock
std::unique_lock<std::shared_mutex> write(container.shared_mutex());
// container work...
} // Downgrades write lock to read lock
Depending on what exception guarantees you want or require, you may need to add a "can re-lock" flag to the re_locker to either do the re-lock or not if an exception is thrown during the container operations/work.

Upgradeable read/write lock Win32

I am in search of an upgradeable read write lock for win32 with the behaviour of pthreads rwlock, where a read lock can be up- and downgraded.
What I want:
pthread_rwlock_rdlock( &lock );
...read...
if( some condition ) {
pthread_rwlock_wrlock( &lock );
...write...
pthread_rwlock_unlock( &lock );
}
...read...
pthread_rwlock_unlock( &lock );
The upgrade behaviour is not required by posix, but it works on linux on mac.
Currently, I have a working implementation (based on an event, a semaphore and a critical section) that is upgradeable, but the upgrade may fail when readers are active. If it fails a read unlock + recheck + write lock is necessary.
What I have:
lock.rdlock();
...read...
if( some condition ) {
if( lock.tryupgrade() ) {
...write...
lock.unlock();
return;
} else {
lock.unlock();
// <- here, other threads may alter the condition ->
lock.wrlock();
if( some condition ) { // so, re-check required
...write...
}
lock.unlock();
return;
}
}
...read...
lock.unlock();
EDIT: The bounty:
I am still in search, but want to add some restrictions: it is used intra-process-only (so based on critical sections is ok, WIN32 mutexes are not ok), and it should be pure WIN32 API (no MFC, ATL etc.). Acquiring read locks should be fast (so, acquiring the read lock should not enter a critical section in its fast path). Maybe an InterlockedIncrement based solution is possible?
The boost shared_mutex class supports reader (shared) and writer (unique) locks and temporary upgrades from shared to unique locks.
Example for boost shared_mutex (multiple reads/one write)?
I don't recommend writing your own, it's a tricky thing to get right and difficult to test thoroughly.
pthread library is a 'Portable Threads' library. That means it's also supported on windows ;) Have a look: Pthreads-w32.
Additionally, consider using OpenMP instead of locks: the compiler extension provides portable critical sections, kernes threading model, tasks and much more! MS C++ supports this technology as well as g++ in Linux.
Cheers! :)
What is wrong with this approach?
// suppose:
struct RwLock
{
void AcquireExclusive();
void AcquireShared();
void Release();
bool TryAcquireExclusive();
};
// rwlock that has TryAcquireSharedToExclusive
struct ConvertableRwLock
{
void AcquireExclusive()
{
writeIntent.AcquireExclusive();
rwlock.AcquireExclusive();
writeIntent.Release();
}
void AcquireShared()
{
readIntent.AcquireShared();
rwlock.AcquireShared();
readIntent.Release();
}
void Release()
{
rwlock.Release();
}
bool TryConvertSharedToExclusive()
{
// Defer to other exclusive including convert.
// Avoids deadlock with other TryConvertSharedToExclusive.
if (!writeIntent.TryAcquireExclusive())
{
rwlock.Release();
return false;
}
// starve readers
readIntent.AcquireExclusive();
// release full lock, but no other can acquire since
// this thread has readIntent and writeIntent.
rwlock.Release();
// Wait for other shared to drain.
rwlock.AcquireExclusive();
readIntent.Release();
writeIntent.Release();
return true;
}
private:
RwLock rwlock;
RwLock readIntent;
RwLock writeIntent;
};