Why does std::counting_semaphore::acquire() suffer deadlock in this case? - c++

Question

I am testing std::counting_semaphore on C++20 with Windows 10 and MinGW x64.
As I learned from https://en.cppreference.com/w/cpp/thread/counting_semaphore, std::counting_semaphore is an atomic counter. We can use release() to increase the counter, and use acquire() to decrease the counter. If the counter equals to 0, than the thread wait.
I build the following simplified example to show my problem.
If I always release() before acquire() in the thread, the internal counter value(v) of std::counting_semaphore should always stay between v and v+1, and this code should never suffer any block.
When I run this example code, it suffers deadlock very often, but sometimes it can finish correctly.
I try to use std::cout message to understand the deadlock situation, but the deadlock disappeared when I using std::cout. In another hand, the deadlock disappeared when I use std::unique_lock.
The example is as follows:
#include <iostream>
#include <thread>
#include <atomic>
#include <vector>
#include <mutex>
#include <semaphore>
using namespace std::literals;
std::mutex mtx;
const int numOfThr {2};
const int numOfForLoop {1000};
const int max_smph {numOfThr* numOfForLoop *2};
std::counting_semaphore<max_smph> smph {numOfThr+1};
void thrf_TestSmph ( const int iThr )
{
for ( int i = 0; i < numOfForLoop; ++i )
{
// std::unique_lock ul(mtx);
//unique_lock can stop deadlock.
smph.release(); //smph counter ++
smph.acquire(); //smph counter --
// if ( i % 1000 == 1 ) std::cout << iThr << " : " << i << "\n";
//print out message can stop deadlock.
}
}
int main()
{
std::cout << "Start testing semaphore ..." << "\n\n";
std::vector<std::thread> thrf_TestSmphVec ( numOfThr );
for ( int iThr = 0; iThr < numOfThr; ++iThr )
{
thrf_TestSmphVec[iThr] = std::thread ( thrf_TestSmph, iThr );
}
for ( auto& thr : thrf_TestSmphVec )
{
if ( thr.joinable() )
thr.join();
}
std::cout << "Test is done." << "\n";
return 0;
}

Answer 1

Update: Found this bug report: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104928
This is not really an answer.
I can reproduce the infinite blocking on my M1 macbook air, when it is compiled with gcc or clang and libstdc++. Printing message did't prevent the blocking. When it is compiled with clang and libc++, the program finished normally.
I noticed this piece of code and comment in my included header include/c++/11/bits/semaphore_base.h of libstdc++:
_GLIBCXX_ALWAYS_INLINE void
_M_release(ptrdiff_t __update) noexcept
{
if (0 < __atomic_impl::fetch_add(&_M_counter, __update, memory_order_release))
return;
if (__update > 1)
__atomic_notify_address_bare(&_M_counter, true);
else
__atomic_notify_address_bare(&_M_counter, true);
// FIXME - Figure out why this does not wake a waiting thread
// __atomic_notify_address_bare(&_M_counter, false);
}
Then I changed the first return to __atomic_notify_address_bare(&_M_counter, true);, and the problem seems disappear.
That comment is commited in this commit.
_GLIBCXX_ALWAYS_INLINE void
_M_release(ptrdiff_t __update) noexcept
{
if (0 < __atomic_impl::fetch_add(&_M_counter, __update, memory_order_release))
return;
if (__update > 1)
__atomic_notify_address_bare(&_M_counter, true);
else
- __atomic_notify_address_bare(&_M_counter, false);
+ __atomic_notify_address_bare(&_M_counter, true);
+ // FIXME - Figure out why this does not wake a waiting thread
+ // __atomic_notify_address_bare(&_M_counter, false);
It seems that the developer team has known the problem, but their short-term solution didn't fix the problem.

Answer 2

After doing a lot of experimentations about std::counting_semaphore::acquire(), I noticed that it will suffer a blocking when two threads trigger std::counting_semaphore::acquire() in a very close time interval. It seems to make the internal counter inside of the std::counting_semaphore be frozen, so
std::counting_semaphore::release() can not increase the internal counter of the std::counting_semaphore correctly. In this situation, the next std::counting_semaphore::acquire() will be blocked, because the internal counter is frozen. This situation happens in a lot of intense threads experimentations with std::counting_semaphore::acquire() on my system. The example code in my question is the most simplified one to reproduce this problem.
I guess it is a kind of collision issue inside of my system. Base on this assumption, I try to use back-off to bypass this problem.
I use while(!std::counting_semaphore::try_acquire_for(1ns)){} to substitude std::counting_semaphore::acquire(), because std::counting_semaphore::try_acquire_for() can return false when it can not decrease the internal counter.
It works well at this moment, even I increase the const int numOfThr {2} to {100`000}.
Here comes the example code as follows:
#include <iostream>
#include <thread>
#include <atomic>
#include <vector>
#include <mutex>
#include <semaphore>
using namespace std::literals;
std::mutex mtx;
const int numOfThr {2};
const int numOfForLoop {1000};
const int max_smph {numOfThr* numOfForLoop * 2};
std::counting_semaphore<max_smph> smph {numOfThr + 1};
void thrf_TestSmph ( const int iThr )
{
for ( int i = 0; i < numOfForLoop; ++i )
{
smph.release(); //smph counter ++
while ( !smph.try_acquire_for ( 1ns ) ) {} //smph counter --
//don't use smph.acquire() directly, it easily makes blocking.
}
}
int main()
{
std::cout << "Start testing semaphore ..." << "\n\n";
std::vector<std::thread> thrf_TestSmphVec ( numOfThr );
for ( int iThr = 0; iThr < numOfThr; ++iThr )
{
thrf_TestSmphVec[iThr] = std::thread ( thrf_TestSmph, iThr );
}
for ( auto& thr : thrf_TestSmphVec )
{
if ( thr.joinable() )
thr.join();
}
std::cout << "Test is done." << "\n";
return 0;
}