Cancelling pthread_cond_wait() hangs with PRIO_INHERIT mutex - c++

Update, 4/10 2012:
Fixed by libc patch
I have a problem canceling threads in pthread_cond_wait, that use mutexes with the PTHREAD_PRIO_INHERIT attribute set. This only happens on certain platforms though.
The following minimal example demonstrates this: (compile with g++ <filename>.cpp -lpthread)
#include <pthread.h>
#include <iostream>
pthread_mutex_t mutex;
pthread_cond_t cond;
void clean(void *arg) {
std::cout << "clean: Unlocking mutex..." << std::endl;
pthread_mutex_unlock((pthread_mutex_t*)arg);
std::cout << "clean: Mutex unlocked..." << std::endl;
}
void *threadFunc(void *arg) {
int ret = 0;
pthread_mutexattr_t mutexAttr;
ret = pthread_mutexattr_init(&mutexAttr); std::cout << "ret = " << ret << std::endl;
//Comment out the following line, and everything works
ret = pthread_mutexattr_setprotocol(&mutexAttr, PTHREAD_PRIO_INHERIT); std::cout << "ret = " << ret << std::endl;
ret = pthread_mutex_init(&mutex, &mutexAttr); std::cout << "ret = " << ret << std::endl;
ret = pthread_cond_init(&cond, 0); std::cout << "ret = " << ret << std::endl;
std::cout << "threadFunc: Init done, entering wait..." << std::endl;
pthread_cleanup_push(clean, (void *) &mutex);
ret = pthread_mutex_lock(&mutex); std::cout << "ret = " << ret << std::endl;
while(1) {
ret = pthread_cond_wait(&cond, &mutex); std::cout << "ret = " << ret << std::endl;
}
pthread_cleanup_pop(1);
return 0;
}
int main() {
pthread_t thread;
int ret = 0;
ret = pthread_create(&thread, 0, threadFunc, 0); std::cout << "ret = " << ret << std::endl;
std::cout << "main: Thread created, waiting a bit..." << std::endl;
sleep(2);
std::cout << "main: Cancelling threadFunc..." << std::endl;
ret = pthread_cancel(thread); std::cout << "ret = " << ret << std::endl;
std::cout << "main: Joining threadFunc..." << std::endl;
ret = pthread_join(thread, NULL); std::cout << "ret = " << ret << std::endl;
std::cout << "main: Joined threadFunc, done!" << std::endl;
return 0;
}
Every time I run it, main() hangs on pthread_join(). A gdb backtrace shows the following:
Thread 2 (Thread 0xb7d15b70 (LWP 257)):
#0 0xb7fde430 in __kernel_vsyscall ()
#1 0xb7fcf362 in __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/lowlevellock.S:142
#2 0xb7fcc9f9 in __condvar_w_cleanup () at ../nptl/sysdeps/unix/sysv/linux/i386/i686/../i486/pthread_cond_wait.S:434
#3 0x08048fbe in threadFunc (arg=0x0) at /home/pthread_cond_wait.cpp:22
#4 0xb7fc8ca0 in start_thread (arg=0xb7d15b70) at pthread_create.c:301
#5 0xb7de73ae in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130
Thread 1 (Thread 0xb7d166d0 (LWP 254)):
#0 0xb7fde430 in __kernel_vsyscall ()
#1 0xb7fc9d64 in pthread_join (threadid=3083950960, thread_return=0x0) at pthread_join.c:89
#2 0x0804914a in main () at /home/pthread_cond_wait.cpp:41
If PTHREAD_PRIO_INHERIT isn't set on the mutex, everything works as it should, and the program exits cleanly.
Platforms with problems:
Embedded AMD Fusion board, running a PTXDist based 32-bit Linux 3.2.9-rt16 (with RTpatch 16). We are using the newest OSELAS i686 cross toolchain (2011.11.1), using gcc 4.6.2, glibc 2.14.1, binutils 2.21.1a, kernel 2.6.39.
Same board with the 2011.03.1 toolchain also (gcc 4.5.2 / glibc 2.13 / binutils 2.18 / kernel 2.6.36).
Platforms with no problems:
Our own ARM-board, also running a PTXDist Linux (32-bit 2.6.29.6-rt23), using OSELAS arm-v4t cross toolchain (1.99.3) with gcc 4.3.2 / glibc 2.8 / binutils 2.18 / kernel 2.6.27.
My laptop (Intel Core i7), running 64-bit Ubuntu 11.04 (virtualized / kernel 2.6.38.15-generic), gcc 4.5.2 / eglibc 2.13-0ubuntu13.1 / binutils 2.21.0.20110327.
I have been looking around the net for solutions, and have come across a few patches that I've tried, but without any effect:
Making the condition variables priority inheritance aware.
Handling EAGAIN from FUTEX_WAIT_REQUEUE_PI
Are we doing something wrong in our code, which just happens to work on certain platforms, or is this a bug in the underlying systems? If anyone has any idea about where to look, or knows of any patches or similar to try out, I'd be happy to hear about it.
Thanks!
Updates:
libc-help mailing list discussion
glibc bug report

This has been fixed by a libc patch.
I've confirmed it to work on my own problematic platform (our custom AMD Fusion board), patched onto glibc-2.14.1.
Thanks go out to Siddhesh Poyarekar for the fix!

Related

C++ multithread program I/O problem with std::endl

I try to learn how to use C++11 thread library and then, I am confused about the output of my following code.
#include <iostream>
#include <thread>
#include <mutex>
std::mutex mtx;
void thread_function()
{
std::cout << "Inside Thread :: ID = " << std::this_thread::get_id() << std::endl;
}
int main()
{
std::thread threadObj1(thread_function);
std::thread threadObj2(thread_function);
if (threadObj1.get_id() != threadObj2.get_id())
std::cout << "Both threads have different id" << std::endl;
threadObj1.join();
threadObj2.join();
std::cout<<"From Main Thread :: ID of Thread 1 = "<<threadObj1.get_id()<<std::endl;
std::cout<<"From Main Thread :: ID of Thread 2 = "<<threadObj2.get_id()<<std::endl;
return 0;
}
I attach every std::cout with std::endl in order to flush the buffer and output the '/n' character. However, finally I got the output as the following.
Both threads have different idInside Thread :: ID = Inside Thread :: ID =
0x700003715000
0x700003692000
From Main Thread :: ID of Thread 1 = 0x0
From Main Thread :: ID of Thread 2 = 0x0
Program ended with exit code: 0
It seems that the '/n' before the Inside Thread disappeared. Could you please help me figure it out? Thank you so much!
You have 3 threads which are accessing cout without any synchronization. You have defined mtx but it is not used, why?
Add lock_guard to protect cout statement:
void thread_function()
{
std::lock_guard<std::mutex> lk{mtx};
std::cout << "Inside Thread :: ID = " << std::this_thread::get_id() << std::endl;
}
if (threadObj1.get_id() != threadObj2.get_id())
{
std::lock_guard<std::mutex> lk{mtx};
std::cout << "Both threads have different id" << std::endl;
}
I think I should also receive three '\n' right?
The three '\n' characters in question are there in your output. They're at the ends of the first three lines of output.
I think maybe you misunderstand what this line from your example means:
std::cout << "Inside Thread :: ID = " << std::this_thread::get_id() << std::endl;
There are four separate function calls explicit in that one line of code. That one line does exactly the same thing as these four lines:
std::cout << "Inside Thread :: ID = ";
auto id = std::this_thread::get_id();
std::cout << id;
std::cout << std::endl;
Even assuming that the std::cout object is fully synchronized, You have done nothing to prevent the various threads from interleaving the separate function calls. E.g.,
main thread calls std::cout << "Both threads have different id";
threadObj1 calls std::cout << "Inside Thread :: ID = ";
threadObj2 calls std::cout << "Inside Thread :: ID = ";
main thread calls std::cout << std::endl;
threadObj1 calls std::cout << std::this_thread::get_id();
threadObj1 calls stc::cout << std::endl;
etc.

Thread Ownership of a mutex

According to several documentation examples the thread can't unlock a mutex unless it locked it explicitly. Here is an excerpt from man page for pthread_mutex_unlock at IBM.
The pthread_mutex_unlock() function unlocks the mutex specified. If
the calling thread does not currently hold the mutex (via a previous
call to pthread_mutex_lock(), pthread_mutex_trylock(), or
pthread_mutex_timedlock_np()) the unlock request fails with the EPERM
error.
Even the new C++ standard says something similar about the thread ownership, yet the following program was able to unlock a mutex locked on a different thread. On gcc & Linux systems the same behavior is seen both on pthread mutex as well as std::mutex (which I believe is implemented based on pthread_mutex anyway).
#include <iostream>
#include <thread>
#include <mutex>
#include <pthread.h>
std::mutex stmutex;
pthread_mutex_t pthrmutex = PTHREAD_MUTEX_INITIALIZER;
void thread1(int i)
{
stmutex.unlock();
std::cout << "Un Locked in thread 1" << std::this_thread::get_id() << std::endl;
}
void pthread1(int i)
{
pthread_mutex_unlock(&pthrmutex);
std::cout << "Un Locked Pthread mutex in thread 1" << std::this_thread::get_id() << std::endl;
}
void thread2(int i)
{
stmutex.lock();
std::cout << "Locked in thread 2" << std::this_thread::get_id() << std::endl;
}
void thread3(int i)
{
stmutex.unlock();
std::cout << "UNLocked in thread 3" << std::this_thread::get_id() << std::endl;
}
int main(int argc, char **argv)
{
try {
stmutex.lock();
std::cout << "Locked in main thread : " << std::this_thread::get_id() << std::endl;
std::thread t1(thread1,1);
t1.join();
stmutex.lock();
std::cout << "Locked in main thread after unlocking in thread1" << std::endl;
stmutex.unlock();
std::cout << "Un Locked in main thread " << std::endl;
pthread_mutex_lock(&pthrmutex);
std::cout << "Locked pthread mutex in main thread : " << std::this_thread::get_id() << std::endl;
std::thread t2(pthread1,1);
t2.join();
pthread_mutex_lock(&pthrmutex);
std::cout << "Locked Pthread mutext in main thread after unlocking in thread1" << std::endl;
pthread_mutex_unlock(&pthrmutex);
std::cout << "Un Locked Pthread mutext in main thread " << std::endl;
std::thread t3(thread2,1);
t3.join();
std::thread t4(thread3,1);
t4.join();
} catch (std::exception& ex)
{
std::cerr << "Exception In main thread: " << ex.what() << std::endl;
}
}
Am I missing anything in my understanding of mutex "Ownership" ?
The request fails with the EPERM error only for mutexes created with PTHREAD_MUTEX_ERRORCHECK.
See pthread_mutex_lock section RATIONALE:
... while being able to extract the thread ID of the owner of a mutex might be desirable, it would require storing the current thread ID when each mutex is locked, and this could incur unacceptable levels of overhead.
I.e. initialize your mutex with PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP.
Example:
void locker(pthread_mutex_t* mutex) {
if(int e = pthread_mutex_lock(mutex))
fprintf(stderr, "pthread_mutex_lock: (%d)%s\n", e, strerror(e));
}
void unlocker(pthread_mutex_t* mutex) {
if(int e = pthread_mutex_unlock(mutex))
fprintf(stderr, "pthread_mutex_unlock: (%d)%s\n", e, strerror(e));
}
int main() {
pthread_mutex_t a = PTHREAD_MUTEX_INITIALIZER;
pthread_mutex_t b = PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP;
std::thread(locker, &a).join();
std::thread(locker, &b).join();
std::thread(unlocker, &a).join();
std::thread(unlocker, &b).join(); // pthread_mutex_unlock: (1)Operation not permitted
}

std::launch::async is blocking like a sync process

I am running Visual Studio 2012 and attempting to learn how std::async works. I have created a very simple C++ console application:
#include "stdafx.h"
#include <future>
#include <iostream>
void foo() {
std::cout << "foo() thread sleep" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(5));
std::cout << "foo() thread awake" << std::endl;
}
int main()
{
std::future<void> res = std::async(std::launch::async, foo);
res.get();
std::cout << "MAIN THREAD" << std::endl;
system("pause");
return 0;
}
My initial expectation was to see "MAIN THREAD" printout appearing before "foo() thread awake" since the two threads are running asynchronously, with the foo() trailing behind due to its sleeping behavior. However, that is not what is actually happening. The call to res.get() blocks until foo() wakes up, and only then does it get to the "MAIN THREAD" printout. This is indicative of a synchronous behavior, so I am wondering what if perhaps I am either missing something, or not fully grasping the implementation. I have looked through numerous posts on this matter, but still cannot make any sense of it. Any help would be appreciated!
res.get();
blocks until the async is done.
http://en.cppreference.com/w/cpp/thread/future/get
Regardless of how you tell it to run, get can't give you the results until it's done.
Well, this is how std::future::get works - it blocks until future has some result or exception to provide.
that doesn't mean that async works synchronously, it is working asynchronously, it's only because you block the thread which waits on the result.
the idea was to to launch some task asynchronously, do something meanwhile and only call get when you need the result, as you might figured out, it is not the most scale-able thing..
if you use Visual Studio 2015, you can access the await keyword both for std::future and concurrency::task (Microsoft PPL library) , and for your own compatible defined types. this achieves non-blocking behavior.
#include "stdafx.h"
#include <future>
#include <iostream>
void foo() {
std::cout << "foo() thread sleep" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(5));
std::cout << "foo() thread awake" << std::endl;
}
std::future<void> entry(){
await std::async(std::launch::async, foo);
std::cout << "foo has finished, back in entry()\n";
}
int main()
{
std::cout << "MAIN THREAD" << std::endl;
entry();
std::cout << "BACK INMAIN THREAD" << std::endl;
system("pause");
return 0;
} //make sure to compile with /await flag
The problem is that res.get() has to wait for its thread to finish before getting its result (if any). To see the concurrency in motion you need to move the get() to after the other code that you want to run at the same time.
This example may make it a little clearer:
#include <ctime>
#include <cstdlib>
#include <future>
#include <iostream>
void foo(int id) {
std::cout << "foo(" << id << ") thread sleep" << std::endl;
// random sleep
std::this_thread::sleep_for(std::chrono::seconds(std::rand() % 10));
std::cout << "foo(" << id << ") thread awake" << std::endl;
}
int main()
{
std::srand(std::time(0));
std::future<void> res1 = std::async(std::launch::async, foo, 1);
std::future<void> res2 = std::async(std::launch::async, foo, 2);
std::future<void> res3 = std::async(std::launch::async, foo, 3);
std::cout << "MAIN THREAD SLEEPING" << std::endl;
std::this_thread::sleep_for(std::chrono::seconds(20));
std::cout << "MAIN THREAD AWAKE" << std::endl;
// now wait for all the threads to end
res1.get();
res2.get();
res3.get();
system("pause");
return 0;
}

boost::scoped_lock appears not to lock std::cout

I'm using boost 1.54.0 and Visual Studio 2010. For the code:
#include <iostream>
#include "boost/thread/thread.hpp"
#include "boost/thread/mutex.hpp"
boost::mutex mx1;
void func1()
{
{
boost::mutex::scoped_lock(mx1);
std::cout << "Thread " << boost::this_thread::get_id() << " starting work." << std::endl;
}
int x = 0;
for (int i=0; i<100; i++)
x++;
{
boost::mutex::scoped_lock(mx1);
std::cout << "Thread " << boost::this_thread::get_id() << " finished." << std::endl;
}
}
int main(void)
{
boost::thread thread1(&func1);
boost::thread thread2(&func1);
thread1.join();
thread2.join();
return 0;
}
About half the time I get the following (with varying thread ids and execution order, obviously):
Thread Thread 15b0 starting work.
1a18 starting work.
Thread 15b0 finished.
Thread 1a18 finished.
...instead of this (which is what I'd expect):
Thread 15b0 starting work.
Thread 1a18 starting work.
Thread 15b0 finished.
Thread 1a18 finished.
However, using
mx1.lock();
std::cout << "Thread " << boost::this_thread::get_id() << " starting work." << std::endl;
mx1.unlock();
...seems to work with no problems.
The output always seems to follow the same pattern. Am I using the mutex incorrectly, or is it something to do with std::cout?
Replace
boost::mutex::scoped_lock(mx1);
with
boost::mutex::scoped_lock lock(mx1);
you fell a victim of the most frequently occurring typo with the scoped lock:-)

Boost threads terminates program?

I have a boost thread program from a tutorial, i can compile it with no problems or warnings but when i run it i don't get any output form eclipse. The program termiates a once. I'm using MiNGW with my eclipse could this be the problem?
Does anybody have a idea? All sugestions are welcome!
Below is the tutorial code that i used, its from this site:
#include <iostream>
#include <boost/thread.hpp>
#include <boost/date_time.hpp>
void workerFunc()
{
boost::posix_time::seconds workTime(10000);
std::cout << "Worker: running" << std::endl;
// Pretend to do something useful...
boost::this_thread::sleep(workTime);
std::cout << "Worker: finished" << std::endl;
}
int main(int argc, char* argv[])
{
std::cout << "main: startup" << std::endl;
boost::thread workerThread(workerFunc);
std::cout << "main: waiting for thread" << std::endl;
workerThread.join();
std::cout << "main: done" << std::endl;
return 0;
}
Update
I have linked pthreads under mingw linker and the program compiles but the result is the same. When i put i breakpoiiny just below main i get this:
The target endianness is set automatically (currently little endian)
[New Thread 4168.0xbf8]
And the program terminates does anybody have idea?
You need to make sure you're linking in both the boost_thread and pthread libraries.