Returning code from pthread creation in C++ is 11 - c++

I have thread creation problem using Pthread. My code is as follows. I show only some portion due to space constraints.
Main.c create Detectdirection instance and send to the function.
d = new Detectdirection();
while(run)
{
int ret = d->run_parallel(d);
if(ret == -1)
run = false;
}
My Detectdirection Class has two functions to run in parallel:
class Detectdirection{
public:
int run_parallel(void*p);
void *Tracking(void *p);
static void *Tracking_helper(void * p);
void *ReadImage(void *p );
static void *ReadImage_helper(void *p );
private:
pthread_t thread[2];
}
void *Detectdirection::ReadImage(void *p){
Detectdirection *app = (Detectdirection*)p;
while(run){
}
pthread_exit(NULL);
}
void *Detectdirection::Tracking(void *p){
Detectdirection *app = (Detectdirection*)p;
while(run){
}
pthread_exit(NULL);
}
void *Detectdirection::Tracking_helper(void *p){
Detectdirection *app = (Detectdirection*)p;
return ((Detectdirection*)p)->Tracking(app);
}
void *Detectdirection::ReadImage_helper(void *p ){
Detectdirection *app = (Detectdirection*)p;
return ((Detectdirection*)p)->ReadImage(app);
}
int Detectdirection::run_parallel(void* p){
Detectdirection *app = (Detectdirection*)p;
int rc = pthread_create(&thread[0], NULL, app->ReadImage_helper, app);
if (rc) {
printf("ERROR; return code from pthread_create() is %d\n", rc);
return -1;
}
rc = pthread_create(&thread[1], NULL, app->Tracking_helper, app);
if (rc) {
printf("ERROR; return code from pthread_create() is %d\n", rc);
return -1;
}
return 0;
}
Compile is ok and when I run, I have thread creation error. That sort of return type 11 happens only when many threads are created. But now I create only two thread and I have that error. What could be wrong?

I believe your are getting EAGAIN (based on the error code 11). That (obivously) means your system doesn't have enough resources to create threads anymore.
POSIX documentation says:
[EAGAIN] The system lacked the necessary resources to create another
thread, or the system-imposed limit on the total number of threads in
a process {PTHREAD_THREADS_MAX} would be exceeded.
I am not quite sure the following is true.
But now I create only two thread and I have that error. What could be wrong?
Here,
while(run)
{
int ret = d->run_parallel(d);
if(ret == -1)
run = false;
}
You are creating in a loop and each call d->run_parallel() creates two threads. So, you are potentially creating infinite number of threads
as the loop only breaks when pthread_create() fails. So, you may want to look at this loop carefully whether you really want to do as it is right now.
You don't seem to join with the threads you create. So, you could detach the threads so that thread-specific resources are released immediately when the thread(s) exit.
You can do:
pthread_detach(pthread_self());
in both ReadImage_helper() and Tracking_helper() functions to detach them. This could potentially solve your resource issue.
If it's still present then you have to look at ways to limit the number of threads that are simultaneously running on your system. One possible option is to use thread pools -- create a fixed number of threads and assign them new tasks as the threads complete their current task(s).

Related

create threads but don't run it immediately in linux

I am trying to execute my program in threads, I use pthread_create(), but it runs the threads immediately. I would like to allow the user to change thread priorities before running. How it is possible to resolve?
for(int i = 0; i < threads; i++)
{
pthread_create(data->threads+i,NULL,SelectionSort,data);
sleep(1);
print(data->array);
}
Set the priority as you create the thread.
Replace
errno = pthread_create(..., NULL, ...);
if (errno) { ... }
with
pthread_attr_t attr;
errno = pthread_attr_init(&attr);
if (errno) { ... }
{
struct sched_param sp;
errno = pthread_attr_getschedparam(&attr, &sp);
if (errno) { ... }
sp.sched_priority = ...;
errno = pthread_attr_setschedparam(&attr, &sp);
if (errno) { ... }
}
/* So our scheduling priority gets used. */
errno = pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
if (errno) { ... }
errno = pthread_create(..., &attr, ...);
if (errno) { ... }
errno = pthread_attr_destroy(&attr);
if (errno) { ... }
For pthreads the priority isn't set after thread creation but rather by passing suitable attributes upon thread creation: the thread attributes go where you have specified NULL in your pthread_create() call. If you want to delay thread creation until the user has given you a priority you can create a function object expecting the priority and upon call of that function object you'd kick off the thread. Of course, you'll still need to keep track of the thus created object (possibly using a std::future<...>-like object) to later join that thread.
Note that providing an answer shouldn't be construed as endorsing thread priorities: as far as I can tell, playing with thread priorities are ill-advised.

pthread_attr_setstacksize and pthread_exit

I have a question about C concurrency programming in Embedded System with about 64Mb Ram.
Especially, I want to reduce the default memory used by a Thread, so I have defined:
pthread_attr_t attr_test;
size_t stacksize = 0x186A0; // 100Kbyte
pthread_attr_init(&attr_test);
pthread_attr_setdetachstate(&attr_test, PTHREAD_CREATE_DETACHED);
pthread_attr_setstacksize(&attr_test, stacksize);
So, When the Thread starts, it uses only 100Kbyte of virtual Memory.
BUT when the Thread ends and calls pthread_exit, the virtual Memory used by the process, increases rapidly!....
Why? What can I do?
Thanks!
UPDATE:
Thread ->
void *thread_test(void *arg1) {
int *param;
param = (int*)arg1;
printf("Thread %d start\n", *param);
pthread_cond_wait(&condition[*param], &mutex[*param]);
printf("Thread %d stop\n",*param);
pthread_exit(0);
}
Main ->
int main(void) {
pthread_t IDthread[MAX_THREADS];
int param[MAX_THREADS];
int pointer;
int i, keyb;
void *stkaddr;
size_t stacksize;
puts("!!! THREAD TEST !!!");
printf("Process ID %d\n\n", getpid());
for(i=0; i<MAX_THREADS; i++)
{
pthread_cond_init(&condition[i], NULL);
pthread_mutex_init(&mutex[i], NULL);
IDthread[i] = 0;
param[i] = i;
}
stacksize = 0x186A0; // 100Kbyte
pthread_attr_init(&attr_test);
pthread_attr_setdetachstate(&attr_test, PTHREAD_CREATE_DETACHED);
/* setting the size of the stack also */
pthread_attr_setstacksize(&attr_test, stacksize);
pointer = 0;
do {
keyb = getchar();
if (keyb == '1')
{
if (pointer < MAX_THREADS)
{
pthread_create(&IDthread[pointer], &attr_test, thread_test, &param[pointer]);
sleep(1);
pointer++;
}
else
puts("MAX Threads Number");
}
if (keyb == '2')
{
if (pointer != 0)
{
pointer--;
pthread_cond_signal(&condition[pointer]);
sleep(1);
}
else
puts("0 Thread is running");
}
} while (keyb != '0');
printf("FINE\n");
return EXIT_SUCCESS;
}
There is a known issue with the joinable or detached threads, quoting from the manual:
Only when a
terminated joinable thread has been joined are the last of its
resources released back to the system. When a detached thread
terminates, its resources are automatically released back to the
system
you can make the thread detachable with:
pthread_attr_setdetachstate(3)
There are some problems with your test.
At first, pthread_attr_setstacksize has the following documentation:
The stack size attribute determines the minimum size (in bytes) that will be allocated for threads created using the thread attributes object attr.
So each thread could use more than what you have set. But more than that, threads may allocate memory from the OS to use as stack. And this also applies to the main thread.
Therefore I don't think there is a way to achieve what you want by looking at the result of top command, since this information is only visible from within the thread itself.
Also note that the virtual memory used by the process is not related to the amount of RAM used by the process.
Here is something you can try to check the total stack of a thread.

Detached pthreads and memory leak

Can somebody please explain to me why this simple code leaks memory?
I believe that since pthreads are created with detached state their resources should be released inmediatly after it's termination, but it's not the case.
My environment is Qt5.2.
#include <QCoreApplication>
#include <windows.h>
void *threadFunc( void *arg )
{
printf("#");
pthread_exit(NULL);
}
int main()
{
pthread_t thread;
pthread_attr_t attr;
while(1)
{
printf("\nStarting threads...\n");
for(int idx=0;idx<100;idx++)
{
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
pthread_create( &thread, &attr, &threadFunc, NULL);
pthread_attr_destroy ( &attr );
}
printf("\nSleeping 10 seconds...\n");
Sleep(10000);
}
}
UPDATE:
I discovered that if I add a slight delay of 5 milliseconds inside the for loop the leak is WAY slower:
for(int idx=0;idx<100;idx++)
{
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
pthread_create( &thread, &attr, &threadFunc, NULL);
pthread_attr_destroy ( &attr );
Sleep(5); /// <--- 5 MILLISECONDS DELAY ///
}
This is freaking me out, could somebody please tell me what is happening? How this slight delay may produce such a significant change? (or alter the behavior in any way)
Any advice would be greatly appreciated.
Thanks.
UPDATE2:
This leak was observed on Windows platforms (W7 and XP), no leak was observed on Linux platforms (thank you #MichaelGoren)
I checked the program with slight modifications on windows using cygwin, and memory consumption was steady. So it must be a qt issue; the pthread library on cygwin works fine without leaking.
#include <pthread.h>
#include <stdio.h>
#include <unistd.h>
void *threadFunc( void *arg )
{
printf("#");
pthread_exit(NULL);
}
int main()
{
pthread_t thread;
pthread_attr_t attr;
int idx;
while(1)
{
printf("\nStarting threads...\n");
for(idx=0;idx<100;idx++)
{
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
pthread_create( &thread, &attr, &threadFunc, NULL);
pthread_attr_destroy ( &attr );
}
printf("\nSleeping 10 seconds...\n");
//Sleep(10000);
sleep(10);
}
}
Compiler optimizations or the OS it self can decide to do loop unrolling. That is your for loop has a constant bound (100 here). Since there is no explicit synchronization to prevent it, a newly created, detached thread can die and have its thread ID reassigned to another new thread before its creator returns from pthread_create() due to this unrolling. The next iteration is already started before the thread was actually destroyed.
This also explains why your added slight delay has less issues; one iteration takes longer and hence the thread functions can actually finish in more cases and hence the threads are actually terminated most of the time.
A possible fix would be to disable compiler optimizations, or add synchronization; that is, you check whether the thread still exist, at the end of the code, if it does you'll have to wait for the function to finish.
A more tricky way would be to use mutexes; you let the thread claim a resource at creation and by definition of PTHREAD_CREATE_DETACHED this resource is automatically released when the thread is exited, hence you can use try_lock to test whether the thread is actually finished. Note that I haven't tested this approach so I'm not actually sure whether PTHREAD_CREATE_DETACHED actually is working according to its definition...
Concept:
pthread_mutex_t mutex;
void *threadFunc( void *arg )
{
printf("#");
pthread_mutex_lock(&mutex);
pthread_exit(NULL);
}
for(int idx=0;idx<100;idx++)
{
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
pthread_create( &thread, &attr, &threadFunc, NULL);
pthread_attr_destroy ( &attr );
pthread_mutex_lock(&mutex); //will block untill "destroy" has released the mutex
pthread_mutex_unlock(&mutex);
}
The delay can induce a large change in behavior because it gives the thread time to exit! Of course how your pthread library is implemented is also a factor here. I suspect it is using a 'free list' optimization.
If you create 1000 threads all at once, then the library allocates memory for them all before any significant number of those threads can exit.
If as in your second code sample you let the previous thread run and probably exit before you start a new thread, then your thread library can reuse that thread's allocated memory or data structures which it now knows are no longer needed and it is now probably holding in a free list just in case someone creates a thread again and it can efficiently recycle the memory.
It has nothing to do with compiler optimisations. Code is fine. Problem could be
a) Windows itself.
b) Qt implementation of pthread_create() with detached attributes
Checking for (a): Try to create many fast detached threads using Windows _beginthreadex directly and see if you get the same picture. Note: CloseHandle(thread_handle) as soon as _beginthreaex returns to make it detached.
Checking for (b): Trace which function Qt uses to create threads. If it is _beginthread then there is your answer. If it is _beginthreadex, then Qt is doing the right thing and you need to check if Qt closes the thread handle handle immediately. If it does not then that is the cause.
cheers
UPDATE 2
Qt5.2.0 does not provide pthreads API and is unlikely responsible for the observed leak.
I wrapped native windows api to see how the code runs without pthread library. You can include this fragment right after includes:
#include <process.h>
#define PTHREAD_CREATE_JOINABLE 0
#define PTHREAD_CREATE_DETACHED 1
typedef struct { int detachstate; } pthread_attr_t;
typedef HANDLE pthread_t;
_declspec(noreturn) void pthread_exit(void *retval)
{
static_assert(sizeof(unsigned) == sizeof(void*), "Modify code");
_endthreadex((unsigned)retval);
}
int pthread_attr_setdetachstate(pthread_attr_t *attr, int detachstate)
{
attr->detachstate = detachstate;
return 0;
}
int pthread_attr_init(pthread_attr_t *attr)
{
attr->detachstate = PTHREAD_CREATE_JOINABLE;
return 0;
}
int pthread_attr_destroy(pthread_attr_t *attr)
{
(void)attr;
return 0;
}
typedef struct {
void *(*start_routine)(void *arg);
void *arg;
} winapi_caller_args;
unsigned __stdcall winapi_caller(void *arglist)
{
winapi_caller_args *list = (winapi_caller_args *)arglist;
void *(*start_routine)(void *arg) = list->start_routine;
void *arg = list->arg;
free(list);
static_assert(sizeof(unsigned) == sizeof(void*), "Modify code");
return (unsigned)start_routine(arg);
}
int pthread_create( pthread_t *thread, pthread_attr_t *attr,
void *(*start_routine)(void *), void *arg)
{
winapi_caller_args *list;
list = (winapi_caller_args *)malloc(sizeof *list);
if (list == NULL)
return EAGAIN;
list->start_routine = start_routine;
list->arg = arg;
*thread = (HANDLE)_beginthreadex(NULL, 0, winapi_caller, list, 0, NULL);
if (*thread == 0) {
free(list);
return errno;
}
if (attr->detachstate == PTHREAD_CREATE_DETACHED)
CloseHandle(*thread);
return 0;
}
With Sleep() line commented out it works OK without leaks. Run time = 1hr approx.
If the code with Sleep line commented out is calling Pthreads-win32 2.9.1 library (prebuilt for MSVC) then the program stops spawning new threads and stops responding after 5..10 minutes.
Test environment: XP Home, MSVC 2010 Expresss, Qt5.2.0 qmake etc.
You forgot to join your thread (even if they are finished already).
Correct code should be:
pthread_t arr[100];
for(int idx=0;idx<100;idx++)
{
pthread_attr_init(&attr);
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_DETACHED);
pthread_create( &arr[idx], &attr, &threadFunc, NULL);
pthread_attr_destroy ( &attr );
}
Sleep(2000);
for(int idx=0;idx<100;idx++)
{
pthread_join(arr[idx]);
}
Note from man page:
Failure to join with a thread that is joinable (i.e., one that is not detached), produces a "zombie thread". Avoid doing this, since each zombie thread consumes some system resources, and when enough zombie threads have
accumulated, it will no longer be possible to create new threads (or processes).

cancelling a thread inside a signal handler

I have started a timer and set the interval as 5 secs and registered a signal handler for it.
When SIGALRM is encountered iam trying to terminate the thread inside the signal handler, bt not able to do that. Thread is not getting terminated , instead of this whole process is killed.
The following is the code:
void signalHandler()
{
printf("Caught signal ...\n");
printf("Now going to terminate thread..\n");
pthread_kill(tid, SIGKILL);
}
void * thread_function()
{
int oldstate;
char result[256] = {0};
time_t startTime = time(NULL);
time_t timerDuration = 5;
time_t endTime = startTime + timerDuration;
while(1) {
printf("Timer is runnuing as dameon..\n");
if(!strcmp(result, "CONNECTED")) {
resp = 1;
pthread_exit(&resp);
}
}
}
int main()
{
int *ptr[2];
signal(SIGALRM, signalHandler);
timer.it_interval.tv_usec = 0;
timer.it_interval. tv_usec = 0;
timer.it_value.tv_sec = INTERVAL;
timer.it_value.tv_usec = 0;
setitimer(ITIMER_REAL, &timer, 0);
pthread_create(&tid, NULL, thread_function, NULL);
pthread_join(tid, (void**)&(ptr[0]));
printf("test %d\n\n",*ptr[0]);
while(1)
printf("1");
}
Platform : Linux , gcc compiler
As far as I'm aware you pretty much can't call anything inside a signal handler as you don't know what state your code is in.
Your best option is to set up a thread to handle your signals. All your other threads should call pthread_setsigmask and to block all signals, and then you create another thread, which calls calls pthread_setsigmask to catch SIGALARM, and then calls sigwait, at which point it can cancel the other thread.
The way of handling signals is much different in a multi-threaded environment as compared to a single threaded environment. In a multi-threaded code, you should block out all the signals for all the threads that have your business logic and then create a seperate thread for handling the signals. This is because, in multi-threaded environment, you cannot be sure to which thread the signal will be delivered.
Please refer to this link for more details:
http://devcry.heiho.net/2009/05/pthreads-and-unix-signals.html
Apart from this, to kill a thread use pthread_cancel which should work fine for you.
You can try using a flag:
int go_on[number_of_threads] = { 1 };
void signalHandler()
{
printf("Caught signal ...\n");
printf("Now going to terminate thread..\n");
go_on[tid] = 0;
}
void * thread_function()
{ /* */
while(go_on[this_thread_id]) {
printf("Timer is runnuing as dameon..\n");
if(!strcmp(result, "CONNECTED")) {
resp = 1;
pthread_exit(&resp);
}
}
}

multiple threads-not able to access class member variables set by constructor

I am spawning threads from one of the functions called by main.
The start routine of this thread is a function in another separate class altogether. So to get access to that class, i have written an extern "C" function, by which i am able to call the start routine.
But the problem is, after getting to the start routine, the thread is not able to access the member variables value set by the constructor of the Class.
This seems strange to me as everything is perfect when i am running the code without using threads.
Can someone please suggest me what would be going wrong?
I am posting some relevant code details below:
`extern "C"{
void* run(void* arg)
{
CFileOp* trans = static_cast<CFileOp*>(arg);
trans->write_block(arg);
return 0;
}
}
int
TestFileOps(int file_size, CGlobalItems &globals){
...
for(i = 0; i < num_chunks; i++)
{
pthread_create( &thread_id[i], NULL, run, buf);
}
...
}`
//there is a class CFileOp which has some private member variables and write_block is a public function of it.
void* CFileOp::write_block(PVOID buf)
{
int rc = my_write(78, buf, m_chunk_size);
if(rc != m_chunk_size)
{
fprintf(stderr, "Can't write block; rc=%d, buf=%p, chunk_size=%d\n", rc, buf, m_chunk_size);
pthread_exit((void *)-1);return 0;;
}
m_cur_pos++;
fprintf(stderr,"m_cur_pos: %d m_chunks_per_file: %d\t",m_cur_pos,m_chunks_per_file);
if(m_cur_pos >= m_chunks_per_file)
{
if(seek(0, SEEK_CUR) == -1)
pthread_exit((void *)-1);return 0;// return -1;
}
pthread_exit((void *)rc);
return 0;
}
I can't post the whole code as its a benchmark code and is very long and detailed.
Please help.
If I understand the question properly you want to call a member function from a thread, you can just do if you have c++11
std::thread th(&my_class::my_mem_func, &my_object);
this will create a thread th and execute the my_mem_func of my_object
EDIT
std::thread th(&my_writer::write_some, &writer_object, data);
th.join();