In my programme, I handle new threads with
pthread_t thread;
pthread_create(&thread, NULL,
c->someFunction, (void *) fd); //where fd is ID of the thread
The question is quite simple - if I just let the someFunction to finish, is it needed then in C++ to call something e.g. join or anything else, to prevenet memory leaks or is the memory freed automatically??
From the opengroup page for pthread_join,
The pthread_join() function provides a simple mechanism allowing an
application to wait for a thread to terminate. After the thread
terminates, the application may then choose to clean up resources that
were used by the thread. For instance, after pthread_join() returns,
any application-provided stack storage could be reclaimed.
The pthread_join() or pthread_detach() function should eventually be
called for every thread that is created with the detachstate attribute
set to PTHREAD_CREATE_JOINABLE so that storage associated with the
thread may be reclaimed.
and from the man page of pthread_join
Failure to join with a thread that is joinable (i.e., one that is not
detached), pro‐
duces a "zombie thread". Avoid doing this, since each zombie thread consumes some
system resources, and when enough zombie threads have accumulated, it will no longer
be possible to create new threads (or processes).
There is no pthreads analog of waitpid(-1, &status, 0), that is, "join with any ter‐
minated thread".
If you believe you need this functionality, you probably need to
rethink your application design.
If you do pthread_detach,
The pthread_detach() function shall indicate to the implementation
that storage for the thread thread can be reclaimed when that thread
terminates
If you don't detach or join a joinable thread, it can cause waste of resources
Related
Is this the equivalent of a leak? To use pointers and dynamically allocated memory as an analogy, if I lose a pointer to a resource that I've allocated memory for I can't delete or free that memory anymore. Likewise if I have a worker thread that's running an infinite loop and I detach the thread then is that a leak? There's no way to stop the thread executing if it's been detached?
What happens is the thread keeps running, potentially forever. If the thread decides on its own to exit, it will clean up its own internal data structures so there won't be any memory leak (at least, not from the thread-class itself; detaching the thread object tells it not to expect any parent thread to do cleanup via join(), so the exiting thread will do that cleanup itself). If it keeps running, then whatever resources it has allocated will remain allocated, although technically it's not a memory leak because code in the thread itself still (presumably) has the ability to free those resources, if it wanted to.
There is still a way to stop the thread from executing; you simply have to ask it nicely to exit. How you ask it is up to you -- you could do it by setting a std::atomic<bool> pleaseQuit; variable to true, and relying on the thread to periodically check the variable's value and exit if it's true, or by sending a byte to the thread on a pipe or a socket, or by some other means. This does require the co-operation of the code running inside the thread, of course; there's no good/safe way to unilaterally kill an uncooperative thread.
Once you've asked the thread to stop running, you usually want to then block until you know that it the thread is guaranteed dead, e.g. so that you can safely deallocate data structures that the thread might try to access while it still exists. To do that, you'll need to join() the thread, which means you shouldn't detach() it since detached threads can't be joined. Detached threads are only useful in situations where you don't care about cleaning up (e.g. because you're not expecting the thread to ever stop running anyway)
So I am using a variable
std::vector<pthread_t> preallocatedThreadsPool;
to hold all the pthreads,
then I use a for loop
preallocatedThreadsPool.resize(preallocatThreadsNumber); // create a threadpoOl
for(pthread_t i : preallocatedThreadsPool) {
pthread_create(&i, NULL, threadFunctionUsedByThreadsPool, NULL);
}
to create the threads pool,
the question is how do I really destory it, for example, when i send signal to the program then i need to manual handle the program to stop all the preallocated pthreads?
I have tried to use another for loop and inside the for loop to call pthread_exit(i),
but the IDE, tell me the for loop will only execute once, which obviously not working
I have tried to use preallocatedThreadsPool.clear() , to clean the vector, however when i use
gdb tool to debug it, inside the info threads , the threads are still there?
is there a good way to destory all the preallocated pthreads in my case?
Threads have to exit themselves. You can't exit another thread.
You can use pthread_join to wait for a thread to exit, or pthread_detach to say that you're never going to call pthread_join. You have to call one of these, or it leaks the thread. pthread_join destroys the thread; pthread_detach doesn't destroy the thread (obviously) but it allows the thread to destroy itself when it exits.
Since this is a thread pool, you must have a queue of things you want the threads in the pool to do. You can add special "please exit" items to the end of the queue (or the beginning), and then wait for the threads to exit. Make it so the threads exit when they see a "please exit" item in the queue.
It's all about thread synchronization. The proper way is that you have to have a global flag (a condition variable or a Win32 Event for example) that threads must periodically check and if set, terminate. When a thread is exiting, you must also wait for it to do so, so each thread should signal another event when "I'm done".
After that, any "handle" allocated to pthread or to std::thread or to CreateThread can be destroyed. In std::thread, you can detach and forget about the handle.
Even if you can kill the thread immediately by a function such as TerminateThread (there should be something similar in pthreads), this is very bad, for you will have leaked memory, possibly.
I developed a multi threaded caching proxy for OpenWrt (Linux for routers, so there is few RAM available). I used Boost and Curl libraries, it works fine but its RAM consumption doesn't stop increasing (Memory leak).
I free all variables allocated dynamically
I close all the opened files
Clean up the initialized curl handles
Global variable are constants
Here is a part of my main code:
while (true) {
ip::tcp::socket* socket = new ip::tcp::socket(io_service);
acceptor.accept(*socket);
pthread_t thread;
pthread_create(&thread, NULL, handleRequest_thread, (void*) socket);
pthread_detach(thread);
}
It's sure that the memory leak occurs "handleRequest_thread", so my question is:
Is there a way to free all the resources (memory, opened files ...) after the thread is finished ?
You cannot free all resources automatically ( only by exiting your process, but that's not very good way ). So you have to control your recourses, use smart pointers where you can, automatically close files ( using fstream for example ) etc.
As for thread: maybe you should use boost.thread instead, it will make sure you don't forget to clean thread objects itself ( what you can easily forget using plain pthreads ).
Do you destroy the ip::tcp::socket* in the detached thread when it finishes? Memory for threads is freed on joining, but this is not neccessary for detached threads. The resources for detached thread are given back to the system when they finish. You could check the exit status of pthread_detach if you are able to detach the thread.
from the man pages man pthread_create:
A thread may either be joinable or detached. If a thread is joinable,
then another thread can call pthread_join(3) to wait for the thread to
terminate and fetch its exit status. Only when a terminated joinable
thread has been joined are the last of its resources released back to
the system. When a detached thread terminates, its resources are auto‐
matically released back to the system: it is not possible to join with
the thread in order to obtain its exit status.
Perhaps you create the threads faster then you can finish them, and hence your memory usage keeps increasing.
I'm working on a project in which I have a main thread and one daemon thread to perform file outputs. In my main thread I have a field pthread_t * _daemon that I would like to delete, but obviously only after _daemon returns NULL (I understand that using pthread_exit() cause memory leaks).
How can I do it without busy-wait? If I try to use a condition variable I have a problem - When I call signal() form _daemon to wake up my main thread, it deletes _daemon before it _daemon returns NULL.
What I did is to just use a mutex lock that is locked when the program is launched and unlocked before _daemon returns. Is there any better way that would not cause busy wait?
pthread_detach() does what you're looking for. It sounds like it will solve your problem (no leaking) with a lot less complexity!
So you can safely call pthread_detatch(_daemon) when you're done with it inside the other thread, without having to worry about if the thread itself is still running. It does not terminate the thread, instead it causes the thread to be cleaned up when it does terminate.
From the documentation:
The pthread_detach() function shall indicate to the implementation
that storage for the thread thread can be reclaimed when that thread
terminates. If thread has not terminated, pthread_detach() shall not
cause it to terminate.
You can actually create a thread in the detached state to start with by setting attr of:
int pthread_create(pthread_t *thread, const pthread_attr_t *attr,
void *(*start_routine) (void *), void *arg);
appropriately at creation time. The pthread_create() manpage says:
When a
detached thread terminates, its resources are automatically released back to
the system. [snip] Making a thread detached is useful for some types of daemon
threads whose exit status the application does not need to care about. By
default, a new thread is created in a joinable state, unless attr was set to
create the thread in a detached state (using pthread_attr_setdetachstate(3)).
pthread_t is an unsigned long int, so I don't really see a need to make a pointer of pthread_t (you can as well use it's address in pthread functions), but if you insist, then you can delete it after creating a pthread (but then you will have no way to communicate with it, because it's a number of created thread - used for joining, detaching functions etc.).
I am working on a multithreaded program using C++ and Boost. I am using a helper thread to eagerly initialize a resource asynchronously. If I detach the thread and all references to the thread go out of scope, have I leaked any resources? Or does the thread clean-up after itself (i.e. it's stack and any other system resources needed for the itself)?
From what I can see in the docs (and what I recall from pthreads 8 years ago), there's not explicit "destory thread" call that needs to be made.
I would like the thread to execute asynchronously and when it comes time to use the resource, I will check if an error has occured. The rough bit of code would look something like:
//Assume this won't get called frequently enough that next_resource won't get promoted
//before the thread finishes.
PromoteResource() {
current_resource_ptr = next_resource_ptr;
next_resource_ptr.reset(new Resource());
callable = bind(Resource::Initialize, next_resource); //not correct syntax, but I hope it's clear
boost::thread t(callable);
t.start();
}
Of course--I understand that normal memory-handling problems still exist (forget to delete, bad exception handling, etc)... I just need confirmation that the thread itself isn't a "leak".
Edit: A point of clarification, I want to make sure this isn't technically a leak:
void Run() {
sleep(10 seconds);
}
void DoSomething(...) {
thread t(Run);
t.run();
} //thread detaches, will clean itself up--the thread itself isn't a 'leak'?
I'm fairly certain everything is cleaned up after 10 seconds-ish, but I want to be absolutely certain.
The thread's stack gets cleaned up when it exits, but not anything else. This means that anything it allocated on the heap or anywhere else (in pre-existing data structures, for example) will get left when it quits.
Additionally any OS-level objects (file handle, socket etc) will be left lying around (unless you're using a wrapper object which closes them in its destructor).
But programs which frequently create / destroy threads should probably mostly free everything that they allocate in the same thread as it's the only way of keeping the programmer sane.
If I'm not mistaken, on Windows Xp all resources used by a process will be released when the process terminates, but that isn't true for threads.
Yes, the resources are automatically released upon thread termination. This is a perfectly normal and acceptable thing to do to have a background thread.
To clean up after a thread you must either join it, or detach it (in which case you can no longer join it).
Here's a quote from the boost thread docs that somewhat explains that (but not exactly).
When the boost::thread object that
represents a thread of execution is
destroyed the thread becomes detached.
Once a thread is detached, it will
continue executing until the
invocation of the function or callable
object supplied on construction has
completed, or the program is
terminated. A thread can also be
detached by explicitly invoking the
detach() member function on the
boost::thread object. In this case,
the boost::thread object ceases to
represent the now-detached thread, and
instead represents Not-a-Thread.
In order to wait for a thread of
execution to finish, the join() or
timed_join() member functions of the
boost::thread object must be used.
join() will block the calling thread
until the thread represented by the
boost::thread object has completed. If
the thread of execution represented by
the boost::thread object has already
completed, or the boost::thread object
represents Not-a-Thread, then join()
returns immediately. timed_join() is
similar, except that a call to
timed_join() will also return if the
thread being waited for does not
complete when the specified time has
elapsed.
In Win32, as soon as the thread's main function, called ThreadProc in the documentation, finishes, the thread is cleaned up. Any resources allocated by you inside the ThreadProc you'll need to clean up explicitly, of course.