I'm new in boost programming, and I've been looking for a reason to use the io_service::work, but I can't figure it out; in some of my tests I removed it and works fine.
The io_service::run() will run operations as long as there are asynchronous operations to perform. If, at any time, there are no asynchronous operations pending (or handlers being invoked), the run() call will return.
However, there are some designs that would prefer that the run() call not exit until all work is done AND the io_service has explicitly been instructed that it's okay to exit. That's what io_service::work is used for. By creating the work object (I usually do it on the heap and a shared_ptr), the io_service considers itself to always have something pending, and therefore the run() method will not return. Once I want the service to be able to exit (usually during shutdown), I will destroy the work object.
io_service::work is base class of all works that can posted to an instance of io_service, for example when you are working with a socket and start an asynchronous read, actually you are adding a work to the io_service. So you normally never use work directly, but there is one exception to this:
io_service::run will return as soon as there is no more work to do, so consider an application that have some producer and consumer threads, producers occasionally produce works and post them to consumer threads with io_service::post, but if all works finished, then io_service::run will return and possibly your consumer thread will be stopped, so you need an arbitrary work to keep io_service busy, in this case you may use io_service::work directly.
Related
It seems like some sources recommend using curl_multi_remove_handle to "invalidate" a curl handle and cause curl_multi_wait to return early. This seems not to be covered under the thread safety guarantee (if done from another thread), or am I wrong (the threads safety guarantees are basically just reentrancy guarantees)?
What is the recommended way signal curl_multi_wait to return early? Is it really required to do it via timeouts? (Under Linux, I would use an eventfd in the epoll set to effectively have the case "wait on these sockets OR this event fd OR the given timeout".) It seems I could use custom curl_waitfd structures, but this would require platform specific setup for dummy sockets.
You must not call curl_multi_remove_handle from thread B if curl_multi_wait for that handle is running in thread A. That will just cause tears and misery.
You can opt to, for example:
user sufficiently short timeouts for curl_multi_wait() so that you don't need to abort it
add a private socket/file descriptor to send data on to abort when you want to
return error from the progress callback (or another callback) for the transfer(s) you need to stop - by setting a flag that they all check (global, or global like)
rework your app logic so that you can consider the transfer to "dead" without it having stopped yet, and have libcurl have its cause and close it later and you don't have to care much about it being done a bit after you decided you can ignore it.
curl_multi_poll()
After I first wrote this answer, we introduced curl_multi_poll in libcurl. This function is very similar to curl_multi_wait but also allows it to pre-emptively return with the use of curl_multi_wakeup, thus offering applications a few more alternative approaches.
Unfortunately, curl_multi is not, what people these days would deem as "thread safe". Yes, you can use a CURLM handle in two different threads, as long, as they don't access it at the same time. But hey, this is true for almost any data structure in C or C++.
So, if you have one thread running an event loop with curl_multi_wait(), you cannot use a second thread to add new jobs via curl_multi_add_handle() or remove jobs via curl_multi_remove_handle(). Well, it will work most of the times, but especially during high load, you will start getting data corruptions and segfaults due to the concurrent access to libcurl's internal data structures.
There are two ways around this problem, but both require a bit of coding:
Use the newer curl_multi_poll() interface, which (unlike curl_multi_wait()) is externally interruptible via curl_multi_wakeup(). Yes, curl_multi_wakeup() is the ONLY function on CURLM handles, that is safe to call concurrently from another thread (or even multiple threads). To add new requests to the event loop or remove requests from it, you would need some request queue and a mutex, which secures access to that queue. Then, to add a new job, you would do:
(thread 1 is running curl_multi_poll() in an endless loop)
thread 2 acquires said mutex
thread 2 posts an "add easy handle request" into the request queue
thread 2 releases said mutex again
thread 2 calls curl_multi_wakeup()
thread 1 acquires the mutex after curl_multi_poll() returns
thread 1 then processes the "add easy handle request" in the job list and performs curl_multi_add_handle()
thread 1 then releases the mutex again
thread 1 does all other necessary work (in particular call curl_multi_perform() and pass finished transfers to the application etc.)
thread 1 calls curl_multi_poll() again
To remove a job, you would use the same procedure, just let thread 2 post an "remove easy handle request" instead of an "add easy handle request" to the request queue and then let thread 1 call curl_multi_remove_handle() instead of curl_multi_add_handle().
In this solution, ALL calls to the CURLM handle are performed from thread 1, with the sole exception of curl_multi_wakeup(), which is used by other threads to signal thread 1 of new work waiting in the request queue.
Or use the curl_action() interface, where you have to provide two callbacks to libcurl, with which it reports file descriptors to watch and a timeout to your application. You then have to call epoll() or a similiar OS function yourself to wait for activity (or timeout) in the event loop thread. Then add a mutex again to serialize access to the CURLM handle: Your event loop thread should lock that mutex just before it calls curl_action() (or any other function on the CURLM handle) and unlock it immediately after. As curl_action() (unlike curl_multi_poll()) does not sleep, that mutex will be locked only for brief intervals. So other threads can then easily directly lock that mutex for themselves, too, and call curl_multi_add_handle() or curl_multi_remove_handle() as needed. Be aware, though, that those intervening additions or removals of handles can modify the active FD set, and that you may need some synchronisation with the event loop thread to notify it of the modified epoll() set.
The first solution is likely easier to implement. You should be able to find libcurl wrappers for both variants on Github, but be sure to test them intensively before using them in any critical application.
io_service::run() is called by thread A. Is it safe to call async_write from thread B?
io_service::run() is called by thread A. Are async operations executed by thread A, or is thread A only guaranteed to call handlers and behind the scenes there could be additional threads that execute the operations?
io_service::run() is called by thread A. Some thread calls async_read and async_write using the same buffer. Is it safe to assume that the buffer will be accessed by at most one operation at a time? Or is it so that only handlers are called serially, but behind the scenes reads and writes can occur simultaneously?
The documentation says "The program must ensure that the stream performs no other read operations (such as async_read, the stream's async_read_some function, or any other composed operations that perform reads) until this operation completes.". Is it correct to interpret this as "You must not perform more than one read operation on a socket at a time. But you may perform 10 read operations on 10 distinct sockets."?
Having a socket that indefinitely accepts data, is it a good idea to call async_read and call it again from async_read's handler?
Does io_service::stop() stop all pending async operations or simply stops accepting new ones and executes the pending ones?
Yes, providing the io_service is tied to whatever is calling async_write. However, it should be noted that it is safe to call async_write from thread B even if the run is not called: it'll get queued in the io_service and wait until one of the run-ing calls are completed.
The callbacks posted to the io_service will run on thread A. Other async operations (such as timer operations) can happen on other threads. What is guarenteed to be on A and what is on its own thread is defined by the specific object being used, not by io_service.
Nope. Yup-ish. Depends on the class calling io_service.
Yes.
Yes, in fact this is super common, as it both ensures that only 1 async_read call is running at a time for a given socket and that there is always "work" for the io_service.
It usually finished the last callback and then stops accepting new ones and stops processing pending ones. It actually still accepts new ones but forces a reset is called before any other callbacks are called.
io_service is a message queue (basically), while a socket that posts its messages to the io_service is something else entirely.
1: Yes
4: Yes, it's okay to perform distinct operations on distinct sockets.
5: Yes, if you check the examples that's how they do it.
6: Considering the reference manual says
All invocations of its run() or run_one() member functions should return as soon as possible.
I would say it might do any.
For number 2 and 6, the source is available so the best way to answer those question is by downloading and reading it.
I would like to have boost thread object being deleted together with exiting from thread entry function. Is it something wrong if I start the thread function and pass a shared pointer to object, which owns thread object instance and then, when thread function exits, it destroys the this object together with thread object at the same time?
EDIT:
Maybe I will describe why I want to do that. I have to use low level dbus API. What I want to do is to create the adapter class, which will start its own thread and wait for incoming messages until the DISCONNECT message arrives. If it arrives I want to close the thread and kill the Adapter itself. The adapter is an Active Object, which runs the method sent to its scheduler. These methods put themselves on the scheduler queue once again after reading message from dbus. But if it is DISCONNECT message, they should not sent the method but just exit scheduler thread, destroying the Adapter object. hmmm looks like it is too complicated...
From the Boost.Thread documentation you can see that a thread object that is joinable should not be deleted, otherwise std::terminate will be called.
So you should assure that if the thread is joinable, either join() or detach() should be called in the destructor of the object owning the thread. Note: if the thread itself is destroying the object, join() is not an option. The thread would attempt to join itself, resulting in a deadlock.
However, if you keep these restrictions in mind, you can destroy a thread from within its own thread of execution.
You can do this, but you probably should not.
The main purpose of the boost::thread object is that you can monitor the associated thread. Having a thread monitor itself does not make much sense in most scenarios.
As was suggested by the other answers, you could just detach the thread and throw the boost::thread object away. Doing this is usually considered bad style, unless the monitoring responsibility has been transferred to another object first. For example, many simple worker threads set a future upon completion. The future already provides all the monitoring we need, so we can detach the thread.
You should never detach a thread completely such that you lose all means of monitoring it. You must at least be able to guarantee a clean shutdown, which becomes impossible for all but the most trivial threads if you detach them completely.
I am not sure if that addresses your use case but it sounds to me like you don't have to do this.
The lifetime of the boost::thread object does not necessarily coincide with the thread itself. Meaning that if you don't care you can just as well start the thread, call detach() on it and have the object run out of scope. Then it is deleted but the thread will still run until it's function is finished. The only thing is, you won't be able to join it. So if your program finishes while the thread still runs it will crash.
In case you do care about this stuff, the question might be wrong because in this case you would store the objects and call join() on them before deleting.
I'm using boost::asio to do some very basic UDP packet collection. The io_service object is instantiated in a worker thread, and io_service.run() is called from inside that thread. My problem is getting io_service.run() to return when I am done collecting packets.
I'm not clear on what methods of io_service can be called from other threads when it comes time to stop my worker thread. I have a reference to the io_service object, and from a different thread I make this call:
ios.dispatch( boost::bind( &udp_server::handle_kill, this ) );
In my udp_server class, the handler for that function cancels the pending work from a single boost::asio::ip::udp::socket and a single boost::asio::deadline_timer object. Both have pending async work to do. At that point I call ios.stop():
void udp_server::handle_kill()
{
m_socket.cancel();
m_timer.cancel();
m_ios.stop();
}
With no work pending, I expect at this point that my call to ios.run() should return - but this does not happen.
So why does it not return? The most likely explanation to me is that I shouldn't be calling io_service::dispatch() from another thread. But the dispatch() method kind of seems like it was built to do just that - dispatch a function call in the thread that io_service::run() is working in. And it seems to do just that.
So this leaves me with a few related questions:
Am I using io_service::dispatch() correctly?
If all tasks are canceled, is there any reason that io_service::run() should not return?
socket::upd::cancel() doesn't seem to be the right way to close a socket and abort all work. What is the right way?
asio is behaving pretty well for me, but I need to get a better understanding of this bit of architecture.
More data
socket::udp::cancel() is apparently an unsupported operation on an open socket under Win32 - so this operation fails by throwing an exception - which does in fact cause an exit from io_service::run(), but definitely not the desired exit.
socket::udp::close() doesn't seem to cancel the pending async_receive_from() task, so calling it instead of socket::udp::cancel() seems to leave the thread somewhere inside io_service::run().
Invoking io_service::stop from another thread is safe, this is well described in the documentation
Thread Safety
Distinct objects: Safe.
Shared objects: Safe, with the
exception that calling reset() while
there are unfinished run(), run_one(),
poll() or poll_one() calls results in
undefined behaviour.
as the comments to your question indicate, you really need to boil this down to a reproducible example.
I have a TCP Server application that serves each client in a new thread using POSIX Threads and C++.
The server calls "listen" on its socket and when a client connects, it makes a new object of class Client. The new object runs in its own thread and processes the client's requests.
When a client disconnects, i want some way to tell my main() thread that this thread is done, and main() can delete this object and log something like "Client disconnected".
My question is, how do i tell to the main thread, that a thread is done ?
The most straightforward way that I can see, is to join the threads. See here. The idea is that on a join call, a command thread will then wait until worker threads exit, and then resume.
Alternatively, you could roll something up with some shared variables and mutexes.
If the child thread is really exiting when it is done (rather than waiting for more work), the parent thread can call pthread_join on it which will block until the child thread exits.
Obviously, if the parent thread is doing other things, it can't constantly be blocking on pthread_join, so you need a way to send a message to the main thread to tell it to call pthread_join. There are a number of IPC mechanisms that you could use for this, but in your particular case (a TCP server), I suspect the main thread is probably a select loop, right? If that's the case, I would recommend using pipe to create a logical pipe, and have the read descriptor for the pipe be one of the descriptors that the main thread selects from.
When a child thread is done, it would then write some sort of message to the pipe saying "I'm Done!" and then the server would know to call pthread_join on that thread and then do whatever else it needs to do when a connection finishes.
Note that you don't have to call pthread_join on a finished child thread, unless you need its return value. However, it is generally a good idea to do so if the child thread has any access to shared resources, since when pthread_join returns without error, it assures you that the child thread is really gone and not in some intermediate state between having sent the "I'm Done!" message and actually having exited.
pthreads return 0 if everything went okay or they return errno if something didn't work.
int ret, joined;
ret = pthread_create(&thread, NULL, connect, (void*) args);
joined = pthread_join(&thread, NULL);
If joined is zero, the thread is done. Clean up that thread's object.
While it is possible to implement IPC mechanisms to notify a main thread when other threads are about to terminate, if you want to do something when a thread terminates you should try to let the terminating thread do it itself.
You might look into using pthread_cleanup_push() to establish a routine to be called when the thread is cancelled or exits. Another option might be to use pthread_key_create() to create a thread-specific data key and associated destructor function.
If you don't want to call pthread_join() from the main thread due to blocking, you should detach the client threads by either setting it as option when creating the thread or calling pthread_detach().
You could use a queue of "thread objects to be deleted", protect access to the queue with a mutex, and then signal a pthread condition variable to indicate that something was available on the queue.
But do you really want to do that? A better model is for each thread to just clean up after itself, and not worry about synchronizing with the main thread in the first place.
Calling pthread_join will block execution of the main thread. Given the description of the problem I don't think it will provide the desired solution.
My preferred solution, in most cases, would be to have the thread perform its own cleanup. If that isn't possible you'll either have to use some kind of polling scheme with shared variables (just remember to make them thread safe, hint:volatile), or perhaps some sort of OS dependant callback mechanism. Remember, you want to be blocked on the call to listen, so really consider having the thread clean itself up.
As others have mentioned, it's easy to handle termination of a given thread with pthread_join. But a weak spot of pthreads is funneling information from several sources into a synchronous stream. (Alternately, you could say its strong spot is performance.)
By far the easiest solution for you would be to handle cleanup in the worker thread. Log the disconnection (add a mutex to the log), delete resources as appropriate, and exit the worker thread without signaling the parent.
Adding mutexes to allow manipulation of shared resources is a tough problem, so be flexible and creative. Always err on caution when synchronizing, and profile before optimizing.
I had exactly the same problem as you described. After ~300 opened client connections my Linux application was not able to create new thread because pthread_join was never called. For me, usage of pthread_tryjoin_np helped.
Briefly:
have a map that holds all opened thread descriptors
from the main thread before new client thread is opened I iterate through map and call pthread_tryjoin_np for each thread recorded in map. If thread is done the result of call is zero meaning that I can clean up resources from that thread. At the same time pthread_tryjoin_np takes care about releasing thread resources. If pthread_tryjoin_np call returns number different from 0 this means that thread is still running and I simply do nothing.
Potential problem with this is that I do not see pthread_tryjoin_np as part official POSIX standard so this solution might not be portable.