Wow, I wish I'd known about this page https://github.com/zaphoyd/websocketpp/blob/experimental/examples/broadcast_server/broadcast_server.cpp written by my c++ hero https://stackoverflow.com/users/1047661/zaphoyd before I started my c++ websocket adventure. Tons of lessons there.
If I'm reading it correctly (that's a stretch lol), it looks like connections and message sends & receipts are all handled in a single thread (can't wait til he "multi-threads" that or whatever it's called since he says in this basic example http://www.zaphoyd.com/websocketpp/manual/common-patterns/server-initiated-messages that WebSocket++ handlers block core networking functions. While this program is running its send loop in on_message, no new connections are being processed and no new messages are being received.) and a separate thread boost::thread(bind(&broadcast_server::process_messages,&server)); is set up to actually process the messages while the primary websocket++ thread simply adds to the queue the necessary information.
Please clear up my lack of knowledge: will the .push() fail if it happens at the same time as this section of code in the link
while(m_actions.empty()) {
m_action_cond.wait(lock);
}
action a = m_actions.front();
m_actions.pop();
lock.unlock();
or does the .push() simply wait for the lock to be released?
std::queue<T> doesn't know anything about threading on its own; however, in the code linked, all calls to push are wrapped like the following:
boost::unique_lock<boost::mutex> lock(m_action_lock);
//std::cout << "on_open" << std::endl;
m_actions.push(action(SUBSCRIBE,hdl));
lock.unlock();
m_action_cond.notify_one();
The constructor of the lock object above internally calls m_action_lock.lock() which blocks until the lock is released.
Note that m_action_cond.wait(lock), in the code you pasted in your question, unlocks the lock while waiting for the condition, and acquires the lock again after it has been woken up (either due to a signal from another thread, or possibly spuriously), so it doesn't prevent the producer (the one doing the push) thread from acquiring the lock while it's waiting: it's only between waking up and the call to lock.unlock() that blocking occurs.
Related
I have a thread with endless loop that receieves and processes data from a socket. I want that thread to block (sleep) until the data becomes available on the socket for reading or the "exit" boolean variable becomes true (set by a different thread). Is it possible to do that in a portable way without polling and using any third-party libraries (except sockets library naturally)? If it is not possible to do in a portable way what would be the best way to do it under Windows (still NO polling and third-party libraries)?
Example code:
bool exit = false; // or "std::atomic<bool> exit" or anything else
void fn()
{
SOCKET s;
// init socket, establish connection, etc
for(;;)
{
// This thread goes to wait (blocks) until data becomes available on socket
// OR exit var is set to true (by a different thread) - how?
if(exit) break;
// receive and process data from socket
}
}
Set up a queue of messages.
These messages are of the form "PleaseExit" or "DataOnSocket".
Your thread, or task, is activated when anything shows up in the queue, processes the queue, then waits on the queue again. If it gets "PleaseExit" it instead starts cleaning up.
Possibly you will have to have a different thread/task waiting on the condition variable and on the socket to ferry the information over to your unified queue.
I say "thread or task", because having an entire thread dedicated to waiting is overkill. Sadly, C++11 threading doesn't support light weight tasks out of the box perfectly.
Basically, this solution allows a thread to wait on multiple events by delegating the waiting on each event to other threads, which send notifications "up the pipe". You could imagine creating a common infrastructure, where your thread that wants to wait on multiple objects tells the dispatch center what it is waiting for, then waits on its own condition condition variable.
The dispatch center waits on each of the things your thread wanted to wait for, and when they occur proceeds to figure out which threads should be notified, then notifies them.
Far, far from ideal, but it does let you do it in fully standards compliant C++11 land. And can give you an interface much like "wait for multiple objects" from windows. (In fact, on windows, you could do away with much of the machinery if the native_handle of your C++11 synchronization primitives are amenable).
I'm working on an application that has a main thread performing some work (message loop of the UI etc.), but I would also like a second thread, which would periodically test if there are any updates available to download. I would also like the possibility for the main thread to ask the secondary thread to force checking for updates, and for the secondary thread to ask the main thread for confirmation on downloading updates.
I don't have that much experience with IPC and multithreading in real life situations, so I'm not sure how I should go about designing this. I would like to eventually have this work on both Windows and POSIX, but let us focus on POSIX for now. Here's my idea so far:
Secondary thread pseudocode:
repeat forever:
check_for_updates()
if (are_any_updates()) {
put the list of available updates on some message queue
send signal SIGUSER1 to main thread
wait for response from that message queue
if (response is positive) download_updates()
}
unblock signal SIGUSER1 on secondary thread
Sleep(one hour)
block signal SIGUSER1
if (any_signal_was_received_while_sleeping)
any_signal_was_received_while_sleeping := false
Sleep(one more hour)
SIGUSER1 handler on secondary thread (main thread has requested us to check for updates):
block signal SIGUSER1 (making sure we don't get signal in signal)
any_signal_was_received_while_sleeping := true
check_for_updates()
...
unblock signal SIGUSER1
Basically, main thread uses SIGUSER1 to ask the secondary thread to force checking for updates, while secondary thread uses SIGUSER1 to ask the main thread to look into the message queue for the available updates and to confirm whether they should be downloaded or not.
I'm not sure if this is a good design or if it would even work properly. One of my problems is related to handling SIGUSER1 received in the main thread, because it's a pretty big application and I'm not really sure when is the right time to block and unblock it (I assume it should be somewhere in the message loop).
Any opinion is appreciated, including advice on what IPC features should I use on Windows (maybe RPC instead of signals?). I could completely remove the use of message queue if I settled on threads, but I might consider using processes instead. I'll clearly use threads on Windows, but I'm not sure about POSIX yet.
You should strongly consider using boost::thread to solve your problem. It is far more comprehensible than directly using posix and is cross platform. Take the time to use a better tool and you will end up saving yourself a great deal of effort.
In particular I think you will find that a condition variable would neatly facilitate your simple interaction.
EDIT:
You can do almost anything with the correct use of mutexes and condition variables. Another piece of advice would be to encapsulate your threads inside class objects. This allows you to write functions that act on the thread and it's data. In your case the main thread could have a method like requestUpdateConfirmation(), inside this you can block the calling thread and wait for the main thread to deal with the request before releasing the caller.
I would like to spawn off threads to perform certain tasks, and use a thread-safe queue to communicate with them. I would also like to be doing IO to a variety of file descriptors while I'm waiting.
What's the recommended way to accomplish this? Do I have to created an inter-thread pipe and write to it when the queue goes from no elements to some elements? Isn't there a better way?
And if I have to create the inter-thread pipe, why don't more libraries that implement shared queues allow you to create the shared queue and inter-thread pipe as a single entity?
Does the fact I want to do this at all imply a fundamental design flaw?
I'm asking this about both C++ and Python. And I'm mildly interested in a cross-platform solution, but primarily interested in Linux.
For a more concrete example...
I have some code which will be searching for stuff in a filesystem tree. I have several communications channels open to the outside world through sockets. Requests that may (or may not) result in a need to search for stuff in the filesystem tree will be arriving.
I'm going to isolate the code that searches for stuff in the filesystem tree in one or more threads. I would like to take requests that result in a need to search the tree and put them in a thread-safe queue of things to be done by the searcher threads. The results will be put into a queue of completed searches.
I would like to be able to service all the non-search requests quickly while the searches are going on. I would like to be able to act on the search results in a timely fashion.
Servicing the incoming requests would generally imply some kind of event-driven architecture that uses epoll. The queue of disk-search requests and the return queue of results would imply a thread-safe queue that uses mutexes or semaphores to implement the thread safety.
The standard way to wait on an empty queue is to use a condition variable. But that won't work if I need to service other requests while I'm waiting. Either I end up polling the results queue all the time (and delaying the results by half the poll interval, on average), blocking and not servicing requests.
Whenever one uses an event driven architecture, one is required to have a single mechanism to report event completion. On Linux, if one is using files, one is required to use something from the select or poll family meaning that one is stuck with using a pipe to initiate all none file related events.
Edit: Linux has eventfd and timerfd. These can be added to your epoll list and used to break out of the epoll_wait when either triggered from another thread or on a timer event respectively.
There is another option and that is signals. One can use fcntl modify the file descriptor such that a signal is emitted when the file descriptor becomes active. The signal handler may then push a file-ready message onto any type of queue of your choosing. This may be a simple semaphore or mutex/condvar driven queue. Since one is now no longer using select/poll, one no longer needs to use a pipe to queue none file based messages.
Health warning: I have not tried this and although I cannot see why it will not work, I don't really know the performance implications of the signal approach.
Edit: Manipulating a mutex in a signal handler is probably a very bad idea.
I've solved this exact problem using what you mention, pipe() and libevent (which wraps epoll). The worker thread writes a byte to its pipe FD when its output queue goes from empty to non-empty. That wakes up the main IO thread, which can then grab the worker thread's output. This works great is actually very simple to code.
You have the Linux tag so I am going to throw this out: POSIX Message Queues do all this, which should fulfill your "built-in" request if not your less desired cross-platform wish.
The thread-safe synchronization is built-in. You can have your worker threads block on read of the queue. Alternatively MQs can use mq_notify() to spawn a new thread (or signal an existing one) when there is a new item put in the queue. And since it looks like you are going to be using select(), MQ's identifier (mqd_t) can be used as a file descriptor with select.
It seems nobody has mentioned this option yet:
Don't run select/poll/etc. in your "main thread". Start a dedicated secondary thread which does the I/O and pushes notifications into your thread-safe queue (the same queue which your other threads use to communicate with the main thread) when I/O operations complete.
Then your main thread just needs to wait on the notification queue.
Duck's and twk's are actually better answers than doron's (the one selected by the OP), in my opinion. doron suggests writing to a message queue from within the context of a signal handler, and states that the message queue can be "any type of queue." I would strongly caution you against this since many C library/system calls cannot safely be called from within a signal handler (see async-signal-safe).
In particuliar, if you choose a queue protected by a mutex, you should not access it from a signal handler. Consider this scenario: your consumer thread locks the queue to read it. Immediately after, the kernel delivers the signal to notify you that a file descriptor now has data on it. You signal handler runs in the consumer thread, necessarily), and tries to put something on your queue. To do this, it first has to take the lock. But it already holds the lock, so you are now deadlocked.
select/poll is, in my experience, the only viable solution to an event-driven program in UNIX/Linux. I wish there were a better way inside a mutlithreaded program, but you need some mechanism to "wake up" your consumer thread. I have yet to find a method that does not involve a system call (since the consumer thread is on a waitqueue inside the kernel during any blocking call such as select).
EDIT: I forgot to mention one Linux-specific way to handle signals when using select/poll: signalfd(2). You get a file descriptor you can select/poll on, and you handling code runs normally instead of in a signal handler's context.
This is a very common seen problem, especially when you are developing network server-side program. Most Linux server-side program's main look will loop like this:
epoll_add(serv_sock);
while(1){
ret = epoll_wait();
foreach(ret as fd){
req = fd.read();
resp = proc(req);
fd.send(resp);
}
}
It is single threaded(the main thread), epoll based server framework. The problem is, it is single threaded, not multi-threaded. It requires that proc() should never blocks or runs for a significant time(say 10 ms for common cases).
If proc() will ever runs for a long time, WE NEED MULTI THREADS, and executes proc() in a separated thread(the worker thread).
We can submit task to the worker thread without blocking the main thread, using a mutex based message queue, it is fast enough.
epoll_add(serv_sock);
while(1){
ret = epoll_wait();
foreach(ret as fd){
req = fd.read();
queue.add_job(req); // fast, non blockable
}
}
Then we need a way to obtain the task result from a worker thread. How? If we just check the message queue directly, before or after epoll_wait().
epoll_add(serv_sock);
while(1){
ret = epoll_wait(); // may blocks for 10ms
resp = queue.check_result(); // fast, non blockable
foreach(ret as fd){
req = fd.read();
queue.add_job(req); // fast, non blockable
}
}
However, the checking action will execute after epoll_wait() to end, and epoll_wait() usually blocks for 10 micro seconds(common cases) if all file descriptors it waits are not active.
For a server, 10 ms is quite a long time! Can we signal epoll_wait() to end immediately when task result is generated?
Yes! I will describe how it is done in one of my open source project:
Create a pipe for all worker threads, and epoll waits on that pipe as well. Once a task result is generated, the worker thread writes one byte into the pipe, then epoll_wait() will end in nearly the same time! - Linux pipe has 5 us to 20 us latency.
In my project SSDB(a Redis protocol compatible in-disk NoSQL database), I create a SelectableQueue for passing messages between the main thread and worker threads. Just like its name, SelectableQueue has an file descriptor, which can be wait by epoll.
SelectableQueue: https://github.com/ideawu/ssdb/blob/master/src/util/thread.h#L94
Usage in main thread:
epoll_add(serv_sock);
epoll_add(queue->fd());
while(1){
ret = epoll_wait();
foreach(ret as fd){
if(fd is queue){
sock, resp = queue->pop_result();
sock.send(resp);
}
if(fd is client_socket){
req = fd.read();
queue->add_task(fd, req);
}
}
}
Usage in worker thread:
fd, req = queue->pop_task();
resp = proc(req);
queue->add_result(fd, resp);
C++11 has std::mutex and std::condition_variable. The two can be used to have one thread signal another when a certain condition is met. It sounds to me like you will need to build your solution out of these primitives. If you environment does not yet support these C++11 library features, you can find very similar ones at boost. Sorry, can't say much about python.
One way to accomplish what you're looking to do is by implementing the Observer Pattern
You would register your main thread as an observer with all your spawned threads, and have them notify it when they were done doing what they were supposed to (or updating during their run with the info you need).
Basically, you want to change your approach to an event-driven model.
At the moment I am using a producer consumer model for the rendering portion of a realtime graphics application. The consumer will continually look for data in our queue(infinite loop); however I am fearful that this may cause my simulation to get out of sync of the main loop. I think this is the fast producer slow consumer problem - compounded by the fact that the simulation is restrained to a certain amount of time.
Question - what is the best method to keep this all in balance and make sure the consumer has enough time to finish, but also that the simulation does not move to the next frame before we are finished rendering our current frame(or at least be able to detect this and skip rendering the next frame - or interrupt the current frame being rendered) I am currently just interrupting and joining after each consumer is finished
Second Question: if you look at the code below you will see that I am currently just calling interrupt and join after adding rendering jobs to the queue - this allows the thread all the time it needs to complete its operation, and to respond to the interrupt when finished. How can I then reuse threads in a thread pool after interrupt_all and join_all are called? (i.e. if i call drawNextFrame again)
The producer is part of the main thread of execution (I don't think this affects anything)
pseudo code:
void renderSystem::init()
create queue to hold work;
create consumer threads of type RenderConsumer set to watch our queue;
add threads to thread_pool of consumers called 'RenderThreads'
void renderSystem::drawNextFrame()
for each thread in 'RenderThreads' divy up work;
add work assignment to queue;
//RenderThreads will now successfully start pulling data from our queue
renderThreads.interupt_all();
renderThreads.join_all();
int main()
renderer = renderSystem class;
renderer.init()
while(not_gameover)
renderer.drawNextFrame();
doOtherCoolStuff();
profit(?)
return(0)
if you need to look at the consumer class see below:
pseudo code:
RenderConsumer::operator () ()
while(true)
try to dequeue from queue
//digest any packet we get
for each ( pixel in packet )
computePrettyStuff()
//we are now done with packet that we got
this_thread::interruption_point();
I tried to make this simple and quick to digest, thank you for your time
#1. I would do this by counting the amount in the queue after each render. If it too high, then either
a. Dump the queue
b. Set a boolean variable to false
That variable will be shared between the threads, and when the producer sees that it is false, it begins waiting on a condition variable. The consumer then notifies the producer when the queue is down to an acceptable level again.
#2. Probably not possible with join_all, as the postcondition to join_all is
Every thread in the group has
terminated.
according to the reference.
It might however be possible, using barriers instead of join_all, but then you would have to find a way to provide them data, which would invariably end up needing some more shared variables.
Simple question, I think.
I have one thread that responds to a callback that is called when a user connects via TCP. That callback wants an answer if I accept or reject the login. Problem is I have to send a login request to a security server via asynchronous message passing and wait for a response.
What is the best way to handle this? Right now I have some code that just loops in the callback testing to see if the security server has sent a reply and when it comes in I read it and return the appropriate boolean. It just seems kind of gross.
TIA
First of all, you need a locking library that contains a monitor class with the ability to:
acquire a lock, guaranteeing mutual exclusion, i.e. that only one thread can hold the lock at any time
sleep on the lock, which means releasing the lock temporarily, and sleeping until the lock can be reacquired
signal on the lock, notifying sleeping threads that they should wake up and reacquire the lock. It is only possible to signal on the lock while holding the lock. This means that the signal will never have the immediate effect of waking up other threads. Typically the signaling thread will signal and then immediately release the lock, allowing the threads that have just been signaled to wake up. Waking up has the effect of a return of the blocking call to sleep, with the lock reacquired.
So with that functionality available from some library, you need to implement a security server proxy that uses the asynchronous security server to implement a synchronous service. When the synchronous authenticate() function is called in some thread (denoted thread 1), this is what should happen:
proxy acquires the lock
proxy sends a request message to the security server
proxy goes to sleep, waiting for the result, thread 1 is now blocked and the lock is available
security server computes a result
security server sends a message with the result to the proxy
the proxy message handler function is called in thread 2, thread 1 is still blocked
the proxy acquires the lock in thread 2
the proxy retrieves the result from the message and stores it in a member variable
the proxy signals on the lock, causing thread 1 blocking on sleep to try to wake up, but it can't, because thread 2 still holds the lock (inside the sleep() function, thread 2 is now blocked on a call to acquire the lock)
the proxy message handler releases its lock
the sleep call in thread 1 reacquires the lock and returns
the synchronous function in thread 1 then immediately releases its lock and returns the result
The last part with thread 1 reacquiring the lock only to immediately release it may seem pointless, but it matters because this ensures that the message handler is done before the synchronous function proceeds.
In pseudocode, it actually looks a lot simpler that you might expect:
class SecutityProxy
{
public:
SecutityProxy( SecurityServer& server ) : m_server(server)
{}
Result authenticate( username, password )
{
m_monitor.lock();
m_server.send_message( username, password );
m_monitor.sleep();
m_monitor.unlock();
return m_result;
}
void message_received( message )
{
m_monitor.lock();
m_result = message;
m_monitor->signal();
m_monitor.unlock();
}
private:
SecurityServer& m_server;
Monitor m_monitor;
Result m_result;
};
Note that this implementation cannot handle more than one request at a time! In order to handle multiple concurrent requests, you need to be able to store multiple results. You also need to store the thread handles of the threads that correspond to each request. In the message handler, you need to figure out which thread is blocking on any given request, and then just wake up the relevant thread in the signal() call, the locking library must support this.
Note also that it is highly recommended to implement a RAII class to handle the lock() and unlock() calls on the monitor.
Presumably, you would block on your asynchronous call to the security server to effectively make it synchronous.
In the function that initiates the login check, after sending the message to request the check block on something. A Windows Event would work, as would a boolean flag and a boost::condition_variable or a boost::unique_future.
In the code that receives the response message from the security server set the event, or future or flag/condition variable. This will then wake up the initial function and allow it to return the appropriate response to the initial caller.