Threading Architecture Question C++ Message Passing - c++

Simple question, I think.
I have one thread that responds to a callback that is called when a user connects via TCP. That callback wants an answer if I accept or reject the login. Problem is I have to send a login request to a security server via asynchronous message passing and wait for a response.
What is the best way to handle this? Right now I have some code that just loops in the callback testing to see if the security server has sent a reply and when it comes in I read it and return the appropriate boolean. It just seems kind of gross.
TIA

First of all, you need a locking library that contains a monitor class with the ability to:
acquire a lock, guaranteeing mutual exclusion, i.e. that only one thread can hold the lock at any time
sleep on the lock, which means releasing the lock temporarily, and sleeping until the lock can be reacquired
signal on the lock, notifying sleeping threads that they should wake up and reacquire the lock. It is only possible to signal on the lock while holding the lock. This means that the signal will never have the immediate effect of waking up other threads. Typically the signaling thread will signal and then immediately release the lock, allowing the threads that have just been signaled to wake up. Waking up has the effect of a return of the blocking call to sleep, with the lock reacquired.
So with that functionality available from some library, you need to implement a security server proxy that uses the asynchronous security server to implement a synchronous service. When the synchronous authenticate() function is called in some thread (denoted thread 1), this is what should happen:
proxy acquires the lock
proxy sends a request message to the security server
proxy goes to sleep, waiting for the result, thread 1 is now blocked and the lock is available
security server computes a result
security server sends a message with the result to the proxy
the proxy message handler function is called in thread 2, thread 1 is still blocked
the proxy acquires the lock in thread 2
the proxy retrieves the result from the message and stores it in a member variable
the proxy signals on the lock, causing thread 1 blocking on sleep to try to wake up, but it can't, because thread 2 still holds the lock (inside the sleep() function, thread 2 is now blocked on a call to acquire the lock)
the proxy message handler releases its lock
the sleep call in thread 1 reacquires the lock and returns
the synchronous function in thread 1 then immediately releases its lock and returns the result
The last part with thread 1 reacquiring the lock only to immediately release it may seem pointless, but it matters because this ensures that the message handler is done before the synchronous function proceeds.
In pseudocode, it actually looks a lot simpler that you might expect:
class SecutityProxy
{
public:
SecutityProxy( SecurityServer& server ) : m_server(server)
{}
Result authenticate( username, password )
{
m_monitor.lock();
m_server.send_message( username, password );
m_monitor.sleep();
m_monitor.unlock();
return m_result;
}
void message_received( message )
{
m_monitor.lock();
m_result = message;
m_monitor->signal();
m_monitor.unlock();
}
private:
SecurityServer& m_server;
Monitor m_monitor;
Result m_result;
};
Note that this implementation cannot handle more than one request at a time! In order to handle multiple concurrent requests, you need to be able to store multiple results. You also need to store the thread handles of the threads that correspond to each request. In the message handler, you need to figure out which thread is blocking on any given request, and then just wake up the relevant thread in the signal() call, the locking library must support this.
Note also that it is highly recommended to implement a RAII class to handle the lock() and unlock() calls on the monitor.

Presumably, you would block on your asynchronous call to the security server to effectively make it synchronous.

In the function that initiates the login check, after sending the message to request the check block on something. A Windows Event would work, as would a boolean flag and a boost::condition_variable or a boost::unique_future.
In the code that receives the response message from the security server set the event, or future or flag/condition variable. This will then wake up the initial function and allow it to return the appropriate response to the initial caller.

Related

What is the best practice for setting a timeout value to socket poll/select?

I am using the poll mechanism to manage upto 100 connections. Is there any standard practice for what the time out value for the poll() call should be or how to determine it.
My case details -
I have one dispatcher thread listening on all the connections. Once a connection becomes read ready, I disable it for polling and forward the connfd to a thread pool processing reads. The dispatcher thread goes back to polling.
The thread pool consumes the read on the connfd and posts it back to the dispatcher so it can add it for polling next. But the dispatcher wouldn't be able to add it for polling until it returns from the poll() call. I need the dispatcher to periodically check if it needs to re-enable polling for any connfd.
What is a good timeout value so the dispatcher thread can periodically stop polling and update its pollfd list.
You don't need to use the timeout (just set it to INF).
Timeout is basically used when an explicit timer operation is needed (some async IO libraries handles this for you).
To wake up a thread sleeping in poll, use the self-pipe trick. On Linux, eventfd is also available for use.
Using timerfd (Linux only), timeout can be completely obsoleted.

Portable C++11 way to wait on a socket and bool var at the same time

I have a thread with endless loop that receieves and processes data from a socket. I want that thread to block (sleep) until the data becomes available on the socket for reading or the "exit" boolean variable becomes true (set by a different thread). Is it possible to do that in a portable way without polling and using any third-party libraries (except sockets library naturally)? If it is not possible to do in a portable way what would be the best way to do it under Windows (still NO polling and third-party libraries)?
Example code:
bool exit = false; // or "std::atomic<bool> exit" or anything else
void fn()
{
SOCKET s;
// init socket, establish connection, etc
for(;;)
{
// This thread goes to wait (blocks) until data becomes available on socket
// OR exit var is set to true (by a different thread) - how?
if(exit) break;
// receive and process data from socket
}
}
Set up a queue of messages.
These messages are of the form "PleaseExit" or "DataOnSocket".
Your thread, or task, is activated when anything shows up in the queue, processes the queue, then waits on the queue again. If it gets "PleaseExit" it instead starts cleaning up.
Possibly you will have to have a different thread/task waiting on the condition variable and on the socket to ferry the information over to your unified queue.
I say "thread or task", because having an entire thread dedicated to waiting is overkill. Sadly, C++11 threading doesn't support light weight tasks out of the box perfectly.
Basically, this solution allows a thread to wait on multiple events by delegating the waiting on each event to other threads, which send notifications "up the pipe". You could imagine creating a common infrastructure, where your thread that wants to wait on multiple objects tells the dispatch center what it is waiting for, then waits on its own condition condition variable.
The dispatch center waits on each of the things your thread wanted to wait for, and when they occur proceeds to figure out which threads should be notified, then notifies them.
Far, far from ideal, but it does let you do it in fully standards compliant C++11 land. And can give you an interface much like "wait for multiple objects" from windows. (In fact, on windows, you could do away with much of the machinery if the native_handle of your C++11 synchronization primitives are amenable).

does .push() fail or wait while locked in this code example?

Wow, I wish I'd known about this page https://github.com/zaphoyd/websocketpp/blob/experimental/examples/broadcast_server/broadcast_server.cpp written by my c++ hero https://stackoverflow.com/users/1047661/zaphoyd before I started my c++ websocket adventure. Tons of lessons there.
If I'm reading it correctly (that's a stretch lol), it looks like connections and message sends & receipts are all handled in a single thread (can't wait til he "multi-threads" that or whatever it's called since he says in this basic example http://www.zaphoyd.com/websocketpp/manual/common-patterns/server-initiated-messages that WebSocket++ handlers block core networking functions. While this program is running its send loop in on_message, no new connections are being processed and no new messages are being received.) and a separate thread boost::thread(bind(&broadcast_server::process_messages,&server)); is set up to actually process the messages while the primary websocket++ thread simply adds to the queue the necessary information.
Please clear up my lack of knowledge: will the .push() fail if it happens at the same time as this section of code in the link
while(m_actions.empty()) {
m_action_cond.wait(lock);
}
action a = m_actions.front();
m_actions.pop();
lock.unlock();
or does the .push() simply wait for the lock to be released?
std::queue<T> doesn't know anything about threading on its own; however, in the code linked, all calls to push are wrapped like the following:
boost::unique_lock<boost::mutex> lock(m_action_lock);
//std::cout << "on_open" << std::endl;
m_actions.push(action(SUBSCRIBE,hdl));
lock.unlock();
m_action_cond.notify_one();
The constructor of the lock object above internally calls m_action_lock.lock() which blocks until the lock is released.
Note that m_action_cond.wait(lock), in the code you pasted in your question, unlocks the lock while waiting for the condition, and acquires the lock again after it has been woken up (either due to a signal from another thread, or possibly spuriously), so it doesn't prevent the producer (the one doing the push) thread from acquiring the lock while it's waiting: it's only between waking up and the call to lock.unlock() that blocking occurs.

Visual c++ thread with mutex not blocking

I'm developing with VC2005, and I'm having a problem with a thread.
I have a thread that dequeue data from a queue and send it. But this thread send one petition and have to wait for the answer to send a new petition (I want to put a timeout to prevent a infinite timeout).
I have a thread like this:
while (true){
dequeue()
send()
WaitForSingleObject(ptClass->getMutex(),10000);
}
But this WaitForSingleObject never stops... I've init event like this:
HANDLE ghMutex = CreateEvent(NULL,FALSE, FALSE, "");
The idea is to block the thread to stop sending data, and when the answer comes, unblock this thread to send a new petition... Why never stops???
regards
This thread you have is waiting for the event to be SET to signaled so it can be woken up again. You have to set the event to signalled using SetEvent. Not sure where you'd do it, as I don't know your architecture, but that's what's missing.
The WaitForSingleObject is taking your thread out of CPU context, but it isn't being woken up again.
Your timeout of your Wait should be set to INFINITE if you want it to wait until the event has been set to signaled and you cannot guarantee that it will happen immediately.
You're not using a mutex, you're using a AutoResetEvent, but you have to set it!

Thread blocked waiting message

I have a pthread running and waiting for messages from a socket. The thread gets blocked to wait a message and doesn't wake up until receiving a new one. Is there a way to send a signal to thread to wake up and for the receive function (recvmsg) to return an error code related to signal?
Yes, by default SIGINT will interrupt all syscalls. From man recv:
EINTR The receive was interrupted by delivery of a signal before any
data were available; see signal(7).
and
MSG_WAITALL (since Linux 2.2)
This flag requests that the operation block until the full request is
satisfied. However, the call may still return less data than requested
if a signal is caught, an error or disconnect occurs, or the next
data to be received is of a different type than that returned.
However, you cannot target a specific thread or specific operation.
If you wish to have this, I suggest using a condition that the receiving thread can explicitely listen for. There is a wellknown trick on linux which allows the receiving thread to use select or poll to listen for the socket and the 'condition' simultaneously[1].
The trick is to open a pipe from the master thread to the client (receiving) thread. The master writes to the pipe upon reaching a certain state (the signal so to speak). The client (receiving) thread can simply poll both the pipe and the socket and only check which of the two awoke it.
[1] normally pthread_cond_wait and poll/select cannot be combined without racing so you'd need to program wait loops with small timeouts. On Win32 by contrast it is as simple as WaitForMultipleObjects and you're done