I have a Windows service written in C++ that functions as a TCP server listening for incoming connections.
I initialized the server socket and put the accept code in a separate thread. This will accept and process the incoming connections.
However, I also need to stop this thread in case the service receives the STOP signal. So I thought of creating an event object using CreateEvent and waiting for it to be signaled. This waiting would happen in the thread that creates the accept thread. So I could use the TerminateThread function to stop the accept thread when the STOP signal is received.
However, MSDN says that
TerminateThread is a dangerous function that should only be used in the most extreme cases.
How strictly should this be followed and is my approach correct? What could be another way of doing this?
In Windows, you can wake up a blocking accept call from another thread simply by calling closesocket. The blocking accept call will return -1 and your code has a chance to break out of whatever loop it is in by checking some other exit condition that you have already set (e.g. global variable)
This also works with Mac (and likely BSD derivatives) with the close function, but not Linux. The more universal UNIX solution to this problem is here.
Some pseduo code for the Windows solution below.
SOCKET _listenSocket;
bool _needToExit = false;
HANDLE _hThread;
void MakeListenThreadExit()
{
_needToExit = true;
closesocket(_listenSocket);
_listenSocket = INVALID_SOCKET;
// wait for the thread to exit
WaitForSingleObject(_hThread, INFINITE);
}
DWORD __stdcall ListenThread(void* context)
{
while (_needToExit == false)
{
SOCKET client = accept(_listenSocket, (sockaddr*)&addr, &addrSize);
if ((client == -1) || _needToExit)
{
break;
}
ProcessClient(client);
}
return 0;
}
In this situation, don't use accept() on a blocking socket. Use a non-blocking socket instead. Then you can use select() with a timeout so your thread can check for a termination condition periodically. Or better, use WSACreateEvent() with WSASelectEvent(). Create two event objects, one to detect client connections, and one to detect thread termination. You can then use WSAWaitForMultipleEvents() to wait on both events at the same time. Use WSASetEvent() to signal the termination event when needed, and call accept() or WSAAccept() whenever the other event is signalled. WSAWaitForMultipleEvents() will tell you which event to act on.
Related
I want a server to accept a client with this code
SOCKADDR_IN cliaddr = { 0 };
int len = sizeof(cliaddr);
_sockClient = accept(_sockServer, (SOCKADDR*)&cliaddr, &len);
Both _sockClient and _sockServer are SOCKETs, _sockServer's SOCKET already connected to the server.
Before I accept, I want to check _sockServer if there is an incoming client request to join. I'm pretty sure there is a way to do this, though I don't know how. Does anyone know?
By default, a socket operates in blocking mode. So, you could simply call accept() and let it block the calling thread until a new connection is accepted. That is the easiest approach. If you don't want to block the calling thread, you could call accept() in another thread instead.
But, provided you don't want to block any thread at all, then yes, there are other ways to handle this, depending on your programming model:
Poll the socket using the readfds parameter of select(). The socket will be in a readable state if it has any pending connections waiting to be accepted.
Use WSAEventSelect() to signal a waitable WSAEVENT object whenever an FD_ACCEPT event occurs, indicating that pending connections are available to accept. You can then poll/wait on that object using WSAWaitForMultipleEvents(), and use WSAEnumNetworkEvents() to reset the object's state for the next wait.
Use WSAAsyncSelect() to receive a window message whenever an FD_ACCEPT event occurs.
Use AcceptEx() instead of accept() to start an asynchronous acceptance in the background. It will report its completion to you via Overlapped I/O or an I/O Completion Port.
See Overlapped I/O and Event Objects in Winsock's documentation for more details.
I 'm using boost beast 1.74.0. in another thread i try close the websocket but the code is broken at "acceptor.accept(socket, endpoint)" and i receive "Signal: SIG32 (Real-time event 32)" after call close.
Part from code to listen connection, What i need change to interrupt the accept correctly the service?
...
_acceptor = &acceptor;
_keepAlive = true;
while (_keepAlive) {
tcp::socket socket{ioc};
// Block until we get a connection
acceptor.accept(socket, endpoint);
// Launch the session, transferring ownership of the socket
std::thread(
&WebSocketServer::doSession,
std::move(socket),
this,
this,
getHeaderServer()
).detach();
}
close function call by another thread
void WebSocketServer::close() {
if (_acceptor != nullptr) this->close();
_keepAlive = false;
}
glibc uses SIG32 to signal the cancellation of threads created using the pthread library. Are you trying to use pthread_kill?
If not, you may be witnessing that only because you are running it under GDB. Which should be fixable by telling GDB to ignore that:
handle SIG32 nostop noprint
Finally to the original question:
there's interupption points in Boost Thread. They could help you iff you can switch to Boost Thread boost::thread instead of std::thread. Also, you have to change the thread's code to actually check for interruptions: https://www.boost.org/doc/libs/1_75_0/doc/html/thread/thread_management.html#thread.thread_management.tutorial.interruption
Since it actually sounds like you want to terminate the accept loop, why not "simply" cancel the acceptor? I'm not entirely sure this works with synchronous operations, but you could of course easily use an async accept.
Take care to synchronize access to the acceptor object itself. This means either run cancel on the same thread doing async_accept or from the same strand. By this point it surely sounds like it's easier to just do the whole thing asynchronously.
Say that I have two threads, the main thread and a thread that is dedicated to continuously listening on a standard TCP socket. Now, say that at some point I want to shutdown everything. From the main thread, I would like to close the connection the listening thread is working on, then join the thread and end the program.
However, this is tricky, since I don't know how to make the listening thread return from the call to read. That call won't return unless something actually is received, and in principle I could be waiting for a long long time until the other endpoint decides to send me something.
When I was working with UDP sockets, I used to work around this problem by sending a packet on that port from my loopback interface, therefore triggering a return from recvfrom. However, this is terribly inelegant and it cannot be done on a TCP socket.
I know that another workaround could be to set a timeout with setsockopt: in this way I am guaranteed that the call will eventually return, but this is inelegant as well, and also quite inefficient since I could be waiting for several seconds before being able to join the thread.
So I was wondering if there is some way to trigger an EAGAIN on a socket read call, not unlike the one I would get on a timeout, so that on my main thread I could just call some force_returnon my socket descriptor and the call to read on the other thread would return?
I usually solve this problem by creating a pipe() and using select() in the reading thread. The reading thread must select on both the TCP socket and one end of the pipe. Whenever you want to shut down the reader, set a flag and write some data to the other end of the pipe.
Setup:
#include <unistd.h>
int signalPipe[2];
...
pipe(signalPipe);
Reader:
while(running)
{
FD_ZERO(&fds);
FD_SET(tcpSocket, &fds);
FD_SET(signalPipe[0], &fds);
select(max(tcpSocket, signalPipe[0]) + 1, &fds, NULL, NULL, NULL);
...
}
Other thread:
// We want to stop now.
running = false;
write(signalPipe[1], "foo", 3);
In my program there is one thread (receiving thread) that is responsible for receiving requests from a TCP socket and there are many threads (worker threads) that are responsible for processing the received requests. Once a request is processed I need to send an answer over TCP.
And here is a question. I would like to send TCP data in the same thread that I use for receiving data. This thread after receiving data usually waits for new data in select(). So once a worker thread finished processing a request and put an answer in the output queue it has to signal the receiving thread that there are data to send. The problem is that I don't know how to cancel waiting in select() in order to get out of waiting and to call send() .
Or shall I use another thread solely for sending data over TCP?
Updated
MSalters, Artyom thank you for you answers!
MSalters, having read your answer I found this site: Winsock 2 I/O Methods and read about WSAWaitForMultipleEvents(). My program in fact must work both on HP-UX and Windows I finally decided to use the approach that had been suggested by Artyom.
You need to use something similar to safe-pipe trick, but in your case you need to use a pair of connected TCP sockets.
Create a pair of sockets.
Add one to the select and wait on it as well
Notify by writing to other socket from other threads.
Select is immediately waken-up as one of the sockets is readable, reads all the
data in this special socket and check all data in queues to send/recv
How to create pair of sockets under Windows?
inline void pair(SOCKET fds[2])
{
struct sockaddr_in inaddr;
struct sockaddr addr;
SOCKET lst=::socket(AF_INET, SOCK_STREAM,IPPROTO_TCP);
memset(&inaddr, 0, sizeof(inaddr));
memset(&addr, 0, sizeof(addr));
inaddr.sin_family = AF_INET;
inaddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
inaddr.sin_port = 0;
int yes=1;
setsockopt(lst,SOL_SOCKET,SO_REUSEADDR,(char*)&yes,sizeof(yes));
bind(lst,(struct sockaddr *)&inaddr,sizeof(inaddr));
listen(lst,1);
int len=sizeof(inaddr);
getsockname(lst, &addr,&len);
fds[0]=::socket(AF_INET, SOCK_STREAM,0);
connect(fds[0],&addr,len);
fds[1]=accept(lst,0,0);
closesocket(lst);
}
Of course some checks should be added for return values.
select is not the native API for Windows. The native way is WSAWaitForMultipleEvents. If you use this to create an alertable wait, you can use QueueUserAPC to instruct the waiting thread to send data. (This might also mean you don't have to implement your own output queue)
See also this post:
How to signal select() to return immediately?
For unix, use an anonymous pipe. For Windows:
Unblocking can be achieved by adding a dummy (unbound) datagram socket to fd_set and then closing it. To make this thread safe, use QueueUserAPC:
The only way I found to make this multi-threadsafe is to close and recreate the socket in the same thread as the select statement is running. Of course this is difficult if the thread is blocking on the select. And then comes in the windows call QueueUserAPC. When windows is blocking in the select statement, the thread can handle Asynchronous Procedure Calls. You can schedule this from a different thread using QueueUserAPC. Windows interrupts the select, executes your function in the same thread, and continues with the select statement. You can now in your APC method close the socket and recreate it. Guaranteed thread safe and you will never loose a signal.
The typical model is for the worker to handle its own writing. Is there a reason why you want to send all the output-IO through selecting thread?
If you're sure of this model, you could have your workers send data back to the master thread using file descriptors as well (pipe(2)) and simply add those descriptors to your select() call.
And, if you're especially sure that you're not going to use pipes to send data back to your master process, the select call allows you to specify a timeout. You can busy-wait while checking your worker threads, and periodically call select to figure out which TCP sockets to read from.
Another quick&dirty solution is to add localhost sockets to the set. Now use those sockets as the inter-thread communication queues. Each worker thread simply sends something to its socket, which ends up on the corresponding socket in your receiving thread. This wakes up the select(), and your receiving thread can then echo the message on the appropriate outgoing socket.
I have a problem with a piece of legacy c++/winsock code that is part of a multi-threaded socket server. The application creates a thread that handles connections from clients, of which there are typically a couple of hundred connected at any one time. It typically runs without a problem for several days (continuously), and then suddenly stops accepting connections. This only happens in production, never test.
It uses WSAEventSelect() to detect FD_ACCEPT network events. The (simplified) code for the connection handler is:
SOCKET listener;
HANDLE hStopEvent;
// ... initialise listener and hStopEvent, and other stuff ...
HANDLE hAcceptEvent = WSACreateEvent();
WSAEventSelect(listener, hAcceptEvent, FD_ACCEPT);
HANDLE rghEvents[] = { hStopEvent, hAcceptEvent };
bool bExit = false;
while(!bExit)
{
DWORD nEvent = WaitForMultipleObjects(2, rghEvents, FALSE, INFINITE);
switch(nEvent)
{
case WAIT_OBJECT_0:
bExit = true;
break;
case WAIT_OBJECT_1:
HandleConnect();
WSAResetEvent(hAcceptEvent);
break;
case WAIT_ABANDONED_0:
case WAIT_ABANDONED_0 + 1:
case WAIT_FAILED:
LogError();
break;
}
}
From detailed logging I know that, when the problem occurs, the thread enters WaitForMultipleObjects() and never emerges, even though there are clients attempting to connect and waiting for an accept. The WAIT_FAILED and WAIT_ABANDONED_x conditions never occur.
While I haven't ruled-out a config problem on the server, or even some kind of resource leak (can't find anything), I am also wondering if the event created by WSACreateEvent() is somehow being 'dissassociated' from the FD_ACCEPT network event - causing it to never fire.
So, am I doing something wrong here? Is there something I should be doing that I'm not? Or a better way? I'd appreciate any suggestions! Thanks.
EDIT
The socket is a non-blocking socket.
EDIT
Problem solved by using the approach suggested by kipkennedy (below). Changed hAcceptEvent to be an auto-reset event, and removed the call to WSAResetEvent() which was no-longer needed.
Maybe an FD_ACCEPT is signaling during HandleConnect() after the accept() and before the return and subsequent ResetEvent(). Then, ResetEvent() ends up resetting all signals and no re-enabling accept() is ever called. For example, the following sequence is possible:
Event signaled, WaitForMultipleObjects() returns
During HandleConnect(), sometime after accept() is called, the Event is signaled again
HandleConnect() returns
ResetEvent() resets the event, masking the second signal
WaitForMultipleObjects() never returns since as far as Windows is concerned, it has already signaled the subsequent event and no subsequent accepts() have re-enabled it
A couple possible solutions: 1) loop on accept() in HandleConnect() until WSAEWOULDBLOCK is returned 2) use an auto-reset event or immediately reset the event before calling HandleConnect()
Code looks fine. The only thing I can suggest is calling WSAWaitForMultipleObjects() instead of the global version.
From reading the docs, it appears that WSAEventSelect() is as parsimonious about notifications as WSAAsyncSelect(). The stack doesn't signal FD_ACCEPT every time a connection comes in. To Winsock, the notification is its way of saying:
You called accept() earlier, and it failed with WSAEWOULDBLOCK. Go ahead and call it again, it should succeed this time.
The solution is to call accept() before calling WSAEventSelect(), and only call WSAEventSelect() after you get WSAEWOULDBLOCK. For this to work as you expect, you need to have set the listening socket to non-blocking. (That might seem obvious, but it isn't actually required.)
After an accept event occured you must not do a WSAResetEvent(hAcceptEven). You must issue a WSAEnumNetworkEvents ( listener, hAcceptEvent, &some_struct). This functions clears the internal state of the socket (ar copies this state into some_struct) and after that you can receive new connections.