At my multithread application I using AF_UNIX type of socket. Seems by default its blocking. The question is whats mean by "blocking"? Is it block the thread when it execute the ::recv or ::send calls or all threads on the application (like fgets do)?
If it block all threads/whole application, I guess need to use non-blocking sockets? If so, please, give the good example of how to set up AF_UNIX non-blocking socket and when need to set non-blocking mode (and how). How to ::recv ? thanks.
Blocking calls make the thread wait for the operation to complete. Use them when your thread cannot continue before the operation has completed, for example due to the data dependency on the input being received.
Non-blocking calls return as soon as the information is buffered for transmission, or the read operation is initiated. Use them when there are no data dependencies.
In general, blocking always means "blocks the current thread", not "block all threads in my process.
It only blocks the thread that makes the recv call.
Related
When my socket connection is terminated normally, then it works fine. But there are cases where the normal termination does not occur and the remote side of the connection simply disappears. When this happens, the sending task gets stuck in send() because the other side has stopped ack'ing the data. My application has a ping request/response going on and so, in another thread, it recognizes that the connection is dead. The question is...what should this other thread do in order to bring the connection to a safe termination. Should it call close()? I see SIGPIPE thrown around when this happens and I just want to make sure I am closing the connection in a safe way. I just don't want it to crash...I don't care about the leftover data. I am using a C++ library that is using synchronous sockets, so moving to async is not an easy option for me.
I avoid this problem by setting setting SIGPIPE to be ignored, and setting all my sockets to non-blocking I/O mode. Once a socket is in non-blocking mode, it will never block inside of send() or recv() -- rather, in any situation where it would normally block, it will instead immediately return -1 and set errno to EWOULDBLOCK instead. Therefore I can never "lose control" of the thread due to bad network conditions.
Of course if you never block, how do you keep your event loop from spinning and using up 100% of a core all the time? The answer is that you can block waiting for I/O inside of a separate call that is designed to do just that, e.g. select() or poll() or similar. These functions are designed to block until any one of a number of sockets becomes ready-to-read (or optionally ready-for-write) or until a pre-specified amount of time elapses, whichever comes first. So by using these, you can have your thread wake up when it needs to wake up and also sleep when there's nothing to do.
Anyway, once you have that (and you've made sure that your code handles short reads, short writes, and -1/EWOULDBLOCK gracefully, as those happen more often in non-blocking mode), you are free to implement your dead-network-detector in any of several ways. You could implement it within your network I/O thread, by keeping track of how long it has been since any data was last sent or received, and by using the timeout argument to select() to cause the blocking function to wake up at the appropriate times based on that. Or you could still use a second thread, but now the second thread has a safe way to wake up the first thread: by calling pipe() or socketpair() you can create a pair of connected file descriptors, and your network I/O thread can select()/poll() on the receiving file descriptor while the other thread holds the sending file descriptor. Then when the other thread wants to wake up the I/O thread, it can send a byte on its file descriptor, or just close() it; either one will cause the network I/O thread to return from select() or poll() and find out that something has happened on its receiving-file-descriptor, which gives it the opportunity to react by exiting (or taking whatever action is appropriate).
I use this technique in almost all of my network programming, and I find it works very well to achieve network behavior that is both reliable and CPU-efficient.
I had a lot of SIGPIPE in my application. Those are not really important: they just tells you that a Pipe (here a SOCKET) is no more available.
I do then, in my main function
signal(SIGPIPE, SIG_IGN);
Another option is to use MSG_NOSIGNAL flag for send, e.g. send(..., MSG_NOSIGNAL);. In that case SIGPIPE is not sent, the call returns -1 and errno == EPIPE.
My application is libevent-like, using epoll (in level-triggered mode) to detect I/O activity and invoke callbacks to handle it.
I have recently discovered that my TCP/IP sockets are blocking, which was an accident, but still I wouldn't expect a recv() call to block on an FD which was reported by epoll as having read activity pending. Even if there were an error on the socket, surely recv() would return and tell me about it.
What am I misunderstanding about this?
What sort of network condition could lead to recv() blocking in such a scenario?
From the Linux select man-page:
Under Linux, select() may report a socket file descriptor as "ready
for reading", while nevertheless a subsequent read blocks. This
could for example happen when data has arrived but upon examination
has wrong checksum and is discarded. There may be other
circumstances in which a file descriptor is spuriously reported as
ready. Thus it may be safer to use O_NONBLOCK on sockets that should
not block.
(yeah, I know epoll() is not the same as select(), but I suspect the same underlying conditions apply to both)
I think if you really want to avoid blocking, the only safe way to accomplish that is to set your socket to non-blocking mode.
If you use Epoll to poll for EPOLLIN event, then a recv call after that should return immediately. Furthermore, I hope you are making use of non blocking sockets. In case you want to look for errors, then you can look for EPOLLERR events. In case the socket gets closed after epoll signals, then recv should fail. Code snippet of your epoll_wait, epoll_ctl and socket creation will be helpful in debugging the problem.
I have read that you are not supposed to use send() on a blocking socket from multiple threads, but I do not know why! And if I want to use send() from multiple threads, is there anything I can do to allow it?
I am using Windows.
The fundamental reason is that synchronous I/O functions use the handle object (sockets are implemented as handles) to keep track of whether the I/O is complete or not.
The result is that if you try to send() to the same socket from multiple threads simultaneously, send() is liable to (a) hang or (b) exit before the I/O is actually complete, with catastrophic results.
You can use a critical section to prevent the sends from overlapping, or have a designated thread that reads data to send from a queue.
Note that this only applies if the sends are to the same socket. Sending to different sockets simultaneously is fine.
This question already has answers here:
How to exit a blocking recv() call? [duplicate]
(2 answers)
Closed 7 years ago.
I have a socket listener which hangs on recv function:
size_t recvLen = recv(sock, buf, 512, 0);
I would like to terminate this thread with interrupting it. MSDN says:
When issuing a blocking Winsock call
such as recv, Winsock may need to wait
for a network event before the call
can complete. Winsock performs an
alertable wait in this situation,
which can be interrupted by an
asynchronous procedure call (APC)
scheduled on the same thread.
How can I do that?
You can interrupt it by queuing an APC to it via QueueUserAPC. However, it's most likely unsafe to terminate the thread in the APC. Queuing an APC doesn't end the recv, it just interrupts it; once the APC returns, it will go back to waiting on recv again.
If you want to stop the recv completely, you should be using select with a timeout to wait until data is available. You can then check whether you should keep waiting for data or continue at each timeout.
If you don't want to receive any more data, you can kill the socket at anytime. Just call close() on it, the function in question will immediately return an error.
What I've done in the past is just run another thread with a timeout, after the waiting period if a "don't die" flag isn't set kill the socket.
Checking the socket buffer prior to recv is more flexible rather than covering a lot for select() support, I think. You can call ioctlsocket(SockHandle, FIONREAD, Longint(CountInBuffer)) to see if there's data in network buffer to read and then call recv(SockHandle, buff, CountInBuffer, 0). This way you can make a single recv call to read the whole network read buffer if you allocate the buff itself enough with CountInBuffer. Otherwise, you need to call recv in a loop to read the network buffer, which is the traditional way. In both cases, you're still in the constraints of the CountInBuffer.
I think the best way to handle this problem is to put the socket into non-blocking I/O mode, so that the thread will never block inside recv() (or in send(), for that matter). The thread should only ever block inside select() (or WaitMultipleObjects()). That way the select() (or WaitMultipleObjects()) call will return if data arrives for the socket (in which case you can then call recv() to get the new data without blocking), but you can also have select()/WaitMultipleObjects() return when something else happens; e.g. when it gets a prompt from the main thread. If you are using select(), that prompt can be the main thread sending a byte on a different socket-pair (with the main thread holding one end of the socket-pair, and the I/O thread holding the other end); if you are using WaitMultipleObjects() then I believe you can use any of the standard Windows event/signaling methods that would cause WaitMultipleObjects() to return.
Errrr... Perform an APC on the same thread? :-))
Seriously, though, it's not clear what your objective here is.
If you just want to end the thread, use the TerminateThread function.
If you want to interrupt this particular call, you can close the socket.
The only way to really interrupt a blocking recv() call and make it fully exit is to close the socket from another thread context than the one that is blocked. If that is not an option, then you need to re-write your socket logic. Best to use non-blocking or asynchronous I/O, that way recv() (or WSARecv() for the latter) will never block, and you can do whatever you need to do (like checking for thread termination conditions) while the reading is performed in the background.
I use Winsock 2 in C++, and wonder how to make my server stop reading from the client connection. The reading thread gets blocked in recv() and I have no idea how to abort it. One way to do this is use non-blocking sockets with select(), but this thread has to wait before checking the new select().
What is the proper way to stop reading the socket?
To abort the blocking call to recv(), you can close the socket with closesocket() from another thread. Even if it's a bit ugly, it should work.
You can try to shutdown() the socket too (I never tested that).
If your program has other things to do besides working with socket I/O, you should not block in the first place.
You claim your thread must wait, but that's just a reflection of your program's current design. You should redesign it so it works with some form of non-blocking sockets. Every networking problem can be addressed using non-blocking sockets.
Since you're using Winsock, you have many alternatives here, not just select(). select() is a good choice only if your application must run on many platforms and you are unable to use the superior (but mutually incompatible) alternatives available on every modern platform.