Using send() on a blocking socket from multiple threads - c++

I have read that you are not supposed to use send() on a blocking socket from multiple threads, but I do not know why! And if I want to use send() from multiple threads, is there anything I can do to allow it?
I am using Windows.

The fundamental reason is that synchronous I/O functions use the handle object (sockets are implemented as handles) to keep track of whether the I/O is complete or not.
The result is that if you try to send() to the same socket from multiple threads simultaneously, send() is liable to (a) hang or (b) exit before the I/O is actually complete, with catastrophic results.
You can use a critical section to prevent the sends from overlapping, or have a designated thread that reads data to send from a queue.
Note that this only applies if the sends are to the same socket. Sending to different sockets simultaneously is fine.

Related

How do I recover when a synchronous call to socket send() gets blocked due to the loss of the other end of the connection?

When my socket connection is terminated normally, then it works fine. But there are cases where the normal termination does not occur and the remote side of the connection simply disappears. When this happens, the sending task gets stuck in send() because the other side has stopped ack'ing the data. My application has a ping request/response going on and so, in another thread, it recognizes that the connection is dead. The question is...what should this other thread do in order to bring the connection to a safe termination. Should it call close()? I see SIGPIPE thrown around when this happens and I just want to make sure I am closing the connection in a safe way. I just don't want it to crash...I don't care about the leftover data. I am using a C++ library that is using synchronous sockets, so moving to async is not an easy option for me.
I avoid this problem by setting setting SIGPIPE to be ignored, and setting all my sockets to non-blocking I/O mode. Once a socket is in non-blocking mode, it will never block inside of send() or recv() -- rather, in any situation where it would normally block, it will instead immediately return -1 and set errno to EWOULDBLOCK instead. Therefore I can never "lose control" of the thread due to bad network conditions.
Of course if you never block, how do you keep your event loop from spinning and using up 100% of a core all the time? The answer is that you can block waiting for I/O inside of a separate call that is designed to do just that, e.g. select() or poll() or similar. These functions are designed to block until any one of a number of sockets becomes ready-to-read (or optionally ready-for-write) or until a pre-specified amount of time elapses, whichever comes first. So by using these, you can have your thread wake up when it needs to wake up and also sleep when there's nothing to do.
Anyway, once you have that (and you've made sure that your code handles short reads, short writes, and -1/EWOULDBLOCK gracefully, as those happen more often in non-blocking mode), you are free to implement your dead-network-detector in any of several ways. You could implement it within your network I/O thread, by keeping track of how long it has been since any data was last sent or received, and by using the timeout argument to select() to cause the blocking function to wake up at the appropriate times based on that. Or you could still use a second thread, but now the second thread has a safe way to wake up the first thread: by calling pipe() or socketpair() you can create a pair of connected file descriptors, and your network I/O thread can select()/poll() on the receiving file descriptor while the other thread holds the sending file descriptor. Then when the other thread wants to wake up the I/O thread, it can send a byte on its file descriptor, or just close() it; either one will cause the network I/O thread to return from select() or poll() and find out that something has happened on its receiving-file-descriptor, which gives it the opportunity to react by exiting (or taking whatever action is appropriate).
I use this technique in almost all of my network programming, and I find it works very well to achieve network behavior that is both reliable and CPU-efficient.
I had a lot of SIGPIPE in my application. Those are not really important: they just tells you that a Pipe (here a SOCKET) is no more available.
I do then, in my main function
signal(SIGPIPE, SIG_IGN);
Another option is to use MSG_NOSIGNAL flag for send, e.g. send(..., MSG_NOSIGNAL);. In that case SIGPIPE is not sent, the call returns -1 and errno == EPIPE.

Can select() be used with blocking sockets?

I want to use select() to monitor if a socket has data to be read, but I do not want to use non-blocking sockets. So can select() be used with blocking sockets?
I am using Windows.
Yes, this is the entire point of select.
It watches for activity on sockets that would block if you tried to read from them without knowing that data were there. Most importantly, it can watch for activity on multiple sockets, which you couldn't do without select on a blocking socket unless you had each socket handled in a separate thread. Also importantly, it tells you when a socket is ready for reading and/or for writing; simply invoking either read or write can't do that.
The behaviour of select is even documented in these terms:
select() and pselect() allow a program to monitor multiple file descriptors, waiting until one or more of the file descriptors become "ready" for some class of I/O operation (e.g., input possible). A file descriptor is considered ready if it is possible to perform the corresponding I/O operation (e.g., read(2)) without blocking.
Of course, you can also use it with non-blocking sockets, because otherwise in order to "wait" for activity you'd have to come up with a read-sleep-read-sleep-… loop and that's suboptimal for a few reasons.

is winsock2 thread safe?

I am writing a small 3 servers and 1 client program. the 2 servers send tcp messages and the last one sends upd datagrams using winsock2.
I am wondering if i can make simulanious recvfrom() by using threads (OpenMP or boost::threads) so that 2 threads listen from the same socket on the same port at the same time.
I am using VC++ 2010 on windows7.
Thank you for your help.
Yes, sockets are thread-safe, however you have to be careful. One common pattern (when using blocking IO) is to have one thread receiving data on a socket and another thread sending data on the same socket. Having multiple threads receiving data from a socket is usually fine for UDP socket, but doesn't make much sense for TCP sockets most of the time. There is a warning in the documentation for WSARecv:
WSARecv should not be called on the same socket simultaneously from
different threads, because it can result in an unpredictable buffer
order.
But this usually isn't of any concern if you are using UDP and the protocol is stateless.
Also note that the WSAEINPROGRESS error code mainly applies to Winsock 1.1:
WSAEINPROGRESS: A blocking Windows Sockets 1.1 call is in progress, or the service provider is still processing a callback function.
And the description of WSAEINPROGRESS further states:
Operation now in progress.
A blocking operation is currently executing. Windows Sockets only allows a single blocking operation—per- task or thread—to be outstanding, and if any other function call is made (whether or not it references that or any other socket) the function fails with the WSAEINPROGRESS error.
Note that this talks about a single blocking operation per-task or thread.
Furthermore there is an additional warning in the documentation for WSARecv:
Issuing another blocking Winsock call inside an APC that interrupted an ongoing blocking Winsock call on the same thread will lead to undefined behavior, and must never be attempted by Winsock clients.
But apart from those warnings you should be fine.
Update: to add some external references:
alt.winsock.programming: Is socket thread-safe?
and Winsock Programmer’s FAQ: Is Winsock thread-safe?
Winsock allows only one blocking IO call on a socket. More than one blocking call from different thread would end up with "WSAEINPROGRESS" error. http://msdn.microsoft.com/en-us/library/windows/desktop/ms740668%28v=vs.85%29.aspx#WSAEINPROGRESS.
If you want to make concurrent IO request you could try using asynchronous IO or overlapped IO (in windows parlance). But I guess you would want concurrent processing of data more than concurrent reading data. In which case you could have one thread issuing IO requests and others for processing.

Synchronizing socket send()

Alright so basically I have 3 threads.
Each one of them use a function which inside uses the send() on a specific socket (let's call it S).
Here's a small design:
How would I synchronize the S socket in such a way so that send() will be called one after another in a queue instead of the threads accessing the socket all at the same time?
First, why not have an actual queue of data to be sent, and a dedicated IO thread popping items off of it and writing to the socket?
Then, if you are using regular blocking socket semantics, you can just write to the same socket from all three threads concurrently - the kernel will provide required locking at the system call level.

Waiting on a condition (pthread_cond_wait) and a socket change (select) simultaneously

I'm writing a POSIX compatible multi-threaded server in c/c++ that must be able to accept, read from, and write to a large number of connections asynchronously. The server has several worker threads which perform tasks and occasionally (and unpredictably) queue data to be written to the sockets. Data is also occasionally (and unpredictably) written to the sockets by the clients, so the server must also read asynchronously. One obvious way of doing this is to give each connection a thread which reads and writes from/to its socket; this is ugly, though, since each connection may persist for a long time and the server thus may have to hold hundred or thousand threads just to keep track of connections.
A better approach would be to have a single thread that handled all communications using the select()/pselect() functions. I.e., a single thread waits on any socket to be readable, then spawns a job to process the input that will be handled by a pool of other threads whenever input is available. Whenever the other worker threads produce output for a connection, it gets queued, and the communication thread waits for that socket to be writable before writing it.
The problem with this is that the communication thread may be waiting in the select() or pselect() function when output is queued by the worker threads of the server. It's possible that, if no input arrives for several seconds or minutes, a queued chunk of output will just wait for the communication thread to be done select()ing. This shouldn't happen, however--data should be written as soon as possible.
Right now I see a couple solutions to this that are thread-safe. One is to have the communication thread busy-wait on input and update the list of sockets it waits on for writing every tenth of a second or so. This isn't optimal since it involves busy-waiting, but it will work. Another option is to use pselect() and send the USR1 signal (or something equivalent) whenever new output has been queued, allowing the communication thread to update the list of sockets it is waiting on for writable status immediately. I prefer the latter here, but still dislike using a signal for something that should be a condition (pthread_cond_t). Yet another option would be to include, in the list of file descriptors on which select() is waiting, a dummy file that we write a single byte to whenever a socket needs to be added to the writable fd_set for select(); this would wake up the communications server because that particular dummy file would then be readable, thus allowing the communications thread to immediately update it's writable fd_set.
I feel intuitively, that the second approach (with the signal) is the 'most correct' way to program the server, but I'm curious if anyone knows either which of the above is the most efficient, generally speaking, whether either of the above will cause race conditions that I'm not aware of, or if anyone knows of a more general solution to this problem. What I really want is a pthread_cond_wait_and_select() function that allows the comm thread to wait on both a change in sockets or a signal from a condition.
Thanks in advance.
This is a fairly common problem.
One often used solution is to have pipes as a communication mechanism from worker threads back to the I/O thread. Having completed its task a worker thread writes the pointer to the result into the pipe. The I/O thread waits on the read end of the pipe along with other sockets and file descriptors and once the pipe is ready for read it wakes up, retrieves the pointer to the result and proceeds with pushing the result into the client connection in non-blocking mode.
Note, that since pipe reads and writes of less then or equal to PIPE_BUF are atomic, the pointers get written and read in one shot. One can even have multiple worker threads writing pointers into the same pipe because of the atomicity guarantee.
Unfortunately, the best way to do this is different for each platform. The canonical, portable way to do it is to have your I/O thread block in poll. If you need to get the I/O thread to leave poll, you send a single byte on a pipe that the thread is polling. That will cause the thread to exit from poll immediately.
On Linux, epoll is the best way. On BSD-derived operating systems (including OSX, I think), kqueue. On Solaris, it used to be /dev/poll and there's something else now whose name I forget.
You may just want to consider using a library like libevent or Boost.Asio. They give you the best I/O model on each platform they support.
Your second approach is the cleaner way to go. It's totally normal to have things like select or epoll include custom events in your list. This is what we do on my current project to handle such events. We also use timers (on Linux timerfd_create) for periodic events.
On Linux the eventfd lets you create such arbitrary user events for this purpose -- thus I'd say it is quite accepted practice. For POSIX only functions, well, hmm, perhaps one of the pipe commands or socketpair I've also seen.
Busy-polling is not a good option. First you'll be scanning memory which will be used by other threads, thus causing CPU memory contention. Secondly you'll always have to return to your select call which will create a huge number of system calls and context switches which will hurt overall system performance.