Alright so basically I have 3 threads.
Each one of them use a function which inside uses the send() on a specific socket (let's call it S).
Here's a small design:
How would I synchronize the S socket in such a way so that send() will be called one after another in a queue instead of the threads accessing the socket all at the same time?
First, why not have an actual queue of data to be sent, and a dedicated IO thread popping items off of it and writing to the socket?
Then, if you are using regular blocking socket semantics, you can just write to the same socket from all three threads concurrently - the kernel will provide required locking at the system call level.
Related
I have read that you are not supposed to use send() on a blocking socket from multiple threads, but I do not know why! And if I want to use send() from multiple threads, is there anything I can do to allow it?
I am using Windows.
The fundamental reason is that synchronous I/O functions use the handle object (sockets are implemented as handles) to keep track of whether the I/O is complete or not.
The result is that if you try to send() to the same socket from multiple threads simultaneously, send() is liable to (a) hang or (b) exit before the I/O is actually complete, with catastrophic results.
You can use a critical section to prevent the sends from overlapping, or have a designated thread that reads data to send from a queue.
Note that this only applies if the sends are to the same socket. Sending to different sockets simultaneously is fine.
Based on this article, calling WSASend() or WSARecv() from multiple threads is not safe. However, is it safe to call WSASend() and WSARecv() at the same time from two threads?
It is always safe to read from a socket in one thread while simultaneously writing to the same socket in another thread. Separate kernel buffers are involved. Regardless of whether you are using blocking, non-blocking, overlapped, or IOCP I/O logic.
What is not safe is simultaneously reading from the same socket in multiple threads, or simultaneously writing to the same socket in multiple threads. That requires synchronization between the threads so their transmitted/received data do not overlap each other.
I am writing an audio streamer (client-server) as a project of mine (C/C++),
and I decided to make a multi threaded UDP server for this project.
The logic behind this is that each client will be handled in his own thread.
The problems I`m having are the interference of threads to one another.
The first thing my server does is create a sort of a thread-pool; it creates 5
threads that all are blocked automatically by a recvfrom() function,
though it seems that, on most of the times when I connect another device
to the server, more than one thread is responding and later on
that causes the server to be blocked entirely and not operate further.
It's pretty difficult to debug this as well so I write here in order
to get some advice on how usually multi-threaded UDP servers are implemented.
Should I use a mutex or semaphore in part of the code? If so, where?
Any ideas would be extremely helpful.
Take a step back: you say
each client will be handled in his own thread
but UDP isn't connection-oriented. If all clients use the same multicast address, there is no natural way to decide which thread should handle a given packet.
If you're wedded to the idea that each client gets its own thread (which I would generally counsel against, but it may make sense here), you need some way to figure out which client each packet came from.
That means either
using TCP (since you seem to be trying for connection-oriented behaviour anyway)
reading each packet, figuring out which logical client connection it belongs to, and sending it to the right thread. Note that since the routing information is global/shared state, these two are equivalent:
keep a source IP -> thread mapping, protected by a mutex, read & access from all threads
do all the reads in a single thread, use a local source IP -> thread mapping
The first seems to be what you're angling for, but it's poor design. When a packet comes in you'll wake up one thread, then it locks the mutex and does the lookup, and potentially wakes another thread. The thread you want to handle this connection may also be blocked reading, so you need some mechanism to wake it.
The second at least gives a seperation of concerns (read/dispatch vs. processing).
Sensibly, your design should depend on
number of clients
I/O load
amount of non-I/O processing (or IO:CPU ratio, or ...)
The first thing my server does is create a sort of a thread-pool; it creates 5 threads that all are blocked automatically by a recvfrom() function, though it seems that, on most of the times when I connect another device to the server, more than one thread is responding and later on that causes the server to be blocked entirely and not operate further
Rather than having all your threads sit on a recvfrom() on the same socket connection, you should protect the connection with a semaphore, and have your worker threads wait on the semaphore. When a thread acquires the semaphore, it can call recvfrom(), and when that returns with a packet, the thread can release the semaphore (for another thread to acquire) and handle the packet itself. When it's done servicing the packet, it can return to waiting on the semaphore. This way you avoid having to transfer data between threads.
Your recvfrom should be in the master thread and when it gets data you should pass the address IP:Port and data of the UDP client to the helper threads.
Passing the IP:port and data can be done by spawning a new thread everytime the master thread receives a UDP packet or can be passed to the helper threads through a message queue
I think that your main problem is the non-persistent udp connection. Udp is not keeping your connections alive, it exchanges only two datagrams per session. Depending on your application, in the worst case, it will have concurrent threads reading from the first available information, ie, recvfrom() will unblock even if it is not it's turn to do it.
I think the way to go is using select in the main thread and, with a concurrent buffer, manage what wich thread will do.
In this solution, you can have one thread per client, or one thread per file, assuming that you keep the clients necessary information to make sure you're sending the right file part.
TCP is another way to do it, since it keeps the connection alive for every thread you run, but is not the best transmission way on data lost allowed applications.
I am writing a small 3 servers and 1 client program. the 2 servers send tcp messages and the last one sends upd datagrams using winsock2.
I am wondering if i can make simulanious recvfrom() by using threads (OpenMP or boost::threads) so that 2 threads listen from the same socket on the same port at the same time.
I am using VC++ 2010 on windows7.
Thank you for your help.
Yes, sockets are thread-safe, however you have to be careful. One common pattern (when using blocking IO) is to have one thread receiving data on a socket and another thread sending data on the same socket. Having multiple threads receiving data from a socket is usually fine for UDP socket, but doesn't make much sense for TCP sockets most of the time. There is a warning in the documentation for WSARecv:
WSARecv should not be called on the same socket simultaneously from
different threads, because it can result in an unpredictable buffer
order.
But this usually isn't of any concern if you are using UDP and the protocol is stateless.
Also note that the WSAEINPROGRESS error code mainly applies to Winsock 1.1:
WSAEINPROGRESS: A blocking Windows Sockets 1.1 call is in progress, or the service provider is still processing a callback function.
And the description of WSAEINPROGRESS further states:
Operation now in progress.
A blocking operation is currently executing. Windows Sockets only allows a single blocking operation—per- task or thread—to be outstanding, and if any other function call is made (whether or not it references that or any other socket) the function fails with the WSAEINPROGRESS error.
Note that this talks about a single blocking operation per-task or thread.
Furthermore there is an additional warning in the documentation for WSARecv:
Issuing another blocking Winsock call inside an APC that interrupted an ongoing blocking Winsock call on the same thread will lead to undefined behavior, and must never be attempted by Winsock clients.
But apart from those warnings you should be fine.
Update: to add some external references:
alt.winsock.programming: Is socket thread-safe?
and Winsock Programmer’s FAQ: Is Winsock thread-safe?
Winsock allows only one blocking IO call on a socket. More than one blocking call from different thread would end up with "WSAEINPROGRESS" error. http://msdn.microsoft.com/en-us/library/windows/desktop/ms740668%28v=vs.85%29.aspx#WSAEINPROGRESS.
If you want to make concurrent IO request you could try using asynchronous IO or overlapped IO (in windows parlance). But I guess you would want concurrent processing of data more than concurrent reading data. In which case you could have one thread issuing IO requests and others for processing.
I have a project where Thread A calls Accept(...) on some socket, then it passes it to another thread, which does receive on the socket and then closes the socket.
The object on which accept was called is never closed in Thread A. Is that safe?
What happens to that object in Thread A?
A socket is not an object - it's just a number (or conceivably a pointer). If you pass an integer from one thread to another, there are no issues - the same with sockets. This assumes you are asking about OS level sockets, and not some socket class that you or a framework you are using have implemented, and that you only access the socket from one of the threads, which seems to be the case.
Yes it will close. But this will not assure that the unused socket is immediately available for garbage collection (reference in ThreadA)
Sockets & Threads, a few golden rule...
Socket objects are not tread safe, make sure that you're using some sort of lock/mutex to access this resource (think synchronization)
Socket access (like any other I/O) is a blocking operation, this can create a LOT of contention (and waste a lot of CPU cycles) in case of multi-threaded access
ALWAYS explicitly close sockets (server or client), just to be double sure & not surprise yourself l8r
if you're continuously going to read data from a socket & other threads are going to consume this data, USE a thread safe data structure (a concurrent queue) for other threads to consume from. so one thread just reads data from the socket & makes it available on the queue, from where the other threads can work on the data