I have a project where Thread A calls Accept(...) on some socket, then it passes it to another thread, which does receive on the socket and then closes the socket.
The object on which accept was called is never closed in Thread A. Is that safe?
What happens to that object in Thread A?
A socket is not an object - it's just a number (or conceivably a pointer). If you pass an integer from one thread to another, there are no issues - the same with sockets. This assumes you are asking about OS level sockets, and not some socket class that you or a framework you are using have implemented, and that you only access the socket from one of the threads, which seems to be the case.
Yes it will close. But this will not assure that the unused socket is immediately available for garbage collection (reference in ThreadA)
Sockets & Threads, a few golden rule...
Socket objects are not tread safe, make sure that you're using some sort of lock/mutex to access this resource (think synchronization)
Socket access (like any other I/O) is a blocking operation, this can create a LOT of contention (and waste a lot of CPU cycles) in case of multi-threaded access
ALWAYS explicitly close sockets (server or client), just to be double sure & not surprise yourself l8r
if you're continuously going to read data from a socket & other threads are going to consume this data, USE a thread safe data structure (a concurrent queue) for other threads to consume from. so one thread just reads data from the socket & makes it available on the queue, from where the other threads can work on the data
Related
I'm developing a program, which consists of bunch of Active Objects, that sending messages to each other. I'm using one same io_service to initialize all these objects. So they're working to end of the software life.
I'm using the Active Objects ,let's say, one for file operation, another for serial IO, another for local database connection and one to communicate all of these.
However I couldn't be sure about the objects with short lives. I'm using the short lived objects to open tcp socket to send a quick message to a remote endpoint then dispose the socket immediately. I'm thinking to make these also asynchronous.
The question is, should I use the same io_service for these short lived objects or should I create a new io_service for each socket ?
I'm developing a program, which consists of bunch of Active Objects, that sending messages to each other. I'm using one same io_service to initialize all these objects. So they're working to end of the software life.
Sounds like a good fit. I would recommend using Chris Kohlhoff's recipe if you need operations to be more efficient on machines with multiple processors.
However I couldn't be sure about the objects with short lives. I'm using the short lived objects to open tcp socket to send a quick message to a remote endpoint then dispose the socket immediately. I'm thinking to make these also asynchronous.
There's nothing wrong with having few(er)long-lived asio io_service objects (e.g. you could create the same number of io_services as there are processors on the machine), and short lived objects that use the io_service. I would say this is more efficient as well since you don't have to fire-up a thread to call io_service::run on each (short-lived?) io_service and you can avoid unnecessary context switching.
Making the sockets asynchronous is also needed if you want/need to avoid blocking in your thread(s), especially if there are network issues, etc.
How independent is the handling of UDP send and receive on same socket in Linux kernel? The use case I have is a worker thread sending UDP test traffic on (up to) 1000 sockets, and receiving the UDP replies in another worker thread. The receiver will be an epoll loop that also receives hardware send and receive timestamps on the socket error queue.
To clarify, when doing a sendmsg() syscall, will this temporarily block (or generate EAGAIN/EWOULDBLOCK) on the receiver thread receiving on the same socket? (i.e. if the send and receive happen to overlap in time) All sockets are set to non-blocking mode.
Another question is granularity of locking in the kernel - if I send and receive with sendmmsg/recvmmsg, is a lock for that socket locked once per sendmmsg, or once per UDP datagram in the sendmmsg?
UPDATE: I took a look at the original patch for sendmmsg in Linux kernel, seems the main benefit is avoiding multiple transitions user-kernel space. If any locking is done, it is probably done inside the individual calls to __sys_sendmsg:
https://lwn.net/Articles/441169/
Each system call is thread independent. So, as far as you don't involve per process kernel data, both will run independently without disturbing each other.
Another different thing is what the kernel does with system calls related to the same inode (in this case, the virtual node assigned to the socket you use to communicate) To serialize and make atomic calls to the filesystem, the kernel normally does an inode lock during the whole system call (being this a read, write or ioctl system call) that stands for the whole system call (even if you do a unique write call to write a zillion bytes, the inode is blocked during the execution of the whole system call)
In the tcp-ip stack, this is made at the socket level, and is controlled in your case by the specific AF_INET socket class software. As udp is concerned, sending a packet or receiving doesn't affect shared resources that need to be locked, but you'll have to look at your udp implementation (or socket level) to see if some locking is done and what the granularity is. Normally, the lock, should it be, should be used only during the load/unload of the udp buffers (normally there aren't buffers in udp, as the socket and network card driver are enough to supply enough buffer resources.
I am writing an audio streamer (client-server) as a project of mine (C/C++),
and I decided to make a multi threaded UDP server for this project.
The logic behind this is that each client will be handled in his own thread.
The problems I`m having are the interference of threads to one another.
The first thing my server does is create a sort of a thread-pool; it creates 5
threads that all are blocked automatically by a recvfrom() function,
though it seems that, on most of the times when I connect another device
to the server, more than one thread is responding and later on
that causes the server to be blocked entirely and not operate further.
It's pretty difficult to debug this as well so I write here in order
to get some advice on how usually multi-threaded UDP servers are implemented.
Should I use a mutex or semaphore in part of the code? If so, where?
Any ideas would be extremely helpful.
Take a step back: you say
each client will be handled in his own thread
but UDP isn't connection-oriented. If all clients use the same multicast address, there is no natural way to decide which thread should handle a given packet.
If you're wedded to the idea that each client gets its own thread (which I would generally counsel against, but it may make sense here), you need some way to figure out which client each packet came from.
That means either
using TCP (since you seem to be trying for connection-oriented behaviour anyway)
reading each packet, figuring out which logical client connection it belongs to, and sending it to the right thread. Note that since the routing information is global/shared state, these two are equivalent:
keep a source IP -> thread mapping, protected by a mutex, read & access from all threads
do all the reads in a single thread, use a local source IP -> thread mapping
The first seems to be what you're angling for, but it's poor design. When a packet comes in you'll wake up one thread, then it locks the mutex and does the lookup, and potentially wakes another thread. The thread you want to handle this connection may also be blocked reading, so you need some mechanism to wake it.
The second at least gives a seperation of concerns (read/dispatch vs. processing).
Sensibly, your design should depend on
number of clients
I/O load
amount of non-I/O processing (or IO:CPU ratio, or ...)
The first thing my server does is create a sort of a thread-pool; it creates 5 threads that all are blocked automatically by a recvfrom() function, though it seems that, on most of the times when I connect another device to the server, more than one thread is responding and later on that causes the server to be blocked entirely and not operate further
Rather than having all your threads sit on a recvfrom() on the same socket connection, you should protect the connection with a semaphore, and have your worker threads wait on the semaphore. When a thread acquires the semaphore, it can call recvfrom(), and when that returns with a packet, the thread can release the semaphore (for another thread to acquire) and handle the packet itself. When it's done servicing the packet, it can return to waiting on the semaphore. This way you avoid having to transfer data between threads.
Your recvfrom should be in the master thread and when it gets data you should pass the address IP:Port and data of the UDP client to the helper threads.
Passing the IP:port and data can be done by spawning a new thread everytime the master thread receives a UDP packet or can be passed to the helper threads through a message queue
I think that your main problem is the non-persistent udp connection. Udp is not keeping your connections alive, it exchanges only two datagrams per session. Depending on your application, in the worst case, it will have concurrent threads reading from the first available information, ie, recvfrom() will unblock even if it is not it's turn to do it.
I think the way to go is using select in the main thread and, with a concurrent buffer, manage what wich thread will do.
In this solution, you can have one thread per client, or one thread per file, assuming that you keep the clients necessary information to make sure you're sending the right file part.
TCP is another way to do it, since it keeps the connection alive for every thread you run, but is not the best transmission way on data lost allowed applications.
Alright so basically I have 3 threads.
Each one of them use a function which inside uses the send() on a specific socket (let's call it S).
Here's a small design:
How would I synchronize the S socket in such a way so that send() will be called one after another in a queue instead of the threads accessing the socket all at the same time?
First, why not have an actual queue of data to be sent, and a dedicated IO thread popping items off of it and writing to the socket?
Then, if you are using regular blocking socket semantics, you can just write to the same socket from all three threads concurrently - the kernel will provide required locking at the system call level.
I'm writing a POSIX compatible multi-threaded server in c/c++ that must be able to accept, read from, and write to a large number of connections asynchronously. The server has several worker threads which perform tasks and occasionally (and unpredictably) queue data to be written to the sockets. Data is also occasionally (and unpredictably) written to the sockets by the clients, so the server must also read asynchronously. One obvious way of doing this is to give each connection a thread which reads and writes from/to its socket; this is ugly, though, since each connection may persist for a long time and the server thus may have to hold hundred or thousand threads just to keep track of connections.
A better approach would be to have a single thread that handled all communications using the select()/pselect() functions. I.e., a single thread waits on any socket to be readable, then spawns a job to process the input that will be handled by a pool of other threads whenever input is available. Whenever the other worker threads produce output for a connection, it gets queued, and the communication thread waits for that socket to be writable before writing it.
The problem with this is that the communication thread may be waiting in the select() or pselect() function when output is queued by the worker threads of the server. It's possible that, if no input arrives for several seconds or minutes, a queued chunk of output will just wait for the communication thread to be done select()ing. This shouldn't happen, however--data should be written as soon as possible.
Right now I see a couple solutions to this that are thread-safe. One is to have the communication thread busy-wait on input and update the list of sockets it waits on for writing every tenth of a second or so. This isn't optimal since it involves busy-waiting, but it will work. Another option is to use pselect() and send the USR1 signal (or something equivalent) whenever new output has been queued, allowing the communication thread to update the list of sockets it is waiting on for writable status immediately. I prefer the latter here, but still dislike using a signal for something that should be a condition (pthread_cond_t). Yet another option would be to include, in the list of file descriptors on which select() is waiting, a dummy file that we write a single byte to whenever a socket needs to be added to the writable fd_set for select(); this would wake up the communications server because that particular dummy file would then be readable, thus allowing the communications thread to immediately update it's writable fd_set.
I feel intuitively, that the second approach (with the signal) is the 'most correct' way to program the server, but I'm curious if anyone knows either which of the above is the most efficient, generally speaking, whether either of the above will cause race conditions that I'm not aware of, or if anyone knows of a more general solution to this problem. What I really want is a pthread_cond_wait_and_select() function that allows the comm thread to wait on both a change in sockets or a signal from a condition.
Thanks in advance.
This is a fairly common problem.
One often used solution is to have pipes as a communication mechanism from worker threads back to the I/O thread. Having completed its task a worker thread writes the pointer to the result into the pipe. The I/O thread waits on the read end of the pipe along with other sockets and file descriptors and once the pipe is ready for read it wakes up, retrieves the pointer to the result and proceeds with pushing the result into the client connection in non-blocking mode.
Note, that since pipe reads and writes of less then or equal to PIPE_BUF are atomic, the pointers get written and read in one shot. One can even have multiple worker threads writing pointers into the same pipe because of the atomicity guarantee.
Unfortunately, the best way to do this is different for each platform. The canonical, portable way to do it is to have your I/O thread block in poll. If you need to get the I/O thread to leave poll, you send a single byte on a pipe that the thread is polling. That will cause the thread to exit from poll immediately.
On Linux, epoll is the best way. On BSD-derived operating systems (including OSX, I think), kqueue. On Solaris, it used to be /dev/poll and there's something else now whose name I forget.
You may just want to consider using a library like libevent or Boost.Asio. They give you the best I/O model on each platform they support.
Your second approach is the cleaner way to go. It's totally normal to have things like select or epoll include custom events in your list. This is what we do on my current project to handle such events. We also use timers (on Linux timerfd_create) for periodic events.
On Linux the eventfd lets you create such arbitrary user events for this purpose -- thus I'd say it is quite accepted practice. For POSIX only functions, well, hmm, perhaps one of the pipe commands or socketpair I've also seen.
Busy-polling is not a good option. First you'll be scanning memory which will be used by other threads, thus causing CPU memory contention. Secondly you'll always have to return to your select call which will create a huge number of system calls and context switches which will hurt overall system performance.