I have a question about threads communication.
there is client and server.
server:
main function- its job is to listen to some port (TCP communication) and get commands from the client
the thread job is to transmit video fluently to the client.
client:
main function- transmit commands to server
thread- watch the video
the TCP\video part works fine.
after the main function of the server, got the command from the client, I need to send the command to the video thread and send back from the video thread to the server's main- "o.k" .
the problem is to send commands from the server's main, to the video thread and vice versa.
its enough that the command will be one variable..
any ideas?
thanks!
The pipe is a bad approach towards two way communication, you could use, shared memory.
In shared memory, both processes have access to some memory that can be used read or write, such that writes in one are visible in the reads of the other and vice versa.
for more details on shared memory http://www.cs.cf.ac.uk/Dave/C/node27.html
if threads and one variable then use atomic variable. if object then use locking (trylock inside video streaming loop and lock write command inside main). if you want commands as queue-ed then use safe concurent queue
I think on your case:
I would do it with two Wait-free ring buffer from boost examples. Making two single producer -consumer.In one producer will be main function consumer other thread,on another one vice versa. (it be like using two pipes in unix but efficient one)
A wait-free ring buffer provides a mechanism for relaying objects from one single "producer" thread to one single "consumer" thread without any locks. The operations on this data structure are "wait-free" which means that each operation finishes within a constant number of steps. This makes this data structure suitable for use in hard real-time systems or for communication with interrupt/signal handlers.
Wait-free ring buffer
But considering that I am not aware of your situation there are might be more right ways.Maybe redesining at all
Related
I currently need to write a Windows 10 application that collects data from four different sensors (encoders and IMUs) via serial. I am writing this in Visual Studio in c++.
Each sensor requires me to write to it once to initialize it and then just read values. I was thinking of having a thread for each sensor (i.e. serial port) and a main thread to gather all of the data and save it in a file.
In other Stack Overflow questions, however, people have suggested having a thread to write and another to read, or a single threaded overlapped approach.
What are the pros/cons of this versus multithreaded? And if I do choose the multithreaded path do I need to use overlapped I/O or can I do non-overlapped?
Windows documentation states that for non-overlapped:
If one thread is blocked waiting for its I/O operation to complete, all other threads that subsequently call a communications API will be blocked until the original operation completes. For instance, if one thread were waiting for a ReadFile function to return, any other thread that issued a WriteFile function would be blocked.
Is this still true if I am using different serial ports for each sensor? Since I need to read these values as fast as possible I cannot afford blocking.
Any help is appreciated! This is my first time designing a system...
I'm a beginner in boost::asio.
I need to code a module which reads from a pipe and puts the data into a ring buffer (I've no problem in how to implement this part).
Another part of the module waits for a consumer to open a new TCP connection or unix domain socket and when the connection is made it sends the full ring buffer contents and then it will send any new data as soon as it is pushed into the ring buffer. Multiple consumers are allowed and one consumer can open a new connection at any time.
The first naive implementation I thought of is to keep a separate asio::streambuf for every connection and push the entire ring buffer into it on connection and then every new data, but it seems a very sub-optimal method to do it both in memory and cpu cycles as data has to be copyed for every connection, maybe multiple times as I don't know if boost::asio::send (or the linux tcp/ip stack) does a copy of the data.
As my idea is to use no multi threading at all, I'm thinking of using some form of custom asio::streambuf derived class which shares the actual buffer with the ring buffer, but keeps a separate state of the read pointer without the need of any lock.
It seems mine it is a pretty unusual need, because I'm unable to find any related documentation/question which deals with a similar subject and the boost documentation seems pretty brief and scarce to me (see e.g.: http://www.boost.org/doc/libs/1_57_0/doc/html/boost_asio/reference/basic_streambuf.html).
It would be nice if someone could point me to some ideas that I could take as starting point to implement my design or point me to an alternative design if he/she considers mine bad, un-implementable and/or improvable.
You should just do what you intend to.
You absolutely don't need a streambuf to use with Boost Asio: http://www.boost.org/doc/libs/release/doc/html/boost_asio/reference/buffer.html
If the problem is how to avoid having the producer "wait" until all consumers (read: connections) are done transmitting the data, you can always use ye olde trick of alternating output buffers.
Many ring buffer implementations allow direct splicing of a complete sequence of elements at once, (e.g. boost lockfree spsc_queue cache memory access). You could use such an operation to your advantage.
Also relevant:
TCP Zero copy using boost
It appears, that performance is a topic here.
Independent of whether boost::asio is used or some hand knitted solution, performance (throughput) might be down the the drain already by the fact (as stated in the comment section of the OP), that single bytes are being traded (read from the pipe).
After the initial "burst phase" when a consumer connects, single bytes trickle from the pipe to the connected consumer sockets with read() and write() operations per byte (or a few bytes, if the application is not constantly polling).
Given that (the fact that the price for system calls read() and write() is paid for small amounts of data), I dare theorize that anything about multiple queues or single queue etc. is already in the shadow of that basic "design flaw". I put "design flaw" in quotes as it cannot always be avoided to have to handle exactly such a situation.
So, if throughput cannot be optimized anyway, I would recommend the most simple and straightforward solution which can be conceived.
The "no threads" statement in the OP implies non-blocking file descriptors for both the accept socket, the consumer data sockets and the pipe. Will this be another 100% CPU/core eating polling application? If this is not some kind of special ops hyper-optimized problem, I would rather not advice to use non-blocking file descriptors. Also, I would not worry about zero-copy or not.
One easy approach with threads would be to have the consumer sockets non-blocking, while pipe is in blocking mode. The thread which reads the pipe then pumps the data into a queue and calls the function which services all currently connected consumers. The listen socket (the one calling accept()) is in signaled state, when new client connections are pending. With mechanisms like kqueue (bsd) or epoll (linux etc.) or WaitForMultipleObjects (windows), the pipe reader thread can react to that situation as well.
In the times when nothing is to be done, your application is sleeping/blocking and friendly to our environment :)
I am writing an audio streamer (client-server) as a project of mine (C/C++),
and I decided to make a multi threaded UDP server for this project.
The logic behind this is that each client will be handled in his own thread.
The problems I`m having are the interference of threads to one another.
The first thing my server does is create a sort of a thread-pool; it creates 5
threads that all are blocked automatically by a recvfrom() function,
though it seems that, on most of the times when I connect another device
to the server, more than one thread is responding and later on
that causes the server to be blocked entirely and not operate further.
It's pretty difficult to debug this as well so I write here in order
to get some advice on how usually multi-threaded UDP servers are implemented.
Should I use a mutex or semaphore in part of the code? If so, where?
Any ideas would be extremely helpful.
Take a step back: you say
each client will be handled in his own thread
but UDP isn't connection-oriented. If all clients use the same multicast address, there is no natural way to decide which thread should handle a given packet.
If you're wedded to the idea that each client gets its own thread (which I would generally counsel against, but it may make sense here), you need some way to figure out which client each packet came from.
That means either
using TCP (since you seem to be trying for connection-oriented behaviour anyway)
reading each packet, figuring out which logical client connection it belongs to, and sending it to the right thread. Note that since the routing information is global/shared state, these two are equivalent:
keep a source IP -> thread mapping, protected by a mutex, read & access from all threads
do all the reads in a single thread, use a local source IP -> thread mapping
The first seems to be what you're angling for, but it's poor design. When a packet comes in you'll wake up one thread, then it locks the mutex and does the lookup, and potentially wakes another thread. The thread you want to handle this connection may also be blocked reading, so you need some mechanism to wake it.
The second at least gives a seperation of concerns (read/dispatch vs. processing).
Sensibly, your design should depend on
number of clients
I/O load
amount of non-I/O processing (or IO:CPU ratio, or ...)
The first thing my server does is create a sort of a thread-pool; it creates 5 threads that all are blocked automatically by a recvfrom() function, though it seems that, on most of the times when I connect another device to the server, more than one thread is responding and later on that causes the server to be blocked entirely and not operate further
Rather than having all your threads sit on a recvfrom() on the same socket connection, you should protect the connection with a semaphore, and have your worker threads wait on the semaphore. When a thread acquires the semaphore, it can call recvfrom(), and when that returns with a packet, the thread can release the semaphore (for another thread to acquire) and handle the packet itself. When it's done servicing the packet, it can return to waiting on the semaphore. This way you avoid having to transfer data between threads.
Your recvfrom should be in the master thread and when it gets data you should pass the address IP:Port and data of the UDP client to the helper threads.
Passing the IP:port and data can be done by spawning a new thread everytime the master thread receives a UDP packet or can be passed to the helper threads through a message queue
I think that your main problem is the non-persistent udp connection. Udp is not keeping your connections alive, it exchanges only two datagrams per session. Depending on your application, in the worst case, it will have concurrent threads reading from the first available information, ie, recvfrom() will unblock even if it is not it's turn to do it.
I think the way to go is using select in the main thread and, with a concurrent buffer, manage what wich thread will do.
In this solution, you can have one thread per client, or one thread per file, assuming that you keep the clients necessary information to make sure you're sending the right file part.
TCP is another way to do it, since it keeps the connection alive for every thread you run, but is not the best transmission way on data lost allowed applications.
I'm writing a POSIX compatible multi-threaded server in c/c++ that must be able to accept, read from, and write to a large number of connections asynchronously. The server has several worker threads which perform tasks and occasionally (and unpredictably) queue data to be written to the sockets. Data is also occasionally (and unpredictably) written to the sockets by the clients, so the server must also read asynchronously. One obvious way of doing this is to give each connection a thread which reads and writes from/to its socket; this is ugly, though, since each connection may persist for a long time and the server thus may have to hold hundred or thousand threads just to keep track of connections.
A better approach would be to have a single thread that handled all communications using the select()/pselect() functions. I.e., a single thread waits on any socket to be readable, then spawns a job to process the input that will be handled by a pool of other threads whenever input is available. Whenever the other worker threads produce output for a connection, it gets queued, and the communication thread waits for that socket to be writable before writing it.
The problem with this is that the communication thread may be waiting in the select() or pselect() function when output is queued by the worker threads of the server. It's possible that, if no input arrives for several seconds or minutes, a queued chunk of output will just wait for the communication thread to be done select()ing. This shouldn't happen, however--data should be written as soon as possible.
Right now I see a couple solutions to this that are thread-safe. One is to have the communication thread busy-wait on input and update the list of sockets it waits on for writing every tenth of a second or so. This isn't optimal since it involves busy-waiting, but it will work. Another option is to use pselect() and send the USR1 signal (or something equivalent) whenever new output has been queued, allowing the communication thread to update the list of sockets it is waiting on for writable status immediately. I prefer the latter here, but still dislike using a signal for something that should be a condition (pthread_cond_t). Yet another option would be to include, in the list of file descriptors on which select() is waiting, a dummy file that we write a single byte to whenever a socket needs to be added to the writable fd_set for select(); this would wake up the communications server because that particular dummy file would then be readable, thus allowing the communications thread to immediately update it's writable fd_set.
I feel intuitively, that the second approach (with the signal) is the 'most correct' way to program the server, but I'm curious if anyone knows either which of the above is the most efficient, generally speaking, whether either of the above will cause race conditions that I'm not aware of, or if anyone knows of a more general solution to this problem. What I really want is a pthread_cond_wait_and_select() function that allows the comm thread to wait on both a change in sockets or a signal from a condition.
Thanks in advance.
This is a fairly common problem.
One often used solution is to have pipes as a communication mechanism from worker threads back to the I/O thread. Having completed its task a worker thread writes the pointer to the result into the pipe. The I/O thread waits on the read end of the pipe along with other sockets and file descriptors and once the pipe is ready for read it wakes up, retrieves the pointer to the result and proceeds with pushing the result into the client connection in non-blocking mode.
Note, that since pipe reads and writes of less then or equal to PIPE_BUF are atomic, the pointers get written and read in one shot. One can even have multiple worker threads writing pointers into the same pipe because of the atomicity guarantee.
Unfortunately, the best way to do this is different for each platform. The canonical, portable way to do it is to have your I/O thread block in poll. If you need to get the I/O thread to leave poll, you send a single byte on a pipe that the thread is polling. That will cause the thread to exit from poll immediately.
On Linux, epoll is the best way. On BSD-derived operating systems (including OSX, I think), kqueue. On Solaris, it used to be /dev/poll and there's something else now whose name I forget.
You may just want to consider using a library like libevent or Boost.Asio. They give you the best I/O model on each platform they support.
Your second approach is the cleaner way to go. It's totally normal to have things like select or epoll include custom events in your list. This is what we do on my current project to handle such events. We also use timers (on Linux timerfd_create) for periodic events.
On Linux the eventfd lets you create such arbitrary user events for this purpose -- thus I'd say it is quite accepted practice. For POSIX only functions, well, hmm, perhaps one of the pipe commands or socketpair I've also seen.
Busy-polling is not a good option. First you'll be scanning memory which will be used by other threads, thus causing CPU memory contention. Secondly you'll always have to return to your select call which will create a huge number of system calls and context switches which will hurt overall system performance.
I am designing a client and server socket program.
I have a file to be transferred to the server from the client using UDP, I repeat I am using UDP.....
I am sending through UDP so, the sending rate is too fast then the receiver, so I have created 3 threads listening on the same socket, so that when one thread is doing some work(I mean writing to a file using fwrite) with the received data the other thread can recv from the client.
My 1st question is when I am using a fwrite with multiple threads I have to use locks as the file pointer is shared between the threads. I am right in thinking???
My 2nd question is "Will there be any improvement in the performance if I use multiple threads to fwrite using locks over using a single thread to do the fwrite work with no locks...??? " ... Please guide me...
I would use one thread. Saves the complications. You can buffer the data and use asynchronous writes
http://www.gnu.org/s/hello/manual/libc/Asynchronous-Reads_002fWrites.html
Cache the data before writing it.
Let the writing happen in another thread.
Doing it the way you do will require locking the socket.
Q1: yes you do need to lock it (very slow!). Why not use a separate file descriptor in each thread? the problem comes mostly with the current file position managed by that descriptor.
Q2: Neither. If data needs ordering (yes, UDP!) you should still buffer it. RAM is much faster then disk IO. Feed a stream to buffer it and handle the data in that stream in a separate thread.
Similar to Ed's answer, I'd suggest using asynchronous I/O and a single thread for your server. Though I find using Boost.Asio easier than posix AIO.
My 1st question is when I am using a fwrite with multiple threads I have to use locks as the file pointer is shared between the threads
Yes, you always have to use locks when multiple threads are writing to a single object (file, memory, etc).
My 2nd question is "Will there be any improvement in the performance if I use multiple threads to fwrite using locks over using a single thread to do the fwrite work with no locks...??? "
I would use two threads. The first thread does nothing but read from the socket and store the data in memory. The second thread reads data from memory and writes it to the file. Treat the memory buffer as a FIFO queue and use a mutex to protect the queue pointers. You'll gain nothing from a third thread. In fact, it would probably harm performance and it definitely makes the problem far more complicated.
First, try to avoid using UDP for bulk transfers. If you use UDP you have to reinvent your own flow control protocol, as well as logic for retransmission and reordering. From the sounds of it, your problems boil down to missing flow control - so why not just use TCP?
Anyway, don't put your file writing in another thread. Modern OSes will internally buffer disk writes in any case - you'll only start blocking if you're writing data much faster than the disk can keep up, in which case buffering inside your process will only buy you another few seconds at most. Switch to TCP, or implement a proper flow control mechanism.