I'm developing a program, which consists of bunch of Active Objects, that sending messages to each other. I'm using one same io_service to initialize all these objects. So they're working to end of the software life.
I'm using the Active Objects ,let's say, one for file operation, another for serial IO, another for local database connection and one to communicate all of these.
However I couldn't be sure about the objects with short lives. I'm using the short lived objects to open tcp socket to send a quick message to a remote endpoint then dispose the socket immediately. I'm thinking to make these also asynchronous.
The question is, should I use the same io_service for these short lived objects or should I create a new io_service for each socket ?
I'm developing a program, which consists of bunch of Active Objects, that sending messages to each other. I'm using one same io_service to initialize all these objects. So they're working to end of the software life.
Sounds like a good fit. I would recommend using Chris Kohlhoff's recipe if you need operations to be more efficient on machines with multiple processors.
However I couldn't be sure about the objects with short lives. I'm using the short lived objects to open tcp socket to send a quick message to a remote endpoint then dispose the socket immediately. I'm thinking to make these also asynchronous.
There's nothing wrong with having few(er)long-lived asio io_service objects (e.g. you could create the same number of io_services as there are processors on the machine), and short lived objects that use the io_service. I would say this is more efficient as well since you don't have to fire-up a thread to call io_service::run on each (short-lived?) io_service and you can avoid unnecessary context switching.
Making the sockets asynchronous is also needed if you want/need to avoid blocking in your thread(s), especially if there are network issues, etc.
Related
I'm writing a networking library that uses Boost asio and am confused on whether I should use a separate thread to run the io_service or not.
I currently have a class that wraps all asio work. It has one io_service, one socket, etc, and uses async_read and async_write methods to communicate with the remote server. This class exposes read and write methods to allow users to communicate with the remote server.
This class is then called by other classes that use it's read/write methods to send and receive data to the remote server. In some cases there are chained calls to read/write data from the server until a final user-provided callback is called to pass on the final result of the computation.
I'm now trying to implement a connection pool and am wondering if I need a thread pool: all reads and writes to the remote server use async methods, none post-read processing involves blocking calls until the final user-provided callback. Should it not be ok to have a series of connection objects running at the same time without the need for a separate thread pool?
If you only have one thread, then when you get the data and process it, you are blocking any other calls. Of course, if the only thing that you do in a async_read or async_write is start the next async call, then the io_service threads is always waiting for new data to arrive, and it populates the relevant connection underlying data structures. No problem with just one thread.
But you probably have some kind of processing that interacts with the read/write data, and this is the part that you can parallelize with the thread pool. So the question is: how big is the fraction of time consumed in this processing? Is it the bottleneck (latency and bandwidth) of the server?
I saw different cases here in the past. One case was a simple server working on one list of jobs to do and dispatching the data to clients. It didn't require threading, I didn't care about the latency, as the clients would come only from time to time, and no bottleneck. Then I had another case where everything needed to be processed quickly and in this instance, I used a thread pool.
So the real question is: where is the bottleneck?
Consider situation where you have 200 detectors that are connected to your program through tcp sockets. They are quite frequently sending their data and
I would like to handle it as efficiently as possible.
I can think of 2 approaches for this problem, but I'm quite new in QT so I don't know which one is better, if any.
Create a threadpool that will be running 200 objects derived from QRunnable(), each object will consist of a socket and slots that will be connected to this socket signals, so that all data concerning one detector will be handled in that one object. (In it's run() method there will be QEventLoop)
Create 200 objects, each object will consist of socket and connect those 200 socket signals to one slot in the main thread. So It will handle data from 200 detectors in one slot.
Which approach would be better consdering the fact that in the first approach there will be created 200 QEventLoops (for each object)?
There is no need to get down to epoll directly. You could use something dedicated like uvw, for example.
I think any solution can work, although I definitely recommend avoiding a thread-per-connection solution, as 200 threads is about 198 threads too many and would not be very efficient.
The way I would do it is to create one thread and run a select() (or poll() or epoll() or whatever) event loop inside that thread to handle the 200 TCP connections there, using non-blocking I/O. When data arrives in that thread, that thread can parse the data into the appropriate chunks and then send the parsed/assembled data on to the main thread (if necessary) via a Queued signal/slot connection (or qApp->postEvent() if you prefer doing it that way). (Doing the networking in a separate thread would help prevent GUI actions from interfering with network performance, and vice-versa)
Creating ~200 QTCPSocket objects in the network thread (and having the network thread run the Qt-standard QEventLoop to handle them) would also probably work well; the last time I tried that I ran into some performance issues with Qt's implementation of networking on some platforms, but that was back in the Qt4 days so I'm optimistic that Qt has improved their implementation's efficiency since then.
In all cases, you don't want more threads than there are logical processor cores. Distribute the objects across the threads. Using a QRunnable that spins an eventloop is fairly pointless, even if I admit to demonstrating it in a SO answer on someone's request. Eventloops aren't cheap either - each takes a few kilobytes of stack, at least on my platform. Thus it's better to just use QThread that has a single eventloop per thread, and then distribute the network objects across the threads in a round-robin fashion.
I am trying to learn C++ (with prior programming knowledge) by creating a server application with multiple clients. Server application will run on Raspberry Pi/Debian (Raspbian). I thought this would also be a good opportunity to learn about low-level concurrent programming with threads (e.g. POSIX). Then I came across with select() function which basically allows usage of blocking functions in a single thread to handle multiple clients, which is interesting. Some people here on StackOverflow mentioned that threads cause a lot of overhead and select() seems to be a nice alternative.
In reality, I will have 1-3 clients connected but I would like to keep my application flexible. As a structure design, I was thinking about a main thread invoking a data thread (processing stuff non-stop) and a server thread (listening for incoming connections). Since accept() call is blocking, the latter one needs to be a separate thread. If a client connects, then for each client, I may need a separate thread as well.
At the end, worker thread will write to the shared memory and client threads will read from there and communicate with the clients. Some people were opposing to the usage of threads but in my understanding, threads are good if they are invoked rarely (and long living) and if there are blocking function calls. For the last one as it seems there is the select() function, which used in a loop, allows for handling of multiple sockets in a single thread.
I think at least for the data processing and server accept() call, I will need 2 separate threads initiated at the beginning. I may handle all clients with select() in a single thread or separate threads. What would be the correct approach and are there smarter alternatives?
I am writing an audio streamer (client-server) as a project of mine (C/C++),
and I decided to make a multi threaded UDP server for this project.
The logic behind this is that each client will be handled in his own thread.
The problems I`m having are the interference of threads to one another.
The first thing my server does is create a sort of a thread-pool; it creates 5
threads that all are blocked automatically by a recvfrom() function,
though it seems that, on most of the times when I connect another device
to the server, more than one thread is responding and later on
that causes the server to be blocked entirely and not operate further.
It's pretty difficult to debug this as well so I write here in order
to get some advice on how usually multi-threaded UDP servers are implemented.
Should I use a mutex or semaphore in part of the code? If so, where?
Any ideas would be extremely helpful.
Take a step back: you say
each client will be handled in his own thread
but UDP isn't connection-oriented. If all clients use the same multicast address, there is no natural way to decide which thread should handle a given packet.
If you're wedded to the idea that each client gets its own thread (which I would generally counsel against, but it may make sense here), you need some way to figure out which client each packet came from.
That means either
using TCP (since you seem to be trying for connection-oriented behaviour anyway)
reading each packet, figuring out which logical client connection it belongs to, and sending it to the right thread. Note that since the routing information is global/shared state, these two are equivalent:
keep a source IP -> thread mapping, protected by a mutex, read & access from all threads
do all the reads in a single thread, use a local source IP -> thread mapping
The first seems to be what you're angling for, but it's poor design. When a packet comes in you'll wake up one thread, then it locks the mutex and does the lookup, and potentially wakes another thread. The thread you want to handle this connection may also be blocked reading, so you need some mechanism to wake it.
The second at least gives a seperation of concerns (read/dispatch vs. processing).
Sensibly, your design should depend on
number of clients
I/O load
amount of non-I/O processing (or IO:CPU ratio, or ...)
The first thing my server does is create a sort of a thread-pool; it creates 5 threads that all are blocked automatically by a recvfrom() function, though it seems that, on most of the times when I connect another device to the server, more than one thread is responding and later on that causes the server to be blocked entirely and not operate further
Rather than having all your threads sit on a recvfrom() on the same socket connection, you should protect the connection with a semaphore, and have your worker threads wait on the semaphore. When a thread acquires the semaphore, it can call recvfrom(), and when that returns with a packet, the thread can release the semaphore (for another thread to acquire) and handle the packet itself. When it's done servicing the packet, it can return to waiting on the semaphore. This way you avoid having to transfer data between threads.
Your recvfrom should be in the master thread and when it gets data you should pass the address IP:Port and data of the UDP client to the helper threads.
Passing the IP:port and data can be done by spawning a new thread everytime the master thread receives a UDP packet or can be passed to the helper threads through a message queue
I think that your main problem is the non-persistent udp connection. Udp is not keeping your connections alive, it exchanges only two datagrams per session. Depending on your application, in the worst case, it will have concurrent threads reading from the first available information, ie, recvfrom() will unblock even if it is not it's turn to do it.
I think the way to go is using select in the main thread and, with a concurrent buffer, manage what wich thread will do.
In this solution, you can have one thread per client, or one thread per file, assuming that you keep the clients necessary information to make sure you're sending the right file part.
TCP is another way to do it, since it keeps the connection alive for every thread you run, but is not the best transmission way on data lost allowed applications.
I'm trying to implement a two way communication using boost:asio. I'm writing the server that will communicate with multiple clients.
I want the writes and reads to and from clients to happen without any synchronization and order - the client can send a command to the server at any time and it still receives some data in a loop. Of course access to shared resources must be protected.
What is the best way to achieve this? Is having two threads - one for reading and one for writing a good option? What about accepting the connections and managing many clients?
//edit
By "no synchronization and order" I mean that the server should stream to the clients its data all the time and that it can respond(change its behaviour) to clients requests at any time regardless of what is now being sent to them.
One key idea behind asio is exactly that you don't need multiple threads to deal with multiple client sessions. Your description is a bit generic, and I'm not sure I understand what you mean by 'I want the writes and reads to and from clients to happen without any synchronization and order'.
A good starting point would be the asio chat server example. Notice how in this example an instance of the class chat_session is created for each connected client. Objects of that class keep on posting asynchronous reads as long as the connection is alive and at the same time they can write data to the connected clients. In the mean time an object of class chat_server keeps accepting new incoming client connections.
At work we're doing something conceptually very similar and there I noticed the big impact a heavy handler has on performance. The writing side of the code/write handler does too much work and occupies a worker thread for too long, thereby jeopardizing the program flow. Especially RST packets (closed connections) weren't detected quick enough by the read handler because the write actions were taking their sweet time and hogging most of the processing time in the worker thread. Currently I fixed that by creating two worker threads so that one line of code was not starved of processing time. Admittedly, this is far from ideal and it is on my lengthy to-do list of optimizations.
Long story short, you can get away with using a single thread for reading and writing if your handlers are light-weight while a second thread handles the rest of your program. Once you notice weird synchronization issues it's time to either lighten your network handlers or add an extra thread to the worker pool.