At work I have been tasked with implementing a TCP server as part of a Modbus slave device. I have done a lot of reading both here on stack exchange and on the internet in general (including the excellent http://beej.us/guide/bgnet/) but I am struggling with a design issue. In summary, my device can accept just 2 connections and on each connection will be incoming modbus requests which I must process in my main controller loop and then reply with success or failure status. I have the following ideas of how to implement this.
Have a listener thread that creates, binds, listens and accepts connections, then spawns a new pthread to listen on the connection for incoming data and close connection after an idle timeout period. If the number of active threads is currently 2, new connections are instantly closed to ensure only 2 are allowed.
Do not spawn new threads from the listener thread, instead use select() to detect incoming connection requests as well as incoming modbus connects on active connections (similar to the approach in Beejs guide).
Create 2 listener threads each of which creates a socket (same IP and port number) which can block on accept() calls, then close the socket fd and deal with the connection. Here I am (perhaps naively) assuming that this will only allow max of 2 connections which I can deal with using blocking reads.
I have been using C++ for a long time but I am fairly new to Linux development. I would really welcome any suggestions as to which of the above approaches is best (if any) and if my inexperience with Linux means that any of them are really really bad ideas. I am keen to avoid fork() and stick to pthreads as incoming modbus requests are going to be queued and read off a main controller loop periodically. Thanks in advance for any advice.
The third alternative won't work, you can only bind to the local address once.
I would probably use your second alternative, unless you need to do a lot of processing in which case a combination of the first to alternatives might be useful.
The combination of the two first alternative I'm thinking of is to have the main thread (the one you always have when a program starts) create two worker threads, then go a blocking accept call to wait for a new connection. When a new connection arrives, tell one of the threads to start working on the new connection and go back to block on accept. When the second connection is accepted you tell the other thread to work on that connection. If both connections are open already, either don't accept until one connection is closed, or wait for new connections but close them immediately.
All of the design option you propose are not very object oriented, and they're all geared more towards C than C++. If your work allows you to use boost, then the Boost.Asio library is fantastic for making simple (and complex) socket servers. You could take nearly any of their examples and trivially extend it to only allow 2 active connections, closing all others as soon as they are opened.
Off the top of my head, their simple HTTP server could be modified to do this by keeping a static counter in the connection class (inc in the constructor, dec in the destructor), and when a new one is created check the count and decide whether to close the connection. The connection class could also gain a boost::asio::deadline_timer to keep track of timeouts.
This would most closely resemble your first design choice, boost could do this in 1 thread and in the background does something similar to select() (usually epoll()). But this is the "C++ way", and in my opinion using select() and raw pthreads is the C way.
Since you are only dealing with 2 connections, thread per connection is perfect for this kind of application. Object oriented approaches using non-blocking or asynchronous I/O would be better if you needed to scale up to thousands of connections. 2 listener threads makes sense, you don't need to close the accept fd. Just come back to accept on it when the connection is completed. In fact, a variation is to have three threads blocked doing accept. If two of the threads are actively handling connections, then the third resets the newly created connection (or returns busy response, whatever is appropriate for your device).
To have all three threads block on accept, you need to have the main thread create and bind your socket before the three threads launch to do their accept/handle processing.
The man page for pthreads on Linux indicates that accept is thread-safe. (The section under thread-safe functions lists the functions that are not thread-safe, go figure.)
Related
I will work on a network design which is using React pattern. In order to have great scalability, I make the design works as one loop per thread. The loop is an abstraction of Linux epoll then I can add socket to the loop.
I can also start multiple threads which each thread has its own loop. But I have a question is can I add a same socket (either listening socket or connection socket) to multiple loops? My feeling is impossible. Otherwise, when the socket get the packet or it is writable, which loop it should trigger when execute epoll_wait() ?
If adding one socket multiple loops is not feasible, but my application is working on a high request volume server which there are large number of requests for the same socket, one thread may not be enough. In this case, how can I improve the performance of my program except increasing the hardware performance ? Depolying the application to multiple servers ?
Thanks in advance.
you can try EPOLLONESHOT, maybe one epoll set is for read and the other is for write~
The program is a client server socket application being developed with C on Linux. There is a remote server to which each client connects and logs itself as being online. There will be most likely be several clients online at any given point of time, all trying to connect to the server to log themselves as being online/busy/idle etc. So how can the server handle these concurrent requests. What's a good design approach (Forking/multithreading for each connection request maybe?)?
personally i would use the event driven approach for servers. there you register a callback that is called as soon as a connection arrives. and event callbacks whenever the socket is ready to read or write.
with a huge amount of connections you will have a great performance and resource benefit compared to threads. But i would also prefere this for a smaler count of connections.
i only would use threads if you really need to use multiple cores or if you have some request that could take longer to process and where it is too complicate to handle it without threads.
i use libev as base library to handle event driven networking.
Generally speaking, you want a thread pool to service requests.
A typical structure will start with a single thread that does nothing but queue up incoming requests. Since it doesn't do very much, it's typically pretty easy for one thread to keep up with the maximum speed of the network.
That puts the items into some sort of concurrent queue. Then you have a pool of other threads reading items from the queue, doing what's needed, then depositing the result in another queue (and repeating, and repeating until the servers shuts down).
Finally, you have another single thread that just takes items from the result queue, and sends replies out to the clients.
Best approach is a combination of event driven model with multithreaded model.
You create a bunch of nonblocking sockets, but threads count should be much fewver. I.e. 10 sockets per thread.
Then you just listen for an event (incoming request) on every thread in a non-blocking mode and process it as it happens.
This technique usually performs better then non-blocking sockets or multithreaded model separately.
Take a look at Comer's "Internetworking with TCP/IP" volume 3 (BSD sockets version), it has detailed examples for different ways of writing servers and clients. The full code (sans explanations, unfortunally) is on the web. Or rummage around in http://tldp.org, there you'll find a collection of tutorials.
select or poll or epoll
These are facilities on *nix systems to aggregate multiple event sources (connections) into a single waiting point. The server adds the connections to a data structure, and then waits by calling select etc. It gets woken up when stuff happens on any of these connections, figures out which one, handles it, and then goes back to sleep. See manual for details.
There are several higher level libraries built on top of these mechanisms, that make programming them somewhat easier e.g. libevent, libev etc.
The native C socket API returns on accept() a new socket descriptor, which is bound to a certain remote socket. That's good because I can create a thread, pass the socket and establish a point-to-point, or better a thread-to-thread connection over the internet. And that's exactly what I want: one thread from the client should be connected to a destined thread on the server. Hence I dont need a workerpool or loadbalancing not even async operation. The server threads save history. ZeroMQ seems great but as far as I understood it does not split up sockets on accept.
Is there a way to establish such an synchronous thread-to-thread connection with ZerMQ?
You're asking how to replicate a particular solution (handing off a socket to a thread) to a broader problem (how to write scalable servers).
The 'one thread per socket' design only works in one pattern which is request-reply, e.g. HTTP. Whereas the really high volume use cases are for data distribution (publish-subscribe), or task distribution (pipeline). Neither fit a 1-to-1 model.
It is a common error when you learn a new tool to ask, "how does this tool do what my old tools do" but you won't get good results like that. Instead, take the time to actually learn how the tool works, and then use that knowledge to re-think your problems and the best solutions for them.
I thought Zmq handle this multi connection for you; I prefer to create a thread-to-thread communication by handling connection within thread callback function, This mean my main zmq connection created in separate thread; which can make separate connection control within threads.
For every single tutorials and examples I have seen on the internet for Linux/Unix socket tutorials, the server side code always involves an infinite loop that checks for client connection every single time.
Example:
http://www.thegeekstuff.com/2011/12/c-socket-programming/
http://tldp.org/LDP/LG/issue74/tougher.html#3.2
Is there a more efficient way to structure the server side code so that it does not involve an infinite loop, or code the infinite loop in a way that it will take up less system resource?
the infinite loop in those examples is already efficient. the call to accept() is a blocking call: the function does not return until there is a client connecting to the server. code execution for the thread which called the accept() function is halted, and does not take any processing power.
think of accept() as a call to join() or like a wait on a mutex/lock/semaphore.
of course, there are many other ways to handle incoming connection, but those other ways deal with the blocking nature of accept(). this function is difficult to cancel, so there exists non-blocking alternatives which will allow the server to perform other actions while waiting for an incoming connection. one such alternative is using select(). other alternatives are less portable as they involve low-level operating system calls to signal the connection through a callback function, an event or any other asynchronous mechanism handled by the operating system...
For C++ you could look into boost.asio. You could also look into e.g. asynchronous I/O functions. There is also SIGIO.
Of course, even when using these asynchronous methods, your main program still needs to sit in a loop, or the program will exit.
The infinite loop is there to maintain the server's running state, so when a client connection is accepted, the server won't quit immediately afterwards, instead it'll go back to listening for another client connection.
The listen() call is a blocking one - that is to say, it waits until it receives data. It does this is an extremely efficient way, using zero system resources (until a connection is made, of course) by making use of the operating systems network drivers that trigger an event (or hardware interrupt) that wakes the listening thread up.
Here's a good overview of what techniques are available - The C10K problem.
When you are implementing a server that listens for possibly infinite connections, there is imo no way around some sort of infinite loops. Usually this is not a problem at all, because when your socket is not marked as non-blocking, the call to accept() will block until a new connection arrives. Due to this blocking, no system resources are wasted.
Other libraries that provide like an event-based system are ultimately implemented in the way described above.
In addition to what has already been posted, it's fairly easy to see what is going on with a debugger. You will be able to single-step through until you execute the accept() line, upon which the 'sigle-step' highlight will disappear and the app will run on - the next line is not reached. If you put a breadkpoint on the next line, it will not fire until a client connects.
We need to follow the best practice on writing client -server programing. The best guide I can recommend you at this time is The C10K Problem . There are specific stuff we need to follow in this case. We can go for using select or poll or epoll. Each have there own advantages and disadvantages.
If you are running you code using latest kernel version, then I would recommend to go for epoll. Click to see sample program to understand epoll.
If you are using select, poll, epoll then you will be blocked until you get an event / trigger so that your server will not run in to infinite loop by consuming your system time.
On my personal experience, I feel epoll is the best way to go further as I observed the threshold of my server machine on having 80k ACTIVE connection was very less on comparing it will select and poll. The load average of my server machine was just 3.2 on having 80k active connection :)
On testing with poll, I find my server load average went up to 7.8 on reaching 30k active client connection :(.
Hi I am working on an assignment writing multi threaded client server.
So far I have done is open a socket in a port and forked two thread for listening and writing to client. But I need to connect two type of clients to the server and service them differently. My question is what would be my best approach?
I am handling connection in a class which has a infinite loop to accept connection. When ever a connection is accepted, the class create two thread to read and write to client? Now if I wnat to handle another client of different type, what should we do?
Do I need to open another port? or is it possible to service through same port? May be if it is possible to identify the type of client in the socket than I can handle messages differently.
Or do you suggest like this?
Fork two thread for two type of client and monitor inbound connection in each thread in different port.
when a connection accepted each thread spawn another two thread for listening and writing.
please make a suggestion.
Perhaps you'll get a better answer from a Unix user, but I'll provide what I know.
Your server needs a thread that opens a 'listening' socket that waits for incoming connections. This thread can be the main thread for simplicity, but can be an alternate thread if you are concerned about UI interaction, for example (in Windows, this would be a concern, not sure about Unix). It sounds like you are at least this far.
When the 'listening' socket accepts a connection, you get a 'connected' socket that is connected to the 'client' socket. You would pass this 'connected' socket to a new thread that manages the reading from and writing to the 'connected' socket. Thus, one change I would suggest is managing the 'connected' socket in a single thread, not two separate threads (one for reading, one for writing) as you have done. Reading and writing against the same socket can be accomplished using the select() system call, as shown here.
When a new client connects, your 'listening' socket will provide a new 'connected' socket, which you will hand off to another thread. At this point, you have two threads - one that is managing the first connection and one that is managing the second connection. As far as the sockets are concerned, there is no distinction between the clients. You simply have two open connections, one to each of your two clients.
At this point, the question becomes what does it mean to "service them differently". If the clients are expected to interact with the server in unique ways, then this has to be determined somehow. The interactions could be determined based on the 'client' socket's IP address, which you can query, but this seems arbitrary and is subject to network changes. It could also be based on the initial block of data received from the 'client' socket which indicates the type of interaction required. In this case, the thread that is managing the 'connected' socket could read the socket for the expected type of interaction and then hand the socket off to a class object that manages that interaction type.
I hope this helps.
You can handle the read-write on a single client connection in one thread. The simplest solution based on multiple-threads will be this:
// C++ like pseudo-code
while (server_running)
{
client = server.accept();
ClientHandlingThread* cth = CreateNewClientHandlingThread(client);
cth->start();
}
class ClientHandlingThread
{
void start()
{
std::string header = client->read_protocol_header();
// We get a specific implementation of the ProtocolHandler abstract class
// from a factory, which create objects by inspecting some protocol header info.
ProtocolHandler* handler = ProtocolHandlerFactory.create(header);
if (handler)
handler->read_write(client);
else
log("unknown protocol")
}
};
To scale better, you can use a thread pool, instead of spawning a new thread for each client. There are many free thread pool implementations for C++.
while (server_running)
{
client = server.accept();
thread_pool->submit(client);
cth->start();
}
The server could be improved further by using some framework that implements the reactor pattern. They use select or poll functions under the hood. You can use these functions directly. But for a production system it is better to use an existing reactor framework. ACE is one of the most widely known C++ toolkits for developing highly scalable concurrent applications.
Different protocols are generally serviced on different ports. However, you could service both types of clients over the same port by negotiating the protocol to be used. This can be as simple as the client sending either HELO or EHLO to request one or another kind of service.