I have a server program (works on all three major os systems), created in c++ which strives to connect two strangers for communication. My current model seems depreciated, and I am simply wondering if there is a better way to go about serving the clients.
-server receives connection request
-checks for ban
-starts thread for this socket
[the thread simply loops through these steps]
-confirm connection with partner (check manager)
-if unconnected request partner from manager class
-receive on my socket <- client sends keep alive packets every 2s or so
-on disconnect inform manager, and close thread.
[the manager class works like this]
-add socket: push_back on my vector of sockPairs
-request partner: find unconnected sockPair, if none exist create new sockPair, once connected, mark sockPair for removal, if already marked, remove it.
-remove socket: add flag to manager id of partner that I've disconnected
-check socket: check manager id for disconnection
I'm thinking a map would be much more efficient, however I'm not positive, as I've never worked with them, what else do you think I should change? I'm hoping to be able to serve 200 clients with this model, I'm really unsure if the current model could handle it...
A thread per connection simply does not scale. You need to use a mechanism such as select(), poll(), epoll(), WSAAsyncSelect() or anything else that will let you query a set of sockets for events. Then you process each socket in order, and repeat.
Related
At work I have been tasked with implementing a TCP server as part of a Modbus slave device. I have done a lot of reading both here on stack exchange and on the internet in general (including the excellent http://beej.us/guide/bgnet/) but I am struggling with a design issue. In summary, my device can accept just 2 connections and on each connection will be incoming modbus requests which I must process in my main controller loop and then reply with success or failure status. I have the following ideas of how to implement this.
Have a listener thread that creates, binds, listens and accepts connections, then spawns a new pthread to listen on the connection for incoming data and close connection after an idle timeout period. If the number of active threads is currently 2, new connections are instantly closed to ensure only 2 are allowed.
Do not spawn new threads from the listener thread, instead use select() to detect incoming connection requests as well as incoming modbus connects on active connections (similar to the approach in Beejs guide).
Create 2 listener threads each of which creates a socket (same IP and port number) which can block on accept() calls, then close the socket fd and deal with the connection. Here I am (perhaps naively) assuming that this will only allow max of 2 connections which I can deal with using blocking reads.
I have been using C++ for a long time but I am fairly new to Linux development. I would really welcome any suggestions as to which of the above approaches is best (if any) and if my inexperience with Linux means that any of them are really really bad ideas. I am keen to avoid fork() and stick to pthreads as incoming modbus requests are going to be queued and read off a main controller loop periodically. Thanks in advance for any advice.
The third alternative won't work, you can only bind to the local address once.
I would probably use your second alternative, unless you need to do a lot of processing in which case a combination of the first to alternatives might be useful.
The combination of the two first alternative I'm thinking of is to have the main thread (the one you always have when a program starts) create two worker threads, then go a blocking accept call to wait for a new connection. When a new connection arrives, tell one of the threads to start working on the new connection and go back to block on accept. When the second connection is accepted you tell the other thread to work on that connection. If both connections are open already, either don't accept until one connection is closed, or wait for new connections but close them immediately.
All of the design option you propose are not very object oriented, and they're all geared more towards C than C++. If your work allows you to use boost, then the Boost.Asio library is fantastic for making simple (and complex) socket servers. You could take nearly any of their examples and trivially extend it to only allow 2 active connections, closing all others as soon as they are opened.
Off the top of my head, their simple HTTP server could be modified to do this by keeping a static counter in the connection class (inc in the constructor, dec in the destructor), and when a new one is created check the count and decide whether to close the connection. The connection class could also gain a boost::asio::deadline_timer to keep track of timeouts.
This would most closely resemble your first design choice, boost could do this in 1 thread and in the background does something similar to select() (usually epoll()). But this is the "C++ way", and in my opinion using select() and raw pthreads is the C way.
Since you are only dealing with 2 connections, thread per connection is perfect for this kind of application. Object oriented approaches using non-blocking or asynchronous I/O would be better if you needed to scale up to thousands of connections. 2 listener threads makes sense, you don't need to close the accept fd. Just come back to accept on it when the connection is completed. In fact, a variation is to have three threads blocked doing accept. If two of the threads are actively handling connections, then the third resets the newly created connection (or returns busy response, whatever is appropriate for your device).
To have all three threads block on accept, you need to have the main thread create and bind your socket before the three threads launch to do their accept/handle processing.
The man page for pthreads on Linux indicates that accept is thread-safe. (The section under thread-safe functions lists the functions that are not thread-safe, go figure.)
I'm trying to build an application with WT which on one end must communicate with some other software through a socket, and on the other end notify every opened session (the WApplications).
The thing is, I can make it work, using a WSocketNotifier to be notified of new events on the socked BUT the WSocketNotifier is linked to a particular session.
In its constructor it does :
sessionId_(WApplication::instance()->sessionId()),
And that's a problem because I have only one socket, not one for each session.
So I feel the WSocketNotifier should be attached to some more general entity, not linked to a particular session. Is there a way to do that?
The only solution I see for now but it's not very elegant is to have a static WSocketNotifier and replace it when sessions die ...
I know nothing of WT, but can you create a "dummy" session just for the purposes of handling the socket notifications, and forward to the real sessions from tehre? Or roll your own notifier that sessions can subscribe to?
in your main(), create a thread that processes the data from your socket. With select, blocking read, ... Then use WServer::post() to notify your sessions that data arrived on your socket.
I'm searching for different options for implementing communication between a service and other services/applications.
What I would like to do:
I have a service that is constantly running, polling a device connected to a serial port. At certain points, this service should send a message to interested clients containing data retrieved from the device. Data is uncomplicated, most likely just a single string.
Ideally, the clients would not have to subscribe to receive these messages, which leads me to some sort of event 'broadcast' setup (similar to Windows events). The message sending process should not block, and does not need a response from any clients (or that there even are any clients for that matter).
I've been reading about IPC (COM in particular) and windows events, but am yet to come across something that really fits with what I want to do.
So is this possible? If so, what technologies should I be using? If not, what are some viable communication alternatives?
Here's the particulars of the setup:
Windows 2000/XP environments
'Server' service is a windows service, using VC++2005
Clients would vary, but always be in the windows environment (usual clients would be VC++6 windows services, VB6 applications)
Any help would be appreciated!
Windows supports broadcasting messages, check here. You can SendMessage to HWND_BROADCAST from the service, and receive it in each client.
There are a number of ways to do a broadcast system, but you'll have to either give up reliability (ie, some messages must be lost) or use a proper subscription system.
If you're willing to give up reliability, you can create a shared memory segment and named manual-reset event object. When a new message arrives, write it to the shared memory segment, signal the event object, then close the event object and create a new one with a different name (the name should be in the shmem segment somewhere). Clients open the shmem segment, find the current event object, wait for it to be signaled, then read off the message and new event segment.
In this option, you must be careful to deal with the case of a client reading at the same time as the shmem segment is updated properly. One way to do this is to have two sequence number fields in the shmem segment - one is updated before the new message is written, one after. Clients read the second sequence number prior to reading the message, then re-read both sequence numbers after, and check that they are all equal (and discard the message and retry after a delay if they are not). Be sure to place memory barriers around accesses to these sequence numbers to ensure the compiler does not reorder them!
Of course, this is all a bit hairy. Named pipes are a lot simpler, but a subscription (of a sort) is required. The server calls CreateNamedPipe, then accepts connections with ConnectNamedPipe. Clients use CreateFile to connect to the server's pipe. The server then just loops to send data (using WriteFile) to all of its clients. Note that you will need to create addititonal instance of the pipe using CreateNamedPipe each time you accept a connection. An example of a named pipe server can be found here: http://msdn.microsoft.com/en-us/library/aa365588(v=vs.85).aspx
Hi I am working on an assignment writing multi threaded client server.
So far I have done is open a socket in a port and forked two thread for listening and writing to client. But I need to connect two type of clients to the server and service them differently. My question is what would be my best approach?
I am handling connection in a class which has a infinite loop to accept connection. When ever a connection is accepted, the class create two thread to read and write to client? Now if I wnat to handle another client of different type, what should we do?
Do I need to open another port? or is it possible to service through same port? May be if it is possible to identify the type of client in the socket than I can handle messages differently.
Or do you suggest like this?
Fork two thread for two type of client and monitor inbound connection in each thread in different port.
when a connection accepted each thread spawn another two thread for listening and writing.
please make a suggestion.
Perhaps you'll get a better answer from a Unix user, but I'll provide what I know.
Your server needs a thread that opens a 'listening' socket that waits for incoming connections. This thread can be the main thread for simplicity, but can be an alternate thread if you are concerned about UI interaction, for example (in Windows, this would be a concern, not sure about Unix). It sounds like you are at least this far.
When the 'listening' socket accepts a connection, you get a 'connected' socket that is connected to the 'client' socket. You would pass this 'connected' socket to a new thread that manages the reading from and writing to the 'connected' socket. Thus, one change I would suggest is managing the 'connected' socket in a single thread, not two separate threads (one for reading, one for writing) as you have done. Reading and writing against the same socket can be accomplished using the select() system call, as shown here.
When a new client connects, your 'listening' socket will provide a new 'connected' socket, which you will hand off to another thread. At this point, you have two threads - one that is managing the first connection and one that is managing the second connection. As far as the sockets are concerned, there is no distinction between the clients. You simply have two open connections, one to each of your two clients.
At this point, the question becomes what does it mean to "service them differently". If the clients are expected to interact with the server in unique ways, then this has to be determined somehow. The interactions could be determined based on the 'client' socket's IP address, which you can query, but this seems arbitrary and is subject to network changes. It could also be based on the initial block of data received from the 'client' socket which indicates the type of interaction required. In this case, the thread that is managing the 'connected' socket could read the socket for the expected type of interaction and then hand the socket off to a class object that manages that interaction type.
I hope this helps.
You can handle the read-write on a single client connection in one thread. The simplest solution based on multiple-threads will be this:
// C++ like pseudo-code
while (server_running)
{
client = server.accept();
ClientHandlingThread* cth = CreateNewClientHandlingThread(client);
cth->start();
}
class ClientHandlingThread
{
void start()
{
std::string header = client->read_protocol_header();
// We get a specific implementation of the ProtocolHandler abstract class
// from a factory, which create objects by inspecting some protocol header info.
ProtocolHandler* handler = ProtocolHandlerFactory.create(header);
if (handler)
handler->read_write(client);
else
log("unknown protocol")
}
};
To scale better, you can use a thread pool, instead of spawning a new thread for each client. There are many free thread pool implementations for C++.
while (server_running)
{
client = server.accept();
thread_pool->submit(client);
cth->start();
}
The server could be improved further by using some framework that implements the reactor pattern. They use select or poll functions under the hood. You can use these functions directly. But for a production system it is better to use an existing reactor framework. ACE is one of the most widely known C++ toolkits for developing highly scalable concurrent applications.
Different protocols are generally serviced on different ports. However, you could service both types of clients over the same port by negotiating the protocol to be used. This can be as simple as the client sending either HELO or EHLO to request one or another kind of service.
I've already developed some online games (like chess, checkers, risk clone) using server side programming (PHP and C++) and Flash (for the GUI). Now, I'd like to develop some kind of game portal (like www.mytopia.com). In order to do so, I must decide what is a good way to structure my server logic.
At first I thought in programming separated game servers for each game. In this way, each game will be an isolated program that opens a specific port to the client. I thought also in creating different servers to each game room (each game room allow 100 clients connected on the same time). Of course I'd use database to link everything (like highscores, etc).
Then, I guess it is not the best way to structure a game portal server. I'm reading about thread programming and I think that is the best way to do it. So, I thought in doing something like a connection thread that will listen only to new connection clients (that way every type of game client will connect in only one port), validate this client (login) and then tranfer this client to the specific game thread (like chess thread, checkers thread, etc). I'll be using select (or variants) to handle the asynchronous clients (I guess the "one thread per client" is not suited this time). This structure seems to be the best but how do I make the communication between threads? I've read about race conditions and global scope variables, so one solution is to have a global clients array (vector or map) that need to be locked by connection thread or game thread everytime it is changed (new connection, logout, change states, etc). Is it right?
Has anyone worked in anything like this? Any recommendations?
Thanks very much
A portal needs to be robust, scalable and extensible so that you can cope with larger audiences, more games/servers being added, etc. A good place to start is to look into the way MMOs and distributed systems are designed. This might help too: http://onlinegametechniques.blogspot.com/
Personally, I'd centralise the users by having an authentication server, then a separate game server for each game that validates users against the authentication server.
If you use threads you might have an easier time sharing data but you'll have to be more careful about security for exactly the same reason. That of course doesn't address MT issues in general.
TBH I've been doing a voip system where the server can send out many streams and the client can listen to many streams. The best architecture I've come up with so far is just to bind to a single port and use sendto and recvfrom to handle communications. If i receive a valid connect packet from a client on a new address then I add the client to an internal list and begin sending audio data to them. The packet receive and response management (RRM) all happens in one thread. The audio, as it becomes ready, then gets sent to all the clients from the audio thread. The clients respond saying they received the audio and that gets handle on the RRM thread. If the client fails to respond for longer than 30 seconds then I send a disconnect and remove the client from my internal list. I don't need to be particularly fault tolerant.
As for how to do this in a games situation my main thought was to send a set of impulse vectors (the current one and 'n' previous ones). This way if the client moves out of sync it can check how out of sync it is by checking the last few impulses it should have received for a given object. If it doesn't correspond to what its got then it can either correct or if it is too far out of sync it can ask for a game state reset. The idea being to try and avoid doig a full game state reset as it is going to be quite an expensive thing to do.
Obviously each packet would be hashed so the client can check the validity of incoming packets but it also allows for the client to ignore an invalid packet and still get the info it needs in the next update and thus helping prevent the state reset.
On top of that its worth doing things like keeping an eye on where the client is. There is no point in sending updates to a client when the client is looking in the other direction or there is something in the way (ie the client can't see the object its being told about). This also limits the effectiveness of a wallhack packet sniffing the incoming packets. Obviously you have to start sending things a tad before the object becomes visible, however, or you will get things popping into existence at inconvenient moments.
Anyway ... thats just some random thoughts. I have to add that I've never actually written a multiplayer engine for a game so I hope my musings help ya a bit :)