I am faced with a design issue regarding thread synchronization in C++, Windows.
I am writing a server application that starts one listening thread, which should stay active the whole time while the server is up.
When the listening thread gets a connect request, it opens a CONTROL socket and starts a new control thread.
This thread is used to send control data between server and a client, initializing server and all the background software to specific client data and starting data processing.
If the initialization (via control socket) is successful, the control thread will open a new socket, DATA socket, which is then used to pass data from server to client. It will also start two new threads, one which is sending on this new, DATA socket, and the other, which is receiving on the CONTROL socket, waiting if the client wants to terminate connection.
When client terminates connection ungracefully, by terminating application without the call to function which sends the server message to close the connection, here is what should happen:
Any of the threads in execution can detect this event. They will get some sort of error (WSAECONNRESET) while sending or receiving on DATA/CONTROL socket and should then signal all the other threads that they should stop executing (except for the server listening thread).
Which is the most natural way to achieve this type of behavior?
(I am using winsock (winsock2.h) for networking, and standard windows api (windows.h) for threading)
If you're writing a multi-threaded winsock server, you should be looking into IO completion ports. Using an IO completion port is the most scalable way to write a network service on the windows platform.
IO completion port based winsock servers use asynchronous communication, so instead of blocking on a socket, your threadpool receives completion packets when something interesting happens.
In any case, you'll be using WSARecv. When WSARecv returns non zero, call WSAGetLastError(). If you don't have WSA_IO_PENDING, then switch on the error and look for the winsock error code you're interested in.
The winsock error code WSA_OPERATION_ABORTED indicates that a socket has closed, although there are others (e.g. WSAECONNABORTED).
Would suggest a good text on the subject (e.g. Windows via C/C++).
You can use WSAEventSelect() function to associate event object with socket and create one event object for your events, then use these event objects in WaitForMultipleObjects() function, so your thread can wait for socket events and your custom events.
Related
I am new to network programming and the usage of Boost Asio library.
I successfully implemented a task for my requirement by modifying the Boost Asio "Blocking TCP Echo Server and Client" which performs transactions of operations between my Client and Server.
Now, I have a requirement where I need to connect multiple Clients with my Server.
I found some relevant links suggesting the usage of async_accept at the Server side.
So, I tried running the Boost Asio example: "Async TCP Echo Server" with the "Blocking TCP Echo client", where the server distinguishes the different clients and addresses them accordingly.
But, my actual requirement should be like, instead of the Server completing the entire process for one Client, it [the server] has to perform same operations for the first client then go to the second client and perform those operations and then again come back to the first client and continue in this order until all operations are complete.
Is there any way or idea which could help me perform this flow using Boost Asio? Also I'm just using the "Blocking TCP Echo Client", which just has a normal connect() and not an async_connect(), now is that a problem?
Also, is it possible to communicate between multiple clients through the server using Boost Asio?
Thanking you very much in advance!
There are 2 models to handling multiple client concurrently on the server.
The one is to spawn a new thread for each client and then each thread handles each client synchronously. The second model is to use asynchronous APIs on a single thread all operating on a single service. When the accept completes, you then create a new worker thread and start the worker off the send and recv required by your protocol. You main thread goes back the accepting new connections.
With async, you prime the pump with an async accept and the call io_service run. When the accept completes, your callback runs. You now prime the pump again with further accepts (for more client) start async send and recv for the newly created client. Since all sends and recvs are non-blocking, the only time your thread sleeps is when it has nothing to do. Otherwise the io_service run method takes care of everything for you.
If you are blocking on sends and recvs, through, you cannot process more than one client concurrently.
I have a question regarding non-blocking sockets in TCP connections.
I have implemented two c++ classes, one for the tcp server and one for the client. The server has two sockets file descriptors, one for the server and one for the client. The client has one socket file descriptor.
My server runs asynchronously and my client runs at a fixed rate. Therefore I would like to have a non-blocking socket for sending data from the client to the server, s.t. the client can send data at a fixed rate without stalling and the server asynchronously reads all data that has been buffered meanwhile.
So my question is: Does it make a difference, if I set the client socket to non-blocking in the client or the server class? (using fcntl(this->newsockfd_, F_SETFL, fcntl(this->newsockfd_, F_GETFL, 0) | O_NONBLOCK), where this->newsockfd_ is the client's socket file descriptor in both classes)
I tried this in my programm and it seemed like setting the client socket to non-blocking in the client-class didn't do the trick, but setting it in the server-class did. However, I don't understand why this should make a difference.
If your socket is set to non blocking mode, you will get just that. It will never block. But that does not mean that your api calls will succeed.
There are buffers that are being used behind the scenes and if they are full, which would mean in blocking mode that the socket would block, you will get a return code EWOULDBLOCK, which means that your sent has failed. This means that you basically have to wait for the buffers to empty and then try again.
Your idea of sending at an even rate despite of the server rate to receive, is impossible. You cannot have a client sending at a fixed rate. The whole idea of TCP is that there is a constant negotiation between client and server and the speed will be heavily depending on the network conditions. Congestion and the like.
Moving to non blocking sockets creates some problems of its own. You have to detect that the send fails, you have to check if the socket becomes writeable again, you have to store the bytes that you tried to send, and reattempt a send as soon as the socket becomes writable again.
There is a lot of difference on both client and server between working with blocking and non blocking sockets. non blocking sockets are in my opinion more difficult to be dealt with. You need the select api, with a timeout very likely to detect all the possible socket states. In case of blocking sockets, you can just use a socket in a thread, and if the socket blocks, it is just the thread that will block as well. If your gui is on a different thread, the GUI will be responsive.
Since your client is only sending data the non-blocking setting will not effect it. According to the excellent beej.us guide on socket programming, only calls to accept() and recv() are effected by the non-blocking setting. Since only your server is calling these you are seeing the change on your server code. If your client received data then the non-blocking setting would effect it and you would have to use select() to check if there is data and read from it accordingly.
I use boost:asio::ip::tcp::acceptor to create server socket in my app. I close this acceptor socket using close function, than stop function in io_service but all connected client sockets closes only when my app is closed. How can i fix that?
Thanks!
Do either of the following:
invoke socket::close() on the sockets.
destroy the socket. See this answer for details on how the socket will be closed during destruction.
io_service::stop() only stops processing of the event loop. Work can still be posted into the io_service, and existing work will remain in the io_service. Thus, the application must call socket::close() on each of the sockets it wishes to close. For a portable graceful closure, call socket::shutdown() before calling close().
It may be worth taking the time to review Boost.Asio's HTTP Server 1 example. It uses a connection_manager to shutdown all connections.
I need to create a server that allow ONE at time client connected.
The rule is that just one client can be connected and if the other one try to connect, can read a messagge like this "another client is connected, do you want disconnect it?".
Then if type yes the client will be disconnected.
My problem is about this step. How can I disconnect a client and connect the other one?
Can someone help me?
Thank you.
First build the abstract server structure. So you write a program which accepts TCP connections in one thread and pass them to a worker thread, which can read and send messages.
You should keep one Singleton containing a reference (or pointer, your choice) to the Worker with the currently connected client (or null, if there is noone connected).
To keep it simple, the acceptor thread should create a new Worker thread everytime it accepts a connection, and the Worker thread is terminated, when the connection breaks up.
Now you have to think about a protocol. For this simple task, 5 messages should be enough. Maybe every message ends with an endl, so you can use methods like readline if there is somthing like this in C++.
First, the CONNECT message. The server should return OK (second message), if noone is connected to it, and ERROR (third message), if there is already one connected.
The fourth message is CONNECTWITHDISCONNECT, it connects the client to the server and disconnects any other client. The newly connected client should receive a OK message from the server, and the disconnected one should receive DISCONNECT (fifth message).
Now, you could use the disconnect message also with the client, so one can disconnect, without requiring another to connect.
The client should send a CONNECT first, if it receives ERROR then, it can ask the user to disconnect the other client, and if the user wants to, the client sends CONNECTWITHDISCONNECT.
Another option (if you don't want to deal with multiple threads or multiple processes) is to use select() or poll() to handle multiple sockets at the same time within a single thread. In particular, you can select()-for-read on your accepting socket, and select() will return with that socket marked as ready-for-read whenever another client is trying to connect. Once you have accept()'d the client, you can pass the client's connection socket (as was returned by accept()) to select()'s read-sockets-set so that you will also be notified whenever the client's socket has bytes ready for you to read. And so on.
I need a client networking thread to be able to respond both to new messages to be transmitted, and the receipt of new data on the network. I wish to avoid this thread performing a polling loop, but rather to process only as needed.
The scenario is as follows:
A client application needs to communicate to a server via a protocol that is largely, but not entirely, synchronous. Typically, the client sends a message to the server and blocks until a response is received.
The server may process client requests asynchronously, in which case the response to client
is not a result, but a notification that processing has begun. A result message is sent to to the client at some point in the future, when the server has finish processing the client request.
The asynchronous result notifications can arrive at the client at any time. These notifications need processed when they are received i.e. it is not possible to process a backlog only when the client transmits again.
The clients networking thread receives and processes notifications from the server, and to transmit outgoing messages from the client.
To achieve this, I need to to make a thread wake to perform processing either when network data is received OR when a message to transmit is enqueued into an input queue.
How can a thread wake to perform processing of an enqueued work item OR data from a socket?
I am interested primarily in using the plain Win32 APIs.
A minimal example or relevant tutorial would be very welcome!
An alternative to I/O Completion Ports for sockets is using WSAEventSelect to associate an event with the socket. Then as others have said, you just need to use another event (or some sort of waitable handle) to signal when an item has been added to your input queue, and use WaitForMultipleObjects to wait for either kind of event.
You can set up an I/O Completion Port for the handles and have your thread wait on the completion port:
http://technet.microsoft.com/en-us/sysinternals/bb963891.aspx
Actually, you can have multiple threads wait on the port (one thread per processor usually works well).
Following on from Michael's suggestion, I have some free code that provides a framework for IO Completion Port style socket stuff; and it includes an IOCP based work queue too. You should be able to grab some stuff from it to solve your problem from here.
Well, if both objects have standard Windows handles, you can have your client call WaitForMultipleObjects to wait on them.
You might want to investiate splitting the servicing of the network port off onto its own thread. That might simplify things greatly. However, it won't help if you just end up having to synchonize something else between that new thread and your main one.