I have a server application with such structure:
There is one object, call him Server, that in endless cycle listens and accepts connections.
I have descendant class from CAsyncSocket, that has overriden event OnReceive, call him ProxySocket.
Also I have a thread pool with early created threads.
When connection is received by server object he accepts the new connection on the new object ProxySocket.
When data arrives to the ProxySocket, he creates a command object and gives it to thread pool. In this command object I giving the socket handle of a ProxySocket. When new object of command is creating - I creating a new Socket in working thread and attach handle to it.
My issue is next:
When command ends, socket doesn't close, I just detach handle it and set CSocket handle to INVALID_SOCKET value, as planned. But my first ProxySocket object doesn't receives messages of new data receiving after that. How can I solve this?
I don't think you can use CAsyncSocket objects (or their descendants) in a thread pool secenario. CAsyncSockets are implemented on top of WSASsyncSelect - which tells the winsock to send notifcations to a window handle.
Because windows have thread affinity, one can never "move" the CAsyncSocket handling to a different thread.
Related
I want to use Overlapped I/O with Completion Routine to handle client connections.
In my UI thread I want to use WSASend(), but in order for the system to call my callback function to inform me that data has been sent, the UI thread must be in a wait state, but this will freeze my UI!
How should I fix this problem?
I agree with #DavidHeffernan - the UI thread should be doing UI things. The IO thread surely needs a binding and port, (server), or peer address and port(client). The socket from ConnectEx or AcceptEx is surely better loaded in the IO thread, but a Socket class with, (at this time, undefined), socket member could surely be created in the UI thread and signaled into the IO thread for handling. Whether buffers form part of your Socket class, or a separate Buffer class, is a design consideration.
One implementation, (that I have used successfully):
Design/define an 'Inter Thread Comms', (ITC'), message class. This has a 'command' enum member that can tell other threads to do stuff, together with any other useful stuff that might be required in such a message
Derive a 'Socket' class from ITC. This has string members for the IP/port, the socket handle and anything else that may be required.
Derive a 'Buffer' class from ITC. This has a 'BoundSocket' member, buffer-space and an 'OVERLAPPED' struct.
Comms with the IO thread is fairly easy. Since it has to wait on something altertably, it can wait on a semaphore that manages a 'Commands' ConcurrentQueue.
If you UI wishes to instruct the IO thread to, say, connect to a server, it creates a Socket instance, (new), loads the IP and Port members from UI elements, sets the Command enum to 'Connect', pushes the socket onto the Commands queue and signals the semaphore, (ReleaseSemaphore).
The alertable wait in the IO thread then returns with WAIT_OBJECT_0, (it needs to ignore returns with WAIT_IO_COMPLETION) and so knows that a command has ben queued. It pops it from the Commands queue and acts upon the command enum, (maybe switching on it), to perform the required action/s. For connect, this would involve an overlapped 'ConnectEx' call to queue up a connect request and set up the connect completion handler.
The connect completion handler, when called, checks for a succesfull connect and, if so, could new up a Buffer, load it, issue a WSARecv with it for the server to send stuff and store the returned Socket object in a container. If failed, it could load the Socket
with a suitable error message and PostMessage it back to the UI thread to inform the user of the fail.
See - it's not that difficult and does not need 10000 lines of code:)
The only thing I don't know how to do immediately is getting the 'this' for the socket object back from the OVERLAPPED struct that is returned in the completion routine. On 32-bit systems, I shoved the Buffer 'this' into the hEvent field of the overlapped struct in the Buffer instance and cast it back in the completion routine. The Buffer instance has a Socket reference, so the job was done. On 64-bit systems, hEvent has not enough room to store the 48/64-bit 'this' Buffer pointer and, (aparrently), this required an extended OVERLAPPED struct:( Not sure how that is done - maybe you will find out:)
[edit] #BenVoigt has advice on the 32/64 bit 'getting the Socket context 'this' back in the completion routine' issue - it's easier than I thought:):
https://stackoverflow.com/a/28660537/758133
I have an out of process ATL COM server (exe). When it is started by a Client the server spawns a worker thread which polls for some data. Every time the data is updated I want to notify all of the clients with the updated value.
I have created a COM Client, which connects via CoCreateInstanceEx, creates its Sink object, gets the connection point and calls Advise, all without error. However, When the server tries to Fire_event, it's IConnectionPointImpl::m_vec is empty and no clients are ever notified.
I assume this is because the server creates its own object, so I end up with two instances of IConnectionPointImpl::m_vec, one from the server and one from the client when it calls Advise. How can I get the server data to the clients?
I'd recommend this approach. Create a plain vanilla C++ singleton (not a COM object) - let's call it S. S would hold a list of weak, non-AddRef'ed references to all outstanding COM objects (a C++ class pointer, rather than a COM interface pointer, would be convenient). Your COM objects would register themselves with S in their constructor, and deregister in destructor.
When something interesting happens, your worker thread would notify (call a method on) S, which would notify all registered COM objects, which would call Fire_event on themselves.
Be careful when firing events from a worker thread. This is illegal, unless both your main thread and worker thread enter MTA. See http://vcfaq.mvps.org/com/1.htm for details and some workarounds.
Hey I am not sure if this has already been asked that way. (I didn´t find anwsers to this specific questions, at least). But:
I have a program, which - at startup - creates an Login-window in a new UI-Thread.
In this window the user can enter data which has to be verified by an server.
Because the window shall still be responsive to the users actions, it (ofc it´s only a UI-thread) shall not handle the transmission and evaluation in it´s own thread.
I want the UI-thread to delegate this work back to the main thread.
In addition: The main thread (My "client" thread) shall manage all actions that go on, like logging in, handle received messages from the server etc... (not window messages)
But I am not sure of how to do this:
1.) Shall I let the UI-Thread Queue an APC to the main thread (but then the main thread does not know about the stuff going on.
2.) May I better use event objects to be waited on and queues to transmit the data from one thread to another?...
Or are there way better options?
For example: I start the client:
1. The client loads data from a file and does some intialization
The client creates a window in a new thread which handles login data input from the user.
The Window Thread shall notifiy and handle the , that has been entered by the user, over to the client.
The Client shall now pack the data and delegate the sending work to another object (e.g. CSingleConnection) which handles sending the data over a network (of course this does not require a new thread, because it can be handle with Overlapped I/O...
One special receiver thread receives the data from the server and handles it back to the client, which - in turn - evaluates the data.
If the data was correct and some special stuff was received from the server, the main thread shall signal the UI thread to close the window and terminate...
The client then creates a new window, which will handle the chatting-UI
The chatting UI thread and the Client thread shall communicate to handle messages to be sent and received...
(Hope this helps to get what I am trying)...
It all depends on what you are prepared to use. If you are developing with Qt, their signals and slots are just the thing to do such a communication. They also supply a network library, so you could easily omit the receiver thread because their network classes do asynchronous communication and will send a signal when you have data, which means your thread does not need to be blocked in the mean time.
If you don't want to use Qt, boost also supplies thread safe signals and slots, but as far as I understand it their slots will be run in the context of the calling thread...
Anyways, I have used Qt sig and slots with great satisfaction for exactly this purpose. I wholeheartedly agree GUI's shouldn't freeze, ever.
I don´t know wether this is good style or not (anwsering Your own question):
But I think I go with Event Objects and two queues (one for the connection between Client and Connection, and one to communicate Client and UI)...
My program uses a NetworkOutput object which can be used to write data to a remote server. The semantic is that in case the object is currently connected (because there is a remote server), then the data is actually sent over the socket. Otherwise, it's silently discarded. Some code sketch:
class NetworkOutput
{
public:
/* Constructs a NetworkOutput object; this constructor should not block, but it
* should start attempting to the given host/port in the background.
*
* In case the connection gets closed for some reason, the object should immediately
* try reconnecting.
*/
NetworkOutput( const std::string &hostName, unsigned short port );
/* Tells whether there is a remote client connected to this NetworkOutput object.
* Clients can use this function to determine whether they need to both serializing
* any data at all before calling the write() function below.
*/
bool isConnected() const;
/* Write data to the remote client, if any. In case this object is not connected
* yet, the function should return immediately. Otherwise it should block until
* all data has been written.
*
* This function must be thread-safe.
*/
void write( const std::vector<char> &data );
};
Right now, I have this implemented using nonblocking sockets. I'n the NetworkOutput constructor, I'm creating a TCP socket as well as an internal helper window. I then do a WSAAsyncSelect call on the socket. This makes the socket nonblocking, and it will cause a magic window message (which I registered myself) to be sent to the internal helper window in case any interesting event (such as 'connection established' or 'connection closed') happens on the socket. Finally, I start a connection attempt using WSAConnect. This returns immediately, and the window procedure of my internal helper window will get notified as soon as the connection succeeded. In case the connection is closed (because the remote client went away), the message procedure will be called and I will attempt to reconnect.
This system allows the me to attach and detach a remote client at will. It works quite well, but unfortunately it requires that I have a message loop running. Without the message loop, the notifications sent by the WSAAsyncSelect call don't seem to arrive at my helper window.
Is there any way to implement a class as described above without requiring a message loop? I was toying around with using blocking sockets in a helper thread, but I couldn't come up with anything reasonable yet. I also considered using a UDP socket, so that I don't even need to connect at all, but I'd like to know whether there is a remote client listening so that in case there is no remote client, the clients of the NetworkOutput class don't need to do any serialization work of complex objects before they can call write().
You can use WSAEventSelect instead of WSAASyncSelect, which takes the handle of a WSAEVENT instead of a message ID, and then use WSAWaitForMultipleEvents to wait for the event to be signalled.
Instead of WSAEVENT you can also use normal Win32 events created with CreateEvent, and the normal synchronisation functions such as WaitForMultipleObjects.
You are looking for the select function:
http://support.sas.com/documentation/onlinedoc/sasc/doc750/html/lr2/select.htm
Basically you specify a set of ports you want to listen to.
When called the select deschedules the thread (thus allowing other threads to work while you do a non busy wait). Your thread is woken up after either a time limit (usually infinite) a signal (if you want to manually make the thread or the system does) or there is some input that needs to be handled on any of the ports.
When your thread wakes up it is usually best to let another thread handle the work so; what usually happens is that you create a work object for each port that has data waiting to be read and add these to a queue where a set of worker threads than start handling the input. Once this is done you call select() again to wait for more input.
Note: You don't have to do this it can be done in a single thread.
How do I execute QTcpSocket functions in a different thread?
It's important to note what you can and can't do in terms of threading QTcpSocket:
you can use it in a non-main thread, but only the thread in which it was created in.
you cannot call different functions on the QTcpSocket from different threads, e.g. read in one thread, write in the other. Instead, you can make a seperate thread for each QTcpSocket, which keeps them from using up time and resources that could be painting your widgets in the main thread.
IMO, putting your IO, including QTcpSocket in a thread other than the main thread is a best practice and must-do for any performant application. I use QTcpSocket in non-main threads all the time using the following idiom:
// Read data from a QTcpSocket in a thread. Assumes this is in some class.
m_thread = std::thread([this]
{
QEventLoop eventLoop;
QTcpSocket* socket = new QTcpSocket(&eventLoop);
socket->connectToHost("localhost", 9999);
// enqueue or process the data
QObject::connect(socket, &QTcpSocket::readyRead, &eventLoop, [socket]
{
m_concurrentQueue.push_back(socket->readAll());
});
// Quit the loop (and thread) if the socket it disconnected. You could also try
// reconnecting
QObject::connect(socket, &QTcpSocket::disconnected, &eventLoop, [&eventLoop]
{
eventLoop.quit();
});
eventLoop.exec();
delete socket;
});
where m_thread is some member thread (basically just ensuring that it has a lifetime greater than the current immediate scope), and m_concurrentQueue is some thread-safe queue, or std container with mutex protection.
You'll also want to connect some signal (I usually call it joinAll) to the event loop quit function, and call it from the class destructor. When using an event-loop-in-a-thread idiom you always have to be careful about making sure you can actually destroy the class correctly, otherwise your program won't exit (or on windows it will be terminated, usually with some destructors not getting called, and it ends up being a silent error).
I also usually use a condition variable to wait after creating the thread until the event loop has started. It's not necessary but if you are putting these threads together in constructors it can help make the program flow make more sense.
The QT docs are explicit that the QTCPSocket should not be used accross threads. I.E, create a QTCPSocket in the main thread and have the signal tied to an object in another thread.
I suspect that you are implementing something like a web server where the listen creates a QTCPSocket on the accept. You then want another thread to handle the task of processing that socket. You can't.
The way I worked around it is I kept the socket in the thread it was born in. I serviced all of the incoming data in that thread and threw it into a queue where another thread could work on that data.
virtual void incomingConnection(qintptr socketDescriptor)
Note: If another socket is created in the reimplementation of this method, it needs to be added to the Pending Connections mechanism by calling addPendingConnection().
Note: If you want to handle an incoming connection as a new QTcpSocket object in another thread you have to pass the socketDescriptor to the other thread and create the QTcpSocket object there and use its setSocketDescriptor() method.
What I read in the docs is that QTcpSocket should not be used across threads. If you want to use it in another thread Qt docs say you should create a new QTcpSocket instance in your thread and set the descriptor on the new instance. To do this you need to reimplement QTcpServer and use QTcpServer::incomingConnection. A simple example is provided here.
Put a QMutex lock around all calls, not just on the "different" thread but on all threads. One easy way to do so is via a QMutexLocker
QTcpSocket and QUdpSocket cannot be moved to a different thread once they have been "opened". This is not supported by Qt and (mostly) doesn't work.
While they can be moved when they are in the "closed" state, that's not generally particularly useful.
What you should do is either:
Create the socket inside the thread it will be used in and use signals to marshal the data to be sent/received between threads.
Pass a socketDescriptor from the thread that starts opening the connection across to the thread where the socket should live, and open the socket there using the socketDescriptor.
Common gotchas:
When using QTcpServer, it will open QTcpSockets in the same thread as itself. As they're already open, they cannot be moved.
If you want QTcpServer to open incoming sockets in a different worker thread to the server itself (very common!), you must subclass it and override QTcpServer::incomingConnection() to get access to the socket descriptor before it creates a QTcpSocket. You cannot use the default implementation.
If you are subclassing QThread, do not create the sockets in the constructor. They must be created in QThread::run()