I have a MFC DLL that I am implementing. The main thread creates a progress bar (CProgressCtrl) and then starts a CWinThread to perform some socket operation. The idea is to have the main thread update the progress bar while the other thread performs sendto socket operation (data request) to request data. Here is the issue though, due to legacy implementation, the receive capability is done via overriding OnReceive function of CAsyncSocket. When OnReceive is called, the code simply copies the data into its own buffer and allow another function to process later.
Currently, the other thread that is doing the sendto operation (data request) is activity checking the buffer to see if the sequence number has been incremented. This all worked fine when the sendto operation is within the main thread (this means UI will freeze), but after I relocated the sendto operations to another thread, the OnReceive is no longer being called even when the data is sent from the other software(verified).
My question is that why is OnReceive not being called when the other side is clearly sending data? I understand this might not be the optimal design but due to legacy design, i would like to keep the current design.
CASyncSocket is bound to the thread in which it is created. Only that thread can receive the OnReceive notification.
Related
I'm trying to create a server with blocking sockets (one new thread for each new client). This thread should be able to receive commands from the client (and send back the result) and periodically send commands to the client (and request back the result).
What I've thought is creating two threads for each client, one for recv, second for send. However:
it's double of the normal thread overhead.
due to request/response design, recv I do in the first thread (to wait for client's commands) can be the request I look for in the second thread (client's result to my send) and vice versa. Making it all properly synced is probably a hell story. So now I'm thinking to do that from a single thread this way:
In a loop:
setsockopt(SO_RCVTIMEO, &small_timeout); // set the timeout for the recv (like 1000 ms).
recv(); // check for client's requests first. if returns WSAETIMEDOUT than I assume no data is requested and do nothing. if I get a normal request I handle it.
if (clientbufferToSend != nullptr) send(clientbufferToSend); // now when client's request has been processed we check the command list we have to send to the client. if there is commands in queue, we send them. SO_SNDTIMEO timeout can be set to a large value so we don't deadlock if client looses connection.
setsockopt(SO_RCVTIMEO, &large_timeout); // set the timeout for the recv (as large as SO_SNDTIMEO, just to not deadlock if anything).
recv(); // now we wait the response from the client.
Is this the legal way to do what I want? Or are there better alternatives (preferrably with blocking sockets and threads)?
P.S. Does recv() with timeout returns WSAETIMEDOUT only if no data is available? Can it return this error if there is the data, but recv() wasn't fast enough to handle it all, thus returning partial data?
One approach is only create a background thread for reading from that socket. Write on whatever random thread your unsolicited events are raised.
You’ll need following stuff.
A critical section or mutex per socket to serialize writes, like when background thread is sending response to client-initiated message, and other thread wants to send message to the same client.
Some other synchronization primitive like a conditional variable for client thread to sleep while waiting for responses.
The background thread which receives messages needs to distinguish client-initiated messages (which need to be responded by the same background thread) from responses to server-initiated messages. If your network protocol doesn’t have that data you’ll have to change the protocol.
This will work OK if your server-initiated events are only happening on a single thread, e.g. they come from some serialized source like a device or OS interface.
If however the event source is multithreaded as well, and you want good performance, you gonna need non-trivial complexity to dispatch the responses to the correct server thread, like 1 conditional variable per client thread, maybe some queues, etc.
I want to use Overlapped I/O with Completion Routine to handle client connections.
In my UI thread I want to use WSASend(), but in order for the system to call my callback function to inform me that data has been sent, the UI thread must be in a wait state, but this will freeze my UI!
How should I fix this problem?
I agree with #DavidHeffernan - the UI thread should be doing UI things. The IO thread surely needs a binding and port, (server), or peer address and port(client). The socket from ConnectEx or AcceptEx is surely better loaded in the IO thread, but a Socket class with, (at this time, undefined), socket member could surely be created in the UI thread and signaled into the IO thread for handling. Whether buffers form part of your Socket class, or a separate Buffer class, is a design consideration.
One implementation, (that I have used successfully):
Design/define an 'Inter Thread Comms', (ITC'), message class. This has a 'command' enum member that can tell other threads to do stuff, together with any other useful stuff that might be required in such a message
Derive a 'Socket' class from ITC. This has string members for the IP/port, the socket handle and anything else that may be required.
Derive a 'Buffer' class from ITC. This has a 'BoundSocket' member, buffer-space and an 'OVERLAPPED' struct.
Comms with the IO thread is fairly easy. Since it has to wait on something altertably, it can wait on a semaphore that manages a 'Commands' ConcurrentQueue.
If you UI wishes to instruct the IO thread to, say, connect to a server, it creates a Socket instance, (new), loads the IP and Port members from UI elements, sets the Command enum to 'Connect', pushes the socket onto the Commands queue and signals the semaphore, (ReleaseSemaphore).
The alertable wait in the IO thread then returns with WAIT_OBJECT_0, (it needs to ignore returns with WAIT_IO_COMPLETION) and so knows that a command has ben queued. It pops it from the Commands queue and acts upon the command enum, (maybe switching on it), to perform the required action/s. For connect, this would involve an overlapped 'ConnectEx' call to queue up a connect request and set up the connect completion handler.
The connect completion handler, when called, checks for a succesfull connect and, if so, could new up a Buffer, load it, issue a WSARecv with it for the server to send stuff and store the returned Socket object in a container. If failed, it could load the Socket
with a suitable error message and PostMessage it back to the UI thread to inform the user of the fail.
See - it's not that difficult and does not need 10000 lines of code:)
The only thing I don't know how to do immediately is getting the 'this' for the socket object back from the OVERLAPPED struct that is returned in the completion routine. On 32-bit systems, I shoved the Buffer 'this' into the hEvent field of the overlapped struct in the Buffer instance and cast it back in the completion routine. The Buffer instance has a Socket reference, so the job was done. On 64-bit systems, hEvent has not enough room to store the 48/64-bit 'this' Buffer pointer and, (aparrently), this required an extended OVERLAPPED struct:( Not sure how that is done - maybe you will find out:)
[edit] #BenVoigt has advice on the 32/64 bit 'getting the Socket context 'this' back in the completion routine' issue - it's easier than I thought:):
https://stackoverflow.com/a/28660537/758133
My aim is to receive messages from a serial device without blocking the main thread (GUI) and to try to separate the platform-dependent logic (GUI and serial port) from the business logic (processing the messages) for ease of porting to other platforms
Context: I'm using Qt, and the QtSerialPort module. The message protocol is simple, 0xff is used to end each message.
I've found 4 solutions so far:
Method 1:
Using one thread to read a serial port and fill a buffer
Using another thread to read the buffer, extract valid messages (into another buffer? not sure how this will work yet)
Using yet another thread to parse the messages
Method 2:
Using one thread to read a serial port, and extract valid messages into a buffer
Using another thread to parse the messages
Method 3:
Using one thread to read a serial port, extract a valid message, and block till that message is processed, making use of QtSerialPort's internal read buffer to buffer incoming data
Method 4:
Using the main thread to asynchronously read serial port, extract a valid message, and for each message, spawn a new thread to process them
Methods 1,2 and 3 differ by the number of threads the general workload is split up into, though I don't know which is best.
I'm currently using method 4, which is horribly inefficient and doesn't work well on lower-end computers, due to the enormous number of threads being spawned, and every time I move or interact with the GUI, serial communication halts. Spawning a thread for each message also makes the order of the messages non-deterministic, which hasn't been a major problem so far...
Are there other methods, what are the pros (if any) and cons of each, and which is the best to use? Thanks!
EDIT: A problem with processing messages in the main thread is that interacting with GUI (even moving the window) would block the message processing function. Is there any way around this?
I think there are two main advantages that you can obtain by using multithreading:
Avoiding poor GUI performance due to the GUI-handling routines being held off by the serial port processing routine
(perhaps more important) Avoid loss of serial data caused by buffer overflow when the GUI routines hold off the serial-data-reading routine for too long.
You should only need to spawn a single thread. Just have that thread read data from the serial port as it comes in (by connecting the QSerialPort's readyRead() signal to a slot that calls read() on the QSerialPort object), and then emit a signal (with a QByteArray argument) whenever it wants to send some serial data to the GUI. Your main/GUI thread can receive the data via a QueuedConnection that will not block either the serial-thread or the main/GUI thread.
That's pretty much all there is to it; the only other thing to worry about is a clean shutdown. Be sure to have another cross-thread signal/slot connection to the QThread's quit() slot, so that when it's time to quit, you can emit that signal and then call wait() on the QThread to wait for it to respond by going away. Once wait() has returned you can safely delete the QThread object.
You can avoid additional threads at all by simply relying on Qt event loop (so far the main thread, the one also handling the GUI to be clear, will be blocked only when a message is actually received by the serial port).
Otherwise if you want to completely handle serial port in a dedicated thread, then the solution is to implement a class deriving from QThread and then override the run() function with something like this:
void MyClass::run()
{
QSerialPort port;
// ... serial port initialization here
// Connect signals/slots
connect(&port, SIGNAL(readyRead()), this, SLOT(readData()));
port.open();
// Start a new message loop on this thread
exec();
}
Where readData is a function implemented in MyClass for handling the received data. Since port is owned by the new thread (being created in run()) then its events will be handled by the thread itself (in a completely independent manner with respect to the main thread).
If you want at some point communicate something to the main thread (e.g.: you received something on serial which should cause a change in your GUI) then you can still use Qt's signals/slots. Simply implement a signal on MyClass and implement a slot on an object handled by the main thread (e.g.: your main form): then simply connect the signal for MyClass and the slot on your main form and you're done: signals/slots is THE solution for cross-thread communication in Qt.
You could also avoid using any (additional) threads and take advantage of Qt event loop. Read about events, QioDevice; then Qt would pass your device file descriptor to its multiplexing loop (e.g. to poll(2)....); probably QSocketNotifier should work (on Posix) on a non-socket file descriptor like a serial device.
Details are probably OS specific
I have a main loop in my program, which calls this method from dbus:
dbus_connection_read_write_dispatch
I have some registered callbacks, which are invoked, when message arrives. Within this callback I am also processing the response and sending back response. Problem is that sometimes it takes much time so probably it will block receiving messages from DBUS.
Question - can I call dbus_connection_read_write_dispatch() method on the same connection from more than one thread? Then it will be probably possible to receive new DBUS messages while the previous one is being processed.
Or maybe better idea is to process responses in another thread than the main loop, from callback is invoked?
Thank you
you can call dbus_connection_read_write_dispatch() from multiple threads if you have called the function dbus_threads_init_default() atleast once.Instead a better approach is to have a single thread running dbus dispatcher and use a thread-pool to process the data from callbacks.
See dbus_threads_init_default() for more info.
By the document provided by freedesktop.org, you can.
But if you operate with same DBusConnection instance from different threads directly, eg. calling dbus_connection_send_with_reply_and_block in a thread while anothoer thread is blocking on dbus_connection_read_write_dispatch, the connection maybe work unproperly. According to official document, DBus connection will be locked when calling callback functions.DBusConnection
In my situation, the dbus_connection_send_with_reply_and_block didn't return even if the return message was send to my process (I had seen it on dbus-monitor). Calling dbus_thread_init does not work at all.
Recently I used a delegate to send / receive / dispatch all dbus messages in one thread, and problem disappeared.
A mail in mailing list of freedesktop.org
My program uses a NetworkOutput object which can be used to write data to a remote server. The semantic is that in case the object is currently connected (because there is a remote server), then the data is actually sent over the socket. Otherwise, it's silently discarded. Some code sketch:
class NetworkOutput
{
public:
/* Constructs a NetworkOutput object; this constructor should not block, but it
* should start attempting to the given host/port in the background.
*
* In case the connection gets closed for some reason, the object should immediately
* try reconnecting.
*/
NetworkOutput( const std::string &hostName, unsigned short port );
/* Tells whether there is a remote client connected to this NetworkOutput object.
* Clients can use this function to determine whether they need to both serializing
* any data at all before calling the write() function below.
*/
bool isConnected() const;
/* Write data to the remote client, if any. In case this object is not connected
* yet, the function should return immediately. Otherwise it should block until
* all data has been written.
*
* This function must be thread-safe.
*/
void write( const std::vector<char> &data );
};
Right now, I have this implemented using nonblocking sockets. I'n the NetworkOutput constructor, I'm creating a TCP socket as well as an internal helper window. I then do a WSAAsyncSelect call on the socket. This makes the socket nonblocking, and it will cause a magic window message (which I registered myself) to be sent to the internal helper window in case any interesting event (such as 'connection established' or 'connection closed') happens on the socket. Finally, I start a connection attempt using WSAConnect. This returns immediately, and the window procedure of my internal helper window will get notified as soon as the connection succeeded. In case the connection is closed (because the remote client went away), the message procedure will be called and I will attempt to reconnect.
This system allows the me to attach and detach a remote client at will. It works quite well, but unfortunately it requires that I have a message loop running. Without the message loop, the notifications sent by the WSAAsyncSelect call don't seem to arrive at my helper window.
Is there any way to implement a class as described above without requiring a message loop? I was toying around with using blocking sockets in a helper thread, but I couldn't come up with anything reasonable yet. I also considered using a UDP socket, so that I don't even need to connect at all, but I'd like to know whether there is a remote client listening so that in case there is no remote client, the clients of the NetworkOutput class don't need to do any serialization work of complex objects before they can call write().
You can use WSAEventSelect instead of WSAASyncSelect, which takes the handle of a WSAEVENT instead of a message ID, and then use WSAWaitForMultipleEvents to wait for the event to be signalled.
Instead of WSAEVENT you can also use normal Win32 events created with CreateEvent, and the normal synchronisation functions such as WaitForMultipleObjects.
You are looking for the select function:
http://support.sas.com/documentation/onlinedoc/sasc/doc750/html/lr2/select.htm
Basically you specify a set of ports you want to listen to.
When called the select deschedules the thread (thus allowing other threads to work while you do a non busy wait). Your thread is woken up after either a time limit (usually infinite) a signal (if you want to manually make the thread or the system does) or there is some input that needs to be handled on any of the ports.
When your thread wakes up it is usually best to let another thread handle the work so; what usually happens is that you create a work object for each port that has data waiting to be read and add these to a queue where a set of worker threads than start handling the input. Once this is done you call select() again to wait for more input.
Note: You don't have to do this it can be done in a single thread.