BSD stream sockets are full duplex, meaning two connected parties can both send/receive at the same time.
A QTcpSocket (qt socket implementation) has asynchronous support, non blocking mode, but can only belong to one thread, see qt docs.
Event driven objects may only be used in a single thread.
Specifically, this applies to the timer mechanism and the network
module.
Let's say I want a transmit/tx thread and a separate receive/rx thread to use the same socket and send/receive data at the same time.
In my understanding this can be 'done' via qt signals/slots, but the socket thread will never really perform the send() and the receive() simultaneously. It just runs the event loop which will do this in a serial fashion and emit the signals when send/receive is done.
Yes, my rx and tx threads can work concurrently and handle the notifications via qt slots, but the socket itself is never really used in full duplex mode.
Is it correct to say that: considering one endpoint only, in the socket thread, its send() and receive() calls are always serial, never simultaneous?
(because the event loop thread is one thread only)
In my understanding this can be 'done' via qt signals/slots, but the
socket thread will never really perform the send() and the receive()
simultaneously. It just runs the event loop which will do this in a
serial fashion and emit the signals when send/receive is done.
True, but keep in mind that the kernel buffers incoming and outgoing data, and QTCPSocket sets the socket to non-blocking, so that the send() and recv() calls always return immediately and never block the event-loop. That means that the actual processes of sending and receiving data will happen simultaneously (inside the kernel), even if the (more-or-less instantaneous) send() and recv() calls technically do not. (*)
Yes, my rx and tx threads can work concurrently and handle the
notifications via qt slots, but the socket itself is never really used
in full duplex mode. Is this correct?
That is not correct -- the socket's data streams can (and do) flow both ways across the network simultaneously, so the socket really is full-duplex. The full-duplex capability is present whether you are using a single thread or multiple threads.
(*) You can test this with a single-threaded Qt program that uses a QTCPSocket to send or receive data, by simply disconnecting your computer's Ethernet cable during a large data transfer. If the QTCPSocket's send() or recv() calls are blocking until completion, that would block the GUI thread and cause your GUI to become unresponsive until you reconnect the cable (or until the TCP connection times out after several minutes).
Related
I have some doubts in regards of IO Completion Port as well as AcceptEx in winsock2
Please correct me if i am wrong.
AcceptEx is an overlapped way of accepting requests or connection. However, as pointed out by multiple posts on this site that AcceptEx is prone to DOS attack if AcceptEx is expecting data but is not sent by the connected client. So, can it be solved by just putting 0 to the dwReceiveDataLength?
Besides, what is the advantages to be able to receive data from client when accepting the respective connection rather than receive the data later using AcceptEx?
After accepting connections from opposite endpoint and associates it with IO completion port, the requests are queued in IO completion port as completion packets which are associated with their respective handle. The worker threads that block on the completion port would be woken up depending on the NumberOfConcurrentThreads to serve the requests. So, is the threads in the completion port the IO threads?
So, where should I implement business logic or operation in the socket server? For example, a request from client that send numbers to the server for processing while the server act like a calculator that responds by echoing back the calculated output. Thus, can this logic be implemented in IO Completion Port?
If the logic is implemented in IO completion port (When IO threads (assumed) that are active in IO completion port are performing WSARecv or WSASend)), would the IO threads block while waiting for the calculation to finish thus making no connection is able to be accepted if the backlog are all taken?
EDITED:
For example, after accepting the client socket and queued/ associated in the IO completion port(main_cpl_port), threads that block on this main_cpl_port call GetQueuedCompletionStatus to dequeue completion packet and subsequently read data into allocated buffer. Before any response is written back to the client, the buffer is processed/ parsed for "command" (eg: GoToCalculator, GoToRecorder).
For example, GoToCalculator is responsible for other calculation related commands.
In this case, GoToCalculator is actually another IO completion port that caters all those requests related to calculation. Let say the completion port is named as calc_completion_port.
Thus, is it possible that the completion packet from main_cpl_port to be posted to calc_completion_port for future IO (send and recv) from the client socket which is currently associated with main_cpl_port. Is this what PostQueuedCompletionStatus is used for?
Is message sent from client after posted to calc_completion_port can be received by threads that block on this completion port?
In other words, how can I redirect the connection to other completion port from another?
1) Avoiding the potential AcceptEx DOS attack is easy, just don't provide any space for data and the AcceptEx will complete as soon as the connection is established.
2) Using AcceptEx means that you don't need to have a separate thread to run an accept loop. This removes one thread from your system and reduces context switching. This is especially useful if you are listening on multiple sockets (different ports/interfaces) as each listening socket would need its own accept thread.
3) Yes, the worker threads that call GetQueuedCompletionStatus on an IOCP can be thought of as I/O threads...
4) It depends. I've built systems with distinct, fixed sized pools of I/O threads which never do any blocking operations and separate expanding thread pools designed to perform blocking operations. The idea being that this would prevent all of the threads being blocked and preventing I/O... This requires that you pass work items off to the other thread pool and it causes unnecessary context switching and complexity but it means that you always have threads to do I/O operations (such as handle new connections as AcceptEx completes)... This kind of design used to work well back when the IOCP APIs used to cancel pending operations if the thread that issued then exited before the operation completed. Now that the OS has changed the rules and pending operations are not cancelled there's no real reason why you don't just have an expanding/contracting pool of I/O threads and do all your work there... You just need to track how many threads are available and create/destroy threads as you need to expand/contract your pool...
5) see 4.
My aim is to receive messages from a serial device without blocking the main thread (GUI) and to try to separate the platform-dependent logic (GUI and serial port) from the business logic (processing the messages) for ease of porting to other platforms
Context: I'm using Qt, and the QtSerialPort module. The message protocol is simple, 0xff is used to end each message.
I've found 4 solutions so far:
Method 1:
Using one thread to read a serial port and fill a buffer
Using another thread to read the buffer, extract valid messages (into another buffer? not sure how this will work yet)
Using yet another thread to parse the messages
Method 2:
Using one thread to read a serial port, and extract valid messages into a buffer
Using another thread to parse the messages
Method 3:
Using one thread to read a serial port, extract a valid message, and block till that message is processed, making use of QtSerialPort's internal read buffer to buffer incoming data
Method 4:
Using the main thread to asynchronously read serial port, extract a valid message, and for each message, spawn a new thread to process them
Methods 1,2 and 3 differ by the number of threads the general workload is split up into, though I don't know which is best.
I'm currently using method 4, which is horribly inefficient and doesn't work well on lower-end computers, due to the enormous number of threads being spawned, and every time I move or interact with the GUI, serial communication halts. Spawning a thread for each message also makes the order of the messages non-deterministic, which hasn't been a major problem so far...
Are there other methods, what are the pros (if any) and cons of each, and which is the best to use? Thanks!
EDIT: A problem with processing messages in the main thread is that interacting with GUI (even moving the window) would block the message processing function. Is there any way around this?
I think there are two main advantages that you can obtain by using multithreading:
Avoiding poor GUI performance due to the GUI-handling routines being held off by the serial port processing routine
(perhaps more important) Avoid loss of serial data caused by buffer overflow when the GUI routines hold off the serial-data-reading routine for too long.
You should only need to spawn a single thread. Just have that thread read data from the serial port as it comes in (by connecting the QSerialPort's readyRead() signal to a slot that calls read() on the QSerialPort object), and then emit a signal (with a QByteArray argument) whenever it wants to send some serial data to the GUI. Your main/GUI thread can receive the data via a QueuedConnection that will not block either the serial-thread or the main/GUI thread.
That's pretty much all there is to it; the only other thing to worry about is a clean shutdown. Be sure to have another cross-thread signal/slot connection to the QThread's quit() slot, so that when it's time to quit, you can emit that signal and then call wait() on the QThread to wait for it to respond by going away. Once wait() has returned you can safely delete the QThread object.
You can avoid additional threads at all by simply relying on Qt event loop (so far the main thread, the one also handling the GUI to be clear, will be blocked only when a message is actually received by the serial port).
Otherwise if you want to completely handle serial port in a dedicated thread, then the solution is to implement a class deriving from QThread and then override the run() function with something like this:
void MyClass::run()
{
QSerialPort port;
// ... serial port initialization here
// Connect signals/slots
connect(&port, SIGNAL(readyRead()), this, SLOT(readData()));
port.open();
// Start a new message loop on this thread
exec();
}
Where readData is a function implemented in MyClass for handling the received data. Since port is owned by the new thread (being created in run()) then its events will be handled by the thread itself (in a completely independent manner with respect to the main thread).
If you want at some point communicate something to the main thread (e.g.: you received something on serial which should cause a change in your GUI) then you can still use Qt's signals/slots. Simply implement a signal on MyClass and implement a slot on an object handled by the main thread (e.g.: your main form): then simply connect the signal for MyClass and the slot on your main form and you're done: signals/slots is THE solution for cross-thread communication in Qt.
You could also avoid using any (additional) threads and take advantage of Qt event loop. Read about events, QioDevice; then Qt would pass your device file descriptor to its multiplexing loop (e.g. to poll(2)....); probably QSocketNotifier should work (on Posix) on a non-socket file descriptor like a serial device.
Details are probably OS specific
Currently I am implementing a simple client-server program with just the basic functionalities of read/write.
However I noticed that if for example my server calls a write() to reply my client, and if my client does not have a corresponding read() function, my server program will just hang there.
Currently I am thinking of using a simple timer to define a timeout count, and then to disconnect the client after a certain count, but I am wondering if there is a more elegant/or standard way of handling such errors?
There are two general approaches to prevent server blocking and to handle multiple clients by a single server instance:
use POSIX threads to handle each client's connection. If one thread blocks because of erroneous client, other threads will still continue to run. If the remote client has just disappeared (crashed, network down, etc.), then sooner or later the TCP stack will signal a timeout and the blocked write operation will fail with error.
use non-blocking I/O together with a polling mechanism, e.g. select(2) or poll(2). It is quite harder to program using polling calls though. Network sockets are made non-blocking using fcntl(2) and in cases where a normal write(2) or read(2) on the socket would block an EAGAIN error is returned instead. You can use select(2) or poll(2) to wait for something to happen on the socket with an adjustable timeout period. For example, waiting for the socket to become writable, means that you will be notified when there is enough socket send buffer space, e.g. previously written data was flushed to the client machine TCP stack.
If the client side isn't going to read from the socket anymore, it should close down the socket with close. And if you don't want to do that because the client still might want to write to the socket, then you should at least close the read half with shutdown(fd, SHUT_RD).
This will set it up so the server gets an EPIPE on the write call.
If you don't control the clients... if random clients you didn't write can connect, the server should handle clients actively attempting to be malicious. One way for a client to be malicious is to attempt to force your server to hang. You should use a combination of non-blocking sockets and the timeout mechanism you describe to keep this from happening.
In general you should write the protocols for how the server and client communicate so that neither the server or client are trying to write to the socket when the other side isn't going to be reading. This doesn't mean you have to synchronize them tightly or anything. But, for example, HTTP is defined in such a way that it's quite clear for either side as to whether or not the other side is really expecting them to write anything at any given point in the protocol.
I have an application wherein serial I/O is conducted with an attached USB device via a virtual COM port. When surprise removal of the device is detected, what would be the best way to stop the serial I/O. Should I simply close the port? Or, should there be a global variable, which is maintained to indicate the presence of the device, that should be checked in each serial I/O function prior to attempting to transmit/receive data? Or, should it be a combination of the two, or something else? Thanks.
I'm assuming you are running Windows.
This depends on how you have designed your communication flow.
I have a BasePort object where I have derived a COMPort object (and many other communication objects). The COMPort object creates one TXThread and RXThread class. These threads are waiting for the "OVERLAP" to signal that the read or write operation finished with WaitForMultipleObjects().
The TXThreads goes to sleep if there is nothing to do and wakes up by the TXWrite function (the data between main process and thread goes through a trhead safe FIFO buffer).
In this case they also need to wait for an event signal that the port has closed, so they actually can cancel any pending operations and exit (the treads exits and gets deleted).
To detect if the USB port is connectd/disconneted I listen for the Windows message DEVICE_CHANGE. If the port is disconnected I set the event and waits for the threads to exit before the Port class deletes and closes the port.
I have found this approach very reliable and safe. It's the core in a communication platform I designed for over 8 years ago and still kicking.
In my program there is one thread (receiving thread) that is responsible for receiving requests from a TCP socket and there are many threads (worker threads) that are responsible for processing the received requests. Once a request is processed I need to send an answer over TCP.
And here is a question. I would like to send TCP data in the same thread that I use for receiving data. This thread after receiving data usually waits for new data in select(). So once a worker thread finished processing a request and put an answer in the output queue it has to signal the receiving thread that there are data to send. The problem is that I don't know how to cancel waiting in select() in order to get out of waiting and to call send() .
Or shall I use another thread solely for sending data over TCP?
Updated
MSalters, Artyom thank you for you answers!
MSalters, having read your answer I found this site: Winsock 2 I/O Methods and read about WSAWaitForMultipleEvents(). My program in fact must work both on HP-UX and Windows I finally decided to use the approach that had been suggested by Artyom.
You need to use something similar to safe-pipe trick, but in your case you need to use a pair of connected TCP sockets.
Create a pair of sockets.
Add one to the select and wait on it as well
Notify by writing to other socket from other threads.
Select is immediately waken-up as one of the sockets is readable, reads all the
data in this special socket and check all data in queues to send/recv
How to create pair of sockets under Windows?
inline void pair(SOCKET fds[2])
{
struct sockaddr_in inaddr;
struct sockaddr addr;
SOCKET lst=::socket(AF_INET, SOCK_STREAM,IPPROTO_TCP);
memset(&inaddr, 0, sizeof(inaddr));
memset(&addr, 0, sizeof(addr));
inaddr.sin_family = AF_INET;
inaddr.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
inaddr.sin_port = 0;
int yes=1;
setsockopt(lst,SOL_SOCKET,SO_REUSEADDR,(char*)&yes,sizeof(yes));
bind(lst,(struct sockaddr *)&inaddr,sizeof(inaddr));
listen(lst,1);
int len=sizeof(inaddr);
getsockname(lst, &addr,&len);
fds[0]=::socket(AF_INET, SOCK_STREAM,0);
connect(fds[0],&addr,len);
fds[1]=accept(lst,0,0);
closesocket(lst);
}
Of course some checks should be added for return values.
select is not the native API for Windows. The native way is WSAWaitForMultipleEvents. If you use this to create an alertable wait, you can use QueueUserAPC to instruct the waiting thread to send data. (This might also mean you don't have to implement your own output queue)
See also this post:
How to signal select() to return immediately?
For unix, use an anonymous pipe. For Windows:
Unblocking can be achieved by adding a dummy (unbound) datagram socket to fd_set and then closing it. To make this thread safe, use QueueUserAPC:
The only way I found to make this multi-threadsafe is to close and recreate the socket in the same thread as the select statement is running. Of course this is difficult if the thread is blocking on the select. And then comes in the windows call QueueUserAPC. When windows is blocking in the select statement, the thread can handle Asynchronous Procedure Calls. You can schedule this from a different thread using QueueUserAPC. Windows interrupts the select, executes your function in the same thread, and continues with the select statement. You can now in your APC method close the socket and recreate it. Guaranteed thread safe and you will never loose a signal.
The typical model is for the worker to handle its own writing. Is there a reason why you want to send all the output-IO through selecting thread?
If you're sure of this model, you could have your workers send data back to the master thread using file descriptors as well (pipe(2)) and simply add those descriptors to your select() call.
And, if you're especially sure that you're not going to use pipes to send data back to your master process, the select call allows you to specify a timeout. You can busy-wait while checking your worker threads, and periodically call select to figure out which TCP sockets to read from.
Another quick&dirty solution is to add localhost sockets to the set. Now use those sockets as the inter-thread communication queues. Each worker thread simply sends something to its socket, which ends up on the corresponding socket in your receiving thread. This wakes up the select(), and your receiving thread can then echo the message on the appropriate outgoing socket.