I have an application in c++/Qt in Windows, Linux and MacOSX, and I have a local client-server mechanism using Qt Local Server/Socket.
When the server refuses a connection, I send a message and close the socket, using QLocalSocket::disconnectFromServer()
From the documentation I see
If there is pending data waiting to be written, QLocalSocket will enter ClosingState and wait until all data has been written.
So in the client side, I connect the signal QLocalSocket::disconnected and do the following in my slot
void MyClientClass::onSocketDisconnected()
{
qDebug() << "Socket disconnected";
socket->readAll();
//I do something with the data read
}
But it happens that sometimes, mostly on slow Linux machines (I try with virtual machines with only 1 processor), I do not receive the last message that the server send just before closing.
Why this happens? It should be in contrast with the documentation, shouldn't it?
Is there some motivation why I noticed it only in Linux and MacOSX (where Qt uses unix domain sockets) and I did not noticed it in Windows (where Qt uses pipes)?
Maybe there is some motivation related to domain sockets? Or I just misunderstood something?
Edit: as written in the comments below, in these cases when I do not receive the message, also the QLocalSocket::ClosingState is missing and I receive directly the QLocalSocket::UnconnectedState
Edit 2: as suggested in comments, I tried the waitForReadyRead() function. I did it onSocketDisconnected() and also onStateChanged() when the state goes to QLocalSocket::ClosingState or QLocalSocket::UnconnectedState. When I receive the QLocalSocket::ClosingState everything goes well, but the problem, as I said in the last edit, is that sometimes QLocalSocket::ClosingState is missing. In this case, the waitForReadyRead() fails without waiting the timeout and I have a warning "QIODevice:read (QTcpSocket): device not open" (even if I am using QLocalSocket, maybe the warning message is not updated well in Qt source). By the way, I am using Qt5.7.
Related
I am creating a network client application that sends requests to a server using a QTcpSocket and expects responses in return. No higher protocol involved (HTTP, etc.), they just exchange somewhat simple custom strings.
In order to test, I have created a TCP server in Python that listens on a socket and logs the strings it receives and those it sends back.
I can send the first request OK and get the expected response. However, when I send the second request, it does not seem to get written to the network.
I have attached debug slots to the QTcpSocket's notification signals, such as bytesWritten(...), connected(), error(), stateChanged(...), etc. and I see the connection being established, the first request sent, the first response processed, the number of bytes written - it all adds up...
Only the second request never seems to get sent :-(
After attempting to send it, the socket sends an error(RemoteHostClosedError) signal followed by ClosingState and UnconnectedState state change signals.
Before I go any deeper into this, a couple of (probably really basic) questions:
do I need to "clear" the underlying socket in any way after reading ?
is it possible / probable that not reading all the data the server has sent me prevents me from writing ?
why does the server close the connection ? Does it always do that so quickly or could that be a sign that something is not right ? I tried setting LowDelay and KeepAlive socket options, but that didn't change anything. I've also checked the socket's state() and isValid() and they're good - although the latter also returns true when unconnected...
In an earlier version of the application, I closed and re-opened the connection before sending a request. This worked ok. I would prefer keeping the connection open though. Is that not a reasonable approach ? What is the 'canonical' way to to implement TCP network communication ? Just read/write or re-open every time ?
Does the way I read from the socket have any impact on how I can write to it ? Most sample code uses readAll(...) to get all available data; I read piece by piece as I need it and << to a QTextStream when writing...
Could this possibly be a bug in the Qt event loop ? I have observed that the output in the Qt Creator console created with QDebug() << ... almost always gets cut short, i.e. just stops. Sometimes some more output is printed when I shut down the application.
This is with the latest Qt 5.4.1 on Mac OS X 10.8, but the issue also occurs on Windows 7.
Update after the first answer and comments:
The test server is dead simple and was taken from the official Python SocketServer.TCPServer Example:
import SocketServer
class MyTCPHandler(SocketServer.StreamRequestHandler):
def handle(self):
request = self.rfile.readline().strip()
print "RX [%s]: %s" % (self.client_address[0], request)
response = self.processRequest(request)
print "TX [%s]: %s" % (self.client_address[0], response)
self.wfile.write(response)
def processRequest(self, message):
if message == 'request type 01':
return 'response type 01'
elif message == 'request type 02':
return 'response type 02'
if __name__ == "__main__":
server = SocketServer.TCPServer(('localhost', 12345), MyTCPHandler)
server.serve_forever()
The output I get is
RX [127.0.0.1]: request type 01
TX [127.0.0.1]: response type 01
Also, nothing happens when I re-send any message after this - which is not surprising as the socket was closed. Guess I'll have to figure out why it is closed...
Next update:
I've captured the network traffic using Wireshark and while all the network stuff doesn't really tell me a lot, I do see the first request and the response. Right after the client [ACK]nowledges the response, the server sends a Connection finish (FIN). I don't see the second request anywhere.
Last update:
I have posted a follow-up question at Python: SocketServer closes TCP connection unexpectedly.
Only the second request never seems to get sent :-(
I highly recommend running a program like WireShark and seeing what packets are actually getting sent and received across the network. (As it is, you can't know for sure whether the bug is on the client side or in the server, and that is the first thing you need to figure out)
do I need to "clear" the underlying socket in any way after reading ?
No.
is it possible / probable that not reading all the data the server has
sent me prevents me from writing ?
No.
why does the server close the connection ?
It's impossible to say without looking at the server's code.
Does it always do that so quickly or could that be a sign that
something is not right ?
Again, this would depend on how the server was written.
This worked ok. I would prefer keeping the connection open though. Is
that not a reasonable approach ?
Keeping the connection open is definitely a reasonable approach.
What is the 'canonical' way to to implement TCP network communication
? Just read/write or re-open every time ?
Neither was is canonical; it depends on what you are attempting to accomplish.
Does the way I read from the socket have any impact on how I can write
to it ?
No.
Could this possibly be a bug in the Qt event loop ?
That's extremely unlikely. The Qt code has been used for years by tens of thousands of programs, so any bug that serious would almost certainly have been found and fixed long ago. It's much more likely that either there is a bug in your client, or a bug in your server, or a mismatch between how you expect some API call to behave and how it actually behaves.
I implemented the Bluetooth server according to the example shown in http://www.codeproject.com/Articles/252882/Bluetooth-Server-Programming-on-Windows.
I modified it a bit to use AF_BTH, SOCK_STREAM, BTHPROTO_RFCOMM.
It works fine.
However, if I restart the computer, the program hangs at accept().
If I then un-pair the two devices and then re-pair them again, the program works.
Any clues as to why this is happening.
So the program hangs on accept, from what I know accept in windows bluetooth sockets and other Bluetooth sockets platforms, accept like functions are a blocking asynchronous call, it pretty much block/freezes the thread that is on, and waits a client to connect, now winsock2 handles one thread operations much better than other platforms yes i'm looking to you android ,but still the program might get unstable if two of those calls occur , furthermore the guy only checks for one error in accept , so try referring to this complete list of winsock errors
(retrieved by winsockgetlasterror()) ,
also can you pls provide the client code and more info like on what platform, ide,devices , you are using ?
I'm trying to connect to a device on COM3 and the code runs until I call open("COM3"), which causes a stack overflow. Here's the relevant code:
asio::io_service io;
asio::basic_serial_port<asio::serial_port_service> scope(io);
//Open the connection and configure it
cout << "OPENING\n";
system::error_code error;
scope.open(PORT, error);
After opening the connection I configure it with the baud rate, etc.
It's hanging in win_iocp_serial_port_service.ipp, inside of SetCommState(handle, &dcb).
I also have some labview code to connect, send a command, and disconnect, which works. If I've run the labview code since starting up my computer, then my C++ program works (connects without hanging), but if I haven't yet run the labview code it gives me a stack overflow. This makes me think that I'm not starting up some driver or setting something persistent but I'm not sure what it would be.
If anyone's run into this issue or has any insight I appreciate the help!
Info from further testing: Connecting from non-labview serial connection clients seems to enable boost to connect as well. If I first connect via hyperterminal it works, and if I connect via command line (per this guide https://learn.sparkfun.com/tutorials/terminal-basics/command-line-windows-mac-linux) then I can subsequently connect via boost as well, which might be a workable solution, even if its dumb. Unfortunately I couldn't successfully send data with System.IO.Ports.SerialPort so the temporary solution is connect using System.IO.Ports.SerialPort, disconnect, then connect using boost asio now that it works. This works reasonably well but the code now only works on windows.
Since you can use your serial instrument from LabVIEW, your hypothesis that you're "not starting up some driver or setting something persistent" is probably correct.
You can see how LabVIEW and VISA are configuring the port and sending commands using a tool provided by NI called I/O Trace [1]. Once you have the working settings and commands in hand, you can match them with your calls to boost::asio and determine if you are over- or under-configuring the port.
In the I/O trace logs, you'll see VISA setting the baud, flow control, and the other traits before opening a session. The driver doesn't share much more than that, however, so if your program is using the same settings and sequence but still hanging, then scrutinize how you're programming to the asio interface [2].
References
[1] Performing a Good NI I/O Trace Capture for Debugging/Troubleshooting
http://digital.ni.com/public.nsf/allkb/282C5D41E2BA04F2862574BA007803B9
[2] Serial ports and C++
http://www.webalice.it/fede.tft/serial_port/serial_port.html
I am working on a client/server application (using qt for tcp).
The clients have to send about 15 messages per second to the server.
The problem is this:
the messages from the clients are received in groups. What i mean:
when i get the readyRead() signal and i read the data from the socket, there are multiple messages in the buffer.
This of-course causes lag in the system.
I tried putting the incoming connections in separate threads (thread per connection) but there was no improvement.
I also tried to rise a thread each time i got a readyRead() signal, but again nothing...
BUT when i run a number of clients on the same pc as the server, everything seems ok. When using different pc's over the network, the lag occurs...
(the network used is 100Mbps LAN, the messages are <200KB, and ping between pc's is <5msec, so i don't believe it's a network issue)
On the client side, the code to write the data is pretty simple:
tcpSocket->write(message.toUtf8());
tcpSocket->waitForBytesWritten();
tcpSocket->flush();
I also tried it without flush() or waitForBytesWritten() but the same...
EDIT: Using Qt 4.8.4 and Windows 7 and XP
Anybody has any idea how to overcome this?
Thank you in advance!
The last time I ran into a similar problem was with the stdin/stdout communication of a QProcess of Qt3.3. It behaved completely different on Linux and Windows.
Finally we found out that on Linux it used select() to react asynchronously when data arrived (fast, in most cases only one line readable) while on Windows the existence of new data was polled via a QTimer from the Qt mainloop (large delay, several messages available). A workaround we tried was to reduce the timer period in the source of Qt, but at the end we switched to shared memory based on the native OS mechanisms.
Your description sounds like you are using a similar Qt version on a Windows OS.
I'm writing program that should control a piece of scientific hardware over COM-port. The program itself is written in wxWidgets and uses ctb library. To test, it before I connect it to 300k€ equipment, I use com0com (Null-modem emulator) to forward COM2 port. To emulate my hardware I use wxTerminal (COM3). Altogether it works nice. One can debug not only in VS or DB but also see the whole data transfer in wxTerminal.
Now to my problem. I use to send data to COM-port ctb::SerialPort::Write() function.
device->Write( (char*)line.c_str(), line.size() );
However, if I disconnect the connection on the side of wxTerminal (i.e. COM2->NULL) than program hangs in this function.
It's obvious that I should add some function to test if my equipment is still there, but to do it I need to send data-packet to it and expect some answer. So I'm back to the Write().
"Just in case" I've also tried ctb::IOBase::Writev (char ∗ buf, size_t len, unsigned int timeout_in_ms) with timeout set to 100ms and I've still got program hanging in the same line. It's actually expected behavior as in this case timeout means only that the connection line is blocked till whole buffer is transferred or timeout is reached.
Connecting of wxTerminal to COM3 leads to un-freezing of debugger or stand-alone program. The Sun is shining, the birds are singing.
Can somebody give me a hint how to overcome my problem? I'd appreciate if comments would be restrained to wxWidgets-world - I really do not want to re-write whole program with other toolkit.
If you COM port library does not provide effective timeouts on write block, (presumably because of hardware flow-control), you could implement your own by threading off the write. You could use a couple of events/semaphores/condvar/whatever. One to signal to the thread that there is something in a buffer to send and another that you can wait on with a timeout that is signaled by the thread after it has sent the buffer. If the 'ack' wait times out, your COM port is stuck and you can pop up some 'Check cable' messageBox. I don't know what other calls your port lib supports, so I don't know how you could implement flushes/retries.