I have a Qt based TCP client and server making use of QTcpServer and QTcpSocket classes for communication. The server is compiled using Qt 5.3.1 and the client is compiled using Qt 4.8.1. This is done so because the client is a part of a framework that uses Qt 4.8.1 running on Ubuntu 12.04.
Since the classes I make use is available in both Qt versions I assume this wont create a problem.
However my client has some weird issues that it does not receive data from the server! I checked the server side and the data is sent from the server and I can also see the data packet on the wire using wireshark. However on my client code, the data does not arrive!
I investigated this a bit and it led me to a strange conclusion that this happens only if I use the read method of QTcpSocket! If I use the native POSIX read system call, I am able to read the data correctly! Please see my code below:
qDebug() << "QTcpSocket::bytesAvailable() gives" << m_pSocket->bytesAvailable();
char nData;
qint32 szReceived;
if(sizeof(char) != (szReceived = m_pSocket->read((char*)&nData,sizeof(char))))
{
qDebug() << "Error reading data from QTcpSocket::read()" << m_pSocket->errorString();
}
else
{
qDebug() << "QTcpSocket::read() returned" << szReceived;
}
int nDesc = m_pSocket->socketDescriptor();
if(sizeof(char) != (szReceived = read(nDesc, &nData,sizeof(char))))
{
perror("Error reading data from POSIX read()");
}
else
{
qDebug() << "POSIX read() returned" << szReceived;
}
This produces the following output:
QTcpSocket::bytesAvailable() gives 0
Error reading data from QTcpSocket::read() "Network operation timed out"
POSIX read() returned 1
How is it that the POSIX system calls reads the buffered data as expected and the Qt class cannot read it? Plus I have not set any socket options and so I don't know why it reports an error that network operation timed out!
"read" is a blocking call in POSIX, it waits till the data is arrived. while QTcpSocket is non-blocking operation it immediately returns the buffered data. Call waitForReadyRead before doing a read
socket->waitForReadyRead();
if(sizeof(char) != (szReceived = m_pSocket->read((char*)&nData,sizeof(char))))
I think that it is misuse of QTcpSocket concept. QTcpSocket implements asynchronous architecture while POSIX read/write calls are blocking until the success or error of I/O on socket. It is much better to process read in slot for readyRead signal. Consider this:
class MyClient
{
Q_OBJECT
...
private slots:
readFromSocket();
};
In your intialization:
QObject::connect(
m_pSocket, SIGNAL(readyRead()),
this, SLOT(readFromSocket()));
And real job done here:
void
MyClient::readFromSocket()
{
QByteArray buffer = m_pSocket->readAll();
// All your data in buffer.
}
I'm aware of the non-blocking nature of QTcpSocket and blocking nature of POSIX read call. Unfortunately I cannot use the signal readFromSocket because my communication architecture expects a header to be sent before each communication (TCP way) to see the payload that is streamed for that particular message. Hence I have to wait till I receive at least the header.
I do believe that this has something to do with the mode (blocking or non-blocking). I did some more tests and none of them were conclusive. In one of my tests, I tried to call a waitForReadyRead with a timeout of 1ms, 2ms, 3ms. This still wasn't sufficent for the read to succeed! I doubt if the read would need such time to read from the kernel buffers to user space as I can clearly see from wireshark that the message was received within 400ms.
When I give -1 as the timeout value of waitForReadyRead, the read succeeds! To put it in another way, the read succeeds only when the socket waits indefinitely like in the case of POSIX read call.
Another strange thing I observed was, this issue was originally observed when I was running a server compiled using Qt 5.3.1 and client compiled using Qt 4.8.1. When I compile my client to use Qt 5.3.1, I do not see this problem!!! I even tried compiling using Qt 4.7.1 and it worked without any issues!!!
Are there any known issues with socket implementation of Qt 4.8.1? I couldn't find much info regarding this unfortunately.
Related
I'm having a strange behaviour with the recv() function.
My C++ (MFC) application with WinSock implements a simple HTTP client (non-blocking socket) for accessing HTML pages on a web server. Some of these pages are taking a few seconds for loading. On Windows 7 this is not a problem, because recv() also returns partial data. But on Windows XP the recv() function always returns SOCKET_ERROR and the error code is WSAEWOULDBLOCK. Only when the connection is finished the data is returned in one access.
Does anyone know this problem? How can I force Windows XP to also receive partial data?
I setted the buffer size (SO_RCVBUF) to 1000 Bytes. On Windows 7 this is also reflected to the TCP Window Size - on XP not.
The real problem which I have with this issue is, that I don't know how to check if the connection is still alive or not. How can I check if a connection is still alive? Or how can I specify a timeout (max time between two received packets from the server)?
By default, a socket operates in blocking mode, so the only way you can get a WSAEWOULDBLOCK error at all is if you explicitly put the socket into non-blocking mode instead. Doing so, you agree to handle WSAEWOULDBLOCK (otherwise, don't use non-blocking mode).
WSAEWOULDBLOCK is not a real error, it is just an indication that the operation you attempted to perform cannot be completed at that moment because it would block the calling thread. You need to detect this "error" and simply retry the same operation again at a later time, preferably after a socket state change is detected.
For recv(), WSAEWOULDBLOCK simply means there is no data available on the socket to be read at that moment. In non-blocking mode, you should be using select() (or WSAEventSelect(), or WSAAsyncSelect(), or Overlapped I/O, or an I/O Completion Port) to detect inbound data before you then read it.
That being said, you are implementing an HTTP client, so you must follow the HTTP protocol properly, regardless of the socket I/O mode you are using, regardless of your socket buffer sizes. You must follow the pseudo code logic I outlined in this answer on another question:
You must follow the rules outlined in RFC 2616. Namely:
Read until the "\r\n\r\n" sequence is encountered. Do not read any more bytes past that yet.
Analyze the received headers, per the rules in RFC 2616 Section 4.4. They tell you the actual format of the remaining response data.
Read the data per the format discovered in #2.
Check the received headers for the presence of a Connection: close header if the response is using HTTP 1.1, or the lack of a Connection: keep-alive header if the response is using HTTP 0.9 or 1.0. If detected, close your end of the socket connection because the server is closing its end. Otherwise, keep the connection open and re-use it for subsequent requests (unless you are done using the connection, in which case do close it).
Process the received data as needed.
In short, you need to do something more like this instead (pseudo code):
string headers[];
byte data[];
string statusLine = read a CRLF-delimited line;
int statusCode = extract from status line;
string responseVersion = extract from status line;
do
{
string header = read a CRLF-delimited line;
if (header == "") break;
add header to headers list;
}
while (true);
if ( !((statusCode in [1xx, 204, 304]) || (request was "HEAD")) )
{
if (headers["Transfer-Encoding"] ends with "chunked")
{
do
{
string chunk = read a CRLF delimited line;
int chunkSize = extract from chunk line;
if (chunkSize == 0) break;
read exactly chunkSize number of bytes into data storage;
read and discard until a CRLF has been read;
}
while (true);
do
{
string header = read a CRLF-delimited line;
if (header == "") break;
add header to headers list;
}
while (true);
}
else if (headers["Content-Length"] is present)
{
read exactly Content-Length number of bytes into data storage;
}
else if (headers["Content-Type"] == "multipart/byteranges")
{
string boundary = extract from Content-Type header;
read into data storage until terminating boundary has been read;
}
else
{
read bytes into data storage until disconnected;
}
}
if (!disconnected)
{
if (responseVersion == "HTTP/1.1")
{
if (headers["Connection"] == "close")
close connection;
}
else
{
if (headers["Connection"] != "keep-alive")
close connection;
}
}
check statusCode for errors;
process data contents, per info in headers list;
As you can see, HTTP requires reading CRLF-delimited lines of text, or fixed lengths of raw bytes. To do that, you must call recv() in a loop until you encounter the terminating CRLF, or have received the expected number of bytes, whichever the case may be. Whether you use a synchronous loop that just ignores WSAEWOULDBLOCK errors while looping, or you use a state machine driven by asynchronous events/callbacks, that is up to you to decide. That doesn't change how you must process the HTTP protocol.
This applies to all versions of Windows (even all platforms that use BSD-style socket APIs). What you are encountering is not a Windows bug at all. It is an underlying flaw in your understanding of how to use socket I/O correctly and effectively.
As for checking if the connection is alive, recv() will return 0 if the server closed the connection gracefully, or will report an error otherwise (usually WSAECONNABORTED or WSAECONNRESET, though there can be others). But an abnormal disconnect may take a long time to detect, so you should implement timeouts in your code instead. In synchronous mode, you can use setsockopt(SO_RCVTIMEO). In non-blocking mode, you can use select(). In asynchronous (overlapped) mode, you can use WaitForSingleObject() on whatever event/object you use to drive your state machine.
You can't expect recv to give you any data on a non-blocking socket. If there's no data available it returns WOULDBLOCK. You just need to call recv again (normally after select notifies you some data is available). Whether you get data on the first (or any) call is going to depend on how fast the server is sending it.
When the socket is closed you'll get a different error from recv, like WSAECONNRESET or WSAENOTCONN. select will also notify you when the socket is closed.
It's very strange.
Today I have changed my software to use blocking sockets. But it still doesn't work on Windows XP. Windows 7 is no problem.
So I thought: Let's try another PC. On this PC (also Windows XP) it does work. Now I tried a 3rd PC with Windows XP and here it also works.
I still don't know what the problem is but I think there must be a bug with the PC.
I am trying to write data to Serial Port. I use this way:
QBluetoothSocket *socket;
socket = new QBluetoothSocket(QBluetoothServiceInfo::RfcommProtocol);
socket->open(QIODevice::WriteOnly);
QByteArray byteArr;
QDataStream out(&byteArr, QIODevice::WriteOnly);
out << 1 << '\n'; //if plusbutton is pushedm then send zero
socket.write(byteArr);
qDebug()<<socket.write(byteArr)<<endl;
But I get in return:
W/libA_for_w8.so( 6443): (null):0 ((null)): qt.bluetooth.android:
Socket::writeData: QBluetoothSocket::ConnectingState false
W/libA_for_w8.so( 6443): (null):0 ((null)): qt.bluetooth.android:
Socket::writeData: QBluetoothSocket::ConnectingState false
D/libA_for_w8.so( 6443): ..\A_for_w8\widget.cpp:68 (void Widget::on_plus_clicked()): -1
D/libA_for_w8.so( 6443):
So, nothing is written to there.
It seems, that the code is ok, but it doesn't work.
Can you tell me what's wrong?
Thank you.
P.S.
I checked socket is open .
socket->isOpen return true and I get:
W/libA_for_w8.so( 9638): (null):0 ((null)): qt.bluetooth.android: Socket::writeData: QBluetoothSocket::UnconnectedState false
D/libA_for_w8.so( 9638): ..\A_for_w8\widget.cpp:70 (void Widget::on_plus_clicked()): -1
You need to connect the socket to an endpoint before you can successfully write anything. There must be a running Bluetooth service somewhere, to which you connect using connectToService(). After the connected() signal has been emitted, you can write data.
It doesn't appear that QBluetoothSocket has a waitForConnected() function, as QAbstractSocket and its subclasses do. This means you can use something like QSignalSpy to wait for the connected() signal, or, if you don't care so much about efficiency, just a busy-wait loop would do.
EDIT:
As pointed out in the comment, waiting for events to happen is the whole point of Qt's main event loop. Using the waitFor* methods or another event loop is pretty inefficient, and definitely goes against the spirit of Qt. So the best solution to writing data only after the socket is connected is to connect a writeData() slot to the connected() signal of QBluetoothSocket.
I have a client program written with raw socket communicating with a server. It uses 'select' to poll the socket descriptor and get the message from the server. After getting message, I use a ZMQ socket to send the message out.
This works perfectly in windows. But in Linux, after I call the ZMQ 'send' function, I cannot receive message from the server any more. It seems the raw socket has been affected somehow.
The program is a multithread program but I have dealt with ZMQ carefully. The client program is running in a thread A and the ZMQ socket is only used by that thread A. I think I used ZMQ correctly as it is working well in Windows. But how comes the issue in Linux?
Anybody knows if this is normal? I am suspecting this is a ZMQ issue. As long as I remove the ZMQ 'send' function, the program works well. Anybody knows how to solve this issue? and why it is working well in Windows, not in Linux?
I found it doesn't work even in single thread situation. ZMQ creates more than ten threads automatically for me. I don't use any multithread this time, and the code flow is like this:
create a raw socket A;
connect to a business server B;
STATE = 1
while (raw socket A is connected){
if (STATE==1){
send(Request 1);
}else if (STATE==2){
send(Request 2);
}
int ret = select(fd + 1, &readSet, &writeSet, &errorSet, &timeout);
if (ret > 0){
if (FD_ISSET(fd, &readSet)) {
char buf[8192];
int nResult = receive( buf, sizeof(buf));
if (buf is X){
zmq.send(messageX)
STATE=2
}else if (buf is Y){
STATE=3
}else{...}
}// socket is ready for reading
}
}
After zmq.send(message), I cannot receive any new message!! The server side code is closed to me so that I cannot debug from server side. If I remove "zmq.send(message)", everything works fine. In Windows, this program works fine too.
Well, it is hard to tell u whats wrong without a code. But you should probably look at ZMQ versions in Linux and Windows because the "formats of sending" etc can differ between versions and stuff that works in one version will not in another.
I have a UDP server using the following code:
void initialize()
{
connect(&_udpSocket, SIGNAL(readyRead()), this, SLOT(onUdpDatagram()));
_udpSocket.bind(QHostAddress::Any, 28283);
}
void onUdpDatagram()
{
qDebug() << "udp packet received!";
_udpSocket.write("Hello");
}
Unfortunately when a UDP packet is received, I have the following error in the log:
QIODevice::write: device not open
How can I make the UDP socket writable? I tried to create another socket for the answer that connect to the sender address and port but the sending won't use the 28283 port anymore...
Any idea?
For info: I'm using Qt 5.2.1 on MacOS 10.9
UDP is not a connection-based protocol. You don't get a separate socket for each peer, instead there's one socket for all communication on a single port.
Therefore, there's some extra effort needed to reply to an incoming UDP packet. You need to retrieve the sender address from the datagram you received, and send back to that same address. In the sockets API this is done by using recvfrom and sendto functions instead of recv (or read) and send (or write) -- the latter are designed for connected sockets like you use with TCP.
You didn't show the declaration (really, the type) for your _udpSocket variable, so I'm assuming that you are using a QUdpSocket. In that case, it looks like you will want to use the readDatagram and writeDatagram functions, which like recvfrom and sendto, have an additional parameter for the peer address (actually, it's a pair, one for the IP address, one for the port).
Here's what the Qt documentation says about that:
The most common way to use this class is to bind to an address and port using bind(), then call writeDatagram() and readDatagram() to transfer data. If you want to use the standard QIODevice functions read(), readLine(), write(), etc., you must first connect the socket directly to a peer by calling connectToHost().
Coincidentally, this warning was introduced by me in Qt upstream:
QIODevice::write: device not open
It should be pretty clear unlike before the introduction of this, namely: you have forgotten to connect to the host with your udp socket. You cannot expect it to write and/or read if it is not even open and/or connected. See the documentation for details:
If you want to use the standard QIODevice functions read(), readLine(), write(), etc., you must first connect the socket directly to a peer by calling connectToHost().
You have to do something like this somewhere in your code:
_udpSocket.connectToHost(myHostAddress, 28283, ReadWrite, AnyIPProtocol);
The last two parameters can be skipped as they are the default. As you can read from the documentation, this method call will open the socket for you, too, which is necessary to get done for QIODevice read and write operations.
That being said, you really should not neglect error checking in your code as it currently seems to stand. It will be difficult to find the issues this way.
Also, it is ice on the cake, but I would encourage you to start using the "new" signal-slot syntax, which is not so new, but much more modern and handier:
void initialize()
{
connect(&_udpSocket, &QUdpSocket::connected, [&_udpSocket]() {
connect(&_udpSocket, &QUdpSocket::readyRead, [&_udpSocket]() {
qDebug() << "udp packet received!";
if (_udpSocket.write("Hello") != 6)
qDebug() << "Failed to write:" << _udpSocket.errorString();
});
});
connect(&_udpSocket, &QUdpSocket::error, [&_udpSocket]() {
qDebug() << "Error occured:" << _udpSocket.errorString();
});
_udpSocket.connectToHost(myHostAddress, 28283, ReadWrite, AnyIPProtocol);
}
I am trying to connect a signal and a slot in C++ using the boost libraries. My code currently opens a file and reads data from it. However, I am trying to improve the code so that it can read and analyze data in real time using a serial port. What I would like to do is have the analyze functions called only once there is data available in the serial port.
How would I go about doing this? I have done it in Qt before, however I cannot use signals and slots in Qt because this code does not use their moc tool.
Your OS (Linux) provides you with the following mechanism when dealing with the serial port.
You can set your serial port to noncanonical mode (by unsetting ICANON flag in termios structure). Then, if MIN and TIME parameters in c_cc[] are zero, the read() function will return if and only if there is new data in the serial port input buffer (see termios man page for details). So, you may run a separate thread responsible for getting the incoming serial data:
ssize_t count, bytesReceived = 0;
char myBuffer[1024];
while(1)
{
if (count = read(portFD,
myBuffer + bytesReceived,
sizeof(myBuffer)-bytesReceived) > 0)
{
/*
Here we check the arrived bytes. If they can be processed as a complete message,
you can alert other thread in a way you choose, put them to some kind of
queue etc. The details depend greatly on communication protocol being used.
If there is not enough bytes to process, you just store them in buffer
*/
bytesReceived += count;
if (MyProtocolMessageComplete(myBuffer, bytesReceived))
{
ProcessMyData(myBuffer, bytesReceived);
AlertOtherThread(); //emit your 'signal' here
bytesReceived = 0; //going to wait for next message
}
}
else
{
//process read() error
}
}
The main idea here is that the thread calling read() is going to be active only when new data arrives. The rest of the time OS will keep this thread in wait state. Thus it will not consume CPU time. It is up to you how to implement the actual signal part.
The example above uses regular read system call to get data from port, but you can use the boost class in the same manner. Just use syncronous read function and the result will be the same.