Qt: Differenciate between messages received using readAll() [duplicate] - c++

I know that TCP guarantees that all packets will arrive. But can a packet be separated into 2 or more? I am using Qt with class QTcpSocket, and I want to know is ReadyRead() signal is emitted only when full packet arrives. Or in other words, is there any sense to send packet size in first bytes and then wait in loop until all bytes had arrived? Or I can just call socket->readAll() and I must get one full packet?

If a large amount of data is sent, the packet can arrive in separate parts. Alternatively, multiple messages can be received in one readyRead slot.
It's good practice to control this by setting the first byte(s) to the number of bytes that will be sent. Then, in readyRead, you read the first bytes and append the data to a buffer until the expected amount of data has been received.
In receiving data, this also means that if multiple messages are received in one call to readyRead(), you can know where the first message ends and the next one begins.
Here's an example of a client that receives data in a readyRead function()
void MyClass::readyRead()
{
// m_pConnection is a QTcpSocket
while(m_pConnection->bytesAvailable())
{
QByteArray buffer;
int dataSize;
m_pConnection->read((char*)&dataSize, sizeof(int));
buffer = m_pConnection->read(dataSize);
while(buffer.size() < dataSize) // only part of the message has been received
{
m_pConnection->waitForReadyRead(); // alternatively, store the buffer and wait for the next readyRead()
buffer.append(m_pConnection->read(dataSize - buffer.size())); // append the remaining bytes of the message
}
QString msg(buffer); // data in this case is JSON, so we can use a QString
emit Log(QString("\tMessage Received: %1").arg(msg));
// Do something with the message
ProcessMessage(msg);
}
}

Related

UART doesn't receive all sent values in Qt

I'm triying to send data from a PSoc via UART to my PC where a want to store data with Qt. The PSoc sends 3 bytes of data. Theses 3 bytes are repeatet with a frequency of 2.5Hz. When I check the signals with my oscilloscope everything is fine. When I receive the data with the software HTerm also everything is as expected. When I use my code written in c++ with Qt I get the problem that not all data are received in Qt, only one third is in the memory. I expected that the signal readyRead is emitted with every new byte? But it seems that the signal is only emitted at the begin of the package of the 3 bytes. Also my qDebug output doesn't react on changes from the PSoc. So when I change values at PSoc the output in qDebug doesn't change.
I already tried reading 3 Bytes (serial->read(3)) and then I first received some single bytes and after a few readings I get the 3 bytes I sended but this is not so reproducible.
connect(serial, SIGNAL(readyRead()), this, SLOT(readData()));
serial->setPortName(gui->ui->comboBox->currentData().toString());
serial->setBaudRate(QSerialPort::Baud115200);
serial->setDataBits(QSerialPort::Data8);
serial->setParity(QSerialPort::NoParity);
serial->setStopBits(QSerialPort::OneStop);
serial->setFlowControl(QSerialPort::NoFlowControl);
void uart::readData()
{
QByteArray data = serial->read(1);
qDebug() << data;
}
I expect an output like "0x01" "0x02" "0x03" 2.5 times a second, but I get only "0x01"
You are only reading a fixed size with read.
Could it be that you get readyRead signals with varying bytes available but you only read fixed size of them
In your readyRead slot try to read all available bytes.
qint64 available = serial->bytesAvailable();
if (available > 0)
{
QByteArray data = serial->read(available);
qDebug() << data;
}
You can also use readAll() function.
I just found the solution!
You have to set the read-buffer size to the right value.
So for reading a package of three bytes I must set:
serial->setReadBufferSize(3);

QTcpSocket data transfer stops when read buffer is full and does not resumes when it frees up

I have server-client Qt application, where client sends data packets to server and server reads them at a set time intervals. It happens that client sends data faster than server can read thus filling all the memory on the server side. I am using QAbstractSocket::setReadBufferSize(size) to set max read buffer size on the server side and when it fills up, socket data transferring stops, and data is buffered on client side, which is what i want, but the problem is when server's QTcpSocket's internal read buffer frees up (is not full anymore), data transfer between client and server does not resume.
I've tried to use QAbstractSocket::resume() which seems to work, but Qt5.10 documentation says:
Continues data transfer on the socket. This method should only be used
after the socket has been set to pause upon notifications and a
notification has been received. The only notification currently
supported is QSslSocket::sslErrors(). Calling this method if the
socket is not paused results in undefined behavior.
I feel like I should not use that function in this situation, but is there any other solution? How do i know if socket is paused? Why data transfer does not continue automaticaly when QTcpSocket's internal read buffer is not full anymore?
EDIT 1 :
I have downloaded Qt(5.10.0) sources and pdb's to debug this situation and I can see that QAbstractSocket::readData() internal function have line "d->socketEngine->setReadNotificationEnabled(true)" which re-enables data transfering, but QAbstractSocket::readData() gets called only when QTcpSocket internal read buffer is empty (qiodevice.cpp; QIODevicePrivate::read(); line 1176) and in My situation it is never empty, because I read it only when it has enough data for complete packet.
Shouldn't QAbstractSocket::readData() be called when read buffer is not full anymore and not when it's completely empty? Or maybe i do something wrong?
Found a Workaround!
In Qt5.10 sources i can clearly see that QTcpSpcket internal read notifications is disabled (qabstractsocket.cpp; bool QAbstractSocketPrivate::canReadNotification(); line 697) when read buffer is full and to enable read notifications you need to read all buffer to make it empty OR use QAbstractSocket::setReadBufferSize(newSize) which internally enables read notifications WHEN newSize is not 0 (unlimited) and not equal to oldSize (qabstractsocket.cpp; void QAbstractSocket::setReadBufferSize(qint64 size); line 2824).
Here's a short function for that:
QTcpSocket socket;
qint64 readBufferSize; // Current max read buffer size.
bool flag = false; // flag for changing max read buffer size.
bool isReadBufferLimitReached = false;
void App::CheckReadBufferLimitReached()
{
if (readBufferSize <= socket.bytesAvailable())
isReadBufferLimitReached = true;
else if (isReadBufferLimitReached)
{
if (flag)
{
readBufferSize++;
flag = !flag;
}
else
{
readBufferSize--;
flag = !flag;
}
socket.setReadBufferSize(readBufferSize);
isReadBufferLimitReached = false;
}
}
In the function which reads data from QTcpSocket at the set intervals, BEFORE reading data, I call this function, which checks if read buffer is full and sets isReadBufferLimitReached if true. Then I read needed amount of data from QTcpSocket and AT THE END I call that function again, which, if buffer were full before, calls QAbstractSocket::setReadBufferSize(size) to set new buffer size and enable internal read notifications. Changing read buffer size by +/-1 should be safe, because you read at least 1 byte from socket.

receive whole data from qtcpsocket [duplicate]

I know that TCP guarantees that all packets will arrive. But can a packet be separated into 2 or more? I am using Qt with class QTcpSocket, and I want to know is ReadyRead() signal is emitted only when full packet arrives. Or in other words, is there any sense to send packet size in first bytes and then wait in loop until all bytes had arrived? Or I can just call socket->readAll() and I must get one full packet?
If a large amount of data is sent, the packet can arrive in separate parts. Alternatively, multiple messages can be received in one readyRead slot.
It's good practice to control this by setting the first byte(s) to the number of bytes that will be sent. Then, in readyRead, you read the first bytes and append the data to a buffer until the expected amount of data has been received.
In receiving data, this also means that if multiple messages are received in one call to readyRead(), you can know where the first message ends and the next one begins.
Here's an example of a client that receives data in a readyRead function()
void MyClass::readyRead()
{
// m_pConnection is a QTcpSocket
while(m_pConnection->bytesAvailable())
{
QByteArray buffer;
int dataSize;
m_pConnection->read((char*)&dataSize, sizeof(int));
buffer = m_pConnection->read(dataSize);
while(buffer.size() < dataSize) // only part of the message has been received
{
m_pConnection->waitForReadyRead(); // alternatively, store the buffer and wait for the next readyRead()
buffer.append(m_pConnection->read(dataSize - buffer.size())); // append the remaining bytes of the message
}
QString msg(buffer); // data in this case is JSON, so we can use a QString
emit Log(QString("\tMessage Received: %1").arg(msg));
// Do something with the message
ProcessMessage(msg);
}
}

Broadcast large data with Qt sockets

I'm using QT. I need to broadcast data, so I try to use QUdpSocket. But data can be too big(after writeDatagram QUdpSocket::error returns DatagramTooLargeError). So I split data and call writeDatagram several times to the parts. But Received socket receive data only once, only first packet. Receive code is
connect(&m_socketReceiver, &QUdpSocket::readyRead, this, &LocalNetSharing::onDataRead);
void LocalNetSharing::onDataRead()
{
while (m_socketReceiver.hasPendingDatagrams())
{
QByteArray datagram;
datagram.resize(m_socketReceiver.pendingDatagramSize());
m_socketReceiver.readDatagram(datagram.data(), datagram.size());
//process data
}
}
From the Qt documentation about QUdpSocket Class :
Note: An incoming datagram should be read when you receive the
readyRead() signal, otherwise this signal will not be emitted for the
next datagram.
So it seems that you are not reading the entire datagram in each call of onDataRead.
You don't specify host and port in readDatagram. I am not sure if it is the reason but the correct form is :
while (m_socketReceiver.hasPendingDatagrams())
{
QByteArray datagram;
datagram.resize(m_socketReceiver.pendingDatagramSize());
m_socketReceiver.readDatagram(datagram.data(), datagram.size(), host, &port);
//process data
}

boost::asio async_read guarantee all bytes are read

I have a server that receives a compressed string (compressed with zlib) from a client, and I was using async_receive from the boost::asio library to receive this string, it turns out however that there is no guarantee that all bytes will be received, so I now have to change it to async_read. The problem I face is that the size of the bytes received is variable, so I am not sure how to use async_read without knowing the number of bytes to be received. With the async_receive I just have a boost::array<char, 1024>, however this is a buffer that is not necessarily filled completely.
I wondered if anyone can suggest a solution where I can use async_read even though I do not know the number of bytes to be received in advance?
void tcp_connection::start(boost::shared_ptr<ResolverQueueHandler> queue_handler)
{
if (!_queue_handler.get())
_queue_handler = queue_handler;
std::fill(buff.begin(), buff.end(), 0);
//socket_.async_receive(boost::asio::buffer(buff), boost::bind(&tcp_connection::handle_read, shared_from_this(), boost::asio::placeholders::error));
boost::asio::async_read(socket_, boost::asio::buffer(buff), boost::bind(&tcp_connection::handle_read, shared_from_this(), boost::asio::placeholders::error));
}
buff is a boost::array<char, 1024>
How were you expecting to do this using any other method?
There are a few general methods to sending data of variable sizes in an async manor:
By message - meaning that you have a header that defines the length of the expected message followed by a body which contains data of the specified length.
By stream - meaning that you have some marker (and this is very broad) method of knowing when you've gotten a complete packet.
By connection - each complete packet of data is sent in a single connection which is closed once the data is complete.
So can your data be parsed, or a length sent etc...
Use async_read_until and create your own match condition, or change your protocol to send a header including the number of bytes to expect in the compressed string.
A single IP packet is limited to an MTU size of ~1500 bytes, and yet still you can download gigabyte-large files from your favourite website, and watch megabyte-sized videos on YouTube.
You need to send a header indicating the actual size of the raw data, and then receive the data by pieces on smaller chunks until you finish receiving all the bytes.
For example, when you download a large file over HTTP, there is a field on the header indicating the size of the file: Content-Length:.