Avoiding Timeout on QSerialPort when handling high speed data - c++

I'm working on a windows application that receives data from a sensor at 600Hz. In two out of five cases, my IO thread reads the 4 bytes of data from the sensor successfully and passes it on to the GUI thread.
The problem is three out of five times, QSerialPort has inexplicable timeouts where QSerialPort's waitForReadyRead() returns false and serial.errorString() has a timeout error. In which case it will never read data. If I read from the serial port despite the timeout error I will read 2000+ bytes of data in the next waitForReadyRead which will be delivered in chunks which renders the realtime data reception aspect of my application obsolete.
I've tried using the readyRead() signal of the serial port but it exhibits the same behaviour ie. if the timeout error appears, no readyRead() signal is ever fired.
UPDATE: I am able to reproduce the issue with Qt's terminal example ([QT_INSTALL_EXAMPLES]/serialport/terminal) which uses a non-blocking read. The frequency of the bug is considerably less but it's definitely still there.
UPDATE: Using Serial Port Monitor, I can see that when it gets stuck, the Qt Terminal Example gets stuck on IOCTL_SERIAL_WAIT_ON_MASK, my example gets stuck on IRP_MJ_WRITE DOWN just after the IOCT_SERIAL_WAIT_ON_MASK. This never happens with other terminal softwares leading me to think the problem is definitely with Qt.
Pastebin of Serial Port Monitor Output
void IOThread::run(){
QSerialPort serial;
serial.setPortName(portname)
serial.setBaudRage(QSerialPort::Baud115200);
serial.setStopBits(QSerialPort::OneStop)
serial.setParity(QSerialPort::NoParity);
serial.setDataBits(QSerialPort::Data8);
serial.setFlowControl(QSerialPort::NoFlowControl);
if(!serial.open(QIODevice::ReadWrite)
{
qDebug() << "Error Opening Port";
return;
}
else
{
qDebug() << "Error Message: " << serial.errorString() // prints "Unknown Error"
}
while(true)
{
if(serial.waitForReadyRead(1000))
{
qDebug() << "Normal read";
reception_buffer = serial.readAll();
}
else
{
qDebug() << "Timeout";
/* serial.readAll() here will read nothing but force next read to read huge chunk of data */
continue;
}
}
// Process data...
}

Try if this makes any difference:
while (true) {
QByteArray reception_buffer;
if (serial.waitForReadyRead(1000)) {
reception_buffer = serial.readAll();
while (serial.waitForReadyRead(10)) {
reception_buffer += serial.readAll();
}
qDebug() << "reception_buffer ready";
}
else {
qDebug() << "Timeout";
}
}
If you want to prevent from timeouting from waitForReadyRead you can set:
if(serial.waitForReadyRead(-1))
bool QSerialPort::waitForReadyRead(int msecs = 30000) will timeout after msecs milliseconds; the default timeout is 30000 milliseconds. If msecs is -1, the function will not time out.

gets stuck on IOCTL_SERIAL_WAIT_ON_MASK
Most likelly a problem is in your HW or driver. QSP use asynchronous notification, based on WaitCommEvent. If WaitCommEvent get stuck - then a problem is in your device or driver (most likelly).

Thanks to the guys at QSerialPort, this bug in Qt 5.10.1 is solved by applying this patch: https://codereview.qt-project.org/#/c/225277/
"QSP may ignore all the read events when the data comes to the device
within opening. In this case, even re-opening of a device does not
help. Reason is that the QWinOverlappedIoNotifier is enabled after
than the startAsyncCommunication() called, that probably, leads to
ignoring for all EV_RXCHAR events. A workaround is to enable the
notifier before than any of I/O operation called." - Denis Shienkov, QSerialPort Maintainer

Related

Sleep inside QTConcurrent run method

I'm using QtConcurrent::run to execute some functions in background and not hang the GUI thread. In one function, I read logs from local SQlite database and send them to server by TCP socket.
Now I want to delay the execution after each log so the server has time to save it (TCP response is read in different thread). I'm stuck with Qt4.8 due to implementation limitations (many embeded devices - no chance to upgrade QT on them) and I can't use QThread::sleep(2) because it is protected in 4.8.
Is it possible to somehow pause the execution of thread inside QtConcurrent::run method or should I redesign it to implement my own class inheriting QThread?
void MainWindow::ReportFinishedHUs(bool asyncWork)
{
if(asyncWork == false)
{
QMutexLocker locker(&localDBmutex);
QList<QSqlRecord> HUsToReport = localDB->getHUsForBook();
qDebug() << "HUs to report" << HUsToReport.count();
if(!HUsToReport.isEmpty())
{
Cls_log::WriteDebugLog("HUs to report: " + QString::number(HUsToReport.count()));
foreach (QSqlRecord record, HUsToReport)
{
int _hu = record.indexOf("hu");
int _logTime = record.indexOf("logTime");
QString logTimeString = record.value(_logTime).toString();
QString hu = record.value(_hu).toString();
qDebug() << hu << logTimeString;
// creating message here ...
qDebug() << message;
emit sig_SendTCPMessage(message);
// this is where I need to wait for 2 seconds
QThread::sleep(2);
}
}
}
else
{
QtConcurrent::run(this, &MainWindow::ReportFinishedHUs, false);
}
}
EDIT:
Solved by usleep(2000000) which I somehow discarded for being platform specific... but hey, half of my aplication is platform specific and I only use it in embeded device with constant OS.
Keeping the question open if anyone can suggest more elegand solution using Qt methods. I like to get inspired.

QSerialPort has bytes available but can't read

I'm writing a Qt GUI application that receives and sends data to an Arduino over serial. It writes correctly, but when trying to read it doesn't work.
My problem is:
I have an Arduino code that sends things over Serial, I programmed it to turn on and off a LED on pin 13 depending on the received data:
#define ARDUINO_TURNED_LED_ON "LEDON"
#define ARDUINO_TURNED_LED_OFF "LEDOFF"
#define PC_REQUESTED_LED_STATE_CHANGE u8"CHANGELED"
void setup()
{
// put your setup code here, to run once:
Serial.begin(9600);
pinMode(13, OUTPUT);
}
String serialCmd = "";
bool ledState=false;
void loop()
{
// put your main code here, to run repeatedly:
if (Serial.available())
{
serialCmd=Serial.readString();
if (serialCmd==PC_REQUESTED_LED_STATE_CHANGE)
{
if (ledState)
{
digitalWrite(13, LOW);
Serial.write(ARDUINO_TURNED_LED_OFF);
}
else
{
digitalWrite(13, HIGH);
Serial.write(ARDUINO_TURNED_LED_ON);
};
ledState=!ledState;
serialCmd = "";
}
}
}
I'm using the following signal and slot at MainWindow class:
#define ARDUINO_TURNED_LED_ON "LEDON"
#define ARDUINO_TURNED_LED_OFF "LEDOFF"
#define PC_REQUESTED_LED_STATE_CHANGE u8"CHANGELED"
MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
ui->setupUi(this);
/* Create Object the Class QSerialPort*/
serialPort = new QSerialPort(this);
/* Create Object the Class Arduino to manipulate read/write*/
arduino = new Arduino(serialPort);
//connect signal:
connect(serialPort, SIGNAL(readyRead()), this, SLOT(ReadData()));
}
//slot
void MainWindow::ReadData()
{
QString received = arduino->Read();
qDebug() << "received data:" << received;
if (received==ARDUINO_TURNED_LED_ON)
{
qDebug() << "led on";
}
else if (received==ARDUINO_TURNED_LED_OFF)
{
qDebug() << "led off";
}
}
And the Arduino::Read() method that is called by the previous slot:
QString Arduino::Read()
{
QString bufRxSerial;
qDebug() << "bytes available:" << serialPort->bytesAvailable();
/* Awaits read all the data before continuing */
while (serialPort->waitForReadyRead(20)) {
bufRxSerial += serialPort->readAll();
}
return bufRxSerial;
}
When writing to the Arduino it works well, the LED changes as it should, the problem happens when Qt tries to read the replied data.
In most of the times that the Arduino sends something, the signal is emitted and serialPort->bytesAvailable() returns a number that is not zero, but
serialPort->waitForReadyRead(20) times out without receiving anything and serialPort->readAll() returns an empty QString. If I use serialPort->waitForReadyRead(-1) the window freezes.
The weirdest thing is that, in a random moment that something is sent, everything works well and the function returns all the data that couldn't be read previously.
Here's an example of what happens:
1st attempt (fails)
Qt writes PC_REQUESTED_LED_STATE_CHANGE (u8"CHANGELED") to the serial port
Arduino LED changes its state (turns on)
Arduino writes ARDUINO_TURNED_LED_ON ("LEDON") to the serial port
Qt reads an empty QString from the Arduino
2nd attempt (fails)
Qt writes PC_REQUESTED_LED_STATE_CHANGE (u8"CHANGELED") to the serial port
Arduino LED changes its state (turns off)
Arduino writes ARDUINO_TURNED_LED_OFF ("LEDOFF") to the serial port
Qt reads an empty QString from the Arduino
3rd attempt (this is a random attempt that everything works)
Qt writes PC_REQUESTED_LED_STATE_CHANGE (u8"CHANGELED") to the serial port
Arduino LED changes its state (turns on)
Arduino writes ARDUINO_TURNED_LED_ON ("LEDON") to the serial port
Qt reads "LEDONLEDOFFLEDON" from the Arduino, that is everything that couldn't be read previosly.
4th attempt (fails)
Qt writes PC_REQUESTED_LED_STATE_CHANGE (u8"CHANGELED") to the serial port
Arduino LED changes its state (turns off)
Arduino writes ARDUINO_TURNED_LED_OFF ("LEDOFF") to the serial port
Qt reads an empty QString from the Arduino
5th attempt (this is another random attempt that everything works)
Qt writes PC_REQUESTED_LED_STATE_CHANGE (u8"CHANGELED") to the serial port
Arduino LED changes its state (turns on)
Arduino writes ARDUINO_TURNED_LED_ON ("LEDON") to the serial port
Qt reads "LEDOFFLEDON" from the Arduino, that is everything that couldn't be read previosly.
I tested using the Arduino IDE serial monitor and it worked as it was supposed to be: I wrote "CHANGELED", then the LED changed and the Arduino replied "LEDON" or "LEDOFF" all the times.
What can I do to solve this problem?
Thanks
Edit: the complete code can be found here
The Read method is incorrect and unnecessary. You should never use the waitFor methods. You also shouldn't be using QString when dealing with simple ASCII data that QByteArray handles fine:
class MainWindow : ... {
QByteArray m_inBuffer;
...
};
void MainWindow::ReadData() {
m_inBuffer.append(arduino->readAll());
QByteArray match;
while (true) {
if (m_inBuffer.startsWith((match = ARDUINO_TURNED_LED_ON))) {
qDebug() << "led on";
} else if (m_inBuffer.startsWith((match = ARDUINO_TURNED_LED_OFF))) {
qDebug() << "led off";
} else {
match = {};
break;
}
}
m_inBuffer.remove(0, match.size());
}
The deal is simple: ReadData can be called with any number of bytes available - even a single byte. Thus you must accumulate data until a full "packet" is received. In your case, packets can only be delineated by matching a full response string.
You'd make your life much easier if Arduino sent full lines - replace Serial.write with Serial.println. Then the lines are packets, and you don't need to buffer the data yourself:
void MainWindow::ReadData() {
while (arduino->canReadLine()) {
auto line = arduino->readLine();
line.chop(1); // remove the terminating '\n'
QDebug dbg; // keeps everything on a single line
dbg << "LINE=" << line;
if (line == ARDUINO_TURNED_LED_ON)
dbg << "led on";
else if (line == ARDUINO_TURNED_LED_OFF)
dbg << "led off";
}
}
See? It's much simpler that way.
You can also leverage Qt to "simulate" Arduino code without running real Arduino hardware, see this answer for an example.
This sounds like a buffering problem. Your Qt application receives the bytes, but there is nothing that tells it the reception is complete. I would add line feeds (\n) as terminator character after each string, just use Serial.println() instead of Serial.write(). Then you can use readLine() in Qt and always be sure that what you get is exactly one complete command string.
If you're on Linux you could also try to stty your device to raw with
stty -F /dev/ttyACM0 raw
to have the serial characters sent straight through instead of default line buffering (see What’s the difference between a “raw” and a “cooked” device driver?).
Note that then you might get a new issue: Your Qt application might be so fast that you receive only one letter at a time in your callback.

How to read data from the serial port in QT?

I am creating a script in QT for reading the format packages (AA), (BB), etc from serial port. I open the serial port, but when I go to check inside the QByteArray values, comes back that I could not read any value.
This is my code
...
QSerialPort *serialPort = new QSerialPort();
serialPort->setPortName("ttyUSB0");
serialPort->setParity(QSerialPort::NoParity);
serialPort->setBaudRate(QSerialPort::Baud9600, QSerialPort::AllDirections);
serialPort->setStopBits(QSerialPort::OneStop);
serialPort->setFlowControl(QSerialPort::NoFlowControl);
serialPort->open(QIODevice::ReadOnly);
if (serialPort->isOpen()) {
qDebug() << "Serial port is open...";
QByteArray datas = serialPort->readAll();
if (datas.size() == 0) {
qDebug() << "Arrived data: 0";
} else {
for (int i = 0; i < datas.size(); i++){
if (datas.at(i)) {
qDebug() << datas[i];
}
}
}
} else {
qDebug() << "OPEN ERROR: " << serialPort->errorString();
}
serialPort->close();
qDebug() << "...serial port is closed!";
return 0;
...
You called readAll() immediately after open(). It probably took the computer a few nanoseconds to get from one to the other.
At 9600 baud, each byte of data takes slightly more than one millisecond to transfer. It would be absolutely impossible for any data to have arrived in that short an interval, so that's why you got no data.
Serial ports don't begin buffering incoming data until you open them (how could they, what baud rate and other settings would be used for receiving and buffering when no program has the port open?)
Use either a blocking read function of some sort (such as readLine()) or an event loop that reacts to data when it arrives.

Arduino doesn't receive data after reconnecting to USB

So, I've built a basic QT GUI where I want to establish communication with an Arduino Nano through USB. I send a number through the GUI and the Arduino receives the number and processes it.
The communication works fine when I upload the code to Arduino and right afterwards open the GUI and start the process. However, when I disconnect the Arduino from the USB (or restart my PC - I've tried both) and reconnect it to use it with the GUI, the Arduino behaves like it received nothing.
More specifically, in the first case Serial.available() returns "1" as it receives the number properly, but in the latter case it returns "0", so it does nothing.
I made the code as simple as I could trying to track down the issue and the problem continues.
So here is the main QT GUI code:
depth_ = insertDepthEdit->text().toInt(); // user input from GUI
myThread *mThread;
mThread = new myThread(this, depth_);
connect(mThread, SIGNAL(valueRead(QString)), this, SLOT(onTextChange(QString)));
//valueRead is the signal emitted from Arduino
//onTextChange the function that processes the received string
mThread->start();
mThread->wait(100);
mThread->quit();
The Arduino thread code (also QT):
void myThread::run() {
QSerialPort serial;
serial.setPortName("COM3");
serial.setBaudRate(QSerialPort::Baud9600);
serial.setDataBits(QSerialPort::Data8);
serial.setParity(QSerialPort::NoParity);
serial.setStopBits(QSerialPort::OneStop);
serial.setFlowControl(QSerialPort::NoFlowControl);
serial.open(QIODevice::ReadWrite);
if (serial.isOpen() && serial.isWritable()) {
qDebug() << "Ready to write..." << endl;
QByteArray ba(QString::number(depth_).toStdString().c_str());
qDebug() << ba << endl;
serial.write(ba);
if (serial.bytesToWrite() > 0) {
serial.flush();
if (serial.waitForBytesWritten(1000)) {
qDebug() << "data has been sent" << endl;
}
}
if (serial.flush()) {
qDebug() << "ok" << endl;
}
}
else {
qDebug() << "Error";
}
if (serial.isOpen() && serial.isReadable()) {
qDebug() << "Ready to read..." <<endl;
while (serial.waitForReadyRead(5000)) {
QByteArray inByteArray = serial.readLine();
input_ = QString(inByteArray);
qDebug() << input_;
qDebug() << "ok" << endl;
emit valueRead(input_);
}
}
serial.close();
}
And finally the Arduino code:
int c = 0;
const int ledPin = 13;
void setup() {
Serial.begin(9600);
}
void loop() {
Serial.print(Serial.available());
while (Serial.available() > 0) {
digitalWrite(ledPin, HIGH);
delay(5);
c = Serial.read() - '0';
Serial.flush();
}
delay(1000);
digitalWrite(ledPin, LOW);
delay(500);
}
When I upload the code to Arduino, it functions properly no matter if I close the GUI and restart it. The problem happens only if Arduino loses power, e.g: when I disconnect it from USB or restart the PC.
~~~~~~~~~~~~~~~~~~~~~~~~~~EDIT~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
COM port remains the same after reconnecting and Arduino Rx LED flashes normally when I send data through the GUI.
~~~~~~~~~~~~~~~~~~~~~~~~~~EDIT 2~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
OK, so, I tried using the code from Arduino Serial documentation and the problem remains. When I upload the code the Arduino receives the character properly and turns the LED on, but once I disconnect it and then connect it back, it does nothing, the LED remains low as it never enters "if".
Here's the code I used:
int incomingByte = 0;
void setup() {
Serial.begin(9600);
}
void loop() {
if (Serial.available() > 0) {
digitalWrite(13, HIGH);
incomingByte = Serial.read();
}
}
~~~~~~~~~~~~~~~~~~~~~~EDIT 3~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
So I have the following 3 scenarios:
Use Scenario A:
Upload code
Run GUI
Send data - It receives properly
Disconnect and reconnect
Run GUI again
Send data - RX blinks but Serial.available returns 0
Use Scenario B:
Upload code
Run Brays
Send data - It receives properly
Disconnect and reconnect
Run Brays again
Send data - It receives properly
Use Scenario C (the most interesting) :
Upload code
Run GUI
Send data - It receives properly
Disconnect and reconnect
Run Brays this time
Send data - It receives properly
Run GUI again after Brays
Send data - It receives properly
I also made the QT GUI code as simple as that but the problem persists:
void myThread::run()
{
QSerialPort *serial = new QSerialPort();
serial->setPortName("COM3");
serial->setBaudRate(QSerialPort::Baud9600);
serial->setDataBits(QSerialPort::Data8);
serial->open(QIODevice::WriteOnly);
if (serial->isOpen() && serial->isWritable())
{
QByteArray ba(QString::number(depth_).toStdString().c_str());
serial->write(ba);
serial->flush();
serial->close();
}
delete serial;
}
~~~~~~~~~~~~~~~~~~~~~~EDIT 4~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
So after much effort and curiosity, I realized that the source of the problem is not the Arduino code but something related to QT or Windows. I added the delays that Jeff recommended and noticed that each time it functioned properly Rx blinked and LED became high as indicated by the code. However, after reconnecting, the problem still remained but I noticed that, this time, immediately after clicking "Send" to send the characters, the LED blinked for some milliseconds (possibly indicating some error??) and then after the 1 second delay the Rx blinked indicating the receipt of data and LED remained LOW as Serial.available remained 0.
So, what I tried next, was to remove one line of code at a time to see what causes the problem. And I ended up with literally blank Arduino code, just empty setup and loop methods, and the following QT GUI code:
void myThread::run()
{
QSerialPort *serial1 = new QSerialPort();
serial1->setPortName("COM5");
serial1->open(QIODevice::WriteOnly);
serial1->close();
}
To summarize, what happens now is:
Upload code to Arduino
Run GUI
Send data
Nothing happens (normal behaviour)
Disconnect and reconnect Arduino to USB
Run GUI
Send data
Arduino LED momentarily blinks once (possibly indicating some kind of error)
OK, so, after hours of debugging I've found what caused the problem.
The root of it was that after reconnecting the Arduino, each time I called serial.open in QT, Arduino did a reset (indicated by the blink of the LED) and by the time it was after the bootloader stage and was running the code, the main program had already passed the serial.write QT command without receiving the data.
So, what I did to solve the problem was to just add a Sleep(uint(2000)); after serial.open in order to let Arduino finish booting and then start sending data.
Thank you all for your help and immediate answers!
In my experience, the issue is not the code in the Arduino. It is because the serial port gets a different name when it is plugged back in.
For example in Linux, originally the port is /dev/ARD0, but when it is disconnected and plugged back in with connections on ARD0, the new plugin is named /dev/ARD1. (In Windows, it might be COM17 then COM18.)
The only way I know to make it become the original port name is to close everything that is connected to it before plugging in again: Close the Arduino IDE, close all programs which have opened the port, etc.
If you use this example for the Arduino Serial documentation do you receive the chars you send?
int incomingByte = 0; // for incoming serial data
void setup() {
Serial.begin(9600); // opens serial port, sets data rate to 9600 bps
}
void loop() {
// send data only when you receive data:
if (Serial.available() > 0) {
// read the incoming byte:
incomingByte = Serial.read();
// say what you got:
Serial.print("I received: ");
Serial.println(incomingByte, DEC);
}
}
Grasping at straws here, replace my comments below a one second delay. Editing on iPhone messed with the format a little, but I believe you can see my intent.
Edit: Also, I think you should not do serial->close inside your loop. I would also try sending a single character repeatedly until we have that working.
void myThread::run()
{
QSerialPort *serial = new QSerialPort();
serial->setPortName("COM3");
serial->setBaudRate(QSerialPort::Baud9600);
serial->setDataBits(QSerialPort::Data8);
serial->open(QIODevice::WriteOnly);
if (serial->isOpen() && serial->isWritable())
{
QByteArray ba(QString::number(depth_).toStdString().c_str());
serial->write("x");
delay 1 second here
serial->flush();
delay 1 second here
}
serial->close();
delay 1 second here
delete serial;
}

QAudioInput::notify() isn't executed while capturing sound from mic

I want to use QAudioInput to capture sound from mic, process it and then play. As I understood, I need to connect to notify signal and inside handler for it user readAll() function to get raw data. But the problem is, this handler function is never executed. Here is my code:
void MainWindow::on_pushButton_clicked()
{
QList<QAudioDeviceInfo> list = QAudioDeviceInfo::availableDevices(QAudio::AudioInput);
if(list.isEmpty())
{
qDebug() << "No audio input device";
return;
}
QAudioDeviceInfo info = QAudioDeviceInfo::defaultInputDevice();
QAudioFormat format;
// Set up the desired format, for example:
format.setSampleRate(44100);
format.setChannelCount(1);
format.setSampleSize(32);
format.setCodec("audio/pcm");
format.setByteOrder(QAudioFormat::LittleEndian);
format.setSampleType(QAudioFormat::UnSignedInt);
if (!info.isFormatSupported(format)) {
qWarning() << "Default format not supported, trying to use the nearest.";
format = info.nearestFormat(format);
}
qDebug() << "input device :" << info.deviceName();
audio = new QAudioInput(info, format);
qDebug("created()");
connect(audio, SIGNAL(notify()), this, SLOT(onNotify()));
audio->setNotifyInterval(10);
bufferInput = audio->start();
}
void MainWindow::onNotify()
{
qDebug() << "onNotify()";
//QByteArray array = bufferInput->readAll();
//bufferOutput->write(array);
}
(audio is of type QAudioInput* and bufferInput is of type QIODevice*)
and when I click the button, "input device = " and "created()" messages are displayed, but "onNotify()" isn't shown.
What am I doing wrong?
QAudioInput seems quite broken. Or I'm utterly misunderstanding something.
The only thing that worked reliably for me was to use the readyRead() signal of the QIODevice buffer returned by start(). Unfortunately, that isn't fired very often on my system (around every 40 to 60ms on my system).
What I've found is that notify() starts firing when I either call resume() on the QAudioInput (because it's in idle state after calling start()) or do a readAll() on the QIODevice (!). But at least with PyQt this leads to a stack overflow after a minute or so.
I would suspect the platform matters as well, as the actual QAudioInput implementation depends on the platform and audio system used (PulseAudio on Fedora 32 in my case).
Connecting a slot to readyRead signal of QAudioInput and reading from QIODevice with readAll is the correct solution. Why do you think 40 or 60 ms interval is inappropriate? QAudioInput object must capture some audio during this interval and send signal after it.