QAudioInput::notify() isn't executed while capturing sound from mic - c++

I want to use QAudioInput to capture sound from mic, process it and then play. As I understood, I need to connect to notify signal and inside handler for it user readAll() function to get raw data. But the problem is, this handler function is never executed. Here is my code:
void MainWindow::on_pushButton_clicked()
{
QList<QAudioDeviceInfo> list = QAudioDeviceInfo::availableDevices(QAudio::AudioInput);
if(list.isEmpty())
{
qDebug() << "No audio input device";
return;
}
QAudioDeviceInfo info = QAudioDeviceInfo::defaultInputDevice();
QAudioFormat format;
// Set up the desired format, for example:
format.setSampleRate(44100);
format.setChannelCount(1);
format.setSampleSize(32);
format.setCodec("audio/pcm");
format.setByteOrder(QAudioFormat::LittleEndian);
format.setSampleType(QAudioFormat::UnSignedInt);
if (!info.isFormatSupported(format)) {
qWarning() << "Default format not supported, trying to use the nearest.";
format = info.nearestFormat(format);
}
qDebug() << "input device :" << info.deviceName();
audio = new QAudioInput(info, format);
qDebug("created()");
connect(audio, SIGNAL(notify()), this, SLOT(onNotify()));
audio->setNotifyInterval(10);
bufferInput = audio->start();
}
void MainWindow::onNotify()
{
qDebug() << "onNotify()";
//QByteArray array = bufferInput->readAll();
//bufferOutput->write(array);
}
(audio is of type QAudioInput* and bufferInput is of type QIODevice*)
and when I click the button, "input device = " and "created()" messages are displayed, but "onNotify()" isn't shown.
What am I doing wrong?

QAudioInput seems quite broken. Or I'm utterly misunderstanding something.
The only thing that worked reliably for me was to use the readyRead() signal of the QIODevice buffer returned by start(). Unfortunately, that isn't fired very often on my system (around every 40 to 60ms on my system).
What I've found is that notify() starts firing when I either call resume() on the QAudioInput (because it's in idle state after calling start()) or do a readAll() on the QIODevice (!). But at least with PyQt this leads to a stack overflow after a minute or so.
I would suspect the platform matters as well, as the actual QAudioInput implementation depends on the platform and audio system used (PulseAudio on Fedora 32 in my case).

Connecting a slot to readyRead signal of QAudioInput and reading from QIODevice with readAll is the correct solution. Why do you think 40 or 60 ms interval is inappropriate? QAudioInput object must capture some audio during this interval and send signal after it.

Related

How can I wait until data comes in when reading data over Serial Port? QT C++

I need to read ID data through Serial Port. When I send the ID read command with the leather port, the ID data comes to me a little late. Therefore, when I try to split the data and display it on the screen, it comes up blank and the application closes itself. How can I wait until the data comes in?
void productDetail::on_pushButton_clicked()
{
QSerialPortInfo info;
QList<QSerialPortInfo> infoList = QSerialPortInfo::availablePorts();
foreach(info, infoList) QTextStream(stdout) << info.portName();
QString curport = info.portName();
serial.begin(curport, 9600, 8, 0, 1, 0, false);
if(serial.isOpen()){
qDebug()<<"serial open";
QString sendWC= "WR+1121"; //Read ID command
serial.send(sendWC);
QString serialID = serial.getString();
serialID = serialID.trimmed();
QStringList buffer_split = serialID.split(",");
ui->IDlabel->setText(buffer_split[2]; //when I write this program closes
}
}
}
error: ASSERT failure in QList::operator[]: "index out of range",
Qt's QIODevice provides a signal readyRead (https://doc.qt.io/qt-5/qiodevice.html#readyRead) which is emitted once every time new data is available for reading from the device.
In the productDetail class constructor, connect serial port readyRead signal to a slot.
connect(serial, &QSerialPort::readyRead, this, &productDetail::readData);
Define readData method as follows:
void productDetail::readData()
{
const QByteArray data = m_serial->readAll();
}
For serial communication there is no guarantee that you will receive entire message data in a single read. I prefer to store the received data in a circular buffer and process the circular buffer in a separate slot.

Avoiding Timeout on QSerialPort when handling high speed data

I'm working on a windows application that receives data from a sensor at 600Hz. In two out of five cases, my IO thread reads the 4 bytes of data from the sensor successfully and passes it on to the GUI thread.
The problem is three out of five times, QSerialPort has inexplicable timeouts where QSerialPort's waitForReadyRead() returns false and serial.errorString() has a timeout error. In which case it will never read data. If I read from the serial port despite the timeout error I will read 2000+ bytes of data in the next waitForReadyRead which will be delivered in chunks which renders the realtime data reception aspect of my application obsolete.
I've tried using the readyRead() signal of the serial port but it exhibits the same behaviour ie. if the timeout error appears, no readyRead() signal is ever fired.
UPDATE: I am able to reproduce the issue with Qt's terminal example ([QT_INSTALL_EXAMPLES]/serialport/terminal) which uses a non-blocking read. The frequency of the bug is considerably less but it's definitely still there.
UPDATE: Using Serial Port Monitor, I can see that when it gets stuck, the Qt Terminal Example gets stuck on IOCTL_SERIAL_WAIT_ON_MASK, my example gets stuck on IRP_MJ_WRITE DOWN just after the IOCT_SERIAL_WAIT_ON_MASK. This never happens with other terminal softwares leading me to think the problem is definitely with Qt.
Pastebin of Serial Port Monitor Output
void IOThread::run(){
QSerialPort serial;
serial.setPortName(portname)
serial.setBaudRage(QSerialPort::Baud115200);
serial.setStopBits(QSerialPort::OneStop)
serial.setParity(QSerialPort::NoParity);
serial.setDataBits(QSerialPort::Data8);
serial.setFlowControl(QSerialPort::NoFlowControl);
if(!serial.open(QIODevice::ReadWrite)
{
qDebug() << "Error Opening Port";
return;
}
else
{
qDebug() << "Error Message: " << serial.errorString() // prints "Unknown Error"
}
while(true)
{
if(serial.waitForReadyRead(1000))
{
qDebug() << "Normal read";
reception_buffer = serial.readAll();
}
else
{
qDebug() << "Timeout";
/* serial.readAll() here will read nothing but force next read to read huge chunk of data */
continue;
}
}
// Process data...
}
Try if this makes any difference:
while (true) {
QByteArray reception_buffer;
if (serial.waitForReadyRead(1000)) {
reception_buffer = serial.readAll();
while (serial.waitForReadyRead(10)) {
reception_buffer += serial.readAll();
}
qDebug() << "reception_buffer ready";
}
else {
qDebug() << "Timeout";
}
}
If you want to prevent from timeouting from waitForReadyRead you can set:
if(serial.waitForReadyRead(-1))
bool QSerialPort::waitForReadyRead(int msecs = 30000) will timeout after msecs milliseconds; the default timeout is 30000 milliseconds. If msecs is -1, the function will not time out.
gets stuck on IOCTL_SERIAL_WAIT_ON_MASK
Most likelly a problem is in your HW or driver. QSP use asynchronous notification, based on WaitCommEvent. If WaitCommEvent get stuck - then a problem is in your device or driver (most likelly).
Thanks to the guys at QSerialPort, this bug in Qt 5.10.1 is solved by applying this patch: https://codereview.qt-project.org/#/c/225277/
"QSP may ignore all the read events when the data comes to the device
within opening. In this case, even re-opening of a device does not
help. Reason is that the QWinOverlappedIoNotifier is enabled after
than the startAsyncCommunication() called, that probably, leads to
ignoring for all EV_RXCHAR events. A workaround is to enable the
notifier before than any of I/O operation called." - Denis Shienkov, QSerialPort Maintainer

How to read HID devices (/dev/hidrawX) with Qt on Linux?

I'm working on a 'RepRap calibration tool' which would use a mouse attached to the printing platform to measure the movement of the platform.
Right now I'm stuck trying to read raw mouse data from /dev/hidrawX, but I'm unable to read any data.
So far I've tried:
First attempt:
QFile f("/dev/hidraw0");
f.readAll();
Reads nothing.
Second attempt:
m_file = new QFile("/dev/hidraw0");
m_sn= new QSocketNotifier(m_file->handle(), QSocketNotifier::Read);
m_sn->setEnabled(true);
connect(m_sn, SIGNAL(activated(int)), this, SLOT(readyRead()));
then on the readyRead SLOT:
qDebug()<<"Ready Read!!"<<m_file.bytesAvailable();
QTextStream d(&m_file);
qDebug()<< d.read(64);
This code fires the readyRead slot once but it gets stuck on the read(64) call, if I comment the read(64) the slot will be fired each time the mouse its moved.
m_file.bytesAvailable() always reports 0.
Which is the right way to read these devices with Qt?
Solution:
I reworked the code like:
bool rcMouseHandler::openHidraw(QString device)
{
int fd =open(device.toLocal8Bit(),O_NONBLOCK);
if(fd <=0)
{
qDebug()<<"[WARN]rcMouseHandler::open-> Cant open!";
return false;
}
m_sn= new QSocketNotifier(fd, QSocketNotifier::Read);
m_sn->setEnabled(true);
connect(m_sn, SIGNAL(activated(int)), this, SLOT(readyRead()));
return true;
}
void rcMouseHandler::readyRead()
{
qDebug()<<"Ready Read!!";
char buffer[4] = {0,0,0,0};
read(m_sn->socket(),&buffer,4);
qDebug()<<(quint8)buffer[0]<<(quint8)buffer[1]<<(quint8)buffer[2]<<(quint8)buffer[3];
}
The right way I suppose here to not use Qt. Why you need portable wrapper above POSIX open and read, when this part of your code is not portable (part that work with /dev/*). Open device with "open" "man 2 open" in O_NONBLOCK and call "read" (man 2 read) to get data from it. And you can still use QSocketNotifier with handle that return "open".

controlling the number of bytes ready to read from Audio card in QT

In QT I can define an Audio input as:
m_audioInput = new QAudioInput(m_Inputdevice, m_format, this);
m_input = m_audioInput->start();
In my application I would like to use a mic and read from audio card.
Now if I want to see how many bytes are ready to read from Audio buffer I use:
qint64 len = m_audioInput->bytesReady();
It looks like the len is a function of sampling rate and number of bits per sample.
My question is that is there a way to control len, without changing the sampling rate? In other words I would like to control the audio card such that it reads data in shorter blocks and emits the ready signal.
You can control the audio card by setting up proper format parameters e.g., frequency, sample size. For this you need to use QAudioFormat class.
Other than this, there is no other way to control audio card from Qt.
Class Reference.
Example from the reference:
QFile outputFile; // class member.
QAudioInput* audio; // class member.
outputFile.setFileName("/tmp/test.raw");
outputFile.open( QIODevice::WriteOnly | QIODevice::Truncate );
QAudioFormat format;
// set up the format you want, eg.
format.setFrequency(8000);
format.setChannels(1);
format.setSampleSize(8);
format.setCodec("audio/pcm");
format.setByteOrder(QAudioFormat::LittleEndian);
format.setSampleType(QAudioFormat::UnSignedInt);
QAudioDeviceInfo info = QAudioDeviceInfo::defaultInputDevice();
if (!info.isFormatSupported(format)) {
qWarning()<<"default format not supported try to use nearest";
format = info.nearestFormat(format);
}
audio = new QAudioInput(format, this);
QTimer::singleShot(3000, this, SLOT(stopRecording()));
audio->start(&outputFile);
// Records audio for 3000ms

How to use QtMultimedia to play a wav file?

My current code is:
void Sound::run() {
QFile audio_file(mResourcePath);
if(audio_file.open(QIODevice::ReadOnly)) {
audio_file.seek(44); // skip wav header
QByteArray audio_data = audio_file.readAll();
audio_file.close();
QBuffer* audio_buffer = new QBuffer(&audio_data);
qDebug() << audio_buffer->size();
QAudioFormat format;
format.setSampleSize(16);
format.setSampleRate(44100);
format.setChannelCount(2);
format.setCodec("audio/pcm");
format.setByteOrder(QAudioFormat::LittleEndian);
format.setSampleType(QAudioFormat::UnSignedInt);
QAudioDeviceInfo info(QAudioDeviceInfo::defaultOutputDevice());
if (!info.isFormatSupported(format)) {
qWarning()<<"raw audio format not supported by backend, cannot play audio.";
return;
}
qDebug() << info.deviceName();
QAudioOutput* output = new QAudioOutput(info, format);
output->start(audio_buffer);
}
}
This whole thing is started as a QRunnable in a QThreadPool and that part works fine. Problem is I never get any audio. My sound device is operational, the buffer is filled. I don't know what's wrong. I use app.exec(). Help appreciated.
The device (QBuffer) has to be open:
QBuffer audio_buffer(&audio_data);
audio_buffer.open(QIODevice::ReadOnly);
QAudioOutput needs an event loop to play anything, and that loop has to be running in the thread it belongs to. Which is the thread it was created in when you don't explicitly move it to another thread:
// Create the device and start playing...
QAudioOutput output(info, format);
output.start(&audio_buffer);
// ...then wait for the sound to finish
QEventLoop loop;
QObject::connect(&output, SIGNAL(stateChanged(QAudio::State)), &loop, SLOT(quit()));
do {
loop.exec();
} while(output.state() == QAudio::ActiveState);
Everything you allocate should be deallocated when the sound has finished playing, or you would have a memory leak, and the event loop will now run inside the function, so you can allocate everything locally.