Convert QAudioBuffer to QByteArray : loss of information? - c++

I have a WAV file that I decode with the QAudioDecoder. As a result I have a QAudioBuffer object. I want to store the data stored in QAudioBuffer in a QByteArray for my QIODevice derived class. I want to use this data in the ReadData method of my derived class for audio output. I now have 2 questions:
How to get a QByteArray from a QAuddioBuffer?
I used the following code, but unfortunately this is not correct. The data in QAudioBuffer is coded to 2Bytes, but each element in a QByteArray is coded to 1Byte (right?). Don't we have a loss of information there? To test if QByteArray contains the original data from the WAV file, I save it to a TXT file.
is this approach appropriate? I actually want to apply some operations on the data stored in QAudioBuffer (e.g. filters) and listen to the result in real time.
Thanks in advance.
Here is the code
QAudioFormat *format_decoder;
format_decoder = new QAudioFormat;
format_decoder->setSampleRate(44100);
format_decoder->setChannelCount(1);
format_decoder->setSampleFormat(QAudioFormat::Int16);
QAudioDecoder decoder;
decoder.setSource(filenameSource);
decoder.setAudioFormat(*format_decoder);
decoder.start();
QObject::connect(&decoder, &QAudioDecoder::bufferReady, this, &MainWindow::slot_bufReady);
and the slot
void MainWindow::slot_bufReady(){
QAudioBuffer buffer = m_audioDecoder->read();
QByteArray buffer_ByteArray(buffer.constData<char>(), buffer.byteCount());
QFile file(filenameTest1);
if(!file.open(QIODevice::WriteOnly|QIODevice::Append)) {
qDebug() << "ERRO "; }
QTextStream strem(&file);
for(auto const dat: buffer_ByteArray) {
strem<< qreal(dat)/128.0<< "\r\n";
}
file.cloe();

This looks suspicious:
for(auto const dat: buffer_ByteArray) {
strem<< qreal(dat)/128.0<< "\r\n";
}
Your audio format is 16-bit mono. Reading it byte by byte is a non-starter. Read it sample by sample. That is, read two bytes at a time and convert. More likely this:
int16_t* data = (int16_t*)(buffer.data());
int samples = buffer.sampleCount();
for (int i = 0; i < samples; i++)
{
strem << data[i] << "\r\n";
}
The above will save your samples into a text file. You could plot it with Excel. But as others have said, that's not as useful as saving in as binary. You could prepend a WAV file header such that it can be played and analyzed with other tools.
Update
If your intent is to transcode from 16-bit to 8-bit, this is how you would likely do it:
int16_t* data = (int16_t*)(buffer.data());
QByteArray buffer_ByteArray(buffer.sampleCount(), '\0');
for (size_t i = 0; i < samples; i++) {
buffer_ByteArray[i] = (char)(data[i] / 256); // 16-bit to 8-bit
}
Note: some audio platforms use unsigned integers for 8-bit audio. That is the zero amplitude sample is 0x80. This is the case for 8-bit WAV files. If that's in play, then change this line:
buffer_ByteArray[i] = (char)(data[i] / 256); // 16-bit to 8-bit
To this:
char c = (char)(data[i] / 256); // 16-bit to 8-bit signed
const unsigned char mask = 0x80;
buffer_ByteArray[i] = (char)(mask ^ c);

Related

Opus encode & decode no error but not the same value

I have to use Opus Codec to encode & decode audio datas in C++ and I have to encapsulate the functions.
So I try to send a floats array to try to encode it and I decode the result of the Opus encoding function. Unfortunately, the result is not the same and I get a table that contains no value from the initial table.
Here is my code.
Encapsulation:
std::vector<float> codec::OpusPlugin::decode(packet_t &packet) {
std::vector<float> out(BUFFER_SIZE * NB_CHANNELS);
int ret = 0;
if (!this->decoder)
throw Exception("Can't decode since there is no decoder.");
ret = opus_decode_float(this->decoder, packet.data.data(), packet.size, reinterpret_cast<float*>(out.data()), FRAME_SIZE, 0);
if (ret < 0)
throw Exception("Error while decoding compressed data.");
return out;
}
// ENCODER
packet_t codec::OpusPlugin::encode(std::vector<float> to_encode) {
std::vector<unsigned char> data(BUFFER_SIZE * NB_CHANNELS * 2);
packet_t packet;
int ret = 0;
if (!this->encoder)
throw Exception("Can't encode since there is no decoder.");
ret = opus_encode_float(this->encoder, reinterpret_cast<float const*>(to_encode.data()), FRAME_SIZE, data.data(), data.size());
if (ret < 0)
throw Exception("Error while encoding data.");
packet.size = ret;
packet.data = data;
return packet;
}
And there is the call of the functions:
packet_t packet;
std::vector<float> floats = {0.23, 0, -0.312, 0.401230, 0.1234, -0.1543};
packet = CodecPlugin->encode(floats);
std::cout << "packet size: " << packet.size << std::endl;
std::vector<float> output = CodecPlugin->decode(packet);
for (int i = 0; i < 10; i++) {
std::cout << output.data()[i] << " ";
}
Here is the packet_t structure, where I stock the return value of encode and the unsigned char array (encoded value)
typedef struct packet_s {
int size;
std::vector<unsigned char> data;
} packet_t;
The output of the program is
*-1.44487e-15 9.3872e-16 -1.42993e-14 7.31834e-15 -5.09662e-14 1.53629e-14 -8.36825e-14 3.9531e-14 -8.72754e-14 1.0791e-13 which is not the array I initialize at the beginning.
I read a lot of times the documentation and code examples but I don't know where I did a mistake.
I hope you will be able to help me.
Thanks :)
We don't see how you initialize your encoder and decoder so we don't know what their sample rate, complexity or number of channels is. No matter how you have initialized them you are still going to have the following problems:
First Opus encoding doesn't support arbitrary frame sizes but instead 2.5ms, 5ms, 10ms, 20, 40ms or 60ms RFC 6716 - Definition of the Opus Audio Codec relevant section 2.1.4. Moreover opus supports only 8kHz, 12kHz, 16kHz, 24kHz or 48kHz sample rates. No matter which of those you have chosen your array of 10 elements doesn't correspond to any of the supported frame sizes.
Secondly Opus codec is a lossy audio codec. This means that after you encode any signal you will never (probably except some edge cases) be able to reconstruct the original signal after decoding the encoded opus frame. The best way to test if your encoder and decoder work is with a real audio sample. Opus encoding preserves the perceptual quality of the audio files. Therefore if you try to test it with arbitrary data you might not get the expected results back even if you implemented the encoding and decoding functions correctly.
What you can easily do is to make a sine function of 2000Hz(there are multiple examples on the internet) for 20ms. This means 160 array elements at a sample rate of 8000Hz if you wish to use 8kHz. A sine wave of 2kHz is within the human hearing range so the encoder is going to preserve it. Then decode it back and see whether the elements of the input and output array are similar as we've already established that it is unlikely that they are the same.
I am not good in C++ so I can't help you with code examples but the problems above hold true no matter what language is used.

convert specific bits of a QByteArray to int,float double etc

I have a QByteArray, inside the QByteArray are multiple values of diferent datatypes which I want to extract. The difficulty by that is, that the values are in an defined length of x-bits, and they have a defined start position (also in bits).
eg: an int8 is stored inside the QByteArray from bit nr. 4 (of the first byte) to bit nr. 12 (inside the second byte).
On the Qt-wiki side i found a method to disassemble a byteArray into a bit array: https://wiki.qt.io/Working_with_Raw_Data
So i'm cutting my bits out of the byte array like that:
QByteArray MyCutter::cutMessage(qint32 bitStart, qint32 bitLength)
{
qDebug() << mBuffer;
QBitArray bits(mBufferLen * 8);
for(quint32 i=0; i<mBufferLen; ++i)
{
for(quint32 b=0; b<8; b++)
{
bits.setBit(i*8 +b, mBuffer.at(i) & (1<<(7-b)));
}
}
qDebug() << bits;
QByteArray bytes;
//round up to the next n*8 length of a byte: (length + y) = x*8
qint32 bitLengthWithZeros = (bitLength + (8 - 1)) & ~(8 - 1);
bytes.resize(bitLengthWithZeros/8);
bytes.fill(0);
for(qint32 b=bitStart ,c=0; b<(bitStart + bitLength); b++, c++)
{
bytes[c/8] = (bytes.at(c/8) | ((bits.testBit(b)?1:0)<<(7-b%8)));
}
qDebug() << bytes;
return bytes.data();
}
This is working fine so far - I can cut my ByteArray into any other.
The Problem is to convert the values into int/float/doubles, and to be more specific into signed values.
To convert i've tried two things:
QByteArray::toHex().toLong(nullptr, 16) ... toLong/toLongLong etc. This is working, but only returns me the UNSIGNED value of the QByteArray. If I'm cutting the mBuffer with the function MyCutter::cutMessage, like the excample above, from the 4. bit to the 12. (which is also 0xFF) im getting 255 as signed int! And that's wrong?
On the other side I've tried to convert it with QStremData:
QDataStream stream(mBuffer);
stream.setByteOrder(QDataStream::LittleEndian);
qint64 result;
stream >> result;
qDebug() << QString::number(result,16);
qDebug() << QString::number(result);
mBuffer are the raw data. If im putting "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" insde mBuffer, the printed value is -1 which is correct.
QByteArray h = cutMessage(0,8);
qDebug() << h.toHex().toLongLong(nullptr, 16);
QDataStream stream2(h);
stream2.setByteOrder(QDataStream::LittleEndian);
qint64 result2;
stream2 >> result2;
qDebug() << QString::number(result2,16);
qDebug() << QString::number(result2);
Converting the cut-Message with the code block above is always returning me "0".
So without cutting, the interpretation of the whole QByteArray is correct, but if I'am cutting something off it's returning me always the unsigned value, or "0".
Somehow I'am loosing some information while the transformation into QBitArray and vice versa.
Hopefully my explanations are understandable ;)

Converting raw pixel data to QPixmap

I'm trying to convert raw pixel data to QPixmap but there is an error while converting qimage to qpixmap.
First, I send raw data from server A by qlocalsocket and Client B gets it.
This is Server A.
if (clientSocket)
{
if (clientSocket->isOpen())
{
QByteArray block;
QDataStream out(&block, QIODevice::WriteOnly);
out.setVersion(QDataStream::Qt_5_7);
const char* rc_data = reinterpret_cast<const char*>(r_data);
out <<sizeof(strlen(rc_data))<< r_width << r_height << r_step;
out.writeBytes(rc_data, strlen(rc_data));
clientSocket->write(block);
clientSocket->flush();
}
}
This is Client B.
QByteArray block = connection->readAll();
QDataStream in(&block, QIODevice::ReadOnly);
in.setVersion(QDataStream::Qt_5_7);
char* data;
size_t size;
int width;
int height;
int step;
while (!in.atEnd())
{
in >> size >> width >> height >> step;
in.readBytes(data,size);
}
mutex.lock();
receivePixmap = QPixmap::fromImage(QImage((unsigned char*)data, width, height, step, QImage::Format::Format_RGB888));
mutex.unlock();
ui.label->setPixmap(receivePixmap.scaled(ui.label->size(), Qt::KeepAspectRatio));
}
There is no error converting (unsigned char*)data to QImage but error while QPixmap::fromImage.
When I debug the code, sended and received data are same.
This is my error.
Error : 0xC0000005: Access violation reading location 0xCDCDCDD1.
And occurs at here.
#ifdef Q_COMPILER_RVALUE_REFS
static QPixmap fromImage(QImage &&image, Qt::ImageConversionFlags flags = Qt::AutoColor)
{
return fromImageInPlace(image, flags);
}
#endif
Please share your idea thanks.
When you serialize, you write the size component as sizeof strlen(rc_data), i.e. sizeof (size_t), which is likely much smaller than the content of data (it's often 4 or 8 bytes). When you reconstruct the image data, you only read that size, so the QImage now refers to memory space that's not as big as it needs to be to represent widthxheight pixels.
You need to write the actual length of data in order to read it back successfully. strlen(rcdata) is almost certainly the wrong measure, as it measures null-terminated strings, which your pixel data is not. Without a complete example, it's not possible to tell how you should determine the length of rc_data.
Use QPixmap's loadFromData method:
bool QPixmap::loadFromData(const uchar * data, uint len, const char * format = 0, Qt::ImageConversionFlags flags = Qt::AutoColor)
Or
bool QPixmap::loadFromData(const QByteArray & data, const char * format = 0, Qt::ImageConversionFlags flags = Qt::AutoColor)
More info: http://doc.qt.io/qt-4.8/qpixmap.html#loadFromData
I Solved this problem
writeBytes(rc_data,r_step*r_height)
readBytes(data,r_step*r_height)
I checked strlen(rc_data), but this size is not correct.
Actual size of image was step*height.
thank you guys~~~!!!

QDataStream won’t work with custom crafted char array

I have an application which consists of two primary modules. One is written in C, uses standard C runtime library and one written in Qt C++. They communicate with each other with IPC. C module creates a char array, fills it with data and sends to the module written in Qt. I want to deserialize received data using QDataStream, but my efforts didn't yield any result yet. Here's a simple example what I'm trying to achieve:
unsigned int pointer = 0;
const int IPC_MSG_LEN = 500;
const int IPC_MSG_HEADER = 200;
const int SOMETHING = 1443;
char api = 55;
char msg[IPC_MSG_LEN] = {0};
memcpy_s(msg, IPC_MSG_LEN, &IPC_MSG_HEADER, sizeof(int));
pointer = sizeof(unsigned int);
memcpy_s(&msg[pointer], IPC_MSG_LEN - pointer, &api, sizeof(char));
++pointer;
memcpy_s(&msg[pointer], IPC_MSG_LEN - pointer, &SOMETHING, sizeof(int));
QByteArray arr(msg, IPC_MSG_LEN);
QDataStream ds(&arr, QIODevice::ReadOnly);
qint32 header = 0, aa = 0;
qint8 t_api = 0;
ds >> header; //Doesn't work
ds >> t_api; //Works
ds >> aa; //Doesn't work
As you can see, the code is pretty simple, but header and aa variables are deserialized to a random number. However t_api (one byte variable) has correct value assigned.
So what's the problem with this code? Does QDataStream uses a private data format which is not compatible with the one I'm using? Should I write my own QIODevice implementation or there is a quick fix I'm not aware of? :)
Thanks, I appreciate your help.
UPDATE
Thank you very much guys, your solution worked perfectly with those primitive data types, but the problem is that I also need to be able to serialize/deserialize char* strings too.
wchar_t* name1 = L"something";
memcpy_s(&msg[pointer], IPC_MSG_LEN - pointer, name1, (wcslen(name1) + 1) * 2);
char* ai = new char[500];
ds >> ai; //ai becomes NULL here :|
Is there a way to achieve that? Thanks again
QDataStream::setByteOrder(QDataStream::LittleEndian);
#include <QDebug>
#include <QByteArray>
#include <QDataStream>
#include <QString>
#include <vector>
template<typename T> void writePtr(char*&dst, T data){
*reinterpret_cast<T*>(dst) = data;
dst += sizeof(T);
}
int main(int argc, char** argv){
const size_t ipcSize = 512;
std::vector<char> buffer(ipcSize, 0);
quint32 sendVal1 = 0x12345678, recvVal1 = 0;
quint8 sendVal2 = 0xee, recvVal2 = 0;
quint32 sendVal3 = 0x9999abcd, recvVal3 = 0;
char* dst = &buffer[0];
writePtr(dst, sendVal1);
writePtr(dst, sendVal2);
writePtr(dst, sendVal3);
QByteArray byteArray(&buffer[0]);
QDataStream stream(&byteArray, QIODevice::ReadOnly);
stream.setByteOrder(QDataStream::LittleEndian);
stream >> recvVal1 >> recvVal2 >> recvVal3;
qDebug() << QString(QObject::tr("sent: %1, received: %2")).arg(sendVal1, 8, 16).arg (recvVal1, 8, 16);
qDebug() << QString(QObject::tr("sent: %1, received: %2")).arg(sendVal2, 2, 16).arg(recvVal2, 2, 16);
qDebug() << QString(QObject::tr("sent: %1, received: %2")).arg(sendVal3, 8, 16).arg(recvVal3, 8, 16);
return 0;
}
but the problem is that I also need to be able to serialize/deserialize char* strings too.
Qt data serialization format is explained (in detail) here. You MUST read that document if you want to use QDataStream for IPC. Qt has nice documentation, so use it.
Also this is not a char* string:
wchar_t* name1 = L"something";
It is a wchar_t* string.
wchar_t has different size depending on compiler - either 4 or 2 bytes per wchar_t. Which means problem for IPC. unlike wchar_t, char is guaranteed to be 1 byte big.
So either encode entire string to UTF8 (or use 8bit-encoded string with known codepage/encoding) and write it as raw data in QByteArray-compatible format:
void writeDataPtr(char*& ptr, const char* data, quint32 size){
if (!data){
size = 0xffffffff;
writePtr(ptr, size);
return;
}
memcpy(ptr, data, size);
ptr += size;
}
Then use QString::fromUtf8 to decode it (or QTextCodec - if you decided to use other 8bit encoding instead of utf8). OR if you can ensure that your wchar_t* string is UTF16-compliant and sizeof(wchar_t) == 2, dump it in QString-compatible format.
By the way - If I were you, I'd avoid memcpy_s. It is not part of C++ standard, which is a very good reason to avoid it.
I want is to read wchar_t*/char* from QDataStream until stream position gets to null terminating character.
If this is homework, tag your post accordingly.
One of those should work:
QString readWcharString(QDataStream& stream){
QVector<ushort> rawData;
ushort tmp;
do{
stream >> tmp;
rawData.push_back(tmp)
}while(tmp);
return QString::fromUtf16(rawData.data());
}
or
QString readWcharString(QDataStream& stream){
QVector<wchar_t> rawData;
ushort tmp;
do{
stream >> tmp;
rawData.push_back(tmp)
}while(tmp);
return QString::fromWCharArray(rawData.data());
}
QDataStream stores the numbers in the big endian format by default.
You can change that with:
ds.setByteOrder(QDataStream::ByteOrder(QSysInfo::ByteOrder));
which will use the detected host endianness instead.

How to transfer larger objects through a socket in QT?

I would like to send/recieve image files and 2 ints as messages in a client server program.
I'm using QLocalSocket and QImage for this.
However I don't know how to read from the socket only after the image and the integers are fully written to the buffer, since the readyRead signal is already fired after the first couple of bytes.
Here's parts of my code:
// sending
QDataStream stream(socket);
stream << image << i << j;
// recieving
void MainWindow::readyRead() {
// ...
if (socket->bytesAvailable() > 400)
{
QByteArray b = socket->readAll();
QDataStream stream(&b, QIODevice::ReadOnly);
QImage image;
int i, j;
stream >> image >> i >> j;
// ...
}
}
I tried guessing the incoming file size, but since QImage is serialized to PNG the data size is variable and sometimes the end of the file doesn't get written to the buffer before I start to read it.
Is there an easy solution to this?
I would send a fixed size header first that describes the data being sent, specifically the type and size in bytes.
Then as you receive readReady events you pull whatever data is available into a buffer. Once you determine you have received all of the necessary data, you can stream it into a QImage object.
The BMP format has size information and PNG format has size information for each chunk. These are formats with what QImage serializes.
If you don't want to extract the information from raw data then serialize QImage first to QBuffer (so you know/control size and format better). Then stream that size and buffer.
Code example:
QBuffer buffer;
image.save(&buffer, "PNG", 100); //can change the compression level to suit the application - see http://qt-project.org/doc/qt-4.8/qimage.html#save
qint64 length = sizeof(quint32) + buffer.data().size(); //http://doc.qt.digia.com/4.7/datastreamformat.html
stream << length;
stream << buffer.data();
Then on the other end, first stream out the qint64 length so you know how big socket->bytesAvailable() has to be to stream out the full QByteArray. Then:
QByteArray ba;
stream >> ba;
QImage image = QImage::fromData(ba); // Get image from buffer data