I have trouble with reading serial data from an Arduino, and sending it to the UI. I found it to be a hassle to split all serial data into 3 seperate variables, and therefore used the left and mid to get the specific numbers. The problem now is that the numbers I get stays the same, eventhough I can see on an LCD display they change.
This is a section of the code:
void Dolle::serialReceived(){
ba = serial->readAll();
serialBuffer += QString::fromStdString(ba.toStdString());
QString bufferSplit = serialBuffer;
QString hum = bufferSplit.left(2);
QString temp = bufferSplit.mid(2, 2);
QString gas = bufferSplit.mid(4, 4);
if((hum.size()==2) && (temp.size()==2) && (gas.size()==4)){
ui->humLabel->setText("Humidity: "+hum+" %");
ui->tempLabel->setText("Temperature: "+temp+(char(176))+ "C");
ui->gasLabel->setText("Gas level: "+gas);
qDebug() << hum << temp << gas;
}
}
There are several problems:
You're appending to serialBuffer, but you never empty it out. The QIODevice already maintains an internal, resizable circular buffer for you, so a second one is not necessary.
You're going from binary representation to a QString via a std::string. That is completely unnecessary.
You're not explicit about what character encoding is used in the binary data.
The UI is updated repeatedly, instead of at most once per readyRead signal.
You're using magic constants. You should UTF-8 encode your source instead, or use a named QChar constant.
You're manually building strings using string operators, this impedes internationalization and maintainability.
You're using spaces as a mechanism to align the display in your UI. Perhaps you should design your UI in a different manner, so that such hacks won't be necessary.
I presume that your strings are separated somehow - perhaps each is in a separate line? In any case, you should keep reading the complete delimited strings from the device as long as they are available. The QIODevice::readLine method makes it easy in case of line-delimited data:
void Dolle::serialReceived() {
QString validLine;
while (serial->canReadLine()) {
auto binLine = serial->readLine();
auto line = QString::fromLatin1(binLine);
if (line.length() < 8)
continue;
validLine = line;
}
if (validLine.isEmpty()) return;
auto hum = line.left(2);
auto temp = line.mid(2, 2);
auto gas = line.mid(4, 4);
ui->humLabel->setText(QStringLiteral("Humidity: %1%").arg(hum));
ui->tempLabel->setText(QStringLiteral("Temperature: %1°C").arg(temp));
ui->gasLabel->setText(QStringLiteral("Gas level: %1").arg(gas));
}
Suppose that instead of line-separated data, your data arrives in fixed-size chunks. You'd process them in a similar fashion:
void Dolle::serialReceived() {
QString validPacket;
while (serial->bytesAvailable() >= 8) {
auto bin = serial->read(8);
auto packet = QString::fromLatin1(bin);
if (packet.length() < 8)
continue;
validPacket = packet;
}
...
}
Related
I'm trying to use IOConnectCallStructMethod so send data in the input field to talk to my driver in DriverKit (iPadOS)
In this particular method I send an input and I expect getting an output. But ignore the output for now.
In my case IOConnectCallStructMethod is called with Swift:
var input:[UInt8] = Array(repeating: 0, count: 200)
for i in 0..<200 {
if (i % 2 == 0) {
input[i] = UInt8(i)
}
}
let arraySize:Int = input.count;
var outputSize = MemoryLayout<UInt8>.size * arraySize
var output:[UInt8] = Array(repeating: 0, count: arraySize)
let inputSize = MemoryLayout<UInt8>.size * input.count
let ret = IOConnectCallStructMethod(connection, Selector.mySelector.rawValue, &input, inputSize, &output, &outputSize)
This is the dispatch in the external method:
[ExternalMethodType_MyMethodRequest] =
{
.function = (IOUserClientMethodFunction) &Client::StaticHandleRequest,
.checkCompletionExists = false,
.checkScalarInputCount = 0,
.checkStructureInputSize = 200,
.checkScalarOutputCount = 0,
.checkStructureOutputSize = 200,
},
The ExternalMethod gets called and when I try to access to the input data with this:
char* input = nullptr;
size_t length = 0;
if (arguments->structureInput != nullptr)
{
input = (char*)arguments->structureInput->getBytesNoCopy();
length = arguments->structureInput->getLength();
Log("Input: %s", input);
Log("length: %zu", length);
}
If I check the variable input with a breakpoint the value is always "" and the length is 200
I tried adding the input in a struct, same result.
I tried using IOConnectCallMethod, same result.
The only alternative I can do it's making the input data extra big to force the system to use the descriptor instead of the OSData, but the input in this case would be always much smaller, so not sure if the best way to go.
Interestingly if I print the output of input with:
#define Log(fmt, ...) os_log(OS_LOG_DEFAULT, "NullDriver - " fmt "\n", ##__VA_ARGS__)
It will give me private:
NullDriver - Input: <private>
Maybe the data is actually there but I can't see it?
Further to discussion in comments:
There is no issue here with the way the data is being sent from app to dext, but rather with the way it is being logged/printed: a format string of %s is not a suitable way of printing the contents of a byte array. That treats it as a nul-terminated string (UTF-8, usually). It looks like you're sending arbitrary bytes, not a human readable string. In particular, your first byte is 0, which is how an empty C string is represented, and hence your empty output.
A format string such as "%02x %02x %02x %02x" with arguments input[0], input[1], input[2], input[3] would be more appropriate for printing the first 4 bytes; you'll have to do this in a loop, possibly writing to a buffer using snprintf before sending it to the system log, to print the entire array.
Separately: to fix the <private> issue when using %s, use %{public}s. (but read the os_log documentation on the privacy ramifications of this before you ship it)
I think the data is there but I can't see it.
If I do:
char realInput[200];
memcpy(realInput, input, 200);
When I loop printing in the debugger: realInput[i] I get my precious data
What I am trying to do is comparin 2 QStrings that have special characters (French)
first I recieved from server as json data saved in txtInfo
txtInfo = "Présenter";
When I am having condition like this it's not gonna work(its not gonna set state.)
if (txtInfo == "Présenter"){
m_appState = 8;
m_appStateString = AppStatesArray[m_appState];
}
else {
m_appState = -1;
m_appStateString = "UNKNOWN";
}
What I am missing? What if I would like to compare not French but Chinese?
Thank you very much
Since Qt 5 QString's operator== performs fromUtf8 conversion on the character array being compared to it. But if your source file (.cpp) isn't using utf8 you need to build your own QString.
Depending on your source file's (.cpp) encoding:
Utf8:
QString compared = QString::fromUtf8("Présenter");
if (txtInfo == QString::fromUtf8("Présenter")){
local 8-bit:
QString compared = QString::fromLocal8Bit("Présenter");
if (txtInfo == QString::fromUtf8("Présenter")){
For 100% correctness, don't forget to normalize your strings:
txtInfo = txtInfo.normalized(QString::NormalizationForm_D);
QString compared = /* the correct form for you */;
if (txtInfo == compared.normalized(QString::NormalizationForm_D)){
Starting from a QByteArray, I'd like to search "\n" char inside my QByteArray and join all the characters from the beginning up to "\n" and save them in a QString; after that, I'd pass to the following bytes up to the next "\n" and save these into a new QString
QByteArray MyArray= (all data from my previous process);
quint16 ByteArrayCount = MyArray.count(); // number of bytes composing MyArray
quint16 mycounter;
QString myString;
while (mycounter < ByteArrayCount)
{
if(MyArray[mycounter] != "\n")
myString.append(MyArray[mycounter]);
mycounter++;
}
This to append all bytes preceeding a new line; my problem is how to evaluate MyArray[counter], since I'm not able to check every byte when the counter increases.
Solution?
You could save yourself the trouble and simply:
QString s(myArray);
QStringList resultStrings = s.split('\n');
This will give you a list of strings split for every new line character, which is what you sound like you want to do.
Also, not to belabor the point, but you don't initialize your counter, and you really should ;)
Here is simple example of using function hello
QString str = "ooops\nhello mama\n daddy cool";
QByteArray bta;
bta.append(str);
for(quint16 index = bta.indexOf('\n');
index != -1;
index = bta.indexOf('\n', index+1)) {
/**
* Do something with index
**/
}
But according to your question there is not so clear when you say that you "not able to check every byte". If you know diapasons of available mem, you can use raw data with:
const char * ptr = MyArray.constData();
and use custom validators:
while(ptr){
if(valid(ptr) && ptr == '\n') {
/**
* do something ...
**/
}
ptr++;
}
ow and also in C/C++:
"\n" != 'n'
because "\n" - is const C string(char[2]) containing \n and EOF('\0')
and '\n' - is just simple C char;
I have to read a text file into a array of structures.I have already written a program but it is taking too much time as there are about 13 lac structures in the file.
Please suggest me the best possible and fastest way to do this in C++.
here is my code:
std::ifstream input_counter("D:\\cont.txt");
/**********************************************************/
int counter = 0;
while( getline(input_counter,line) )
{
ReadCont( line,&contract[counter]); // function to read data to structure
counter++;
line.clear();
}
input_counter.close();
keep your 'parsing' as simple as possible: where you know the field' format apply the knowledge, for instance
ReadCont("|PE|1|0|0|0|0|1|1||2|0||2|0||3|0|....", ...)
should apply fast char to integer conversion, something like
ReadCont(const char *line, Contract &c) {
if (line[1] == 'P' && line[2] == 'E' && line[3] == '|') {
line += 4;
for (int field = 0; field < K_FIELDS_PE; ++field) {
c.int_field[field] = *line++ - '0';
assert(*line == '|');
++line;
}
}
well, beware to details, but you got the idea...
I would use Qt entirely in this case.
struct MyStruct {
int Col1;
int Col2;
int Col3;
int Col4;
// blabla ...
};
QByteArray Data;
QFile f("D:\\cont.txt");
if (f.open(QIODevice::ReadOnly)) {
Data = f.readAll();
f.close();
}
MyStruct* DataPointer = reinterpret_cast<MyStruct*>(Data.data());
// Accessing data
DataPointer[0] = ...
DataPointer[1] = ...
Now you have your data and you can access it as array.
In case your data is not binary and you have to parse it first you will need a conversion routine. For example if you read csv file with 4 columns:
QVector<MyStruct> MyArray;
QString StringData(Data);
QStringList Lines = StringData.split("\n"); // or whatever new line character is
for (int i = 0; i < Lines.count(); i++) {
String Line = Lines.at(i);
QStringList Parts = Line.split("\t"); // or whatever separator character is
if (Parts.count() >= 4) {
MyStruct t;
t.Col1 = Parts.at(0).toInt();
t.Col2 = Parts.at(1).toInt();
t.Col3 = Parts.at(2).toInt();
t.Col4 = Parts.at(3).toInt();
MyArray.append(t);
} else {
// Malformed input, do something
}
}
Now your data is parsed and in MyArray vector.
As user2617519 says, this can be made faster by multithreading. I see that you are reading each line and parsing it. Put these lines in a queue. Then let different threads pop them off the queue and parse the data into structures.
An easier way to do this (without the complication of multithreading) is to split the input data file into multiple files and run an equal number of processes to parse them. The data can then be merged later.
QFile::readAll() may cause a memory problem and std::getline() is slow (as is ::fgets()).
I faced a similar problem where I needed to parse very large delimited text files in a QTableView. Using a custom model, I parsed the file to find the offsets to the start of a each line. Then when data is needed to display in the table I read the line and parse it on demand. This results in a lot of parsing, but that is actually fast enough to not notice any lag in scrolling or update speed.
It also has the added benefit of low memory usage as I do not read the file contents into memory. With this strategy nearly any size file is possible.
Parsing code:
m_fp = ::fopen(path.c_str(), "rb"); // open in binary mode for faster parsing
if (m_fp != NULL)
{
// read the file to get the row pointers
char buf[BUF_SIZE+1];
long pos = 0;
m_data.push_back(RowData(pos));
int nr = 0;
while ((nr = ::fread(buf, 1, BUF_SIZE, m_fp)))
{
buf[nr] = 0; // null-terminate the last line of data
// find new lines in the buffer
char *c = buf;
while ((c = ::strchr(c, '\n')) != NULL)
{
m_data.push_back(RowData(pos + c-buf+1));
c++;
}
pos += nr;
}
// squeeze any extra memory not needed in the collection
m_data.squeeze();
}
RowData and m_data are specific to my implementation, but they are simply used to cache information about a row in the file (such as the file position and number of columns).
The other performance strategy I employed was to use QByteArray to parse each line, instead of QString. Unless you need unicode data, this will save time and memory:
// optimized line reading procedure
QByteArray str;
char buf[BUF_SIZE+1];
::fseek(m_fp, rd.offset, SEEK_SET);
int nr = 0;
while ((nr = ::fread(buf, 1, BUF_SIZE, m_fp)))
{
buf[nr] = 0; // null-terminate the string
// find new lines in the buffer
char *c = ::strchr(buf, '\n');
if (c != NULL)
{
*c = 0;
str += buf;
break;
}
str += buf;
}
return str.split(',');
If you need to split each line with a string, rather than a single character, use ::strtok().
I'm writing networking programming using socket.h to my studies. I have written server and client simple programs that can transfer files between them using buffer size given by user.
Server
void transfer(string name)
{
char *data_to_send;
ifstream myFile;
myFile.open(name.c_str(),ios::binary);
if(myFile.is_open))
{
while(myFile.eof))
{
data_to_send = new char [buffer_size];
myFile.read(data_to_send, buffer_size);
send(data_to_send,buffer_size);
delete [] data_to_send;
}
myFile.close();
send("03endtransmission",buffer_size);
}
else
{
send("03error",buffer_size);
}
}
Client
void download(string name)
{
char *received_data;
fstream myFile;
myFile.open(name.c_str(),ios::out|ios::binary);
if(myFile.is_open())
{
while(1)
{
received_data = new char[rozmiar_bufora];
if((receivedB = recv(sockfd, received_data, buffer_size,0)) == -1) {
perror("recv");
close(sockfd);
exit(1);
}
if(strcmp(received_data,"03endoftransmission") == 0)
{
cout<<"End of transmission"<<endl;
break;
}
else if (strcmp(received_data,"03error") == 0)
{
cout<<"Error"<<endl;
break;
}
myFile.write(received_data,buffer_size);
}
myFile.close();
}
The problem occurs, when I want to implement my own protocol- two chars (control), 32 chars hash, and the rest of package is data. I tried few times to split it and I end up with this code:
Server
#define PAYLOAD 34
void transfer(string name)
{
char hash[] = "12345678901234567890123456789012"; //32 chars
char *data_to_send;
ifstream myFile;
myFile.open(name.c_str(),ios::binary);
if(myFile.is_open))
{
while(myFile.eof))
{
data_to_send = new char [buffer_size-PAYLOAD];
myFile.read(data_to_send, buffer_size-PAYLOAD);
concatenation = new char[buffer_size];
strcpy(concatenation,"02");
strcat(concatenation,hash);
strcat(concatenation,data_to_send);
send(concatenation,buffer_size);
delete [] data_to_send;
delete [] concatenation;
}
myFile.close();
send("03endtransmission",buffer_size);
}
else
{
send("03error",buffer_size);
}
}
Client
void download(string name)
{
char *received_data;
fstream myFile;
myFile.open(name.c_str(),ios::out|ios::binary);
if(myFile.is_open())
{
while(1)
{
received_data = new char[buffer_size];
if((receivedB = recv(sockfd, received_data, buffer_size,0)) == -1) {
perror("recv");
close(sockfd);
exit(1);
}
if(strcmp(received_data,"03endoftransmission") == 0)
{
cout<<"End of transmission"<<endl;
break;
}
else if (strcmp(received_data,"03error") == 0)
{
cout<<"Error"<<endl;
break;
}
control = new char[3];
strcpy(control,"");
strncpy(control, received_data,2);
control[2]='\0';
hash = new char[33];
strcpy(hash,"");
strncpy(hash,received_data+2,32);
hash[32]='\0';
data = new char[buffer_size-PAYLOAD+1];
strcpy(data,"");
strncpy(data,received_data+34,buffer_size-PAYLOAD);
myFile.write(data,buffer_size-PAYLOAD);
}
myFile.close();
}
But this one inputs to file some ^# instead of real data. Displaying "data" to console looks the same on server and client. If you know how I can split it up, I would be very grateful.
You have some issues which may or may not be your problem.
(1) send/recv can return less than you requested. You may ask to receive 30 bytes but only get 10 on the recv call so all of these have to be coded in loops and buffered somewhere until you actually get the number you wanted. Your first set of programs was lucky to work in this regard and probably only because you tested on a limited amount of data. Once you start to push through more data your assumptions on what you are reading (and comparing) will fail.
(2) There is no need to keep allocating char buffers in the loops; allocate them before the loop or just use a local buffer rather than the heap. What you are doing is inefficient and in the second program you have memory leaks because you don't delete them.
(3) You can get rid of the strcpy/strncpy statements and just use memmove()
Your specific problem is not jumping out at me but maybe this will push in the right direction. More information what is being transmitted properly and exactly where in the data you are seeing problems would be helpful.
But this one inputs to file some ^# instead of real data. Displaying
"data" to console looks the same on server and client. If you know how
I can split it up, I would be very grateful.
You say that the data (I presume the complete file rather than the '^#') is the same on both client and server? If this is the case, then your issue is likely writing the data to file, rather than the actual transmission of the data itself.
If this is the case, you'll probably want to check assumptions about how the program writes to file - for example, are you passing in text data to be written to file, or binary data? If you're writing binary data, but it uses the NULL-terminated string, chances are it will quit early treating valid binary information as a NULL.
If it's text mode, you might want to consider initialising all strings with memset to a default character (other than NULL) to see if it's garbage data being out put.
If both server and client display the '^#' (or whatever data), binary based char data would be incompatible with the strcpy/strcat functions as this rely on NULL termination (where-as binary uses size termination instead).
I can't track down the specific problem, but maybe this might offer an insight or two that helps.