Until now I've been using structs to store data and then putting these structs inside a QHash.
struct Event {
QString languageCode;
QString message;
//QMap<QString, QString> messages; Ignore it. Won't be using it.
QString prop;
qint64 dateSec;
qint64 dateUsec;
qint64 startDateSec;
qint64 startDateUsec;
};
QHash<QString, Event> eventList;
Event event;
event.languageCode = "en";
event.message = "";
event.prop = "www.google.com";
event.dateSec = 1429554399;
event.dateUsec = 0;
event.startDateSec = 0;
event.startDateUsec = 0;
eventList.insert("ab1443c323956", event);
// Test...
qDebug() << eventList.value("ab1443c323956").prop;
Is this the best approach to store data that's constantly being accessed and sometimes being modified? This is small but there won't be only one key in the Hash.
From code you posted:
eventList.insert("ab1443c323956", event);
// Test...
qDebug() << eventList.value("ab1443c323956").prop;
I assume, that you either hard-code identifiers of events (which are of type QString now) or save them somewhere. In this case, I would use QVector:
extern int ab1443c323956_index;
QVector<Event> eventList;
Event event;
//fill data
eventList.push_back(event);
ab1443c323956_index = eventList.size() - 1;
// Test...
qDebug() << eventList[ab1443c323956_index].prop;
Now, since you always append events (QVector::push_back() is equivalent to QVector::append()), you have following complexities:
insertion: constant amortized ( Amort. O(1) )
index lookup: constant ( O(1) )
For data, that is constantly accessed - nothing can beat this in terms of speed.
Of course, this solution is valid, if you can use integers as your "keys", but when it comes to operations like this one, even QHash can introduce significant overhead.
Related
I have a select query from which i want to store results in QT lists after query execution. Some of my code is:
QSqlQuery querySelect;
QStringList rfid;
QList<QDateTime> datetimeIN;
QList<QDateTime> datetimeOUT;
... Some other code + querySelect prepare.....
if(querySelect.exec())
{
while( querySelect.next() )
{
rfid.append(querySelect.value( 0 ).toString());
datetimeIN.append(querySelect.value( 1 ).toDateTime());
datetimeOUT.append(querySelect.value( 2 ).toDateTime());
}
}
else
{
qDebug() << querySelect.lastError();
}
I want to know is there another, more optimized faster approach to store the results in the lists, other than with the query.next while cycle because i think this is rather slow?
On the Qt side, the only optimization that I can think of is
querySelect.setForwardOnly(true)
By the way, I think you can change approach and create a single class and a list for it, rather then three separated lists.
struct Data
{
QString rfid;
QDateTime dtIn;
QDateTime dtOut;
};
Then store the results this way:
QList<Data> list;
while( querySelect.next() ) {
list.append( {querySelect.value(0).toString(),
querySelect.value(1).toDateTime(),
querySelect.value(2).toDateTime()} );
}
I need to serialize and parse multiple objects from my project, in order to save/load them when needed.
My objects will have exactly the same components : a QString name, an integer id, a QString description, and two integer x, y.
I'll need something like this :
{"name":"toto", "id":"42", "description":"tata", "x":"20", "y":"50"}
So I'll build my QJsonObject like this :
QJsonObject json;
json["id"] = object_to_serialize.get_id();
json["name"] = object_to_serialize.get_name();
json["description"] = object_to_serialize.get_description();
json["x"] = object_to_serialize.get_x();
json["y"] = object_to_serialize.get_y();
QJsonDocument filedoc(json);
file.write(filedoc.toJson);`
And in the file it will appear like this :
{"name":"toto", "id":"42", "description":"tata", "x":"20", "y":"50"}
{"name":"toto2", "id":"44", "description":"tata2", "x":"25", "y":"547"}
{"name":"toto3", "id":"46", "description":"tata3", "x":"21", "y":"580"}
My serialiser will take in parameter the object, the savefile name, and transform the object into a QJsonObject. It will need then to read the file to check if an object with the same id is here. If it is here, it will need to replace it, and if it is not, it will append it.
I'm a little lost between my serialization options and how to read it ;
Should I make a QJsonArray with multiple QJsonObject inside or QJsonObject with QJsonArrays ?
When I read it, I will need to check for the id ; but will a
foreach(object.value["id"] == 42)
//create the QJsonObject from the one with 42 and change it with the new data
will do to parse the object and not all of them ? Is there a better way ?
Thank you in advance for your answers.
You can have an array of json object, each of them having an ID so you can parse the relevant ones.
Although you could also parse all of them and add them in a map, as long as you don't have very heavy files it should be fine.
void parseJson(const QString &data)
{
QJsonDocument doc = QJsonDocument::fromJson(data.toUtf8());
if (doc.isNull())
{
war("invalid json document");
return;
}
QJsonArray jsonArray = doc.array();
foreach (const QJsonValue & value, jsonArray) {
QJsonObject obj = value.toObject();
if (obj.contains("id"))
{
if (obj["id"].toInt() == yourId) parseObject(obj);
}
}
}
void parseObject(const QJsonObject &obj)
{
if (obj.contains("valueA")) valueA = obj["valueA"].toDouble();
if (obj.contains("valueB")) valueB = obj["valueB"].toDouble();
}
This will work just fine if your file is not too big
Bigger Files
Now if you have very large file, it might be an issue to load it all in memory and parse it.
Since your structure is always the same and quite simple, JSON might not be the best choice, one more efficient method would be to do your own parser (or use probably some existing ones) that could read the file and process it as a stream.
Another method, would be to have one JSON entry per line preceded by an ID with a fixed number of digit. Load this in a QHash lookup and then only read id of interest from the file and only parse a small section.
// This code is not tested and is just to show the principle.
#define IDSIZE 5
QHash<int64, int64> m_lookup; // has to be global var
// For very large file, this might take some time and can be done on a separate thread.
// it needs to be done only once at startup (given the file is not modified externally)
void createLookup(const QString &fileName)
{
QFile inputFile(fileName);
if (inputFile.open(QIODevice::ReadOnly))
{
QTextStream in(&inputFile);
while (!in.atEnd())
{
int position = in.pos(); // store the position in the file
QString line = in.readLine();
int id = line.mid(0,IDSIZE).toInt(); // 5 digit id (like 00001, 00002, etc...
m_lookup[id] = position + IDSIZE;
}
inputFile.close();
}
}
QString getEntry(const QString &fileName, int64 id)
{
if (m_lookup.contains(id))
{
QFile inputFile(fileName);
if (inputFile.open(QIODevice::ReadOnly))
{
inputFile.seek(m_lookup[id]);
QString data = inputFile.readLine();
inputFile.close();
return data;
} else {
return QString(); // or handle error
}
} else {
return QString(); // or handle error
}
}
// use example
QString data = getEntry(id);
if (data.length() > 0)
{
QJsonDocument doc = QJsonDocument::fromJson(data.toUtf8());
if (!doc.isNull())
{
// assign your variables
}
}
and your data file looking like this:
00042{"name":"toto", "id":"42", "description":"tata", "x":"20", "y":"50"}
00044{"name":"toto2", "id":"44", "description":"tata2", "x":"25", "y":"547"}
00046{"name":"toto3", "id":"46", "description":"tata3", "x":"21", "y":"580"}
The advantage of this method, it will only read the entry of interest, and avoid having to load MB or GB of data in memory just to get a specific entry.
This could further be improved with a lookup table stored at the beginning of the file.
I am using QT and QCustomPlot to create a real time plotting tool, and the data of the plot is read from the Arduino UNO board. My application succeeded in plotting while the data is a total mess. Here is my code below (Some code is from QCustomPlot website):
void Dialog::realtimeDataSlot()
{
bool currentPortNameChanged = false;
QString currentPortName;
if (currentPortName != portName) {
currentPortName = portName;
currentPortNameChanged = true;
}
QString currentRequest = request;
QSerialPort serial;
if (currentPortNameChanged) {
serial.close();
serial.setPortName(currentPortName);
if (!serial.open(QIODevice::ReadOnly)) {
return;
}
}
static QTime time(QTime::currentTime());
// calculate two new data points:
double key = time.elapsed()/1000.0;
static double lastPointKey = 0;
if (key-lastPointKey > 0.002) // at most add point every 2 ms
{
// add data to lines:
if(serial.waitForReadyRead(-1)){
data = serial.readAll();
QTextStream(stdout) << "HERE:" << data.toDouble() << endl;
customPlot->graph(0)->addData(key, data.toDouble());
customPlot->graph(0)->rescaleValueAxis(); //rescale value (vertical) axis to fit the current data:
lastPointKey = key;
customPlot->xAxis->setRange(key, 8, Qt::AlignRight);
customPlot->replot();
static double lastFpsKey;
static int frameCount;
++frameCount;
if (key-lastFpsKey > 2) // average fps over 2 seconds
{
lastFpsKey = key;
frameCount = 0;
}
}
}
// calculate frames per second:
if (currentPortName != portName) {
currentPortName = portName;
currentPortNameChanged = true;
} else {
currentPortNameChanged = false;
}
}
When I Tried to print out the data I read from the serial port, I found the following:
HERE:1
HERE:15
HERE:150
HERE:149
HERE:149
HERE:149
HERE:150
HERE:150
HERE:15
HERE:150
HERE:149
HERE:49
HERE:150
HERE:150
HERE:1
HERE:150
The values around 150 are normal while the value that are 0, 1 to others are not. Also it is not print out at a stable speed. I don't know what happened to this, and thanks to whoever may help, and I would appreciate it if there is any better ways to implement this.
The problem here is that it is not guaranteed that the serial transmission is received all at once. So it is better to let the serial to be processed somewhere else, for instance:
// in the class definition
QSerialPort serialPort;
private slots:
void handleReadyRead();
private:
QByteArray serialBuffer;
volatile double lastSerialValue;
// In the initialization part (not the realtimeDataSlot function)
lastSerialValue = qQNaN();
serialPort.setPortName(currentPortName);
connect(&serialPort, &QSerialPort::readyRead, this, &Dialog::handleReadyRead, Qt::UniqueConnection);
if (!serialPort.open(QIODevice::ReadOnly)) {
return;
}
serialBuffer.clear();
// Other functions:
void Dialog::realtimeDataSlot()
{
...
if (key-lastPointKey > 0.002) // at most add point every 2 ms
{
if (!qIsNaN(lastSerialData))
{
// use lastSerialValue as the data.toDouble() you had before, then, at the end
lastSerialValue = qQNaN();
}
...
}
void Dialog::handleReadyRead()
{
serialBuffer.append(serialPort.readAll());
int serPos;
while ((serPos = serialBuffer.indexOf('\n')) >= 0)
{
bool ok;
double tempValue = QString::fromLatin1(serialBuffer.left(serPos)).toDouble(&ok);
if (ok) lastSerialValue = tempValue;
serialBuffer = serialBuffer.mid(serPos+1);
}
}
Explanation: whenever you receive something from the arduino the bytes are appended to a buffer. Then the byte array is parsed looking for a terminator, and if found the byte array is split and analysed. When the other function needs the data, it simply pulls the most recent one saved in the variable.
NOTE 1: I saw that you used a binary transmission. The problem is that you do not have any way to determine where the data begins and end in this way. For instance, if you receive 0x01 0x02 0x03 0x04 and you know that there are 3 bytes, are they 01..03 or 02..04 or 03, 04 and a missing one or...? The version I implemented requires you to send data in string format with a new-line terminator (simplest version, you just have to write Serial.println(doubleValue); in the arduino code), but if you need the binary version I can give you some hints
NOTE 2: The code I wrote is NOT thread safe. It will work only if the realtimeDataSlot and the handleReadyRead are called in the same thread. Note that if they belong to the same object and are called through signals this is guaranteed.
Now, this should work. But I highly discourage you from doing this. I don't know who needs to call the realtimeDataSlot(), but I think that the most correct version is something like this:
// in the class definition
QSerialPort serialPort;
private slots:
void handleReadyRead();
void receivedData(double val);
private:
QByteArray serialBuffer;
signals:
void newData(double data);
// In the initialization part (not the realtimeDataSlot function)
serialPort.setPortName(currentPortName);
connect(&serialPort, &QSerialPort::readyRead, this, &Dialog::handleReadyRead, Qt::UniqueConnection);
connect(this, &Dialog::newData, this, &Dialog::receivedData, Qt::UniqueConnection);
if (!serialPort.open(QIODevice::ReadOnly)) {
return;
}
serialBuffer.clear();
// Other functions:
void Dialog::receivedData(double val)
{
double key = time.elapsed()/1000.0;
static double lastPointKey = 0;
if (key-lastPointKey > 0.002) // at most add point every 2 ms
{
QTextStream(stdout) << "HERE:" << data.toDouble() << endl;
customPlot->graph(0)->addData(key, data.toDouble());
customPlot->graph(0)->rescaleValueAxis();
...
}
}
void Dialog::handleReadyRead()
{
serialBuffer.append(serialPort.readAll());
int serPos;
while ((serPos = serialBuffer.indexOf('\n')) >= 0)
{
bool ok;
double tempValue = QString::fromLatin1(serialBuffer.left(serPos)).toDouble(&ok);
if (ok) emit newData(tempValue);
serialBuffer = serialBuffer.mid(serPos+1);
}
}
So keep the graph responsive to events (received a new data) instead of to a timer.
One more thing: I removed the port change on purpose. I suggest you to handle it in another way: put a button to start and stop the serial, and when the serial port is started prevent the user from changing the port name. This way the user will explicitely need to shut it down when he needs to change the port. If you want your version, however, don't include it in your code, but make a slot on its own to call whenever you need to change the port name:
void changeSerialPortName(QString newName)
{
if (newName != serialPort.portName()) {
if (serialPort.isOpen())
serialPort.close();
serialPort.setPortName(newName);
if (!serialPort.open(QIODevice::ReadOnly)) {
return;
}
}
}
The signal/slot mechanism in Qt, is a static mechanism. The classes have to be preprocessed by the moc compiler.
Now I want to create signals and slots dynamically at run-time.
I already have a working solution, but it feels to me like a hack, although I am using publicly available methods.
This is the code for dynamic slots:
bool DynamicQObject::connectDynamicSlot(const QString &objectName, QObject *pSourceObject, QMetaMethod signalMethod)
{
QByteArray slotName = signalMethod.name().prepend("on").append("(");
QStringList parameters;
for (int i = 0, j = signalMethod.parameterCount(); i < j; ++i)
{
parameters << QMetaType::typeName(signalMethod.parameterType(i));
}
slotName.append(parameters.join(",")).append(")");
QByteArray theSignal = QMetaObject::normalizedSignature(signalMethod.methodSignature().constData());
QByteArray theSlot = QMetaObject::normalizedSignature(slotName);
if (!QMetaObject::checkConnectArgs(theSignal, theSlot))
{
return false;
}
int signalId = pSourceObject->metaObject()->indexOfSignal(theSignal);
if (signalId < 0)
{
return false;
}
int slotId = slotIndices.value(theSlot, -1);
if (slotId < 0)
{
slotId = slotList.size();
slotIndices[theSlot] = slotId;
slotList.append(createSlot(theSlot, objectName, signalMethod));
}
return QMetaObject::connect(pSourceObject, signalId, this, slotId + metaObject()->methodCount());
}
As you can see, I make intensive use of the QMetaObject, and particularly the index of the slots (method count).
The code for dynamic signals is comparable.
Now my question is: how future proof is this solution, especially because I assume that the index must be at least one bigger than the methodCount()?
Now my question is: how future proof is this solution, especially because I assume that the index must be at least one bigger than the methodCount()?
It should work as of now.
As for future proof... may be. The snippet is using unsupported features which means those might break at any point in the future. It is likely though it will continue working.
As a separate option, if all your things are very similar in nature (like in a vector or something), consider connecting to a lambda function. eg:
QObject::connect(iter->checkbox, &QCheckBox::stateChanged,
[&, iter->startup_status](int new_val) {
if (new_val == Qt::CheckState::Checked) {
startup_status = true;
} else {
startup_status = false;
}
});
where iter is a struct/class that has public fields
QCheckBox *checkbox;
bool startup_status;
by this method, it is possible to have a variable number of very similar "slots" (which aren't actually slots, but act like slots)
I'm capable to fill the database table in wxListCtrl,
my problem is to handle high range of data, I want to do this with the help of thread concept , perhaps it will save to hang the frame because of high amount of data.
I'm new in thread concept so your single lines will be a book for me.
Update:
My question was- how to display large data in wxListCtrl with using concept of wxThread
so for this I used thread concept I add two more files thread.c and thread.cpp
my entry thread code is shown below
void *MyThread :: Entry()
{
int i=1,j,k=0;
while(i!=400)
{
long index=this->temp->data_list_control->InsertItem(i,wxT("amit"));
for(j=1; j<3; j++) {
this->temp->data_list_control->SetItem(index,j,wxT("pathak"));
}
k++;
if(k==30) {
this->Sleep(1000);
k=0;
}
i++;
}
}
It is sometimes working fine but when I try to increase the value of i, it shows an error like
-*showingdatainwxlistctrl: ../../src/XlibInt.c:595: _XPrivSyncFunction: Assertion `dpy->synchandler == _XPrivSyncFunction' failed.*
or sometime it gives error like
***[Xcb] xcb_io.c:378: _XAllocID: Assertion `ret != inval_id' failed***
Why it is happening to me?
You can define your own thread object in wxWidgets in the following way:
class MyThread : public wxThread
{
private:
wxListCtrl* m_pListCtrl;
public:
MyThread(wxListCtrl* pListCtrl, wxThreadKind kind = wxTHREAD_DETACHED) :
wxThread(kind), m_pListCtrl(pListCtrl) {
}
virtual ~MyThread() {
}
virtual void* Entry() {
// here you have to place your code that will be running in separate thread
// m_pListCtrl-> ...
}
};
And this is the way how you can start your thread (assume you have your pListCtrl pointer here):
MyThread * pMyThread = new MyThread (pListCtrl);
wxThreadError ThreadError = pMyThread->Create();
if (wxTHREAD_NO_ERROR!=ThreadError) {
wxLogError(L"Can not create thread, wxThreadError '%d'", (int)ThreadError);
delete pMyThread;
return false;
}
ThreadError = pMyThread->Run();
if (wxTHREAD_NO_ERROR!=ThreadError) {
wxLogError(L"Can not run thread, wxThreadError '%d'", (int)ThreadError);
delete pMyThread;
return false;
}
// here, everything is ok.
Anyway, this is not the best solution for your problem. As far as I've understood, you need to display large amount of data in your wxListCtrl. To do this, you can use virtual ctrl (created with flag wxLC_VIRTUAL) and provide data source:
class MyListCtrl : public wxListCtrl
{
public:
MyListCtrl( ...) { ... }
virtual ~MyListCtrl();
protected:
virtual int OnGetItemImage(long item) const {
// You need this only if you want to provide specific image for your item.
// If you do not need it, just do not overload this method.
}
virtual wxString OnGetItemText(long item, long column) const {
// This is where you have to provide data for [item, column].
// Suppose, you have matrix A[n,m] which represents actually the data
// you want to display. The elements of this matrix can be of any type
// (strings, doubles, integers etc).
// You should return here wxString object that contains
// representation of the matrix's element A[item, column].
return ToWxString(A[item, column]);
// where ToWxString is your method that converts data to string
// So, you do not need to load all the data from A to wxListCtrl.
// Instead of it, wxListCtrl will determine which rows of the matrix should be
// displayed based on sizes and scroll position of wxListCtrl, and will
// call this method to obtain corresponding strings.
}
};
To create, you may use:
m_pListCtrl = new MyListCtrl( ..., ..., wxLC_REPORT | wxLC_SINGLE_SEL | wxLC_VIRTUAL | wxSUNKEN_BORDER | wxLC_VRULES | wxLC_HRULES);
Best regards!
When you are performing high range of data you are bound to use WXThread in your program
Firstly was trying to fill wxListCtrl from wxEntry point, it was wrong u can not hit any main thread control from entry point, it does not give error, but it is a wrong concept
Here u need to pass the data to handler, handler will use it to fill wxListCtrl
code look like this->
void *MyThread :: Entry()
{
int a;
Handler handler_obj;
char *database_name=DATABASE_NAME;
connection =handler_obj.handler(101,database_name);
if(connection==NULL)
{
wxMessageBox(wxT("CAN NOT CONNECT TO DATABASE"), wxT("Message"), wxOK | wxICON_INFORMATION, NULL, -1, -1);
}
else
{
List_Ctrl_Data list_ctrl_data_object;
table_data=list_ctrl_data_object.fetch_table(connection);
MYSQL_ROW row;
while((row=mysql_fetch_row(table_data))!=NULL)
{
wxCommandEvent event( wxEVT_COMMAND_TEXT_UPDATED, 100000 );
void *row_data;
row_data=(void *)row;
event.SetClientData(row_data);
temp->GetEventHandler()->AddPendingEvent( event );
this->Sleep(1000);
}
}
}
to handle the row data we will use
void Id_Search_Report::onNumberUpdate(wxCommandEvent& evt)
{
int j;
void* hold_row;
hold_row=(void *)evt.GetClientData();
MYSQL_ROW row;
row=(MYSQL_ROW)hold_row;
const char* chars1 = row[0];
wxString mystring1(chars1, wxConvUTF8);
long index=data_list_control->InsertItem(this->counter,mystring1);
this->counter++;
for(j=1;j<12;j++)
{
const char* chars2=row[j];
wxString mystring2(chars2,wxConvUTF8);
data_list_control->SetItem(index,j,mystring2);
}
}
thread is returning a row , this method will handle the row and fill ListCtrl , it is a proper way to fill wxListCtrl.