Qt - QTreeView and SQL - Performance advice - c++

Question

I have an app where I want to show SQL query results in a hierarchical structure. I have something work that is based on this example.
The main part of my code where the tree nodes are created currently looks like this:
void TreeModel::setupModelData(TreeItem *parent)
{
QList<TreeItem*> parents;
QList<int> indentations;
parents << parent;
QList<QVariant> columnData;
QVector<QString> vecFileNames = getFileNames();
QVector<QString> vecTableNames = getTableNames();
for(int i = 0; i < vecFileNames.size(); i++)
{
columnData.clear();
columnData << vecFileNames[i];
parents.last()->appendChild(new TreeItem(columnData, parents.last()));
int childCount = parents.last()->childCount() - 1;
parents << parents.last()->child(childCount); //add the current parent's last child as a parent
for(int j = 0; j < vecTableNames.size(); j++)
{
columnData.clear();
columnData << vecTableNames[j];
parents.last()->appendChild(new TreeItem(columnData, parents.last()));
QVector<QString> vecTableValues = getTableValues(&vecTableNames[j]);
int childCount = parents.last()->childCount() - 1;
parents << parents.last()->child(childCount); //add the current parent's last child as a parent
for(int k = 0; k < vecTableValues.size(); k++)
{
columnData.clear();
columnData << vecTableValues[j];
parents.last()->appendChild(new TreeItem(columnData, parents.last()));
}
}
parents.pop_back();
}
}
QVector<QString> TreeModel::getFileNames()
{
db.open();
QVector<QString> vecFileNames;
QSqlQuery query(db);
QString strQuery = "SELECT PK_fileName FROM fileproperties";
query.prepare(strQuery);
if(query.exec() == true)
{
while(query.next())
{
vecFileNames.push_back(query.value(0).toString());
}
}
db.close();
return vecFileNames;
}
However, it is incredibly slow retrieving 2000 queries worth of data.
Can anyone suggest another approach to the one I'm using now?

Answer 1

You should implement function hasChildren() and use lazy population of model data. You should basically read this article and documentation of QAbstractItemModel class (especially canFetchMore() function).

Answer 2

I would guess that the perfomance suffers by inserting 2000 entries individually, triggering 2000 view updates, maybe 2000 sorts etc... you should provide a way of adding data to your model that takes "batches" of items and only signals changes once...

Answer 3

For MS SQL Servers I always use QSqlQuery::setForward(true) to speed up the queries by up to 10 times.
This "forward" mode just disables the row caches and forces the sql driver to request all results as one fat reply, instead of getting the query results as multiple parts (one ore more rows).
I discovered the problem with a MS SQL Server 2005+2008 with more than 10 million entries, where I searched for only 200-400 entries for a special day to display them inside a QTableView using an QSqlTableModel.
With the forward mode enabled my query times went from over 10s to only 200-300 milliseconds - on a database with over 10 million entries!
Example:
QSqlQuery query(database);
query.setForwardOnly(m_bForwardOnly);
query.exec(statement);
if ( query.lastError().isValid() || database.lastError().isValid() ) {
...evaluate the results...
}