QTableView: endMoveRows in model resets horizontal header sizes - c++

I'm trying to implement drag'n'drop reordering of rows in a QTableView.
The operation needs to actually reorder the underlying data, therefore reordering the vertical header rows doesn't seem like a good solution.
I instead opted for the underlying model to handle this by providing the row indices in its mimeData function and accepting this list in dropMimeData. In the dropMimeData function, I use the beginMoveRows and endMoveRows to move individual rows visually, in addition to moving the underlying data. This approach seems to work, with only one shortcoming: as soon as I call endMoveRows, the QTableView's horizontal header resets all its section sizes to defaults.
Am I missing something I need to do, or is this a bug in Qt (or even intended behavior)? Is there any other way to handle drag'n'drop reordering?
Here is a simplified piece of code that shows the same behavior:
class Model: public QAbstractTableModel
{
public:
Model():
m_Items({"A", "B", "C", "D", "E"})
{
}
virtual QVariant data(const QModelIndex & a_Parent, int a_Role) const override
{
return QString("%1 - %2").arg(m_Items[a_Parent.row()]).arg(a_Parent.column());
}
virtual int rowCount(const QModelIndex & a_Parent) const override
{
return m_Items.count();
}
virtual int columnCount(const QModelIndex & a_Parent) const override
{
return 4;
}
void moveNow()
{
// Moves the last row as the 3rd row:
auto count = m_Items.count();
beginMoveRows(QModelIndex(), count, count, QModelIndex(), 2);
m_Items.insert(2, m_Items.last());
m_Items.pop_back();
endMoveRows();
}
protected:
QStringList m_Items;
};
static Model g_Model;
MainWindow::MainWindow(QWidget * parent):
QMainWindow(parent),
ui(new Ui::MainWindow)
{
// "ui" created by QtCreator, has a "tableView" and a "pushButton"
ui->setupUi(this);
ui->tableView->setModel(&g_Model);
ui->tableView->horizontalHeader()->resizeSection(0, 200);
ui->tableView->horizontalHeader()->resizeSection(1, 100);
ui->tableView->horizontalHeader()->resizeSection(2, 50);
ui->tableView->horizontalHeader()->resizeSection(3, 80);
connect(ui->pushButton, &QPushButton::clicked,
[this]()
{
// Outputs 200 on first click
qDebug() << ui->tableView->horizontalHeader()->sectionSize(0);
g_Model.moveNow();
// Outputs 100
qDebug() << ui->tableView->horizontalHeader()->sectionSize(0);
}
);
}

I found out that calling emit layoutAboutToBeChanged() just before the move operation fixes the issue:
void moveNow()
{
emit layoutAboutToBeChanged();
auto count = m_Items.count();
beginMoveRows(QModelIndex(), count, count, QModelIndex(), 2);
m_Items.insert(2, m_Items.last());
m_Items.pop_back();
endMoveRows();
}
This seems to contradict the documentation for beginMoveRows, which indicates that it is a replacement for calling this function. Also there must not be a matching emit layoutChanged() after endMoveRows, otherwise the columns are, again, reset.
This all together leads me to believe it's an actual bug in Qt.

Related

Context depending drag and drop in QListView

In one of my projects I have to manage a list of items, that can be rearranged in their orders by using drag and drop.
Now, all items came with a priority, that cannot be changed by the user. There is an restriction on the order of the elements in the list, namely that elements with a lower priority must come first, but elements with the same priority can be in interchanged.
For example, the following list is sane:
(A,1),(B,1),(C,1),(D,2),(E,3)
whereas the following is broken:
(A,1),(B,1),(E,3),(D,2)
The following code shows a starting point of my problem:
#include <QApplication>
#include <QFrame>
#include <QHBoxLayout>
#include <QListView>
#include <QStandardItemModel>
QStandardItem* create(const QString& text, int priority) {
auto ret = new QStandardItem(text);
ret->setData(priority);
return ret;
}
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
auto frame = new QFrame;
frame->setLayout(new QVBoxLayout);
auto view = new QListView;
frame->layout()->addWidget(view);
auto model = new QStandardItemModel;
view->setModel(model);
model->appendRow(create("1. A", 1));
model->appendRow(create("1. B", 1));
model->appendRow(create("2. X", 2));
model->appendRow(create("2. Y", 2));
model->appendRow(create("2. Z", 2));
view->setDragEnabled(true);
view->viewport()->setAcceptDrops(true);
view->setDropIndicatorShown(true);
view->setDragDropMode(QAbstractItemView::DragDropMode::InternalMove);
view->setDefaultDropAction(Qt::DropAction::MoveAction);
view->setDragDropOverwriteMode(false);
frame->show();
return a.exec();
}
Now, the DefaultDropAction must change context depended on the item going to be moved and also the item where it is going to be dropped.
If the priorities of the two elements are equal, then I have a MoveAction. In case the priorities of the two elements differ, I have a IgnoreAction.
Can this behavior be achieved without implementing my on QListView and what
can be achieved by adapting a custom QAbstractItemModel.
A possible workaround might be even to abandon the drag and drop interface and using the arrow up and down keys to move items around. Or even more general an action with cut and paste operation. But, I really prefer to stick with drag and drop interface.
You could reimplement QStandardItemModel and override the canDropMimeData() method. There are other ways, though they would probably be more involved if you're happy with QStandardItemModel already. Implementing your own model could have performance advantages, especially if your data structure is fairly simple (like a single-column list). This would also let you customize the drag/drop behavior much more in general.
Note that this ignores the action type entirely (QStandardItemModel only allows move and copy by default). Moving an item onto another item will remove the destination item entirely -- which may not be what you want but is a separate issue (see comments in code below).
You could also implement the same logic in dropMimeData() method (before calling the base class method), but I'm not sure I see any advantage. And by using canDropMimeData() the user also gets visual feedback about what is and isn't going to work.
#include <QStandardItemModel>
class ItemModel : public QStandardItemModel
{
public:
using QStandardItemModel::QStandardItemModel;
bool canDropMimeData(const QMimeData *data, Qt::DropAction action, int row, int column, const QModelIndex &parent) const override
{
if (!QStandardItemModel::canDropMimeData(data, action, row, column, parent))
return false;
const int role = Qt::UserRole + 1; // what QStandardItem uses for setData() by default
int originPriority;
int destPriority;
// Find destination item priority.
if (parent.isValid()) {
// dropping onto an item
// Note: if you don't want MoveAction to overwrite items you could:
// if (action == Qt::MoveAction) return false;
destPriority = parent.data(role).toInt();
}
else if (row > -1) {
// dropping between items
destPriority = this->data(index(row, 0), role).toInt();
}
else {
// dropping somewhere else onto the view, treat it as drop after last item in model
destPriority = this->data(index(rowCount() - 1, 0), role).toInt();
}
// Need to find priority of item(s) being dragged (encoded in mime data). Could be several.
// This part decodes the mime data in a way compatible with how QAbstractItemModel encoded it.
// (QStandardItemModel includes it in the mime data alongside its own version)
QByteArray ba = data->data(QAbstractItemModel::mimeTypes().first());
QDataStream ds(&ba, QIODevice::ReadOnly);
while (!ds.atEnd()) {
int r, c;
QMap<int, QVariant> v;
ds >> r >> c >> v;
// If there were multiple columns of data we could also do a
// check on the column number, for example.
originPriority = v.value(role).toInt();
if (originPriority != destPriority)
break; //return false; Could exit here but keep going to print our debug info.
}
qDebug() << "Drop parent:" << parent << "row:" << row <<
"destPriority:" << destPriority << "originPriority:" << originPriority;
if (originPriority != destPriority)
return false;
return true;
}
};
For reference, here's how QAbstractItemModel encodes data (and decodes it in the next method down).
ADDED:
OK, it was bugging me a bit, so here is a more efficient version... :-) It saves a lot of decoding time by embedding the dragged item's priority right into the mime data when the drag starts.
#include <QStandardItemModel>
#define PRIORITY_MIME_TYPE QStringLiteral("application/x-priority-data")
class ItemModel : public QStandardItemModel
{
public:
using QStandardItemModel::QStandardItemModel;
QMimeData *mimeData(const QModelIndexList &indexes) const override
{
QMimeData *mdata = QStandardItemModel::mimeData(indexes);
if (!mdata)
return nullptr;
// Add our own priority data for more efficient evaluation in canDropMimeData()
const int role = Qt::UserRole + 1; // data role for priority value
int priority = -1;
bool ok;
for (const QModelIndex &idx : indexes) {
// Priority of selected item
const int thisPriority = idx.data(role).toInt(&ok);
// When dragging multiple items, check that the priorities of all selected items are the same.
if (!ok || (priority > -1 && thisPriority != priority))
return nullptr; // Cannot drag items with different priorities;
priority = thisPriority;
}
if (priority < 0)
return nullptr; // couldn't find a priority, cancel the drag.
// Encode the priority data
QByteArray ba;
ba.setNum(priority);
mdata->setData(PRIORITY_MIME_TYPE, ba);
return mdata;
}
bool canDropMimeData(const QMimeData *data, Qt::DropAction action, int row, int column, const QModelIndex &parent) const override
{
if (!QStandardItemModel::canDropMimeData(data, action, row, column, parent))
return false;
if (!data->hasFormat(PRIORITY_MIME_TYPE))
return false;
const int role = Qt::UserRole + 1; // what QStandardItem uses for setData() by default
int destPriority = -1;
bool ok = false;
// Find destination item priority.
if (parent.isValid()) {
// dropping onto an item
destPriority = parent.data(role).toInt(&ok);
}
else if (row > -1) {
// dropping between items
destPriority = this->data(index(row, 0), role).toInt(&ok);
}
else {
// dropping somewhere else onto the view, treat it as drop after last item in model
destPriority = this->data(index(rowCount() - 1, 0), role).toInt(&ok);
}
if (!ok || destPriority < 0)
return false;
// Get priority of item(s) being dragged which we encoded in mimeData() method.
const int originPriority = data->data(PRIORITY_MIME_TYPE).toInt(&ok);
qDebug() << "Drop parent:" << parent << "row:" << row
<< "destPriority:" << destPriority << "originPriority:" << originPriority;
if (!ok || originPriority != destPriority)
return false;
return true;
}
};

QAbstractTableModel and emit dataChanged for a single row

I derived a model from QAbstractTableModel and now I want to notify, that the data of a whole row has been changed. If for example the data of a row with index 5 is changed (4 columns), than using the following code works as expected.
emit dataChanged(index(5,0), index(5, 0));
emit dataChanged(index(5,1), index(5, 1));
emit dataChanged(index(5,2), index(5, 2));
emit dataChanged(index(5,3), index(5, 3));
But if I try to achieve the same with only one emit, ALL columns of ALL rows in the view are updated.
emit dataChanged(index(5, 0), index(5, 3));
What I am doing wrong here?
Minimal example (C++11, QTCreator 4.7.1, Windows 10 (1803), 64 Bit)
demo.h
#pragma once
#include <QAbstractTableModel>
#include <QTime>
#include <QTimer>
class Demo : public QAbstractTableModel
{
Q_OBJECT
QTimer * t;
public:
Demo()
{
t = new QTimer(this);
t->setInterval(1000);
connect(t, SIGNAL(timeout()) , this, SLOT(timerHit()));
t->start();
}
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override
{
int c = index.column();
if (role == Qt::DisplayRole)
{
QString strTime = QTime::currentTime().toString();
if (c == 0) return "A" + strTime;
if (c == 1) return "B" + strTime;
if (c == 2) return "C" + strTime;
if (c == 3) return "D" + strTime;
}
return QVariant();
}
int rowCount(const QModelIndex &) const override { return 10; }
int columnCount(const QModelIndex &) const override { return 4; }
private slots:
void timerHit()
{
//Works
emit dataChanged(index(5,0), index(5, 0));
emit dataChanged(index(5,1), index(5, 1));
emit dataChanged(index(5,2), index(5, 2));
emit dataChanged(index(5,3), index(5, 3));
//emit dataChanged(index(5,0), index(5, 3)); // <-- Doesn't work
}
};
main.cpp
#include "demo.h"
#include <QApplication>
#include <QTreeView>
int main(int argc, char *argv[])
{
QApplication a(argc, argv);
QTreeView dataView;
Demo dataModel{};
dataView.setModel( &dataModel );
dataView.show();
return a.exec();
}
I think the problem lies with certain assumptions you're making with regard the behaviour of QTreeView when the QAbstractItemModel::dataChanged signal is emitted.
Specifically, you assume that the view will only invoke QAbstractItemModel::data on those indexes that are specified in the signal. That's not necessarily the case.
Looking at the source for QAbstractItemView::dataChanged (Qt 5.11.2) you'll see...
void QAbstractItemView::dataChanged(const QModelIndex &topLeft, const QModelIndex &bottomRight, const QVector<int> &roles)
{
Q_UNUSED(roles);
// Single item changed
Q_D(QAbstractItemView);
if (topLeft == bottomRight && topLeft.isValid()) {
const QEditorInfo &editorInfo = d->editorForIndex(topLeft);
//we don't update the edit data if it is static
if (!editorInfo.isStatic && editorInfo.widget) {
QAbstractItemDelegate *delegate = d->delegateForIndex(topLeft);
if (delegate) {
delegate->setEditorData(editorInfo.widget.data(), topLeft);
}
}
if (isVisible() && !d->delayedPendingLayout) {
// otherwise the items will be update later anyway
update(topLeft);
}
} else {
d->updateEditorData(topLeft, bottomRight);
if (isVisible() && !d->delayedPendingLayout)
d->viewport->update();
}
#ifndef QT_NO_ACCESSIBILITY
if (QAccessible::isActive()) {
QAccessibleTableModelChangeEvent accessibleEvent(this, QAccessibleTableModelChangeEvent::DataChanged);
accessibleEvent.setFirstRow(topLeft.row());
accessibleEvent.setFirstColumn(topLeft.column());
accessibleEvent.setLastRow(bottomRight.row());
accessibleEvent.setLastColumn(bottomRight.column());
QAccessible::updateAccessibility(&accessibleEvent);
}
#endif
d->updateGeometry();
}
The important point is that this code behaves differently depending on whether or not the signal specifies a single QModelIndex -- e.g. topLeft is the same as bottomRight. If they are the same then the view tries to ensure that only that model index is updated. However, if multiple model indexes are specified then it will invoke...
d->viewport->update();
which will, presumably, result in the data for all visible model indexes being queried.
Since your implementation of Demo::data always returns new data based on the current time you will see the entire visible part of the view update giving the impression that the dataChanged signal was emitted for all rows and columns.
So the fix is really to make your data model more ``stateful'' -- it needs to keep track of values rather than simply generating them on demand.
Not sure whether this is what you're looking for but I'll put it up anyways.
Even using emit dataChanged(...), you would still see that clicks/selection on rows will cause them to self-update (doing this from a Mac, so might be different).
Instead of using the QAbstractItemModel::dataChanged signal, I will be using the QAbstractItemModel::setData() function.
This is my implementation of demo.h
#pragma once
#include <QAbstractTableModel>
#include <QTime>
#include <QTimer>
class Demo : public QAbstractTableModel
{
Q_OBJECT
public:
Demo()
{
int cCount = columnCount(index(0, 0));
int rCount = rowCount(index(0, 0));
// populate model data with *static* values
QString strTime = QTime::currentTime().toString();
QStringList temp;
for (int j = 0; j < cCount; j++)
temp.append(strTime);
for (int i = 0; i < rCount; i++)
demoModelData.append(temp);
// nothing new here
t = new QTimer(this);
t->setInterval(1000);
connect(t, SIGNAL(timeout()) , this, SLOT(timerHit()));
t->start();
}
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override
{
// tells the *view* what to display
// if this was dynamic (e.g. like your original strTime implementation)
// then the view (QTreeView in main.cpp) will constantly update
if (role == Qt::DisplayRole)
return demoModelData.at(index.row()).at(index.column()); // retrieve data from model
return QVariant();
}
// reimplemented from QAbstractTableModel
virtual bool setData(const QModelIndex &index, const QVariant &value, int role = Qt::DisplayRole) override
{
if (role == Qt::DisplayRole)
{
demoModelData[index.row()][index.column()] = value.toString(); // set the new data
emit dataChanged(index, index); // explicitly emit dataChanged signal, notifies TreeView to update by
// calling this->data(index, Qt::DisplayRole)
}
return true;
}
int rowCount(const QModelIndex &) const override { return 10; }
int columnCount(const QModelIndex &) const override { return 4; }
private slots:
void timerHit()
{
QString strTime = QTime::currentTime().toString();
setData(index(5, 0), QVariant(strTime), Qt::DisplayRole); // only changes index at (row = 5, col = 0)
}
private:
QTimer *t;
QList<QStringList> demoModelData; // stores the table model's data
};
Since the class is a "model", there should be some way of storing/retrieving data for display. Here, I've used a QList<QStringList>, but you can store data in other ways that suit you as well (e.g. a tree, QVector, QMap).
This is a genuine efficiency bug in Qt.
The Qt project is no longer accepting changes to Qt 5, so I made the change and pushed it to my fork on GitHub. You can see a fix for the issue you've encountered here.
If you want to build your own copy of 5.15.2, you may be interested in my other fixes.

Algorithm for splitting large amount of text data

I have a text area which i populate dynamically(to be specific I have a QPlainTextEdit in Qt, but its not important for algorithm suggestion).
Now problem is sometimes Large amounts of data comes and as more data comes in my application becomes heavy,since all the text data is in main memory.
So I thought of the following. We can use a file for storing all the text data and display only limited amount of data dynamically, but at the same time I have to illusion the user that the data size is that of the file, by creating scroll events that trigger when new lines comes.
Is there any standard algorithm for such problem?
Subclass QAbstractListModel implement cache there.
When cell value is read you are fetching data from cache and update it if value is not present in cache.
Tweak QTableView, by altering delegate to achieve needed visualization of cells. Note you have to use QTableView since other QAbstractItemViews have broken items recycling and they don't handle very large models well (QTableView doesn't have such issue).
Some time ego I've wrote hex viewer of large files and tested that with file size 2GB and it was working perfectly.
Ok, I found my old code which could be a good example:
#include <QAbstractTableModel>
class LargeFileCache;
class LageFileDataModel : public QAbstractTableModel
{
Q_OBJECT
public:
explicit LageFileDataModel(QObject *parent);
// QAbstractTableModel
int rowCount(const QModelIndex &parent) const;
int columnCount(const QModelIndex &parent) const;
QVariant data(const QModelIndex &index, int role) const;
signals:
public slots:
void setFileName(const QString &fileName);
private:
LargeFileCache *cachedData;
};
// ----- cpp file -----
#include "lagefiledatamodel.h"
#include "largefilecache.h"
#include <QSize>
static const int kBytesPerRow = 16;
LageFileDataModel::LageFileDataModel(QObject *parent)
: QAbstractTableModel(parent)
{
cachedData = new LargeFileCache(this);
}
int LageFileDataModel::rowCount(const QModelIndex &parent) const
{
if (parent.isValid())
return 0;
return (cachedData->FileSize() + kBytesPerRow - 1)/kBytesPerRow;
}
int LageFileDataModel::columnCount(const QModelIndex &parent) const
{
if (parent.isValid())
return 0;
return kBytesPerRow;
}
QVariant LageFileDataModel::data(const QModelIndex &index, int role) const
{
if (index.parent().isValid())
return QVariant();
if (index.isValid()) {
if (role == Qt::DisplayRole) {
qint64 pos = index.row()*kBytesPerRow + index.column();
if (pos>=cachedData->FileSize())
return QString();
return QString("%1").arg((unsigned char)cachedData->geByte(pos), 2, 0x10, QChar('0'));
} else if (role == Qt::SizeHintRole) {
return QSize(30, 30);
}
}
return QVariant();
}
void LageFileDataModel::setFileName(const QString &fileName)
{
beginResetModel();
cachedData->SetFileName(fileName);
endResetModel();
}
Here is a cache implementation:
class LargeFileCache : public QObject
{
Q_OBJECT
public:
explicit LargeFileCache(QObject *parent = 0);
char geByte(qint64 pos);
qint64 FileSize() const;
signals:
public slots:
void SetFileName(const QString& filename);
private:
static const int kPageSize;
struct Page {
qint64 offset;
QByteArray data;
};
private:
int maxPageCount;
qint64 fileSize;
QFile file;
QQueue<Page> pages;
};
// ----- cpp file -----
#include "largefilecache.h"
const int LargeFileCache::kPageSize = 1024*4;
LargeFileCache::LargeFileCache(QObject *parent)
: QObject(parent)
, maxPageCount(1024)
, fileSize(0)
{
}
char LargeFileCache::geByte(qint64 pos)
{
// largefilecache
if (pos>=fileSize)
return 0;
for (int i=0, n=pages.size(); i<n; ++i) {
int k = pos - pages.at(i).offset;
if (k>=0 && k< pages.at(i).data.size()) {
pages.enqueue(pages.takeAt(i));
return pages.back().data.at(k);
}
}
Page newPage;
newPage.offset = (pos/kPageSize)*kPageSize;
file.seek(newPage.offset);
newPage.data = file.read(kPageSize);
pages.push_front(newPage);
while (pages.count()>maxPageCount)
pages.dequeue();
return newPage.data.at(pos - newPage.offset);
}
qint64 LargeFileCache::FileSize() const
{
return fileSize;
}
void LargeFileCache::SetFileName(const QString &filename)
{
file.close();
pages.clear();
file.setFileName(filename);
file.open(QFile::ReadOnly);
fileSize = file.size();
}
I wrote cache manually since I was handling a row data, but you can use QCache which should help you do a caching logic.
Using mmap only addresses how you may read the file while only having pieces of it in memory. It doesn't address how the edit control would only have pieces at a time.
I have to think that any such system would be fairly specific to the text editing widget involved. In this case, you would either need to figure out how to extend QPlainTextEdit with the desired functionality, or make a new text editing widget (possible forking an existing one). There are numerous text editing widgets available as open source that could be used as a starting point.
I've been assuming so far that you want to edit the text in this large file. If you are only using QPlainTextEdit as a read-only viewer, then that writing your own that is only a large text stream reading widget may be much easier than extending an existing editor widget.
It's my two cents,
When I googled the similar question, I've found the answer from Fast textfile reading in c++
In short, Memory mapped files in the boost Lib. might be helpful not only for performance but also for handling a large amount of data.
The sample in the link, I could check the number of lines and getting data from Lib.
Good Luck

C++/QML: ListView is not updated on dataChanged signal from QAbstractListModel

I am trying to write a QML Gui for a large dynamic C/Fortran simulation. The data I want to display is stored in Fortran Common blocks and updated on fixed time steps. My problem is that QML ListView does not refresh when the dataChanged signal is emitted after each time step, although the signal is received by the Gui (test is in the code below).
I am probably missing out something really obvious because when I flick my ListView down and up again, the displayed data is updated and correct (I guess because the QML engine re-renders elements when they get "out of sight" and back in again). So the only thing that does not work is that the ListView gets updated every time the dataChanged signal is received and not only when it is re-rendered. Below is a more detailed description of my approach and the relevant code parts.
Each simulation entity has several attributes (alive, position...), so I decided to create a ListModel containing a DataObject for each entity. This is the corresponding header file (the actual simulation data is declared as extern structs in "interface.h", so I can access it via pointer):
"acdata.h"
#include <QtCore>
#include <QObject>
#include <QtGui>
extern "C" {
#include "interface.h"
}
class AcDataObject : public QObject
{
Q_OBJECT
public:
explicit AcDataObject(int id_, int *pac_live, double *pac_pos_x, QObject *parent = 0) :
QObject(parent)
{
entity_id = id_;
ac_live = pac_live;
ac_pos_x = pac_pos_x;
}
int entity_id;
int *ac_live;
double *ac_pos_x;
};
class AcDataModel : public QAbstractListModel
{
Q_OBJECT
public:
enum RoleNames {
IdRole = Qt::UserRole,
LiveRole = Qt::UserRole + 1,
PosXRole = Qt::UserRole + 2
};
explicit AcDataModel(QObject *parent = 0);
virtual int rowCount(const QModelIndex &parent) const;
virtual QVariant data(const QModelIndex &index, int role) const;
Q_INVOKABLE Qt::ItemFlags flags(const QModelIndex &index) const Q_DECL_OVERRIDE;
void do_update();
protected:
virtual QHash<int, QByteArray> roleNames() const;
private:
QList<AcDataObject*> data_list;
QHash<int, QByteArray> m_roleNames;
QModelIndex start_index;
QModelIndex end_index;
signals:
void dataChanged(const QModelIndex &start_index, const QModelIndex &end_index);
};
Like the header, the .cpp file is also adapted from what you can find in the Qt5 Cadaques Book here, except that my constructor iterates over all simulation entities to set the pointers. Additionally, there is the do_update function that emits the dataChanged signal for the whole list.
"acdata.cpp"
#include "acdata.h"
AcDataModel::AcDataModel(QObject *parent) :
QAbstractListModel(parent)
{
m_roleNames[IdRole] = "entity_id";
m_roleNames[LiveRole] = "ac_live";
m_roleNames[PosXRole] = "ac_pos_x";
for (int i = 0; i < MAX_ENTITIES; i++) // MAX_ENTITIES is defined in interface.h
{
AcDataObject *data_object = new AcDataObject( i,
&fdata_ac_.ac_live[i], // fdata_ac_ is the C struct/Fortran common block defined in interface.h
&fdata_ac_.ac_pos_x[i] );
data_list.append(data_object);
}
}
int AcDataModel::rowCount(const QModelIndex &parent) const {
Q_UNUSED(parent);
return data_list.count();
}
QVariant AcDataModel::data(const QModelIndex &index, int role) const
{
int row = index.row();
if(row < 0 || row >= data_list.count()) {
return QVariant();
}
const AcDataObject *data_object = data_list.at(row);
switch(role) {
case IdRole: return data_object->entity_id;
case LiveRole: return *(data_object->ac_live);
case PosXRole: return *(data_object->ac_pos_x);
}
return QVariant();
}
QHash<int, QByteArray> AcDataModel::roleNames() const
{
return m_roleNames;
}
void AcDataModel::do_update() {
start_index = createIndex(0, 0);
end_index = createIndex((data_list.count() - 1), 0);
dataChanged(start_index, end_index);
}
Qt::ItemFlags AcDataModel::flags(const QModelIndex &index) const
{
if (!index.isValid()) {return 0;}
return Qt::ItemIsEditable | QAbstractItemModel::flags(index);
}
When the simulation is running, do_update() is called every second. I have created a test Gui with a ListView and exposed my model to it with:
Excerpt from "threadcontrol.cpp"
acdata = new AcDataModel();
viewer = new QtQuick2ApplicationViewer();
viewer->rootContext()->setContextProperty("acdata", acdata);
viewer->setMainQmlFile(QStringLiteral("../lib/qml_gui/main.qml"));
viewer->showExpanded();
(This code is part of a larger file that controls the different threads. I am quite sure the rest is not relevant to the actual problem and this question is getting really long...)
So finally there is main.qml. It contains a list with MAX_ENTITIES elements and each elements holds text fields to display my data. I have also added a Connections element to check if the dataChanged signal is received by the Gui.
"main.qml"
ListView {
id: listviewer
model: acdata
delegate: Rectangle {
/* ... some formatting stuff like height etc ... */
Row {
anchors.fill: parent
Text {
/* ... formatting stuff ... */
text: model.entity_id
}
Text {
/* ... formatting stuff ... */
text: model.ac_live
}
Text {
/* ... formatting stuff ... */
text: model.ac_pos_x
}
}
}
Connections {
target: listviewer.model // EDIT: I drew the wrong conclusions here, see text below!
onDataChanged: {
console.log("DataChanged received")
}
}
}
When running the simulation, the "DataChanged received" message is printed every second.
Edit: I was connecting to the ListModel and not to the ListView here, although the ListView has to receive the dataChanged signal. As the console log does not work when connecting to listviewer, I am probably missing the connection between listView and dataChanged signal. However, I think this should work automatically when implementing the dataChanged signal?
Additional information: I have found a similar problem here with Qt Map and it actually seemed to be a bug that was fixed in Qt 5.6. However, running qmake with Qt 5.7 did not fix my problem.
You mustn't declare the dataChanged() signal in your class, because you want to emit the signal AbstractItemModel::dataChanged(). If you re-declare it you add a comleptely new and different Signal that is not connected anywhere. If you remove the declaration in acdata.h everything should work fine.

QListView with millions of items slow with keyboard

I'm using a QListView with a custom model derived from QAbstractItemModel. I have on the order of millions of items. I have called listView->setUniformItemSizes(true) to prevent a bunch of layout logic from being called when I'm adding items to the model. So far, everything works as expected.
The problem is that using the keyboard to navigate the list is slow. If I select an item in the list, then press up/down, the selection moves fast until the selection needs to scroll the list. Then it becomes extremely laggy. Pressing page-up or page-down is also very laggy. The problem seems to be when an item is selected (aka the "current item") with the keyboard and the list is also scrolled up/down.
If I use the mouse, navigating the list is fast. I can use the mouse wheel, which is fast. I can drag the scroll bar up/down as fast as I want--from the top of the list to the bottom--and the list view updates wickedly fast.
Any ideas on why the combination of changing selections and scrolling the list is so slow? Is there a viable work-around?
Update 9/9/15
In order to better illustrate the issue, I'm providing amplifying information in this update.
Performance Issues with KEYBOARD + SCROLLING
This is mostly a performance question, although it does tie in with the user experience (UX) somewhat. Check out what happens as I use the keyboard to scroll through a QListView:
Notice the slow-down near the bottom? This is the focal point of my question. Let me explain how I am navigating the list.
Explanation:
Starting at the top, the first item in the list is selected.
Pressing and holding the down arrow key, the current item (selection) is changed to the next item.
Changing selection is fast for all of the items that are currently in view.
As soon as the list needs to bring the next item into view, the selection rate slows down significantly.
I expect that the list should be able to scroll as fast as the typematic rate of my keyboard--in other words, the time it takes to select the next item should not slow down when the list is scrolled.
Fast Scrolling with MOUSE
Here's what it looks like when I use the mouse:
Explanation:
Using the mouse, I select the scroll bar handle.
Quickly dragging the scroll bar handle up and down, the list is scrolled accordingly.
All movements are extremely fast.
Note that no selections are being made.
This proves two main points:
The model is not the problem. As you can see, the model has no problem whatsoever performance-wise. It can deliver the elements faster than they can be displayed.
Performance is degraded when selecting AND scrolling. The "perfect storm" of selecting and scrolling (as illustrated by using the keyboard to navigate through the list) causes the slowdown. As a result, I surmise that Qt is somehow doing a lot of processing when selections are being made during scrolling that aren't normally performed.
Non-Qt Implementation is FAST
I want to point out that my issue seems to be specific to Qt.
I have already implemented this type of thing before using a different framework. What I am trying to do is within the scope of model-view theory. I can do exactly what I am describing at blazing fast speeds using juce::ListBoxModel with a juce::ListBox. It's stupid fast (plus, there's no need to create a duplicate index such as a QModelIndex for every single item when each item already has a unique index). I get that Qt needs a QModelIndex for each item for its model-view architecture, and although I don't like the overhead cost, I think I get the rational and I can live with it. Either way, I don't suspect that these QModelIndexes are what is causing my performance slow-down.
With a JUCE implementation, I can even use the page-up & page-down keys to navigate the list, and it just blazes through the list. Using the Qt QListView implementation, it chugs along and is laggy, even with a release build.
A model-view implementation using the JUCE framework is extremely fast. Why is the Qt QListView implementation such a dog?!
Motivating Example
Is it hard to imagine why you'd need so many items in a list view? Well, we've all seen this kind of thing before:
This is the Visual Studio Help Viewer index. Now, I haven't counted all of the items--but I think we'd agree that there are a lot of them! Of course to make this list "useful," they added a filter box that narrows down what is in the list view according to an input string. There aren't any tricks here. It's all practical, real-world stuff we've all seen for decades in desktop applications.
But are there millions of items? I'm not sure it matters. Even if there were "only" 150k items (which is roughly accurate based on some crude measurements), it's easy to point out that you have to do something to make it useable--which is what a filter will do for you.
My specific example uses a list of German words as a plain text file with slightly more than 1.7 million entries (including inflected forms). This is probably only a partial (but still significant) sample of words from the German text corpus that was used to assemble this list. For linguistic study, this is a reasonable use case.
Concerns about improving the UX (user experience) or filtering are great design goals, but they are out of the scope of this question (I'll certainly address them later in the project).
Code
Want a code example? You got it! I'm not sure how useful it will be; it's as vanilla as it gets (about 75% boilerplate), but I suppose it will provide some context. I realize that I'm using a QStringList and that there is a QStringListModel for this, but the QStringList that I'm using to hold the data is a placeholder--the model will eventually be somewhat more complicated, so in the end, I need to use a custom model derived from QAbstractItemModel.
//
// wordlistmodel.h ///////////////////////////////////////
//
class WordListModel : public QAbstractItemModel
{
Q_OBJECT
public:
WordListModel(QObject* parent = 0);
virtual QModelIndex index(int row, int column, const QModelIndex& parent = QModelIndex()) const;
virtual QModelIndex parent(const QModelIndex& index) const;
virtual int rowCount(const QModelIndex& parent = QModelIndex()) const;
virtual int columnCount(const QModelIndex & parent = QModelIndex()) const;
virtual QVariant data(const QModelIndex& index, int role = Qt::DisplayRole) const;
public slots:
void loadWords();
signals:
void wordAdded();
private:
// TODO: this is a temp backing store for the data
QStringList wordList;
};
//
// wordlistmodel.cpp ///////////////////////////////////////
//
WordListModel::WordListModel(QObject* parent) :
QAbstractItemModel(parent)
{
wordList.reserve(1605572 + 50); // testing purposes only!
}
void WordListModel::loadWords()
{
// load items from file or database
// Due to taking Kuba Ober's advice to call setUniformItemSizes(true),
// loading is fast. I'm not using a background thread to do
// loading because I was trying to visually benchmark loading speed.
// Besides, I am going to use a completely different method using
// an in-memory file or a database, so optimizing this loading by
// putting it in a background thread would obfuscate things.
// Loading isn't a problem or the point of my question; it takes
// less than a second to load all 1.6 million items.
QFile file("german.dic");
if (!file.exists() || !file.open(QIODevice::ReadOnly))
{
QMessageBox::critical(
0,
QString("File error"),
"Unable to open " + file.fileName() + ". Make sure it can be located in " +
QDir::currentPath()
);
}
else
{
QTextStream stream(&file);
int numRowsBefore = wordList.size();
int row = 0;
while (!stream.atEnd())
{
// This works for testing, but it's not optimal.
// My real solution will use a completely different
// backing store (memory mapped file or database),
// so I'm not going to put the gory details here.
wordList.append(stream.readLine());
++row;
if (row % 10000 == 0)
{
// visual benchmark to see how fast items
// can be loaded. Don't do this in real code;
// this is a hack. I know.
emit wordAdded();
QApplication::processEvents();
}
}
if (row > 0)
{
// update final word count
emit wordAdded();
QApplication::processEvents();
// It's dumb that I need to know how many items I
// am adding *before* calling beginInsertRows().
// So my begin/end block is empty because I don't know
// in advance how many items I have, and I don't want
// to pre-process the list just to count the number
// of items. But, this gets the job done.
beginInsertRows(QModelIndex(), numRowsBefore, numRowsBefore + row - 1);
endInsertRows();
}
}
}
QModelIndex WordListModel::index(int row, int column, const QModelIndex& parent) const
{
if (row < 0 || column < 0)
return QModelIndex();
else
return createIndex(row, column);
}
QModelIndex WordListModel::parent(const QModelIndex& index) const
{
return QModelIndex(); // this is used as the parent index
}
int WordListModel::rowCount(const QModelIndex& parent) const
{
return wordList.size();
}
int WordListModel::columnCount(const QModelIndex& parent) const
{
return 1; // it's a list
}
QVariant WordListModel::data(const QModelIndex& index, int role) const
{
if (!index.isValid())
{
return QVariant();
}
else if (role == Qt::DisplayRole)
{
return wordList.at(index.row());
}
else
{
return QVariant();
}
}
//
// mainwindow.h ///////////////////////////////////////
//
class MainWindow : public QMainWindow
{
Q_OBJECT
public:
explicit MainWindow(QWidget *parent = 0);
~MainWindow();
public slots:
void updateWordCount();
private:
Ui::MainWindow *ui;
WordListModel* wordListModel;
};
//
// mainwindow.cpp ///////////////////////////////////////
//
MainWindow::MainWindow(QWidget *parent) :
QMainWindow(parent),
ui(new Ui::MainWindow)
{
ui->setupUi(this);
ui->listView->setModel(wordListModel = new WordListModel(this));
// this saves TONS of time during loading,
// but selecting/scrolling performance wasn't improved
ui->listView->setUniformItemSizes(true);
// these didn't help selecting/scrolling performance...
//ui->listView->setLayoutMode(QListView::Batched);
//ui->listView->setBatchSize(100);
connect(
ui->pushButtonLoadWords,
SIGNAL(clicked(bool)),
wordListModel,
SLOT(loadWords())
);
connect(
wordListModel,
SIGNAL(wordAdded()),
this,
SLOT(updateWordCount())
);
}
MainWindow::~MainWindow()
{
delete ui;
}
void MainWindow::updateWordCount()
{
QString wordCount;
wordCount.setNum(wordListModel->rowCount());
ui->labelNumWordsLoaded->setText(wordCount);
}
As noted, I've already reviewed and taken Kuba Ober's advice:
QListView takes too long to update when given 100k items
My question is not a duplicate of that question! In the other question, the OP was asking about loading speed, which as I've noted in my code above, is not a problem due to the call to setUniformItemSizes(true).
Summary Questions
Why is navigating a QListView (with millions of items in the model) using the keyboard so slow when the list is scrolled?
Why does the combination of selecting and scrolling items cause a slow-down?
Are there any implementation details that I am missing, or have I reached a performance threshold for QListView?
1. Why is navigating a QListView (with millions of items in the model)
using the keyboard so slow when the list is scrolled?
Because when you navigate through your list using the keyboard, you enter the internal Qt function QListModeViewBase::perItemScrollToValue, see stack:
Qt5Widgetsd.dll!QListModeViewBase::perItemScrollToValue(int index, int scrollValue, int viewportSize, QAbstractItemView::ScrollHint hint, Qt::Orientation orientation, bool wrap, int itemExtent) Ligne 2623 C++
Qt5Widgetsd.dll!QListModeViewBase::verticalScrollToValue(int index, QAbstractItemView::ScrollHint hint, bool above, bool below, const QRect & area, const QRect & rect) Ligne 2205 C++
Qt5Widgetsd.dll!QListViewPrivate::verticalScrollToValue(const QModelIndex & index, const QRect & rect, QAbstractItemView::ScrollHint hint) Ligne 603 C++
Qt5Widgetsd.dll!QListView::scrollTo(const QModelIndex & index, QAbstractItemView::ScrollHint hint) Ligne 575 C++
Qt5Widgetsd.dll!QAbstractItemView::currentChanged(const QModelIndex & current, const QModelIndex & previous) Ligne 3574 C++
Qt5Widgetsd.dll!QListView::currentChanged(const QModelIndex & current, const QModelIndex & previous) Ligne 3234 C++
Qt5Widgetsd.dll!QAbstractItemView::qt_static_metacall(QObject * _o, QMetaObject::Call _c, int _id, void * * _a) Ligne 414 C++
Qt5Cored.dll!QMetaObject::activate(QObject * sender, int signalOffset, int local_signal_index, void * * argv) Ligne 3732 C++
Qt5Cored.dll!QMetaObject::activate(QObject * sender, const QMetaObject * m, int local_signal_index, void * * argv) Ligne 3596 C++
Qt5Cored.dll!QItemSelectionModel::currentChanged(const QModelIndex & _t1, const QModelIndex & _t2) Ligne 489 C++
Qt5Cored.dll!QItemSelectionModel::setCurrentIndex(const QModelIndex & index, QFlags<enum QItemSelectionModel::SelectionFlag> command) Ligne 1373 C++
And this function does:
itemExtent += spacing();
QVector<int> visibleFlowPositions;
visibleFlowPositions.reserve(flowPositions.count() - 1);
for (int i = 0; i < flowPositions.count() - 1; i++) { // flowPositions count is +1 larger than actual row count
if (!isHidden(i))
visibleFlowPositions.append(flowPositions.at(i));
}
Where flowPositions contains as many items as your QListView, so this basically iterates through all your items, and this will definitely take a while to process.
2. Why does the combination of selecting and scrolling items cause a slow-down?
Because "selecting and scrolling" makes Qt call QListView::scrollTo (to scroll the view to a specific item) and this is what ends up calling QListModeViewBase::perItemScrollToValue. When you scroll using the scroll bar, the system does not need to ask the view to scroll to a specific item.
3. Are there any implementation details that I am missing, or have I reached a performance threshold for QListView?
I'm afraid you are doing the things right. This is definitely a Qt bug. A bug report must be done to hope having this fixed in later releases. I submitted a Qt bug here.
As this code is internal (private data classes) and not conditionnal to any QListView setting, I see no way to fix it except by modifying and recompiling the Qt source code (but I don't know exactly how, this would require more investigation). The first function overidable in the stack is QListView::scrollTo but I doubt it would be easy to oevrride it without calling QListViewPrivate::verticalScrollToValue...
Note: The fact that this function goes through all items of the view was apparently introduced in Qt 4.8.3 when this bug was fixed (see changes). Basically, if you don't hide any items in your view, you could modify Qt code as below:
/*QVector<int> visibleFlowPositions;
visibleFlowPositions.reserve(flowPositions.count() - 1);
for (int i = 0; i < flowPositions.count() - 1; i++) { // flowPositions count is +1 larger than actual row count
if (!isHidden(i))
visibleFlowPositions.append(flowPositions.at(i));
}*/
QVector<int>& visibleFlowPositions = flowPositions;
Then you'll have to recompile Qt and I'm pretty sure this will fix the issue (not tested however). But then you'll see new problems if you one day hide some items...to support filtering for instance!
Most likely the right fix would have been to have the view maintain both flowPositions and visibleFlowPositions to avoid creating it on the fly...
I have made the following test:
First of all i create a class to check in the calls:
struct Test
{
static void NewCall( QString function, int row )
{
function += QString::number( row );
map[ function ]++;
}
static void Summary( )
{
qDebug() << "-----";
int total = 0;
QString data;
for( auto pair : map )
{
data = pair.first + ": " + QString::number( pair.second );
total += pair.second;
qDebug( ) << data;
}
data = "total: " + QString::number( total ) + " calls";
qDebug() << data;
map.clear();
}
static std::map< QString, int > map;
};
std::map<QString,int> Test::map;
Then I insert a call to NewCall in index, parent and data methods of WordListModel. Finally i add a QPushButton in the dialog, the clicked signal is linked to a method which call to Test::Summary.
The steps of the test are the next:
Select the last showed item of the list
Press the Summary button to clear the calling list
With tab key select the list view again
Perform a scroll with the direction keys
Press Summary button again
The printed list shows the problem. QListView widget makes a big number of calls. It seems the widget is reloading all the data from the model.
I don't know if it can be improved but you can't do anything but filter the list to limit the number of items to show.
Unfortunately, I believe that you can't do much about this.
We don't have much control over widgets.
Although you can avoid that issue by using ListView instead.
If you try my quick example below you'll notice how fast it can be even using delegates which is costly.
Here is the example:
Window{
visible: true
width: 200
height: 300
property int i: 0;
Timer {
interval: 5
repeat: true
running: true
onTriggered: {
i += 1
lv.positionViewAtIndex(i, ListView.Beginning)
}
}
ListView {
id:lv
anchors.fill: parent
model: 1605572
delegate: Row {
Text { text: index; width: 300; }
}
}
}
I put a Timer to simulate the scrolling, but of course you can turn on or off that timer depending on whether keys are pressed as well as changing i += 1 by i += -1 if ▲ is pressed instead of ▼. You'd have to add overflow and underflow checks too.
You can also choose the scrolling speed by changing interval of Timer. Then it's just a matter of modifying the selected element's color etc. to show it's selected.
On top of which you can use cacheBuffer with ListView to cache more elements but I don't think it is necessary.
If you want to use QListView anyway take a look at this example: http://doc.qt.io/qt-5/qtwidgets-itemviews-fetchmore-example.html
Using the fetch method allow to keep performance even with big datasets. It allows you to fill the list as you scroll.