Qt 4.8 endInsert/RemoveRows cause memory leak? - c++

Recently I've made some stress tests on my Qt 4.8 application. I've seen using valgrind massif tool, that it cause heap memory expansion...
Using that tool I found out this memory is allocated with this stack trace (this is other massif call, so values vary from the one on screenshot):
->02.11% (1,133,952B) 0x221FD9EB: ??? (in /usr/lib/x86_64-linux-gnu/qt4/plugins/accessible/libqtaccessiblewidgets.so)
| ->02.11% (1,133,952B) 0x80ADE69: QAccessible::queryAccessibleInterface(QObject*) (in /usr/lib/x86_64-linux-gnu/libQtGui.so.4.8.7)
| ->01.18% (637,824B) 0x80B5156: ??? (in /usr/lib/x86_64-linux-gnu/libQtGui.so.4.8.7)
| | ->01.18% (637,824B) 0x8BA0F6E: QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (in /usr/lib/x86_64-linux-gnu/libQtCore.so.4.8.7)
| | ->01.18% (637,824B) 0x8BF1472: QAbstractItemModel::rowsInserted(QModelIndex const&, int, int) (in /usr/lib/x86_64-linux-gnu/libQtCore.so.4.8.7)
| | ->01.18% (637,824B) 0x8B86510: QAbstractItemModel::endInsertRows() (in /usr/lib/x86_64-linux-gnu/libQtCore.so.4.8.7)
| | ->01.18% (637,824B) in 5 places, all below massif's threshold (1.00%)
The cause of expansions are endInsertRows and endRemoveRows functions. My ModelView implementation looks like this:
void TrainScheduleModelView::addTrain(const model::object::Train &train)
{
if (this->m_rows == TrainScheduleModelView::MAX_TRAIN_SCHEDULE_SIZE)
{
beginRemoveRows(QModelIndex(),
TrainScheduleModelView::MAX_TRAIN_SCHEDULE_SIZE - 1,
TrainScheduleModelView::MAX_TRAIN_SCHEDULE_SIZE - 1);
endRemoveRows();
}
beginInsertRows(QModelIndex(), 0, 0);
this->m_trains[this->m_head].second = train;
this->m_trains[this->m_head].first = true;
if (0 == this->m_head)
{
this->m_head = TrainScheduleModelView::MAX_TRAIN_SCHEDULE_SIZE - 1;
}
else
{
--(this->m_head);
}
if (this->m_rows < TrainScheduleModelView::MAX_TRAIN_SCHEDULE_SIZE)
{
++(this->m_rows);
}
endInsertRows();
}
This model is based on C-arrays intentionaly, every time the new object is added, last one is truncated.
Could anyone tell me is there a bug, or I'm using it wrong?

I had two attempts to solve it:
First of all I tried to eliminate Accesibility module - I hoped it is not necessary. I had to build Qt from source with ./configure ... -no-accessibility... option. The case was resolved, but UI of the application dramatically suffered. I was not able to enter password, because keyboard was blocked. So this solution was not acceptable.
Next workaround was elimination of dynamic row insertion. I've changed approach and instead of emulating row insertion, I return always fixed number of rows (empty rows were displayed, however there were no data inside). When I put some data inside the table I emit dataChanged signal with all rows affected (because each one was shifted). This solution eliminated using queryAccessibleInterface function and there were no leak. So the code looked like this:
void TrainScheduleModelView::addTrain(const model::object::Train &train)
{
this->m_trains[this->m_head].second = train;
this->m_trains[this->m_head].first = true;
if (0 == this->m_head)
{
this->m_head = TrainScheduleModelView::MAX_TRAIN_SCHEDULE_SIZE - 1;
}
else
{
--(this->m_head);
}
if (this->m_rows < TrainScheduleModelView::MAX_TRAIN_SCHEDULE_SIZE)
{
++(this->m_rows);
}
emit dataChanged(this->index(0, 0),
this->index(TrainScheduleModelView::MAX_TRAIN_SCHEDULE_SIZE-1,
TrainScheduleModelView::COLUMNS_AMOUNT - 1));
}
The case is not completely solved. I suppose there might be a problem with porting Qt on Ubuntu or init code of application.
UPDATE
I found leak source. queryAccessibleInterface function returns allocated with new pointer and code loose reference to it.
void QAbstractItemViewPrivate::_q_layoutChanged()
{
doDelayedItemsLayout();
#ifndef QT_NO_ACCESSIBILITY
#ifdef Q_WS_X11
Q_Q(QAbstractItemView);
if (QAccessible::isActive()) {
QAccessible::queryAccessibleInterface(q)->table2Interface()->modelReset();
QAccessible::updateAccessibility(q, 0, QAccessible::TableModelChanged);
}
#endif
#endif
}

Related

How do I appropriately call constructors and pass objects around?

I have the following code, but something is wrong with it. It compiles, but nothing happens on the NeoPixel matrix. I've loaded up a strand test to verify that the hardware is working properly. Through some manual debugging, I've worked out that the line containing PixelArt art = PixelArt(matrix); triggers the issue. If I comment that out, I get a panel of orange light coming out of the light matrix as expected. Putting it back in simply results in darkness.
What about including that one line is causing issues? I'm new to C++ and this is driving me up the wall.
For some code context, I plan on including more modes, each with a series of submodes. PixelArt would show off different pixel art pictures as its sub modes. Other modes would include different animation patterns. The different modes should all share a reference to the same matrix object.
#include <Adafruit_NeoMatrix.h>
#include <gamma.h>
#define MODE_PIN 2
#define SUB_MODE_PIN 3
#define MATRIX_PIN 9
#define MATRIX_COLUMNS 16
#define MATRIX_ROWS 16
#define MATRIX_NUM_PIXELS (MATRIX_COLUMNS * MATRIX_ROWS)
class Mode {
public:
Mode(Adafruit_NeoMatrix &neomatrix) : matrix{&neomatrix} {}
virtual void toggleSubMode() = 0;
virtual void update(uint64_t timestamp) = 0;
protected:
Adafruit_NeoMatrix * const matrix;
};
class PixelArt : public Mode {
public:
PixelArt(Adafruit_NeoMatrix &neomatrix):Mode(neomatrix) {}
void toggleSubMode();
void update(uint64_t timestamp);
};
void PixelArt::toggleSubMode() {
// Stubbed.
}
// Green: 0x80c000
// Brown: 0xc04000
// Tan: 0xffa040
uint32_t link[16][16] = { [REDACTED FOR BREVITY] };
void PixelArt::update(uint64_t timestamp) {
matrix->clear();
for (uint16_t row = 0; row < 16; row++) {
for (uint16_t column = 0; column < 16; column++) {
matrix->setPassThruColor(link[row][column]);
matrix->drawPixel(column, row, 0);
matrix->setPassThruColor();
}
}
matrix->show();
}
//
Adafruit_NeoMatrix matrix = Adafruit_NeoMatrix(
MATRIX_COLUMNS,
MATRIX_ROWS,
MATRIX_PIN,
NEO_MATRIX_TOP | NEO_MATRIX_LEFT | NEO_MATRIX_COLUMNS | NEO_MATRIX_ZIGZAG,
NEO_GRB | NEO_KHZ800
);
PixelArt art = PixelArt(matrix);
void setup() {
matrix.begin();
matrix.show();
matrix.setBrightness(30); // Max ~80
}
uint8_t mode = 0;
void loop() {
// art.update(millis());
matrix.fillScreen(matrix.Color(255, 127, 0));
matrix.show();
}
My suggestion would be that because the code runs on a microcontroller, it may run out of memory. Especially if the target board is an Arduino Uno or similar, with small dynamic memory. After compilation, the memory usage calculation is done for compile-time variables. The run-time variables and function calls may exceed the available memory.
Compiled for an Arduino Uno, the compiler calculates a 1204 bytes (58%) of dynamic memory usage with PixelArt object. Without it is 168 bytes (8%).
Try reducing the inheritance and function call levels.

Exporting vulkan memory allocation handle cause an out of device memory

I'm trying to export the handle of a memory allocation made by Vulkan to import it into OpenGL. But when I add a vk::ExportMemoryAllocateInfo in the pNext chain of the vk::MemoryAllocateInfo, vk::Device::allocateMemory throws an OutOfDeviceMemory exception.
When I don't export the handle, the allocation does not fail but the returned handle is not valid.
Here is the guilty code (based jherico's example: vulkan-opengl-interop-example):
// The physical device and the device are correct
// requirements are for a RGBA image of 1024x1024 pixels
// memory properties is just vk::MemoryPropertyFlagBits::eDeviceLocal
void IMemoryObject::Allocate(vk::PhysicalDevice physicalDevice, vk::Device device, const vk::MemoryRequirements& requirements, vk::MemoryPropertyFlags properties)
{
unsigned int memoryTypeIndex = 0;
bool memoryIndexTypeFound = false;
vk::PhysicalDeviceMemoryProperties memoryProperties = physicalDevice.getMemoryProperties();
for (unsigned int i = 0; i < memoryProperties.memoryTypeCount && !memoryIndexTypeFound; i++)
{
vk::MemoryType memoryType = memoryProperties.memoryTypes[i];
if (requirements.memoryTypeBits & 1 << i && (memoryType.propertyFlags & properties) == properties)
{
memoryTypeIndex = i;
memoryIndexTypeFound = true;
}
}
if (!memoryIndexTypeFound)
throw std::exception();
vk::ExportMemoryAllocateInfo exportAllocInfo;
exportAllocInfo.setHandleTypes(vk::ExternalMemoryHandleTypeFlagBits::eOpaqueWin32);
vk::MemoryAllocateInfo allocateInfo;
allocateInfo.setPNext(&exportAllocInfo); // Remove this line and the allocation won't fail
allocateInfo.setAllocationSize(requirements.size);
allocateInfo.setMemoryTypeIndex(memoryTypeIndex);
deviceMemory_ = device.allocateMemoryUnique(allocateInfo);
// Call VkBindBufferMemory or VkBindImageMemory, never reached anyway when allocateInfo.pNext == &exportAllocInfo;
BindMemory(*deviceMemory_, 0);
}
My system is:
Windows 7
Nvidia Quadro RTX 4000
Driver version 431.02 or 431.94
The validation layer is present in my instance but stays silencious, extensions VK_KHR_external_memory and VK_KHR_external_memory_win32 are available in my device, the allocation size respect the API limitations and the memoryIndexType is correct.
Am I doing something wrong or there is a limitation I missed?
Thanks !
Edit:
I tried to export the handle as a vk::ExternalMemoryHandleTypeFlagBits::eOpaqueWin32Kmt and the allocation worked. The code below is how I test if the allocation require an dedicated allocation to export an handle type.
bool RequireDedicatedAllocation(vk::PhysicalDevice physicalDevice, const vk::ImageCreateInfo& createInfo, vk::ExternalMemoryHandleTypeFlagBits handleType)
{
vk::PhysicalDeviceExternalImageFormatInfo externalImageFormatInfo;
externalImageFormatInfo.setHandleType(handleType);
vk::PhysicalDeviceImageFormatInfo2 imageFormatInfo;
imageFormatInfo.setUsage(createInfo.usage)
.setFormat(createInfo.format)
.setTiling(createInfo.tiling)
.setType(createInfo.imageType)
.setPNext(&externalImageFormatInfo);
vk::StructureChain<vk::ImageFormatProperties2, vk::ExternalImageFormatProperties> imageFormatPropertiesChain = physicalDevice.getImageFormatProperties2<vk::ImageFormatProperties2, vk::ExternalImageFormatProperties>(imageFormatInfo);
vk::ExternalImageFormatProperties externalImageProperties = imageFormatPropertiesChain.get<vk::ExternalImageFormatProperties>();
return static_cast<bool>(externalImageProperties.externalMemoryProperties.externalMemoryFeatures & vk::ExternalMemoryFeatureFlagBits::eDedicatedOnly);
}
On my system, it throws an ErrorFormatNotSupported error (on vk::PhysicalDevice::getImageFormatProperties2) with vk::ExternalMemoryHandleTypeFlagBits::eOpaqueWin32 and return false with vk::ExternalMemoryHandleTypeFlagBits::eOpaqueWin32Kmt.
I finally ended up with a vk::ExternalMemoryHandleTypeFlagBits::eOpaqueWin32Kmt. I don't need to interact with it throught the Windows' API, just OpenGL.

result of call warning - want to use the result?

I have some difficulties with the new Swift 3.0. First I know the “result of call is unused” warning has been debated a few times already. I have read the topics in question and usually the suggested answer is to use #discardableResult or using _ = before the function. But in my case this isn’t working. Before upgrading xCode the code worked great. I tested it on my iPad and everything was ok. But then once I upgraded xCode and had to convert the code to comply with Swift 3.0 the problem appeared.
The thing is that whatever I do, the object in the game isn’t showing as it should. Like I said, previously it worked, but now it doesn’t anymore. When the character runs into the object and it crashes it, the game should display a crashed object, but instead it shows an image with a red “X” as if the image isn’t there.
Here is the code. I would appreciate any suggestion you guys have. Thanks in advance
for block in Blocks
{
block.move()
block.isColB()
if(block.manager.HasFloorL2)
{
if(block.isOnL2())
{
if(block.manager.FloorL2Obstact)
{
block.ObsColL2() // here, result of call to ObscColL2() is unused
}
break;
}
}
if(block.manager.HasFloorL1)
{
if(block.isOnL1())
{
if(block.manager.FloorL1Obstact)
{
block.ObsColL1() // here, result of call to ObscColL1() is unused
}
break;
}
}
}
Here is the function it refers to.
func ObsColL1() -> Bool
{
if(kong.Node.position.x+kong.Node.size.width*0.7 > self.holder.position.x+ObstacleL1.Node.position.x
&& kong.Node.position.x+kong.Node.size.width*0.7 < self.holder.position.x+ObstacleL1.Node.position.x+ObstacleL1.Node.size.width*0.3)
{
if(kong.Y() <= self.holder.position.y+(ObstacleL1.Node.position.y)+ObstacleL1.Node.size.height*0.7 && kong.Y() >= self.holder.position.y+ObstacleL1.Node.position.y)
{
if(kong.state != heroStates.flash && !self.ObstacleL1.crashed)
{
kong.Node.position.x = self.holder.position.x+ObstacleL1.Node.position.x-(kong.Node.size.width*0.55)
kong.Node.run(SKAction.moveTo(y: self.holder.position.y+ObstacleL1.Node.position.y+ObstacleL1.Node.size.height/2, duration: 0.5))
kong.die()
}
else
{
ObstacleL1.crash()
return true
}
}
}
return false
}
EDIT:
before and after
before and after
before and after 2

Crash while inserting int a set

My program seems to be crashing while inserting and int into a set, and I cannot track down the reason for this. Here is there relevant code:
bool request::check_list(std::vector<int> search_vec)
{
std::set<int> *tmp_ptr = create_int_set();
boost::shared_ptr<std::set<int> > c_list(tmp_ptr);
if(aerospike_query_foreach(as, &err, NULL, &query, process_set, &c_list) != AEROSPIKE_OK)
{
return false;
}
for(int i = 0; i < search_vec.size(); i++)
{
if(c_list->find(search_vec[i]) != c_list->end())
{
c_list_value_ = search_vec[i];
return true;
}
}
return false;
}
bool request::process_set(const as_val *val, void * udata)
{
try
{
boost::shared_ptr<std::set<int> > c_set = *(boost::shared_ptr<std::set<int> > *)(udata);
if(val == NULL)
{
return true;
}
if(val->type == AS_REC)
{
if (val!=NULL)
{
as_record *rec = as_record_fromval(val);
if (rec!=NULL)
{
as_integer* c_id = as_record_get_integer(rec,"c_id");
int cid = 0;
cid = boost::lexical_cast<int>(c_id->value);
if(c_set != nullptr)
{
c_set->insert(c_id);
}
as_record_destroy(rec);
as_integer_destroy(c_id);
}
}
return true;
}catch(...){}
return false;
}
The line c_set->insert(c_id); is causing a segfault. Here is this backtrace of the crash:
#0 0x00007f2064299f94 in std::_Rb_tree_rotate_right(std::_Rb_tree_node_base*, std::_Rb_tree_node_base*&) () from /usr/lib64/libstdc++.so.6
#1 0x00007f206429a12b in std::_Rb_tree_insert_and_rebalance(bool, std::_Rb_tree_node_base*, std::_Rb_tree_node_base*, std::_Rb_tree_node_base&) () from /usr/lib64/libstdc++.so.6
#2 0x00000000004829d9 in std::_Rb_tree<int, int, std::_Identity<int>, std::less<int>, std::allocator<int> >::_M_insert_<int const&> (this=0x7f1fcc005440, __x=0x0, __p=0x7f1f3c0009a0, __v=#0x7f20159e729c)
at /opt/centos/devtoolset-1.1/root/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/bits/stl_tree.h:981
#3 0x000000000047f1e0 in std::_Rb_tree<int, int, std::_Identity<int>, std::less<int>, std::allocator<int> >::_M_insert_unique<int const&> (this=0x7f1fcc005440, __v=#0x7f20159e729c)
at /opt/centos/devtoolset-1.1/root/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/bits/stl_tree.h:1299
#4 0x000000000047c473 in std::set<int, std::less<int>, std::allocator<int> >::insert (this=0x7f1fcc005440, __x=#0x7f20159e729c)
at /opt/centos/devtoolset-1.1/root/usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/bits/stl_set.h:415
#5 0x00000000004765ee in request::process_set (val=0x7f20159e73e0, udata=0x7f200b9d6620) at ../../request.cpp:1862
I am assuming there is a problem where the set is not being initialized, or something similar to that. Here is how I create and pass the set from the another function, I have tried two ways to create it:
boost::shared_ptr<std::set<int> > c_list(new std::set<int>());
and
std::set<int> *tmp_ptr = create_int_set();
boost::shared_ptr<std::set<int> > c_list(tmp_ptr);
std::set<int>* request::create_int_set()
{
return new std::set<int>();
}
The calling function is a callback function from a database driver, that takes in a few different objects, most notably however are the process_set and the c_list, which is passed as a void*:
aerospike_query_foreach(as, &err, NULL, &query, process_set, &c_list)
This crash does not happen all the time, in fact it is fairly rare, which makes me think there is something I am doing wrong, some sort of undefined behavior. Any help would be greatly appreciated!
The aerospike APi documentation says for the callback (i.e. process_set(), here):
Execute a query and call the callback function for each result item.
Multiple threads will likely be calling the callback in parallel.
Therefore, your callback implementation should be thread safe.
As several threads might insert at the same time in the same set (the one pointed to by your shared pointer), you'll get race conditions and hence undefined behaviour !
So I think that you should protect at least your set insertion block with a lock_guard on a mutex.
Important edit: boost::shared_ptr<> can't assumed to be thread safe. In the examples on boost.org, they suggest that a shared pointer going out of scope might cause a race. It may therefore be highly advisable to follow Matthew Moss's suggestion in the comments and use a raw pointer to the set within the bounds of process_set().

std::map::iterator crashes program on increment

What could cause this?
Here's the stack trace:
#0 0x0645c0f5 in std::_Rb_tree_increment (__x=0x83ee5b0)
at ../../../../libstdc++-v3/src/tree.cc:69
#1 0x0805409a in std::_Rb_tree_iterator<std::pair<std::string const, Widget*> >::operator++ (
this=0xbffff144)
at /usr/lib/gcc/i586-redhat-linux/4.4.1/../../../../include/c++/4.4.1/bits/stl_tree.h:192
#2 0x08053d32 in Generic::StartLayout (this=0x8287d68) at Generic.cpp:195
#3 0x0804f6e1 in LCDControl::ConfigSetup (this=0xbffff26c) at LCDControl.cpp:91
#4 0x0804ed7c in LCDControl::Start (this=0xbffff26c, argc=1, argv=0xbffff404) at LCDControl.cpp:21
#5 0x08050964 in main (argc=1, argv=0xbffff404) at Main.cpp:11
And here's the code:
for(std::map<std::string,Widget *>::iterator w = widgets_.begin();
w != widgets_.end(); w++){
if( w->second->GetType() & WIDGET_TYPE_BAR) {
w->second->SetupChars();
}
w->second->Start();
}
Edit: This next problem is related, so I won't open a whole new question. I'll leave the answer acceptance like it is. I just need to know something. I have two iterators, one main and one within that main after a function call. They both relate to the same map. Well, the one inside gets all corrupted, and the main one's loop stops iterating.
Here's the code.
Here's StartLayout:
void Generic::StartLayout() {
Error("StartLayout: %s", key.c_str());
for(std::map<std::string,Widget *>::iterator w = widgets_.begin();
w != widgets_.end(); w++){
Error("Starting widget %s", w->first.c_str());
if( w->second->GetType() & WIDGET_TYPE_SPECIAL) {
w->second->SetupChars();
}
w->second->Start();
}
}
And here's SetupChars():
void WidgetGif::SetupChars() {
Error("SetupChars <%s> <%s>", name_.c_str(), widget_base_.c_str());
Error("Size of widgets: %d", visitor_->Widgets().size());
std::map<std::string, Widget *> widgets = visitor_->Widgets();
for(std::map<std::string, Widget *>::iterator ii=visitor_->Widgets().begin();
ii != visitor_->Widgets().end(); ii++) {
Error("<%s> Widget base %s == %s", ii->first.c_str(), ii->second->GetWidgetBase().c_str(), widget_base_.c_str());
if(ii->second->GetWidgetBase() == widget_base_ &&
((WidgetGif *)ii->second)->HasChars()) {
Error("Using chars from %s", ii->first.c_str());
for(unsigned int i = 0; i < rows_ * cols_; i++ ) {
if(i >= visitor_->GetLCDText()->CHARS) {
Error("1) GIF too large: %s, %d", name_.c_str(), visitor_->GetLCDText()->CHARS);
if(update_) delete update_;
update_ = new Property(visitor_, section_, "", new Json::Value("-1"));
return;
}
ch_[i] = ((WidgetGif *)widgets[ii->first])->GetChars()[i];
}
hasChars_ = true;
return;
}
}
// It goes on, but I snipped it here.
}
And this is what happens:
StartLayout: display_qt
Starting widget widget_gif_american_flag:layout_american_flag:0
SetupChars <widget_gif_american_flag:layout_american_flag:0> <layout_american_flag>
Size of widgets: 5
<widget_gif_american_flag:layout_american_flag:1> Widget base layout_american_flag == layout_american_flag
<widget_gif_american_flag:layout_american_flag:4> Widget base layout_american_flag == layout_american_flag
<(n
(n
��S> Widget base ГS == layout_american_flag
^C
Last edit: I figured it out. I just needed a copy of the original map for the new iterator.
There could be quite a few reasons for that. For one, it may be that GetType or SetupChars or Start do something that causes your map to change - which would invalidate the current iterator (note that using operator[] on the map, even just to read the value, is technically a mutating operation, and can cause a crash with iterator debugging enabled!). Alternatively, your map could be corrupted in memory by some code that executed before, e.g. because of a buffer overrun overwriting part of the map's tree.
You must not modify widgets_ in any of your methods GetType, SetupChars, or Start. This is most likely your problem.
If you must modify widgets_, then you will need to restart the iterator whenever such a change is made. To avoid duplicating your changes, you can use a simple marker dictionary outside the loop, or a marker member of the Widget class.