I have a model as follows:
class SomeModel(ndb.Model):
parent = ndb.KeyProperty()
someint = ndb.IntegerProperty(default = 2)
sum_of_child_some_int = ndb.IntegerProperty()
The parent property is either nothing (in the case of a top level parent) or is a Key pointing at another SomeModel entity.
My question is, what is the most efficient way of calculating the sum of all the someint's of all children of a SomeModel parent? It is recursive, so a child can itself have a child, and the someint would also need to be counted.
I see that it can be done at write - simply write to the entities parents recursively. But can it be done at read, or is that always going to be too expensive?
If you don't want to put that number at write time, you could use an ancestor query to get all the children of a given entity and extract the someint property. If you are going to do this, I think the most efficient way would be to use projection queries.
Just remember that if you don't write that sum when you put the entity, you will have to make the sum every time it is requested (although you may get a cached result), which sounds a bit wasteful. Memory is cheap, and adding another property updated at write time seems like the best idea to me.
By the way, maybe you don't need the parent as a KeyProperty, an ancestor may be a better way to do it.
Related
How to add multiple node to relations here is my query
MATCH (n:Customer{name:"motoM"})-[:RECENT {default:TRUE}]-(l:Location{name:"Adugodi"}) return l how to write a query to add one more "location" node to the relation "recent" if location node is not found and setting default to true to newly created realtion
What about this?
MATCH (n:Customer{name:"motoM"})-[:RECENT {default:TRUE}]-(l:Location{name:"Adugodi"})
MERGE (n)-[:RECENT]->(l2:Location)
ON CREATE SET l2.default = true
RETURN l, l2
The direction needs to be specified so I made it up, but it might need to go the other way.
Well, I don't know if I understood what you were looking for, but this might help you :)
Try with this query:
MATCH (n:Customer{name:"motoM"})-[r:RECENT {default:TRUE}]-(:Location{name:"Adugodi"})
CREATE (l2:Location{name:"Wherever You need"})
With r,n,l,l2
Set r.default = false
With n,l2
CREATE (n)-[r2:RECENT{default:TRUE}]->(l2)
I'm using Withto make the query easier to read, but you can do it in a single query.
In fact, I think your problem is your Graph model.
You should probably do something like a Customer node, related to Location nodes with a "VISITED" relation, and when you create your VISITED relation, you set date property to timestamp. Then, when you get your relations, you can simply compare timestamps to get the closest one, and you know which one is the one your need. Also, if you need a default property, set it on the node, it'll be easier to match.
Tell me if you need a code example for match, create and set data with this graph model.
I have a list with about 2500 custom items. I set them with:
const std::vector<const Items::AbstractItem *> results = _engine.request(text);
if (!results.empty())
{
for (auto i : results){
QListWidgetItem *lwi = new QListWidgetItem;
_results->addItem(lwi);
ListItemWidget *w = new ListItemWidget;
w->setName(i->name());
w->setTooltip(i->path());
_results->setItemWidget(lwi, w);
}
_results->setFixedHeight(std::min(5,_results->count()) * 48); // TODO
_results->show();
}
This takes about 5 seconds on an i5-4590. Hiding the widget is twice as fast. Is this normal or do I have look for errors?
A few ideas:
Try assigning proper parents to your QWidgets, thats way the layout doesn't have to do this
mapping for you. This should help performance.
Call setUpdatesEnable(false) before starting the insert, and to true after it's done
As for hiding the widget while adding large amounts of items, this will help to alleviate extraneous update calls. The second suggestion above should mitigate that.
I think this is fully expected behavior for controls like Lists or Trees that are not based on any data model. And I believe that the data model was invented mainly to fix this issue.
In your situation you have a ListWidget control that stores its data on its own. You need to pass all 2500 items before your app can go on, and you need to do this even if your list shows only 10 items at a time. Even if you just run and close your app, the user won't see all the items but you still need to pass them to your ListWidget. Some GUI frameworks use internal allocation of items and in such case they can optimize things a bit, you could do the same if you allocated your Items in chunks but it's still not a good solution.
Now let's say you introduce some object that could be asked about item properties. The Control will ask about some item and your object will respond with the contents. Your object don't even need to know about all your items, it will just learn when needed.
You Control can ask about few first items and stop when it realize it can fill up its entire height. This way you can avoid work that is not needed for now. The Control can also ask about the item count, so it can set-up its vertical slider.
It needs to be said that the model will not solve your problem automatically, it's just a programming paradigm that allows you to do it better.
So the solution for you would be to replace your QListWidget with a QListView and implement you own data model inheriting QAbstractListModel. You could pass the results to the model and it will pass the items data when needed.
If your QListWidgetItem's always has fixed size, call setUniformItemSizes on your QListWidget, pass true.
I have an association like this
class Parent
List children
static hasMany =[children:Child]
I need to be able to know the order of a Child object when I look at it outside of Parent context. So that I can tell if it is a 1st child, second child, etc.
What do you mean with 'outside of Parent context'? To get index of a object in a list - you need to load it before. If you want to get index without loading Parent, then you have to use an raw SQL for this, but i'm not sure that it will be faster than loading Parent, because it's the same logic, except making mapping resultset to a model.
Btw, to get this index when you have Parent instance, you have to use:
int idx = parent.children.indexOf(child)
And don't forget to implement .equals of your Child domain.
I'm using a small tree/graph package (django_dag) that has that give my model a Many-to-many children field which refers to itself. The basic structure can be shown as the following models
#models
class Foo(FooBase):
class Meta:
abstract = True
children = models.ManyToManyField('self', symmetrical = False,
through = Bar)
class Bar():
parent = models.ForeignKey(Foo)
child = models.ForeignKey(Foo)
All is fine with the models and all the functionality of the package.
FooBase adds a variety of functions to the model, including a way of recursively finding all children of a Foo and the children's children and so forth.
My concern is with the following function within FooBase:
def descendants_tree(self):
tree = {}
for f in self.children.all():
tree[f] = f.descendants_tree()
return tree
It outputs something like {Foo1:{}, Foo2: {Child_of_Foo2: {Child_of_Child_of_Foo2:{}}}} where the progeny are in a nested dictionary.
The alert reader may notice that this method calls a new query for each child.
While these db hits are pretty quick, they can add up quickly when there might be 50+ children. And eventually, there will be tens of thousands of db entries. Right now, each query averages 0.6 msec with a row count of almost 2000.
Is there a more efficient way of doing this nested query?
In my mind, doing a select_related().all() beforehand would get it down to one query but that smells like trouble in the future. At what point is one large query better or worse than many small ones?
---Edit---
Here's what I'm trying to test the select_related().all() option with, but it's still hitting every iteration:
all_foo = Foo.objects.select_related('children').all()
def loop(baz):
tree = {}
for f in all_foo.get(id = baz).children.all()
tree[f] = loop(f)
return tree
I assume the children.all() is causing the hit. Is there another way to get all of Many-to-Many children without using the callable attribute?
You'll have to test under your own environment with your own circumstances. select_related is generally always recommended, but in cases where there will be many recursive levels, that one large query is generally slower than the multiple queries.
The amount of children doesn't really matter, the levels of recursion is what matters most. If you're doing 3 or so, select_related() might be better, but much more than that would likely result in a slow down. The plugin author likely did it this way to allow for many, many levels of recursion, because it only really hurts when there's just a few, and that's only a few extra queries.
Greetings,
I've been writing some nasty code to support the undo/redo of deletion of an arbitrary set of objects from my model. I feel like I'm going about this correctly, as all the other mutators (adding/copy-pasting) are subsets of this functionality.
The code is nastier than it needs to me, mostly because the only way to mutate the model involves calling beginInsertRows/beginRemoveRows and removing the rows in a range (just doing 1 row at a time, no need to optimize "neighbors" into a single call yet)
The problem with beginInsertRows/beginRemoveRows is that removal of a row could affect another QModelIndex (say, one cached in a list). For instance:
ParentObj
->ChildObj1
->ChildObj2
->ChildObj3
Say I select ChildObj1 and ChildObj3 and delete them, if I remove ChildObj1 first I've changed ChildObj3's QModelIndex (row is now different). Similar issues occur if I delete a parent object (but I've fixed this by "pruning" children from the list of objects).
Here are the ways I've thought of working around this interface limitation, but I thought I'd ask for a better one before forging ahead:
Move "backwards", assuming a provided list of QModelIndices is orderered from top to bottom just go from bottom up. This really requires sorting to be reliable, and the sort would probably be something naive and slow (maybe there's a smart way of sorting a collection of QModelIndexes? Or does QItemSelectionModel provide good (ordered) lists?)
Update other QModelIndeces each time an object is removed/added (can't think of a non-naive solution, search the list, get new QModelIndeces where needed)
Since updating the actual data is easy, just update the data and rebuild the model. This seems grotesque, and I can imagine it getting quite slow with large sets of data.
Those are the ideas I've got currently. I'm working on option 1 right now.
Regards,
Dan O
Think of beginRemoveRows/endRemoveRows, etc. as methods to ask the QAbstractItemModel base class to fix up your persistent model indexes for you instead of just a way of updating views, and try not to confuse the QAbstractItemModel base class in its work on those indexes. Check out http://labs.trolltech.com/page/Projects/Itemview/Modeltest to exercise your model and see if you are keeping the QAbstractItemModel base class happy.
Where QPersistentModelIndex does not help is if you want to keep your undo/redo data outside of the model. I built a model that is heavily edited, and I did not want to try keeping everything in the model. I store the undo/redo data on the undo stack. The problem is that if you edit a column, storing the persistent index of that column on the undo stack, and then delete the row holding that column, the column's persistent index becomes invalid.
What I do is keep both a persistent model index, and a "historical" regular QModelIndex. When it's time to undo/redo, I check if the persistent index has become invalid. If it has, I pass the historical QModelIndex to a special method of my model to ask it to recreate the index, based on the row, column, and internalPointer. Since all of my edits are on the undo stack, by the time I've backed up to that column edit on the undo stack, the row is sure to be there in the model. I keep enough state in the internalPointer to recreate the original index.
I would consider using a "all-data" model and a filter proxy model with the data model. Data would only be added to the all-data model, and never removed. That way, you could store your undo/redo information with references to that model. (May I suggest QPersistentModelIndex?). Your data model could also track what should be shown, somehow. The filter model would then only return information for the items that should be shown at a given time.