PyQt4: Load large data sets in QStandardItemModel - python-2.7

Well, in my current project, the records are loaded from a database directly into QStandardItemModel and then view via QTreeView(). Everything works fine too. But I thought about the ram-memory. I think that the QStandardItemModel, including records, will be loaded into ram-memory. Similar to the data structure (for instance list), right? But what happens when I have large data sets? I read in Qt Documentation: Fetch More Example that I have to add items to the model in batches and preferably only when the items are needed by the view. It sounds good, but I have no idea how to do that. Does anyone of you know a few examples in python codes? Unfortunately, I do not understand the C ++ examples. Or does anyone know a better solution?
Don't get me wrong. I don't want a ready solution because I'm too lazy. No! I want to learn and work on it.

Related

Django Upload File, Analyise Conents and write to DB or update form

I'm pretty new to Django, Trying to get my grips with it, and expand what I think its capable of doing, and maybe one of you more intelligent people here can point me in the right direction.
I'm basically trying to build a system similar to a so called "Asset Management" system, to track software version of a product, so when an engineer updates the software version, they run a script which gathers all the information (Version, Install date, Hardware etc), which is stored in a .txt file, The engineer then comes back to the website and upload this .txt file for that customer, and it automatically updates the fields in the form, or directly to the database.
While, I've search a bit on here for concepts, I haven't been able to find something similar (Maybe my search terms aren't correct?), and wanted to ask if anyone knows, what I'm doing is even feasible, or am I lost down the rabbit hole of limitations :) Maybe its not do-able within Django, Any suggestions on how I should approach such a problem would be greatly appreciated.
What you're asking is doable both in the form and post-submit of a form or file upload.
In the form approach you'll want a live-reload of the form should the data come from a .txt file. That can be done with JavaScript. This will mean that the data will come from the text file and be input into the forms in the manner you define. This also means form validation will work as you want it to.
Another option is to require a txt file in a specified format, and parse it in the view, form_valid() for Class-Based Views, and request.FILES[] for function based views, and then perform all the required validations and then save the values to the database as the model instance.

How can I check if a C++ class description has changed?

I have a set of complex C++ classes.
After creating objects of the class, all the data can be saved on disk.
I want to load two such saved instances and tell if they are identical.
Any ideas on how to do this in a way that is maintainable?
I've tried doing sorted text reports that print all the data and compare that. The problem is fields can get added to the classes over time and its not possible to tell if the report is "complete".
Any way introspection or reflection can be used to accomplish this?
this is a complex problem which those wonderful guys at boost.org have already solved for you:
http://www.boost.org/doc/libs/1_58_0/libs/serialization/doc/index.html

Updating a field in all records in elasticsearch

I'm new to ElasticSearch, so this is probably something quite trivial, but I haven't figured out anything better that fetching everything, processing with a script and updating the registers one by one.
I want to make something like a simple SQL update:
UPDATE RECORD SET SOMEFIELD = SOMEXPRESSION
My intent is to replace the actual bogus data with some data that makes more sense (so the expression is basically randomly choosing from a pool of valid values).
There are a couple of open issues about making possible to update documents by query.
The technical challenge is that lucene (the text search engine library that elasticsearch uses under the hood) segments are read only. You can never modify an existing document. What you need to do is delete the old version of the document (which by the way will only be marked as deleted till a segment merge happens) and index the new one. That's what the existing update api does. Therefore, an update by query might take a long time and lead to issues, that's why it's not released yet. A mechanism that allows to interrupt running queries would be a nice to have too for this case.
But there's the update by query plugin that exposes exactly that feature. Just beware of the potential risks before using it.

c++ Reading big text file to string (bigger than string::max_size)

I have a huge text file (~5GB) which is the database for my program. During run this database is read completely many times with string functions like string::find(), string::at(), string::substr()...
The problem is that this text file cannot be loaded in one string, because string::max_size is definitely too small.
How would you implement this? I had the idea of loading a part to string->reading->closing->loading another part to same string->reading->closing->...
Is there a better/more efficient way?
How would you implement this?
With a real database, for instance SQLite. The performance improvement from having indexes is more than going to make up for your time learning another API.
Since this is a database, I'm assuming it'd have many records. That to me implies best idea would be to implement a data class for each records and populate a list/vector/etc depending upon how you plan to use it. I'd also look into persistent cache as the file is big.
And within in your container class of all records, you could implement search etc functions as you see fit. But as suggested for a db of this size, you're probably best of using a database.

How do I create levels for my puzzle game ? Obj-C & Cocos2d

I want to create levels in my cocos2d game and I do not know how to do that with .plist files ... I searched the Internet but unfortunately I couldn't find significant information on how to implement these property lists. Can you please help out ?
Check out Tiled Map Editor. Tiled's TMX format is supported by Cocos2D.
As with any Apple technologies, the first place you should start searching for is the developer.apple.com website. In this case, here's the Property List (plist) Programming Guide.
However, I find property lists very awkward to work with, specifically if you want to create them manually and whenever they contain more than just a few entries. It certainly can't hurt to evaluate rolling out your own file format, text-based plain and simple. I would always rather work with simple text files like these rather than messing with property lists:
X=10;Y=10;Tile=30;
X=12;Y=11;Tile=28;
X=16;Y=19;Tile=22;
It's a different story if you actually design the data with a tool or within an app, where you'll be able to make use of the various collection convenience methods that save and load property lists, for example to and from a dictionary or array.