I've been trying to figure out the approach to take to allow a JavaFX TableView (or any other JavaFX thing) to represent some Clojure data, and allow the user to manipulate the data through the GUI.
For this discussion, let's assume I have a list/vector of maps, ie something like
[{:col1 "happy1" :col2 "sad1"} {:col1 "happy2" :col2 "sad2"}] and I want it to display in a graphical table as follows:
mykey1 mykey2
------------------
happy1 sad1
happy2 sad2
Pretty straightforward. This has been done a gazillion times in the history of the world.
The problem is the TableView insists on taking an ObservableList, etc., which is inherently a mutable thing, as are all the Observables in JavaFX. This is great for keeping the table up to date, and in a mutable model, it's also great for allowing the user to directly manipulate the data via the GUI. I'm not an expert, but in this case it seems JavaFX wants the GUI object to actually contain the real data. This seems funny (not ha ha) to me. To maintain my own model and communicate between GUI and model via some API or interface also implies that I'm maintaining the data in two places: in my own model, and in the GUI. Is this the Correct Way of Doing Things? Maybe this is ok since the GUI only ever displays a small fraction of the total data, and it lets my model data just be normal model data, not an instance of some Java-derived type.
So this leads to the following three general questions when trying to put a GUI on a stateless/immutable model:
How can the model underneath be truly immutable if the GUI necessarily allows you to change things? I'm thinking specifically of some sort of design tool, editor, etc., where the user is explicitly changing things around. For example LightTable is an editor, yet the story is it is based on immutable data. How can this be? I'm not interested in FRP for this discussion.
Assuming at some level there is at least one Atom or other Clojure mutable type (ref/var/agent/etc) (whether it be a single Atom containing the entire in-memory design database, or whether the design database is an immutable list of mutable Atoms), which of the [MVP, MCP, MVVM, etc.] models is best for this type of creation?
JavaFX has littered the class hierarchy with every imaginable variation of Observable interface (http://docs.oracle.com/javafx/2/api/javafx/beans/Observable.html), with such gems as Observable[whatever]Value, including for example ObservableMap and ObservableMapValue, and then dozens upon dozens of implementing classes such as IntegerProperty and SimpleIntegerProperty... geez! wtf?. Assuming that I have to create some Clojure objects (defrecord, etc.) and implement some of the Observable interface methods on my mostly immutable objects, can I just stick with Observable, or must I implement each one down to the leaf node, ie ObservableIntegerValue, etc.?
What is the correct high level approach? Maintain a single top-level atom which is replaced every time the user changes a value? Maintain a thousand low-level atoms? Let my values live as JavaFX Observables, and forget about Clojure data structs? Implement my own set of Observables in Clojure using some reify/proxy/gen-class, but implement them as immutables that get replaced each time a change is made? Is there a need or place for Clojure's add-watch function? I'd very much prefer that my data just be normal data in Clojure, not a "type" or an implementation of an interface of anything. An Integer should be an Integer, etc.
thanks
I would add a watch to the atom, and then update your observable based on the diff.
http://clojuredocs.org/clojure.core/add-watch
Related
Is an QAbstractListModel implementation with say roles
name, image, enabled, color, editable
Better than an implementation with a single role (forgive me for not being able to come up with a better name)
thing
which causes a QObject* with aforementioned Q_PROPERTYs to be returned?
The difference on QML side would be an extra "indirection" when accessing the values in say, a delegate:
model.thing.name
vs
model.name
I lean towards the QObject approach, because it disentangles the data view from the list model, and has a potential for reuse. It also doesn't require you to also implement the setData function. As far as I can tell, there are no specific pro's for the "many roles" option, unless of course it's impossible to properly define a proper QObject subclass wrapping the data (because e.g. it could make little sense conceptuallyfor specific cases).
Having roles in a model is helpful when you want to use it with features that rely on roles.
For example you want to feed your model to a ComboBox, if it has roles you can just specify a textRole to the ComboBox and it will do its thing. If using a single object role, you will have to modify the delegate to display your correct object property.
Roles are also needed if you want to make use of ListView's section feature.
Like #derm said, it's also useful with a QSortFilterProxyModel (shameless plug: if your model has roles, it's super easy to filter or sort it with my SortFilterProxyModel library).
You mention reusability and having to reimplement stuff. That's a good point but it's totally doable to have a generic model with roles based on QObject properties.
In fact it has been done and it is available with a very permissible license here : Thomas Boutroue's QQmlObjectListModel. From c++ it has the same API as a QList but it exposes a QAbstractListModel that is usable by QML.
The roles are based on the properties of your objects, and when a notify signal is emitted, it emits the corresponding dataChanged signal. insertRows and friends are also handled automatically.
With that said, I don't see much point in doing a model with a single object role. If you want a proper model with rowsInserted/rowRemoved signals (and you should, unless you just have a static model, in this case a QVariantList or a QObjectList is enough), you'd have to implement it yourself anyway when you can just use QQmlObjectListModel and be done with it.
Using roles means you have to stick to a pre-defined model schema.
Roles are good if your objects are low level, so you can expose various object members as roles.
But in the case of a QObject * model, that is entirely unnecessary. You are already utilizing Qt's meta object system, which can facilitate the same functionality in conjunction with the QML engine..
As long as the properties are properly implemented and have notify signals, it will all work seamlessly in QML.
And it is a big time saver too, as the model is generic, and you can populate it with objects, defined in QML without having to recompile the C++ code or have a dedicated model for each and every model.
Keep in mind that QObjects are rather big, and to present a significant overhead if you have scores of model items. If your model is huge or you are dealing with an already existing data set that is not available as QObject*s out of the box, it will be more efficient to implement a "classical" model with roles and whatnot.
As long as you don't run into its limitations, I'd say a single role QObject* model is the simpler, easier, faster, more flexible and more elegant solution.
As for sorting and filtering issue mentioned, the stock solutions obviously won't cut it. As mentioned here, it is possible to implement sorting and filtering which takes in a JS functor in which you can run your custom logic, which I dare say is actually better than the stock role based functionality. And again, you can get new code in it without having to recompile, it is even possible to drive it via functors generated at runtime.
Yes, you should.
The most fundamental reason to use roles is: It is the Qt way to write models.
Ok, this reason is nothing that really matters --- unless there are things that expect you to do it the Qt way...
So, there are reasons why you should implement roles, but that does not mean, you have to use them in QML - and depending on what you plan to do with your model, it will be unnecessary.
In my opinion, the most important reason is, that you have to implement that dataChanged-signal. If you don't do that, your model is worth as much as QVariantList.
Changes of the the properties of your object would only be reflected in the delegates that are generated but objects that are relying on the dataChanged-signal won't receive updates - unless you trigger it for the object-role when ever a property of that object has changed.
This on the other hand will have large impact on the delegates, since than all bindings have to be reevaluated.
Take for example the QSortFilterProxyModel - if you don't have the dataChanged-signal, it won't update the filter or sorting, if a value has been changed, since it is listening to the dataChanged-signal for that.
Ad hoc, I don't know any other cases that would use that from the Qt libraries, but there might be others.
However, as long as you don't intend to use any of those - you don't need to fire the dataChanged-signal.
Whether you have to really implement roles for that, I don't know. I have not tried to implement the dataChanged-signal without roles.
Code of QSortFilterProxyModel, that uses the dataChanged-signal
You don't need to
When you either don't use anything that relies on properly implemented roles or you reimplement that functionality that would rely on the roles your self.
I'm developing a cross-platform C++ data model with almost 1,000 different kinds of data items (but < 100 different data structures/classes). To handle the Model to Controller messages (i.e. notification for the Controller that some data item has changed) I'm thinking about using boost:signals2. I think this will help create a unified Observer pattern that remains the same across different OS platforms. Initial implementation is on Mac OS / IOS, with subsequent UI developed on .net and unix.
Questions:
1) To set up observation of the data model, what type of object should a Controller connect to the signals2 object/slots? Should there be specific functions/methods/selectors in the Controller for EACH data item observed? Should these functions be C, C++, or Objective-C/C++ functions?
2) What level of granularity should a signal have? Should each item of data have its own? Or should the Model's management of related structures/records maintain a single signal for each type of structure/record? For example — should application preferences have ONE signal for all preferences data — passing information about what data item was changed?
3) Should the process of sending signals (or the slots receiving signals) be performed in a SEPARATE THREAD?
4) I understand Cocoa has its own system for Key-Value Observing. But would such a system be advantageous to COMBINE with the Model's signals2-based Observer paradigm — or just redundant?
UPDATE:
Regarding "granularity" of actual signal2 objects (not observers), I think having one per document and one for application prefs might be a good place to start. My document data already has a "key" concept, so it may be possible to generalize common UI cases where a UI component is tied to a specific item of model data.
There are quite a lot of facets to this question, so I'll tackle what I can.
Why do we want to be able to observe something in Objective-C and .NET?
I think it's instructive to think about why we'd want to observe a data-model in a 4th generation language? Aside from decoupling subsystems, we can also bind UI interface elements directly to the data model in some cases, avoiding the need to write a great deal of code in the controllers. This is possible in a WPF application built on .NET and in AppKit on MacOSX. There are fewer opportunities in UIKit on iOS.
Meta-model
The first obvious thing to point out is that two of the language run-times you want to target (Objective-C and .NET) provide language-level implementations of the observer pattern which are built around object properties. If you want to integrate them without writing thousands of lines of boiler-plate code, you will want to do this too.
This strongly suggests that all of your C++ model classes need to inherit from a meta-model that provides a generic property mechanism. Wrapped up with this will be an mechanism for observing them.
The alternative will be to generate an enormous quantity of boiler-plate setter/getters in C++, and binding objects to make use of them in .NET and Objective-C.
Adaptors
For both .NET and Objective-C you would design a generic adaptor class that uses reflection to do their dirty work. In the case of Objective-C you do this by overriding methods in NSObject to forward property sets/get messages onto the underlying model's property mechanism. I believe in .NET you can use reflection to inject properties corresponding to those in your model into the adaptors.
[Aside] IDL/ORM
Surely by the point you have a data model built on top of a meta-model, you might as well use some kind of IDL or ORM to describe and automatically generate the very large number of model classes? The huge number of classes you are dealing with also suggest this too.
Granularity of Observers
In terms of granularity, you could do either - it really depends on the way in which your data model changes and the UI needs to react to it. KVO in Objective-C is designed around changes to named properties on an object, but is restricted in that an observer has a single delegate method for all objects and properties it observes. This is, however, only an issue for your Objective-C UI components.
Integrating with this is definitely worthwhile unless you want to write masses of adaptor methods for value-changed events that bridge between C++ and Objective C: Remember that whilst there is toll-free interwork between C++ and Objective-C++, objective-C++ classes can't inherit interfaces C++ classes, not can you send a message to an objective-C object from C++ directly without a bridge and some nasty casting in an objective-C++ compilation unit.
With .NET you'd probably get into all kinds of similar machinations with an intermediate managed-C++ layer.
Threading
The only sane way to do this involving more than one thread is to queue signals for processing on the UI thread. After all, if generating signals in another thread you'll need to do this at some point later to update the UI. Using a separate thread for model and View/Controller portions of the app sounds like a recipe for dead-locks. It'll also be very hard to debug the source of a signal it it'll be long gone in another thread. Your life will be much easier keeping it all on one thread.
Problem domain
I'm working on a rather big application, which uses a hierarchical data model. It takes images, extracts images' features and creates analysis objects on top of these. So the basic model is like Object-(1:N)-Image_features-(1:1)-Image. But the same set of images may be used to create multiple analysis objects (with different options).
Then an object and image can have a lot of other connected objects, like the analysis object can be refined with additional data or complex conclusions (solutions) can be based on the analysis object and other data.
Current solution
This is a sketch of the solution. Stacks represent sets of objects, arrows represent pointers (i.e. image features link to their images, but not vice versa). Some parts: images, image features, additional data, may be included in multiple analysis objects (because user wants to make analysis on different sets of object, combined differently).
Images, features, additional data and analysis objects are stored in global storage (god-object). Solutions are stored inside analysis objects by means of composition (and contain solution features in turn).
All the entities (images, image features, analysis objects, solutions, additional data) are instances of corresponding classes (like IImage, ...). Almost all the parts are optional (i.e., we may want to discard images after we have a solution).
Current solution drawbacks
Navigating this structure is painful, when you need connections like the dotted one in the sketch. If you have to display an image with a couple of solutions features on top, you first have to iterate through analysis objects to find which of them are based on this image, and then iterate through the solutions to display them.
If to solve 1. you choose to explicitly store dotted links (i.e. image class will have pointers to solution features, which are related to it), you'll put very much effort maintaining consistency of these pointers and constantly updating the links when something changes.
My idea
I'd like to build a more extensible (2) and flexible (1) data model. The first idea was to use a relational model, separating objects and their relations. And why not use RDBMS here - sqlite seems an appropriate engine to me. So complex relations will be accessible by simple (left)JOIN's on the database: pseudocode "images JOIN images_to_image_features JOIN image_features JOIN image_features_to_objects JOIN objects JOIN solutions JOIN solution_features") and then fetching actual C++ objects for solution features from global storage by ID.
The question
So my primary question is
Is using RDBMS an appropriate solution for problems I described, or it's not worth it and there are better ways to organize information in my app?
If RDBMS is ok, I'd appreciate any advice on using RDBMS and relational approach to store C++ objects' relationships.
You may want to look at Semantic Web technologies, such as RDF, RDFS and OWL that provide an alternative, extensible way of modeling the world. There are some open-source triple stores available, and some of the mainstream RDBMS also have triple store capabilities.
In particular take a look at Manchester Universities Protege/OWL tutorial: http://owl.cs.manchester.ac.uk/tutorials/protegeowltutorial/
And if you decide this direction is worth looking at further, I can recommend "SEMANTIC WEB for the WORKING ONTOLOGIST"
Just based on the diagram, I would suggest that an RDBMS solution would indeed work. It has been years since I was a developer on an RDMS (called RDM, of course!), but I was able to renew my knowledge and gain very many valuable insights into data structure and layout very similar to what you describe by reading the fabulous book "The Art of SQL" by Stephane Faroult. His book will go a long way to answer your questions.
I've included a link to it on Amazon, to ensure accuracy: http://www.amazon.com/The-Art-SQL-Stephane-Faroult/dp/0596008945
You will not go wrong by reading it, even if in the end it does not solve your problem fully, because the author does such a great job of breaking down a relation in clear terms and presenting elegant solutions. The book is not a manual for SQL, but an in-depth analysis of how to think about data and how it interrelates. Check it out!
Using an RDBMS to track the links between data can be an efficient way to store and think about the analysis you are seeking, and the links are "soft" -- that is, they go away when the hard objects they link are deleted. This ensures data integrity; and Mssr Fauroult can answer what to do to ensure that remains true.
I don't recommend RDBMS based on your requirement for an extensible and flexible model.
Whenever you change your data model, you will have to change DB schema and that can involve more work than change in code.
Any problems with DB queries are discovered only at runtime. This can make a lot of difference to the cost of maintenance.
I strongly recommend using standard C++ OO programming with STL.
You can make use of encapsulation to ensure any data change is done properly, with updates to related objects and indexes.
You can use STL to build highly efficient indexes on the data
You can create facades to get you the information easily, rather than having to go to multiple objects/collections. This will be one-time work
You can make unit test cases to ensure correctness (much less complicated compared to unit testing with databases)
You can make use of polymorphism to build different kinds of objects, different types of analysis etc
All very basic points, but I reckon your effort would be best utilized if you improve the current solution rather than by look for a DB based solution.
http://www.boost.org/doc/libs/1_51_0/libs/multi_index/doc/index.html
"you'll put very much effort maintaining consistency of these pointers
and constantly updating the links when something changes."
With the help of Boost.MultiIndex you can create almost every kind of index on a "table". I think the quoted problem is not so serious, so the original solution is manageable.
I hope I can phrase this question correctly. I have a concern when dealing with state and test-ability in classes with static and instance mutable fields.
Do the static fields essentially constitute a different class/responsibility/instance due to the difference in their lifetime/scope?
If so: then shouldn't the instance fields also be a separate class/data structure?
And then: and if that is so, then shouldn't all classes be stateless only receiving their dependencies on construction and should all then be immutable?
And finally, would this mean that functional programming is the right way to do object oriented programming as well?
You shouldn't have (really) mutable static fields. That is crappy design. Functional programming makes things a lot easier. I would separate the concerns so:
what is data, should only have the necessary code to manipulate it in class. like "string" "Integer" "Person" , these are the entities
what manipulates data, depends on data, but should not have internal state (Formatters etc)
what drives the execution , must usually have some internal state ,but it is either carried with the call, or is "configuration" like
database connections, requests etc. This is either from an active
request , or it is pure configuration (injected)
what views results (pure views) should be stateless in itself, and state should be fed to it with a context
then there is the glue in between which usually is the hairy exception..minimize that.
etc..
For testability
data should be mockable to a fake database.
"what manipulates data" should be easy to test since it is stateless.
State contained in contexts is easy to mock.
"what drives the execution" should be easy to test since dependencies are injectable and you can test it piece by piece.
views are easy to test since mocked state can be fed to it. the difficulty comes from actually verifying "what it looks like" , but that is up to either gui automation or human testers.
In essence ALL of this could be done in a functional way if database (disk) and the request layer (web, ui, whatever) would comply. In practise you try to do the "pure" part in between pretty and functionalish , and use design patterns to shield it from the outside dirt.
I just starting on WPF with MVVM, so forgive any misconceptions. I have a model (not a view model but the actual model) that has many List inside classes that are in other List making trees of data.
Important, the data should be XML Serializabled, currently there is no problem doing that with regular properties and List.
The View Model of this class is taking a lot more work than I expected and I'm thinking on converting some or maybe all of the List to ObservableCollections, what are the pros and cons of that?
Also, what would be best in a "bindable model", NotifyPropertyChange or DependencyProperties? I think there would be some memory gains with DependencyProperty since many objects will use default values on most of the properties, what about performance in general, and has it any problems with serialization?
Regards,
The View Model of this class is taking a lot more work than I expected and I'm thinking on converting some or maybe all of the List to ObservableCollections, what are the pros and cons of that?
There is a bit more overhead in ObservableCollection<T> when compared to List<T>, but it does make it easier as data binding directly to your model collections will work.
Also, what would be best in a "bindable model", NotifyPropertyChange or DependencyProperties?
I would not use DependencyProperties within your model. This is taking a dependency on the WPF libraries, and very platform specific. If you want to be able to data bind directly to your model, implementing INotifyPropertyChanged is a reasonable approach.
However, you really should stop and take a minute before making any of these changes. If you're changing your model classes primarily to use them in your View, you're violating the basic principles of MVVM (and most other architectural patterns). One of the main goals here is that your Model is unaware of the presentation layer being used - this makes it more flexible in the future, as you can change presentation layers without changing your model at all.