I was wondering whether it makes sense to add a version number of a timestamp meta data every time my ref is changed, so that the freshness of data can used to determine whether data is updated by GUI components.
If you add metadata (or plain data) to the ref, then UI components will have to poll the ref to know whether to update. You might be better off to use an agent send within the ref update to notify interested parties.
you can use the answer to your question from yesterday for this purpose also.
(def my-ref (ref {}))
(def my-ref-version (atom 0))
(add-watch my-ref (fn [key ref old new] (swap! my-ref-version inc)))
If you stick to immutable data structures then you can save a copy of the data you last served and compare it to the data you are considering serving. This would be a lot simpler and you would not resend data that had been updated to the same value. immutable data is great for caching using time stamps it good when you cant directly compare the data to what you last sent. With languages that dont offer efficient copy functions for the collections this is necessary because you can't efficiently save a copy of your data before sending it. with clojure's collections saving a copy before you send is both easy and efficient.
Although metadata is an option, you could just put your ref data in a map. Either way you have to do a map lookup since Clojure's metadata lives in a map. Using metadata just makes you jump through the extra hurdle. So when you define/update your ref just make it a map and you will be functionally equivalent without having to use meta to get at the information.
I guess, you will need to remember the last time you updated the gui with the value of the ref, in order to know whether updating the gui is necessary or not. If the value is a big datastructure or the update is expensive this might make sense.
Related
I'm new to clojure and I'm having trouble to understand some concepts, specially pure functions and immutability.
One thing that I still can't comprehend is how you solve a problem like this in clojure:
A simple console application with a login method, where the user can't try to login more than 3 times in a 1 minute interval.
In C# for example I could add the UserId and a timestamp to a collection every time the user tries to login, then I would check if there has been more 3 than attempts in the last minute.
How would I do that in Clojure considering that I can't alter my collection ?
This is not a pratical question (although some code examples would be welcomed), I want to understand how you approach a problem like this.
You don't alter objects in most cases, you create new versions of the old objects:
(loop [attempt-dates []]
(if (login-is-correct)
(login)
(recur (conj attempt-dates (current-date-stamp)))))
In this case I'm using loop. Whatever I give to recur will be passed to the next iteration of loop. I'm creating a new list that contains the new stamp when I write (conj attempt-dates (current-date-stamp)), then that new list is passed to the next iteration of loop.
This is how it's done in most cases. Instead of thinking about altering the object, think about creating a transformed copy of the object and passing the copy along.
If you really truly do need mutable state though, you can use a mutable atom to hold an immutable state:
(def mut-state (atom []))
(swap! mut-state conj 1)
(println #mut-state) ; Prints [1]
[] is still immutable here, the new version is just replacing the old version in the mutable atom container.
Unless you're needing to communicate with UI callbacks or something similar though, you usually don't actually need mutability. Practice using loop/recur and reduce instead.
Every textbook says that Clojure data structures are 'immutable and persistent'. They go different lengths explaining the concept, but so far I failed to figure out what is the difference between immutability and persistence. Is there an entity persistent but mutable? or immutable but not persistent?
Immutable means that the value can't be changed and persistence means that the path to the value is copied if the value already exists within the program. Clojure uses this as a part of it's structural sharing implementation. If the data doesn't exist, it's created. If the data exists, the new data builds on the old version of the data without altering or removing it.
Atoms are persistent but safely mutable.
user> (def +a+ (atom 0))
#'user/+a+
user> #+a+
0
user> (swap! +a+ inc)
1
user> #+a+
1
Transients are mutable but should be made persistent after mutation
user> (def t (transient []))
#'user/t
user> (conj! t 1)
#<TransientVector clojure.lang.PersistentVector$TransientVector#658ee462>
user> (persistent! t)
[1]
Understanding Clojure's Persistent Vectors, pt. 1 =>
http://hypirion.com/musings/understanding-persistent-vector-pt-1
Persistent data structure => https://en.wikipedia.org/wiki/Persistent_data_structure
Persistent Data Structures and Managed References =>
http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey
Purely Functional Data Structures by Chris Okasaki refers to an article [1] which appears to contain the original definition of the term persistent:
Ordinary data structures are ephemeral in the sense that making a change to the structure destroys the old version, leaving only the new one. … We call a data structure persistent if it supports access to multiple versions. The structure is partially persistent if all versions can be accessed but only the newest version can be modified, and fully persistent if every version can be both accessed and modified.
[1] James R. Driscoll, Neil Sarnak, Daniel D. Sleator, and Robert E. Tarjan. Making data structures persistent. Journal of Computer and System Sciences, 38(1):86–124, February 1989.
Immutable implies persistent, but persistent does not imply immutable. So you could have something that's persistent but not immutable.
An example of a mutable and persistent data structure is Java's CopyOnWriteArrayList.
Persistence does not imply shared structure, nor does it say anything about performance. Of course, shared structure and good performance are both highly desirable, and are both provided by Clojure's persistent data structures. But it would be quite possible to create something that had no structure sharing and awful performance (see CopyOnWriteArrayList, for example ;-)) but was still persistent.
Basically immutable == can't be changed, and persistent == immutable, with shared structure.
If I have a language where arrays can't be changed, then arrays are immutable. To "change" the array, I must create a new array and copy every element (except the one(s) to be changed) into the new array. This makes any update O(n), where n is the number of elements in the array. This is obviously inefficient for large n.
On the other hand, if I use a persistent data structure instead of an array, then instead of copying every element every time the data structure is "altered", the new version shares most of the same structure with the old one.
The details depend on the structure, but often usually there is a tree involved. If the tree is balanced, replacing an element means creating new copies of nodes along the path from the root to the leaf containing the element. The rest of the nodes are shared with the original version. The length of this path is O(n log(n)). Since the nodes are O(1) size, the entire operation takes O(n log(n)) time and extra space.
Note that not all persistent structures support the same operations efficiently. In Clojure, for example, Lists are singly-linked lists. You can efficiently add and remove elements to/from the front, but that's about it. Vectors, on the other hand, allow you to efficiently get any element and add/remove elements at the back.
I'd love to hear what advice the Clojure gurus here have about managing state in hierarchies. I find I'm often using {:structures {:like {:this {:with {:many 'levels}} } } } and if I want to track changes in state at multiple levels, by throwing atoms around values (atom {:like (atom 'this)} ), I find myself thinking this must be wrong. Is it generally better to use just one atom at the top level, and have none as values in a map ?
Don't use nested atoms in a data structure if at all possible.
The main reason is that immutability is your friend. Clojure is a functional language that thrives on immutable data structures. Most libraries assume immutable data structures. Clojure's STM assumes immutable data structures to get the best possible concurrency. Immutability gives you the opportunity to take consistent snapshots of the entire state at any one instant. Pure functions that operate on immutable data are easy to develop and test.
If you put atoms inside your data structures then you lose all the advantages of immutability and risk making your code very complex - it's a lot harder to reason about a data structure if it contains a lot of mutable components.
Some suggested alternative approaches:
Put your entire data structure in a single ref or atom. This can be a huge data structure with no problem - I once wrote a game where the entire game map was held in a single atom without any difficulty.
Use the various methods that are designed for accessing and changing nested immutable data structures: assoc-in, get-in, update-in etc.
Use recursive functions to make navigating your data structure more managable. If one node of your structure has sub-nodes of the same "type" then it's usually a good hint that you should be using some form of recursive function.
You can use assoc-in, get-in, update-in, and dissoc-in functions to work with nested structures.
They are very convenient, but I don't know if they can handle atoms and such directly. In the worst case you should be able to nest them up to deref, e.g.:
(def m (atom {:like {:this {:nested (atom {:value 5})}}}))
#(get-in #m [:like :this :nested])
; => {:value 5}
(get-in #(get-in #m [:like :this :nested]) [:value])
; => 5
You can use -> to make this more readable:
(-> #m
(get-in [:like :this :nested])
deref
(get-in [:value]))
; => 5
Regarding nested atoms/refs/agents, etc. I think it depends on what you're trying to achieve. It's certainly easier to reason about things, if there's just one of them at the top and the changes are synchronized.
On the other hand, if you don't need this synchronization, you're wasting time in doing it, and you'll be better off with nested atoms/refs/agents.
The bottom line is, I don't think either way is "the right way", they both have their usages.
I would prefer to use one atom at top level as that would make things really simple and also that indicate that the data represent a state which is modified at once n all by an operation. If you put atoms at each level then it would become way too complex to figure out what is going on. Also if in your case the nesting is going way too deep then I would suggest you to sit back and think carefully whether you need such a structure or there can be any better alternate possible because this will certainly lead to complexity until the nested data is recursive (i.e same structure at each level)
I have quite a few records in my program that I end up putting in a map using one of their fields as key. For example
(defrecord Foo. [id afield anotherfield])
And then I'd add that to a map with the id as key. This is all perfectly doable, but a bit tedious, e.g. when adding a new instance of Foo to a map I need to extract the key first. I'm wondering if somewhere in clojure.core a data structure to do this already exist?
Basically I'd like to construct a set of Foo's by giving the set a value to key mapping function (i.e. :id) at construction time of the set, and then have that used when I want to add/find/remove/... a value.
So instead of:
(assoc my-map (:id a-foo) a-foo))
I could do, say:
(conj my-set a-foo)
And more interestingly, merge and merge-with support.
Sounds like a simple case where you would want to use a function to eliminate the "tedious" part.
e.g.
(defn my-assoc [some-map some-record]
(assoc some-map (:id some-record) some-record))
If you are doing this a lot and need different key functions, you might want to try a higher order function:
(defn my-assoc-builder [id-function]
(fn [some-map some-record]
(assoc some-map (id-function some-record) some-record)))
(def my-assoc-by-id (my-assoc-builder :id))
Finally, note that you could do the same with a macro. However a useful general rule with macros is not to use them unless you really need them. Thus in this case, since it can be done easily with a function, I'd recommend sticking to functions.
Well as (AFAIK) there is no such datasctructure (and even if there were, it would probably do same tedious stuff in the background), you can build upon your record fns for desired operations (which will in background do same tedious stuff that needs to be done).
Basically I'd like to construct a set
of Foo's by giving the set a value to
key mapping function (i.e. :id) at
construction time of the set, and then
have that used when I want to
add/find/remove/...
Didn't get this.. If you are holding your records in a set and then want to e.g. find one by id you would have to do even more tidious work looking at every record until you find the right one.. that's O(n), and when using map you will have O(1).
Did I use tedious too much? My suggestion is use map and do some tedious stuff.. It's all 1s and 0s after all :)
I would like to watch for changes to different parts of a Clojure Hash map (accessed via a STM ref), which forms quite a large tree, and on changes to those parts I would like to invoke some registered listeners. How can I do this in clojure as I understand that "add-watch" only works on an entire reference?
Since Clojure maps are immutable, there isn't really such a thing a a change to a single part of a tree from a conceptual perspective.
I can see a couple of good options however:
Add a watch to the entire tree, but test whether the particular part you are interested in has changed. This should be quite quick and easy to test (use "get-in" to look up the right part of the tree)
Marshall all changes to the tree through a library of helper functions, which can intercept the kind of changes you are interested in.
I, too, would watch the whole tree and check subsets with get-in. You can quickly test whether the subtree has been changed by using an identical? test against the previous state. Something like
(defn change-tester [tree path]
(let [orig (get-in tree path)]
(fn [tree]
(not (identical? (get-in tree path) orig)))))
I don't use watchers very often so I don't know the syntax, but you could attach the above function somehow, I'm sure.
Clojure maps are immutable which means they are also thread safe which is a good thing. When you modify one with 'assoc' or similar you are creating a new copy in which your changed values are present. (Note that a full copy isn't made, but rather an efficient technique is employed to create a copy.)
I think perhaps the best way to do what you want is to create your own data structure, because essentially what you are asking for is a mutable HashMap as in Java, but not a Clojure Map.
You could create a wrapper around an existing Java HashMap which overrides the 'put' and 'putAll' methods so that you can detect what's being changed. If you have a HashMap within a HashMap you will want the sub HashMap to be of your new type as well so that you can detect changes at any level.
You might call it something like 'WatchfulHashMap'. Then, you will want to create an instance of this new HashMap like:
(def m (ref (WatchfulHashMap.)))
Thus making a single instance of it modifiable from anywhere in your app.