Clojure question
I have written the following function in clojure:
In the first loop it iterates a list of maps and creates a map.
Then the second loop iterates a list, matches data from the map previously created
and a vector and returns a new map. However, different runs with the same data produce
diffenet results. See below.
(defn resolve-case-per-period
"Constructs a map by matching data existing in input parameter vectors"
[dd case-details periods]
(let [cases ((fn [in]
(loop [case-details in, result-map {}]
(if (= (count case-details) 0)
result-map
(recur (rest case-details)
(assoc result-map
(:priority (first case-details))
(:caseid (first case-details)))))))
case-details)
periods periods]
(info "Mapping cases to periods step 1 completed " cases)
(loop [periods periods, result-map {}]
(if (= (count periods) 0)
result-map
(recur (rest periods)
(conj result-map
{ (str (:period (first periods)))
(get cases (:priority (first periods)))}))))))
The returned output is of a map like the following:
{31-10-10 20 10020101030122036M, 31-10-10 10 10020101030122036M, 31-10-10 21 10020101030122036M, 30-10-10 21 10020101030200157M, 31-10-10 00 10020101030122036M, 31-10-10 11 10020101030122036M, 31-10-10 22 10020101031112152M, 30-10-10 22 10020101030122036M, 31-10-10 01 10020101030122036M, 31-10-10 12 10020101030122036M, 30-10-10 23 10020101030122036M, 31-10-10 02 10020101030122036M, 31-10-10 13 10020101030122036M, 31-10-10 03 10020101030122036M, 31-10-10 14 10020101030122036M, 31-10-10 04 10020101030122036M, 31-10-10 15 10020101030122036M, 31-10-10 05 10020101030122036M, 31-10-10 16 10020101030122036M, 31-10-10 06 10020101030122036M, 31-10-10 17 10020101030122036M, 31-10-10 07 10020101030122036M, 31-10-10 18 10020101030122036M, 31-10-10 08 10020101030122036M, 31-10-10 19 10020101030122036M, 31-10-10 09 10020101030122036M}
Executing the function with the same parameters sometimes yields
{31-10-10 20 nil, 31-10-10 10 nil, 31-10-10 21 nil, 30-10-10 21 nil, 31-10-10 00 nil, 31-10-10 11 nil, 31-10-10 22 nil, 30-10-10 22 nil, 31-10-10 01 nil, 31-10-10 12 nil, 30-10-10 23 nil, 31-10-10 02 nil, 31-10-10 13 nil, 31-10-10 03 nil, 31-10-10 14 nil, 31-10-10 04 nil, 31-10-10 15 nil, 31-10-10 05 nil, 31-10-10 16 nil, 31-10-10 06 nil, 31-10-10 17 nil, 31-10-10 07 nil, 31-10-10 18 nil, 31-10-10 08 nil, 31-10-10 19 nil, 31-10-10 09 nil}
Everything in this function is deterministic and pure (except the info calls, which shouldn't matter), so it should return the same thing every time. You don't provide any sample inputs, so I can't disprove this assumption.
The code is hard to read, and with no context I don't really understand what you're doing. But I went through your code in several refactoring passes to try to make it clearer what's going on. Hopefully this will help someone else who is reading, or even make it clearer to you where your problem is.
First
Remove all the crazy formatting and pointless variable-copying, and use seq instead of testing count=0
(defn resolve-case-per-period
"Constructs a map by matching data existing in input parameter vectors"
[dd case-details periods]
(let [cases (loop [case-details case-details, result-map {}]
(if (seq case-details)
(recur (rest case-details)
(assoc result-map
(:priority (first case-details))
(:caseid (first case-details))))
result-map))]
(info "Mapping cases to periods step 1 completed " cases)
(loop [periods periods, result-map {}]
(if (seq periods)
(recur (rest periods)
(assoc result-map
(str (:period (first periods)))
(get cases (:priority (first periods)))))
(do (info "Mapping cases to periods step 2 completed " result-map)
result-map)))))
Second
Destructuring and if-let instead of primitive keyword lookups, ifs, and seqs:
(defn resolve-case-per-period
"Constructs a map by matching data existing in input parameter vectors"
[dd case-details periods]
(let [cases (loop [case-details case-details, result-map {}]
(if-let [[{:keys [priority case-id]} & more] (seq case-details)]
(recur more
(assoc result-map priority caseid))
result-map))]
(info "Mapping cases to periods step 1 completed " cases)
(loop [periods periods, result-map {}]
(if-let [[{:keys [period priority]} & more] (seq periods)]
(recur more
(assoc result-map
(str period)
(get cases priority)))
(do (info "Mapping cases to periods step 2 completed " result-map)
result-map)))))
Third
At this point it's finally clear that we're just iterating over a sequence and building up a result value as we go, so we can drop all the first/rest nonsense and just use reduce to traverse the sequence for us:
(defn resolve-case-per-period
"Constructs a map by matching data existing in input parameter vectors"
[dd case-details periods]
(let [cases (reduce (fn [result-map {:keys [priority case-id]}]
(assoc result-map priority caseid))
{}, case-details)]
(info "Mapping cases to periods step 1 completed " cases)
(reduce (fn [result-map {:keys [period priority]}]
(assoc result-map
(str period)
(get cases priority)))
{}, periods)))
It will be difficult to answer your question if we don't know what your data (function input) looks like, but a few points:
dd is never used in your function, so you can get rid of that.
It is idiomatic in Clojure to have fairly short functions, so I would suggest factoring the part that you do in let out into another function. That will also make testing and experimentation at the repl easier.
Remapping periods to periods in the let has no effect, so get rid of that.
You shadow a lot of locals (case-details in the let and periods in the loop) , this can be confusing and I would advise against it.
The keys in your resulting map are strings, I presume of the form "31-10-10 20". This is easier to discern with the quotes.
I honestly don't see how this Clojure function could be responsible for giving you different outputs, are you absolutely sure that the input is identical? As an observation in the first case you get BigDecimals for the values, so my guess is that in the second case something that couldn't handle BigDecimals was in contact with the data. But I don't see how this could have happend in the function you provide.
Related
This question already has answers here:
Recursive function causing a stack overflow
(2 answers)
Closed 3 years ago.
I am just learning Clojure and, as usual when lerning new programming languages, one of the first things I tried is implementing the Sieve of Eratosthenes.
I came up with the following solution:
(defn primes
"Calculate all primes up to the given number"
[n]
(loop
[
result []
numbers (range 2 (inc n))
]
(if (empty? numbers)
result
(let [[next & rest] numbers]
(recur (conj result next) (filter (fn [n] (not= 0 (mod n next))) rest)))
)
)
)
It works fine and quite fast for small numbers but for large inputs a StackOverflowError is raised with a suspiciously short stacktrace, eg.:
(primes 100000)
Execution error (StackOverflowError) at (REPL:1).
null
(pst)
StackOverflowError
clojure.lang.LazySeq.sval (LazySeq.java:42)
clojure.lang.LazySeq.seq (LazySeq.java:51)
clojure.lang.RT.seq (RT.java:531)
clojure.core/seq--5387 (core.clj:137)
clojure.core/filter/fn--5878 (core.clj:2809)
clojure.lang.LazySeq.sval (LazySeq.java:42)
clojure.lang.LazySeq.seq (LazySeq.java:51)
clojure.lang.RT.seq (RT.java:531)
clojure.core/seq--5387 (core.clj:137)
clojure.core/filter/fn--5878 (core.clj:2809)
clojure.lang.LazySeq.sval (LazySeq.java:42)
clojure.lang.LazySeq.seq (LazySeq.java:51)
=> nil
I was under the impression that recur implements tail recursion if it is evaluated last in a loop form and my first question is if this is really the case here. My second question is why the stack trace is so short for a StackOverflowError. I also have problems interpreting the stacktrace, ie. what line corresponds to what form.
I am only interested in better or more Clojure-like solutions if they provide insights for these questions, since otherwise I would like to find them by myself. Thank you!
Slightly modified, with comments to describe what is happening on each line, this is your function:
(defn primes
"Calculate all primes up to the given number"
[n]
;; `loop` is not lazy, it runs until it produces a result:
(loop [result []
;; a lazy sequence implemented by clojure.lang.LongRange:
numbers (range 2 (inc n))]
(if (not (nil? (seq numbers)))
result
(let [current (first numbers)
remaining (rest numbers)]
(recur
;; `conj` on a vector returns a vector (non-lazy):
(conj result current)
;; `filter` on a lazy sequence returns a new lazy sequence:
(filter (fn [n] (not= 0 (mod n next)))
remaining))))))
The key is that filter at the end.
Most lazy sequence operations such as filter work by wrapping one lazy sequence in another. On each iteration of the loop, filter adds another layer of lazy sequence, like this:
(filter (fn [n] (not= 0 (mod n 5))) ; returns a LazySeq
(filter (fn [n] (not= 0 (mod n 4))) ; returns a LazySeq
(filter (fn [n] (not= 0 (mod n 3))) ; returns a LazySeq
(filter (fn [n] (not= 0 (mod n 2))) ; returns a LazySeq
remaining))))
The LazySeq objects stack up, each one holding a reference to the previous.
With most lazy sequences, the wrapping doesn't matter because they automatically "unwrap" as soon as you request a value. That happens in LazySeq.seq.
This is one case where it does matter, because your loop builds up so many layers of lazy sequence objects that the nested calls to LazySeq.seq and .sval overflow the maximum stack size allowed by the JVM. That's what you see in the stacktrace.
(This also has memory implications, since a reference to the start of the sequence prevents any of the others from being garbage-collected, what Clojure programmers call "holding on to the head" of the sequence.)
The more general issue with this function is mixing lazy (filter) and non-lazy (loop) operations. That's often a source of problems, so Clojure programmers learn to avoid it out of habit.
As Alan suggests, you can avoid the problem by using only non-lazy operations, such as filterv instead of filter, which forces the lazy sequence into a vector.
Almost any style of lazy evaluation can exhibit some variation of this problem. I described it in Clojure don'ts: concat. For another example see foldr versus foldl in Haskell.
Even without laziness, deeply-nested object trees can cause a StackOverflow, for examples in Java I found xstream#88 or circe#1074.
Here is a version that works:
(ns tst.demo.core
(:use tupelo.core tupelo.test))
(defn primes
"Calculate all primes up to the given number"
[n]
(loop [result []
numbers (range 2 (inc n))]
(if (empty? numbers)
result
(let [[new-prime & candidate-primes] numbers]
(recur
(conj result new-prime)
(filterv (fn [n] (not= 0 (mod n new-prime)))
candidate-primes))) )))
(dotest
(spyx (primes 99999))
)
with result:
-------------------------------
Clojure 1.10.1 Java 13
-------------------------------
Testing tst.demo.core
(primes 99999) => [2 3 5 7 11 13 17 19 23 29 31 37 41 43 47 53 59 61
67 71 73 79 83 89 97 101 103 107 109 113 127 131 137 139 149 151 157 163
167 173 179 181 191 193 197 199 211 223 227 229 233 239 241 251 257 263
269 271 277 281 283 293 307 311 313 317 331 337 347 349 353 359 367 373
379 383 389 397 401 409 419 421 431 433 439 443 449 457 461 463 467 479
487 491 499 503 509 521 523 541 547 557 563 569 571 577 587 593 599 601
...<snip>...
99401 99409 99431 99439 99469 99487 99497 99523 99527 99529 99551 99559
99563 99571 99577 99581 99607 99611 99623 99643 99661 99667 99679 99689
99707 99709 99713 99719 99721 99733 99761 99767 99787 99793 99809 99817
99823 99829 99833 99839 99859 99871 99877 99881 99901 99907 99923 99929
99961 99971 99989 99991]
I renames your variables a bit to make them clearer. If you look closely, you'll see the only substantive difference is the change from the lazy filter to the eager filterv.
Before this change, it worked for a N of 9999 but failed for 99999.
I'm not sure about the implementation of the lazy filter function, but that is clearly the problem.
Strange (& unpredictable) problems like this reinforce my dislike of excessive laziness in Clojure code. It appears you have crashed into a variant of the Clojure Don'ts: Concat problem. In this instance, you code looks like:
(filter ...
(filter ...
(filter ...
(filter ...
...<many, many more>... ))))
Lazy sequences are implemented as nested function calls. Since the last loop that finds prime 99991 is dependent on the first call that finds prime 2, the earlier lazy sequences (and their nested function calls on the stack) cannot be released and you eventually blow the stack.
On my computer, a simple recursive implementation of factorial(N) blows up around N=4400. The above found 9592 primes, so it the specific cause seems to be a bit more complex than 1 stack frame per prime.
Possibly N=32 chunking could play a part.
In order to avoid bugs due to unnecessary laziness, you may be interested in replacing concat with glue, and replacing for with forv. You can also see the full API docs.
I am trying to generate a new key that doesn't exist in my map (atom), then immediately add it to my map and return the key. However, the check for the key and the update are not done atomically. I am wondering how to do this atomically so that it is safe for concurrency.
Ideally this key is short enough to type but hard to guess (so a user can create a session, and his/her friends can join with the key). So 0,1,2,3... is not ideal since a user can try enter sessions n-1. Something like UUID where I don't have to worry about collisions is also not ideal. I was planning on generating a short random string (e.g. "udibwi") but I've used rand-int 25 in the code snippet below to simplify the problem.
I've written a function which randomly generates a key. It checks if the map contains it. If it already does then try a new key. If it doesn't, associate it to my map and then return the key.
This works but I don't think it is safe for multiple threads. Is there a way to do this using atoms or is there a better way?
(defonce sessions (atom {}))
(defn getNewSessionId []
(let [id (rand-int 25)]
(if (contains? #sessions id)
(createNewId)
(do
(swap! sessions assoc id "")
id))))
You're trying to do too much at once. Having that one function generate an ID and update the atom is complicating things. I'd break this down into three functions:
A function that generates an ID based on an existing map
A function that updates a plain, immutable map using the above function
A function that updates an atom (although this will be so simple after implementing the previous two functions that it may not be necessary at all).
Something like:
; Notice how this doesn't deal with atoms at all
(defn generate-new-id [old-map]
(let [new-id (rand-int 25)]
(if (old-map new-id) ; or use "contains?"
(recur old-map) ; Using "recur" so we don't get a StackOverflow
new-id)))
; Also doesn't know anything about the atom
(defn assoc-new-id [old-map]
(let [new-id (generate-new-id old-map)]
(assoc old-map new-id "")))
(defonce data (atom {}))
(defn swap-new-id! []
(swap! data assoc-new-id))
The main changes:
Everything that could be removed from the atom swapping logic was moved to its own function. This allows you to just pass the function handling all the logic to swap! and it will be handled atomically.
Clojure uses dash-case, not camelCase.
I used recur instead of actual recursion so you won't get a StackOverflow while the ID is being brute-forced.
Of course though, this suffers from problems if the available number of IDs left is small. It may take a long time for it to "find" an available ID via brute-force. You might be better off using a "generator" backed by an atom to produce IDs atomically starting from 0:
(defn new-id-producer []
(atom -1))
(defn generate-id [producer]
(swap! producer inc)) ; "swap!" returns the new value that was swapped in
(let [producer (new-id-producer)]
; Could be run on multiple threads at once
(doseq [id (repeatedly 5 #(generate-id producer))]
(println id)))
0
1
2
3
4
=> nil
I tried to write an example of this operating on multiple threads at once:
(let [producer (new-id-producer)
; Emulate the "consumption" of IDs
consume (fn []
(doseq [id (repeatedly 20 #(generate-id producer))]
(println (.getId (Thread/currentThread)) id)))]
(doto (Thread. consume)
(.start))
(doto (Thread. consume)
(.start)))
37 0
3738 1
38 3
38 4
38 5
38 6
38 7
38 8
38 9
38 10
38 11
38 12
38 13
38 14
38 15
38 16
38 17
38 18
38 19
38 20
38 21
2
37 22
37 23
37 24
37 25
37 26
37 27
37 28
37 29
37 30
37 31
37 32
37 33
37 34
37 35
37 36
37 37
37 38
37 39
But the un-synchronized nature of the printing to the outstream made this output a mess. If you squint a bit though, you can see that the threads (with Thread IDs of 37 and 38) are taking turns.
If you need the new ID returned, the only clean way I know of that doesn't involve locking is to use a second atom to get the returned ID out of the swapping function. This requires getting rid of assoc-new-id:
(defn generate-new-id [old-map]
(let [new-id (rand-int 25)]
(if (old-map new-id)
(recur old-map)
new-id)))
(defn swap-new-id! [old-map]
(let [result-atom (atom nil)]
(swap! data (fn [m]
(let [id (generate-new-id m)]
(reset! result-promise id) ; Put the ID in the result atom
(assoc m id ""))))
#result-promise)) ; Then retrieve it here
Or, if a very inefficient solution is fine and you're using Clojure 1.9.0, you can just search the maps to find what key was added using clojure.set.difference:
(defn find-new-id [old-map new-map]
(clojure.set/difference (set (keys new-map))
(set (keys old-map))))
(defn swap-new-id! []
(let [[old-map new-map] (swap-vals! data assoc-new-id)] ; New in 1.9.0
(find-new-id new-map old-map)))
But again, this is very inefficient. It requires two iterations of each map.
Can you please update your question with the reason you are trying to do this? There are almost certainly better solutions than the one you propose.
If you really want to generate unique keys for a map, there are 2 easy answers.
(1) For coordinated keys, you could use an atom to hold an integer of the last key generated.
(def last-map-key (atom 0))
(defn new-map-key (swap! last-map-key inc))
which is guaranteed to generate unique new map keys.
(2) For uncoordinated keys, use a UUID as with clj-uuid/v1
(3) If you really insist on your original algorithm, you could use a Clojure ref, but that is an abuse of it's intended purpose.
You can store the information about which id was the last one in the atom as well.
(defonce data
(atom {:sessions {}
:latest-id nil}))
(defn generate-session-id [sessions]
(let [id (rand-int 25)]
(if (contains? sessions id)
(recur sessions)
id)))
(defn add-new-session [{:keys [sessions] :as data}]
(let [id (generate-session-id sessions)]
(-> data
(assoc-in [:sessions id] {})
(assoc :latest-id id))))
(defn create-new-session! []
(:latest-id (swap! data add-new-session)))
As Carcigenicate shows, by using swap-vals! it is derivable from the before and after states, but it's simpler to just keep around.
this setup is straight outta the docs here:
https://clojuredocs.org/clojure.core/commute
I'll just copy the code as is, with my comments:
(def counter (ref 0))
(defn alter-inc! [counter]
(dosync (Thread/sleep 100) (alter counter inc)))
(defn commute-inc! [counter]
(dosync (Thread/sleep 100) (commute counter inc)))
(defn bombard-counter! [n f counter]
(apply pcalls (repeat n #(f counter))))
(dosync (ref-set counter 0))
Running with the alter produces the randomly ordered list and takes 2000 ms, like in the example:
> (time (doall (bombard-counter! 20 alter-inc! counter)))
"Elapsed time: 2078.859995 msecs"
(7 6 1 5 4 2 3 9 12 10 8 14 11 13 15 18 16 17 20 19)
But running with commute does something very different from the claim in the official doc - I get duplicates:
> (time (doall (bombard-counter! 20 commute-inc! counter)))
"Elapsed time: 309.615195 msecs"
(5 1 1 6 5 4 1 8 8 10 10 12 14 13 15 16 17 18 19 20)
And that's definitely not the result promised in the docs! The difference in the running time is as advertised, but what with the duplicates? I'm prone to typos, so I've re-done it from scratch - same problem.
“commute returns the new value of the ref. However, the last in-transaction value you see from a commute will not always match the end-of-transaction value of a ref, because of reordering. If another transaction sneaks in and alters a ref that you are trying to commute, the STM will not restart your transaction. Instead, it will simply run your commute function again, out of order. Your transaction will never even see the ref value that your commute function finally ran against."
Since Clojure’s STM can reorder commutes behind your back, you can use
them only when you do not care about ordering.”
Excerpt From: Stuart Halloway. “Programming Clojure.”
This is the reason you see out of order update results in your output.
I have a lazy-seq where each item takes some time to calculate:
(defn gen-lazy-seq [size]
(for [i (range size)]
(do
(Thread/sleep 1000)
(rand-int 10))))
Is it possible to evaluate this sequence step by step and print the results. When I try to process it with for or doseq clojure always realizes the whole lazy-seq before printing anything out:
(doseq [item (gen-lazy-seq 10)]
(println item))
(for [item (gen-lazy-seq 10)]
(println item))
Both expressions will wait for 10 seconds before printing anything out. I have looked at doall and dorun as a solution, but they require that the lazy-seq producing function contain the println. I would like to define a lazy-seq producing function and lazy-seq printing function separately and make them work together item by item.
Motivation for trying to do this:
I have messages coming in over a network, and I want to start processing them before all have been received. At the same time it would be nice to save all messages corresponding to a query in a lazy-seq.
Edit 1:
JohnJ's answer shows how to create a lazy-seq that will be evaluated step by step. I would like to know how to evaluate any lazy-seq step by step.
I'm confused because running (chunked-seq? (gen-lazy-seq 10)) on gen-lazy-seq as defined above OR as defined in JohnJ's answer both return false. So then the problem can't be that one creates a chunked sequence and the other doesn't.
In this answer, a function seq1 which turns a chunked lazy-seq into a non-chunked one is shown. Trying that function still gives the same problem with delayed output. I thought that maybe the delay has to do with the some sort of buffering in the repl, so I tried to also print the time when each item in the seq is realized:
(defn seq1 [s]
(lazy-seq
(when-let [[x] (seq s)]
(cons x (seq1 (rest s))))))
(let [start-time (java.lang.System/currentTimeMillis)]
(doseq [item (seq1 (gen-lazy-seq 10))]
(let [elapsed-time (- (java.lang.System/currentTimeMillis) start-time)]
(println "time: " elapsed-time "item: " item))))
; output:
time: 10002 item: 1
time: 10002 item: 8
time: 10003 item: 9
time: 10003 item: 1
time: 10003 item: 7
time: 10003 item: 2
time: 10004 item: 0
time: 10004 item: 3
time: 10004 item: 5
time: 10004 item: 0
Doing the same thing with JohnJ's version of gen-lazy-seq works as expected
; output:
time: 1002 item: 4
time: 2002 item: 1
time: 3002 item: 6
time: 4002 item: 8
time: 5002 item: 8
time: 6002 item: 4
time: 7002 item: 5
time: 8002 item: 6
time: 9003 item: 1
time: 10003 item: 4
Edit 2:
It's not only sequences generated with for which have this problem. This sequence generated with map cannot be processed step by step regardless of seq1 wrapping:
(defn gen-lazy-seq [size]
(map (fn [_]
(Thread/sleep 1000)
(rand-int 10))
(range 0 size)))
But this sequence, also created with map works:
(defn gen-lazy-seq [size]
(map (fn [_]
(Thread/sleep 1000)
(rand-int 10))
(repeat size :ignored)))
Clojure's lazy sequences are often chunked. You can see the chunking at work in your example if you take large sizes (it will be helpful to reduce the thread sleep time in this case). See also these related SO posts.
Though for seems to be chunked, the following is not and works as desired:
(defn gen-lazy-seq [size]
(take size (repeatedly #(do (Thread/sleep 1000)
(rand-int 10)))))
(doseq [item (gen-lazy-seq 10)]
(println item))
"I have messages coming in over a network, and I want to start processing them before all have been received." Chunked or no, this should actually be the case if you process them lazily.
I have an atom,
(def a (atom {:a <some-value>}))
and it needs to be updated continuously, what would be the most memory efficient call in the long run...?
(swap! a assoc :a <next-value>)
or
(swap! a (fn [_] {:a <next-value>}))
Intuitively, based on the talks I have heard on persistent structures, I'm thinking that the second way would be a little slower but better in the long-run... but would like a second opinion.
The first form doesn't work.
Memory efficiency is irrelevant: how you get to the new value has no impact on the long-term amount of memory usage once the old value is thrown away.
You appear to want reset!, not swap!.
Consider why you're updating an atom so often, especially if you don't care about its previous value at all. You can usually accomplish something similar more easily with a purely functional approach, or at least with a swap! function that takes the old value into account.
Following up on Ankur's advice as well as a shameless plug for my wrapper of the sigar analytics lib https://github.com/zcaudate/sigmund I ran some quick and dirty diagnostics for update-in and reset!
the code:
(ns test-memory
(:require [sigmund.core :as sig])
(:import java.lang.management.ManagementFactory))
(def counter (atom 0))
(def data (atom {:data {:a 0
:b 0}}))
(defn update-loop [a]
(swap! a update-in [:data :a] (fn [_] (Math/random)))
(swap! a update-in [:data :b] (fn [_] (Math/random)))
(swap! counter + 2)
(recur a))
(defn update-loop1 [a]
(reset! a {:data {:a (Math/random)
:b (:b (:data #a))}})
(reset! a {:data {:b (Math/random)
:a (:a (:data #a))}})
(swap! counter + 2)
(recur a))
(defn print-loop [sec]
(println "date: " (.toString (java.util.Date.)))
(println "memory: " (/ (:resident (sig/ps-memory)) 1000000.) "MB")
(println "counter: " #counter)
(println "")
(println "")
(Thread/sleep (* 1000 sec))
(recur sec))
(def loop1 (future (update-loop1 data)))
(def loop2 (future (update-loop1 data)))
(def loop-pr (future (print-loop 60)))
Results for the update-in loop:
date: Tue Oct 09 21:13:06 EST 2012
memory: 152.072192 MB
counter: 0
date: Tue Oct 09 21:15:06 EST 2012
memory: 157.904896 MB
counter: 53109426
date: Tue Oct 09 21:18:06 EST 2012
memory: 158.007296 MB
counter: 134171090
date: Tue Oct 09 21:21:06 EST 2012
memory: 157.896704 MB
counter: 214766350
date: Tue Oct 09 21:23:06 EST 2012
memory: 158.011392 MB
counter: 268002504
As you can see, gc does kick in but the data structure is definitely growing.
Results for the reset loop
date: Tue Oct 09 21:25:01 EST 2012
memory: 157.667328 MB
counter: 0
date: Tue Oct 09 21:26:01 EST 2012
memory: 158.86336 MB
counter: 215137676
date: Tue Oct 09 21:27:01 EST 2012
memory: 150.474752 MB
counter: 428276080
date: Tue Oct 09 21:30:02 EST 2012
memory: 150.478848 MB
counter: 1052419088
date: Tue Oct 09 21:33:02 EST 2012
memory: 150.663168 MB
counter: 1697444032
date: Tue Oct 09 21:36:02 EST 2012
memory: 150.77376 MB
counter: 2360045388
reset! is slower but takes less memory.