clojure Atom example codes: why the difference between let and def? - clojure

I read the doc https://clojure.org/reference/atoms and triy the codes about Fibonacci.
According the outputs below, in test 3, it takes as short as expected, using def. But in test 2,it take quite long time, using let instead of def to redefine fib. I wonder why let does NOT work as the same way as def?
(defn fib "original fib function" [n]
(if (<= n 1)
n
(+ (fib (dec n)) (fib (- n 2)))))
(defn memorize [f]
(let [mem (atom {})]
(fn [& args]
(if-let [e (find #mem args)]
(val e)
(let [ret (apply f args)]
(swap! mem assoc args ret)
ret)))))
(deftest test-memorize
(testing "test memoize 1 - using fib"
(println "test 1")
(is (> (time (fib 35)) 0)))
(testing "test memoize 2 - uising `(let [fib (memorize fib)])"
(println "test 2")
(let [fib (memorize fib)]
(is (> (time (fib 35)) 0))))
(testing "test memoize 3 - using `(def fib (memorize fib))"
(println "test 3")
(def fib (memorize fib))
(is (> (time (fib 35)) 0)))
The outputs:
$ lein test :only app.model-test/test-memorize
lein test app.model-test
test 1
"Elapsed time: 1418.226187 msecs"
test 2
"Elapsed time: 1391.479784 msecs"
test 3
"Elapsed time: 0.215439 msecs"
Ran 1 tests containing 3 assertions.
0 failures, 0 errors.

... why let does NOT work as the same way as def? ... Because they serve different purposes.
def creates, or updates the value of, a global var. It does not creat a local binding. (Compare to Racket where define is used at the top level to create global bindings and within functions for local bindings.) https://clojure.org/reference/special_forms#def
let creates local bindings. https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/let
(testing "test memoize 3 - using `(def fib (memorize fib))"
(println "test 3")
(def fib (memorize fib))
(is (> (time (fib 35)) 0)))
Is redefining the global value of fib to be the value returned by memorize fib. When the called with (fib 35) the memorize code will call what used to be named fib. That, original, fib calls, by name, fib which is now the memoized version, of fib to calculate fib 34. The calculation of fib 34 calls fib to calculate fib 33. So when (fib 34) returns and (fib 35) calls (fib 33), but that value is cached already. The recursive implementation of fib creates a tree of computation. By redefining fib at the top to inject memoization the work, on the first call to (fib 35) is trimming the tree to a single branch.
For let to do the same thing would require Clojure to use dynamic scope instead of lexical scope.
Some ways to observe that fib is being redefined at the top. After running you tests, (doc fib) and see that your documentation string is not there. Because fib is no longer defined as the function you wrote.
Swap the order of the let and def tests. Then let will be getting the memoized fib.
If you define fib using fn as such:
(def fib (fn fib [n]
(if (<= n 1)
n (+ (fib (dec n))
(fib (- n 2))))))
The call to fib inside will always go to itself, because it is using the local binding for fib from the fn form.

As Shannon Severance already answered. def creates a global var. let creates a local binding.
Which effect has this difference on your function?
Case 2 (let):
You create a new function (a memoized version of fib) and bind it locally to the name fib. Then you call fib. fib calls fib recursively, but inside defn fib ... fib refers to the globally defined fib, which is not memoized. So effectively, there is no caching.
(testing "test memoize 2 - uising `(let [fib (memorize fib)])"
(println "test 2")
(let [fib (memorize fib)] ;-> new memoized version of fib, which is bound to the name fib
(is (> (time (fib 35)) 0)))); but it is only available inside the let block
Effect on defn fib:
(defn fib "original fib function" [n]
(if (<= n 1)
n
(+ (fib (dec n)) (fib (- n 2))))) ;-> calls fib, but fib refers to the globally defined fib, which is not memoized
Case 3 (def):
You create a memoized version of fib, then you globally redeclare fib's value to be the new memoized version. You call fib. Fib calls itself, but now it refers to the new defined version, which is memoized.
(def fib (memorize fib)) ; -> you create a new global fn fib, which is memoized
(is (> (time (fib 35)) 0))
Effect on defn fib:
(defn fib "original fib function" [n]
(if (<= n 1)
n
(+ (fib (dec n)) (fib (- n 2))))) ; -> now, the memoized version of fib is called

Related

How to end dotimes clojure

I am trying to make a guess the number game in clojure but I keep getting an error saying I can only recur from tail position
(def n (rand-int 100))
(prn n)
(println "You have 10 guesses :D")
(println "HINT: My number is between 1 and 100")
(dotimes [i 10]
(def guess (read-line))
(if (= guess str(n))
(recur (println "Correct!") (println "Incorrect"))))
(I am new to clojure)
dotimes is used to execute the body for sideeffects that exact amount given; there is no means to break - except throwing
loop (or functions) are recur targets. Next you would have to count down the attempts so you can stop, if the user did not guess it:
(loop [attempts 10]
; ...
(recur (dec attempts)))
There are also other problematic things:
Don't def inside other forms. Use let instead.
str(n) will throw, as it will try to call n (ClassCastException java.lang.Long cannot be cast to clojure.lang.IFn)
recuring with println looks fishy, since println returns always nil
How do you end dotimes? You don't. Try using loop instead. There are a lot of issues with your code but that's a start.
though this is discouraged and counterclojurish to even think of short circuiting the execution this way, it is still totally possible with macros (purely for education and fun)
(defmacro return [& x]
`(list '~'return (do ~#x)))
(defmacro dotimes+ [[i n] & body]
`(loop [~i 0 res# nil]
(cond (and (list? res#) (= '~'return (first res#))) (second res#)
(< ~i ~n) (recur (inc ~i) (do ~#body))
:else res#)))
can be used like this:
user> (dotimes+ [i 10]
(println i)
(if (== i 5) (return :short-circuited)))
;; 0
;; 1
;; 2
;; 3
;; 4
;; 5
:short-circuited
user> (dotimes+ [i 10]
(println i)
(if (== i 5) (return)))
;; 0
;; 1
;; 2
;; 3
;; 4
;; 5
nil
user> (dotimes+ [i 10]
(println i))
;; 0
;; 1
;; 2
;; 3
;; 4
;; 5
;; 6
;; 7
;; 8
;; 9
nil
notice, that it still expects the return macro to be called in tail position (similar to recur in loop macro)
(dotimes+ [x 4]
(println "attempt" (inc x))
(let [answer (read-line)]
(println "answer is:" answer)
(if (= answer "yes")
(return "YEAH!!!")
(println "WRONG!"))))

Memoize over one parameter

I have a function which takes two inputs which I would like to memoize. The output of the function only depends on the value of the first input, the value of the second input has no functional effect on the outcome (but it may affect how long it takes to finish). Since I don't want the second parameter to affect the memoization I cannot use memoize. Is there an idiomatic way to do this or will I just have to implement the memoization myself?
I'd recommend using a cache (like clojure.core.cache) for this instead of function memoization:
(defonce result-cache
(atom (cache/fifo-cache-factory {})))
(defn expensive-fun [n s]
(println "Sleeping" s)
(Thread/sleep s)
(* n n))
(defn cached-fun [n s]
(cache/lookup
(swap! result-cache
#(cache/through
(fn [k] (expensive-fun k s))
%
n))
n))
(cached-fun 111 500)
Sleeping 500
=> 12321
(cached-fun 111 600) ;; returns immediately regardless of 2nd arg
=> 12321
(cached-fun 123 600)
Sleeping 600
=> 15129
memoize doesn't support caching only on some args, but's pretty easy to make it yourself:
(defn search* [a b]
(* a b))
(def search
(let [mem (atom {})]
(fn [a b]
(or (when-let [cached (get #mem a)]
(println "retrieved from cache")
cached)
(let [ret (search* a b)]
(println "storing in cache")
(swap! mem assoc a ret)
ret)))))
You can wrap you function into another function (with one parameter) and call it the function with second default parameter. Then you can memoize the new function.
(defn foo
[param1]
(baz param1 default-value))

Why does reducing this lazy sequence slow down this Clojure program 20x?

I have a Clojure program that returns the sum of a lazy sequence of even Fibonacci numbers below n:
(defn sum-of-even-fibonaccis-below-1 [n]
(defn fib [a b] (lazy-seq (cons a (fib b (+ b a)))))
(reduce + (take-while (partial >= n) (take-nth 3 (fib 0 1)))))
(time (dotimes [n 1000] (sum-of-even-fibonaccis-below-1 4000000))) ;; => "Elapsed time: 98.764msecs"
It's not very efficient. But if I don't reduce the sequence and simply return a list of the values (0 2 8 34 144...) it can do its job 20x faster:
(defn sum-of-even-fibonaccis-below-2 [n]
(defn fib [a b] (lazy-seq (cons a (fib b (+ b a)))))
(take-while (partial >= n) (take-nth 3 (fib 0 1))))
(time (dotimes [n 1000] (sum-of-even-fibonaccis-below-2 4000000))) ;; => "Elapsed time: 5.145msecs"
Why is reduce so costly to this lazy Fibonacci sequence, and how can I speed it up without abandoning idiomatic Clojure?
The difference in the execution time is a result of lazyness. In sum-of-even-fibonaccis-below-2 you only produce a lazy seq of Fibonacci numbers which is not realised (dotimes only calls sum-of-even-fibonaccis-below-2 to create a lazy sequence, but does not evaluate all of its contents). So in fact your second time expression doesn't return a list of values but a lazy seq that will produce its elements only when you ask for them.
To force realisation of the lazy sequence you can use dorun if you don't need to preserve it as a value or doall if you want to get the realised seq (be careful with inifinite seqs).
If you measure the second case with sum-of-even-fibonaccis-below-2 wrapped in dorun you will get time comparable to sum-of-even-fibonaccis-below-1.
Results from my machine:
(time (dotimes [n 1000] (sum-of-even-fibonaccis-below-1 4000000))) ;; => "Elapsed time: 8.544193 msecs"
(time (dotimes [n 1000] (dorun (sum-of-even-fibonaccis-below-2 4000000)))) ;; => "Elapsed time: 8.012638 msecs"
I also noticed that you defined your fib function with defn inside another defn. You shouldn't do that as defn will always define function at the top level in your namespace. So your code should look like:
(defn fib [a b] (lazy-seq (cons a (fib b (+ b a)))))
(defn sum-of-even-fibonaccis-below-1 [n]
(reduce + (take-while (partial >= n) (take-nth 3 (fib 0 1)))))
(defn sum-of-even-fibonaccis-below-2 [n]
(take-while (partial >= n) (take-nth 3 (fib 0 1))))
If you do want to define a locally scoped function you can take a look at letfn.
Comment
You can refactor your functions - and give them better names - thus:
(defn fib [a b] (lazy-seq (cons a (fib b (+ b a)))))
(defn even-fibonaccis-below [n]
(take-while (partial >= n) (take-nth 3 (fib 0 1))))
(defn sum-of-even-fibonaccis-below [n]
(reduce + (even-fibonaccis-below n)))
This is easier to understand and therefore to answer.

Is there a simpler way to memoize a recursive let fn?

Let's say you have a recursive function defined in a let block:
(let [fib (fn fib [n]
(if (< n 2)
n
(+ (fib (- n 1))
(fib (- n 2)))))]
(fib 42))
This can be mechanically transformed to utilize memoize:
Wrap the fn form in a call to memoize.
Move the function name in as the 1st argument.
Pass the function into itself wherever it is called.
Rebind the function symbol to do the same using partial.
Transforming the above code leads to:
(let [fib (memoize
(fn [fib n]
(if (< n 2)
n
(+ (fib fib (- n 1))
(fib fib (- n 2))))))
fib (partial fib fib)]
(fib 42))
This works, but feels overly complicated. The question is: Is there a simpler way?
I take risks in answering since I am not a scholar but I don't think so. You pretty much did the standard thing which in fine is a partial application of memoization through a fixed point combinator.
You could try to fiddle with macros though (for simple cases it could be easy, syntax-quote would do name resolution for you and you could operate on that). I'll try once I get home.
edit: went back home and tried out stuff, this seems to be ok-ish for simple cases
(defmacro memoize-rec [form]
(let [[fn* fname params & body] form
params-with-fname (vec (cons fname params))]
`(let [f# (memoize (fn ~params-with-fname
(let [~fname (partial ~fname ~fname)] ~#body)))]
(partial f# f#))))
;; (clojure.pprint/pprint (macroexpand '(memoize-rec (fn f [x] (str (f x))))))
((memoize-rec (fn fib [n]
(if (< n 2)
n
(+ (fib (- n 1))
(fib (- n 2)))))) 75) ;; instant
((fn fib [n]
(if (< n 2)
n
(+ (fib (- n 1))
(fib (- n 2))))) 75) ;; slooooooow
simpler than what i thought!
I'm not sure this is "simpler" per se, but I thought I'd share an approach I took to re-implement letfn for a CPS transformer I wrote.
The key is to introduce the variables, but delay assigning them values until they are all in scope. Basically, what you would like to write is:
(let [f nil]
(set! f (memoize (fn []
<body-of-f>)))
(f))
Of course this doesn't work as is, because let bindings are immutable in Clojure. We can get around that, though, by using a reference type — for example, a promise:
(let [f (promise)]
(deliver! f (memoize (fn []
<body-of-f>)))
(#f))
But this still falls short, because we must replace every instance of f in <body-of-f> with (deref f). But we can solve this by introducing another function that invokes the function stored in the promise. So the entire solution might look like this:
(let [f* (promise)]
(letfn [(f []
(#f*))]
(deliver f* (memoize (fn []
<body-of-f>)))
(f)))
If you have a set of mutually-recursive functions:
(let [f* (promise)
g* (promise)]
(letfn [(f []
(#f*))
(g []
(#g*))]
(deliver f* (memoize (fn []
(g))))
(deliver g* (memoize (fn []
(f))))
(f)))
Obviously that's a lot of boiler-plate. But it's pretty trivial to construct a macro that gives you letfn-style syntax.
Yes, there is a simpler way.
It is not a functional transformation, but builds on the impurity allowed in clojure.
(defn fib [n]
(if (< n 2)
n
(+ (#'fib (- n 1))
(#'fib (- n 2)))))
(def fib (memoize fib))
First step defines fib in almost the normal way, but recursive calls are made using whatever the var fib contains. Then fib is redefined, becoming the memoized version of its old self.
I would suppose that clojure idiomatic way will be using recur
(def factorial
(fn [n]
(loop [cnt n acc 1]
(if (zero? cnt)
acc
(recur (dec cnt) (* acc cnt))
;; Memoized recursive function, a mash-up of memoize and fn
(defmacro mrfn
"Returns an anonymous function like `fn` but recursive calls to the given `name` within
`body` use a memoized version of the function, potentially improving performance (see
`memoize`). Only simple argument symbols are supported, not varargs or destructing or
multiple arities. Memoized recursion requires explicit calls to `name` so the `body`
should not use recur to the top level."
[name args & body]
{:pre [(simple-symbol? name) (vector? args) (seq args) (every? simple-symbol? args)]}
(let [akey (if (= (count args) 1) (first args) args)]
;; name becomes extra arg to support recursive memoized calls
`(let [f# (fn [~name ~#args] ~#body)
mem# (atom {})]
(fn mr# [~#args]
(if-let [e# (find #mem# ~akey)]
(val e#)
(let [ret# (f# mr# ~#args)]
(swap! mem# assoc ~akey ret#)
ret#))))))
;; only change is fn to mrfn
(let [fib (mrfn fib [n]
(if (< n 2)
n
(+ (fib (- n 1))
(fib (- n 2)))))]
(fib 42))
Timings on my oldish Mac:
original, Elapsed time: 14089.417441 msecs
mrfn version, Elapsed time: 0.220748 msecs

Mutually recursive definitions in Clojure

How do I do mutually recursive definitions in Clojure?
Here is a code in Scala to find prime numbers which uses recursive definitions:
val odds: Stream[Int] = cons(3, odds map { _ + 2 })
val primes: Stream[Int] = cons(2, odds filter isPrime)
def primeDivisors(n: Int) =
primes takeWhile { _ <= Math.ceil(Math.sqrt(n))} filter { n % _ == 0 }
def isPrime(n: Int) = primeDivisors(n) isEmpty
primes take 10
I translated this to Clojure:
(def odds (iterate #(+ % 2) 3))
(def primes (cons 2 (filter is-prime odds)))
(defn prime-divisors [n]
(filter #(zero? (mod n %))
(take-while #(<= % (Math/ceil (Math/sqrt n)))
primes)))
(defn is-prime [n] (empty? (prime-divisors n)))
(take 10 primes)
But writing the definitions in the Clojure REPL one by one gives
java.lang.Exception: Unable to resolve symbol: is-prime in this context (NO_SOURCE_FILE:8)
after I write (def primes (cons 2 (filter is-prime odds))).
Is there a way to do mutually recursive definitions in Clojure?
You need to (declare is-prime) before the first time you reference it.
This is referred to as a "forward declaration".
Greg's answer is correct. However you have to rearrange your code: (def odds ...) (declare primes) (defn prime-divisors ...) (defn prime? ...) (def primes ...). This should do the trick.
The problem is, that the definition of primes is not a function. It is executed immediately and hence tries to dereference the prime? Var which is not bound, yet. Hence the exception. Re-arranging should take care of that.
(Disclaimer: I haven't checked that the code works with the re-arrangement.)
And I think prime? is broken. (prime? 2) should give false, no?