Integer overflow using lazy sequences in Clojure - clojure

I'm just learning to use lazy sequences in Clojure, and I'm not sure what I'm doing wrong in the following code:
(defn sum [seqn]
(reduce + seqn))
(defn fib
([] (concat [0 1] (fib 0 1)))
([a b] (lazy-seq (cons (+ a b) (fib b (+ a b))))))
(defn up-to [n seqn]
(filter (fn [x] (< x n)) seqn))
(sum (up-to 100 (fib))) => ArithmeticException integer overflow clojure.lang.Numbers.throwIntOverflow (Numbers.java:1388)
The numbers being summed shouldn't be larger than 100, so what is causing the integer overflow?

Filtering an infinite seq produces an infinite seq and reducing over this causes filter to keep looking for another matching item even after the predicate stops returning true.
Replace filter with take-while. The infinite sequence generated by (fib) will cause filter to run forever, but before that it will break due to the ArithmeticException you're experiencing. take-while will stop further evaluation of the list after the (fn [x] (< x n)) predicate evaluates to false.
(defn up-to [n seqn]
(take-while (fn [x] (< x n)) seqn))
(sum (up-to 100 (fib))) ;; => 232

starting with clojure 1.3.0 numbers don't auto-promote to bigInt/bigDecimal.
to fix this use +' instead
your 100th fibinachi number is too large for an integer
user> (nth (fib) 100)
354224848179261915075N

Related

Getting Execution error (StackOverflowError) when evaluating a function on larger range of values

I'm learning Clojure and solving exercises from SICP book in the process. I get this error when evaluating (search-for-primes 1001 10000) but when called on smaller values (search-for-primes 101 1000) it works fine.
This seems to be a memory issue but I'm unable to zero in on the cause of it. Would love your help. Thank you. Below is the code.
(defn square [x]
(* x x))
(defn divides? [a b]
(= (rem b a) 0))
(defn find-divisor [n test-divisor]
(cond
(> (square test-divisor) n) n
(divides? test-divisor n) test-divisor
:else (find-divisor n (+ test-divisor 1))))
(defn smallest-divisor [n]
(find-divisor n 2))
;;return true if prime
(defn prime?
[n]
(= n (smallest-divisor n)))
;;return the first of the three consecutive prime numbers
(defn search-for-primes [low high]
(if (< low high)
(cond
(and (prime? low) (prime? (+ low 2)) (prime? (+ low 4))) low
:else (search-for-primes (+ low 2) high))))
You don't show the actual error you're getting, but I'll guess that it has something to do with the stack overflowing.
Unlike Scheme, Clojure does not support tail call elimination, so you probably need to look at loop/recur, see https://clojuredocs.org/clojure.core/loop for example.

Standard Deviation in clojure

I am trying to write a function in clojure to find the standard deviation of a sequence (vector). So far I have defined a function to find the average of a set of numbers, but I am having an issue with a couple of things.
First I am confused over how to use a square root and powers in clojure. Second I am trying to figure out how to pull out each element individually out the vector and subtract the mean from it and then square it.
So far this is my function
(defn mean [a] (/ (reduce + a) (count a)))
(defn standarddev [a] (Math/sqrt (/ (reduce + (map square #(- % (mean a) a))) (- (count a) 1 ))))
As long as you have a double, you can use Java's Math class (https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html) to perform calculations like sqrt. You don't need to do anything special to access the Math class, because Clojure make all java.lang classes available to you w/o import.
You are pretty close.
Assuming you already have the following functions
(defn square [n] (* n n))
(defn mean [a] (/ (reduce + a) (count a)))
2 problems with your standarddev function
(defn standarddev [a] (Math/sqrt (/ (map square (map - a (mean a))) (- (count a) 1 ))))
1) (map - a (mean a))
Doesn't work because you are subtracting a "number" from a "vector".
To fix repeat (mean a) as many times as there are elements in "a"
Easiest and by no means efficient solution would be
(map - a (repeat (mean a)))
2) (map square (map - a (mean a))) Doesn't work because of #1 above and because map returns a "vector".
To fix sum the elements of the vector
(reduce + (map square (map - a (repeat (mean a)))))
Your standard dev function should now be
(defn standarddev [a]
(Math/sqrt (/
(reduce + (map square (map - a (repeat (mean a)))))
(- (count a) 1 ))))
You can gently increase performance by getting rid of the map altogether
(def square #(* % %))
(defn standard-deviation
[a]
(let [mn (mean a)]
(Math/sqrt
(/ (reduce #(+ %1 (square (- %2 mn))) 0 a)
(dec (count a))))))
First I am confused over how to use a square root and powers in clojure.
To square something, just multiply it by itself:
(defn square [n]
(* n n))
If you want a power higher than 2, you could also use an exponentiation function:
(defn exp [x n]
(reduce * (repeat n x)))
Second I am trying to figure out how to pull out each element individually out the vector and subtract the mean from it and then square it.
The Clojure (functional) way of iterating through a seq is to use map. Map takes a function and a collection and returns the result of applying that function to each element of the collection.
(defn squares [avg coll] (map #(square (- % avg)) coll))
Final standard-deviation function, using the above 2 functions and your mean:
(defn standard-deviation [coll]
(let [avg (mean coll)
squares (squares avg coll)
total (count coll)]
(Math/sqrt (/ (reduce + squares) (- total 1)))))
inspiration from: https://github.com/clojure-cookbook/clojure-cookbook/blob/master/01_primitive-data/1-20_simple-statistics.asciidoc
Corrected sample standard deviation, same as in R sd function
(defn sd [abc]
(Math/sqrt
(/ (reduce + (map #(* % %)
(map #(- % (/ (reduce + abc) (count abc))) abc)))
(dec (count abc))
)
)
)

clojure laziness: prevent unneded mapcat results to realize

Consider a query function q that returns, with a delay, some (let say ten) results.
Delay function:
(defn dlay [x]
(do
(Thread/sleep 1500)
x))
Query function:
(defn q [pg]
(lazy-seq
(let [a [0 1 2 3 4 5 6 7 8 9 ]]
(println "q")
(map #(+ (* pg 10) %) (dlay a)))))
Wanted behaviour:
I would like to produce an infinite lazy sequence such that when I take a value only needed computations are evaluated
Wrong but explicative example:
(drop 29 (take 30 (mapcat q (range))))
If I'm not wrong, it needs to evaluate every sequence because it really doesn't now how long the sequences will be.
How would you obtain the correct behaviour?
My attempt to correct this behaviour:
(defn getq [coll n]
(nth
(nth coll (quot n 10))
(mod n 10)))
(defn results-seq []
(let [a (map q (range))]
(map (partial getq a)
(iterate inc 0)))) ; using iterate instead of range, this way i don't have a chunked sequence
But
(drop 43 (take 44 (results-seq)))
still realizes the "unneeded" q sequences.
Now, I verified that a is lazy, iterate and map should produce lazy sequences, so the problem must be with getq. But I can't understand really how it breaks my laziness...perhaps does nth realize things while walking through a sequence? If this would be true, is there a viable alternative in this case or my solution suffers from bad design?

Cleaning up Clojure function

Coming from imperative programming languages, I am trying to wrap my head around Clojure in hopes of using it for its multi-threading capability.
One of the problems from 4Clojure is to write a function that generates a list of Fibonacci numbers of length N, for N > 1. I wrote a function, but given my limited background, I would like some input on whether or not this is the best Clojure way of doing things. The code is as follows:
(fn fib [x] (cond
(= x 2) '(1 1)
:else (reverse (conj (reverse (fib (dec x))) (+ (last (fib (dec x))) (-> (fib (dec x)) reverse rest first))))
))
The most idiomatic "functional" way would probably be to create an infinite lazy sequence of fibonacci numbers and then extract the first n values, i.e.:
(take n some-infinite-fibonacci-sequence)
The following link has some very interesting ways of generating fibonnaci sequences along those lines:
http://en.wikibooks.org/wiki/Clojure_Programming/Examples/Lazy_Fibonacci
Finally here is another fun implementation to consider:
(defn fib [n]
(let [next-fib-pair (fn [[a b]] [b (+ a b)])
fib-pairs (iterate next-fib-pair [1 1])
all-fibs (map first fib-pairs)]
(take n all-fibs)))
(fib 6)
=> (1 1 2 3 5 8)
It's not as concise as it could be, but demonstrates quite nicely the use of Clojure's destructuring, lazy sequences and higher order functions to solve the problem.
Here is a version of Fibonacci that I like very much (I took the implementation from the clojure wikibook: http://en.wikibooks.org/wiki/Clojure_Programming)
(def fib-seq (lazy-cat [0 1] (map + (rest fib-seq) fib-seq)))
It works like this: Imagine you already have the infinite sequence of Fibonacci numbers. If you take the tail of the sequence and add it element-wise to the original sequence you get the (tail of the tail of the) Fibonacci sequence
0 1 1 2 3 5 8 ...
1 1 2 3 5 8 ...
-----------------
1 2 3 5 8 13 ...
thus you can use this to calculate the sequence. You need two initial elements [0 1] (or [1 1] depending on where you start the sequence) and then you just map over the two sequences adding the elements. Note that you need lazy sequences here.
I think this is the most elegant and (at least for me) mind stretching implementation.
Edit: The fib function is
(defn fib [n] (nth fib-seq n))
Here's one way of doing it that gives you a bit of exposure to lazy sequences, although it's certainly not really an optimal way of computing the Fibonacci sequence.
Given the definition of the Fibonacci sequence, we can see that it's built up by repeatedly applying the same rule to the base case of '(1 1). The Clojure function iterate sounds like it would be good for this:
user> (doc iterate)
-------------------------
clojure.core/iterate
([f x])
Returns a lazy sequence of x, (f x), (f (f x)) etc. f must be free of side-effects
So for our function we'd want something that takes the values we've computed so far, sums the two most recent, and returns a list of the new value and all the old values.
(fn [[x y & _ :as all]] (cons (+ x y) all))
The argument list here just means that x and y will be bound to the first two values from the list passed as the function's argument, a list containing all arguments after the first two will be bound to _, and the original list passed as an argument to the function can be referred to via all.
Now, iterate will return an infinite sequence of intermediate values, so for our case we'll want to wrap it in something that'll just return the value we're interested in; lazy evaluation will stop the entire infinite sequence being evaluated.
(defn fib [n]
(nth (iterate (fn [[x y & _ :as all]] (cons (+ x y) all)) '(1 1)) (- n 2)))
Note also that this returns the result in the opposite order to your implementation; it's a simple matter to fix this with reverse of course.
Edit: or indeed, as amalloy says, by using vectors:
(defn fib [n]
(nth (iterate (fn [all]
(conj all (->> all (take-last 2) (apply +)))) [1 1])
(- n 2)))
See Christophe Grand's Fibonacci solution in Programming Clojure by Stu Halloway. It is the most elegant solution I have seen.
(defn fibo [] (map first (iterate (fn [[a b]] [b (+ a b)]) [0 1])))
(take 10 (fibo))
Also see
How can I generate the Fibonacci sequence using Clojure?

Mutually recursive definitions in Clojure

How do I do mutually recursive definitions in Clojure?
Here is a code in Scala to find prime numbers which uses recursive definitions:
val odds: Stream[Int] = cons(3, odds map { _ + 2 })
val primes: Stream[Int] = cons(2, odds filter isPrime)
def primeDivisors(n: Int) =
primes takeWhile { _ <= Math.ceil(Math.sqrt(n))} filter { n % _ == 0 }
def isPrime(n: Int) = primeDivisors(n) isEmpty
primes take 10
I translated this to Clojure:
(def odds (iterate #(+ % 2) 3))
(def primes (cons 2 (filter is-prime odds)))
(defn prime-divisors [n]
(filter #(zero? (mod n %))
(take-while #(<= % (Math/ceil (Math/sqrt n)))
primes)))
(defn is-prime [n] (empty? (prime-divisors n)))
(take 10 primes)
But writing the definitions in the Clojure REPL one by one gives
java.lang.Exception: Unable to resolve symbol: is-prime in this context (NO_SOURCE_FILE:8)
after I write (def primes (cons 2 (filter is-prime odds))).
Is there a way to do mutually recursive definitions in Clojure?
You need to (declare is-prime) before the first time you reference it.
This is referred to as a "forward declaration".
Greg's answer is correct. However you have to rearrange your code: (def odds ...) (declare primes) (defn prime-divisors ...) (defn prime? ...) (def primes ...). This should do the trick.
The problem is, that the definition of primes is not a function. It is executed immediately and hence tries to dereference the prime? Var which is not bound, yet. Hence the exception. Re-arranging should take care of that.
(Disclaimer: I haven't checked that the code works with the re-arrangement.)
And I think prime? is broken. (prime? 2) should give false, no?