I was trying to solve Euler Project problem 1. I have noticed a sequence leading to a quicker solution of every 15th number.
This is the Clojure code
(defn fifteenator [n]
(* 15 (+ (* (+ 1 n) 3) (* (/ (+ (* n n) n) 2) 7))))
for 15 n is 0 for 30 n is 1 and so on.
So I can calculate the nearest number divisible by 15 and do only a few recursive calculations. But still one of the HackerRank test cases times out. Before I start profiling the code I would like to make sure if my reasoning is correct. Is there a quicker way to calculate it, or should I learn to profile Clojure?
I am not sure about your approach. Clojure has excellent support for ranges and filters. With these solving Euler 1 is not too difficult:
(defn euler1
[n]
(reduce +
(filter #(or (= (rem % 5) 0) (= (rem % 3) 0)) (range n))))
Testing if we are getting the right result:
user=> (euler1 10)
23
Related
I am learning clojure. While solving one of the problem, I had to use first + filter. I noted that the filter is running unnecessarily for all the inputs.
How can I make the filter to run lazily so that it need not apply the predicate for the whole input.
The below is an example showing that it is not lazy,
(defn filter-even
[n]
(println n)
(= (mod n 2) 0))
(first (filter filter-even (range 1 4)))
The above code prints
1
2
3
Whereas it need not go beyond 2. How can we make it lazy?
This happens because range is a chunked sequence:
(chunked-seq? (range 1))
=> true
And it will actually take the first 32 elements if available:
(first (filter filter-even (range 1 100)))
1
2
. . .
30
31
32
=> 2
This overview shows an unchunk function that prevents this from happening. Unfortunately, it isn't standard:
(defn unchunk [s]
(when (seq s)
(lazy-seq
(cons (first s)
(unchunk (next s))))))
(first (filter filter-even (unchunk (range 1 100))))
2
=> 2
Or, you could apply list to it since lists aren't chunked:
(first (filter filter-even (apply list (range 1 100))))
2
=> 2
But then obviously, the entire collection needs to be realized pre-filtering.
This honestly isn't something that I've ever been too concerned about though. The filtering function usually isn't too expensive, and 32 element chunks aren't that big in the grand scheme of things.
I have implemented the Sieve of Eratosthenes using Clojure's standard library.
(defn primes [below]
(remove (set (mapcat #(range (* % %) below %)
(range 3 (Math/sqrt below) 2)))
(cons 2 (range 3 below 2))))
I think this should be amenable to parallelism as there is no recursion and the reducer versions of remove and mapcat can be dropped in. Here is what I came up with:
(defn pprimes [below]
(r/foldcat
(r/remove
(into #{} (r/mapcat #(range (* % %) below %)
(into [] (range 3 (Math/sqrt below) 2))))
(into [] (cons 2 (range 3 below 2))))))
I've poured the initial set and the generated multiples into vectors as I understand that LazySeqs can't be folded. Also, r/foldcat is used to finally realize the collection.
My problem is that this is a little slower than the first version.
(time (first (primes 1000000))) ;=> approx 26000 seconds
(time (first (pprimes 1000000))) ;=> approx 28500 seconds
Is there too much overhead from the coordinating processes or am I using reducers wrong?
Thanks to leetwinski this seems to work:
(defn pprimes2 [below]
(r/foldcat
(r/remove
(into #{} (r/foldcat (r/map #(range (* % %) below %)
(into [] (range 3 (Math/sqrt below) 2)))))
(into [] (cons 2 (range 3 below 2))))))
Apparently I needed to add another fold operation in order to map #(range (* % %) below %) in parallel.
(time (first (pprimes 1000000))) ;=> approx 28500 seconds
(time (first (pprimes2 1000000))) ;=> approx 7500 seconds
Edit: The above code doesn't work. r/foldcat isn't concatenating the composite numbers it is just returning a vector of the multiples for each prime number. The final result is a vector of 2 and all the odd numbers. Replacing r/map with r/mapcat gives the correct answer but it is again slower than the original primes.
as far as remember, the r/mapcat and r/remove are not parallel themselves, thy are just producing foldable collections, which are in turn can be subject to parallelization by r/fold. in your case the only parallel operation is r/foldcat, which is according to documentation "Equivalent to (fold cat append! coll)", meaning that you just potentially do append! in parallel, which isn't what you want at all.
To make it parallel you should probably use r/fold with remove as a reduce function and concat as a combine function, but it won't really make your code faster i guess, due to the nature of your algorithm (i mean you will try to remove a big set of items from every chunk of a collection)
I have the following bit of code that produces the correct results:
(ns scratch.core
(require [clojure.string :as str :only (split-lines join split)]))
(defn numberify [str]
(vec (map read-string (str/split str #" "))))
(defn process [acc sticks]
(let [smallest (apply min sticks)
cuts (filter #(> % 0) (map #(- % smallest) sticks))]
(if (empty? cuts)
acc
(process (conj acc (count cuts)) cuts))))
(defn print-result [[x & xs]]
(prn x)
(if (seq xs)
(recur xs)))
(let [input "8\n1 2 3 4 3 3 2 1"
lines (str/split-lines input)
length (read-string (first lines))
inputs (first (rest lines))]
(print-result (process [length] (numberify inputs))))
The process function above recursively calls itself until the sequence sticks is empty?.
I am curious to know if I could have used something like take-while or some other technique to make the code more succinct?
If ever I need to do some work on a sequence until it is empty then I use recursion but I can't help thinking there is a better way.
Your core problem can be described as
stop if count of sticks is zero
accumulate count of sticks
subtract the smallest stick from each of sticks
filter positive sticks
go back to 1.
Identify the smallest sub-problem as steps 3 and 4 and put a box around it
(defn cuts [sticks]
(let [smallest (apply min sticks)]
(filter pos? (map #(- % smallest) sticks))))
Notice that sticks don't change between steps 5 and 3, that cuts is a fn sticks->sticks, so use iterate to put a box around that:
(defn process [sticks]
(->> (iterate cuts sticks)
;; ----- 8< -------------------
This gives an infinite seq of sticks, (cuts sticks), (cuts (cuts sticks)) and so on
Incorporate step 1 and 2
(defn process [sticks]
(->> (iterate cuts sticks)
(map count) ;; count each sticks
(take-while pos?))) ;; accumulate while counts are positive
(process [1 2 3 4 3 3 2 1])
;-> (8 6 4 1)
Behind the scene this algorithm hardly differs from the one you posted, since lazy seqs are a delayed implementation of recursion. It is more idiomatic though, more modular, uses take-while for cancellation which adds to its expressiveness. Also it doesn't require one to pass the initial count and does the right thing if sticks is empty. I hope it is what you were looking for.
I think the way your code is written is a very lispy way of doing it. Certainly there are many many examples in The Little Schema that follow this format of reduction/recursion.
To replace recursion, I usually look for a solution that involves using higher order functions, in this case reduce. It replaces the min calls each iteration with a single sort at the start.
(defn process [sticks]
(drop-last (reduce (fn [a i]
(let [n (- (last a) (count i))]
(conj a n)))
[(count sticks)]
(partition-by identity (sort sticks)))))
(process [1 2 3 4 3 3 2 1])
=> (8 6 4 1)
I've changed the algorithm to fit reduce by grouping the same numbers after sorting, and then counting each group and reducing the count size.
I have a higher order predicate
(defn not-factor-of-x? [x]
(fn [n]
(cond
(= n x) true
(zero? (rem n x)) false
:else true)))
which returns a predicate that checks if the given argument n is not a factor of x.
Now I want to filter a list of numbers and find which are not factors of say '(2 3). One way to do this would be :
(filter (not-factor-of-x? 3) (filter (not-factor-of-x? 2) (range 2 100)))
But one can only type so much. In order to do this dynamically I tried function composition :
(comp (partial filter (not-factor-of-x? 2)) (partial filter (not-factor-of-x? 3)))
And it works. So I tried reducing the filters, like this:
(defn compose-filters [fn1 fn2]
(comp (partial filter fn1) (partial filter fn2)))
(def composed-filter (reduce compose-filters (map not-factor-of-x? '(2 3 5 7 11))))
(composed-filter (range 2 122)) ; returns (2 3 4 5 6 7 8 9 10 .......)
So, why the filter composition is not working as intended ?
There are many ways to compose functions and/or improve your code. Here's one:
(defn factor? [n x]
(and (not= n x) (zero? (rem n x))))
(->> (range 2 100)
(remove #(factor? % 2))
(remove #(factor? % 3)))
;; the same as the above
(->> (range 2 100)
(remove (fn [n] (some #(factor? n %) [2 3]))))
To see your problem with (reduce compose-filters ... let's look a bit at what that actually does. First, it uses filter on the first two predicates and composes them.. The result of that is a new function from sequences to sequences. The next iteration then calls filter on that function, when filter expects a predicate. Every sequence is a truthy value, so that new filter will now never remove any values because it's using a "predicate" which always returns truthy values. So in the end, only the very last filter actually does any filtering - in my REPL your code removes the numbers 22, 33, 44 and so on because 11 is a factor in them. I think the reduce you want to do here is more like
(reduce comp (map (comp (partial partial filter) not-factor-of-x?) '(2 3 5 7 11)))
Note how because we only want to call (partial filter) once per number, you can move that into the mapping step of the mapreduce. As to how I'd do this, considering that you produce all your predicates together:
(map not-factor-of-x? '(2 3 5 7 11))
it seems more natural to me to just combine the predicates at that point using every-pred
(apply every-pred (map not-factor-of-x? '(2 3 5 7 11)))
and use one filter on that predicate. It seems to communicate the intent a little more clearly ("I want values satisfying every one of these preds") and unlike composition of (partial filter ...) it avoids making an intermediate sequence for each predicate.
(In Clojure 1.7+ you can also avoid this by composing the transducer version of filter).
I am hoping to generate all the multiples of two less than 10 using the following code
(filter #(< % 10) (iterate (partial + 2) 2))
Expected output:
(2 4 6 8)
However, for some reason repl just doesn't give any output?
But, the below code works just fine...
(filter #(< % 10) '(2 4 6 8 10 12 14 16))
I understand one is lazy sequence and one is a regular sequence. That's the reason. But how can I overcome this issue if I wish to filter all the number less than 10 from a lazy sequence...?
(iterate (partial + 2) 2)
is an infinite sequence. filter has no way to know that the number of items for which the predicate is true is finite, so it will keep going forever when you're realizing the sequence (see Mark's answer).
What you want is:
(take-while #(< % 10) (iterate (partial + 2) 2))
I think I should note that Diego Basch's answer is not fully correct in its argumentation:
filter has no way to know that the number of items for which the predicate is true is finite, so it will keep going forever
Why should filter know something about that? Actually filter works fine in this case. One can apply filter on a lazy sequence and get another lazy sequence that represent potentially infinite sequence of filtered numbers:
user> (def my-seq (iterate (partial + 2) 2)) ; REPL won't be able to print this
;; => #'user/my-seq
user> (def filtered (filter #(< % 10) my-seq)) ; filter it without problems
;; => #'user/filtered
user>
Crucial detail here is that one should never try to realize (by printing in OP's case) lazy sequence when actual sequence is not finite for sure (so that Clojure knows that).
Of course, this example is only for demonstration purposes, you should use take-while here, not filter.