I want to rewrite (I'm not sure if the original implementation is lazy of not) a lazy implementation of clojure interleaving using lazy-seq that works like this :
(take 4 (lazy-interleave ’( 1 2 3) ’( a b c)))
(1 a 2 b)
I came up with something like this, but I'm not sure why it doesn't work:
(defn lazy-interleave [v1 v2]
(lazy-seq (concat (list (first v1) (first v2))) (lazy-interleave (next v1) (next v2)))
)
Edit:
Thanks to Arthur's answer, here is a modified working solution:
(defn lazy-interleave [v1 v2]
(lazy-seq
(cons (first v1) (cons (first v2) (lazy-interleave (rest v1) (rest v2))))
)
)
A bit of reformatting reveals the problem:
(defn lazy-interleave [v1 v2]
(lazy-seq
(concat (list (first v1) (first v2)))
(lazy-interleave (next v1) (next v2))))
In other words, you're constructing a lazy sequence that, when realized, will evaluate (concat (list (first v1) (first v2))), ignore the result, and then try to evaluate and return (lazy-interleave (next v1) (next v2)). This call to lazy-interleave does the same thing, again dropping the first elements of v1 and v2, and so on, ad infinitum.
You never get to the bottom because you have no empty check, and so since (next nil) returns nil, it just keeps going even after you exhaust both sequences. You don't get a StackOverflowError because you're using lazy sequences instead of recursion.
A correct implementation would look like this:
(defn lazy-interleave [v1 v2]
(when (and (seq v1) (seq v2))
(lazy-cat [(first v1) (first v2)]
(lazy-interleave (rest v1) (rest v2)))))
interleave is already lazy:
user> (interleave (take 5 (iterate #(do (println "sequence A:" %) (inc %)) 0))
(take 5 (iterate #(do (println "sequence B:" %) (inc %)) 100)))
sequence A: 0
sequence B: 100
sequence A: 1
sequence B: 101
sequence A: 2
sequence B: 102
sequence A: 3
sequence B: 103
(0 100 1 101 2 102 3 103 4 104)
user> (take 4 (interleave (take 5 (iterate #(do (println "sequence A:" %) (inc %)) 0))
(take 5 (iterate #(do (println "sequence B:" %) (inc %)) 100))))
sequence A: 0
sequence B: 100
(0 100 1 101)
And the core of it's implementation looks much like your example:
(lazy-seq
(let [s1 (seq c1) s2 (seq c2)]
(when (and s1 s2)
(cons (first s1) (cons (first s2)
(interleave (rest s1) (rest s2)))))))
Except it also works on more than two sequences, so it has another arity that handles that case.
Related
Why the program runs endlessly?
(defn lazycombine
([s] (lazycombine s []))
([s v] (let [a (take 1 s)
b (drop 1 s)]
(if (= a :start)
(lazy-seq (lazycombine b v))
(if (= a :end)
(lazy-seq (cons v (lazycombine b [])))
(lazy-seq (lazycombine b (conj v a))))))))
(def w '(:start 1 2 3 :end :start 7 7 :end))
(lazycombine w)
I need a function that returns a lazy sequence of elements by taking elements from another sequence of the form [: start 1 2: end: start: 5: end] and combining all the elements between: start and: end into a vector
You need to handle the termination condition - i.e. what should return when input s is empty?
And also the detection of :start and :end should use first instead of (take 1 s). And you can simplify that with destructuring.
(defn lazycombine
([s] (lazycombine s []))
([[a & b :as s] v]
(if (empty? s)
v
(if (= a :start)
(lazy-seq (lazycombine b v))
(if (= a :end)
(lazy-seq (cons v (lazycombine b [])))
(lazy-seq (lazycombine b (conj v a))))))))
(def w '(:start 1 2 3 :end :start 7 7 :end))
(lazycombine w)
;; => ([1 2 3] [7 7])
To reduce cyclomatic complexity a bit, you can use condp to replace couple if:
(defn lazycombine
([s] (lazycombine s []))
([[a & b :as s] v]
(if (empty? s)
v
(lazy-seq
(condp = a
:start (lazycombine b v)
:end (cons v (lazycombine b []))
(lazycombine b (conj v a)))))))
I would do it like so, using take-while:
(ns tst.demo.core
(:use tupelo.core tupelo.test))
(def data
[:start 1 2 3 :end :start 7 7 :end])
(defn end-tag? [it] (= it :end))
(defn start-tag? [it] (= it :start))
(defn lazy-segments
[data]
(when-not (empty? data)
(let [next-segment (take-while #(not (end-tag? %)) data)
data-next (drop (inc (count next-segment)) data)
segment-result (vec (remove #(start-tag? %) next-segment))]
(cons segment-result
(lazy-seq (lazy-segments data-next))))))
(dotest
(println "result: " (lazy-segments data)))
Running we get:
result: ([1 2 3] [7 7])
Note the contract when constructing a sequence recursively using cons (lazy or not). You must return either the next value in the sequence, or nil. Supplying nil to cons is the same as supplying an empty sequence:
(cons 5 nil) => (5)
(cons 5 []) => (5)
So it is convenient to use a when form to test the termination condition (instead of using if and returning an empty vector when the sequence must end).
Suppose we wrote the cons as a simple recursion:
(cons segment-result
(lazy-segments data-next))
This works great and produces the same result. The only thing the lazy-seq part does is to delay when the recursive call takes place. Because lazy-seq is a Clojure built-in (special form), it it is similar to loop/recur and does not consume the stack like ordinary recursion does . Thus, we can generate millions (or more) values in the lazy sequence without creating a StackOverflowError (on my computer, the default maximum stack size is ~4000). Consider the infinite lazy-sequence of integers beginning at 0:
(defn intrange
[n]
(cons n (lazy-seq (intrange (inc n)))))
(dotest
(time
(spyx (first (drop 1e6 (intrange 0))))))
Dropping the first million integers and taking the next one succeeds and requires only a few milliseconds:
(first (drop 1000000.0 (intrange 0))) => 1000000
"Elapsed time: 49.5 msecs"
Why the wrapper lazy-cons is required? There are two functions with the same result.
(defn seq1 [s]
(lazy-seq
(when-let [x (seq s)]
(cons (first x) (seq1 (rest x))))))
(defn seq2 [s]
(when-let [x (seq s)]
(cons (first x) (seq2 (rest x)))))
Both case I got the same result none chunked sequences.
repl.core=> (first (map println (seq1 (range 1000))))
0
nil
repl.core=> (first (map println (seq2 (range 1000))))
0
nil
repl.core=> (chunked-seq? (seq2 (range 1000)))
false
repl.core=> (chunked-seq? (seq1 (range 1000)))
false
The first is lazy. It only evaluates elements of the sequence as needed. The second however is strict and goes through the entire sequence immediately. This can be seen if you add some println calls in each:
(defn seq1 [s]
(lazy-seq
(when-let [x (seq s)]
(println "Seq1" (first x))
(cons (first x) (seq1 (rest x))))))
(defn seq2 [s]
(when-let [x (seq s)]
(println "Seq2" (first x))
(cons (first x) (seq2 (rest x)))))
(->> (range 10)
(seq1)
(take 5))
Seq1 0
Seq1 1
Seq1 2
Seq1 3
Seq1 4 ; Only iterated over what was asked for
=> (0 1 2 3 4)
(->> (range 10)
(seq2)
(take 5))
Seq2 0
Seq2 1
Seq2 2
Seq2 3
Seq2 4
Seq2 5
Seq2 6
Seq2 7
Seq2 8
Seq2 9 ; Iterated over everything immediately
=> (0 1 2 3 4)
So, to answer the question, lazy-seq is only required if you intend the iteration of the sequence to only happen as needed. Prefer lazy solutions if you think you may not need the entire sequence, or the sequence is infinite, or you want to sequence several transformations using map or filter. Use a strict solution if you're likely going to need the entire sequence at some point and/or you need fast random access.
I have completed this problem on hackerrank and my solution passes most test cases but it is not fast enough for 4 out of the 11 test cases.
My solution looks like this:
(ns scratch.core
(require [clojure.string :as str :only (split-lines join split)]))
(defn ascii [char]
(int (.charAt (str char) 0)))
(defn process [text]
(let [parts (split-at (int (Math/floor (/ (count text) 2))) text)
left (first parts)
right (if (> (count (last parts)) (count (first parts)))
(rest (last parts))
(last parts))]
(reduce (fn [acc i]
(let [a (ascii (nth left i))
b (ascii (nth (reverse right) i))]
(if (> a b)
(+ acc (- a b))
(+ acc (- b a))))
) 0 (range (count left)))))
(defn print-result [[x & xs]]
(prn x)
(if (seq xs)
(recur xs)))
(let [input (slurp "/Users/paulcowan/Downloads/input10.txt")
inputs (str/split-lines input)
length (read-string (first inputs))
texts (rest inputs)]
(time (print-result (map process texts))))
Can anyone give me any advice about what I should look at to make this faster?
Would using recursion instead of reduce be faster or maybe this line is expensive:
right (if (> (count (last parts)) (count (first parts)))
(rest (last parts))
(last parts))
Because I am getting a count twice.
You are redundantly calling reverse on every iteration of the reduce:
user=> (let [c [1 2 3]
noisey-reverse #(doto (reverse %) println)]
(reduce (fn [acc e] (conj acc (noisey-reverse c) e))
[]
[:a :b :c]))
(3 2 1)
(3 2 1)
(3 2 1)
[(3 2 1) :a (3 2 1) :b (3 2 1) :c]
The reversed value could be calculated inside the containing let, and would then only need to be calculated once.
Also, due to the way your parts is defined, you are doing linear time lookups with each call to nth. It would be better to put parts in a vector and do indexed lookup. In fact you wouldn't need a reversed parts, and could do arithmetic based on the count of the vector to find the item to look up.
Is there a function that could replace subsequences? For example:
user> (good-fnc [1 2 3 4 5] [1 2] [3 4 5])
;; => [3 4 5 3 4 5]
I know that there is clojure.string/replace for strings:
user> (clojure.string/replace "fat cat caught a rat" "a" "AA")
;; => "fAAt cAAt cAAught AA rAAt"
Is there something similar for vectors and lists?
Does this work for you?
(defn good-fnc [s sub r]
(loop [acc []
s s]
(cond
(empty? s) (seq acc)
(= (take (count sub) s) sub) (recur (apply conj acc r)
(drop (count sub) s))
:else (recur (conj acc (first s)) (rest s)))))
Here is a version that plays nicely with lazy seq inputs. Note that it can take an infinite lazy sequence (range) without looping infinitely as a loop based version would.
(defn sq-replace
[match replacement sq]
(let [matching (count match)]
((fn replace-in-sequence [[elt & elts :as sq]]
(lazy-seq
(cond (empty? sq)
()
(= match (take matching sq))
(concat replacement (replace-in-sequence (drop matching sq)))
:default
(cons elt (replace-in-sequence elts)))))
sq)))
#'user/sq-replace
user> (take 10 (sq-replace [3 4 5] ["hello, world"] (range)))
(0 1 2 "hello, world" 6 7 8 9 10 11)
I took the liberty of making the sequence argument the final argument, since this is the convention in Clojure for functions that walk a sequence.
My previous (now deleted) answer was incorrect because this was not as trivial as I first thought, here is my second attempt:
(defn seq-replace
[coll sub rep]
(letfn [(seq-replace' [coll]
(when-let [s (seq coll)]
(let [start (take (count sub) s)
end (drop (count sub) s)]
(if (= start sub)
(lazy-cat rep (seq-replace' end))
(cons (first s) (lazy-seq (seq-replace' (rest s))))))))]
(seq-replace' coll)))
I'd like to write an implementation to an algorithm that produces an infinite sequence of results, where each element represents the calculation of a single iteration of the algorithm. Using a lazy sequence is convenient, as it decouples the logic of the number of iterations (by using take) and burn-in iterations (by using drop) from the implementation.
Here's an example of two algorithm implementations, one that produces a lazy sequence (yadda-lazy), and one that does not (yadda-loop).
(defn yadda-iter
[v1 v2 v3]
(+ (first v1)
(first v2)
(first v3)))
(defn yadda-lazy
[len]
(letfn [(inner [v1 v2 v3]
(cons (yadda-iter v1 v2 v3)
(lazy-seq (inner (rest v1)
(rest v2)
(rest v3)))))]
(let [base (cycle (range len))]
(inner base
(map #(* %1 %1) base)
(map #(* %1 %1 %1) base)))))
(defn yadda-loop
[len iters]
(let [base (cycle (range len))]
(loop [result nil
i 0
v1 base
v2 (map #(* %1 %1) base)
v3 (map #(* %1 %1 %1) base)]
(if (= i iters)
result
(recur (cons (yadda-iter v1 v2 v3) result)
(inc i)
(rest v1)
(rest v2)
(rest v3))))))
(prn (take 11 (yadda-lazy 4)))
(prn (yadda-loop 4 11))
Is there a way to create a lazy sequence using the same style as loop/recur? I like yadda-loop better, because:
It's more obvious what the initial conditions are and how the algorithm progresses to the next iteration.
It won't suffer from a stack overflow due to tail optimization.
Your loop version would be better written to (1) pull the addition out of the loop so you don't have to recur on so many sequences, and (2) use conj on a vector accumulator so your results are in the same order as your yadda-lazy.
(defn yadda-loop-2 [len iters]
(let [v1 (cycle (range len))
v2 (map * v1 v1)
v3 (map * v1 v2)
s (map + v1 v2 v3)]
(loop [result [], s s, i 0]
(if (= i iters)
result
(recur (conj result (first s)), (rest s), (inc i))))))
However, at this point it becomes clear that the loop is pointless as this is just
(defn yadda-loop-3 [len iters]
(let [v1 (cycle (range len))
v2 (map * v1 v1)
v3 (map * v1 v2)
s (map + v1 v2 v3)]
(into [] (take iters s))))
and we might as well pull out the iters parameter, return simply s and take from it.
(defn yadda-yadda [len]
(let [v1 (cycle (range len))
v2 (map * v1 v1)
v3 (map * v1 v2)]
(map + v1 v2 v3)))
This produces the same results as your yadda-lazy, is also lazy, and is quite clear
(take 11 (yadda-yadda 4)) ;=> (0 3 14 39 0 3 14 39 0 3 14)
You could also, equivalently
(defn yadda-yadda [len]
(as-> (range len) s
(cycle s)
(take 3 (iterate (partial map * s) s))
(apply map + s)))
Addendum
If you are looking for a pattern for converting an eager loop like yours to a lazy-sequence
(loop [acc [] args args] ...) -> ((fn step [args] ...) args)
(if condition (recur ...) acc) -> (when condition (lazy-seq ...)
(recur (conj acc (f ...)) ...) -> (lazy-seq (cons (f ...) (step ...)))
Applying this to your yadda-lazy
(defn yadda-lazy-2 [len iters]
(let [base (cycle (range len))]
((fn step [i, v1, v2, v3]
(when (< i iters)
(lazy-seq
(cons (yadda-iter v1 v2 v3)
(step (inc i), (rest v1), (rest v2), (rest v3))))))
0, base, (map #(* %1 %1) base), (map #(* %1 %1 %1) base))))
And at this point you'd probably want to pull out the iters
(defn yadda-lazy-3 [len]
(let [base (cycle (range len))]
((fn step [v1, v2, v3]
(lazy-seq
(cons (yadda-iter v1 v2 v3)
(step (rest v1), (rest v2), (rest v3)))))
base, (map #(* %1 %1) base), (map #(* %1 %1 %1) base))))
So you can
(take 11 (yadda-lazy-3 4)) ;=> (0 3 14 39 0 3 14 39 0 3 14)
And then you might say, hey, my yadda-iter is just applying + on the first and step is applied on the rest, so why not combine my v1, v2, v3 and make this a bit clearer?
(defn yadda-lazy-4 [len]
(let [base (cycle (range len))]
((fn step [vs]
(lazy-seq
(cons (apply + (map first vs))
(step (map rest vs)))))
[base, (map #(* %1 %1) base), (map #(* %1 %1 %1) base)])))
And lo and behold, you have just re-implemented variadic map
(defn yadda-lazy-5 [len]
(let [base (cycle (range len))]
(map + base, (map #(* %1 %1) base), (map #(* %1 %1 %1) base))))
#A.Webb's answer is perfect, but if your love for loop/recur overcomes his arguments, know that you can still combine both styles of recursion.
For example, take a look at the implementation of range:
(defn range
(...)
([start end step]
(lazy-seq
(let [b (chunk-buffer 32)
comp (cond (or (zero? step) (= start end)) not=
(pos? step) <
(neg? step) >)]
(loop [i start] ;; chunk building through loop/recur
(if (and (< (count b) 32)
(comp i end))
(do
(chunk-append b i)
(recur (+ i step)))
(chunk-cons (chunk b)
(when (comp i end)
(range i end step))))))))) ;; lazy recursive call
Here's another example, an alternate implementation of filter:
(defn filter [pred coll]
(letfn [(step [pred coll]
(when-let [[x & more] (seq coll)]
(if (pred x)
(cons x (lazy-seq (step pred more))) ;; lazy recursive call
(recur pred more))))] ;; eager recursive call
(lazy-seq (step pred coll))))
The Tupelo library has a new lazy-gen/yield feature that mimics generator functions in Python. It can generate a lazy sequence from any point in a looping structure. Here is version of yadda-loop that shows lazy-gen & yield in action:
(ns tst.xyz
(:use clojure.test tupelo.test)
(:require [tupelo.core :as t] ))
(defn yadda-lazy-gen
[len iters]
(t/lazy-gen
(let [base (cycle (range len))]
(loop [i 0
v1 base
v2 (map #(* %1 %1) base)
v3 (map #(* %1 %1 %1) base)]
(when (< i iters)
(t/yield (yadda-iter v1 v2 v3))
(recur
(inc i)
(rest v1)
(rest v2)
(rest v3)))))))
Testing tst.clj.core
(take 11 (yadda-lazy 4)) => (0 3 14 39 0 3 14 39 0 3 14)
(yadda-loop 4 11) => (0 3 14 39 0 3 14 39 0 3 14)
(yadda-lazy-gen 4 11) => (0 3 14 39 0 3 14 39 0 3 14)
Ran 1 tests containing 1 assertions.
0 failures, 0 errors.