(def coll [10 27 7 12])
Desired result is:
==> (10 37 44 56)
I tried:
(map #(+ % (next %)) coll)
with no success
reductions can do that:
(reductions + [10 27 7 12])
; → (10 37 44 56)
Two related questions about sequence:
Given a transducer, e.g. (def xf (comp (filter odd?) (map inc))),
What's the relationship between (into [] xf (range 10)) or (into () xf (range 10)), and (sequence xf (range 10))? Is it just that there's no syntax for a lazy sequence that can be used as the second argument of into, so we need a separate function sequence for this purpose? (I know that sequence has another, non-transducer use, coercing a collection into a sequence of one kind or another.)
The Clojure transducers page says, about uses of sequence like the one above,
The resulting sequence elements are incrementally computed. These sequences will consume input incrementally as needed and fully realize intermediate operations. This behavior differs from the equivalent operations on lazy sequences.
To me that sounds as if sequence doesn't return a lazy sequence, yet the docstring for sequence says "When a transducer is supplied, returns a lazy sequence of applications of the transform to the items in coll(s), ....", and in fact (class (sequence xf (range 10))) returns clojure.lang.LazySeq. I think I don't understand the last sentence quoted above from the Clojure transducers page.
(sequence xform from) creates lazy-seq (RT.chunkIteratorSeq) over TransformerIterator to which xform and from are passed. When next value is requested, xform (composition of transformations) is invoked over next value from from.
This behavior differs from the equivalent operations on lazy sequences.
What would be equivalent operations on lazy sequences? With your xf as an example,
applying filter odd? to (range 10), producing intermediate lazy sequence, and applying map inc to intermediate lazy sequence, producing final lazy sequence as result.
I would say that (into to xform from) is similar to (into to (sequence xform from)) when from is some collection which does not implement IReduceInit.
into internally uses (transduce xform conj to from) which does the
same as (reduce (xform conj) to from) and at the end clojure.core.protocols/coll-reduce is called:
(into [] (sequence xf (range 10)))
;[2 4 6 8 10]
(into [] xf (range 10))
;[2 4 6 8 10]
(transduce xf conj [] (range 10))
;[2 4 6 8 10]
(reduce (xf conj) [] (range 10))
;[2 4 6 8 10]
I modified a bit your transducer into:
(defn hof-pr
"Prints char c on each invocation of function f within higher order function"
([hof f c]
(hof (fn [e] (print c) (f e))))
([hof f c coll]
(hof (fn [e] (print c) (f e)) coll)))
(def map-inc-pr (partial hof-pr map inc \m))
(def filter-odd-pr (partial hof-pr filter odd? \f))
(def xf (comp (filter-odd-pr) (map-inc-pr)))
so that it prints out character on each transformation step.
Create s1 in REPL as follows:
(def s1 (into [] xf (range 10)))
ffmffmffmffmffm
s1 is eagerly evaluated (printed f for filtering and m for mapping). No evaluation when s1 is requested again:
s1
[2 4 6 8 10]
Let's create s2:
(def s2 (sequence xf (range 10)))
ffm
Only first item in s2 is evaluated. Next items will be evaluated when requested:
s2
ffmffmffmffm(2 4 6 8 10)
Additionally, create s3, old way:
(def s3 (map-inc-pr (filter-odd-pr (range 10))))
s3
ffffffffffmmmmm(2 4 6 8 10)
As you can see, no evaluation when s3 is defined. When s3 is requested, filtering over 10 elements is applied and after that mapping over remaining 5 elements is applied, producing final sequence.
I didn't find the current answer clear enough, so here goes...
sequence does return a LazySeq, but it is a chunked one, so when you play around with it in the REPL, you will often have the impression it is eager, because your collection will probably be too small, and the chunking will make it look eager. The chunk size I think is a bit dynamic, and it won't always be exactly the same size chunks, but in general it seems to be of size 32. So your transducer will be applied to the input collection 32 elements at a time, lazily.
Here's a simple transducer that just prints the elements it reduces over and returns them untouched:
(defn printer
[xf]
(fn
([] (xf))
([result] (xf result))
([result input]
(println input)
(xf result input))))
If we create a sequence s of 100 elements with it:
(def s
(sequence
printer
(range 100)))
;;> 0
We see that it prints 0, but nothing else. On the call to sequence, the first element will thus be consumed from (range 100), and it will be passed to the xf chain to be transformed, which in our case just prints it. No other elements except the first one have thus been consumed yet.
Now if we take one element from s:
(take 1 s)
;;> 0
;;> 1
;;> 2
;;> 3
;;> 4
;;> 5
;;> 6
;;> 7
;;> 8
;;> 9
;;> 10
;;> 11
;;> 12
;;> 13
;;> 14
;;> 15
;;> 16
;;> 17
;;> 18
;;> 19
;;> 20
;;> 21
;;> 22
;;> 23
;;> 24
;;> 25
;;> 26
;;> 27
;;> 28
;;> 29
;;> 30
;;> 31
;;> 32
We see that it printed the first 32 elements. This is the normal behavior of chunked lazy sequence in Clojure. You can think of it as semi-lazy, in that it consumes chunk-size elements at a time, instead of 1 at a time.
Now if we try to take any element from 1 to 32, nothing else will be printed, because the first 32 elements have already been processed:
(take 1 s)
;; => (0)
(take 10 s)
;; => (0 1 2 3 4 5 6 7 8 9)
(take 24 s)
;; => (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23)
(take 32 s)
;; => (0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31)
Nothing gets printed, and each take returns the expected set of result. I'm using ;; => for return values, and ;;> for printed output.
Okay, now if we take the 33rd element, we expect to see the next chunk of 32 elements being printed:
(take 33 s)
;;> 33
;;> 34
;;> 35
;;> 36
;;> 37
;;> 38
;;> 39
;;> 40
;;> 41
;;> 42
;;> 43
;;> 44
;;> 45
;;> 46
;;> 47
;;> 48
;;> 49
;;> 50
;;> 51
;;> 52
;;> 53
;;> 54
;;> 55
;;> 56
;;> 57
;;> 58
;;> 59
;;> 60
;;> 61
;;> 62
;;> 63
;;> 64
Awesome! So once more, we see that only the next 32 were taken, which brings us to a total of 64 elements now processed.
Well, this demonstrates that sequence called with a transducer does in fact creates a lazy chunked sequence where elements will only be processed when needed (chunk-size at a time).
So what's this about?:
The resulting sequence elements are incrementally computed. These sequences will consume input incrementally as needed and fully realize intermediate operations. This behavior differs from the equivalent operations on lazy sequences.
This is about the order in which the operations happen. With sequence and a transducer:
(sequence (comp A B C) coll)
Will for each elements in the chunk have them go through: A -> B -> C, so you get:
A(e1) -> B(e1) -> C(e1)
A(e2) -> B(e2) -> C(e2)
...
A(e32) -> B(e32) -> C(e32)
While for a normal lazy seq like:
(->> coll A B C)
Will first have all chunked elements go through A, and then have them all go through B and then C:
A(e1)
A(e2)
...
A(e32)
|
B(e1)
B(e2)
...
B(e32)
|
C(e1)
C(e2)
...
C(e32)
This requires an intermediate collection between each step, as the result of A have to be collected into a collection to then loop over and apply B, etc.
We can see this with our previous example:
(def s
(sequence
(comp (filter odd?)
printer
(map vector)
printer)
(range 10)))
(take 1 s)
;;> 1
;;> [1]
;;> 3
;;> [3]
;;> 5
;;> [5]
;;> 7
;;> [7]
;;> 9
;;> [9]
(def l
(->> (range 10)
(filter odd?)
(map #(do (println %) %))
(map vector)
(map #(do (println %) %))))
(take 1 l)
;;> 1
;;> 3
;;> 5
;;> 7
;;> 9
;;> [1]
;;> [3]
;;> [5]
;;> [7]
;;> [9]
See how the first will filter -> vector -> filter -> vector, etc. While the second will filter all -> vector all. Well this is what the quote from the doc means.
Now one more thing, there is a difference in how the chunking is applied as well between the two. With sequence and a transducer, it will process elements until the transducer result has chunk-size count of elements. While in the lazy-seq case, it will process in chunks at each level until all steps have enough for what they need to do.
Here's what I mean:
(def s
(sequence
(comp printer
(filter odd?))
(range 100)))
(take 1 s)
;;> 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
(def l
(->> (range 100)
(map #(do (print % "") %))
(filter odd?)))
(take 1 l)
;;> 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
Here I modified the printing logic to be on the same line, so it doesn't take as much space. And if you look closely, s processed 66 elements of the input range, while l only consumed 32 elements.
The reason for this is what I said above. With sequence, we will continue taking in chunks until we have chunk-size number of results. In this case, the chunk-size is 32, and since we filter on odd?, it takes us two chunks to reach 32 results.
With lazy-seq, it doesn't try and grab the first chunk of results, but only enough chunks from the input to satisfy the logic, in this case, that only needs one chunk of 32 elements from the input for us to find a single odd number to take.
In clojure, I would like to calculate several subvectors out of a big lazy sequence (maybe an infinite one).
The naive way would be to transform the lazy sequence into a vector and then to calculate the subvectors. But when doing that, I am losing the laziness.
I have a big sequence big-sequence and positions, a list of start and end positions. I would like to do the following calculation but lazilly:
(let [positions '((5 7) (8 12) (18 27) (28 37) (44 47))
big-sequence-in-vec (vec big-sequence)]
(map #(subvec big-sequence-in-vec (first %) (second %)) positions))
; ([5 6] [8 9 10 11] [18 19 20 21 22 23 24 25 26] [28 29 30 31 32 33 34 35 36] [44 45 46])
Is it feasible?
Remark: If big-sequence is infinite, vec will never return!
You are asking for a lazy sequence of sub-vectors of a lazy sequence. We can develop it layer by layer as follows.
(defn sub-vectors [spans c]
(let [starts (map first spans) ; the start sequence of the spans
finishes (map second spans) ; the finish sequence of the spans
drops (map - starts (cons 0 starts)) ; the incremental numbers to drop
takes (map - finishes starts) ; the numbers to take
tails (next (reductions (fn [s n] (drop n s)) c drops)) ; the sub-sequences from which the sub-vectors will be taken from the front of
slices (map (comp vec take) takes tails)] ; the sub-vectors
slices))
For example, given
(def positions '((5 7) (8 12) (18 27) (28 37) (44 47)))
we have
(sub-vectors positions (range))
; ([5 6] [8 9 10 11] [18 19 20 21 22 23 24 25 26] [28 29 30 31 32 33 34 35 36] [44 45 46])
Both the spans and the basic sequence are treated lazily. Both can be infinite.
For example,
(take 10 (sub-vectors (partition 2 (range)) (range)))
; ([0] [2] [4] [6] [8] [10] [12] [14] [16] [18])
This works out #schauho's suggestion in a form that is faster than #alfredx's solution, even as improved by OP. Unlike my previous solution, it does not assume that the required sub-vectors are sorted.
The basic tool is an eager analogue of split-at:
(defn splitv-at [n v tail]
(if (and (pos? n) (seq tail))
(recur (dec n) (conj v (first tail)) (rest tail))
[v tail]))
This removes the first n items from tail, appending them to vector v, returning the new v and tail as a vector. We use this to capture just as much more of the big sequence in the vector as is necessary to supply each sub-vector as it comes along.
(defn sub-spans [spans coll]
(letfn [(sss [spans [v tail]]
(lazy-seq
(when-let [[[from to] & spans-] (seq spans)]
(let [[v- tail- :as pair] (splitv-at (- to (count v)) v tail)]
(cons (subvec v- from to) (sss spans- pair))))))]
(sss spans [[] coll])))
For example
(def positions '((8 12) (5 7) (18 27) (28 37) (44 47)))
(sub-spans positions (range))
; ([8 9 10 11] [5 6] [18 19 20 21 22 23 24 25 26] [28 29 30 31 32 33 34 35 36] [44 45 46])
Since subvec works in short constant time, it takes linear time in the
amount of the big sequence consumed.
Unlike my previous solution, it does not forget its head: it keeps
all of the observed big sequence in memory.
(defn pos-pair-to-vec [[start end] big-sequence]
(vec (for [idx (range start end)]
(nth big-sequence idx))))
(let [positions '((5 7) (8 12) (18 27) (28 37) (44 47))
big-seq (range)]
(map #(pos-pair-to-vec % big-seq) positions))
You could use take on the big sequence with the maximum of the positions. You need to compute the values up to this point anyway to compute the subvectors, so you don't really "lose" anything.
The trick is to write a lazy version of subvec using take and drop:
(defn subsequence [coll start end]
(->> (drop start coll)
(take (- end start))))
(let [positions '((5 7) (8 12) (18 27) (28 37) (44 47))
big-sequence (range)]
(map (fn [[start end]] (subsequence big-sequence start end)) positions))
;((5 6) (8 9 10 11) (18 19 20 21 22 23 24 25 26) (28 29 30 31 32 33 34 35 36) (44 45 46))
Hi I just starting with Clojure and I cannot manage to reverse a vector of integers.
;generate a vector of random integers
(defn generate-rands
[x]
(vector (take x (repeatedly #(rand-int 100))))
)
;vector of integers
(def my-vector (generate-rands 10))
;reverse the vector of integers
(def my-vector-reversed (reverse my-vector))
;main
(defn main-app
[]
(println "My Vector: \t\t\t" my-vector)
(println "My Vector Reversed: \t" my-vector-reversed))
The output is
=> (main-app)
My Vector: [(14 49 29 3 66 7 60 60 34 19)]
My Vector Reversed: [((14 49 29 3 66 7 60 60 34 19))]
nil
#'startingclojure.app/main-app
=> (vector? my-vector-reversed)
false
Can someone kindly explain me why my-vector-reversed is not a vector? And how can I reverse the content of 'my-vector'?
Thanks
Also, it's preferable to use rseq instead of reverse when you work with vector or sorted-map. It has constant time, because they are indexed and can be efficiently walked in either direction.
from reverse's doc:
Returns a seq of the items in coll in reverse order. Not lazy.
reverse turns anything into a seq, meaning, a list. in order to get back a vector you should turn it into a vector:
(into [] (reverse [1 2 3 4])) ; =>[4 3 2 1]
In your case, look at your "my-vector": [(14 49 29 3 66 7 60 60 34 19)] - its a vector containing a single element - a seq. so reversing it wouldnt change anything. you should use the same technique to turn your seq into a vector:
(defn generate-rands
[x]
(into [] (take x (repeatedly #(rand-int 100)))))
reverse function returns always a seq, not a vector. You can again convert the result into a vector with something like: (apply vector (reverse [1 2 3]))
I want to take a sequence [44 1 11] generated using (map #(nth %1 0 nil) v1) and feed (map) that into successive calls to the same function. I am just not sure which Clojure builtin or builtins to use other than for.
Here are the details.
Given these two vectors:
(def v1 [[44 2 3 4 5]
[1 6 7 5 10]
[11 12 13 14 15]])
(def v2 [[1 2 3 4 44]
[1 6 7 5 1]
[11 12 13 14 44]])
and this function
(defn ret-non-match-rows
"Expects a sequence of sequences, like what is returned from clojure-csv.
Returns all csv rows that do not match cmp-val at cmp-col-idx."
[s-o-s cmp-val cmp-col-idx]
(filter (complement nil?)
(map #(if (ret-col-match %1 cmp-val cmp-col-idx) nil %1) s-o-s) ))
So I am asking for help in how to feed (map) [44 1 11] into ret-non-match-rows like this
(ret-non-match-rows v2 44 4)
(ret-non-match-rows v2 44 1)
(ret-non-match-rows v2 44 11)
but using Clojure built-ins to generate those individual calls.
Thank You.
Edit:
The following gives me what I want, but I'm wondering if there is a cleaner way to do it.
(def ssn-1 [44 1 11])
(def tst (partial ret-non-match-rows v2 4))
(map #(tst %1) ssn-1)
I get back a sequence of sequences and will parse that to get my results.
Maybe you want this:
(map (partial ret-non-match-rows v2 44) (map first v1))
(assuming the 4 in the first example call is a typo and should be 44)