I need to modify map function behavior to provide mapping not with minimum collection size but with maximum and use zero for missing elements.
Standard behavior:
(map + [1 2 3] [4 5 6 7 8]) => [5 7 9]
Needed behavior:
(map + [1 2 3] [4 5 6 7 8]) => [5 7 9 7 8]
I wrote function to do this, but it seems not very extensible with varargs.
(defn map-ext [f coll1 coll2]
(let [mx (max (count coll1) (count coll2))]
(map f
(concat coll1 (repeat (- mx (count coll1)) 0))
(concat coll2 (repeat (- mx (count coll2)) 0)))))
Is there a better way to do this?
Your method is concise, but inefficient (it calls count). A more efficient solution, which does not require the entirety of its input sequences to be stored in memory follows:
(defn map-pad [f pad & colls]
(lazy-seq
(let [seqs (map seq colls)]
(when (some identity seqs)
(cons (apply f (map #(or (first %) pad) seqs))
(apply map-pad f pad (map rest seqs)))))))
Used like this:
user=> (map-pad + 0 [] [1] [1 1] (range 1 10))
(3 3 3 4 5 6 7 8 9)
Edit: Generalized map-pad to arbitrary arity.
Another lazy variant, usable with an arbitrary number of input sequences:
(defn map-ext [f ext & seqs]
(lazy-seq
(if (some seq seqs)
(cons (apply f (map #(if (seq %) (first %) ext) seqs))
(apply map-ext f ext (map rest seqs)))
())))
Usage:
user> (map-ext + 0 [1 2 3] [4 5 6 7 8])
(5 7 9 7 8)
user> (map-ext + 0 [1 2 3] [4 5 6 7 8] [3 4])
(8 11 9 7 8)
If you just want it to work for any number of collections, try:
(defn map-ext [f & colls]
(let [mx (apply max (map count colls))]
(apply map f (map #(concat % (repeat (- mx (count %)) 0)) colls))))
Clojure> (map-ext + [1 2] [1 2 3] [1 2 3 4])
(3 6 6 4)
I suspect there may be better solutions though (as Trevor Caira suggests, this solution isn't lazy due to the calls to count).
How about that:
(defn map-ext [f x & xs]
(let [colls (cons x xs)
res (apply map f colls)
next (filter not-empty (map #(drop (count res) %) colls))]
(if (empty? next) res
(lazy-seq (concat res (apply map-ext f next))))))
user> (map-ext + [1 2 3] [4] [5 6] [7 8 9 10])
(17 16 12 10)
Along the lines of #LeNsTR's solution, but simpler and faster:
(defn map-ext [f & colls]
(lazy-seq
(let [colls (filter seq colls)
firsts (map first colls)
rests (map rest colls)]
(when (seq colls)
(cons (apply f firsts) (apply map-ext f rests))))))
(map-ext + [1 2 3] [4] [5 6] [7 8 9 10])
;(17 16 12 10)
I've just noticed Michał Marczyk's accepted solution, which is superior: it deals properly with asymmetric mapping functions such as -.
We can make Michał Marczyk's answer neater by using the convention - which many core functions follow - that you get a default or identity value by calling the function with no arguments. For examples:
(+) ;=> 0
(concat) ;=> ()
The code becomes
(defn map-ext [f & seqs]
(lazy-seq
(when (some seq seqs)
(cons (apply f (map #(if (seq %) (first %) (f)) seqs))
(apply map-ext f (map rest seqs)))
)))
(map-ext + [1 2 3] [4 5 6 7 8] [3 4])
;(8 11 9 7 8)
I've made the minimum changes. It could be speeded up a bit.
We may need a function that will inject such a default value into a function that lacks it:
(defn with-default [f default]
(fn
([] default)
([& args] (apply f args))))
((with-default + 6)) ;=> 6
((with-default + 6) 7 8) ;=> 15
This could be speeded up or even turned into a macro.
Related
I'd like to have a function, such that,
(f '([1 4 7] [2 5 9] [3 6]))
would give
([1 2 3] [4 5 6] [7 9])
I tried
(apply map vector '([1 4 7] [2 5 9] [3 6]))
would only produce:
([1 2 3] [4 5 6])
I find it hard to describe my requirements that it's difficult for me to search for a ready solution.
Please help me either to improve my description, or pointer to a solution.
Thanks in advance!
I'd solve a more general problem which means you might reuse that function in the future. I'd change map so that it keeps going past the smallest map.
(defn map-all
"Like map but if given multiple collections will call the function f
with as many arguments as there are elements still left."
([f] (map f))
([f coll] (map f coll))
([f c1 & colls]
(let [step (fn step [cs]
(lazy-seq
(let [ss (keep seq cs)]
(when (seq ss)
(cons (map first ss)
(step (map rest ss)))))))]
(map #(apply f %) (step (conj colls c1))))))
(apply map-all vector '([1 4 7] [2 5 9] [3 6]))
(apply map-all vector '([1 false 7] [nil 5 9] [3 6] [8]))
Note, that as opposed to many other solutions, this one works fine even if any of the sequences contain nil or false.
or this way with loop/recur:
user> (defn transpose-all-2 [colls]
(loop [colls colls res []]
(if-let [colls (seq (filter seq colls))]
(recur (doall (map next colls))
(conj res (mapv first colls)))
res)))
#'user/transpose-all-2
user> (transpose-all-2 x)
[[1 2 3] [4 5 6] [7 9]]
user> (transpose-all-2 '((0 1 2 3) (4 5 6 7) (8 9)))
[[0 4 8] [1 5 9] [2 6] [3 7]]
If you know the maximum length of the vectors ahead of time, you could define
(defn tx [colls]
(lazy-seq
(cons (filterv identity (map first colls))
(tx (map rest colls)))))
then
(take 3 (tx '([1 4 7] [2 5 9] [3 6])))
A simple solution is
(defn transpose-all
[colls]
(lazy-seq
(let [ss (keep seq colls)]
(when (seq ss)
(cons (map first ss) (transpose-all (map rest ss)))))))
For example,
(transpose-all '([1 4 7] [2 5 9] [3 6] [11 12 13 14]))
;((1 2 3 11) (4 5 6 12) (7 9 13) (14))
Here is my own attempt:
(defn f [l]
(let [max-count (apply max (map count l))
l-patched (map (fn [e] (if (< (count e) max-count)
(concat e (take (- max-count (count e)) (repeat nil)))
e)) l)]
(map (fn [x] (filter identity x)) (apply map vector l-patched))
))
Another simple solution:
(->> jagged-list
(map #(concat % (repeat nil)))
(apply map vector)
(take-while (partial some identity)))
A jagged-list like this
'([1 4 7 ]
[2 5 9 ]
[3 6 ]
[11 12 13 14])
will produce:
'([1 2 3 11]
[4 5 6 12]
[7 9 nil 13]
[nil nil nil 14])
Here is another go that doesn't require you to know the vector length in advance:
(defn padzip [& [colls]]
(loop [acc [] colls colls]
(if (every? empty? colls) acc
(recur (conj acc (filterv some?
(map first colls))) (map rest colls)))))
I'm looking for a sequential data structure which is perfect for the following operation. The lenght of the list remains constant, it will never be longer or shorter than a fixed length.
Omit the first item and add x to the end.
(0 1 2 3 4 5 6 7 8 9)
(pop-and-push "10")
(1 2 3 4 5 6 7 8 9 10)
There is only one other reading-operation that has to be done equally often:
(last coll)
pop-and-push could be implemented like this:
(defn pop-and-push [coll x]
(concat (pop coll) ["x"]))
(unfortunately this does not work with sequences produced by e.g. range, it just pops when the sequence declared by the literals '(..) is passed.)
but is this optimal?
The main issue here (once we change "x" to x) is that concat returns a lazy-seq, and lazy-seqs are invalid args to pop.
user=> (defn pop-and-push [coll x] (concat (pop coll) [x]))
#'user/pop-and-push
user=> (pop-and-push [1 2 3] 4)
(1 2 4)
user=> (pop-and-push *1 5)
ClassCastException clojure.lang.LazySeq cannot be cast to clojure.lang.IPersistentStack clojure.lang.RT.pop (RT.java:730)
My naive preference would be to use a vector. This function is easy to implement with subvec.
user=> (defn pop-and-push [v x] (conj (subvec (vec v) 1) x))
#'user/pop-and-push
user=> (pop-and-push [1 2 3] 4)
[2 3 4]
user=> (pop-and-push *1 5)
[3 4 5]
as you can see, this version can actually operate on its own return value
As suggested in the comments, PersistentQueue is made for this situation:
user=> (defn pop-and-push [v x] (conj (pop v) x))
#'user/pop-and-push
user=> (pop-and-push (into clojure.lang.PersistentQueue/EMPTY [1 2 3]) 4)
#object[clojure.lang.PersistentQueue 0x50313382 "clojure.lang.PersistentQueue#7c42"]
user=> (into [] *1)
[2 3 4]
user=> (pop-and-push *2 5)
#object[clojure.lang.PersistentQueue 0x4bd31064 "clojure.lang.PersistentQueue#8023"]
user=> (into [] *1)
[3 4 5]
The PersistentQueue data structure, though less convenient to use in some ways, is actually optimized for this usage.
I would like to partition a seq, based on a seq of values
(partition-by-seq [3 5] [1 2 3 4 5 6])
((1 2 3)(4 5)(6))
The first input is a seq of split points.
The second input is a seq i would like to partition.
So, that the first list will be partitioned at the value 3 (1 2 3) and the second partition will be (4 5) where 5 is the next split point.
another example:
(partition-by-seq [3] [2 3 4 5])
result: ((2 3)(4 5))
(partition-by-seq [2 5] [2 3 5 6])
result: ((2)(3 5)(6))
given: the first seq (split points) is always a subset of the second input seq.
I came up with this solution which is lazy and quite (IMO) straightforward.
(defn part-seq [splitters coll]
(lazy-seq
(when-let [s (seq coll)]
(if-let [split-point (first splitters)]
; build seq until first splitter
(let [run (cons (first s) (take-while #(<= % split-point) (next s)))]
; build the lazy seq of partitions recursively
(cons run
(part-seq (rest splitters) (drop (count run) s))))
; just return one partition if there is no splitter
(list coll)))))
If the split points are all in the sequence:
(part-seq [3 5 8] [0 1 2 3 4 5 6 7 8 9])
;;=> ((0 1 2 3) (4 5) (6 7 8) (9))
If some split points are not in the sequence
(part-seq [3 5 8] [0 1 2 4 5 6 8 9])
;;=> ((0 1 2) (4 5) (6 8) (9))
Example with some infinite sequences for the splitters and the sequence to split.
(take 5 (part-seq (iterate (partial + 3) 5) (range)))
;;=> ((0 1 2 3 4 5) (6 7 8) (9 10 11) (12 13 14) (15 16 17))
the sequence to be partitioned is a splittee and the elements of split-points (aka. splitter) marks the last element of a partition.
from your example:
splittee: [1 2 3 4 5 6]
splitter: [3 5]
result: ((1 2 3)(4 5)(6))
Because the resulting partitions is always a increasing integer sequence and increasing integer sequence of x can be defined as start <= x < end, the splitter elements can be transformed into end of a sequence according to the definition.
so, from [3 5], we want to find subsequences ended with 4 and 6.
then by adding the start, the splitter can be transformed into sequences of [start end]. The start and end of the splittee is also used.
so, the splitter [3 5] then becomes:
[[1 4] [4 6] [6 7]]
splitter transformation could be done like this
(->> (concat [(first splittee)]
(mapcat (juxt inc inc) splitter)
[(inc (last splittee))])
(partition 2)
there is a nice symmetry between transformed splitter and the desired result.
[[1 4] [4 6] [6 7]]
((1 2 3) (4 5) (6))
then the problem becomes how to extract subsequences inside splittee that is ranged by [start end] inside transformed splitter
clojure has subseq function that can be used to find a subsequence inside ordered sequence by start and end criteria. I can just map the subseq of splittee for each elements of transformed-splitter
(map (fn [[x y]]
(subseq (apply sorted-set splittee) <= x < y))
transformed-splitter)
by combining the steps above, my answer is:
(defn partition-by-seq
[splitter splittee]
(->> (concat [(first splittee)]
(mapcat (juxt inc inc) splitter)
[(inc (last splittee))])
(partition 2)
(map (fn [[x y]]
(subseq (apply sorted-set splittee) <= x < y)))))
This is the solution i came up with.
(def a [1 2 3 4 5 6])
(def p [2 4 5])
(defn partition-by-seq [s input]
(loop [i 0
t input
v (transient [])]
(if (< i (count s))
(let [x (split-with #(<= % (nth s i)) t)]
(recur (inc i) (first (rest x)) (conj! v (first x))))
(do
(conj! v t)
(filter #(not= (count %) 0) (persistent! v))))))
(partition-by-seq p a)
I would like to "chunk" a seq into subseqs the same as partition-by, except that the function is not applied to each individual element, but to a range of elements.
So, for example:
(gather (fn [a b] (> (- b a) 2))
[1 4 5 8 9 10 15 20 21])
would result in:
[[1] [4 5] [8 9 10] [15] [20 21]]
Likewise:
(defn f [a b] (> (- b a) 2))
(gather f [1 2 3 4]) ;; => [[1 2 3] [4]]
(gather f [1 2 3 4 5 6 7 8 9]) ;; => [[1 2 3] [4 5 6] [7 8 9]]
The idea is that I apply the start of the list and the next element to the function, and if the function returns true we partition the current head of the list up to that point into a new partition.
I've written this:
(defn gather
[pred? lst]
(loop [acc [] cur [] l lst]
(let [a (first cur)
b (first l)
nxt (conj cur b)
rst (rest l)]
(cond
(empty? l) (conj acc cur)
(empty? cur) (recur acc nxt rst)
((complement pred?) a b) (recur acc nxt rst)
:else (recur (conj acc cur) [b] rst)))))
and it works, but I know there's a simpler way. My question is:
Is there a built in function to do this where this function would be unnecessary? If not, is there a more idiomatic (or simpler) solution that I have overlooked? Something combining reduce and take-while?
Thanks.
Original interpretation of question
We (all) seemed to have misinterpreted your question as wanting to start a new partition whenever the predicate held for consecutive elements.
Yet another, lazy, built on partition-by
(defn partition-between [pred? coll]
(let [switch (reductions not= true (map pred? coll (rest coll)))]
(map (partial map first) (partition-by second (map list coll switch)))))
(partition-between (fn [a b] (> (- b a) 2)) [1 4 5 8 9 10 15 20 21])
;=> ((1) (4 5) (8 9 10) (15) (20 21))
Actual Question
The actual question asks us to start a new partition whenever pred? holds for the beginning of the current partition and the current element. For this we can just rip off partition-by with a few tweaks to its source.
(defn gather [pred? coll]
(lazy-seq
(when-let [s (seq coll)]
(let [fst (first s)
run (cons fst (take-while #((complement pred?) fst %) (next s)))]
(cons run (gather pred? (seq (drop (count run) s))))))))
(gather (fn [a b] (> (- b a) 2)) [1 4 5 8 9 10 15 20 21])
;=> ((1) (4 5) (8 9 10) (15) (20 21))
(gather (fn [a b] (> (- b a) 2)) [1 2 3 4])
;=> ((1 2 3) (4))
(gather (fn [a b] (> (- b a) 2)) [1 2 3 4 5 6 7 8 9])
;=> ((1 2 3) (4 5 6) (7 8 9))
Since you need to have the information from previous or next elements than the one you are currently deciding on, a partition of pairs with a reduce could do the trick in this case.
This is what I came up with after some iterations:
(defn gather [pred s]
(->> (partition 2 1 (repeat nil) s) ; partition the sequence and if necessary
; fill the last partition with nils
(reduce (fn [acc [x :as s]]
(let [n (dec (count acc))
acc (update-in acc [n] conj x)]
(if (apply pred s)
(conj acc [])
acc)))
[[]])))
(gather (fn [a b] (when (and a b) (> (- b a) 2)))
[1 4 5 8 9 10 15 20 21])
;= [[1] [4 5] [8 9 10] [15] [20 21]]
The basic idea is to make partitions of the number of elements the predicate function takes, filling the last partition with nils if necessary. The function then reduces each partition by determining if the predicate is met, if so then the first element in the partition is added to the current group and a new group is created. Since the last partition could have been filled with nulls, the predicate has to be modified.
Tow possible improvements to this function would be to let the user:
Define the value to fill the last partition, so the reducing function could check if any of the elements in the partition is this value.
Specify the arity of the predicate, thus allowing to determine the grouping taking into account the current and the next n elements.
I wrote this some time ago in useful:
(defn partition-between [split? coll]
(lazy-seq
(when-let [[x & more] (seq coll)]
(lazy-loop [items [x], coll more]
(if-let [[x & more] (seq coll)]
(if (split? [(peek items) x])
(cons items (lazy-recur [x] more))
(lazy-recur (conj items x) more))
[items])))))
It uses lazy-loop, which is just a way to write lazy-seq expressions that look like loop/recur, but I hope it's fairly clear.
I linked to a historical version of the function, because later I realized there's a more general function that you can use to implement partition-between, or partition-by, or indeed lots of other sequential functions. These days the implementation is much shorter, but it's less obvious what's going on if you're not familiar with the more general function I called glue:
(defn partition-between [split? coll]
(glue conj []
(fn [v x]
(not (split? [(peek v) x])))
(constantly false)
coll))
Note that both of these solutions are lazy, which at the time I'm writing this is not true of any of the other solutions in this thread.
Here is one way, with steps split up. It can be narrowed down to fewer statements.
(def l [1 4 5 8 9 10 15 20 21])
(defn reduce_fn [f x y]
(cond
(f (last (last x)) y) (conj x [y])
:else (conj (vec (butlast x)) (conj (last x) y)) )
)
(def reduce_fn1 (partial reduce_fn #(> (- %2 %1) 2)))
(reduce reduce_fn1 [[(first l)]] (rest l))
keep-indexed is a wonderful function. Given a function f and a vector lst,
(keep-indexed (fn [idx it] (if (apply f it) idx))
(partition 2 1 lst)))
(0 2 5 6)
this returns the indices after which you want to split. Let's increment them and tack a 0 at the front:
(cons 0 (map inc (.....)))
(0 1 3 6 7)
Partition these to get ranges:
(partition 2 1 nil (....))
((0 1) (1 3) (3 6) (6 7) (7))
Now use these to generate subvecs:
(map (partial apply subvec lst) ....)
([1] [4 5] [8 9 10] [15] [20 21])
Putting it all together:
(defn gather
[f lst]
(let [indices (cons 0 (map inc
(keep-indexed (fn [idx it]
(if (apply f it) idx))
(partition 2 1 lst))))]
(map (partial apply subvec (vec lst))
(partition 2 1 nil indices))))
(gather #(> (- %2 %) 2) '(1 4 5 8 9 10 15 20 21))
([1] [4 5] [8 9 10] [15] [20 21])
If I use the reductions function like so:
(reductions + [1 2 3 4 5])
Then I get
(1 3 6 10 15)
Which is great - but I'd like to apply a binary function in the same way without the state being carried forward - something like
(magic-hof + [1 2 3 4 5])
leads to
(1 3 5 7 9)
ie it returns the operation applied to the first pair, then steps 1 to the next pair.
Can someone tell me the higher-order function I'm looking for? (Something like reductions)
This is my (non-working) go at it:
(defn thisfunc [a b] [(+ a b) b])
(reduce thisfunc [1 2 3 4 5])
You can do it with map:
(map f coll (rest coll))
And if you want a function:
(defn map-pairwise [f coll]
(map f coll (rest coll)))
And if you really need the first element to remain untouched (thanx to juan.facorro's comment):
(defn magic-hof [f [x & xs :as s]]
(cons x (map f s xs)))
partition will group your seq:
user> (->> [1 2 3 4 5] (partition 2 1) (map #(apply + %)) (cons 1))
(1 3 5 7 9)
So, you want to apply a function to subsequent pairs of elements?
(defn pairwise-apply
[f sq]
(when (seq sq)
(->> (map f sq (next sq))
(cons (first sq)))))
Let's try it:
(pairwise-apply + (range 1 6))
;; => (1 3 5 7 9)
This is sufficient:
(#(map + (cons 0 %) %) [1 2 3 4 5])
;; => (1 3 5 7 9)