This question already has answers here:
How do I avoid Clojure's chunking behavior for lazy seqs that I want to short circuit?
(3 answers)
Closed 8 years ago.
This code:
(first (map (fn [d]
(apply * (repeat d 10)))
(range)))
yealds an integer overflow exception, while this code:
(first (map (fn [d]
(apply * (repeat d 10)))
(range 0 1)))
yealds 1.
Both codes should yeald 1 but for some reason the laziness of range has a strange behaviour. It seems to get chuncks of data instead of only one at a time. Is it possible to make range behave in the desired way?
range is chunked, as a performance optimization. There are a few ways to fix this including using (iterate inc 0) or unchunk (there are a few versions, this one is copied from math.combinatorics)
(defn unchunk
[s]
(lazy-seq
(when (seq s)
(cons (first s)
(unchunk (rest s))))))
user=> (first (map (fn [d]
(apply * (repeat d 10)))
(unchunk (range)))
1
Related
I need help with an assignment that uses Clojure. It is very small but the language is a bit confusing to understand. I need to create a function that behaves like count without actually using the count funtion. I know a loop can be involved with it somehow but I am at a lost because nothing I have tried even gets my code to work. I expect it to output the number of elements in list. For example:
(defn functionname []
...
...)
(println(functionname '(1 4 8)))
Output:3
Here is what I have so far:
(defn functionname [n]
(def n 0)
(def x 0)
(while (< x n)
do
()
)
)
(println(functionname '(1 4 8)))
It's not much but I think it goes something like this.
This implementation takes the first element of the list and runs a sum until it can't anymore and then returns the sum.
(defn recount [list-to-count]
(loop [xs list-to-count sum 0]
(if (first xs)
(recur (rest xs) (inc sum))
sum
)))
user=> (recount '(3 4 5 9))
4
A couple more example implementations:
(defn not-count [coll]
(reduce + (map (constantly 1) coll)))
or:
(defn not-count [coll]
(reduce (fn [a _] (inc a)) 0 coll))
or:
(defn not-count [coll]
(apply + (map (fn [_] 1) coll)))
result:
(not-count '(5 7 8 1))
=> 4
I personally like the first one with reduce and constantly.
I need a function that maps a function only on every other element, e.g.
(f inc '(1 2 3 4))
=> '(2 2 4 4)
I came up with:
(defn flipflop [f l]
(loop [k l, b true, r '()]
(if (empty? k)
(reverse r)
(recur (rest k)
(not b)
(conj r (if b
(f (first k))
(first k)))))))
Is there a prettier way to achieve this ?
(map #(% %2)
(cycle [f identity])
coll)
It's a good idea to look at Clojure's higher level functions before using loop and recur.
user=> (defn flipflop
[f coll]
(mapcat #(apply (fn ([a b] [(f a) b])
([a] [(f a)]))
%)
(partition-all 2 coll)))
#'user/flipflop
user=> (flipflop inc [1 2 3 4])
(2 2 4 4)
user=> (flipflop inc [1 2 3 4 5])
(2 2 4 4 6)
user=> (take 11 (flipflop inc (range))) ; demonstrating laziness
(1 1 3 3 5 5 7 7 9 9 11)
this flipflop doesn't need to reverse the output, it is lazy, and I find it much easier to read.
The function uses partition-all to split the list into pairs of two items, and mapcat to join a series of two element sequences from the calls back into a single sequence.
The function uses apply, plus multiple arities, in order to handle the case where the final element of the partitioned collection is a singleton (the input was odd in length).
also, since you want to apply the function to some specific indiced items in the collection (even indices in this case) you could use map-indexed, like this:
(defn flipflop [f coll]
(map-indexed #(if (even? %1) (f %2) %2) coll))
Whereas amalloy's solution is the one, you could simplify your loop - recur solution a bit:
(defn flipflop [f l]
(loop [k l, b true, r []]
(if (empty? k)
r
(recur (rest k)
(not b)
(conj r ((if b f identity) (first k)))))))
This uses couple of common tricks:
If an accumulated list comes out in the wrong order, use a vector
instead.
Where possible, factor out common elements in a conditional.
What if map and doseq had a baby? I'm trying to write a function or macro like Common Lisp's mapc, but in Clojure. This does essentially what map does, but only for side-effects, so it doesn't need to generate a sequence of results, and wouldn't be lazy. I know that one can iterate over a single sequence using doseq, but map can iterate over multiple sequences, applying a function to each element in turn of all of the sequences. I also know that one can wrap map in dorun. (Note: This question has been extensively edited after many comments and a very thorough answer. The original question focused on macros, but those macro issues turned out to be peripheral.)
This is fast (according to criterium):
(defn domap2
[f coll]
(dotimes [i (count coll)]
(f (nth coll i))))
but it only accepts one collection. This accepts arbitrary collections:
(defn domap3
[f & colls]
(dotimes [i (apply min (map count colls))]
(apply f (map #(nth % i) colls))))
but it's very slow by comparison. I could also write a version like the first, but with different parameter cases [f c1 c2], [f c1 c2 c3], etc., but in the end, I'll need a case that handles arbitrary numbers of collections, like the last example, which is simpler anyway. I've tried many other solutions as well.
Since the second example is very much like the first except for the use of apply and the map inside the loop, I suspect that getting rid of them would speed things up a lot. I have tried to do this by writing domap2 as a macro, but the way that the catch-all variable after & is handled keeps tripping me up, as illustrated above.
Other examples (out of 15 or 20 different versions), benchmark code, and times on a Macbook Pro that's a few years old (full source here):
(defn domap1
[f coll]
(doseq [e coll]
(f e)))
(defn domap7
[f coll]
(dorun (map f coll)))
(defn domap18
[f & colls]
(dorun (apply map f colls)))
(defn domap15
[f coll]
(when (seq coll)
(f (first coll))
(recur f (rest coll))))
(defn domap17
[f & colls]
(let [argvecs (apply (partial map vector) colls)] ; seq of ntuples of interleaved vals
(doseq [args argvecs]
(apply f args))))
I'm working on an application that uses core.matrix matrices and vectors, but feel free to substitute your own side-effecting functions below.
(ns tst
(:use criterium.core
[clojure.core.matrix :as mx]))
(def howmany 1000)
(def a-coll (vec (range howmany)))
(def maskvec (zero-vector :vectorz howmany))
(defn unmaskit!
[idx]
(mx/mset! maskvec idx 1.0)) ; sets element idx of maskvec to 1.0
(defn runbench
[domapfn label]
(print (str "\n" label ":\n"))
(bench (def _ (domapfn unmaskit! a-coll))))
Mean execution times according to Criterium, in microseconds:
domap1: 12.317551 [doseq]
domap2: 19.065317 [dotimes]
domap3: 265.983779 [dotimes with apply, map]
domap7: 53.263230 [map with dorun]
domap18: 54.456801 [map with dorun, multiple collections]
domap15: 32.034993 [recur]
domap17: 95.259984 [doseq, multiple collections interleaved using map]
EDIT: It may be that dorun+map is the best way to implement domap for multiple large lazy sequence arguments, but doseq is still king when it comes to single lazy sequences. Performing the same operation as unmask! above, but running the index through (mod idx 1000), and iterating over (range 100000000), doseq is about twice as fast as dorun+map in my tests (i.e. (def domap25 (comp dorun map))).
You don't need a macro, and I don't see why a macro would be helpful here.
user> (defn do-map [f & lists] (apply mapv f lists) nil)
#'user/do-map
user> (do-map (comp println +) (range 2 6) (range 8 11) (range 22 40))
32
35
38
nil
note do-map here is eager (thanks to mapv) and only executes for side effects
Macros can use varargs lists, as the (useless!) macro version of do-map demonstrates:
user> (defmacro do-map-macro [f & lists] `(do (mapv ~f ~#lists) nil))
#'user/do-map-macro
user> (do-map-macro (comp println +) (range 2 6) (range 8 11) (range 22 40))
32
35
38
nil
user> (macroexpand-1 '(do-map-macro (comp println +) (range 2 6) (range 8 11) (range 22 40)))
(do (clojure.core/mapv (comp println +) (range 2 6) (range 8 11) (range 22 40)) nil)
Addendum:
addressing the efficiency / garbage-creation concerns:
note that below I truncate the output of the criterium bench function, for conciseness reasons:
(defn do-map-loop
[f & lists]
(loop [heads lists]
(when (every? seq heads)
(apply f (map first heads))
(recur (map rest heads)))))
user> (crit/bench (with-out-str (do-map-loop (comp println +) (range 2 6) (range 8 11) (range 22 40))))
...
Execution time mean : 11.367804 µs
...
This looks promising because it doesn't create a data structure that we aren't using anyway (unlike mapv above). But it turns out it is slower than the previous (maybe because of the two map calls?).
user> (crit/bench (with-out-str (do-map-macro (comp println +) (range 2 6) (range 8 11) (range 22 40))))
...
Execution time mean : 7.427182 µs
...
user> (crit/bench (with-out-str (do-map (comp println +) (range 2 6) (range 8 11) (range 22 40))))
...
Execution time mean : 8.355587 µs
...
Since the loop still wasn't faster, let's try a version which specializes on arity, so that we don't need to call map twice on every iteration:
(defn do-map-loop-3
[f a b c]
(loop [[a & as] a
[b & bs] b
[c & cs] c]
(when (and a b c)
(f a b c)
(recur as bs cs))))
Remarkably, though this is faster, it is still slower than the version that just used mapv:
user> (crit/bench (with-out-str (do-map-loop-3 (comp println +) (range 2 6) (range 8 11) (range 22 40))))
...
Execution time mean : 9.450108 µs
...
Next I wondered if the size of the input was a factor. With larger inputs...
user> (def test-input (repeatedly 3 #(range (rand-int 100) (rand-int 1000))))
#'user/test-input
user> (map count test-input)
(475 531 511)
user> (crit/bench (with-out-str (apply do-map-loop-3 (comp println +) test-input)))
...
Execution time mean : 1.005073 ms
...
user> (crit/bench (with-out-str (apply do-map (comp println +) test-input)))
...
Execution time mean : 756.955238 µs
...
Finally, for completeness, the timing of do-map-loop (which as expected is slightly slower than do-map-loop-3)
user> (crit/bench (with-out-str (apply do-map-loop (comp println +) test-input)))
...
Execution time mean : 1.553932 ms
As we see, even with larger input sizes, mapv is faster.
(I should note for completeness here that map is slightly faster than mapv, but not by a large degree).
What would be a more idiomatic way to partition a seq based on a seq of integers instead of just one integer?
Here's my implementation:
(defn partition-by-seq
"Return a lazy sequence of lists with a variable number of items each
determined by the n in ncoll. Extra values in coll are dropped."
[ncoll coll]
(let [partition-coll (mapcat #(repeat % %) ncoll)]
(->> coll
(map vector partition-coll)
(partition-by first)
(map (partial map last)))))
Then (partition-by-seq [2 3 6] (range)) yields ((0 1) (2 3 4) (5 6 7 8 9 10)).
Your implementation looks fine, but there could be a more simple solution which uses simple recursion wrapped in lazy-seq(and turns out to be more efficient) than using map and existing partition-by as in your case.
(defn partition-by-seq [ncoll coll]
(if (empty? ncoll)
'()
(let [n (first ncoll)]
(cons (take n coll)
(lazy-seq (partition-by-seq (rest ncoll) (drop n coll)))))))
A variation on Ankur's answer, with a minor addition of laziness and when-let instead of an explicit test for empty?.
(defn partition-by-seq [parts coll]
(lazy-seq
(when-let [s (seq parts)]
(cons
(take (first s) coll)
(partition-by-seq (rest s) (nthrest coll (first s)))))))
(first (reduce (fn [[r l] n]
[(conj r (take n l)) (drop n l)])
[[] (range)]
[2 3 6]))
=> [(0 1) (2 3 4) (5 6 7 8 9 10)]
In a Clojure program, I have a sequence of numbers:
(2 3 4 6 8 1)
I want to find the longest sub-sequence where the items are sequential:
(2 3 4)
I am assuming that it will involve (take-while ...) or (reduce ...).
Any ideas?
Clarification: I need the longest initial list of sequential items. Much easier, I'm sure. Thanks for the solutions to the more difficult problem I initially posed.
If you are only interested in the longest initial sequence, it's a 1-liner:
(defn longest-initial-sequence [[x :as s]]
(take-while identity (map #(#{%1} %2) s (iterate inc x))))
Taking into account the OP's comment on the question -- which completely changes the game! -- this can be written very simply:
(let [doubletons (partition 2 1 [1 2 3 5 6])
increment? (fn increment? [[x y]]
(== (inc x) y))]
(cons (ffirst doubletons)
(map second (take-while increment? doubletons))))
;; returns (1 2 3)
Note that this is actually lazy. I expect it not to hold onto the head of doubletons thanks to locals clearing. Another version:
(cons (first [1 2 3 5 6])
(map second (take-while increment? (partition 2 1 [1 2 3 5 6]))))
The original version of the question is more fun, though! :-) A super-simple solution to that could be built using the above, but of course that would be significantly less performant than using reduce. I'll see if I have anything substantially different from zmila's and dnolen's solutions -- and yet still reasonably performant -- to add to that part of this thread later. (Not very likely, I guess.)
Answer to original:
(defn conj-if-sequential
([] [])
([a] a)
([a b] (let [a (if (vector? a) a [a])]
(if (= (inc (last a)) b)
(conj a b)
a))))
(reduce conj-if-sequential [2 3 4 6 8 1])
A more generic solution for those interested:
(defn sequential-seqs
([] [])
([a] a)
([a b] (let [n (last (last a))]
(if (and n (= (inc n) b))
(update-in a [(dec (count a))] conj b)
(conj a [b])))))
(defn largest
([] nil)
([a] a)
([a b] (if (> (count b) (count a)) b a)))
(reduce largest (reduce sequential-seqs [] [2 3 4 6 8 1 4 5 6 7 8 9 13]))
I think this is much better.
(defn find-max-seq [lst]
(let [[f & r] lst,
longest-seq (fn [a b] (if (> (count a) (count b)) a b)),
[last-seq max-seq] (reduce
(fn [ [[prev-num & _ :as cur-seq] max-seq] cur-num ]
(if (== (inc prev-num) cur-num)
[(conj cur-seq cur-num) max-seq]
[(list cur-num) (longest-seq cur-seq max-seq)]
))
[(list f) ()]
r)]
(reverse (longest-seq last-seq max-seq))))
(find-max-seq '(2 3 4 6 8 1)) ; ==> (2 3 4)
(find-max-seq '(3 2 3 4 6 8 9 10 11)) ; ==> (8 9 10 11)