As a newbie to Clojure I often have difficulties to express the simplest things. For example, for replacing the last element in a vector, which would be
v[-1]=new_value
in python, I end up with the following variants in Clojure:
(assoc v (dec (count v)) new_value)
which is pretty long and inexpressive to say the least, or
(conj (vec (butlast v)) new_value)
which even worse, as it has O(n) running time.
That leaves me feeling silly, like a caveman trying to repair a Swiss watch with a club.
What is the right Clojure way to replace the last element in a vector?
To support my O(n)-claim for butlast-version (Clojure 1.8):
(def v (vec (range 1e6)))
#'user/v
user=> (time (first (conj (vec (butlast v)) 55)))
"Elapsed time: 232.686159 msecs"
0
(def v (vec (range 1e7)))
#'user/v
user=> (time (first (conj (vec (butlast v)) 55)))
"Elapsed time: 2423.828127 msecs"
0
So basically for 10 time the number of elements it is 10 times slower.
I'd use
(defn set-top [coll x]
(conj (pop coll) x))
For example,
(set-top [1 2 3] :a)
=> [1 2 :a]
But it also works on the front of lists:
(set-top '(1 2 3) :a)
=> (:a 2 3)
The Clojure stack functions - peek, pop, and conj - work on the natural open end of a sequential collection.
But there is no one right way.
How do the various solutions react to an empty vector?
Your Python v[-1]=new_value throws an exception, as does your (assoc v (dec (count v)) new_value) and my (defn set-top [coll x] (conj (pop coll) x)).
Your (conj (vec (butlast v)) new_value) returns [new_value]. The butlast has no effect.
If you insist on being "pure", your 2nd or 3rd solutions will work. I prefer to be simpler & more explicit using the helper functions from the Tupelo library:
(s/defn replace-at :- ts/List
"Replaces an element in a collection at the specified index."
[coll :- ts/List
index :- s/Int
elem :- s/Any]
...)
(is (= [9 1 2] (replace-at (range 3) 0 9)))
(is (= [0 9 2] (replace-at (range 3) 1 9)))
(is (= [0 1 9] (replace-at (range 3) 2 9)))
As with drop-at, replace-at will throw an exception for invalid values of index.
Similar helper functions exist for
insert-at
drop-at
prepend
append
Note that all of the above work equally well for either a Clojure list (eager or lazy) or a Clojure vector. The conj solution will fail unless you are careful to always coerce the input to a vector first as in your example.
Related
I am trying to get into Lisps and FP by trying out the 99 problems.
Here is the problem statement (Problem 15)
Replicate the elements of a list a given number of times.
I have come up with the following code which simply returns an empty list []
I am unable to figure out why my code doesn't work and would really appreciate some help.
(defn replicateList "Replicates each element of the list n times" [l n]
(loop [initList l returnList []]
(if (empty? initList)
returnList
(let [[head & rest] initList]
(loop [x 0]
(when (< x n)
(conj returnList head)
(recur (inc x))))
(recur rest returnList)))))
(defn -main
"Main" []
(test/is (=
(replicateList [1 2] 2)
[1 1 2 2])
"Failed basic test")
)
copying my comment to answer:
this line: (conj returnList head) doesn't modify returnlist, rather it just drops the result in your case. You should restructure your program to pass the accumulated list further to the next iteration. But there are better ways to do this in clojure. Like (defn replicate-list [data times] (apply concat (repeat times data)))
If you still need the loop/recur version for educational reasons, i would go with this:
(defn replicate-list [data times]
(loop [[h & t :as input] data times times result []]
(if-not (pos? times)
result
(if (empty? input)
(recur data (dec times) result)
(recur t times (conj result h))))))
user> (replicate-list [1 2 3] 3)
;;=> [1 2 3 1 2 3 1 2 3]
user> (replicate-list [ ] 2)
;;=> []
user> (replicate-list [1 2 3] -1)
;;=> []
update
based on the clarified question, the simplest way to do this is
(defn replicate-list [data times]
(mapcat (partial repeat times) data))
user> (replicate-list [1 2 3] 3)
;;=> (1 1 1 2 2 2 3 3 3)
and the loop/recur variant:
(defn replicate-list [data times]
(loop [[h & t :as data] data n 0 res []]
(cond (empty? data) res
(>= n times) (recur t 0 res)
:else (recur data (inc n) (conj res h)))))
user> (replicate-list [1 2 3] 3)
;;=> [1 1 1 2 2 2 3 3 3]
user> (replicate-list [1 2 3] 0)
;;=> []
user> (replicate-list [] 10)
;;=> []
Here is a version based on the original post, with minimal modifications:
;; Based on the original version posted
(defn replicateList "Replicates each element of the list n times" [l n]
(loop [initList l returnList []]
(if (empty? initList)
returnList
(let [[head & rest] initList]
(recur
rest
(loop [inner-returnList returnList
x 0]
(if (< x n)
(recur (conj inner-returnList head) (inc x))
inner-returnList)))))))
Please keep in mind that Clojure is mainly a functional language, meaning that most functions produce their results as a new return value instead of updating in place. So, as pointed out in the comment, the line (conj returnList head) will not have an effect, because it's return value is ignored.
The above version works, but does not really take advantage of Clojure's sequence processing facilities. So here are two other suggestions for solving your problem:
;; Using lazy seqs and reduce
(defn replicateList2 [l n]
(reduce into [] (map #(take n (repeat %)) l)))
;; Yet another way using transducers
(defn replicateList3 [l n]
(transduce
(comp (map #(take n (repeat %)))
cat
)
conj
[]
l))
One thing is not clear about your question though: From your implementation, it looks like you want to create a new list where each element is repeated n times, e.g.
playground.replicate> (replicateList [1 2 3] 4)
[1 1 1 1 2 2 2 2 3 3 3 3]
But if you would instead like this result
playground.replicate> (replicateList [1 2 3] 4)
[1 2 3 1 2 3 1 2 3 1 2 3]
the answer to your question will be different.
If you want to learn idiomatic Clojure you should try to find a solution without such low level facilities as loop. Rather try to combine higher level functions like take, repeat, repeatedly. If you're feeling adventurous you might throw in laziness as well. Clojure's sequences are lazy, that is they get evaluated only when needed.
One example I came up with would be
(defn repeat-list-items [l n]
(lazy-seq
(when-let [s (seq l)]
(concat (repeat n (first l))
(repeat-list-items (next l) n)))))
Please also note the common naming with kebab-case
This seems to do what you want pretty well and works for an unlimited input (see the call (range) below), too:
experi.core> (def l [:a :b :c])
#'experi.core/
experi.core> (repeat-list-items l 2)
(:a :a :b :b :c :c)
experi.core> (repeat-list-items l 0)
()
experi.core> (repeat-list-items l 1)
(:a :b :c)
experi.core> (take 10 (drop 10000 (repeat-list-items (range) 4)))
(2500 2500 2500 2500 2501 2501 2501 2501 2502 2502)
When doing
(map f [0 1 2] [:0 :1])
f will get called twice, with the arguments being
0 :0
1 :1
Is there a simple yet efficient way, i.e. without producing more intermediate sequences etc., to make f get called for every value of the first collection, with the following arguments?
0 :0
1 :1
2 nil
Edit Addressing question by #fl00r in the comments.
The actual use case that triggered this question needed map to always work exactly (count first-coll) times, regardless if the second (or third, or ...) collection was longer.
It's a bit late in the game now and somewhat unfair after having accepted an answer, but if a good answer gets added that only does what I specifically asked for - mapping (count first-coll) times - I would accept that.
You could do:
(map f [0 1 2] (concat [:0 :1] (repeat nil)))
Basically, pad the second coll with an infinite sequence of nils. map stops when it reaches the end of the first collection.
An (eager) loop/recur form that walks to end of longest:
(loop [c1 [0 1 2] c2 [:0 :1] o []]
(if (or (seq c1) (seq c2))
(recur (rest c1) (rest c2) (conj o (f (first c1) (first c2))))
o))
Or you could write a lazy version of map that did something similar.
A general lazy version, as suggested by Alex Miller's answer, is
(defn map-all [f & colls]
(lazy-seq
(when-not (not-any? seq colls)
(cons
(apply f (map first colls))
(apply map-all f (map rest colls))))))
For example,
(map-all vector [0 1 2] [:0 :1])
;([0 :0] [1 :1] [2 nil])
You would probably want to specialise map-all for one and two collections.
just for fun
this could easily be done with common lisp's do macro. We could implement it in clojure and do this (and much more fun things) with it:
(defmacro cl-do [clauses [end-check result] & body]
(let [clauses (map #(if (coll? %) % (list %)) clauses)
bindings (mapcat (juxt first second) clauses)
nexts (map #(nth % 2 (first %)) clauses)]
`(loop [~#bindings]
(if ~end-check
~result
(do
~#body
(recur ~#nexts))))))
and then just use it for mapping (notice it can operate on more than 2 colls):
(defn map-all [f & colls]
(cl-do ((colls colls (map next colls))
(res [] (conj res (apply f (map first colls)))))
((every? empty? colls) res)))
in repl:
user> (map-all vector [1 2 3] [:a :s] '[z x c v])
;;=> [[1 :a z] [2 :s x] [3 nil c] [nil nil v]]
I know I can destructure a vector "from the front" like this:
(fn [[a b & rest]] (+ a b))
Is there any (short) way to access the last two elements instead?
(fn [[rest & a b]] (+ a b)) ;;Not legal
My current alternative is to
(fn [my-vector] (let [[a b] (take-last 2 my-vector)] (+ a b)))
and it was trying to figure out if there is way to do that in a more convenient way directly in the function arguments.
You can peel off the last two elements and add them thus:
((fn [v] (let [[b a] (rseq v)] (+ a b))) [1 2 3 4])
; 7
rseq supplies a reverse sequence for a vector in quick time.
We just destructure its first two elements.
We needn't mention the rest of it, which we don't do anything with.
user=> (def v (vec (range 0 10000000)))
#'user/v
user=> (time ((fn [my-vector] (let [[a b] (take-last 2 my-vector)] (+ a b))) v))
"Elapsed time: 482.965121 msecs"
19999997
user=> (time ((fn [my-vector] (let [a (peek my-vector) b (peek (pop my-vector))] (+ a b))) v))
"Elapsed time: 0.175539 msecs"
19999997
My advice would be to throw convenience to the wind and use peek and pop to work with the end of a vector. When your input vector is very large, you'll see tremendous performance gains.
(Also, to answer the question in the title: no.)
What if map and doseq had a baby? I'm trying to write a function or macro like Common Lisp's mapc, but in Clojure. This does essentially what map does, but only for side-effects, so it doesn't need to generate a sequence of results, and wouldn't be lazy. I know that one can iterate over a single sequence using doseq, but map can iterate over multiple sequences, applying a function to each element in turn of all of the sequences. I also know that one can wrap map in dorun. (Note: This question has been extensively edited after many comments and a very thorough answer. The original question focused on macros, but those macro issues turned out to be peripheral.)
This is fast (according to criterium):
(defn domap2
[f coll]
(dotimes [i (count coll)]
(f (nth coll i))))
but it only accepts one collection. This accepts arbitrary collections:
(defn domap3
[f & colls]
(dotimes [i (apply min (map count colls))]
(apply f (map #(nth % i) colls))))
but it's very slow by comparison. I could also write a version like the first, but with different parameter cases [f c1 c2], [f c1 c2 c3], etc., but in the end, I'll need a case that handles arbitrary numbers of collections, like the last example, which is simpler anyway. I've tried many other solutions as well.
Since the second example is very much like the first except for the use of apply and the map inside the loop, I suspect that getting rid of them would speed things up a lot. I have tried to do this by writing domap2 as a macro, but the way that the catch-all variable after & is handled keeps tripping me up, as illustrated above.
Other examples (out of 15 or 20 different versions), benchmark code, and times on a Macbook Pro that's a few years old (full source here):
(defn domap1
[f coll]
(doseq [e coll]
(f e)))
(defn domap7
[f coll]
(dorun (map f coll)))
(defn domap18
[f & colls]
(dorun (apply map f colls)))
(defn domap15
[f coll]
(when (seq coll)
(f (first coll))
(recur f (rest coll))))
(defn domap17
[f & colls]
(let [argvecs (apply (partial map vector) colls)] ; seq of ntuples of interleaved vals
(doseq [args argvecs]
(apply f args))))
I'm working on an application that uses core.matrix matrices and vectors, but feel free to substitute your own side-effecting functions below.
(ns tst
(:use criterium.core
[clojure.core.matrix :as mx]))
(def howmany 1000)
(def a-coll (vec (range howmany)))
(def maskvec (zero-vector :vectorz howmany))
(defn unmaskit!
[idx]
(mx/mset! maskvec idx 1.0)) ; sets element idx of maskvec to 1.0
(defn runbench
[domapfn label]
(print (str "\n" label ":\n"))
(bench (def _ (domapfn unmaskit! a-coll))))
Mean execution times according to Criterium, in microseconds:
domap1: 12.317551 [doseq]
domap2: 19.065317 [dotimes]
domap3: 265.983779 [dotimes with apply, map]
domap7: 53.263230 [map with dorun]
domap18: 54.456801 [map with dorun, multiple collections]
domap15: 32.034993 [recur]
domap17: 95.259984 [doseq, multiple collections interleaved using map]
EDIT: It may be that dorun+map is the best way to implement domap for multiple large lazy sequence arguments, but doseq is still king when it comes to single lazy sequences. Performing the same operation as unmask! above, but running the index through (mod idx 1000), and iterating over (range 100000000), doseq is about twice as fast as dorun+map in my tests (i.e. (def domap25 (comp dorun map))).
You don't need a macro, and I don't see why a macro would be helpful here.
user> (defn do-map [f & lists] (apply mapv f lists) nil)
#'user/do-map
user> (do-map (comp println +) (range 2 6) (range 8 11) (range 22 40))
32
35
38
nil
note do-map here is eager (thanks to mapv) and only executes for side effects
Macros can use varargs lists, as the (useless!) macro version of do-map demonstrates:
user> (defmacro do-map-macro [f & lists] `(do (mapv ~f ~#lists) nil))
#'user/do-map-macro
user> (do-map-macro (comp println +) (range 2 6) (range 8 11) (range 22 40))
32
35
38
nil
user> (macroexpand-1 '(do-map-macro (comp println +) (range 2 6) (range 8 11) (range 22 40)))
(do (clojure.core/mapv (comp println +) (range 2 6) (range 8 11) (range 22 40)) nil)
Addendum:
addressing the efficiency / garbage-creation concerns:
note that below I truncate the output of the criterium bench function, for conciseness reasons:
(defn do-map-loop
[f & lists]
(loop [heads lists]
(when (every? seq heads)
(apply f (map first heads))
(recur (map rest heads)))))
user> (crit/bench (with-out-str (do-map-loop (comp println +) (range 2 6) (range 8 11) (range 22 40))))
...
Execution time mean : 11.367804 µs
...
This looks promising because it doesn't create a data structure that we aren't using anyway (unlike mapv above). But it turns out it is slower than the previous (maybe because of the two map calls?).
user> (crit/bench (with-out-str (do-map-macro (comp println +) (range 2 6) (range 8 11) (range 22 40))))
...
Execution time mean : 7.427182 µs
...
user> (crit/bench (with-out-str (do-map (comp println +) (range 2 6) (range 8 11) (range 22 40))))
...
Execution time mean : 8.355587 µs
...
Since the loop still wasn't faster, let's try a version which specializes on arity, so that we don't need to call map twice on every iteration:
(defn do-map-loop-3
[f a b c]
(loop [[a & as] a
[b & bs] b
[c & cs] c]
(when (and a b c)
(f a b c)
(recur as bs cs))))
Remarkably, though this is faster, it is still slower than the version that just used mapv:
user> (crit/bench (with-out-str (do-map-loop-3 (comp println +) (range 2 6) (range 8 11) (range 22 40))))
...
Execution time mean : 9.450108 µs
...
Next I wondered if the size of the input was a factor. With larger inputs...
user> (def test-input (repeatedly 3 #(range (rand-int 100) (rand-int 1000))))
#'user/test-input
user> (map count test-input)
(475 531 511)
user> (crit/bench (with-out-str (apply do-map-loop-3 (comp println +) test-input)))
...
Execution time mean : 1.005073 ms
...
user> (crit/bench (with-out-str (apply do-map (comp println +) test-input)))
...
Execution time mean : 756.955238 µs
...
Finally, for completeness, the timing of do-map-loop (which as expected is slightly slower than do-map-loop-3)
user> (crit/bench (with-out-str (apply do-map-loop (comp println +) test-input)))
...
Execution time mean : 1.553932 ms
As we see, even with larger input sizes, mapv is faster.
(I should note for completeness here that map is slightly faster than mapv, but not by a large degree).
I am looking for a function that returns the first element in a sequence for which an fn evaluates to true. For example:
(first-map (fn [x] (= x 1)) '(3 4 1))
The above fake function should return 1 (the last element in the list). Is there something like this in Clojure?
user=> (defn find-first
[f coll]
(first (filter f coll)))
#'user/find-first
user=> (find-first #(= % 1) [3 4 1])
1
Edit: A concurrency. :) No. It does not apply f to the whole list. Only to the elements up to the first matching one due to laziness of filter.
In your case, the idiom is
(some #{1} [1 2 3 4])
How it works: #{1} is a set literal. A set is also a function evaluating to its arg if the arg is present in the set and to nil otherwise. Any set element is a "truthy" value (well, except for a boolean false, but that's a rarity in a set). some returns the return value of the predicate evaluated against the first collection member for which the result was truthy.
I tried several methods mentioned in this thread (JDK 8 and Clojure 1.7), and did some benchmark tests:
repl> (defn find-first
[f coll]
(first (filter f coll)))
#'cenx.parker.strategies.vzw.repl/find-first
repl> (time (find-first #(= % 50000000) (range)))
"Elapsed time: 5799.41122 msecs"
50000000
repl> (time (some #{50000000} (range)))
"Elapsed time: 4386.256124 msecs"
50000000
repl> (time (reduce #(when (= %2 50000000) (reduced %2)) nil (range)))
"Elapsed time: 993.267553 msecs"
50000000
The results show that reduce way may be the most efficient solution as in clojure 1.7.
In 2016 there was a patch submitted to clojure core that added an efficient shortcut for (first (filter pred coll)) idiom, it was called seek.
The implementation avoided problems in herent with both the (first (filter)) and (some #(when (pred))) alternatives. That is, it works efficiently with chunked sequences and plays nice with nil? and false? predicates.
Patch:
(defn seek
"Returns first item from coll for which (pred item) returns true.
Returns nil if no such item is present, or the not-found value if supplied."
{:added "1.9" ; note, this was never accepted into clojure core
:static true}
([pred coll] (seek pred coll nil))
([pred coll not-found]
(reduce (fn [_ x]
(if (pred x)
(reduced x)
not-found))
not-found coll)))
Examples:
(seek odd? (range)) => 1
(seek pos? [-1 1]) => 1
(seek pos? [-1 -2] ::not-found) => ::not-found
(seek nil? [1 2 nil 3] ::not-found) => nil
Eventually the patch was rejected:
Upon review, we've decided that we do not wish to include this. Use of linear search (and in particular nested linear search) leads to poor performance - often it's better to use other kinds of data structures and that's why this functionality has not been included in the past. ~Alex Miller 12/May/17 3:34 PM
I think some is the best tool for the job:
(some #(if (= % 1) %) '(3 4 1))
Using drop-while instead of filter should address "over-application" of f for chunked sequences:
(defn find-first [f coll]
(first (drop-while (complement f) coll)))
;;=> #'user/find-first
(find-first #(= % 1) [3 4 1])
;;=> 1
The way I do this in clojure is sort like you might do it in Scheme.
(defn call-with-found
"Call the given predicate, pred, on successive elements of the collection
until the first time pred returns a truthy value, at which time if-found
is called with that element of the collection, and call-with-found returns
the return value of if-found. If no such element of collection is found
(including if collection is empty) then the value if-not-found (defaulting
to false) is returned."
([pred coll & {:keys [if-found if-not-found]
:or {if-found (constantly true)
if-not-found false}}]
(reduce (fn [_ item]
(if (pred item)
(reduced (if-found item))
if-not-found)) if-not-found coll)))
The function call-with-found is called with a predicate and a collection. We search the collection until we find an element which satisfies the predicate, at which point we call the if-found continuation with that value, else we return the if-not-found value.