I'm trying to make a conditional transducer in Clojure as follows:
(defn if-xf
"Takes a predicate and two transducers.
Returns a new transducer that routes the input to one of the transducers
depending on the result of the predicate."
[pred a b]
(fn [rf]
(let [arf (a rf)
brf (b rf)]
(fn
([] (rf))
([result]
(rf result))
([result input]
(if (pred input)
(arf result input)
(brf result input)))))))
It is pretty useful in that it lets you do stuff like this:
;; multiply odd numbers by 100, square the evens.
(= [0 100 4 300 16 500 36 700 64 900]
(sequence
(if-xf odd? (map #(* % 100)) (map (fn [x] (* x x))))
(range 10)))
However, this conditional transducer does not work very well with transducers that perform cleanup in their 1-arity branch:
;; negs are multiplied by 100, non-negs are partitioned by 2
;; BUT! where did 6 go?
;; expected: [-600 -500 -400 -300 -200 -100 [0 1] [2 3] [4 5] [6]]
;;
(= [-600 -500 -400 -300 -200 -100 [0 1] [2 3] [4 5]]
(sequence
(if-xf neg? (map #(* % 100)) (partition-all 2))
(range -6 7)))
Is it possible to tweak the definition of if-xf to handle the case of transducers with cleanup?
I'm trying this, but with weird behavior:
(defn if-xf
"Takes a predicate and two transducers.
Returns a new transducer that routes the input to one of the transducers
depending on the result of the predicate."
[pred a b]
(fn [rf]
(let [arf (a rf)
brf (b rf)]
(fn
([] (rf))
([result]
(arf result) ;; new!
(brf result) ;; new!
(rf result))
([result input]
(if (pred input)
(arf result input)
(brf result input)))))))
Specifically, the flushing happens at the end:
;; the [0] at the end should appear just before the 100.
(= [[-6 -5] [-4 -3] [-2 -1] 100 200 300 400 500 600 [0]]
(sequence
(if-xf pos? (map #(* % 100)) (partition-all 2))
(range -6 7)))
Is there a way to make this branching/conditional transducer without storing the entire input sequence in local state within this transducer (i.e. doing all the processing in the 1-arity branch upon cleanup)?
The idea is to complete every time the transducer switches over. IMO this is the only way to do it without buffering:
(defn if-xf
"Takes a predicate and two transducers.
Returns a new transducer that routes the input to one of the transducers
depending on the result of the predicate."
[pred a b]
(fn [rf]
(let [arf (volatile! (a rf))
brf (volatile! (b rf))
a? (volatile! nil)]
(fn
([] (rf))
([result]
(let [crf (if #a? #arf #brf)]
(-> result crf rf)))
([result input]
(let [p? (pred input)
[xrf crf] (if p? [#arf #brf] [#brf #arf])
switched? (some-> #a? (not= p?))]
(if switched?
(-> result crf (xrf input))
(xrf result input))
(vreset! a? p?)))))))
(sequence (if-xf pos? (map #(* % 100)) (partition-all 2)) [0 1 0 1 0 0 0 1])
; => ([0] 100 [0] 100 [0 0] [0] 100)
I think your question is ill-defined. What exactly do you want to happen when the transducers have state? For example, what do you expect this do:
(sequence
(if-xf even? (partition-all 3) (partition-all 2))
(range 14))
Furthermore, sometimes reducing functions have work to do at the beginning and the end and can't be restarted arbitrarily. For example, here is a reducer that computes the mean:
(defn mean
([] {:count 0, :sum 0})
([result] (double (/ (:sum result) (:count result))))
([result x]
(update-in
(update-in result [:count] inc)
[:sum] (partial + x))))
(transduce identity mean [10 20 40 40]) ;27.5
Now let's take the average, where anything below 20 counts for 20, but everything else is decreased by 1:
(transduce
(if-xf
(fn [x] (< x 20))
(map (constantly 20))
(map dec))
mean [10 20 40 40]) ;29.25
My answer is the following: I think your original solution is best. It works well using map, which is how you stated the usefulness of the conditional transducer in the first place.
Related
I'd like to have a version of sequence that doesn't do the chunking of 32 elements. Currently, this code will output
(def peek #(doto % (print " ")))
(def pause #(do (Thread/sleep 10)
%))
(take 2 (->> (range 100)
(sequence (comp (map peek)
(map pause)
(map inc)))))
;; prints 0 1 2 3 4 <..etc..> 32
;; => (0, 1)
I'd like a version of it so that it only iterates through the elements that it needs
(take 2 (->> (range 100)
(iter-sequence (comp (map peek)
(map pause)
(map inc)))))
;; prints 0 1
;; => (0, 1)
Is there a way to do this?
I had to change a couple of things to get it working. The first is to cut and paste sequence code and replace clojure.lang.RT/chunkIteratorSeq with an alternative version of clojure.lang.IteratorSeq that has exposed public constructor methods.
The reason being is that the clojure.lang.IteratorSeq/create has a check to iter.next() on L27 which will block if the source is blocking.
(defn seqiter
{:added "1.0"
:static true}
([coll] coll)
([xform coll]
(IteratorSeq.
(TransformerIterator/create xform (clojure.lang.RT/iter coll))))
([xform coll & colls]
(IteratorSeq.
(TransformerIterator/createMulti
xform
(map #(clojure.lang.RT/iter %) (cons coll colls))))))
I am trying to get into Lisps and FP by trying out the 99 problems.
Here is the problem statement (Problem 15)
Replicate the elements of a list a given number of times.
I have come up with the following code which simply returns an empty list []
I am unable to figure out why my code doesn't work and would really appreciate some help.
(defn replicateList "Replicates each element of the list n times" [l n]
(loop [initList l returnList []]
(if (empty? initList)
returnList
(let [[head & rest] initList]
(loop [x 0]
(when (< x n)
(conj returnList head)
(recur (inc x))))
(recur rest returnList)))))
(defn -main
"Main" []
(test/is (=
(replicateList [1 2] 2)
[1 1 2 2])
"Failed basic test")
)
copying my comment to answer:
this line: (conj returnList head) doesn't modify returnlist, rather it just drops the result in your case. You should restructure your program to pass the accumulated list further to the next iteration. But there are better ways to do this in clojure. Like (defn replicate-list [data times] (apply concat (repeat times data)))
If you still need the loop/recur version for educational reasons, i would go with this:
(defn replicate-list [data times]
(loop [[h & t :as input] data times times result []]
(if-not (pos? times)
result
(if (empty? input)
(recur data (dec times) result)
(recur t times (conj result h))))))
user> (replicate-list [1 2 3] 3)
;;=> [1 2 3 1 2 3 1 2 3]
user> (replicate-list [ ] 2)
;;=> []
user> (replicate-list [1 2 3] -1)
;;=> []
update
based on the clarified question, the simplest way to do this is
(defn replicate-list [data times]
(mapcat (partial repeat times) data))
user> (replicate-list [1 2 3] 3)
;;=> (1 1 1 2 2 2 3 3 3)
and the loop/recur variant:
(defn replicate-list [data times]
(loop [[h & t :as data] data n 0 res []]
(cond (empty? data) res
(>= n times) (recur t 0 res)
:else (recur data (inc n) (conj res h)))))
user> (replicate-list [1 2 3] 3)
;;=> [1 1 1 2 2 2 3 3 3]
user> (replicate-list [1 2 3] 0)
;;=> []
user> (replicate-list [] 10)
;;=> []
Here is a version based on the original post, with minimal modifications:
;; Based on the original version posted
(defn replicateList "Replicates each element of the list n times" [l n]
(loop [initList l returnList []]
(if (empty? initList)
returnList
(let [[head & rest] initList]
(recur
rest
(loop [inner-returnList returnList
x 0]
(if (< x n)
(recur (conj inner-returnList head) (inc x))
inner-returnList)))))))
Please keep in mind that Clojure is mainly a functional language, meaning that most functions produce their results as a new return value instead of updating in place. So, as pointed out in the comment, the line (conj returnList head) will not have an effect, because it's return value is ignored.
The above version works, but does not really take advantage of Clojure's sequence processing facilities. So here are two other suggestions for solving your problem:
;; Using lazy seqs and reduce
(defn replicateList2 [l n]
(reduce into [] (map #(take n (repeat %)) l)))
;; Yet another way using transducers
(defn replicateList3 [l n]
(transduce
(comp (map #(take n (repeat %)))
cat
)
conj
[]
l))
One thing is not clear about your question though: From your implementation, it looks like you want to create a new list where each element is repeated n times, e.g.
playground.replicate> (replicateList [1 2 3] 4)
[1 1 1 1 2 2 2 2 3 3 3 3]
But if you would instead like this result
playground.replicate> (replicateList [1 2 3] 4)
[1 2 3 1 2 3 1 2 3 1 2 3]
the answer to your question will be different.
If you want to learn idiomatic Clojure you should try to find a solution without such low level facilities as loop. Rather try to combine higher level functions like take, repeat, repeatedly. If you're feeling adventurous you might throw in laziness as well. Clojure's sequences are lazy, that is they get evaluated only when needed.
One example I came up with would be
(defn repeat-list-items [l n]
(lazy-seq
(when-let [s (seq l)]
(concat (repeat n (first l))
(repeat-list-items (next l) n)))))
Please also note the common naming with kebab-case
This seems to do what you want pretty well and works for an unlimited input (see the call (range) below), too:
experi.core> (def l [:a :b :c])
#'experi.core/
experi.core> (repeat-list-items l 2)
(:a :a :b :b :c :c)
experi.core> (repeat-list-items l 0)
()
experi.core> (repeat-list-items l 1)
(:a :b :c)
experi.core> (take 10 (drop 10000 (repeat-list-items (range) 4)))
(2500 2500 2500 2500 2501 2501 2501 2501 2502 2502)
In Clojure I want to find the result of multiple reductions while only consuming the sequence once. In Java I would do something like the following:
double min = Double.MIN_VALUE;
double max = Double.MAX_VALUE;
for (Item item : items) {
double price = item.getPrice();
if (price > min) {
min = price;
}
if (price < max) {
max = price;
}
}
In Clojure I could do much the same thing by using loop and recur, but it's not very composable - I'd like to do something that lets you add in other aggregation functions as needed.
I've written the following function to do this:
(defn reduce-multi
"Given a sequence of fns and a coll, returns a vector of the result of each fn
when reduced over the coll."
[fns coll]
(let [n (count fns)
r (rest coll)
initial-v (transient (into [] (repeat n (first coll))))
fns (into [] fns)
reduction-fn
(fn [v x]
(loop [v-current v, i 0]
(let [y (nth v-current i)
f (nth fns i)
v-new (assoc! v-current i (f y x))]
(if (= i (- n 1))
v-new
(recur v-new (inc i))))))]
(persistent! (reduce reduction-fn initial-v r))))
This can be used in the following way:
(reduce-multi [max min] [4 3 6 7 0 1 8 2 5 9])
=> [9 0]
I appreciate that it's not implemented in the most idiomatic way, but the main problem is that it's about 10x as slow as doing the reductions one at at time. This might be useful for lots performing lots of reductions where the seq is doing heavy IO, but surely this could be better.
Is there something in an existing Clojure library that would do what I want? If not, where am I going wrong in my function?
that's what i would do: simply delegate this task to a core reduce function, like this:
(defn multi-reduce
([fs accs xs] (reduce (fn [accs x] (doall (map #(%1 %2 x) fs accs)))
accs xs))
([fs xs] (when (seq xs)
(multi-reduce fs (repeat (count fs) (first xs))
(rest xs)))))
in repl:
user> (multi-reduce [+ * min max] (range 1 10))
(45 362880 1 9)
user> (multi-reduce [+ * min max] [10])
(10 10 10 10)
user> (multi-reduce [+ * min max] [])
nil
user> (multi-reduce [+ * min max] [1 1 1000 0] [])
[1 1 1000 0]
user> (multi-reduce [+ * min max] [1 1 1000 0] [1])
(2 1 1 1)
user> (multi-reduce [+ * min max] [1 1 1000 0] (range 1 10))
(46 362880 1 9)
user> (multi-reduce [max min] (range 1000000))
(999999 0)
The code for reduce is fast for reducible collections. So it's worth trying to base multi-reduce on core reduce. To do so, we have to be able to construct reducing functions of the right shape. An ancillary function to do so is ...
(defn juxt-reducer [f g]
(fn [[fa ga] x] [(f fa x) (g ga x)]))
Now we can define the function you want, which combines juxt with reduce as ...
(defn juxt-reduce
([[f g] coll]
(if-let [[x & xs] (seq coll)]
(juxt-reduce (list f g) [x x] xs)
[(f) (g)]))
([[f g] init coll]
(reduce (juxt-reducer f g) init coll)))
For example,
(juxt-reduce [max min] [4 3 6 7 0 1 8 2 5 9]) ;=> [9 0]
The above follows the shape of core reduce. It can clearly be extended to cope with more than two functions. And I'd expect it to be faster than yours for reducible collections.
Here is how I would do it:
(ns clj.core
(:require [clojure.string :as str] )
(:use tupelo.core))
(def data (flatten [ (range 5 10) (range 5) ] ))
(spyx data)
(def result (reduce (fn [cum-result curr-val] ; reducing (accumulator) fn
(it-> cum-result
(update it :min-val min curr-val)
(update it :max-val max curr-val)))
{ :min-val (first data) :max-val (first data) } ; inital value
data)) ; seq to reduce
(spyx result)
(defn -main [] )
;=> data => (5 6 7 8 9 0 1 2 3 4)
;=> result => {:min-val 0, :max-val 9}
So the reducing function (fn ...) carries along a map like {:min-val xxx :max-val yyy} through each element of the sequence, updating the min & max values as required at each step.
While this does make only one pass through the data, it is doing a lot of extra work calling update twice per element. Unless your sequence is very unusual, it is probably more efficient to make two (very efficient) passes through the data like:
(def min-val (apply min data))
(def max-val (apply max data))
(spyx min-val)
(spyx max-val)
;=> min-val => 0
;=> max-val => 9
What is the best way of implementing map function together with an updatable state between applications of function to each element of sequence? To illustrate the issue let's suppose that we have a following problem:
I have a vector of the numbers. I want a new sequence where each element is multiplied by 2 and then added number of 10's in the sequence up to and including the current element. For example from:
[20 30 40 10 20 10 30]
I want to generate:
[40 60 80 21 41 22 62]
Without adding the count of 10 the solution can be formulated using a high level of abstraction:
(map #(* 2 %) [20 30 40 10 20 10 30])
Having count to update forced me to "go to basic" and the solution I came up with is:
(defn my-update-state [x state]
(if (= x 10) (+ state 1) state)
)
(defn my-combine-with-state [x state]
(+ x state))
(defn map-and-update-state [vec fun state update-state combine-with-state]
(when-not (empty? vec)
(let [[f & other] vec
g (fun f)
new-state (update-state f state)]
(cons (combine-with-state g new-state) (map-and-update-state other fun new-state update-state combine-with-state))
)))
(map-and-update-state [20 30 40 50 10 20 10 30 ] #(* 2 %) 0 my-update-state my-combine-with-state )
My question: is it the appropriate/canonical way to solve the problem or I overlooked some important concepts/functions.
PS:
The original problem is walking AST (abstract syntax tree) and generating new AST together with updating symbol table, so when proposing the solution to the problem above please keep it in mind.
I do not worry about blowing up stack, so replacement with loop+recur is not
my concern here.
Is using global Vars or Refs instead of passing state as an argument a definite no-no?
You can use reduce to accumulate a pair of the number of 10s seen so far and the current vector of results.:
(defn map-update [v]
(letfn [(update [[ntens v] i]
(let [ntens (+ ntens (if (= 10 i) 1 0))]
[ntens (conj v (+ ntens (* 2 i)))]))]
(second (reduce update [0 []] v))))
To count # of 10 you can do
(defn count-10[col]
(reductions + (map #(if (= % 10) 1 0) col)))
Example:
user=> (count-10 [1 2 10 20 30 10 1])
(0 0 1 1 1 2 2)
And then a simple map for the final result
(map + col col (count-10 col)))
Reduce and reductions are good ways to traverse a sequence keeping a state. If you feel your code is not clear you can always use recursion with loop/recur or lazy-seq like this
(defn twice-plus-ntens
([coll] (twice-plus-ntens coll 0))
([coll ntens]
(lazy-seq
(when-let [s (seq coll)]
(let [item (first s)
new-ntens (if (= 10 item) (inc ntens) ntens)]
(cons (+ (* 2 item) new-ntens)
(twice-plus-ntens (rest s) new-ntens)))))))
have a look at map source code evaluating this at your repl
(source map)
I've skipped chunked optimization and multiple collection support.
You can make it a higher-order function this way
(defn map-update
([mf uf coll] (map-update mf uf (uf) coll))
([mf uf val coll]
(lazy-seq
(when-let [s (seq coll)]
(let [item (first s)
new-status (uf item val)]
(cons (mf item new-status)
(map-update mf uf new-status (rest s))))))))
(defn update-f
([] 0)
([item status]
(if (= item 10) (inc status) status)))
(defn map-f [item status]
(+ (* 2 item) status))
(map-update map-f update-f in)
The most appropriate way is to use function with state
(into
[]
(map
(let [mem (atom 0)]
(fn [val]
(when (== val 10) (swap! mem inc))
(+ #mem (* val 2)))))
[20 30 40 10 20 10 30])
also see
memoize
standard function
I need to take some amount of elements from a sequence based on some quantity rule. Here is a solution I came up with:
(defn take-while-not-enough
[p len xs]
(loop [ac 0
r []
s xs]
(if (empty? s)
r
(let [new-ac (p ac (first s))]
(if (>= new-ac len)
r
(recur new-ac (conj r (first s)) (rest s)))))))
(take-while-not-enough + 10 [2 5 7 8 2 1]) ; [2 5]
(take-while-not-enough #(+ %1 (%2 1)) 7 [[2 5] [7 8] [2 1]]) ; [[2 5]]
Is there any better way to achieve the same?
Thanks.
UPDATE:
Somebody posted that solution, but then removed it. It does the same is the answer that I accepted, but is more readable. Thank you, anonymous well-wisher!
(defn take-while-not-enough [reducer-fn limit data]
(->> (reductions reducer-fn 0 data) ; 1. the sequence of accumulated values
(map vector data) ; 2. paired with the original sequence
(take-while #(< (second %) limit)) ; 3. until a certain accumulated value
(map first))) ; 4. then extract the original values
My first thought is to view this problem as a variation on reduce and thus to break the problem into two steps:
count the number of items in the result
take that many from the input
I also took some liberties with the argument names:
user> (defn take-while-not-enough [reducer-fn limit data]
(take (dec (count (take-while #(< % limit) (reductions reducer-fn 0 data))))
data))
#'user/take-while-not-enough
user> (take-while-not-enough #(+ %1 (%2 1)) 7 [[2 5] [7 8] [2 1]])
([2 5])
user> (take-while-not-enough + 10 [2 5 7 8 2 1])
(2 5)
This returns a sequence and your examples return a vector, if this is important then you can add a call to vec
Something that would traverse the input sequence only once:
(defn take-while-not-enough [r v data]
(->> (rest (reductions (fn [s i] [(r (s 0) i) i]) [0 []] data))
(take-while (comp #(< % v) first))
(map second)))
Well, if you want to use flatland/useful, this is a kinda-okay way to use glue:
(defn take-while-not-enough [p len xs]
(first (glue conj []
(constantly true)
#(>= (reduce p 0 %) len)
xs)))
But it's rebuilding the accumulator for the entire "processed so far" chunk every time it decides whether to grow the chunk more, so it's O(n^2), which will be unacceptable for larger inputs.
The most obvious improvement to your implementation is to make it lazy instead of tail-recursive:
(defn take-while-not-enough [p len xs]
((fn step [acc coll]
(lazy-seq
(when-let [xs (seq coll)]
(let [x (first xs)
acc (p acc x)]
(when-not (>= acc len)
(cons x (step acc xs)))))))
0 xs))
Sometimes lazy-seq is straight-forward and self-explaining.
(defn take-while-not-enough
([f limit coll] (take-while-not-enough f limit (f) coll))
([f limit acc coll]
(lazy-seq
(when-let [s (seq coll)]
(let [fst (first s)
nacc (f acc fst)]
(when (< nxt-sd limit)
(cons fst (take-while-not-enough f limit nacc (rest s)))))))))
Note: f is expected to follow the rules of reduce.