I am currently working on a tree search algorithm to walk the optimal, cost efficient path through a tree structure.
(def tree
'([1]
[3 5]
[5 6 4]
[2 3 4 5]))
The function initiates multiple "workers" through a recursive function call, each time a new node is reached.
(defn treewalk [tree]
(let [worker_history {}]
(letfn [(get-neighbours [level current_position]
(;; get adjacent nodes one level below)
(worker [current_level current_position cost individual_history]
(if (;; current level above bottom)
(let [neighbours (get-neighbours current_level current_position)]
;; recursive worker call for next 2 branches
(worker (+ level1 1) (first_neighbour) (+ cost (current_cost))
(;; updated individual_history))
(worker (+ level1 1) (second neighbour) (+ cost (current_cost))
(;; updated individual_history)))
;; else: worker at bottom -> insert cost and individual history into worker_history
(assoc worker_history cost individual_history))
)))))
The worker_history map is supposed to store the costs and individual paths and is updated with these values when each individual worker reaches bottom of the tree structure. I am aware that working with these sides effects is not the most elegant way to approach that problem in Clojure!
Currently, I am running into the problem that worker_history only returns ine entry of the very last worker that has finished and does not behave as a static variable within the function scope, so there is no concurrent access of that ofject. How could I still modify my approach in order to achieve this level of "concurrency"?
Your structure isn't tree. In functional languages, trees are usually written as nested lists (or vectors), like this: '(3 (4) (5)) or this '(3 (4 (2) (8)) (5)). If I stick to your instructions recursive tree search, return costs and paths for all workers, this should work:
(defn tree-costs [tree]
(let [worker-history (atom [])]
(letfn [(treewalk [tree cost history]
(if (= (count tree) 1) (swap! worker-history
conj [(+ (first tree) cost)
(conj history (first tree))])
(doseq [node (rest tree)]
(treewalk node
(+ cost (first tree))
(conj history (first tree))))))]
(treewalk tree 0 [])
#worker-history
)))
(tree-costs '(3 (4) (5))) => [[7 [3 4]] [8 [3 5]]]
(tree-costs '(3 (4 (2) (8)) (5))) => [[9 [3 4 2]] [15 [3 4 8]] [8 [3 5]]]
Also check clojure.core.async for thread concurrency.
In order to update worker_hisoty concurently, you can try atom
...
(let [worker_history (atom {})]
...
then update it with swap!
...
(swap! worker_history assoc cost individual_history)
...
Simplistically you think about atom as a global mutable variable with atomic updates
I would caution against using stateful techniques as you miss out on the benefits of pure functions. Without knowing the full context of the problem you are trying to solve I cannot say if using atom will serve you well.
Rather than update an atom for each tree path, a map can be constructed using reduce:
(defn paths [tree]
(when-let [root (ffirst tree)]
(if-let [r (next tree)]
(let [left (paths r)
right (paths (map rest r))]
(map #(cons root %) (concat left right)))
[[root]])))
(defn tree-costs [tree]
(reduce (fn [m path] (update m (reduce + path) (fnil conj #{}) path))
{}
(paths tree)))
Related
I often have to run my data through a function if the data fulfill certain criteria. Typically, both the function f and the criteria checker pred are parameterized to the data. For this reason, I find myself wishing for a higher-order if-then-else which knows neither f nor pred.
For example, assume I want to add 10 to all even integers in (range 5). Instead of
(map #(if (even? %) (+ % 10) %) (range 5))
I would prefer to have a helper –let's call it fork– and do this:
(map (fork even? #(+ % 10)) (range 5))
I could go ahead and implement fork as function. It would look like this:
(defn fork
([pred thenf elsef]
#(if (pred %) (thenf %) (elsef %)))
([pred thenf]
(fork pred thenf identity)))
Can this be done by elegantly combining core functions? Some nice chain of juxt / apply / some maybe?
Alternatively, do you know any Clojure library which implements the above (or similar)?
As Alan Thompson mentions, cond-> is a fairly standard way of implicitly getting the "else" part to be "return the value unchanged" these days. It doesn't really address your hope of being higher-order, though. I have another reason to dislike cond->: I think (and argued when cond-> was being invented) that it's a mistake for it to thread through each matching test, instead of just the first. It makes it impossible to use cond-> as an analogue to cond.
If you agree with me, you might try flatland.useful.fn/fix, or one of the other tools in that family, which we wrote years before cond->1.
to-fix is exactly your fork, except that it can handle multiple clauses and accepts constants as well as functions (for example, maybe you want to add 10 to other even numbers but replace 0 with 20):
(map (to-fix zero? 20, even? #(+ % 10)) xs)
It's easy to replicate the behavior of cond-> using fix, but not the other way around, which is why I argue that fix was the better design choice.
1 Apparently we're just a couple weeks away from the 10-year anniversary of the final version of fix. How time flies.
I agree that it could be very useful to have some kind of higher-order functional construct for this but I am not aware of any such construct. It is true that you could implement a higher order fork function, but its usefulness would be quite limited and can easily be achieved using if or the cond-> macro, as suggested in the other answers.
What comes to mind, however, are transducers. You could fairly easily implement a forking transducer that can be composed with other transducers to build powerful and concise sequence processing algorithms.
The implementation could look like this:
(defn forking [pred true-transducer false-transducer]
(fn [step]
(let [true-step (true-transducer step)
false-step (false-transducer step)]
(fn
([] (step))
([dst x] ((if (pred x) true-step false-step) dst x))
([dst] dst))))) ;; flushing not performed.
And this is how you would use it in your example:
(eduction (forking even?
(map #(+ 10 %))
identity)
(range 20))
;; => (10 1 12 3 14 5 16 7 18 9 20 11 22 13 24 15 26 17 28 19)
But it can also be composed with other transducers to build more complex sequence processing algorithms:
(into []
(comp (forking even?
(comp (drop 4)
(map #(+ 10 %)))
(comp (filter #(< 10 %))
(map #(vector % % %))
cat))
(partition-all 3))
(range 20))
;; => [[18 20 11] [11 11 22] [13 13 13] [24 15 15] [15 26 17] [17 17 28] [19 19 19]]
Another way to define fork (with three inputs) could be:
(defn fork [pred then else]
(comp
(partial apply apply)
(juxt (comp {true then, false else} pred) list)))
Notice that in this version the inputs and output can receive zero or more arguments. But let's take a more structured approach, defining some other useful combinators. Let's start by defining pick which corresponds to the categorical coproduct (sum) of morphisms:
(defn pick [actions]
(fn [[tag val]]
((actions tag) val)))
;alternatively
(defn pick [actions]
(comp
(partial apply apply)
(juxt (comp actions first) rest)))
E.g. (mapv (pick [inc dec]) [[0 1] [1 1]]) gives [2 0]. Using pick we can define switch which works like case:
(defn switch [test actions]
(comp
(pick actions)
(juxt test identity)))
E.g. (mapv (switch #(mod % 3) [inc dec -]) [3 4 5]) gives [4 3 -5]. Using switch we can easily define fork:
(defn fork [pred then else]
(switch pred {true then, false else}))
E.g. (mapv (fork even? inc dec) [0 1]) gives [1 0]. Finally, using fork let's also define fork* which receives zero or more predicate and action pairs and works like cond:
(defn fork* [& args]
(->> args
(partition 2)
reverse
(reduce
(fn [else [pred then]]
(fork pred then else))
identity)))
;equivalently
(defn fork* [& args]
(->> args
(partition 2)
(map (partial apply (partial partial fork)))
(apply comp)
(#(% identity))))
E.g. (mapv (fork* neg? -, even? inc) [-1 0 1]) gives [1 1 1].
Depending on the details, it is often easiest to accomplish this goal using the cond-> macro and friends:
(let [myfn (fn [val]
(cond-> val
(even? val) (+ val 10))) ]
with result
(mapv myfn (range 5)) => [10 1 14 3 18]
There is a variant in the Tupelo library that is sometimes helpful:
(mapv #(cond-it-> %
(even? it) (+ it 10))
(range 5))
that allows you to use the special symbol it as you thread the value through multiple stages.
As the examples show, you have the option to define and name the transformer function (my favorite), or use the function literal syntax #(...)
I want to do the following in Clojure as idiomatically as possible:
transduce a collection
associate each element of the input collection with the corresponding element in the output collection
return the result in a hashmap
Is there a succinct way to do this using core library functions?
If not, what improvements can you suggest to the following implementation?
(defn to-hash [coll xform]
(reduce
merge
(map
#(apply hash-map %)
(mapcat hash-map coll (into [] xform coll)))))
something like this should do the trick without intermediate collections:
(defn process [data xform]
(zipmap data (eduction xform data)))
user> (process [1 2 3] (comp (map inc) (map #(* % %))))
;;=> {1 4, 2 9, 3 16}
the docs on eduction say the following:
Returns a reducible/iterable application of the transducers
to the items in coll. Transducers are applied in order as if
combined with comp. Note that these applications will be
performed every time reduce/iterator is called.
so no additional collection is created.
This is any good, of course, as long as there is one-to-one relationship between input and output elements. What is desired output for (process [1 -2 3] (filter pos?)) or (process [1 1 1 2 2 2] (dedupe)) ?
(by the way, your to-hash implementation has the same flaw)
A transducer is a function that takes a reducing function and returns a new reducing function. To make it work with transducers where there is not a one-to-one mapping from elements in the input collection to the output, you will have to use your transducer to create a new reducing function (step2 in the code below) that will associate elements into your hash map. Something like this.
(def ^:dynamic assoc-k nil)
(defn assoc-step [dst x]
(assoc dst assoc-k x))
(defn to-hash [coll xform]
(let [step (xform (completing assoc-step))
step2 (fn [dst x] (binding [assoc-k x] (step dst x)))]
(reduce step2 {} coll)))
This implementation is quite basic and I am not sure to which extent it will work with stateful transducers. But it will work with the stateless ones, such as map and filter.
And we can test it with a transducer that keeps odd elements in the input collection and squares them:
(defn square [x] (* x x))
(to-hash (range 10) (comp (filter odd?) (map square)))
;; => {1 1, 3 9, 5 25, 7 49, 9 81}
I am constructing a list of hash maps which is then passed to another function. When I try to print each hash maps from the list using map it is not working. I am able to print the full list or get the first element etc.
(defn m [a]
(println a)
(map #(println %) a))
The following works from the repl only.
(m (map #(hash-map :a %) [1 2 3]))
But from the program that I load using load-file it is not working. I am seeing the a but not its individual elements. What's wrong?
In Clojure tranform functions return a lazy sequence. So, (map #(println %) a) return a lazy sequence. When consumed, the map action is applied and only then the print-side effect is visible.
If the purpose of the function is to have a side effect, like printing, you need to eagerly evaluate the transformation. The functions dorun and doall
(def a [1 2 3])
(dorun (map #(println %) a))
; returns nil
(doall (map #(println %) a))
; returns the collection
If you actually don't want to map, but only have a side effect, you can use doseq. It is intended to 'iterate' to do side effects:
(def a [1 2 3])
(doseq [i a]
(println i))
If your goal is simply to call an existing function on every item in a collection in order, ignoring the returned values, then you should use run!:
(run! println [1 2 3])
;; 1
;; 2
;; 3
;;=> nil
In some more complicated cases it may be preferable to use doseq as #Gamlor suggests, but in this case, doseq only adds boilerplate.
I recommend to use tail recursion:
(defn printList [a]
(let [head (first a)
tail (rest a)]
(when (not (nil? head))
(println head)
(printList tail))))
In the book Clojure for the Brave and True at the end of the section covering reduce there's a challenge:
If you want an exercise that will really blow your hair back, try implementing map using reduce
It turns out that this was a lot harder (at least for me, a Clojure beginner) than I thought it would be. After quite a few hours I came up with this:
(defn map-as-reduce
[f coll]
(reduce #(cons (f %2) %1) '() (reverse coll)))
Is a better way to do this? I'm particularly frustrated by the fact that I have to reverse the input collection in order for this to work correctly. It seems somehow inelegant!
Remember that you can efficiently insert at the end of a vector:
(defn map' [f coll]
(reduce #(conj %1 (f %2)) [] coll))
Example:
(map' inc [1 2 3])
;=> [2 3 4]
One difference between this map' and the original map is that the original map returns an ISeq instead of just a Seqable:
(seq? (map inc [1 2 3]))
;=> true
(seq? (map' inc [1 2 3]))
;=> false
You could remedy this by composing the above implementation of map' with seq:
(defn map' [f coll]
(seq (reduce #(conj %1 (f %2)) [] coll)))
The most important difference now is that, while the original map is lazy, this map' is eager, because reduce is eager.
just for fun:
map really accepts more than one collection as an argument. Here is an extended implementation:
(defn map-with-reduce
([f coll] (seq (reduce #(conj %1 (f %2)) [] coll)))
([f coll & colls]
(let [colls (cons coll colls)]
(map-with-reduce (partial apply f)
(partition (count colls)
(apply interleave colls))))))
in repl:
user> (map-with-reduce inc [1 2 3])
(2 3 4)
user> (map-with-reduce + [1 2 3] [4 5] [6 7 8])
(11 14)
The real map calls seq on its collection argument(s) and returns a lazy seq, so maybe this to get it a little closer to the real map?
(defn my-map
[f coll]
(lazy-seq (reduce #(conj %1 (f %2)) [] (seq coll))))
I would have added that as a comment, but I don't have the reputation. :)
You can use conj to append to a vector instead of prepending to a list:
(defn my-map [f coll]
(reduce (fn [result item]
(conj result (f item)))
[] coll))
(my-map inc [1 2 3]) => [2 3 4]
It is more common to reverse the result, not the input. This is a common idiom when handling singly-linked lists in a recursive fashion. It preserves linear complexity with this data structure.
You might want to offer different implementations for other seqs, e. g., vectors, maybe based on conj instead of cons.
I would not worry too much about elegance with this kind of exercise.
As it was already pointed out. You do not have to reverse the input.
cons add an item to the beginning of a sequence (even on vectors) whereas conj will always add in the most natural way, it always add an item in the fastest way possible for the collection. it will add from left to right for list, and from left to right for vectors.
I noticed that most suggested answers use 'reduce' so allow me to suggest this one using mainly recursion:
(defn my-map [f coll]
(loop [f f coll coll acc []]
(if (empty? coll)
acc
(recur f (rest coll) (conj acc (f (first coll)))))))
The thing i need - is simple queue, something, where i can put tasks, and retrieve them one-by-one from workers(without maintaining order of tasks).
I wrote something like this:
;; Definition
(def q (ref []))
;; Put
(defn put-in-queue [i]
(dosync (alter q conj i)))
;; Get
(defn get-from-queue []
(dosync
(let [v (peek q)]
(alter q pop)
v)))
Is it correct solution? (maybe there are also better solutions for this task)
You should use a clojure.lang.PersistentQueue, right tool for the job and all that :).
Sample below "borrowed" from Hidden features of Clojure:
user> (-> (clojure.lang.PersistentQueue/EMPTY)
(conj 1 2 3)
pop)
(2 3)
See the remainder of that answer for specifics.
The correct solution is probably to use a java.util.concurrent queue, most likely java.util.concurrent.LinkedBlockingQueue. j.u.c queues are robust, perfectly suited to the task and work in Clojure just fine.
See my answer to the Producer consumer with qualifications SO question for a scenario with 5 producers, 2 consumers and a limited-size queue implemented in two ways for comparison: first with c.l.PersistentQueue, second with j.u.c.LinkedBlockingQueue.
as dsm points out, when you need a queue, use a queue, and for times where you really need a vector and want to add things to the end and remove them from the front, subvec and conj are both O(1) functions on vectors.
user> (subvec [1 2 3] 1)
[2 3]
user> (conj [1 2 3] 4)
[1 2 3 4]
user> (-> [] (conj 1 2) (conj 3) (subvec 1))