I'm working on my first-ever functional program in Clojure. I'm having some issues figuring out how to step through each item in a list, in each list in a list, and operate on it while keeping return values. I'm sure the issue comes from my unfamiliarity with Clojure and functional programming and was hoping someone could explain the best method to do the following:
psuedo-code algorithm:
for each lst in list
for each item in lst
return_values.append = do_something(item)
I first tried nesting two doseq functions and then calling my do_something function, which worked to call the function on the item, but didn't save my return values. I then tried a for and cons to an empty list, but was unable to get my return values outside of the for.
Would it be possible/preferable to break the list of lists down first? Could I still get a list of lists of return values?
In the end, I would like the result to be a list of lists of return values to match the input list of lists.
If anyone could explain the best method for doing this in Clojure, and why, it would be much appreciated.
Nested for loop will do the trick:
(for [lst my-list]
(for [item lst] (do_something item)))
It will take nested list my-list (list of lists) and convert it into another nested list by applying do_something to each element.
In clojure, for returns a list of values already, so there is no need to handle it yourself. Furthermore, since all data structures in clojure are immutable, you can't do this by appending elements to initially empty list with cons.
If you have a deeply nested list and you want to keep its structure, but transform the values, you can use clojure.walk/postwalk to operate on each value, e.g.:
(def nested '(1 (2 3 (4 5)) 6))
(defn transform-values [coll f]
(clojure.walk/postwalk #(if (not (list? %))
(f %)
%)
coll))
(transform-values nested inc)
=> (2 (3 4 (5 6)) 7)
You can, of course, pass any function to transform-values.
This can be done as a simple recursive walk. The first implementation that comes to mind for this would be the following for sequences:
(defn deep-walk
[f data]
(map (fn [s] (if (seq? s)
(deep-walk f s)
(f s)))
data))
And this slight variation for vectors:
(defn vec-deep-walk
[f data]
(vec (map (fn [s] (if (vector? s)
(vec-deep-walk f s)
(f s)))
data)))
Just a quick test with the following:
(vec-deep-walk (partial + 1) [1 [2 3] 4 [5 [6 7]]])
Gives the following output:
[2 [3 4] 5 [6 [7 8]]]
The walk functions take two parameters, the first is a function that takes a single parameter. This will be called for each non-seq/vector element in your data, which is passed as the second parameter. The results will be returned in a nested structure that is identical to the input structure.
Related
From what I gather a transformer is the use of functions that change , alter , a collection of elements . Like if I did added 1 to each element in a collection of
[1 2 3 4 5]
and it became
[2 3 4 5 6]
but writing the code for this looks like
(map inc)
but I keep getting this sort of code confused with a reducer. Because it produces a new accumulated result .
The question I ask is , what is the difference between a transformer and a reducer ?
You are likely just confusing various nomenclature (as the comments above suggest), but I'll answer what I think is your question by taking some liberties in interpreting what you mean to be reducer and transformer.
Reducing:
A reducing function (what you probably think is a reducer), is a function that takes an accumulated value and a current value, and returns a new accumulated value.
(accumulated, current) => accumulated
These functions are passed to reduce, and they successively step through a sequence performing whatever the body of the reducing function says with it's two arguments (accumulated and current), and then returning a new accumulated value which will be used as the accumulated value (first argument) to the next call of the reducing function.
For example, plus can be viewed as a reducing function.
(reduce + [0 1 2]) => 3
First, the reducing function (plus in this example) is called with 0 and 1, which returns 1. On the next call, 1 is now the accumulated value, and 2 is the current value, so plus is called with 1 and 2, returning 3, which completes the reduction as there are no further elements in the collection to process.
It may help to look at a simplified version of a reduce implementation:
(defn reduce1
([f coll] ;; f is a reducing function
(let [[x y & xs] coll]
;; called with the accumulated value so far "x"
;; and cur value in input sequence "y"
(if y (reduce1 f (cons (f x y) xs))
x)))
([f start coll]
(reduce1 f (cons start coll))))
You can see that the function "f" , or the "reducing function" is called on each iteration with two arguments, the accumulated value so far, and the next value in the input sequence. The return value of this function is used as the first argument in the next call, etc. and thus has the type:
(x, y) => x
Transforming:
A transformation, the way I think you mean it, suggests the shape of the input does not change, but is simply modified according to an arbitrary function. This would be functions you pass to map, as they are applied to each element and build up a new collection of the same shape, but with that function applied to each element.
(map inc [0 1 2]) => '(1 2 3)
Notice the shape is the same, it's still a 3 element sequence, whereas in the reduction above, you input a 3 element sequence and get back an integer. Reductions can change the shape of the final result, map does not.
Note that I say the "shape" doesn't change, but the type of each element may change depending on what your "transforming" function does:
(map #(list (inc %)) [0 1 2]) => '((1) (2) (3))
It's still a 3 element sequence, but now each element is a list, not an integer.
Addendum:
There are two related concepts in Clojure, Reducers and Transducers, which I just wanted to mention since you asked about reducers (which have as specific meaning in Clojure) and transformers (which are the names Clojurists typically assign to a transducing function via the shorthand "xf"). It would turn this already long answer into a short-story if I tried to explain the details of both here, and it's been done better than I can do by others:
Transducers:
http://elbenshira.com/blog/understanding-transducers/
https://www.youtube.com/watch?v=6mTbuzafcII
Reducers and Transducers:
https://eli.thegreenplace.net/2017/reducers-transducers-and-coreasync-in-clojure/
It turns out that many transformations of collections can be expressed in terms of reduce. For instance map could be implemented as
(defn map [f coll] (reduce (fn [x y] (conj x (f y))) [] [0 1 2 3 4]))
and then you would call
(map inc [1 2 3 4 5])
to obtain
[2 3 4 5 6]
In our homemade implementation of map, the function that we pass to reduce is
(fn [x y] (conj x (f y))))
where f is the function that we would like to apply to every element. So we can write a function that produces such a function for us, passing the function that we would like to map.
(defn mapping-with-conj [f] (fn [x y] (conj x (f y))))
But we still see the presence of conj in the above function assuming we want to add elements to a collection. We can get even more flexibility by extra indirection:
(defn mapping [f] (fn [step] (fn [x y] (step x (f y)))))
Then we can use it like this:
(def increase-by-1 (mapping inc))
(reduce (increase-by-1 conj) [] [1 2 3])
The (map inc) you are referring does what our call to (mapping inc) does. Why would you want to do things this way? The answer is that it gives us a lot of flexibility to build things. For instance, instead of building up a collection, we can do
(reduce ((map inc) +) 0 [1 2 3 4 5])
Which will give us the sum of the mapped collection [2 3 4 5 6]. Or we can add extra processing steps just by simple function composition.
(reduce ((comp (filter odd?) (map inc)) conj) [] [1 2 3 4 5])
which will first remove even elements from the collection before we map. The transduce function in Clojure does essentially what the above line does, but takes care of another few extra details, too. So you would actually write
(transduce (comp (filter odd?) (map inc)) conj [] [1 2 3 4 5])
To sum up, the map function in Clojure has two arities. Calling it like (map inc [1 2 3 4 5]) will map every element of a collection so that you obtain [2 3 4 5 6]. Calling it just like (map inc) gives us a function that behaves pretty much like our mapping function in the above explanation.
Updating a vector works fine:
(update [{:idx :a} {:idx :b}] 1 (fn [_] {:idx "Hi"}))
;; => [{:idx :a} {:idx "Hi"}]
However trying to do the same thing with a list does not work:
(update '({:idx :a} {:idx :b}) 1 (fn [_] {:idx "Hi"}))
;; => ClassCastException clojure.lang.PersistentList cannot be cast to clojure.lang.Associative clojure.lang.RT.assoc (RT.java:807)
Exactly the same problem exists for assoc.
I would like to do update and overwrite operations on lazy types rather than vectors. What is the underlying issue here, and is there a way I can get around it?
The underlying issue is that the update function works on associative structures, i.e. vectors and maps. Lists can't take a key as a function to look up a value.
user=> (associative? [])
true
user=> (associative? {})
true
user=> (associative? `())
false
update uses get behind the scenes to do its random access work.
I would like to do update and overwrite operations on lazy types
rather than vectors
It's not clear what want to achieve here. You're correct that vectors aren't lazy, but if you wish to do random access operations on a collection then vectors are ideal for this scenario and lists aren't.
and is there a way I can get around it?
Yes, but you still wouldn't be able to use the update function, and it doesn't look like there would be any benefit in doing so, in your case.
With a list you'd have to walk the list in order to access an index somewhere in the list - so in many cases you'd have to realise a great deal of the sequence even if it was lazy.
You can define your own function, using take and drop:
(defn lupdate [list n function]
(let [[head & tail] (drop n list)]
(concat (take n list)
(cons (function head) tail))))
user=> (lupdate '(a b c d e f g h) 4 str)
(a b c d "e" f g h)
With lazy sequences, that means that you will compute the n first values (but not the remaining ones, which after all is an important part of why we use lazy sequences). You have also to take into account space and time complexity (concat, etc.). But if you truly need to operate on lazy sequences, that's the way to go.
Looking behind your question to the problem you are trying to solve:
You can use Clojure's sequence functions to construct a simple solution:
(defn elf [n]
(loop [population (range 1 (inc n))]
(if (<= (count population) 1)
(first population)
(let [survivors (->> population
(take-nth 2)
((if (-> population count odd?) rest identity)))]
(recur survivors)))))
For example,
(map (juxt identity elf) (range 1 8))
;([1 1] [2 1] [3 3] [4 1] [5 3] [6 5] [7 7])
This has complexity O(n). You can speed up count by passing the population count as a redundant argument in the loop, or by dumping the population and survivors into vectors. The sequence functions - take-nth and rest - are quite capable of doing the weeding.
I hope I got it right!
I've got a function like this:
(defn magic
[a b c]
(flatten (conj [] a b c)))
So on these inputs I get the following:
(magic 1 2 3) => (1 2 3)
(magic 1 [2 3] 4) => (1 2 3 4)
My question is, is there a better way of doing this?
The problem can be summarised as:
I don't know whether I will get numbers or vectors as input, but I need to return a single flat list
This could be slightly simplified (and generalized) as:
(defn magic [& args]
(flatten (apply list args)))
Or, as pointed out in the comments, it can be simplified even further (since args above is already a seq):
(defn magic [& args]
(flatten args))
Other than that, I don't see much else that can be improved about this. Is there anything in particular that's bothering you about your implementation?
If you can get seqs of seqs then you need to be more careful. And will have to recursively go into the list. There is a clojure native function for this tree-seq see the examples here:
http://clojuredocs.org/clojure_core/clojure.core/tree-seq
You'd want something like this (untested):
(defn nonempty-seq [x]
"returns x as a seq if it's a non-empty seq otherwise nil/false"
(and (coll? x) (seq x)))
(tree-seq nonempty-seq seq expr)
I see some examples that show we can get a nice head/tail destructuring of a sequence in clojure as follows:
(if-let [[x & xs] (seq coll)]
However I assume this won't work as desired for lazy sequences because this puts the values into a vector, which aren't lazy. I tried changing the vector form to a list form, and it gave me binding errors, quoted or not.
Without having binding like this, it seems that if I've got a lazy sequence where each element was a computationally-intensive equation of the previous element, I'd have to do that computation twice to get the head and tail as separate statements, right?
(let [head (first my-lazy-seq) ;; has to calculate the value of head.
tail (rest my-lazy-seq)] ;; also has to calculate the value of head to prepare the rest of the sequence.
Is there any way around this, or am I making an incorrect assumption somewhere?
user=> (let [[x & xs] (range)] [x (take 10 xs)])
[0 (1 2 3 4 5 6 7 8 9 10)]
xs is still a lazy seq, so you can use the destructuring with no problems. This will force the first element of xs, though. (Destructuring uses vector notation, but it doesn't necessarily use vectors under the covers.)
With respect to your second question: lazy seqs cache their results, so your second option would also work without extra recalculation. The head will only be calculated once.
The binding vector [x & xs] isn't actually constructing a vector at runtime. It's just the notation used for destructuring into head & tail.
So it works fine on infinite sequences:
(if-let [[x & xs] (range)]
(apply str x (take 9 xs)))
=> "0123456789"
The destructuring form is actually producing a lazy sequence in this case, which you can observe as follows:
(if-let [[x & xs :as my-seq] (range)]
(class my-seq))
=> clojure.lang.LazySeq
In lisp, I can pass an argument to a function and have it altered within the function. (AKA destructive functions). However, in Clojure, I've read somewhere that it is not permissible to alter the given arguments within that same function. For example:
(defn add-two-lists [list1 list2]
(for [n (range (count list1))]
(+ (nth list1 n) (nth list2 n))))
This is a normal function and its output is the addition of the two identical lists. However, I want something like this:
(defn add-two-lists [list1 list2 added_list]
(set! added_list
(for [n (range (count list1))]
(+ (nth list1 n) (nth list2 n)))))
Perhaps my use of set! is wrong or misused, and I still get errors. Is there a elegant way to destructively modify arguments in Clojure?
Destructive modification is discouraged in Clojure - I would encourage you to find ways to write your code without resorting to destructive updates.
In the spirit of giving a Clojurey solution, I would write your add-two-lists function as follows:
(defn add-two-lists [list1 list2]
(map + list1 list2))
This has a few advantages:
It's purely functional
It's lazy, so you can even add lists of infinite length (try doing that with a destructively updated argument!)
It's performance is O(n) which is optimal - the versions in the question are actually O(n^2) since nth is itself an O(n) operation on lists.
It's nice and concise :-)
Clojure Provides several mutable types that would work well in this situation, for instance you could pass an atom to the function and have it set the value in that atom.
(defn add-two-lists [list1 list2 added_list]
(reset! added_list
(for [n (range (count list1))]
(+ (nth list1 n) (nth list2 n)))))
then after you call this you get the value out of the atom with #/deref
edit: if efficiency is the goal then using a transient collection may help
The with-local-vars macro lets you create thread-locally bound vars that you can modify with var-set. You also have to access the var's value with var-get, which can be shortened to just #.
(defn add-two-lists [list1 list2 added-list]
(var-set added-list
(for [n (range (count list1))]
(+ (nth list1 n) (nth list2 n)))))
(with-local-vars [my-list nil]
(add-two-lists '(1 2 3) '(3 4 5) my-list)
#my-list)
EDIT:
On a stylistic note, you could use map to add the two lists without using the nth function to random-access each index in each list:
(defn add-two-lists [list1 list2 added-list]
(var-set added-list (map + list1 list2)))
From the clojure documentation on set!
Note - you cannot assign to function params or local bindings. Only Java fields, Vars, Refs and Agents are mutable in Clojure.
Typically in courses where functional languages are chosen, you are encouraged not to use for-loops and assignments. Instead you should favor recursion and composition of functions.
So if I wanted to add 2 to each element of a list, in an imperative language, I would just do a for loop, but in a functional language, I would use recursion
user=> (def add2
(fn [mylist]
(if
(empty? mylist)
nil
(cons (+ (first mylist) 2) (add2 (rest mylist))))))
user=> (add2 (list 1 2 3))
(3 4 5)