I am having a problem understanding how these functions update the underlying ref, atom etc.
The docs say:
(apply f current-value-of-identity args)
(def one (atom 0))
(swap! one inc) ;; => 1
So I am wondering how it got "expanded" to the apply form. It's not mentioned what exactly 'args' in the apply form is. Is it a sequence of arguments or are these separate values?
Was it "expanded" to:
(apply inc 0) ; obviously this wouldnt work, so that leaves only one possibility
(apply inc 0 '())
(swap! one + 1 2 3) ;; #=> 7
Was it:
(apply + 1 1 2 3 '()) ;or
(apply + 1 [1 2 3])
(def two (atom []))
(swap! two conj 10 20) ;; #=> [10 20]
Was it:
(apply conj [] [10 20]) ;or
(apply conj [] 10 20 '())
The passage you quoted from swap!'s docstring means that what happens is the equivalent of swapping in a new value for the Atom obtained from the old one with (apply f old-value args), where args is a seq of all additional arguments passed to swap!.
What actually happens is different, but that's just an implementation detail. For the sake of curiosity: Atoms have a Java method called swap, which is overloaded to take from one to four arguments. The first one is always an IFn (the f passed to swap!); the second and third, in present, are the first two extra arguments to that IFn; the fourth, if present, is an ISeq of extra arguments beyond the first two. apply is never involved and the fixed arity cases don't even call the IFn's applyTo method (they just use invoke). This improves performance in the common case where not too many extra arguments are passed to swap!.
Related
It appears that apply forces the realization of four elements given a lazy sequence.
(take 1
(apply concat
(repeatedly #(do
(println "called")
(range 1 10)))))
=> "called"
=> "called"
=> "called"
=> "called"
Is there a way to do an apply which does not behave this way?
Thank You
Is there a way to do an apply which does not behave this way?
I think the short answer is: not without reimplementing some of Clojure's basic functionality. apply's implementation relies directly on Clojure's implementation of callable functions, and tries to discover the proper arity of the given function to .invoke by enumerating the input sequence of arguments.
It may be easier to factor your solution using functions over lazy, un-chunked sequences / reducers / transducers, rather than using variadic functions with apply. For example, here's your sample reimplemented with transducers and it only invokes the body function once (per length of range):
(sequence
(comp
(mapcat identity)
(take 1))
(repeatedly #(do
(println "called")
(range 1 10))))
;; called
;; => (1)
Digging into what's happening in your example with apply, concat, seq, LazySeq, etc.:
repeatedly returns a new LazySeq instance: (lazy-seq (cons (f) (repeatedly f))).
For the given 2-arity (apply concat <args>), apply calls RT.seq on its argument list, which for a LazySeq then invokes LazySeq.seq, which will invoke your function
apply then calls a Java impl. method applyToHelper which tries to get the length of the argument sequence. applyToHelper tries to determine the length of the argument list using RT.boundedLength, which internally calls next and in turn seq, so it can find the proper overload of IFn.invoke to call
concat itself adds another layer of lazy-seq behavior.
You can see the stack traces of these invocations like this:
(take 1
(repeatedly #(do
(clojure.stacktrace/print-stack-trace (Exception.))
(range 1 10))))
The first trace descends from the apply's initial call to seq, and the subsequent traces from RT.boundedLength.
in fact, your code doesn't realize any of the items from the concatenated collections (ranges in your case). So the resulting collection is truly lazy as far as elements are concerned. The prints you get are from the function calls, generating unrealized lazy seqs. This one could easily be checked this way:
(defn range-logged [a b]
(lazy-seq
(when (< a b)
(println "realizing item" a)
(cons a (range-logged (inc a) b)))))
user> (take 1
(apply concat
(repeatedly #(do
(println "called")
(range-logged 1 10)))))
;;=> called
;; called
;; called
;; called
;; realizing item 1
(1)
user> (take 10
(apply concat
(repeatedly #(do
(println "called")
(range-logged 1 10)))))
;; called
;; called
;; called
;; called
;; realizing item 1
;; realizing item 2
;; realizing item 3
;; realizing item 4
;; realizing item 5
;; realizing item 6
;; realizing item 7
;; realizing item 8
;; realizing item 9
;; realizing item 1
(1 2 3 4 5 6 7 8 9 1)
So my guess is that you have nothing to worry about, as long as the collection returned from repeatedly closure is lazy
From what I gather a transformer is the use of functions that change , alter , a collection of elements . Like if I did added 1 to each element in a collection of
[1 2 3 4 5]
and it became
[2 3 4 5 6]
but writing the code for this looks like
(map inc)
but I keep getting this sort of code confused with a reducer. Because it produces a new accumulated result .
The question I ask is , what is the difference between a transformer and a reducer ?
You are likely just confusing various nomenclature (as the comments above suggest), but I'll answer what I think is your question by taking some liberties in interpreting what you mean to be reducer and transformer.
Reducing:
A reducing function (what you probably think is a reducer), is a function that takes an accumulated value and a current value, and returns a new accumulated value.
(accumulated, current) => accumulated
These functions are passed to reduce, and they successively step through a sequence performing whatever the body of the reducing function says with it's two arguments (accumulated and current), and then returning a new accumulated value which will be used as the accumulated value (first argument) to the next call of the reducing function.
For example, plus can be viewed as a reducing function.
(reduce + [0 1 2]) => 3
First, the reducing function (plus in this example) is called with 0 and 1, which returns 1. On the next call, 1 is now the accumulated value, and 2 is the current value, so plus is called with 1 and 2, returning 3, which completes the reduction as there are no further elements in the collection to process.
It may help to look at a simplified version of a reduce implementation:
(defn reduce1
([f coll] ;; f is a reducing function
(let [[x y & xs] coll]
;; called with the accumulated value so far "x"
;; and cur value in input sequence "y"
(if y (reduce1 f (cons (f x y) xs))
x)))
([f start coll]
(reduce1 f (cons start coll))))
You can see that the function "f" , or the "reducing function" is called on each iteration with two arguments, the accumulated value so far, and the next value in the input sequence. The return value of this function is used as the first argument in the next call, etc. and thus has the type:
(x, y) => x
Transforming:
A transformation, the way I think you mean it, suggests the shape of the input does not change, but is simply modified according to an arbitrary function. This would be functions you pass to map, as they are applied to each element and build up a new collection of the same shape, but with that function applied to each element.
(map inc [0 1 2]) => '(1 2 3)
Notice the shape is the same, it's still a 3 element sequence, whereas in the reduction above, you input a 3 element sequence and get back an integer. Reductions can change the shape of the final result, map does not.
Note that I say the "shape" doesn't change, but the type of each element may change depending on what your "transforming" function does:
(map #(list (inc %)) [0 1 2]) => '((1) (2) (3))
It's still a 3 element sequence, but now each element is a list, not an integer.
Addendum:
There are two related concepts in Clojure, Reducers and Transducers, which I just wanted to mention since you asked about reducers (which have as specific meaning in Clojure) and transformers (which are the names Clojurists typically assign to a transducing function via the shorthand "xf"). It would turn this already long answer into a short-story if I tried to explain the details of both here, and it's been done better than I can do by others:
Transducers:
http://elbenshira.com/blog/understanding-transducers/
https://www.youtube.com/watch?v=6mTbuzafcII
Reducers and Transducers:
https://eli.thegreenplace.net/2017/reducers-transducers-and-coreasync-in-clojure/
It turns out that many transformations of collections can be expressed in terms of reduce. For instance map could be implemented as
(defn map [f coll] (reduce (fn [x y] (conj x (f y))) [] [0 1 2 3 4]))
and then you would call
(map inc [1 2 3 4 5])
to obtain
[2 3 4 5 6]
In our homemade implementation of map, the function that we pass to reduce is
(fn [x y] (conj x (f y))))
where f is the function that we would like to apply to every element. So we can write a function that produces such a function for us, passing the function that we would like to map.
(defn mapping-with-conj [f] (fn [x y] (conj x (f y))))
But we still see the presence of conj in the above function assuming we want to add elements to a collection. We can get even more flexibility by extra indirection:
(defn mapping [f] (fn [step] (fn [x y] (step x (f y)))))
Then we can use it like this:
(def increase-by-1 (mapping inc))
(reduce (increase-by-1 conj) [] [1 2 3])
The (map inc) you are referring does what our call to (mapping inc) does. Why would you want to do things this way? The answer is that it gives us a lot of flexibility to build things. For instance, instead of building up a collection, we can do
(reduce ((map inc) +) 0 [1 2 3 4 5])
Which will give us the sum of the mapped collection [2 3 4 5 6]. Or we can add extra processing steps just by simple function composition.
(reduce ((comp (filter odd?) (map inc)) conj) [] [1 2 3 4 5])
which will first remove even elements from the collection before we map. The transduce function in Clojure does essentially what the above line does, but takes care of another few extra details, too. So you would actually write
(transduce (comp (filter odd?) (map inc)) conj [] [1 2 3 4 5])
To sum up, the map function in Clojure has two arities. Calling it like (map inc [1 2 3 4 5]) will map every element of a collection so that you obtain [2 3 4 5 6]. Calling it just like (map inc) gives us a function that behaves pretty much like our mapping function in the above explanation.
We can define and use an anonymous function like this:
repl=> (#(+ 10 %) 1)
11
But -> macro won't accept such anonymous functions. Say I want to add 10 and then multiply by 2. I'd try to write:
(-> 5 #(+ 10 %) #(* 2 %))
But that is not the correct code for some reason, the correct code is
(-> 5 (+ 10) (* 2))
What is the difference between (+ 10) and #(+ 10 %), and why won't -> macro accept anonymous functions defined with #()?
Here's my attempt at an explanation. There are two parts.
First, the anonymous literal syntax. When you write #(+ 10 %), it gets expanded into something that is functionally similar to the following:
(fn [x] (+ 10 x))
For ex.
=> (macroexpand '(#(+ 10 %))
Would return something like:
(fn* [p1__7230#] (+ 10 p1__7230#))
The second part. When you use the threading macro, as the docs say, the macro expands by inserting the first argument as the second item into the first form. And if there are more forms, inserts the first form as the second item in second form, and so on.
The key term here is second item. It doesn't care about what forms you are providing as arguments, it will just do an expansion using that rule.
So, to combine both the points, when you use
(-> 5 #(+ 10 %) #(* 2 %))
following the rules, it gets expanded into something that is functionally similar to this
(fn (fn 5 [x] (+ 10 x)) [y] (* 2 y))
which doesn't compile.
Also, as a side note, the form (+ 10) is not an anonymous function. It is a partial function call that gets updated with arguments during macro expansion. And by 'partial', I mean in the literal sense, not in the functional programming sense.
Update
To explain why it works when you enclose the anonymous literal within parentheses (as a comment on the question says), you can infer the results from these two rules. For ex.
=> (macroexpand '(#(+ 10 %)))
would result in the functional equivalent of
((fn [x] (+ 10 x)))
So, when an item is inserted in its second place, it would look like
((fn [x] (+ 10 x)) 5)
Which is equivalent to
(#(+ 10 %) 5)
I'm a newbie to Clojure and I was wondering if there is a way to define a function that can be called like this:
(strange-adder 1 2 3 :strange true)
That is, a function that can receive a variable number of ints and a keyword argument.
I know that I can define a function with keyword arguments this way:
(defn strange-adder
[a b c & {:keys [strange]}]
(println strange)
(+ a b c))
But now my function can only receive a fixed number of ints.
Is there a way to use both styles at the same time?
unfortunately, no.
The & destructuring operator uses everything after it on the argument list so it does not have the ability to handle two diferent sets of variable arity destructuring groups in one form.
one option is to break the function up into several arities. Though this only works if you can arrange it so only one of them is variadic (uses &). A more universal and less convenient solution is to treat the entire argument list as one variadic form, and pick the numbers off the start of it manually.
user> (defn strange-adder
[& args]
(let [nums (take-while number? args)
opts (apply hash-map (drop-while number? args))
strange (:strange opts)]
(println strange)
(apply + nums)))
#'user/strange-adder
user> (strange-adder 1 2 3 4 :strange 4)
4
10
Move the variadic portion to the the tail of the argument list and pass the options as a map:
(defn strange-adder [{:keys [strange]} & nums]
(println strange)
(apply + nums))
(strange-adder {:strange true} 1 2 3 4 5)
There is no formal support that I know of, but something like this should be doable:
(defn strange-adder
[& args]
(if (#{:strange} (-> args butlast last))
(do (println (last args))
(apply + (drop-last 2 args)))
(apply + args)))
I don't know if this can be generalized (check for keywords? how to expand to an arbitrary number of final arguments?). One option may be putting all options in a hashmap as the final argument, and checking if the last argument is a hashmap (but this would not work for some functions that expect arbitrary arguments that could be hashmaps).
I have a sequence (foundApps) returned from a function and I want to map a function to all it's elements. For some reason, apply and count work for the sequnece but map doesn't:
(apply println foundApps)
(map println rest foundApps)
(map (fn [app] (println app)) foundApps)
(println (str "Found " (count foundApps) " apps to delete"))))
Prints:
{:description another descr, :title apptwo, :owner jim, :appstoreid 1235, :kind App, :key #<Key App(2)>} {:description another descr, :title apptwo, :owner jim, :appstoreid 1235, :kind App, :key #<Key App(4)>}
Found 2 apps to delete for id 1235
So apply seems to happily work for the sequence, but map doesn't. Where am I being stupid?
I have a simple explanation which this post is lacking. Let's imagine an abstract function F and a vector. So,
(apply F [1 2 3 4 5])
translates to
(F 1 2 3 4 5)
which means that F has to be at best case variadic.
While
(map F [1 2 3 4 5])
translates to
[(F 1) (F 2) (F 3) (F 4) (F 5)]
which means that F has to be single-variable, or at least behave this way.
There are some nuances about types, since map actually returns a lazy sequence instead of vector. But for the sake of simplicity, I hope it's pardonable.
Most likely you're being hit by map's laziness. (map produces a lazy sequence which is only realised when some code actually uses its elements. And even then the realisation happens in chunks, so that you have to walk the whole sequence to make sure it all got realised.) Try wrapping the map expression in a dorun:
(dorun (map println foundApps))
Also, since you're doing it just for the side effects, it might be cleaner to use doseq instead:
(doseq [fa foundApps]
(println fa))
Note that (map println foundApps) should work just fine at the REPL; I'm assuming you've extracted it from somewhere in your code where it's not being forced. There's no such difference with doseq which is strict (i.e. not lazy) and will walk its argument sequences for you under any circumstances. Also note that doseq returns nil as its value; it's only good for side-effects. Finally I've skipped the rest from your code; you might have meant (rest foundApps) (unless it's just a typo).
Also note that (apply println foundApps) will print all the foundApps on one line, whereas (dorun (map println foundApps)) will print each member of foundApps on its own line.
A little explanation might help. In general you use apply to splat a sequence of elements into a set of arguments to a function. So applying a function to some arguments just means passing them in as arguments to the function, in a single function call.
The map function will do what you want, create a new seq by plugging each element of the input into a function and then storing the output. It does it lazily though, so the values will only be computed when you actually iterate over the list. To force this you can use the (doall my-seq) function, but most of the time you won't need to do that.
If you need to perform an operation immediately because it has side effects, like printing or saving to a database or something, then you typically use doseq.
So to append "foo" to all of your apps (assuming they are strings):
(map (fn [app] (str app "foo")) found-apps)
or using the shorhand for an anonymous function:
(map #(str % "foo") found-apps)
Doing the same but printing immediately can be done with either of these:
(doall (map #(println %) found-apps))
(doseq [app found-apps] (println app))