Clojure struggling to visualise how the arrow macros work - clojure

Question

I am struggling to understand how the -> and ->> expand.
If I have the following:
(-> query
(write-query-to-excel db csv-file)
(archiver archive-password)
(send-with-attachment send-to subject body)
(clean-up))
How does this expand, I understand that -> is a thread-first macro which inserts the the value as the second item as the first form, and after looking at examples I'm still unsure.
Also how does ->> differ, I know it inserts the item as the last argument of the function but again the wording confuses me.
(->> query
(write-query-to-excel db csv-file)
(archiver archive-password)
(send-with-attachment send-to subject body)
(clean-up))
When expanded how would these two macros look?

Answer 1

-> and ->> always remind me of Matryoshka dolls because they nest all expressions passed to them.
Let's consider (-> a (b c d) (e f g)). We can now visually cut them (with a pipe) at the 1st argument position in each expression but the first: (-> a (b | c d) (e | f g)). Now that the expressions are cut in halves we can nest them: first (-> (b a c d) (e | f g)) and then (-> (e (b a c d) f g)) which simplifies to (e (b a c d) f g).
->> works in a very similar manner except the insert point is at the end (last argument position): (-> a (b c d |) (e f g |)) which successively expands to (-> (b c d a) (e f g |)), (-> (e f g (b c d a))) and (e f g (b c d a)).

Answer 2

Using clojure.walk/macroexpand to expand the macros:
(clojure.walk/macroexpand-all '(-> query
(write-query-to-excel db csv-file)
(archiver archive-password)
(send-with-attachment send-to subject body)
(clean-up)))
... produces
(clean-up (send-with-attachment (archiver (write-query-to-excel query db csv-file) archive-password) send-to subject body))
Whereas
(clojure.walk/macroexpand-all '(->> query
(write-query-to-excel db csv-file)
(archiver archive-password)
(send-with-attachment send-to subject body)
(clean-up)))
... produces
(clean-up (send-with-attachment send-to subject body (archiver archive-password (write-query-to-excel db csv-file query))))
Since the result doesn't have to make sense, we can use simpler examples to illustrate the difference:
(clojure.walk/macroexpand-all '(-> a b c d))
;(d (c (b a)))
, the same with -> or ->>. But if there are arguments:
(clojure.walk/macroexpand-all '(-> a (b 1 2 3)))
;(b a 1 2 3)
whereas
(clojure.walk/macroexpand-all '(->> a (b 1 2 3)))
;(b 1 2 3 a)

Answer 3

The answer from #cgrand is good, but I would like to add a bit more.
You could achieve something very similar by putting your thread-first example into a (let...) form as:
; better if descriptive names used for intermediate results
(let [tmp1 (write-query-to-excel db csv-file)
tmp2 (archiver tmp1 archive-password)
tmp3 (send-with-attachment tmp2 send-to subject body)
tmp4 (clean-up tmp3) ]
<process result in tmp4>
...)
This is often the best solution since you can use descriptive variable names for the intermediate results. If, however, you won't really benefit from naming the intermediate results, you can use a generic variable it as a placeholder and still see exactly where each intermediate result is used in the next form. This is how the it-> macro from the Tupelo Core library works:
Imagine using a variable it to mark the location in each form where the result of the previous expression is inserted. Then in your -> example, we get:
(ns my.proj
(:require [tupelo.core :refer [it->]]
... ))
(it-> (write-query-to-excel db csv-file)
(archiver it archive-password)
(send-with-attachment it send-to subject body)
(clean-up it))
Then in your ->> example we would have:
(it-> (write-query-to-excel db csv-file)
(archiver archive-password it)
(send-with-attachment send-to subject body it)
(clean-up it))
However, what do we do if the expressions do not all chain together in either the first or last argument position? Consider this code:
(it-> 1
(inc it)
(+ it 3)
(/ 10 it)
(str "We need to order " it " items." )
;=> "We need to order 2 items." )
In this case, we have expressions where the previous result is chained into the only, first, last, and middle argument positions. In this case, neither -> nor ->> will work, and one must use something like it-> or a generic (let ...) form.
So for simple expression pipelines, using it-> (or the similar as->) can make the code more explicit about how the expressions are chained together. It also allows you to chain expressions that are not all "thread-first" or "thread-last". For more complicated processing chains, using a let form (possible nested) is the best answer as you can name the intermediate results and then combine them in arbitrarily complicated ways.

Answer 4

Would you mind taking a little tour around programming languages?
Ruby
In the world of Ruby there is a common practice of making so-called "method chains". What it is, essentially, is:
calling a method
calling a method on its result
calling a method on that method's result
etc.
Ruby is object-oriented, and such chains look like this:
(-5..5).map(&:abs)
.select(&:even?)
.count
.to_s
take a range of integers from -5 to 5 inclusive
map them into their absolute values
only select those that are even
count how many you have
convert the resuling number into a string
In Ruby, collections are objects, methods like map, count and select are their methods. map returns a collection, on which you can call select. On the result of select we call count that outputs an integer. Then we call to_s on that integer to convert it into a string. The dot-call of object.method calls a method on an object. Familiar stuff.
In some languages you may be exposed to terrible truth that object's methods may be nothing but a simple function that takes an object it acts upon and arguments. Some languages thoroughly hide that fact as an implementation detail, but could that convention actually be a limitation?
Lua
Lua is a nice little and honest language that doesn't hide from you what object's methods are. Consider these two equivalent lines:
obj.increase(obj, 5)
obj:increase(5)
It's a convention enforced on language-level: a method is a function that takes its owner object as a first argument.
...and some others with "static extension"
Defining all the possibly needed methods in a type has proven to be impossible, so some languages have adopted "static extension": it allows you to create a function that takes an "object acted upon" as its first argument (without access to sources of the type itself!). Then with some syntactic sugar you can use x.ext(y) calls instead of ext(x, y):
C# has that.
Haxe has that.
C++ doesn't have that, but there are proposals to add this, that highlight the semantic similarities of x.f(y) and f(x, y)
That feature allows us to chain operations a slightly cleaner way.
g(f(x, y), z) # Before
x.f(y).g(z) # After
Clojure
And we're back! So what does all this have to do with -> and ->>? They provide similar-looking syntax that looks good in longer chains.
We know for sure that our map and filter (same as Ruby's select) are just functions that take collections they act upon as their last (oops, that was unexpected!) argument.
(->> (range -5 6) (map #(Math/abs %)) ; 6 not included
(filter even?)
(count)
(str))
Oh, so we see a bit different way here: functions we treat as methods here take an object they act upon as their last argument. But we just make the "chain" using ->> and that fact becomes less noticeable.
If we had functions that took "object acted upon" as their first argument, as we've seen in Lua, we'd use ->. Depending on the functions you're chaining, -> might make more sense.
We can have it both ways, we can even mix them with a tad more verbose as->.