Functional alternative to "let" - clojure

I find myself writing a lot of clojure in this manner:
(defn my-fun [input]
(let [result1 (some-complicated-procedure input)
result2 (some-other-procedure result1)]
(do-something-with-results result1 result2)))
This let statement seems very... imperative. Which I don't like. In principal, I could be writing the same function like this:
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
The problem with this is that it involves recomputation of some-complicated-procedure, which may be arbitrarily expensive. Also you can imagine that some-complicated-procedure is actually a series of nested function calls, and then I either have to write a whole new function, or risk that changes in the first invocation don't get applied to the second:
E.g. this works, but I have to have an extra shallow, top-level function that makes it hard to do a mental stack trace:
(defn some-complicated-procedure [input] (lots (of (nested (operations input)))))
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
E.g. this is dangerous because refactoring is hard:
(defn my-fun [input]
(do-something-with-results (lots (of (nested (operations (mistake input))))) ; oops made a change here that wasn't applied to the other nested calls
(some-other-procedure (lots (of (nested (operations input))))))))
Given these tradeoffs, I feel like I don't have any alternatives to writing long, imperative let statements, but when I do, I cant shake the feeling that I'm not writing idiomatic clojure. Is there a way I can address the computation and code cleanliness problems raised above and write idiomatic clojure? Are imperitive-ish let statements idiomatic?

The kind of let statements you describe might remind you of imperative code, but there is nothing imperative about them. Haskell has similar statements for binding names to values within bodies, too.

If your situation really needs a bigger hammer, there are some bigger hammers that you can either use or take for inspiration. The following two libraries offer some kind of binding form (akin to let) with a localized memoization of results, so as to perform only the necessary steps and reuse their results if needed again: Plumatic Plumbing, specifically the Graph part; and Zach Tellman's Manifold, whose let-flow form furthermore orchestrates asynchronous steps to wait for the necessary inputs to become available, and to run in parallel when possible. Even if you decide to maintain your present course, their docs make good reading, and the code of Manifold itself is educational.

I recently had this same question when I looked at this code I wrote
(let [user-symbols (map :symbol states)
duplicates (for [[id freq] (frequencies user-symbols) :when (> freq 1)] id)]
(do-something-with duplicates))
You'll note that map and for are lazy and will not be executed until do-something-with is executed. It's also possible that not all (or even not any) of the states will be mapped or the frequencies calculated. It depends on what do-something-with actually requests of the sequence returned by for. This is very much functional and idiomatic functional programming.

i guess the simplest approach to keep it functional would be to have a pass-through state to accumulate the intermediate results. something like this:
(defn with-state [res-key f state]
(assoc state res-key (f state)))
user> (with-state :res (comp inc :init) {:init 10})
;;=> {:init 10, :res 11}
so you can move on to something like this:
(->> {:init 100}
(with-state :inc'd (comp inc :init))
(with-state :inc-doubled (comp (partial * 2) :inc'd))
(with-state :inc-doubled-squared (comp #(* % %) :inc-doubled))
(with-state :summarized (fn [st] (apply + (vals st)))))
;;=> {:init 100,
;; :inc'd 101,
;; :inc-doubled 202,
;; :inc-doubled-squared 40804,
;; :summarized 41207}

The let form is a perfectly functional construct and can be seen as syntactic sugar for calls to anonymous functions. We can easily write a recursive macro to implement our own version of let:
(defmacro my-let [bindings body]
(if (empty? bindings)
body
`((fn [~(first bindings)]
(my-let ~(rest (rest bindings)) ~body))
~(second bindings))))
Here is an example of calling it:
(my-let [a 3
b (+ a 1)]
(* a b))
;; => 12
And here is a macroexpand-all called on the above expression, that reveal how we implement my-let using anonymous functions:
(clojure.walk/macroexpand-all '(my-let [a 3
b (+ a 1)]
(* a b)))
;; => ((fn* ([a] ((fn* ([b] (* a b))) (+ a 1)))) 3)
Note that the expansion doesn't rely on let and that the bound symbols become parameter names in the anonymous functions.

As others write, let is actually perfectly functional, but at times it can feel imperative. It's better to become fully comfortable with it.
You might, however, want to kick the tires of my little library tl;dr that lets you write code like for example
(compute
(+ a b c)
where
a (f b)
c (+ 100 b))

Related

Using let style destructuring for def

Is there a reasonable way to have multiple def statements happen with destructing the same way that let does it? For Example:
(let [[rtgs pcts] (->> (sort-by second row)
(apply map vector))]
.....)
What I want is something like:
(defs [rtgs pcts] (->> (sort-by second row)
(apply map vector)))
This comes up a lot in the REPL, notebooks and when debugging. Seriously feels like a missing feature so I'd like guidance on one of:
This exists already and I'm missing it
This is a bad idea because... (variable capture?, un-idiomatic?, Rich said so?)
It's just un-needed and I must be suffering from withdrawals from an evil language. (same as: don't mess up our language with your macros)
A super short experiment give me something like:
(defmacro def2 [[name1 name2] form]
`(let [[ret1# ret2#] ~form]
(do (def ~name1 ret1#)
(def ~name2 ret2#))))
And this works as in:
(def2 [three five] ((juxt dec inc) 4))
three ;; => 3
five ;; => 5
Of course and "industrial strength" version of that macro might be:
checking that number of names matches the number of inputs. (return from form)
recursive call to handle more names (can I do that in a macro like this?)
While I agree with Josh that you probably shouldn't have this running in production, I don't see any harm in having it as a convenience at the repl (in fact I think I'll copy this into my debug-repl kitchen-sink library).
I enjoy writing macros (although they're usually not needed) so I whipped up an implementation. It accepts any binding form, like in let.
(I wrote this specs-first, but if you're on clojure < 1.9.0-alpha17, you can just remove the spec stuff and it'll work the same.)
(ns macro-fun
(:require
[clojure.spec.alpha :as s]
[clojure.core.specs.alpha :as core-specs]))
(s/fdef syms-in-binding
:args (s/cat :b ::core-specs/binding-form)
:ret (s/coll-of simple-symbol? :kind vector?))
(defn syms-in-binding
"Returns a vector of all symbols in a binding form."
[b]
(letfn [(step [acc coll]
(reduce (fn [acc x]
(cond (coll? x) (step acc x)
(symbol? x) (conj acc x)
:else acc))
acc, coll))]
(if (symbol? b) [b] (step [] b))))
(s/fdef defs
:args (s/cat :binding ::core-specs/binding-form, :body any?))
(defmacro defs
"Like def, but can take a binding form instead of a symbol to
destructure the results of the body.
Doesn't support docstrings or other metadata."
[binding body]
`(let [~binding ~body]
~#(for [sym (syms-in-binding binding)]
`(def ~sym ~sym))))
;; Usage
(defs {:keys [foo bar]} {:foo 42 :bar 36})
foo ;=> 42
bar ;=> 36
(defs [a b [c d]] [1 2 [3 4]])
[a b c d] ;=> [1 2 3 4]
(defs baz 42)
baz ;=> 42
About your REPL-driven development comment:
I don't have any experience with Ipython, but I'll give a brief explanation of my REPL workflow and you can maybe comment about any comparisons/contrasts with Ipython.
I never use my repl like a terminal, inputting a command and waiting for a reply. My editor supports (emacs, but any clojure editor should do) putting the cursor at the end of any s-expression and sending that to the repl, "printing" the result after the cursor.
I usually have a comment block in the file where I start working, just typing whatever and evaluating it. Then, when I'm reasonably happy with a result, I pull it out of the "repl-area" and into the "real-code".
(ns stuff.core)
;; Real code is here.
;; I make sure that this part always basically works,
;; ie. doesn't blow up when I evaluate the whole file
(defn foo-fn [x]
,,,)
(comment
;; Random experiments.
;; I usually delete this when I'm done with a coding session,
;; but I copy some forms into tests.
;; Sometimes I leave it for posterity though,
;; if I think it explains something well.
(def some-data [,,,])
;; Trying out foo-fn, maybe copy this into a test when I'm done.
(foo-fn some-data)
;; Half-finished other stuff.
(defn bar-fn [x] ,,,)
(keys 42) ; I wonder what happens if...
)
You can see an example of this in the clojure core source code.
The number of defs that any piece of clojure will have will vary per project, but I'd say that in general, defs are not often the result of some computation, let alone the result of a computation that needs to be destructured. More often defs are the starting point for some later computation that will depend on this value.
Usually functions are better for computing a value; and if the computation is expensive, then you can memoize the function. If you feel you really need this functionality, then by all means, use your macro -- that's one of the sellings points of clojure, namely, extensibility! But in general, if you feel you need this construct, consider the possibility that you're relying too much on global state.
Just to give some real examples, I just referenced my main project at work, which is probably 2K-3K lines of clojure, in about 20 namespaces. We have about 20 defs, most of which are marked private and among them, none are actually computing anything. We have things like:
(def path-prefix "/some-path")
(def zk-conn (atom nil))
(def success? #{200})
(def compile* (clojure.core.memoize/ttl compiler {} ...)))
(def ^:private nashorn-factory (NashornScriptEngineFactory.))
(def ^:private read-json (comp json/read-str ... ))
Defining functions (using comp and memoize), enumerations, state via atom -- but no real computation.
So I'd say, based on your bullet points above, this falls somewhere between 2 and 3: it's definitely not a common use case that's needed (you're the first person I've ever heard who wants this, so it's uncommon to me anyway); and the reason it's uncommon is because of what I said above, i.e., it may be a code smell that indicates reliance on too much global state, and hence, would not be very idiomatic.
One litmus test I have for much of my code is: if I pull this function out of this namespace and paste it into another, does it still work? Removing dependencies on external vars allows for easier testing and more modular code. Sometimes we need it though, so see what your requirements are and proceed accordingly. Best of luck!

Create a clojure map threading macro ( map-> )

I'm inspired by clojure's 1.5 cond-> macro.
Similarily, I want to create a macro of the same idea, applied to the function map. However, I have no idea where to start.
For example, I can't find the source for cond->. (probably because it's not released yet)
Any suggestions?
There is the source of cond-> https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L6742
there are a variety of threading macros from the pallet project folks including apply-map-> which looks close to, though not exactly what you are looking for.
(letfn [(apply-map-
[arg f arg-coll]
`(let [arg# ~arg]
(apply ~f arg#
~#(butlast arg-coll)
(apply concat ~(last arg-coll)))))]
(defmacro apply-map->
"Apply in a threaded expression.
e.g.
(-> :a
(apply-map-> hash-map 1 {:b 2}))
=> {:a 1 :b 2}"
[arg f & arg-coll]
(apply-map- arg f arg-coll))
Perhaps there will be enough examples there for you to pick out what you need.
If I understand -- you want to write a macro that takes a list of partial function calls, and for each one, adds map (or apply map) to the beginning, and the previous result to the end?
While this doesn't directly answer how to write that macro, I wanted to point out that you have a couple of alternatives.
Factor out map
This is always true for pure functions:
(=
(map g (map f coll))
(map (comp g f) coll))
The refactored version only walks the collection once, and no intermediate collections need to be made.
Here's what it looks like with threading:
(=
(->> coll
(map f)
(map g))
(map #(->> % f g) coll))
Here's a concrete example in JS.
Transducers
Transducers are another pattern for doing this kind of thing in Clojure that work on more than just map. They're sort of an abstraction over reducer functions. Clojure's map / filter / reduce (etc.) will create a transducer if called without a collection. You can chain them with comp and use them in various contexts (lazy, eager, observable, whatever). Rich Hickey's talk on them is a good intro.

Could Clojure do without let?

I find I very rarely use let in Clojure. For some reason I took a dislike to it when I started learning and have avoided using it ever since. It feels like the flow has stopped when let comes along. I was wondering, do you think we could do without it altogether ?
You can replace any occurrence of (let [a1 b1 a2 b2...] ...) by ((fn [a1 a2 ...] ...) b1 b2 ...) so yes, we could. I am using let a lot though, and I'd rather not do without it.
Let offers a few benefits. First, it allows value binding in a functional context. Second, it confers readability benefits. So while technically, one could do away with it (in the sense that you could still program without it), the language would be impoverished without a valuable tool.
One of the nice things about let is that it helps formalize a common (mathematical) way of specifying a computation, in which you introduce convenient bindings and then a simplified formula as a result. It's clear the bindings only apply to that "scope" and it's tie in with a more mathematical formulation is useful, especially for more functional programmers.
It's not a coincidence that let blocks occur in other languages like Haskell.
Let is indispensable to me in preventing multiple execution in macros:
(defmacro print-and-run [s-exp]
`(do (println "running " (quote ~s-exp) "produced " ~s-exp)
s-exp))
would run s-exp twice, which is not what we want:
(defmacro print-and-run [s-exp]
`(let [result# s-exp]
(do (println "running " (quote ~s-exp) "produced " result#)
result#))
fixes this by binding the result of the expression to a name and referring to that result twice.
because the macro is returning an expression that will become part of another expression (macros are function that produce s-expressions) they need to produce local bindings to prevent multiple execution and avoid symbol capture.
I think I understand your question. Correct me if it's wrong. Some times "let" is used for imperative programming style. For example,
... (let [x (...)
y (...x...)
z (...x...y...)
....x...y...z...] ...
This pattern comes from imperative languages:
... { x = ...;
y = ...x...;
...x...y...;} ...
You avoid this style and that's why you also avoid "let", don't you?
In some problems imperative style reduces amount of code. Furthermore, some times It's more efficient to write in java or c.
Also in some cases "let" just holds values of subexpressions regardless of evaluation order. For example,
(... (let [a (...)
b (...)...]
(...a...b...a...b...) ;; still fp style
There are at least two important use cases for let-bindings:
First, using let properly can make your code clearer and shorter. If you have an expression that you use more than once, binding it in a let is very nice. Here's a portion of the standard function map that uses let:
...
(let [s1 (seq c1) s2 (seq c2)]
(when (and s1 s2)
(cons (f (first s1) (first s2))
(map f (rest s1) (rest s2)))))))
...
Even if you use an expression only once, it can still be helpful (to future readers of the code) to give it a semantically meaningful name.
Second, as Arthur mentioned, if you want to use the value of an expression more than once, but only want it evaluated once, you can't simply type out the entire expression twice: you need some kind of binding. This would be merely wasteful if you have a pure expression:
user=> (* (+ 3 2) (+ 3 2))
25
but actually changes the meaning of the program if the expression has side-effects:
user=> (* (+ 3 (do (println "hi") 2))
(+ 3 (do (println "hi") 2)))
hi
hi
25
user=> (let [x (+ 3 (do (println "hi") 2))]
(* x x))
hi
25
Stumbled upon this recently so ran some timings:
(testing "Repeat vs Let vs Fn"
(let [start (System/currentTimeMillis)]
(dotimes [x 1000000]
(* (+ 3 2) (+ 3 2)))
(prn (- (System/currentTimeMillis) start)))
(let [start (System/currentTimeMillis)
n (+ 3 2)]
(dotimes [x 1000000]
(* n n))
(prn (- (System/currentTimeMillis) start)))
(let [start (System/currentTimeMillis)]
(dotimes [x 1000000]
((fn [x] (* x x)) (+ 3 2)))
(prn (- (System/currentTimeMillis) start)))))
Output
Testing Repeat vs Let vs Fn
116
18
60
'let' wins over 'pure' functional.

Branching function composition: composition version of if/cond?

In Clojure, there are several option for composition of functions. There are composition functions for:
Apply: for 'unwrapping' arguments
Partial: for arguments that are not yet given
Comp: for piping consecutive results through multiple functions
Juxt: for applying one argument on multiple functions
However, AFAIK there are no such composition functions that include branching. Are there any functions that compose functions in a branching way, like a functional version of if or cond ?
Of course an if version is easy to make (though this implementation might not be the quickest):
(defn iff
([pred rtrue] (iff pred rtrue identity))
([pred rtrue rfalse]
(fn [& args]
(if (apply pred args)
(apply rtrue args)
(apply rfalse args)))))
There could be discussion about by default returning identity in the 'else' case is the right choice, or if nil should be returned in such case.
The use of such function could produce more easy to read code. Instead of #(if (string? %) (trim %) %) it would become (iff string? trim), or with a cond version:
(condf string? trim,
vector? (partial apply str),
:else identity)
Do other FP languages have such constructs ? I can imagine it might be handy in compositions with comp and juxt. Why doesn't Clojure ?
Bonus points for nice iff / condf implementations :)
I'm not sure if this is a direct match for what you're looking for (the question, to me, is somewhat vague), but you should look into Monads and Arrows.
Monads allow you to chain together functions with a specific "bind" function that defines how to chain them. It could do some sort of if/else pipelining, as in the Maybe and Either monads, or it could simulate state, as in the State monad.
Monads are built into Haskell (as monads) and F# (as "Workflows"). I have seen monad libraries for Clojure (check this out for one), and there are probably Arrow libraries too.
Well there could be many such composition pattern you can come up and ask why this isn't in the core language. The reason is obvious, it is not feasible. The core of the language provide you all the constructs to build such patterns. These sort of features are more of a contrib kind of thing rather than core of the language.
As far as implementation is concerned it would as simple as something shown below:
(defn condf [& args]
(let [chain (partition 2 args)]
(fn [& params]
(first (for [[p f] chain :when (or (= :else p) (apply p params))]
(apply f params))))))
(def my-func (condf string? clojure.string/trim
vector? (partial apply str)
:else identity))
(my-func "Ankur ") ==> "Ankur"
(my-func [1 2 3]) ==> "123"
(my-func '(1 2 3)) ==> (1 2 3)
This approaches the idea of Strategic Programming. You may find the following paper of interest
The Essence of Strategic Programming by
Ralf Lämmel and Eelco Visser and Joost Visser
http://homepages.cwi.nl/~ralf/eosp/
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1969

What is the standard way to write nested define statements (like in scheme) for clojure?

All examples are taken from the SICP Book: http://sicpinclojure.com/?q=sicp/1-3-3-procedures-general-methods
This was motivated from the MIT video series on LISP - http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/2a-higher-order-procedures/
In scheme, you can put 'define' inside another 'define':
(define (close-enough? v1 v2)
(define tolerance 0.00001)
(< (abs (- v1 v2)) tolerance ) )
In clojure, there is the 'let' statement with the only difference that it is nested:
(defn close-enough? [v1 v2]
(let [tolerance 0.00001]
(< (Math/abs (- v1 v2) )
tolerance) ) )
But what about rewriting in clojure something bigger like this?:
(define (sqrt x)
(define (fixed-point f first-guess)
(define (close-enough? v1 v2)
(define tolerance 0.00001)
(< (abs (- v1 v2)) tolerance))
(define (try guess)
(let ((next (f guess)))
(if (close-enough? guess next)
next
(try next))))
(try first-guess))
(fixed-point (lambda (y) (average y (/ x y)))
1.0))
This does in fact work but looks very unconventional...
(defn sqrt [n]
(let [precision 10e-6
abs #(if (< % 0) (- %) %)
close-enough? #(-> (- %1 %2) abs (< precision))
averaged-func #(/ (+ (/ n %) %) 2)
fixed-point (fn [f start]
(loop [old start
new (f start)]
(if (close-enough? old new)
new
(recur new (f new) ) ) ) )]
(fixed-point averaged-func 1) ) )
(sqrt 10)
UPDATED Mar/8/2012
Thanks for the answer!
Essentially 'letfn' is not too different from 'let' - the functions being called have to be nested in the 'letfn' definition (as opposed to Scheme where the functions are used in the next sexp after its definitions and only existing within the scope of the top-level function in which it is defined).
So another question... Why doesn't clojure give the capability of doing what scheme does? Is it some sort of language design decision? What I like about the scheme organization is:
1) The encapsulation of ideas so that I as the programmer have an idea as to what little blocks are being utilized bigger block - especially if I am only using the little blocks once within the big block (for whatever reason, even if the little blocks are useful in their own right).
2) This also stops polluting the namespace with little procedures that are not useful to the end user (I've written clojure programs, came back to them a week later and had to re-learn my code because it was in a flat structure and I felt that I was looking at the code inside out as opposed to in a top down manner).
3) A common method definition interface so I can pull out a particular sub-method, de-indent it test it, and paste the changed version back without too much fiddling around.
Why isn't this implemented in clojure?
the standard way to write nested, named, procedures in clojure is to use letfn.
as an aside, your example use of nested functions is pretty suspicious. all of the functions in the example could be top-level non-local functions since they're more or less useful on their own and don't close over anything but each other.
The people critizing his placement of this in all one function, don't understand the context of why its been done in such a manner. SICP, which is where the example is from, was trying to illustrate the concept of a module, but without adding any other constructs to the base language. So "sqrt" is a module, with one function in its interface, and the rest are local or private functions within that module. This was based on R5RS scheme I believe, and later schemes have since added a standard module construct I think(?). But regardless, its more demonstrating the principle of hiding implementation.
The seasoned schemer also goes through similar examples of nested local functions, but usually to both hide implementation and to close over values as well.
But even if this wasn't a pedagogical example, you can see how this is a very lightweight module and I probably would write it this way within a larger "real" module. Reuse is ok, if its planned for. Otherwise, you are just exposing functions that probably won't be a perfect fit for what you need later on and, at the same time, burdening those functions with unexpected use cases that could break them later on.
letfn is the standard way.
But since Clojure is a Lisp, you can create (almost) any semantics you want. Here's a proof of concept that defines define in terms of letfn.
(defmacro define [& form]
(letfn [(define? [exp]
(and (list? exp) (= (first exp) 'define)))
(transform-define [[_ name args & exps]]
`(~name ~args
(letfn [~#(map transform-define (filter define? exps))]
~#(filter #(not (define? %)) exps))))]
`(defn ~#(transform-define `(define ~#form)))))
(define sqrt [x]
(define average [a b] (/ (+ a b) 2))
(define fixed-point [f first-guess]
(define close-enough? [v1 v2]
(let [tolerance 0.00001]
(< (Math/abs (- v1 v2)) tolerance)))
(define tryy [guess]
(let [next (f guess)]
(if (close-enough? guess next)
next
(tryy next))))
(tryy first-guess))
(fixed-point (fn [y] (average y (/ x y)))
1.0))
(sqrt 10) ; => 3.162277660168379
For real code, you'd want to change define to behave more like R5RS: allow non-fn values, be available in defn, defmacro, let, letfn, and fn, and verify that the inner definitions are at the beginning of the enclosing body.
Note: I had to rename try to tryy. Apparently try is a special non-function, non-macro construct for which redefinition silently fails.