Assumptions:
by proxy I mean in the sense of a wrapper like in Spring
by your function I mean something as simple as (fn [x] (+ x 1))
Consider the following from Konrad Hinsen:
(defn f [x]
(let [a x
b (inc a)]
(* a b)))
(defn f [x]
(domonad maybe-m
[a x
b (inc a)]
(* a b)))
(defn m-bind [value function]
(if (nil? value)
nil
(function value)))
Now the benefit is that if the value is nil, then m-bind returns nil and the rest of the computation is never called.
My question is: Is the essence of the Maybe Monad a Proxy for your function?
Clojurians seem to prefer syntactic transformations, such as the some-> threading macro for short-circuiting on nil, over algebraic formalisms like the maybe monad.
In either case, whether by code transformation or by monadic structure, the key is manipulating the continuation so that it is never called after a nil or Nothing is encountered.
You cannot achieve this short-circuiting behavior with just pure individual function wrappers on each individual function in the chain, precisely because they pure. You need to to control the chain of computation itself, which can be done either with code transformation via macros or installation of a continuation algebra via monads.
I'm not sure how the Spring wrappers work but the point about the maybe monad as described in that article is that you write code like this:
(defn f [x]
(domonad maybe-m
[a x
b (inc a)]
(* a b)))
And the monad mechanisms adds checks between the lines. So after each of these statements it checks if the result is nil and if so just returns nil and quits.
So you don't just get nil if you pass in nil, but it also works if (inc 8) suddenly returns nil.
Related
I find myself writing a lot of clojure in this manner:
(defn my-fun [input]
(let [result1 (some-complicated-procedure input)
result2 (some-other-procedure result1)]
(do-something-with-results result1 result2)))
This let statement seems very... imperative. Which I don't like. In principal, I could be writing the same function like this:
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
The problem with this is that it involves recomputation of some-complicated-procedure, which may be arbitrarily expensive. Also you can imagine that some-complicated-procedure is actually a series of nested function calls, and then I either have to write a whole new function, or risk that changes in the first invocation don't get applied to the second:
E.g. this works, but I have to have an extra shallow, top-level function that makes it hard to do a mental stack trace:
(defn some-complicated-procedure [input] (lots (of (nested (operations input)))))
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
E.g. this is dangerous because refactoring is hard:
(defn my-fun [input]
(do-something-with-results (lots (of (nested (operations (mistake input))))) ; oops made a change here that wasn't applied to the other nested calls
(some-other-procedure (lots (of (nested (operations input))))))))
Given these tradeoffs, I feel like I don't have any alternatives to writing long, imperative let statements, but when I do, I cant shake the feeling that I'm not writing idiomatic clojure. Is there a way I can address the computation and code cleanliness problems raised above and write idiomatic clojure? Are imperitive-ish let statements idiomatic?
The kind of let statements you describe might remind you of imperative code, but there is nothing imperative about them. Haskell has similar statements for binding names to values within bodies, too.
If your situation really needs a bigger hammer, there are some bigger hammers that you can either use or take for inspiration. The following two libraries offer some kind of binding form (akin to let) with a localized memoization of results, so as to perform only the necessary steps and reuse their results if needed again: Plumatic Plumbing, specifically the Graph part; and Zach Tellman's Manifold, whose let-flow form furthermore orchestrates asynchronous steps to wait for the necessary inputs to become available, and to run in parallel when possible. Even if you decide to maintain your present course, their docs make good reading, and the code of Manifold itself is educational.
I recently had this same question when I looked at this code I wrote
(let [user-symbols (map :symbol states)
duplicates (for [[id freq] (frequencies user-symbols) :when (> freq 1)] id)]
(do-something-with duplicates))
You'll note that map and for are lazy and will not be executed until do-something-with is executed. It's also possible that not all (or even not any) of the states will be mapped or the frequencies calculated. It depends on what do-something-with actually requests of the sequence returned by for. This is very much functional and idiomatic functional programming.
i guess the simplest approach to keep it functional would be to have a pass-through state to accumulate the intermediate results. something like this:
(defn with-state [res-key f state]
(assoc state res-key (f state)))
user> (with-state :res (comp inc :init) {:init 10})
;;=> {:init 10, :res 11}
so you can move on to something like this:
(->> {:init 100}
(with-state :inc'd (comp inc :init))
(with-state :inc-doubled (comp (partial * 2) :inc'd))
(with-state :inc-doubled-squared (comp #(* % %) :inc-doubled))
(with-state :summarized (fn [st] (apply + (vals st)))))
;;=> {:init 100,
;; :inc'd 101,
;; :inc-doubled 202,
;; :inc-doubled-squared 40804,
;; :summarized 41207}
The let form is a perfectly functional construct and can be seen as syntactic sugar for calls to anonymous functions. We can easily write a recursive macro to implement our own version of let:
(defmacro my-let [bindings body]
(if (empty? bindings)
body
`((fn [~(first bindings)]
(my-let ~(rest (rest bindings)) ~body))
~(second bindings))))
Here is an example of calling it:
(my-let [a 3
b (+ a 1)]
(* a b))
;; => 12
And here is a macroexpand-all called on the above expression, that reveal how we implement my-let using anonymous functions:
(clojure.walk/macroexpand-all '(my-let [a 3
b (+ a 1)]
(* a b)))
;; => ((fn* ([a] ((fn* ([b] (* a b))) (+ a 1)))) 3)
Note that the expansion doesn't rely on let and that the bound symbols become parameter names in the anonymous functions.
As others write, let is actually perfectly functional, but at times it can feel imperative. It's better to become fully comfortable with it.
You might, however, want to kick the tires of my little library tl;dr that lets you write code like for example
(compute
(+ a b c)
where
a (f b)
c (+ 100 b))
Reading all over (except the clojure source code) it is somewhat hard to fathom how do transducers avoid the usage of intermediate collections, which is supposed to make them more lean and performant.
A related question arises as to whether or not, they assume, that each input transformation is applied to each element of its input independently from the other elements of it, a limitation that may exist if transducers were to work by squashing the input transformations on the input collection ― element-by-element.
Do they inspect the code of their input functions to determine how to interweave them such that they yield the correct result of their composition?
Can you please detail how do transducers in clojure work under the hood, in those regards?
A related question arises as to whether or not, they assume, that each
input transformation is applied to each element of its input
independently from the other elements of it
They are named transducers because they may have (implicit) state.
A transducer is a function that takes a reducing function and returns a reducing function.
A reducing function is a function that expects two parameters: an accumulator and an item and returns an updated accumulator.
This is the reducing function which holds the mutable state (if any).
To get transducers you have to understand they work in two times: composition-time then computation-time. That's why they are functions returning functions.
Let's start with an easy reducing function: conj.
The transducer returned by (map inc) is (fn [rf] (fn [acc x] (rf acc (inc x)))). When called with conj it returns a function equivalent to (fn [acc x] (conj acc (inc x))).
The transducer returned by (filter odd?) is (fn [rf] (fn [acc x] (if (odd? x) (rf acc x) acc))). When called with conj it returns a function equivalent to (fn [acc x] (if (odd? x) (conj acc x) acc))). This one is interesting because rf (the downstream reducing function is sometimes short-circuited).
If you want to chain these two transducers you just do (comp (map inc) (filter odd?)) if you pass conj to this composite transducer, (filter odd?) is going to be the first to wrap conj (because comp applies functions from right to left). Then the resulting filtered-rffunction is passed to (map inc) which yields a function equiavlent to:
(fn [acc x] (filtered-rf acc (inc x))) where filtered-rf is (fn [acc x] (if (odd? x) (conj acc x) acc))). If you inline filtered-rf you get: (fn [acc x] (let [x+1 (inc x)] (if (odd? x+1) (conj acc x+1) acc))).
As you may see no intermediate collection or sequence is allocated.
For stateful transducers that's the same story except that reducing functions have mutable state (as little as possible and avoid keeping all previous items in it): usually a volatile box (see volatile!) or a mutable Java object.
You may also have remarked that in the example items are first mapped then filtered: computations are applied from left to right which seems in contradiction to comp. This is not: remember comp here composes transducers, fns that wraps reducing fns. So at compoition time the wrapping occurs right to left (conj wrapped by the "filtering rf" then by the "mapping rf") but at computation time the wrapping layers are traversed inwards: map, filter and then conj.
There are finicky implementation details to know to implement your own transducers (reduced, init and completion arities) but the general idea is the one exposed above.
We can see that we can use reduce/foldl1 as the function by which we can define other higher order functions such as map, filter and reverse.
(defn mapl [f coll]
(reduce (fn [r x] (conj r (f x)))
[] coll))
(defn filterl [pred coll]
(reduce (fn [r x] (if (pred x) (conj r x) r))
[] coll))
(defn mapcatl [f coll]
(reduce (fn [r x] (reduce conj r (f x)))
[] coll))
We also appear to be able to do this in terms of foldr. Here is map and filter in terms of foldr from Rich Hickey's Transducers talk at 17:25.
(defn mapr [f coll]
(foldr (fn [x r] (cons (f x) r))
() coll))
(defn filterr [pred coll]
(foldr (fn [x r] (if (pred x) (cons x r) r))
() coll))
Now we can define map, foldl (reduce) and foldr in terms of first, rest and cons (car, cdr and cons):
(defn foldr [f z xs]
(if (null? xs)
z
(f (first xs) (foldr f z (rest xs)))))
(defn foldl [f z xs]
(if (null? xs)
z
(foldl f (f z (first xs)) (rest xs))))
(defn map [f lst]
(if (null? lst)
'()
(cons (f (first lst)) (map f (rest lst)))))
My question is Why is fold and reduce considered fundamental - surely everything is defined in terms of cons, cdr and car? Isn't this looking at things at the wrong level?
I thought Rich Hickey explained exactly that in his talk about transducers.
He sees folds as a fundamental concept of transformation. It doesn't need to know what structure it is working on and how to operate on that structure.
You just defined fold in terms of itself, cdr, car and rest. What Rich is arguing for is that fold in itself is an abstract concept separate from the data structure that it operates on and, as long as we provide it certain functions that actually operate on the data structure, it will work as expected.
So in the end it's all about separation of concerns and reusability.
Folds are a unifying framework for data processing in principled way, because they correspond to inductive data definitions. They are not ad-hoc. cons/car/cdr are building blocks for creating data (and code) but in unprincipled way (we can do anything, and process it ad-hoc). Folds are higher level, more disciplined, more predictable, easier to reason about.
Given the ad-hoc implementations for map, filter, mapcat for lists, we can see there's something similar in them - the structure of code that follows the structure of data (list, built of cons nodes, to which correspond the two arguments of the combining function). And that's fold.
In that talk, Rich Hickey abstracts away the step function. But he doesn't abstract over the data. Any combining function used to fold over labeled trees has to take three parameters, not two. So what his -ing functions do is, still, folding on lists, conceptually; it's just that they abstract over the concrete implementations of lists - either as cons cells, hashed array trees, whatever.
Most operators in a lispy language will have a partner that you can use to define it or vice versa. This attribute is called "metacircularity". There is no one specific set of basic operators that is the fundamental minimum to allow the full set of programmability given by the language.
You can read more at this paper:
http://home.pipeline.com/~hbaker1/MetaCircular.html
I find I very rarely use let in Clojure. For some reason I took a dislike to it when I started learning and have avoided using it ever since. It feels like the flow has stopped when let comes along. I was wondering, do you think we could do without it altogether ?
You can replace any occurrence of (let [a1 b1 a2 b2...] ...) by ((fn [a1 a2 ...] ...) b1 b2 ...) so yes, we could. I am using let a lot though, and I'd rather not do without it.
Let offers a few benefits. First, it allows value binding in a functional context. Second, it confers readability benefits. So while technically, one could do away with it (in the sense that you could still program without it), the language would be impoverished without a valuable tool.
One of the nice things about let is that it helps formalize a common (mathematical) way of specifying a computation, in which you introduce convenient bindings and then a simplified formula as a result. It's clear the bindings only apply to that "scope" and it's tie in with a more mathematical formulation is useful, especially for more functional programmers.
It's not a coincidence that let blocks occur in other languages like Haskell.
Let is indispensable to me in preventing multiple execution in macros:
(defmacro print-and-run [s-exp]
`(do (println "running " (quote ~s-exp) "produced " ~s-exp)
s-exp))
would run s-exp twice, which is not what we want:
(defmacro print-and-run [s-exp]
`(let [result# s-exp]
(do (println "running " (quote ~s-exp) "produced " result#)
result#))
fixes this by binding the result of the expression to a name and referring to that result twice.
because the macro is returning an expression that will become part of another expression (macros are function that produce s-expressions) they need to produce local bindings to prevent multiple execution and avoid symbol capture.
I think I understand your question. Correct me if it's wrong. Some times "let" is used for imperative programming style. For example,
... (let [x (...)
y (...x...)
z (...x...y...)
....x...y...z...] ...
This pattern comes from imperative languages:
... { x = ...;
y = ...x...;
...x...y...;} ...
You avoid this style and that's why you also avoid "let", don't you?
In some problems imperative style reduces amount of code. Furthermore, some times It's more efficient to write in java or c.
Also in some cases "let" just holds values of subexpressions regardless of evaluation order. For example,
(... (let [a (...)
b (...)...]
(...a...b...a...b...) ;; still fp style
There are at least two important use cases for let-bindings:
First, using let properly can make your code clearer and shorter. If you have an expression that you use more than once, binding it in a let is very nice. Here's a portion of the standard function map that uses let:
...
(let [s1 (seq c1) s2 (seq c2)]
(when (and s1 s2)
(cons (f (first s1) (first s2))
(map f (rest s1) (rest s2)))))))
...
Even if you use an expression only once, it can still be helpful (to future readers of the code) to give it a semantically meaningful name.
Second, as Arthur mentioned, if you want to use the value of an expression more than once, but only want it evaluated once, you can't simply type out the entire expression twice: you need some kind of binding. This would be merely wasteful if you have a pure expression:
user=> (* (+ 3 2) (+ 3 2))
25
but actually changes the meaning of the program if the expression has side-effects:
user=> (* (+ 3 (do (println "hi") 2))
(+ 3 (do (println "hi") 2)))
hi
hi
25
user=> (let [x (+ 3 (do (println "hi") 2))]
(* x x))
hi
25
Stumbled upon this recently so ran some timings:
(testing "Repeat vs Let vs Fn"
(let [start (System/currentTimeMillis)]
(dotimes [x 1000000]
(* (+ 3 2) (+ 3 2)))
(prn (- (System/currentTimeMillis) start)))
(let [start (System/currentTimeMillis)
n (+ 3 2)]
(dotimes [x 1000000]
(* n n))
(prn (- (System/currentTimeMillis) start)))
(let [start (System/currentTimeMillis)]
(dotimes [x 1000000]
((fn [x] (* x x)) (+ 3 2)))
(prn (- (System/currentTimeMillis) start)))))
Output
Testing Repeat vs Let vs Fn
116
18
60
'let' wins over 'pure' functional.
I'm really curious to see why vector's implementation is so verbose? What's the reason it can't just do [], [a] and [a & args]?
Here is what I get from clj-1.4.0.
=> (source vector)
(defn vector
"Creates a new vector containing the args."
{:added "1.0"
:static true}
([] [])
([a] [a])
([a b] [a b])
([a b c] [a b c])
([a b c d] [a b c d])
([a b c d & args]
(. clojure.lang.LazilyPersistentVector (create (cons a (cons b (cons c (cons d args))))))))
nil
The first few cases have direct calls to make them faster because they are the most commonly called. The rare cases where it is called with many arguments may require more calls and hence more time, but this keeps the common cases concise. It was a deliberate speed, verbosity tradeoff. It also makes the use of the function clear from looking at the argument list without cluttering people's IDEs with a huge list of arities.
Most of Clojure's core functions have similar signatures.
Further to #ArthurUlfeldt's solution.
There is a case for having reference implementations for the core functions, say in namespace clojure.core.reference. These would be shorter, hence clearer, though slower than their standard counterparts. The testing regime would make sure that they produced the same results.
For example, the reference implementation for vector could be
(ns clojure.core.reference)
(defn vector
"Creates a new vector containing the args."
{:added "1.0"
:static true}
[& args]
(. clojure.lang.LazilyPersistentVector (create args)))
In addition to testing the various optimisations and speed-ups employed by the core library, the reference implementations would be first port of call for those trying to understand what the code does.