In the book The Seasoned Schemer - the author writes the following code:
(define intersectall
(lambda (lset)
(letcc hop
(letrec
((A (lambda (lset)
(cond
((null? (car lset)) (hop (quote ())))
((null? (cdr lset)) (car lset))
(else
(intersect (car lset)
(A (cdr lset))))))))
(cond
((null? lset) (quote ()))
(else (A lset)))))))
Here is potentially how it could look in Clojure:
(defmacro letcc
[name & body]
`(letfn [(~name [arg#]
(throw (ex-info (str '~name) {:name '~name :value arg#})))]
(try ~#body
(catch clojure.lang.ExceptionInfo e#
(if (= '~name (:name (ex-data e#)))
(:value (ex-data e#))
(throw e#))))))
(defn intersectall
[lset]
(letcc hop
(letfn [(A [lset]
(cond (empty? (first lset))
(hop ())
(empty? (rest lset))
(first lset)
:else
(intersect (first lset) (A (rest lset)))))]
(cond (empty? lset)
()
:else
(A lset)))))
My question is: How do you do letcc in Clojure?
Background
The core Clojure language does not support first-class continuations. That, and the fact that the JVM does not provide a way to capture the current continuation, means there is no way of implementing letcc that is satisfactory for all situations.
However, it is possible to implement continuations in some situations. Specifically, if you own all the code (that is, the code in which you must capture continuations) then you can employ continuation-passing-style (CPS). Basically, you add an extra parameter to each function. This parameter is a function that represents the continuation of that call. You "return" a value by calling the continuation function. Of course, this style is a pain to write by itself -- but fortunately this is a transform we can easily apply to specific code via macros.
By itself, CPS is unsuitable for platforms that do not do tail-call optimization (TCO). Because the last step of any function in CPS is to invoke another function, without TCO the stack quickly overflows except for the most trivial of computations. This problem can be solved by employing thunking and trampolining.
Solutions
As I alluded above, you can write your own CPS transform using macros. However, I would invite you to checkout my pulley.cps library, which already does this for you. There are alternatives, but as far as I'm aware pulley.cps is the only Clojure library that provides all of the following:
call-cc/let-cc
Seamless calls between "native" (non-transformed) and transformed code
Exception (try/catch/finally) support
binding forms (they're properly tail-recursive too!)
Allows you to provide a CPS version of an existing native function (this is necessary if you want to capture a continuation within that function)
Alternatives include:
delimc provides a library for delimited continuations. This doesn't appear to be very complete (e.g., binding fails because it doesn't understand the try/finally block) and hasn't been touched in 4 years.
algo.monads is a monad library for Clojure. There is a strong and interesting relationship between monads and continuations, and algo.monads provides a continuation monad. Although monadic style isn't quite as covenient, it does have the advantage of making the effect more explicit, which can aid in encapsulating the code that uses control effects from the code that doesn't. Plus, do notation (e.g., the domonad macro) greatly blurs the lines between direct and monadic style.
The continuation caught by (letcc hop ...) in your example is used as an "upwards continuation". One could have used the name return instead: (letcc return ... (return () ...). When the continuation named return is called, the entire letcc-expression evaluates to the value given to return -- which is then returned as the result of intersectall.
This means that 1. the continuation goes up (we return) and 2. the continuation is used once only. When these conditions are met, one can implement letcc in terms of try and catch as you have done.
So as I see it, by writing your letcc macro, you have answered your own question.
Now as Nathan Davis mentions there are other use cases of continuations, but Clojure does not support them directly.
Note: There is a related question here: The Seasoned Schemer, letcc and guile
Related
I find myself writing a lot of clojure in this manner:
(defn my-fun [input]
(let [result1 (some-complicated-procedure input)
result2 (some-other-procedure result1)]
(do-something-with-results result1 result2)))
This let statement seems very... imperative. Which I don't like. In principal, I could be writing the same function like this:
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
The problem with this is that it involves recomputation of some-complicated-procedure, which may be arbitrarily expensive. Also you can imagine that some-complicated-procedure is actually a series of nested function calls, and then I either have to write a whole new function, or risk that changes in the first invocation don't get applied to the second:
E.g. this works, but I have to have an extra shallow, top-level function that makes it hard to do a mental stack trace:
(defn some-complicated-procedure [input] (lots (of (nested (operations input)))))
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
E.g. this is dangerous because refactoring is hard:
(defn my-fun [input]
(do-something-with-results (lots (of (nested (operations (mistake input))))) ; oops made a change here that wasn't applied to the other nested calls
(some-other-procedure (lots (of (nested (operations input))))))))
Given these tradeoffs, I feel like I don't have any alternatives to writing long, imperative let statements, but when I do, I cant shake the feeling that I'm not writing idiomatic clojure. Is there a way I can address the computation and code cleanliness problems raised above and write idiomatic clojure? Are imperitive-ish let statements idiomatic?
The kind of let statements you describe might remind you of imperative code, but there is nothing imperative about them. Haskell has similar statements for binding names to values within bodies, too.
If your situation really needs a bigger hammer, there are some bigger hammers that you can either use or take for inspiration. The following two libraries offer some kind of binding form (akin to let) with a localized memoization of results, so as to perform only the necessary steps and reuse their results if needed again: Plumatic Plumbing, specifically the Graph part; and Zach Tellman's Manifold, whose let-flow form furthermore orchestrates asynchronous steps to wait for the necessary inputs to become available, and to run in parallel when possible. Even if you decide to maintain your present course, their docs make good reading, and the code of Manifold itself is educational.
I recently had this same question when I looked at this code I wrote
(let [user-symbols (map :symbol states)
duplicates (for [[id freq] (frequencies user-symbols) :when (> freq 1)] id)]
(do-something-with duplicates))
You'll note that map and for are lazy and will not be executed until do-something-with is executed. It's also possible that not all (or even not any) of the states will be mapped or the frequencies calculated. It depends on what do-something-with actually requests of the sequence returned by for. This is very much functional and idiomatic functional programming.
i guess the simplest approach to keep it functional would be to have a pass-through state to accumulate the intermediate results. something like this:
(defn with-state [res-key f state]
(assoc state res-key (f state)))
user> (with-state :res (comp inc :init) {:init 10})
;;=> {:init 10, :res 11}
so you can move on to something like this:
(->> {:init 100}
(with-state :inc'd (comp inc :init))
(with-state :inc-doubled (comp (partial * 2) :inc'd))
(with-state :inc-doubled-squared (comp #(* % %) :inc-doubled))
(with-state :summarized (fn [st] (apply + (vals st)))))
;;=> {:init 100,
;; :inc'd 101,
;; :inc-doubled 202,
;; :inc-doubled-squared 40804,
;; :summarized 41207}
The let form is a perfectly functional construct and can be seen as syntactic sugar for calls to anonymous functions. We can easily write a recursive macro to implement our own version of let:
(defmacro my-let [bindings body]
(if (empty? bindings)
body
`((fn [~(first bindings)]
(my-let ~(rest (rest bindings)) ~body))
~(second bindings))))
Here is an example of calling it:
(my-let [a 3
b (+ a 1)]
(* a b))
;; => 12
And here is a macroexpand-all called on the above expression, that reveal how we implement my-let using anonymous functions:
(clojure.walk/macroexpand-all '(my-let [a 3
b (+ a 1)]
(* a b)))
;; => ((fn* ([a] ((fn* ([b] (* a b))) (+ a 1)))) 3)
Note that the expansion doesn't rely on let and that the bound symbols become parameter names in the anonymous functions.
As others write, let is actually perfectly functional, but at times it can feel imperative. It's better to become fully comfortable with it.
You might, however, want to kick the tires of my little library tl;dr that lets you write code like for example
(compute
(+ a b c)
where
a (f b)
c (+ 100 b))
What's the (most) idiomatic Clojure representation of no-op? I.e.,
(def r (ref {}))
...
(let [der #r]
(match [(:a der) (:b der)]
[nil nil] (do (fill-in-a) (fill-in-b))
[_ nil] (fill-in-b)
[nil _] (fill-in-a)
[_ _] ????))
Python has pass. What should I be using in Clojure?
ETA: I ask mostly because I've run into places (cond, e.g.) where not supplying anything causes an error. I realize that "most" of the time, an equivalent of pass isn't needed, but when it is, I'd like to know what's the most Clojuric.
I see the keyword :default used in cases like this fairly commonly.
It has the nice property of being recognizable in the output and or logs. This way when you see a log line like: "process completed :default" it's obvious that nothing actually ran. This takes advantage of the fact that keywords are truthy in Clojure so the default will be counted as a success.
There are no "statements" in Clojure, but there are an infinite number of ways to "do nothing". An empty do block (do), literally indicates that one is "doing nothing" and evaluates to nil. Also, I agree with the comment that the question itself indicates that you are not using Clojure in an idiomatic way, regardless of this specific stylistic question.
The most analogous thing that I can think of in Clojure to a "statement that does nothing" from imperative programming would be a function that does nothing. There are a couple of built-ins that can help you here: identity is a single-arg function that simply returns its argument, and constantly is a higher-order function that accepts a value, and returns a function that will accept any number of arguments and return that value. Both are useful as placeholders in situations where you need to pass a function but don't want that function to actually do much of anything. A simple example:
(defn twizzle [x]
(let [f (cond (even? x) (partial * 4)
(= 0 (rem x 3)) (partial + 2)
:else identity)]
(f (inc x))))
Rewriting this function to "do nothing" in the default case, while possible, would require an awkward rewrite without the use of identity.
In Clojure, there are several option for composition of functions. There are composition functions for:
Apply: for 'unwrapping' arguments
Partial: for arguments that are not yet given
Comp: for piping consecutive results through multiple functions
Juxt: for applying one argument on multiple functions
However, AFAIK there are no such composition functions that include branching. Are there any functions that compose functions in a branching way, like a functional version of if or cond ?
Of course an if version is easy to make (though this implementation might not be the quickest):
(defn iff
([pred rtrue] (iff pred rtrue identity))
([pred rtrue rfalse]
(fn [& args]
(if (apply pred args)
(apply rtrue args)
(apply rfalse args)))))
There could be discussion about by default returning identity in the 'else' case is the right choice, or if nil should be returned in such case.
The use of such function could produce more easy to read code. Instead of #(if (string? %) (trim %) %) it would become (iff string? trim), or with a cond version:
(condf string? trim,
vector? (partial apply str),
:else identity)
Do other FP languages have such constructs ? I can imagine it might be handy in compositions with comp and juxt. Why doesn't Clojure ?
Bonus points for nice iff / condf implementations :)
I'm not sure if this is a direct match for what you're looking for (the question, to me, is somewhat vague), but you should look into Monads and Arrows.
Monads allow you to chain together functions with a specific "bind" function that defines how to chain them. It could do some sort of if/else pipelining, as in the Maybe and Either monads, or it could simulate state, as in the State monad.
Monads are built into Haskell (as monads) and F# (as "Workflows"). I have seen monad libraries for Clojure (check this out for one), and there are probably Arrow libraries too.
Well there could be many such composition pattern you can come up and ask why this isn't in the core language. The reason is obvious, it is not feasible. The core of the language provide you all the constructs to build such patterns. These sort of features are more of a contrib kind of thing rather than core of the language.
As far as implementation is concerned it would as simple as something shown below:
(defn condf [& args]
(let [chain (partition 2 args)]
(fn [& params]
(first (for [[p f] chain :when (or (= :else p) (apply p params))]
(apply f params))))))
(def my-func (condf string? clojure.string/trim
vector? (partial apply str)
:else identity))
(my-func "Ankur ") ==> "Ankur"
(my-func [1 2 3]) ==> "123"
(my-func '(1 2 3)) ==> (1 2 3)
This approaches the idea of Strategic Programming. You may find the following paper of interest
The Essence of Strategic Programming by
Ralf Lämmel and Eelco Visser and Joost Visser
http://homepages.cwi.nl/~ralf/eosp/
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1969
All examples are taken from the SICP Book: http://sicpinclojure.com/?q=sicp/1-3-3-procedures-general-methods
This was motivated from the MIT video series on LISP - http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-001-structure-and-interpretation-of-computer-programs-spring-2005/video-lectures/2a-higher-order-procedures/
In scheme, you can put 'define' inside another 'define':
(define (close-enough? v1 v2)
(define tolerance 0.00001)
(< (abs (- v1 v2)) tolerance ) )
In clojure, there is the 'let' statement with the only difference that it is nested:
(defn close-enough? [v1 v2]
(let [tolerance 0.00001]
(< (Math/abs (- v1 v2) )
tolerance) ) )
But what about rewriting in clojure something bigger like this?:
(define (sqrt x)
(define (fixed-point f first-guess)
(define (close-enough? v1 v2)
(define tolerance 0.00001)
(< (abs (- v1 v2)) tolerance))
(define (try guess)
(let ((next (f guess)))
(if (close-enough? guess next)
next
(try next))))
(try first-guess))
(fixed-point (lambda (y) (average y (/ x y)))
1.0))
This does in fact work but looks very unconventional...
(defn sqrt [n]
(let [precision 10e-6
abs #(if (< % 0) (- %) %)
close-enough? #(-> (- %1 %2) abs (< precision))
averaged-func #(/ (+ (/ n %) %) 2)
fixed-point (fn [f start]
(loop [old start
new (f start)]
(if (close-enough? old new)
new
(recur new (f new) ) ) ) )]
(fixed-point averaged-func 1) ) )
(sqrt 10)
UPDATED Mar/8/2012
Thanks for the answer!
Essentially 'letfn' is not too different from 'let' - the functions being called have to be nested in the 'letfn' definition (as opposed to Scheme where the functions are used in the next sexp after its definitions and only existing within the scope of the top-level function in which it is defined).
So another question... Why doesn't clojure give the capability of doing what scheme does? Is it some sort of language design decision? What I like about the scheme organization is:
1) The encapsulation of ideas so that I as the programmer have an idea as to what little blocks are being utilized bigger block - especially if I am only using the little blocks once within the big block (for whatever reason, even if the little blocks are useful in their own right).
2) This also stops polluting the namespace with little procedures that are not useful to the end user (I've written clojure programs, came back to them a week later and had to re-learn my code because it was in a flat structure and I felt that I was looking at the code inside out as opposed to in a top down manner).
3) A common method definition interface so I can pull out a particular sub-method, de-indent it test it, and paste the changed version back without too much fiddling around.
Why isn't this implemented in clojure?
the standard way to write nested, named, procedures in clojure is to use letfn.
as an aside, your example use of nested functions is pretty suspicious. all of the functions in the example could be top-level non-local functions since they're more or less useful on their own and don't close over anything but each other.
The people critizing his placement of this in all one function, don't understand the context of why its been done in such a manner. SICP, which is where the example is from, was trying to illustrate the concept of a module, but without adding any other constructs to the base language. So "sqrt" is a module, with one function in its interface, and the rest are local or private functions within that module. This was based on R5RS scheme I believe, and later schemes have since added a standard module construct I think(?). But regardless, its more demonstrating the principle of hiding implementation.
The seasoned schemer also goes through similar examples of nested local functions, but usually to both hide implementation and to close over values as well.
But even if this wasn't a pedagogical example, you can see how this is a very lightweight module and I probably would write it this way within a larger "real" module. Reuse is ok, if its planned for. Otherwise, you are just exposing functions that probably won't be a perfect fit for what you need later on and, at the same time, burdening those functions with unexpected use cases that could break them later on.
letfn is the standard way.
But since Clojure is a Lisp, you can create (almost) any semantics you want. Here's a proof of concept that defines define in terms of letfn.
(defmacro define [& form]
(letfn [(define? [exp]
(and (list? exp) (= (first exp) 'define)))
(transform-define [[_ name args & exps]]
`(~name ~args
(letfn [~#(map transform-define (filter define? exps))]
~#(filter #(not (define? %)) exps))))]
`(defn ~#(transform-define `(define ~#form)))))
(define sqrt [x]
(define average [a b] (/ (+ a b) 2))
(define fixed-point [f first-guess]
(define close-enough? [v1 v2]
(let [tolerance 0.00001]
(< (Math/abs (- v1 v2)) tolerance)))
(define tryy [guess]
(let [next (f guess)]
(if (close-enough? guess next)
next
(tryy next))))
(tryy first-guess))
(fixed-point (fn [y] (average y (/ x y)))
1.0))
(sqrt 10) ; => 3.162277660168379
For real code, you'd want to change define to behave more like R5RS: allow non-fn values, be available in defn, defmacro, let, letfn, and fn, and verify that the inner definitions are at the beginning of the enclosing body.
Note: I had to rename try to tryy. Apparently try is a special non-function, non-macro construct for which redefinition silently fails.
Thanks a lot for all the beautiful answers! Cannot mark just one as correct
Note: Already a wiki
I am new to functional programming and while I can read simple functions in Functional programming, for e.g. computing the factorial of a number, I am finding it hard to read big functions.
Part of the reason is I think because of my inability to figure out the smaller blocks of code within a function definition and also partly because it is becoming difficult for me to match ( ) in code.
It would be great if someone could walk me through reading some code and give me some tips on how to quickly decipher some code.
Note: I can understand this code if I stare at it for 10 minutes, but I doubt if this same code had been written in Java, it would take me 10 minutes. So, I think to feel comfortable in Lisp style code, I must do it faster
Note: I know this is a subjective question. And I am not seeking any provably correct answer here. Just comments on how you go about reading this code, would be welcome and highly helpful
(defn concat
([] (lazy-seq nil))
([x] (lazy-seq x))
([x y]
(lazy-seq
(let [s (seq x)]
(if s
(if (chunked-seq? s)
(chunk-cons (chunk-first s) (concat (chunk-rest s) y))
(cons (first s) (concat (rest s) y)))
y))))
([x y & zs]
(let [cat (fn cat [xys zs]
(lazy-seq
(let [xys (seq xys)]
(if xys
(if (chunked-seq? xys)
(chunk-cons (chunk-first xys)
(cat (chunk-rest xys) zs))
(cons (first xys) (cat (rest xys) zs)))
(when zs
(cat (first zs) (next zs)))))))]
(cat (concat x y) zs))))
I think concat is a bad example to try to understand. It's a core function and it's more low-level than code you would normally write yourself, because it strives to be efficient.
Another thing to keep in mind is that Clojure code is extremely dense compared to Java code. A little Clojure code does a lot of work. The same code in Java would not be 23 lines. It would likely be multiple classes and interfaces, a great many methods, lots of local temporary throw-away variables and awkward looping constructs and generally all kinds of boilerplate.
Some general tips though...
Try to ignore the parens most of the time. Use the indentation instead (as Nathan Sanders suggests). e.g.
(if s
(if (chunked-seq? s)
(chunk-cons (chunk-first s) (concat (chunk-rest s) y))
(cons (first s) (concat (rest s) y)))
y))))
When I look at that my brain sees:
if foo
then if bar
then baz
else quux
else blarf
If you put your cursor on a paren and your text editor doesn't syntax-highlight the matching one, I suggest you find a new editor.
Sometimes it helps to read code inside-out. Clojure code tends to be deeply nested.
(let [xs (range 10)]
(reverse (map #(/ % 17) (filter (complement even?) xs))))
Bad: "So we start with numbers from 1 to 10. Then we're reversing the order of the mapping of the filtering of the complement of the wait I forgot what I'm talking about."
Good: "OK, so we're taking some xs. (complement even?) means the opposite of even, so "odd". So we're filtering some collection so only the odd numbers are left. Then we're dividing them all by 17. Then we're reversing the order of them. And the xs in question are 1 to 10, gotcha."
Sometimes it helps to do this explicitly. Take the intermediate results, throw them in a let and give them a name so you understand. The REPL is made for playing around like this. Execute the intermediate results and see what each step gives you.
(let [xs (range 10)
odd? (complement even?)
odd-xs (filter odd? xs)
odd-xs-over-17 (map #(/ % 17) odd-xs)
reversed-xs (reverse odd-xs-over-17)]
reversed-xs)
Soon you will be able to do this sort of thing mentally without effort.
Make liberal use of (doc). The usefulness of having documentation available right at the REPL can't be overstated. If you use clojure.contrib.repl-utils and have your .clj files on the classpath, you can do (source some-function) and see all the source code for it. You can do (show some-java-class) and see a description of all the methods in it. And so on.
Being able to read something quickly only comes with experience. Lisp is no harder to read than any other language. It just so happens that most languages look like C, and most programmers spend most of their time reading that, so it seems like C syntax is easier to read. Practice practice practice.
Lisp code, in particular, is even harder to read than other functional languages because of the regular syntax. Wojciech gives a good answer for improving your semantic understanding. Here is some help on syntax.
First, when reading code, don't worry about parentheses. Worry about indentation. The general rule is that things at the same indent level are related. So:
(if (chunked-seq? s)
(chunk-cons (chunk-first s) (concat (chunk-rest s) y))
(cons (first s) (concat (rest s) y)))
Second, if you can't fit everything on one line, indent the next line a small amount. This is almost always two spaces:
(defn concat
([] (lazy-seq nil)) ; these two fit
([x] (lazy-seq x)) ; so no wrapping
([x y] ; but here
(lazy-seq ; (lazy-seq indents two spaces
(let [s (seq x)] ; as does (let [s (seq x)]
Third, if multiple arguments to a function can't fit on a single line, line up the second, third, etc arguments underneath the first's starting parenthesis. Many macros have a similar rule with variations to allow the important parts to appear first.
; fits on one line
(chunk-cons (chunk-first s) (concat (chunk-rest s) y))
; has to wrap: line up (cat ...) underneath first ( of (chunk-first xys)
(chunk-cons (chunk-first xys)
(cat (chunk-rest xys) zs))
; if you write a C-for macro, put the first three arguments on one line
; then the rest indented two spaces
(c-for (i 0) (< i 100) (add1 i)
(side-effects!)
(side-effects!)
(get-your (side-effects!) here))
These rules help you find blocks within the code: if you see
(chunk-cons (chunk-first s)
Don't count parentheses! Check the next line:
(chunk-cons (chunk-first s)
(concat (chunk-rest s) y))
You know that the first line is not a complete expression because the next line is indented beneath it.
If you see the defn concat from above, you know you have three blocks, because there are three things on the same level. But everything below the third line is indented beneath it, so the rest belongs to that third block.
Here is a style guide for Scheme. I don't know Clojure, but most of the rules should be the same since none of the other Lisps vary much.
First remember that functional program consists of expressions, not statements. For example, form (if condition expr1 expr2) takes its 1st arg as a condition to test for the boolean falue, evaluates it, and if it eval'ed to true then it evaluates and returns expr1, otherwise evaluates and returns expr2. When every form returns an expression some of usual syntax constructs like THEN or ELSE keywords may just disappear. Note that here if itself evaluates to an expression as well.
Now about the evaluation: In Clojure (and other Lisps) most forms you encounter are function calls of the form (f a1 a2 ...), where all arguments to f are evaluated before actual function call; but forms can be also macros or special forms which don't evaluate some (or all) of its arguments. If in doubt, consult the documentation (doc f) or just check in REPL:
user=> apply
#<core$apply__3243 clojure.core$apply__3243#19bb5c09> a function
user=> doseq
java.lang.Exception: Can't take value of a macro: #'clojure.core/doseq a macro.
These two rules:
we have expressions, not statements
evaluation of a subform may occur or not, depending of how outer form behaves
should ease your groking of Lisp programs, esp. if they have nice indentation like the example you gave.
Hope this helps.