What is the idiomatic way of printing values inside a let binding ?
When I started developing in Clojure, I wrote code in the REPL, that I then turned into simple let expressions. Being a beginner, I often made mistakes during this (simple) transformation phase.
(let [a (aFn ...)
b (bFn ... a)]
;; error above
)
So I would transform it back to something like that, basically inlining things :
(println "a is" (aFn ...))
(println "b is" (bFn ... (aFn ...)))
(let [a (aFn ...)
b (bFn ... a)]
;; ...
)
It works most of the time thanks to Clojure being nice (immutability, referential transparency..).
Now I do something along the lines of :
(let [a (aFn ...)
_ (println "a is" a)
b (bFn ... a)
_ (println "b is" b)]
;; ...
)
It is an improvement, but it still feels clumsy. What is the proper way to do this ?
You could define a print function that returns its argument:
(defn cl-print [x] (doto x (print)))
Then, it is only a matter of wrapping your expressions:
(let [a (cl-print (aFn ...))
b (cl-print (bFn ... a))]
...)
I tend to take a totally different approach. I never put print statements in my let bindings. I also think you need to be careful about calling a function just to get the value for debugging purposes. While we would like all our functions to be side-effect free, this is not always the case, so calling the funciton just to get a value to print may have unexpected results. There is also the issue of how printing values can impact on laziness and realising of lazy sequences etc.
My approach is to define some debugging functions, which I stick in a 'debug' namespace. I then call these debug functions when needed from inside the body of the function - not in the let binding section. Often, I also define a debug-level var so that I can have some control over debugging verbosity. this allows me to change one var and increase or decrease the amount of information logged/printed.
I've experimented with 'clever' macros to make debugging easier - but to be honest, these usually take more effort to get right than the benefit they provide.
I like having my debug functions in a separate namespace as this helps me ensure I've not left any debugging code in my production version - or it allows me to leave debug statements in there, but have them 'do nothing' by setting an appropriate debug level.
As mentioned by another post, using a debugger can eliminate/reduce the need to have these print statements or debug functions. However, I think debuggers can be a double edged sword as well. Too often, people get into bad debugging hapits where they rely on trace and inspect rather than thinking about and analysing exactly what is going on. This can tend to development driven by too much trial and error and not enough analysis and understanding.
You could start with something as simple as
(def debug-level 20)
(defn debug [lvl prefix val]
(if (>= lvl debug-level)
(println (str prefix ": " val)))
(defn debug1 [prefix v]
(debug 10 prefix v))
(defn debug2 [prefix v]
(debug 20 prefix v))
etc
and then just call
(debug2 :a a)
in the body of your function to have the value of a printed when debug-level is 20 or higher.
Related
I find myself writing a lot of clojure in this manner:
(defn my-fun [input]
(let [result1 (some-complicated-procedure input)
result2 (some-other-procedure result1)]
(do-something-with-results result1 result2)))
This let statement seems very... imperative. Which I don't like. In principal, I could be writing the same function like this:
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
The problem with this is that it involves recomputation of some-complicated-procedure, which may be arbitrarily expensive. Also you can imagine that some-complicated-procedure is actually a series of nested function calls, and then I either have to write a whole new function, or risk that changes in the first invocation don't get applied to the second:
E.g. this works, but I have to have an extra shallow, top-level function that makes it hard to do a mental stack trace:
(defn some-complicated-procedure [input] (lots (of (nested (operations input)))))
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
E.g. this is dangerous because refactoring is hard:
(defn my-fun [input]
(do-something-with-results (lots (of (nested (operations (mistake input))))) ; oops made a change here that wasn't applied to the other nested calls
(some-other-procedure (lots (of (nested (operations input))))))))
Given these tradeoffs, I feel like I don't have any alternatives to writing long, imperative let statements, but when I do, I cant shake the feeling that I'm not writing idiomatic clojure. Is there a way I can address the computation and code cleanliness problems raised above and write idiomatic clojure? Are imperitive-ish let statements idiomatic?
The kind of let statements you describe might remind you of imperative code, but there is nothing imperative about them. Haskell has similar statements for binding names to values within bodies, too.
If your situation really needs a bigger hammer, there are some bigger hammers that you can either use or take for inspiration. The following two libraries offer some kind of binding form (akin to let) with a localized memoization of results, so as to perform only the necessary steps and reuse their results if needed again: Plumatic Plumbing, specifically the Graph part; and Zach Tellman's Manifold, whose let-flow form furthermore orchestrates asynchronous steps to wait for the necessary inputs to become available, and to run in parallel when possible. Even if you decide to maintain your present course, their docs make good reading, and the code of Manifold itself is educational.
I recently had this same question when I looked at this code I wrote
(let [user-symbols (map :symbol states)
duplicates (for [[id freq] (frequencies user-symbols) :when (> freq 1)] id)]
(do-something-with duplicates))
You'll note that map and for are lazy and will not be executed until do-something-with is executed. It's also possible that not all (or even not any) of the states will be mapped or the frequencies calculated. It depends on what do-something-with actually requests of the sequence returned by for. This is very much functional and idiomatic functional programming.
i guess the simplest approach to keep it functional would be to have a pass-through state to accumulate the intermediate results. something like this:
(defn with-state [res-key f state]
(assoc state res-key (f state)))
user> (with-state :res (comp inc :init) {:init 10})
;;=> {:init 10, :res 11}
so you can move on to something like this:
(->> {:init 100}
(with-state :inc'd (comp inc :init))
(with-state :inc-doubled (comp (partial * 2) :inc'd))
(with-state :inc-doubled-squared (comp #(* % %) :inc-doubled))
(with-state :summarized (fn [st] (apply + (vals st)))))
;;=> {:init 100,
;; :inc'd 101,
;; :inc-doubled 202,
;; :inc-doubled-squared 40804,
;; :summarized 41207}
The let form is a perfectly functional construct and can be seen as syntactic sugar for calls to anonymous functions. We can easily write a recursive macro to implement our own version of let:
(defmacro my-let [bindings body]
(if (empty? bindings)
body
`((fn [~(first bindings)]
(my-let ~(rest (rest bindings)) ~body))
~(second bindings))))
Here is an example of calling it:
(my-let [a 3
b (+ a 1)]
(* a b))
;; => 12
And here is a macroexpand-all called on the above expression, that reveal how we implement my-let using anonymous functions:
(clojure.walk/macroexpand-all '(my-let [a 3
b (+ a 1)]
(* a b)))
;; => ((fn* ([a] ((fn* ([b] (* a b))) (+ a 1)))) 3)
Note that the expansion doesn't rely on let and that the bound symbols become parameter names in the anonymous functions.
As others write, let is actually perfectly functional, but at times it can feel imperative. It's better to become fully comfortable with it.
You might, however, want to kick the tires of my little library tl;dr that lets you write code like for example
(compute
(+ a b c)
where
a (f b)
c (+ 100 b))
Is there a reasonable way to have multiple def statements happen with destructing the same way that let does it? For Example:
(let [[rtgs pcts] (->> (sort-by second row)
(apply map vector))]
.....)
What I want is something like:
(defs [rtgs pcts] (->> (sort-by second row)
(apply map vector)))
This comes up a lot in the REPL, notebooks and when debugging. Seriously feels like a missing feature so I'd like guidance on one of:
This exists already and I'm missing it
This is a bad idea because... (variable capture?, un-idiomatic?, Rich said so?)
It's just un-needed and I must be suffering from withdrawals from an evil language. (same as: don't mess up our language with your macros)
A super short experiment give me something like:
(defmacro def2 [[name1 name2] form]
`(let [[ret1# ret2#] ~form]
(do (def ~name1 ret1#)
(def ~name2 ret2#))))
And this works as in:
(def2 [three five] ((juxt dec inc) 4))
three ;; => 3
five ;; => 5
Of course and "industrial strength" version of that macro might be:
checking that number of names matches the number of inputs. (return from form)
recursive call to handle more names (can I do that in a macro like this?)
While I agree with Josh that you probably shouldn't have this running in production, I don't see any harm in having it as a convenience at the repl (in fact I think I'll copy this into my debug-repl kitchen-sink library).
I enjoy writing macros (although they're usually not needed) so I whipped up an implementation. It accepts any binding form, like in let.
(I wrote this specs-first, but if you're on clojure < 1.9.0-alpha17, you can just remove the spec stuff and it'll work the same.)
(ns macro-fun
(:require
[clojure.spec.alpha :as s]
[clojure.core.specs.alpha :as core-specs]))
(s/fdef syms-in-binding
:args (s/cat :b ::core-specs/binding-form)
:ret (s/coll-of simple-symbol? :kind vector?))
(defn syms-in-binding
"Returns a vector of all symbols in a binding form."
[b]
(letfn [(step [acc coll]
(reduce (fn [acc x]
(cond (coll? x) (step acc x)
(symbol? x) (conj acc x)
:else acc))
acc, coll))]
(if (symbol? b) [b] (step [] b))))
(s/fdef defs
:args (s/cat :binding ::core-specs/binding-form, :body any?))
(defmacro defs
"Like def, but can take a binding form instead of a symbol to
destructure the results of the body.
Doesn't support docstrings or other metadata."
[binding body]
`(let [~binding ~body]
~#(for [sym (syms-in-binding binding)]
`(def ~sym ~sym))))
;; Usage
(defs {:keys [foo bar]} {:foo 42 :bar 36})
foo ;=> 42
bar ;=> 36
(defs [a b [c d]] [1 2 [3 4]])
[a b c d] ;=> [1 2 3 4]
(defs baz 42)
baz ;=> 42
About your REPL-driven development comment:
I don't have any experience with Ipython, but I'll give a brief explanation of my REPL workflow and you can maybe comment about any comparisons/contrasts with Ipython.
I never use my repl like a terminal, inputting a command and waiting for a reply. My editor supports (emacs, but any clojure editor should do) putting the cursor at the end of any s-expression and sending that to the repl, "printing" the result after the cursor.
I usually have a comment block in the file where I start working, just typing whatever and evaluating it. Then, when I'm reasonably happy with a result, I pull it out of the "repl-area" and into the "real-code".
(ns stuff.core)
;; Real code is here.
;; I make sure that this part always basically works,
;; ie. doesn't blow up when I evaluate the whole file
(defn foo-fn [x]
,,,)
(comment
;; Random experiments.
;; I usually delete this when I'm done with a coding session,
;; but I copy some forms into tests.
;; Sometimes I leave it for posterity though,
;; if I think it explains something well.
(def some-data [,,,])
;; Trying out foo-fn, maybe copy this into a test when I'm done.
(foo-fn some-data)
;; Half-finished other stuff.
(defn bar-fn [x] ,,,)
(keys 42) ; I wonder what happens if...
)
You can see an example of this in the clojure core source code.
The number of defs that any piece of clojure will have will vary per project, but I'd say that in general, defs are not often the result of some computation, let alone the result of a computation that needs to be destructured. More often defs are the starting point for some later computation that will depend on this value.
Usually functions are better for computing a value; and if the computation is expensive, then you can memoize the function. If you feel you really need this functionality, then by all means, use your macro -- that's one of the sellings points of clojure, namely, extensibility! But in general, if you feel you need this construct, consider the possibility that you're relying too much on global state.
Just to give some real examples, I just referenced my main project at work, which is probably 2K-3K lines of clojure, in about 20 namespaces. We have about 20 defs, most of which are marked private and among them, none are actually computing anything. We have things like:
(def path-prefix "/some-path")
(def zk-conn (atom nil))
(def success? #{200})
(def compile* (clojure.core.memoize/ttl compiler {} ...)))
(def ^:private nashorn-factory (NashornScriptEngineFactory.))
(def ^:private read-json (comp json/read-str ... ))
Defining functions (using comp and memoize), enumerations, state via atom -- but no real computation.
So I'd say, based on your bullet points above, this falls somewhere between 2 and 3: it's definitely not a common use case that's needed (you're the first person I've ever heard who wants this, so it's uncommon to me anyway); and the reason it's uncommon is because of what I said above, i.e., it may be a code smell that indicates reliance on too much global state, and hence, would not be very idiomatic.
One litmus test I have for much of my code is: if I pull this function out of this namespace and paste it into another, does it still work? Removing dependencies on external vars allows for easier testing and more modular code. Sometimes we need it though, so see what your requirements are and proceed accordingly. Best of luck!
I want to write functions without think of the order of function declarations, I don't want to use declare function because I need to declare all function names which I don't want to do it.
I want some macro or some function that does the magic for me. Long story short, I need to write functions like Java(method declaration order does not matter )
One of the best things I like most about functional programming is, that its flow of writing is the same as the flow of thinking. It's like peeling an onion, and at every moment, I only need to concentrate on working on this single layer, and take the inner part for granted. And don't worry about function names, foo and bar would be fine at first. In this style of writing, functions are defined and implemented from the end of the source file back to the top. In cases when one function calls multiple other functions, this linear structure becomes a tree-like structure, but there is always a single point in file to insert new functions. No choices and worries.
Yes, there are times when we need to work on some code snippets with no other dependencies. They can be put at the top of the source file of course.
To answer the question upfront, macros are not magic. If there exists such a macro, this macro will need to take the whole source file as input, analyze the dependency between each code blocks, and re-flow them in the correct order. The analysis of dependency is non-trivial because of lexical scoping. It's almost like writing a compiler. I don't believe such macro exists (has been written by anyone), and the goods it can do is, well to me, not so big.
It is kind of unneeded, but easily possible as an exercise (up to some point). You can write macro, that would forward declare all the top level functions wrapped in it:
(defmacro with-forward-declaration [& body]
(let [names (keep #(when (#{'defn 'defn-} (first %))
(if (map? (second %)) (nth % 2) (second %)))
body)]
`(do (declare ~#names)
~#body)))
that is how it is expanded:
(with-forward-declaration
(defn ^long ff1 [x] (* 2 (ff2)))
(println "just to be sure it doesn't eat other forms")
(defn ^{:name 'xx} ff2 [] (+ 10 (ff3)))
(defn ff3 [] 101)
(defn- ff4 [] :x))
would do the following:
(do
(declare ff1 ff2 ff3 ff4)
(defn ff1 [x] (* 2 (ff2)))
(println "just to be sure it doesn't eat other forms")
(defn ff2 [] (+ 10 (ff3)))
(defn ff3 [] 101)
(defn- ff4 [] :x))
so if you wrap all your namespace's code into this macro, it would predeclare all the functions for you. If you want to go deeper, like predeclaring all some possible defns that are not in the top level, you could update this toy macro by using clojure.walk to find these inner forms.. But i guess this one is just enough, to play with clojure quickly, without thinking about functions' order. I wouldn't do that in production though (at least not without the heavy testing).
Currently when I'm refactoring in Clojure I tend to use the pattern:
(defn my-func [arg1 arg2]
(assert (= demo.core.Record1 (class arg1) "Incorrect class for arg1: " (class arg1))
(assert (= demo.core.Record1 (class arg2) "Incorrect class for arg2: " (class arg2))
...
That is, I find myself manually checking the return types in case a downstream part of the system modifies them to something I don't expect. (As in, if I refactor and get a stack-trace I don't expect, then I express my assumptions as invariants, and step forward from there).
In one sense, this is exactly the kind of invariant checking that Bertrand Meyer anticipated. (Author of Object Oriented Software Construction, and proponent of the idea of Design by Contract).
The challenge is that I don't find these out until run-time. I would be nice to find these out at compile-time - by simply stating what the function expects.
Now I know Clojure is essentially a dynamic language. (Whilst Clojure has a 'compiler' of sorts, we should expect the application of values to a function to only come to realisation at run-time.)
I just want a good pattern to make refactoring easier. (ie see all the flow-on effects of changing an argument to a function, without seeing it break on the first call, then moving onto the next, then moving onto the next breakage.)
My question is: Is there a good way to check return types when refactoring?
If I understand you right, prismatic/schema should be your choice.
https://github.com/plumatic/schema
(s/defn ^:always-validate my-func :- SomeResultClass
[arg1 :- demo.core.Record1
arg2 :- demo.core.Record1]
...)
you should just turn off all the validation before release, so it won't affect performance.
core.typed is nice, but as far as i remember, it enforces you to annotate all your code, while schema lets you only annotate critical parts.
You have a couple of options, only one of which is "compile" time:
Tests
As Clojure is a dynamic language, tests are absolutely essential. They are your safety net when refactoring. Even in statically typed languages tests are still of use.
Pre and Post Conditions
They allow you to verify your invariants by adding metadata to your functions such as in this example from Michael Fogus' blog:
(defn constrained-fn [f x]
{:pre [(pos? x)]
:post [(= % (* 2 x))]}
(f x))
(constrained-fn #(* 2 %) 2)
;=> 4
(constrained-fn #(float (* 2 %)) 2)
;=> 4.0
(constrained-fn #(* 3 %) 2)
;=> java.lang.Exception: Assert failed: (= % (* 2 x)
core.typed
core.typed is the only option in this list that will give you compile time checking. Your example would then be expressed like so:
(ann my-func (Fn [Record1 Record1 -> ResultType]))
(defn my-func [arg1 arg2]
...)
This comes at the expense of running core.typed as a seperate action, possibly as part of your test suite.
And still on the realm of runtime validation/checking, there are even more options such as bouncer and schema.
Are the -> and ->> macros only to make the code more readable or they have also other specific functions?
The thread-first (->) and thread-last (->>) are there to make code more readable. But that's already pretty important !
It allows to un-nest function calls (example taken from clojuredocs) :
;; Arguably a bit cumbersome to read:
user=> (first (.split (.replace (.toUpperCase "a b c d") "A" "X") " "))
"X"
;; Perhaps easier to read:
user=> (-> "a b c d"
.toUpperCase
(.replace "A" "X")
(.split " ")
first)
It also allows you to put the emphasis on something. For instance, consider :
(-> {... }
...
...
(count))
(count (...(... {...})))
In the first example, it is very clear that you start with a map, and then do something to it. In the second, it is clear that you count something.
The preference depends on what you want to put forward.
Also, the choice of -> vs ->> is a clear visual indicator : are you operating or not on collections ? This is made immediately obvious (this is a convention in Clojure to have the collection as a last parameter).
So yes, it is "only" readability, but that is a very important part of building programs, and Clojure gives you tools to make your code read naturally. When choosing one solution, I try to consider the way Clojure will be read.