I'm having some trouble understanding how the delay macro works in Clojure. It doesn't seem to do what expect it to do (that is: delaying evaluation). As you can see in this code sample:
; returns the current time
(defn get-timestamp [] (System/currentTimeMillis))
; var should contain the current timestamp after calling "force"
(def current-time (delay (get-timestamp)))
However, calling current-time in the REPL appears to immediately evaluate the expression, even without having used the force macro:
user=> current-time
#<Delay#19b5217: 1276376485859>
user=> (force current-time)
1276376485859
Why was the evaluation of get-timestamp not delayed until the first force call?
The printed representation of various objects which appears at the REPL is the product of a multimethod called print-method. It resides in the file core_print.clj in Clojure's sources, which constitutes part of what goes in the clojure.core namespace.
The problem here is that for objects implementing clojure.lang.IDeref -- the Java interface for things deref / # can operate on -- print-method includes the value behind the object in the printed representation. To this end, it needs to deref the object, and although special provisions are made for printing failed Agents and pending Futures, Delays are always forced.
Actually I'm inclined to consider this a bug, or at best a situation in need of an improvement. As a workaround for now, take extra care not to print unforced delays.
Related
I'm playing around with clojure macros and I'm finding that a lot of macro behavior I can just replicate with function composition.
A good example of this is the threading macro:
(defn add1 [n] (+ n 1))
(defn mult10 [n] (* n 10))
(defn threadline [arg]
(-> arg
add1
mult10))
I can replicate this easily with a higher order function like pipe:
(defn pipe [& fns]
(reduce (fn [f g] (fn [arg] (g(f arg)))) fns))
(def pipeline
(pipe
#(+ % 1)
#(* % 10)))
There must be instances where a macro can not be replaced by a function. I was wondering if someone had some good examples of these sorts of situations, and the reoccurring themes involved.
One important advantage of macros is their ability to transform code at compile-time without evaluating any of it. Macros receive code as data during compilation, but functions receive values at run-time. Macros allow you to extend the compiler in a sense.
For example, Clojure's and and or are implemented as recursive macros that expand into nested if forms. This allows lazy evaluation of the and/or's inner forms i.e. if the first or form is truthy, its value will be returned and none of the others will be evaluated. If you wrote and/or as a function, all its arguments would be evaluated before they could be examined.
Short-circuiting control flow isn't an issue in your pipe function example, but pipe adds considerable run-time complexity compared to -> which simply unrolls to nested forms. A more interesting macro to try to implement as a function might be some->.
I'm finding that a lot of macro behavior I can just replicate with function composition
If your functions are amenable to it, you can certainly replace a simple threading macro with function composition with comp, similar to "point free" style in other functional languages: #(-> % inc str) is functionally equivalent to (comp str inc) and #(str (inc %)).
It's generally advised to prefer functions when possible, and even when writing a macro you can usually farm out most of the "work" to function(s).
The first macro I ever learned is a good example of a macro that can't be written as a plain function:
(defmacro infix [[arg1 f arg2]]
(list f arg1 arg2))
(infix (1 + 2))
=> 3
Of course this exact macro would never be used in the wild, but it sets the stage for more useful macros that act as readability helpers. It should also be noted that while you can replicate a lot of basic macro's behavior with plain functions, should you? It would be hard to argue that your pipe example leads to easier to read/write code than, say, as->.
The "reoccurring themes" you're looking for are cases where you're manipulating data at compile-time ("data" being the code itself), not run-time. Any case that requires the function to take its argument unevaluated must be a macro. You can partially "cheat" in some cases and just wrap the code in a function to delay evaluation, but that doesn't work for all cases (like the infix example).
Macros are not interchangeable with functions, but you examples are:
(macroexpand '#(+ % 1))
; ==> (fn* [p1__1#] (+ p1__1# 1))
The reason why it works is because the argument expect a function and you use a macro that becomes a function. However I know that cond is a macro. It cannot be replaced with a function implementation since the arguments of a function gets evaluated and the whole point of cond is to only evaluate some parts of the arguments in a specific order based on evaluation of their predicates. eg. making a recursive function with that would never terminate since the default case will also always be called before the body of the function cond is evaluated.
The whole point of macros is to expand the lamguage and since the evaluation is controlled by the result you can make all sorts of new features that would be impossible with function except if one passed all arguments as functions to delay evaluation.
In any language, macros -- compile-time functions from code to code -- let you do three things:
Define new binding forms (e.g. Clojure's destructuring let).
Change the order of evaluation (e.g. or, and).
Present a domain-specific language (e.g. Instaparse).
You can debate 3 -- whether implementing DSLs truly requires macros. Sure you can do code generators that are functions from text files to text files. Or, sure you can do Ruby style runtime DSLs. But if you want a DSL that's integrated into the compiler at compile-time, then macros are effectively your "compiler API".
Having said that, it makes sense to use macros only for these special purposes. Use functions and/or data-driven code as much as possible. Including to do work behind the "facade" provide by a macro.
The two big things macros do is control evaluation of their arguments and transform code at compile time. You can do both with functions by requiring calling code to quote their arguments.
For instance, you could write a version of defn that is called this way:
(defn 'name '[arg1 arg2]
'(expression1)
'(expression2)
'etc)
Then you could eval arguments at will, evaluating them or not, changing the order of execution, or altering forms before you evaluate them, exactly the things macros are good for.
What macros can do that functions can't is gain this ability without any cooperation from the calling code. Users can call macros as if they were plain functions and don't have to treat their arguments any differently.
That's how macros allow you to extend the language: you don't have to treat macro code any differently than regular code, unlike, say, JavaScript, Ruby, or Python, where the language can only be extended with new control flow constructs by doing what you've done in your example, wrapping code in a block, lambda, or function.
I have recently watched Rich Hickeys talk at Cojure Conj 2016 and although it was very interesting, I didn't really understand the point in clojure.spec or when you'd use it. It seemed like most of the ideas, such as conform, valid etc, had similar functions in Clojure already.
I have only been learning clojure for around 3 months now so maybe this is due to lack of programming/Clojure experience.
Do clojure.spec and cljs.spec work in similar ways to Clojure and Cljs in that, although they are not 100% the same, they are based on the same underlying principles.
Are you tired of documenting your programs?
Does the prospect of making up yet more tests cause procrastination?
When the boss says "test coverage", do you cower with fear?
Do you forget what your data names mean?
For smooth expression of hard specifications, you need Clojure.Spec!
Clojure.spec gives you a uniform method of documenting, specifying, and automatically testing your programs, and of validating your live data.
It steals virtually every one of its ideas. And it does nothing you can't do for yourself.
But in my - barely informed - opinion, it changes the economy of specification, making it worth while doing properly. A game-changer? - quite possibly.
At the clojure/conj conference last week, probably half of the presentations featured spec in some way, and it's not even out of alpha yet. spec is a major feature of clojure; it is here to stay, and it is powerful.
As an example of its power, take static type checking, hailed as a kind of safety net by so many, and a defining characteristic of so many programming languages. It is incredibly limited in that it's only good at compile time, and it only checks types. spec, on the other hand, validates and conforms any predicate (not just type) for the args, the return, and can also validate relationships between the two. All of this is external to the function's code, separating the logic of the function from being commingled with validation and documentation about the code.
Regarding WORKFLOW:
One archetypal example of the benefits of relationship-checking, versus only type-checking, is a function which computes the substring of a string. Type checking ensures that in (subs s start end) the s is a string and start and end are integers. However, additional checking must be done within the function to ensure that start and end are positive integers, that end is greater than start, and that the resulting substring is no larger than the original string. All of these things can be spec'd out, for example (forgive me if some of this is a bit redundant or maybe even inaccurate):
(s/fdef clojure.core/subs
:args (s/and (s/cat :s string? :start nat-int? :end (s/? nat-int?))
(fn [{:keys [s start end]}]
(if end
(<= 0 start end (count s))
(<= 0 start (count s)))))
:ret string?
:fn (fn [{{:keys [s start end]} :args, substring :ret}]
(and (if end
(= (- end start) (count substring))
(= (- (count s) start) (count substring)))
(<= (count substring) (count s)))))
Call the function with sample data meeting the above args spec:
(s/exercise-fn `subs)
Or run 1000 tests (this may fail a few times, but keep running and it will work--this is due to the built-in generator not being able to satisfy the second part of the :args predicate; a custom generator can be written if needed):
(stest/check `subs)
Or, want to see if your app makes calls to subs that are invalid while it's running in real time? Just run this, and you'll get a spec exception if the function is called and the specs are not met:
(stest/instrument `subs)
We have not integrated this into our work flow yet, and can't in production since it's still alpha, but the first goal is to write specs. I'm putting them in the same namespace but in separate files currently.
I foresee our work flow being to run the tests for spec'd functions using this (found in the clojure spec guide):
(-> (stest/enumerate-namespace 'user) stest/check)
Then, it would be advantageous to turn on instrumenting for all functions, and run the app under load as we normally would test it, and ensure that "real world" data works.
You can also use s/conform to destructure complex data in functions themselves, or use s/valid as pre- and post- conditions for running functions. I'm not too keen on this, as it's overhead in a production system, but it is a possibility.
The sky's the limit, and we've just scratched the surface! Cool things coming in the next months and years with spec!
I defined a function play like so:
(let [play-switch (fn [c]
(condp = c
:scales (play-scale)
:intervals (play-interval)
:chords (play-chord)
:inversions (play-inversion)
(print "play error, invalid :practice_mode state")))]
(def ^:dynamic play
"if no argument, evaluates the appropriate play-<mode> function, based on the current state of #opts.
if one argument, does the corresponding play-<mode> function."
(fn
([] (play-switch (get #opts :practice_mode)))
([method] (play-switch method)))))
When I call play in my application like (play), its intended side effect does not occur. However, when I play in my application like (clojure.tools.trace/dotrace [play] (play)). How could tracing a function affect what it does?
Tracing a function can force the realization of otherwise unrealized lazy sequences.
Such problems are very commonly instances of "the dreaded lazy bug" where the side effects (aka the work) of one of the functions is being run in a lazy sequence. This has the frusterating effect that if you call the function directly from the REPL the side effect happens as the result is printed and if you call it with trace the lazy sequence is realized when trace prints the result. Though in the normal case the side effects never happen because the lazy sequence it not realized.
put calls to doall (if you need the result) or dorun (if you don't need the result) around each side effecting lazy sequence.
I'm calling clojure.core/time which is documented as "Evaluates expr and prints the time it took. Returns the value of expr"
Eg:
(time (expensive))
Macroexpanding it shows that it does store the value as a let so after outputting the time it should return immediately with the value in the let expression.
When I make the call however with an expensive computation, I see a delay and then get the time back, but then have to wait for a significant time (sometimes +10 seconds or more) for the result to appear.
Can anyone explain what's going on here?
PS: This is using clojure 1.3.0 if that makes any difference.
Maybe you are returning something lazy, of which the elements are only produced when fed to the REPL? In that case, you might want to wrap it in a dorun which forces all elements to be produced.
If you could provide the details of your expensive computation, we could see if this is true.
Useful addition from Savanni D'Gerinel's comment:
The proper syntax is probably (time (doall (computation))) if you want to return the result and (time (dorun (computation))) if you do not.
I have a defrecord called a bag. It behaves like a list of item to count. This is sometimes called a frequency or a census. I want to be able to do the following
(def b (bag/create [:k 1 :k2 3])
(keys bag)
=> (:k :k1)
I tried the following:
(defrecord MapBag [state]
Bag
(put-n [self item n]
(let [new-n (+ n (count self item))]
(MapBag. (assoc state item new-n))))
;... some stuff
java.util.Map
(getKeys [self] (keys state)) ;TODO TEST
Object
(toString [self]
(str ("Bag: " (:state self)))))
When I try to require it in a repl I get:
java.lang.ClassFormatError: Duplicate interface name in class file compile__stub/techne/bag/MapBag (bag.clj:12)
What is going on? How do I get a keys function on my bag? Also am I going about this the correct way by assuming clojure's keys function eventually calls getKeys on the map that is its argument?
Defrecord automatically makes sure that any record it defines participates in the ipersistentmap interface. So you can call keys on it without doing anything.
So you can define a record, and instantiate and call keys like this:
user> (defrecord rec [k1 k2])
user.rec
user> (def a-rec (rec. 1 2))
#'user/a-rec
user> (keys a-rec)
(:k1 :k2)
Your error message indicates that one of your declarations is duplicating an interface that defrecord gives you for free. I think it might actually be both.
Is there some reason why you cant just use a plain vanilla map for your purposes? With clojure, you often want to use plain vanilla data structures when you can.
Edit: if for whatever reason you don't want the ipersistentmap included, look into deftype.
Rob's answer is of course correct; I'm posting this one in response to the OP's comment on it -- perhaps it might be helpful in implementing the required functionality with deftype.
I have once written an implementation of a "default map" for Clojure, which acts just like a regular map except it returns a fixed default value when asked about a key not present inside it. The code is in this Gist.
I'm not sure if it will suit your use case directly, although you can use it to do things like
user> (:earth (assoc (DefaultMap. 0 {}) :earth 8000000000))
8000000000
user> (:mars (assoc (DefaultMap. 0 {}) :earth 8000000000))
0
More importantly, it should give you an idea of what's involved in writing this sort of thing with deftype.
Then again, it's based on clojure.core/emit-defrecord, so you might look at that part of Clojure's sources instead... It's doing a lot of things which you won't have to (because it's a function for preparing macro expansions -- there's lots of syntax-quoting and the like inside it which you have to strip away from it to use the code directly), but it is certainly the highest quality source of information possible. Here's a direct link to that point in the source for the 1.2.0 release of Clojure.
Update:
One more thing I realised might be important. If you rely on a special map-like type for implementing this sort of thing, the client might merge it into a regular map and lose the "defaulting" functionality (and indeed any other special functionality) in the process. As long as the "map-likeness" illusion maintained by your type is complete enough for it to be used as a regular map, passed to Clojure's standard function etc., I think there might not be a way around that.
So, at some level the client will probably have to know that there's some "magic" involved; if they get correct answers to queries like (:mars {...}) (with no :mars in the {...}), they'll have to remember not to merge this into a regular map (merge-ing the other way around would work fine).