It tried the following in REPL and got no error (because of lazy evaluation):
(defn square [n] (* n n))
(if (= 0 0)
(println "hello")
(map square ["a" "b"]))
The following gives error (because it gets evaluated):
(defn square [n] (* n n))
(if (= 0 1)
(println "hello")
(map square ["a" "b"]))
My concern is that if in a large piece of code, there are some parts which my test cases could never touch, then for example, a code like above will crash in production!
Is there a way to check these things at compile time?
Thanks
It's not lazy evaluation. It's conditional evaluation.
Special form (if condition expr1 expr2) evaluates expr2 only if condition was false.
Your concern is not related to Clojure nor laziness.
Try this in Ruby:
irb(main):003:0> raise "wannabe error" if false
=> nil
[edit: answering your question more precisely]
You can hardly have such error detection at compile time in a dynamic language.
At runtime you can evaluate tests with different values of conditions to reach possibly all revelant code branches. I guess the situation is the same in all dynamic languages. It's the reason we write the tests for.
So you'd want to cover both cases, true and false, for the condition you're concerned about in your tests.
Personally I'm a fan of solutions like QuickCheck which generate a wide range of possible values (guarded by some conditions if you need) for the purpose of testing. In the case above you'd only want to test 2 cases, true and false, but imagine other tests triggered by a wide range of input values. Unfortunately there are not many implementations of such generators. The only QuickCheck clones for dynamic typing languages I know are for Erlang (proprietary and quite expensive) and RushCheck for Ruby. The Clojure implementation is in its infancy.
Nope. Then again, if code behaves bad in cases not tested by test cases, it will not be caught by the test cases.
You could add a type system on top of Clojure using macros, but then you run into limitations that the type system has and the limitations it would impose.
(Side note: this isn't really lazy evaluation; this is the behavior of the if special form)
Related
I'm new in clojure so I would like to know some things
There are 2 ways to typehint your code, I would like to know the difference
^String and others using #^String
When I saw the code from when I was a little disappointed, it's an if if that is not true could explain me why (assert is a when with throw)
(defmacro when
"Evaluates test. If logical true, evaluates body in an implicit do."
{:added "1.0"}
[test & body]
(list 'if test (cons 'do body)))
#^ is the old, deprecated syntax. Just use ^ now going forward, unless you're running a very old version of Clojure (pre-1.2).
Why be disappointed? This is part of the beauty of languages like Clojure! when is just defined in terms of if. It looks like a language construct, but it's actually just a macro that anyone can write. when literally turns into an if with a do at compile time. It would arguably complicate the language to have made when a special form when it's entirely possible to define it in terms of existing forms.
The purpose of when is to carry out side effects when the condition holds, but do nothing otherwise. It's simply a readability helper. It's similar to the if-not, and when-not macros that just turn into ifs with their condition negated. They aren't necessary by any means, but they certainly help clean up code.
I have recently watched Rich Hickeys talk at Cojure Conj 2016 and although it was very interesting, I didn't really understand the point in clojure.spec or when you'd use it. It seemed like most of the ideas, such as conform, valid etc, had similar functions in Clojure already.
I have only been learning clojure for around 3 months now so maybe this is due to lack of programming/Clojure experience.
Do clojure.spec and cljs.spec work in similar ways to Clojure and Cljs in that, although they are not 100% the same, they are based on the same underlying principles.
Are you tired of documenting your programs?
Does the prospect of making up yet more tests cause procrastination?
When the boss says "test coverage", do you cower with fear?
Do you forget what your data names mean?
For smooth expression of hard specifications, you need Clojure.Spec!
Clojure.spec gives you a uniform method of documenting, specifying, and automatically testing your programs, and of validating your live data.
It steals virtually every one of its ideas. And it does nothing you can't do for yourself.
But in my - barely informed - opinion, it changes the economy of specification, making it worth while doing properly. A game-changer? - quite possibly.
At the clojure/conj conference last week, probably half of the presentations featured spec in some way, and it's not even out of alpha yet. spec is a major feature of clojure; it is here to stay, and it is powerful.
As an example of its power, take static type checking, hailed as a kind of safety net by so many, and a defining characteristic of so many programming languages. It is incredibly limited in that it's only good at compile time, and it only checks types. spec, on the other hand, validates and conforms any predicate (not just type) for the args, the return, and can also validate relationships between the two. All of this is external to the function's code, separating the logic of the function from being commingled with validation and documentation about the code.
Regarding WORKFLOW:
One archetypal example of the benefits of relationship-checking, versus only type-checking, is a function which computes the substring of a string. Type checking ensures that in (subs s start end) the s is a string and start and end are integers. However, additional checking must be done within the function to ensure that start and end are positive integers, that end is greater than start, and that the resulting substring is no larger than the original string. All of these things can be spec'd out, for example (forgive me if some of this is a bit redundant or maybe even inaccurate):
(s/fdef clojure.core/subs
:args (s/and (s/cat :s string? :start nat-int? :end (s/? nat-int?))
(fn [{:keys [s start end]}]
(if end
(<= 0 start end (count s))
(<= 0 start (count s)))))
:ret string?
:fn (fn [{{:keys [s start end]} :args, substring :ret}]
(and (if end
(= (- end start) (count substring))
(= (- (count s) start) (count substring)))
(<= (count substring) (count s)))))
Call the function with sample data meeting the above args spec:
(s/exercise-fn `subs)
Or run 1000 tests (this may fail a few times, but keep running and it will work--this is due to the built-in generator not being able to satisfy the second part of the :args predicate; a custom generator can be written if needed):
(stest/check `subs)
Or, want to see if your app makes calls to subs that are invalid while it's running in real time? Just run this, and you'll get a spec exception if the function is called and the specs are not met:
(stest/instrument `subs)
We have not integrated this into our work flow yet, and can't in production since it's still alpha, but the first goal is to write specs. I'm putting them in the same namespace but in separate files currently.
I foresee our work flow being to run the tests for spec'd functions using this (found in the clojure spec guide):
(-> (stest/enumerate-namespace 'user) stest/check)
Then, it would be advantageous to turn on instrumenting for all functions, and run the app under load as we normally would test it, and ensure that "real world" data works.
You can also use s/conform to destructure complex data in functions themselves, or use s/valid as pre- and post- conditions for running functions. I'm not too keen on this, as it's overhead in a production system, but it is a possibility.
The sky's the limit, and we've just scratched the surface! Cool things coming in the next months and years with spec!
I find it hard to reason about macro-expansion and was wondering what the best practices were for testing them.
So if I have a macro, I can perform one level of macro expansion via macroexpand-1.
(defmacro incf-twice (n)
`(progn
(incf ,n)
(incf ,n)))
for example
(macroexpand-1 '(incf-twice n))
evaluates to
(PROGN (INCF N) (INCF N))
It seems simple enough to turn this into a test for the macro.
(equalp (macroexpand-1 '(incf-twice n))
'(progn (incf n) (incf n)))
Is there an established convention for organizing tests for macros? Also, is there a library for summarizing differences between s-expressions?
Generally testing macros is not one of the strong parts of Lisp and Common Lisp. Common Lisp (and Lisp dialects in general) uses procedural macros. The macros can depend on the runtime context, the compile-time context, the implementation and more. They also can have side effects (like registering things in the compile-time environment, registering things in the development environment and more).
So one might want to test:
that the right code gets generated
that the generated code actually does the right thing
that the generated code actually works in code contexts
that the macro arguments are actually parsed correctly in case of complex macros. Think loop, defstruct, ... macros.
that the macro detects wrongly formed argument code. Again, think of macros like loop and defstruct.
the side effects
From above list on can infer that it is best to minimize all these problem areas when developing a macro. BUT: there are really really complex macros out there. Really scary ones. Especially those who are used to implemented new domain specific languages.
Using something like equalp to compare code works only for relatively simple macros. Macros often introduce new, uninterned and unique symbols. Thus equalp will fail to work with those.
Example: (rotatef a b) looks simple, but the expansion is actually complicated:
CL-USER 28 > (pprint (macroexpand-1 '(rotatef a b)))
(PROGN
(LET* ()
(LET ((#:|Store-Var-1234| A))
(LET* ()
(LET ((#:|Store-Var-1233| B))
(PROGN
(SETQ A #:|Store-Var-1233|)
(SETQ B #:|Store-Var-1234|))))))
NIL)
#:|Store-Var-1233| is a symbol, which is uninterned and newly created by the macro.
Another simple macro form with a complex expansion would be (defstruct s b).
Thus one would need a s-expression pattern matcher to compare the expansions. There are a few available and they would be useful here. One needs to make sure in the test patterns that the generated symbols are identical, where needed.
There are also s-expression diff tools. For example diff-sexp.
I agree with Rainer Joswig's answer; in general, this is a very difficult task to solve because macros can do a whole lot. However, I would point out that in many cases, the easiest way to unit test your macros is by making the macros do as little as possible. In many cases, the easiest implementation of a macro is just syntactic sugar around a simpler function. E.g., there's a typical pattern of with-… macros in Common Lisp (e.g., with-open-file), where the macro simply encapsulates some boilerplate code:
(defun make-frob (frob-args)
;; do something and return the resulting frob
(list 'frob frob-args))
(defun cleanup-frob (frob)
(declare (ignore frob))
;; release the resources associated with the frob
)
(defun call-with-frob (frob-args function)
(let ((frob (apply 'make-frob frob-args)))
(unwind-protect (funcall function frob)
(cleanup-frob frob))))
(defmacro with-frob ((var &rest frob-args) &body body)
`(call-with-frob
(list ,#frob-args)
(lambda (,var)
,#body)))
The first two functions here, make-frob and cleanup-frob are relatively straightforward to unit test. The call-with-frob is a bit harder. The idea is that it's supposed to handle the boilerplate code of creating the frob and ensuring that the cleanup call happens. That's a bit harder to check, but if the boilerplate only depends on some well defined interfaces, then you'll probably be able to create mock up a frob that can detect whether it's cleaned up correctly. Finally, the with-frob macro is so simple that you can probably test it the way you've been considering, i.e., checking its expansion. Or you might say that it's simple enough that you don't need to test it.
On the other hand, if you're looking at a much more complex macro, such as loop, which is really a kind of compiler in its own right, you're almost certainly already going to have the expansion logic in some separate functions. E.g., you might have
(defmacro loop (&body body)
(compile-loop body))
in which case you really don't need to test loop, you need to test compile-loop, and then you're back in the realm of your usual unit testing.
I'd generally just test the functionality, not the shape of the expansion.
Yes, there are all kinds of contexts and surroundings that might influence what happens, but if you rely on such things, it should be no problem to set them up the same for your test.
Some common cases:
binding macros: test that the variables are bound as intended inside and that any shadowed outside variables are unaffected
unwind-protect wrappers: provoke a nonlocal exit from the inside and check that the cleanup is working
definition/registration: test that you can define/register what you want and use it afterwards
I've been developing in Java and Perl for a long time but wanted to learn something new so I've begun looking into clojure. One of the first things I've tried was a solution for the Towers of Hanoi puzzle but I've been getting strange behavior on my pretty-print function. Basically, my for loop is never entered when I run it with 'lein run' but it seems to work fine when I run it from the repl. Here's a stripped down example:
(ns test-app.core
(:gen-class))
(defn for-print
"Print the same thing 3 times"
[ p-string ]
(println (str "Checkpoint: " p-string))
(for
[x [1 2 3]]
(printf "FOR: %s\n" p-string)
))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(for-print "Haldo wurld!"))
When I run this with 'lein run' I only see the output from the "Checkpoint" println. If I remove that line I get no output at all. But, if I run 'lein repl' and then type (-main) it prints the string 3 times, as expected:
test-app.core=> (-main)
Checkpoint: Haldo wurld!
(FOR: Haldo wurld!
FOR: Haldo wurld!
FOR: Haldo wurld!
nil nil nil)
test-app.core=>
What's going on here? I have a feeling that I'm approaching this the wrong way, trying to use my past Perl/Java mentality to write clojure. What would be an idiomatic way to run the same task a set number of times?
The for loop is returning a lazy sequence that is evaluated only as needed.
When you run the program inside the repl the for result is realized in order to display the result on screen.
But when you run using lein run result is never used so the collection is not realized.
You have a couple alternatives:
1) use doall outside the for loop for force lazy sequence realization
Ex:
(defn for-print
"Print the same thing 3 times"
[ p-string ]
(println (str "Checkpoint: " p-string))
(doall (for
[x [1 2 3]]
(printf "FOR: %s\n" p-string))))
2) since you're only printing which is a side effect and not really creating a collection you can use doseq.
Ex:
(defn for-print
"Print the same thing 3 times"
[ p-string ]
(println (str "Checkpoint: " p-string))
(doseq [x [1 2 3]]
(printf "FOR: %s\n" p-string)))
Clojure for is not an imperative loop (you should avoid thinking about loops in Clojure at all), it's a list comprehension, which returns lazy sequence. It's made for creating a sequence, not for printing anything. You can realize it, and make it work, but it is bad way to go.
As Guillermo said, you're looking for doseq macro, which is designed for side effects. It's probably the most idiomatic Clojure in your particular case.
From my point of view the most similar to imperative loops construction in Clojure is tail recursion, made with loop/recur. Still it's rather low level construct in Clojure and certainly should not be used in a imperative loop manner. Have a better look into functional programming principles, as well as Clojure core functions. Trying to transfer Java/Perl thinking to Clojure may harm you.
The other answers are correct, and provide details. I want to add some higher-level clarification that might be helpful. In most languages, "for" means "for such and conditions, perform such and such actions (which could be of an arbitrary type, including side-effects)." This kind of thing can be done in Clojure, and I have seen experienced Clojure programmers do it when it's useful and convenient. However, using loops with side-effects works usually against the strengths of the language. So in Clojure the word "for" has a different meaning in most languages: Generating (lazy) sequences. It means something like "For these inputs, when/while/etc. they meet such and such conditions, bind then temporarily to these variables, generate a (lazy) sequence by processing each set of values in such and such a way."
Common Lisp has return-from; is there any sort of return in Clojure for when you want to return early from a function?
When you need to bail out of a computation early, you need a way to do that, not an argument from purists. Usually you need it when you're reducing a big collection and a certain value indicates that there's no point in further processing the collection. To that end, the ever-practical Clojure provides the reduced function.
A simple example to illustrate is that when multiplying a sequence of numbers, if you encounter a zero, you already know that the final result will be zero, so you don't need to look at the rest of the sequence. Here's how you code that with reduced:
(defn product [nums]
(reduce #(if (zero? %2)
(reduced 0.0)
(* %1 %2))
1.0
nums))
reduced wraps the value you give it in a sentinel data structure so that reduce knows to stop reading from the collection and simply return the reduced value right now. Hey, it's pure-functional, even!
You can see what's going on if you wrap the above if in a do with a (println %1 %2):
user=> (product [21.0 22.0 0.0 23.0 24.0 25.0 26.0])
1.0 21.0
21.0 22.0
462.0 0.0
0.0
user=> (product [21.0 22.0 23.0 24.0 25.0 26.0])
1.0 21.0
21.0 22.0
462.0 23.0
10626.0 24.0
255024.0 25.0
6375600.0 26.0
1.657656E8
There isn't any explicit return statement in clojure. You could hack something together using a catch/throw combination if you want to, but since clojure is much more functional than common lisp, the chances you actually need an early return right in the middle of some nested block is much smaller than in CL. The only 'good' reason I can see for return statements is when you're dealing with mutable objects in a way that's not idiomatic in clojure.
I wouldn't go as far as saying that it's never useful, but I think in Clojure, if your algorithm needs a return statement, it's a major code smell.
Unless you're writing some really funky code, the only reason you'd ever want to return early is if some condition is met. But since functions always return the result of the last form evaluated, if is already this function — just put the value that you want to return in the body of the if and it will return that value if the condition is met.
I'm no expert in Clojure, but it seems it does not have those construct to try to be more functional. Take a look at what Stuart Halloway says here:
Common Lisp also supports a return-from macro to "return" from the
middle of a function. This encourages an imperative style of
programming, which Clojure discourages.
However, you can solve the same problems in a different way. Here is
the return-from example, rewritten in a functional style so that no
return-from is needed:
(defn pair-with-product-greater-than [n]
(take 1 (for [i (range 10) j (range 10) :when (> (* i j) n)] [i j])))
That is, use lazy sequences and returning values based on conditions.
As an alternative you could use cond . And if on some conditions you would need to evaluate multiple expressions use do.
Here is an example:
(defn fact [x]
(cond
(< x 0) (do (println (str x " is negative number"))
(throw (IllegalArgumentException. "x should be 0 or higher")))
(<= x 1) 1
:else (* x (fact (- x 1)))))
The if option already given is probably the best choice, and note since maps are easy, you can always return {:error "blah"} in the error condition, and{result: x} in the valid condition.
There isn't a return statement in Clojure. Even if you choose not to execute some code using a flow construct such as if or when, the function will always return something, in these cases nil. The only way out is to throw an exception, but even then it will either bubble all the way up and kill your thread, or be caught by a function - which will return a value.
In short, no. If this is a real problem for you then you can get around it with
"the maybe monad" Monads have a high intellectual overhead so for many cases clojurians tend to avoid the "if failed return" style of programming.
It helps to break the function up into smaller functions to reduce the friction from this.