This is a follow up to my question "Recursively reverse a sequence in Clojure".
Is it possible to reverse a sequence using the Clojure "for" macro? I'm trying to better understand the limitations and use-cases of this macro.
Here is the code I'm starting from:
((defn reverse-with-for [s]
(for [c s] c))
Possible?
If so, I assume the solution may require wrapping the for macro in some expression that defines a mutable var, or that the body-expr of the for macro will somehow pass a sequence to the next iteration (similar to map).
Clojure for macro is being used with arbitrary Clojure sequences.
These sequences may or may not expose random access like vectors do. So, in general case, you do not have access to the last element of a Clojure sequence without traversing all the way to it, which would make making a pass through it in reverse order not possible.
I'm assumming you had something like this in mind (Java-like pseudocode):
for(int i = n-1; i--; i<=0){
doSomething(array[i]);
}
In this example we know array size n in advance and we can access elements by its index. With Clojure sequences we don't know that. In Java it makes sense to do that with arrays and ArrayLists. Clojure sequences are however much more like linked lists - you have an element, and a reference to next one.
Btw, even if there were a (probably non-idiomatic)* way to do that, its time complexity would be something like O(n^2) which is just not worth the effort compared to much easier solution in the linked post which is O(n^2) for lists and a much better O(n) for vectors (and it is quite elegant and idiomatic. In fact, the official reverse has that implementation).
EDIT:
A general advice: Don't try to do imperative programming in Clojure, it wasn't designed for it. Although many things may seem strange or counter-intuitive (as opposed to well known idioms from imperative programming) once you get used to the functional way of doing things it is a lot, and I mean a lot easier.
Specifically for this question, despite the same name Java (and other C-like) for and Clojure for are not the same thing! First is an actual loop - it defines a flow control. The second one is a comprehension - look at it conceptually as a higher function of a sequence and a function f to be done for each of its element, which returns another sequence of f(element) s. Java for is a statement, it doesn't evaluate to anything, Clojure for (as well as anything else in Clojure) is an expression - it evaluates to the sequence of f(element) s.
Probably the easiest way to get the idea is to play with sequence functions library: http://clojure.org/sequences. Also, you can solve some problems on http://www.4clojure.com/. The first problems are very easy but they gradually get harder as you progress through them.
*As shown in Alexandre's answer the solution to the problem in fact is idiomatic and quite clever. Kudos for that! :)
Here's how you could reverse a string with for:
(defn reverse-with-for [s]
(apply str
(for [i (range (dec (count s)) -1 -1)]
(get s i))))
Note that this code is mutation free. It's the same as:
(defn reverse-with-map [s]
(apply str
(map (partial get s) (range (dec (count s)) -1 -1))))
A simpler solution would be:
(apply str (reverse s))
First of all, as Goran said, for is not a statement - it is an expression, namely sequence comprehension. It construct sequences by iteration through other sequences. So in the form it is meant to be used it is pure function (without side-effects). for can be seen as enhanced map infused with filter. Because of this it cannot be used to hold iteration state as e.g. reduce do.
Secondly, you can express sequence reversal using for and mutable state, e.g. using an atom, which is rough equivalent (not taking into account its concurrency properties) of java variable. But doing so you are facing several problems:
You are breaking main language paradigm so you will definitely get worse looking and behaving code.
Since all clojure mutable state cells are designed to be thread-safe, they all use some kind of illegal concurrent modification protection, and there is no ability to remove it. Consequently, you will get poorer performance characteristics.
In this particular case, like Goran said, sequences are one of the wide-used Clojure abstractions. For example, there are lazy sequences, which could be potentially infinite, so you just cannot walk them to the end. You certainly will have difficulties trying to work with such sequences with imperative techniques.
So don't do it, at least in Clojure :)
EDIT: I forgot to mention it. for returns lazy sequence, so you have to evaluate it in some way in order to apply all state mutations you do in it. Another reason not to do so :)
Related
I have been using Clojure, ClojureScript, lein, shadow-cljs, re-frame,
reagent, Emacs, and CIDER to work on a Clojure/ClojureScript dynamic
web app project. I am new to Clojure.
At some point in the codebase there is a big use of doall command followed by the use of reduce in order to generate hiccup (HTML renderer):
(doall
(reduce
(fn ...) ...)
[] ; hiccup-output
project-variable)
I am new to Clojure. But this felt weird to me considering documentation:
When lazy sequences are produced via functions that have side
effects, any effects other than those needed to produce the first
element in the seq do not occur until the seq is consumed. doall can
be used to force any effects. Walks through the successive nexts of
the seq, retains the head and returns it, thus causing the entire
seq to reside in memory at one time.
1 - Isn't doall supposed to be used with lazy sequences?
2 - I believe reduce is not one. Am I wrong?
3 - If doall should not be used with reduce in this case, what would be the recommendation for a refactoring here?
Yes, doall only makes sense with lazy collections
reduce does not by itself create a lazy collection. But the reducing function (the first argument to reduce) might end up returning a lazy collection - in that case, doall might make sense
Impossible to say because the code that you've provided is incomplete and the parentheses are unbalanced
How in Clojure process collections like in Java streams - one by one thru all the functions instead of evaluating all the elements in all the stack frame. Also I would describe it as Unix pipes (next program pulls chunk by chunk from previous one).
As far as I understand your question, you may want to look into two things.
First, understand the sequence abstraction. This is a way of looking at collections which consumes them one by one and lazily. It is an important Clojure idiom and you'll meet well known functions like map, filter, reduce, and many more. Also the macro ->>, which was already mentioned in a comment, will be important.
After that, when you want to dig deeper, you probably want to look into transducers and reducers. In a grossly oversimplifying summary, they allow you combine several lazy functions into one function and then process a collection with less laziness, less memory consumption, more performance, and possibly on several threads. I consider these to be advanced topics, though. Maybe the sequences are already what you were looking for.
Here is a simple example from ClojureDocs.org
;; Use of `->` (the "thread-first" macro) can help make code
;; more readable by removing nesting. It can be especially
;; useful when using host methods:
;; Arguably a bit cumbersome to read:
user=> (first (.split (.replace (.toUpperCase "a b c d") "A" "X") " "))
"X"
;; Perhaps easier to read:
user=> (-> "a b c d"
.toUpperCase
(.replace "A" "X")
(.split " ")
first)
"X"
As always, don't forget the Clojure CheatSheet or Clojure for the Brave and True.
I am learning clojure, and implementing my standard test project, Tic Tac Toe 10. I have written the same AI in many languages before and have had problems scaling it beyond 6 moves ahead in the other languages also.
I got the AI algorithm basically working, but I'm trying to improve the speed with pmap. Since everything is immutable, I should be able to just drop in pmap in place of map and get the same results, but I'm not seeing that.
(defn get-spot
[board player win-cond levels]
(def avail (get-available board))
(def final
(apply merge
(map #(array-map % (calc-score board player win-cond levels %)) avail)))
final)
But pmap in that spot returns inconsistent results. Not sure where to start looking. I can post more of the code if it's needed.
Replacing def with let everywhere solved the problem. My functions had lots of inconsistent side effects if I didn't use let.
(defn get-spot
[board player win-cond levels]
(let [avail (get-available board)
final (apply merge
(pmap #(array-map % (calc-score board player win-cond levels %)) avail))]
final))
I've written a bunch of clojure code, and read through many tutorials. I don't know how I missed that really important detail.
Please read a book or two about clojure before you even consider to do performance tuning. Having immutability built into a language does not mean you can just replace map by pmap.
Two more things to consider are the absence of side effects for your code and the fact that operations will have to be commutativ.
If they are not, you will have to consider this in your merge algorithm.
As others already said, don't create def bindings insider functions, use let instead. Also, the last expression will be returned from a function anyway, so you don't need to bind final before returning it.
I am sorry for not giving a direct solution to your problem, I just think you need to understand that replacement of map is not as easy as it may seem.
If you still want your problem solved I guess we need to see more code here.
In Common Lisp, the let uses a list for a bindings, i.e:
(let ((var1 1)
(var2 2))
...)
While Clojure uses a vector instead:
(let [a 1
b 2]
...)
Is there any specific reason, other than readability, for Clojure to use a vector?
You can find Rich Hickey's argument at Simple Made Easy - slide 14, about 26 minutes in:
Rich's line on this was as follows
"Since we were talking about syntax, let’s look at
classic Lisp. It seems to be the simplest of syntax, everything is a
parenthesized list of symbols, numbers, and a few other things. What
could be simpler? But in reality, it is not the simplest, since to
achieve that uniformity, there has to be substantial overloading of
the meaning of lists. They might be function calls, grouping
constructs, or data literals, etc. And determining which requires
using context, increasing the cognitive load when scanning code to
assess its meaning. Clojure adds a couple more composite data literals
to lists, and uses them for syntax. In doing so, it means that lists
are almost always call-like things, and vectors are used for grouping,
and maps have their own literals. Moving from one data structure to
three reduces the cognitive load substantially."
One of the things he believes was overloaded in the standard syntax was access time. So vector syntax in arguments is related to the constant access time when you used them. He said:
Seems odd though as it only is valid for that one form...as soon as it is stored in a variable or passed in any way the information is 'lost'. For example...
(defn test [a]
(nth a 0)) ;;<- what is the access time of and element of a?
I personally prefer harsh syntax changes like brackets to be reserved for when the programmer has to switch mental models e.g. for embedded languages.
;; Example showing a possible syntax for an embedded prolog.
{size [],0}
{size([H|T],N) :- size(T,N1), N is N1+1}
(size '(1 2 3 4) 'n) ;; here we are back to lisp code
Such a concept is syntactically constant. You don't 'pass around' structure at runtime. Before runtime (read/macro/compile time) is another matter though so where possible it is often better to keep things as lists.
[edit]
The original source seems to have gone, but here is another record of the interview: https://gist.github.com/rduplain/c474a80d173e6ae78980b91bc92f43d1#file-code-quarterly-rich-hickey-2011-md
Clojure has a number of libraries for generative testing such as test.check, test.generative or data.generators.
It is possible to use higher order functions to create random data generators that are composable such as:
(defn gen [create-fn content-fn lazy]
(fn [] (reduce #(create-fn %1 %2) (for [a lazy] (content-fn)))))
(def a (gen str #(rand-nth [\a \b \c]) (range 10)))
(a)
(def b (gen vector #(rand-int 10) (range 2)))
(b)
(def c (gen hash-set b (range (rand-int 10))))
(c)
This is just an example and could be modified with different parameters, filters, partials, etc to create data generating functions which are quite flexible.
Is there something that any of the generative libraries can do that isn't also just as (or more) succinctly achievable by composing some higher order functions?
As a side note to the stackoverflow gods: I don't believe this question is subjective. I'm not asking for an opinion on which library is better. I want to know what specific feature(s) or technique(s) of any/all data generative libraries differentiate them from composing vanilla higher order functions. An example answer should illustrate generating random data using any of the libraries with an explanation as to why this would be more complex to do by composing HOFs in the way I have illustrated above.
test.check does this way better. Most notably, suppose you generate a random list of 100 elements, and your test fails: something about the way you handled that list is wrong. What now? How do you find the basic bug? It surely doesn't depend on exactly those 100 inputs; you could probably reproduce it with a list of just a few elements, or even an empty list if something is wrong with your base case.
The feature that makes all this actually useful isn't the random generators, it is the "shrinking" of those generators. Once test.check finds an input that breaks your tests, it tries to simplify the input as much as possible while still making your tests break. For a list of integers, the shrinks are simple enough you could maybe do them yourself: remove any element, or decrease any element. Even that may not be true: choosing the order to do shrinks in is probably a harder problem than I realize. And for larger inputs, like a list of maps from vectors to a 3-tuple of [string, int, keyword], you'll find it totally unmanageable, whereas test.check has done all the hard work already.