I am trying to really understand partial in Clojure. partial takes a function f and fewer than the normal arguments to f, and returns a fn that takes a variable number of additional args. When called, the returned function calls f with args + additional args. This is the working example for partial seems not making sense to me ((partial map *) [1 2 3] [4 5 6] [7 8 9]). map and * are functions; therefore partial is taking more than one function and invalidating the definition. Anyone can help me understand better?
There is nothing wrong with a function being an argument. In your case map * is created, which can be used later:
(def will-map-multiply (partial map *))
As with any partial function the rest of its arguments can be supplied later:
(will-map-multiply [1 2 3] [4 5 6] [7 8 9]) ;=> (28 80 162)
That was 'partialising' one of the arguments. It could have been two:
(def will-map-multiply-again (partial map * [1 2 3]))
(will-map-multiply-again [4 5 6] [7 8 9]) ;=> (28 80 162)
When using partial the first parameter must be a function, and the others can be whatever - no reason they can't also be functions.
Your example is a bit complex because multiplying requires two or more arguments, and hence map needs two or more sequences.
You're over-thinking it. The first argument to partial must be a function. The next arguments must be whatever would be valid arguments to that function you just named as the first argument -- including other functions, if your first parameter is a higher-order function like map or reduce or many others.
(partial map *)
The first argument is a function, map. The next argument is whatever would be a valid argument to map. Since map requires a function as its first argument, this partial expression requires a function as its second argument.
As simple as that. You can think of the * as an argument to the map first, and by extension, an argument to partial.
Functions are first-class values in Clojure. This means, you can pass them around like, for example, integers, maps or strings.
So, there’s nothing strange with (partial map *), where function * is just a map’s first argument. In your case, partial constructs a function which then passes additional collections to map. Let's see:
(partial map *)
;; becomes
(fn [coll1 coll2 coll3] ;; other arities ommited
(map * coll1 coll2 coll3))
;; let’s call this fn with your collections
((fn [coll1 coll2 coll3]
(map * coll1 coll2 coll3)) [1 2 3] [4 5 6] [7 8 9])
;; becomes
(map * coll1 coll2 coll3)
Again, the main point is that functions are first-class in Clojure and many other functional languages.
Hope this helps!
(partial map *) looks strange, but when one realizes that map takes a function as its first argument, it becomes more clear. * is just a function that takes variable number of arguments and multiplies them. Therefore,
(map * [1 2 3] [4 5 6] [7 8 9]) applies * to three triples of numbers and produces [28 80 162]. All this has nothing to do with partial. The confusion stems from the fact that in clojure functions are often passed as arguments to other functions (actually this is becoming common in all modern languages, even JavaScript!).
So, (partial map *) means exactly: take map with pre-selected first argument (*) and create a new function that uses its arguments as vectors to perform the mapping.
Related
For me as, a new Clojurian, some core functions seem rather counter-intuitive and confusing when it comes to arguments order/position, here's an example:
> (nthrest (range 10) 5)
=> (5 6 7 8 9)
> (take-last 5 (range 10))
=> (5 6 7 8 9)
Perhaps there is some rule/logic behind it that I don't see yet?
I refuse to believe that the Clojure core team made so many brilliant technical decisions and forgot about consistency in function naming/argument ordering.
Or should I just remember it as it is?
Thanks
Slightly offtopic:
rand&rand-int VS random-sample - another example where function naming seems inconsistent but that's a rather rarely used function so it's not a big deal.
There is an FAQ on Clojure.org for this question: https://clojure.org/guides/faq#arg_order
What are the rules of thumb for arg order in core functions?
Primary collection operands come first. That way one can write → and its ilk, and their position is independent of whether or not they have variable arity parameters. There is a tradition of this in OO languages and Common Lisp (slot-value, aref, elt).
One way to think about sequences is that they are read from the left, and fed from the right:
<- [1 2 3 4]
Most of the sequence functions consume and produce sequences. So one way to visualize that is as a chain:
map <- filter <- [1 2 3 4]
and one way to think about many of the seq functions is that they are parameterized in some way:
(map f) <- (filter pred) <- [1 2 3 4]
So, sequence functions take their source(s) last, and any other parameters before them, and partial allows for direct parameterization as above. There is a tradition of this in functional languages and Lisps.
Note that this is not the same as taking the primary operand last. Some sequence functions have more than one source (concat, interleave). When sequence functions are variadic, it is usually in their sources.
Adapted from comments by Rich Hickey.
Functions that work with seqs usually has the actual seq as last argument.
(map, filter, remote etc.)
Accessing and "changing" individual elements takes a collection as first element: conj, assoc, get, update
That way, you can use the (->>) macro with a collection consistenly,
as well as create transducers consistently.
Only rarely one has to resort to (as->) to change argument order. And if you have to do so, it might be an opportunity to check if your own functions follow that convention.
For some functions (especially functions that are "seq in, seq out"), the args are ordered so that one can use partial as follows:
(ns tst.demo.core
(:use tupelo.core tupelo.test))
(dotest
(let [dozen (range 12)
odds-1 (filterv odd? dozen)
filter-odd (partial filterv odd?)
odds-2 (filter-odd dozen) ]
(is= odds-1 odds-2
[1 3 5 7 9 11])))
For other functions, Clojure often follows the ordering of "biggest-first", or "most-important-first" (usually these have the same result). Thus, we see examples like:
(get <map> <key>)
(get <map> <key> <default-val>)
This also shows that any optional values must, by definition, be last (in order to use "rest" args). This is common in most languages (e.g. Java).
For the record, I really dislike using partial functions, since they have user-defined names (at best) or are used inline (more common). Consider this code:
(let [dozen (range 12)
odds (filterv odd? dozen)
evens-1 (mapv (partial + 1) odds)
evens-2 (mapv #(+ 1 %) odds)
add-1 (fn [arg] (+ 1 arg))
evens-3 (mapv add-1 odds)]
(is= evens-1 evens-2 evens-3
[2 4 6 8 10 12]))
Also
I personally find it really annoying trying to parse out code using partial as with evens-1, especially for the case of user-defined functions, or even standard functions that are not as simple as +.
This is especially so if the partial is used with 2 or more args.
For the 1-arg case, the function literal seen for evens-2 is much more readable to me.
If 2 or more args are present, please make a named function (either local, as shown for evens-3), or a regular (defn some-fn ...) global function.
As also explained in the answers of this question, Clojure determines the number of parameters of an anonymous function (defined through #()), by the maximal parameter index referenced in the body (e.g. if the maximal referenced parameter is %4, then that anonymous function has 4 parameters).
Question: is there some syntax to tell the Clojure compiler that an anonymous function expects one parameter, even not referencing that parameter? Or, in this case, the only "clean way"* is to use the fn syntax? (* "Clean way": I'm aware that in most cases, you could reference the dummy parameter without affecting the function -- see below, but I would like to avoid such workarounds.)
My use case: I have a function (defn foo [predicate]), where predicate expects exactly one argument, and I would like to test foo with a predicate that always returns true. (I.e., something like #(true).) This causes an ArityException since the interpreter thinks that #(true) expects zero arguments.
In this particular case, I could use some clever hack like #(or true %), but:
It is almost as much to type as the fn-variant.
I'm interested if there is a generic solution.
Edit: so to be clear, I'm looking for something like this:
#[1](true) ;; the anonymous function takes one parameter
No. Just use fn. That's what it's for.
If it's a constant return you are after you can use constantly:
(map (constantly 10) '(1 2 3 4 5))
; ==> (10 10 10 10 10)
If it's not you can use fn with a rest argument:
(fn [& _]
some-side-effect-expression)
; ==> non functional value
In fact this can be done on higher order functions where you don't need all the passes arguments:
(some-fun (fn [a b & _] (+ a b)) '(1 2 3 4 5))
; ==> (3 5 7 9)
If you insist on the #( ... ) syntax, you can write
#(do % true)
... incurring no run-time penalty, whatever you want to return.
But (constantly true), as #Sylvester suggests, is idiomatic, and accepts any argument list:
((constantly true) 1 2 "Buckle my shoe");true
I need some help about higher-order functions' definition or explanation.
Some guys write a statement like this:
(map inc [9 5 4 8]) ; statement
And after this, an explanation of this statement:
[(inc 9) (inc 5) (inc 4) (inc 8)] ; explanation.
or in (this link) this first message, (reduce + (list 1 2 3 4 5))
"translates to":
(+ (+ (+ (+ 1 2) 3) 4) 5)
Is there a function which gives an explanation of a statement in Clojure?
The trick with understanding higher-order functions is not to over think them - they are actually quite simple. The higher-order function is really just a function which takes another function as one of its arguments and applies that function to each of the arguments in turn and does something with the results generated by applying the function.
You can even think of a higher order function as a mini-program. It takes an argument (a function) which says what to do with the data input (arguments). Each time the function which is passed in is applied to an argument, it generates some new value (which could be nil). The higher order function takes that result and does something with it. In the case of map, it adds it to a new sequence and it is this new sequence that will be returned as the overall result.
Consider a higher order sorting function, lets call it 'sort'. Its first argument is the function which will be used to compare elements to determine which comes first in the sort order and the remaining argument is the list of things to be sorted.
The actual sort function is really just scaffolding or the basic sort engine which ensures the list representing the data is processed until it is sorted. It might implement the basic mechanics for a bubble sort, a quicksort or some other sorting algorithm. Because it is a higher order function, it does not care or even know about how to compare elements to determine the correct order. It relies on the function which is passed in as its first argument to do that. All it wants is for that function to tell it if one value is higher, lower or the same rank as the other.
The function which is passed in to the sort function is the comparison function which determines your sort order. The actual sort function has no knowledge of how to sort the data. It will just keep processing the data until some criteria is met, such as one iteration of the data items where no change in order occurs. It expects the comparison function to take two arguments and return 1, 0 or -1 depending on whether the first argument passed to it is greater, equal or less than the second. If the function returns 1 or 0 it does nothing, but if the value is -1 it swaps A and B and then calls the function again with the new B value and the next value from the list of data. After the first iteration through the input, it will have a new list of items (same items, but different order). It may iterate through this process until no elements are swapped and then returns the final list, which will be sorted according to the sort criteria specified in the comparison function.
The advantage of this higher order function is that you can now sort data according to different sort criteria by simply defining a new comparison function - it just has to take two arguments and return 1, 0 or -1. We don't have to re-write all the low level sorting engine.
Clojure provides a number of basic scaffolding functions which are higher order functions. The most basic of these is map. The map function is very simple - it takes a function and one or more sequences. It iterates through the sequences, taking one element from each sequence and passing them to the function supplied as its first argument. It then puts the result from each call to this argument into a new sequence, which is returned as the final result. This is a little simplified as the map function can take more than one collection. When it does, it takes an element from each collection and expects that the function that was passed as the first argument will accept as many arguments as there are collections - but this is just a generalisation of the same principal, so lets ignore it for now.
As the execution of the map function doesn't change, we don't need to look at it in any detail when trying to understand what is going on. All we need to do is look at the function passed in as the first argument. We look at this function and see what it does based on the arguments passed in and know that the result of the call to map will be a new sequence consisting of all the values returned from applying the supplied function to the input data i.e. collections passed in to map.
If we look at the example you provided
(map inc [9 5 4 8])
we know what map does. It will apply the passed in function (inc) to each of the elements in the supplied collection ([9 5 4 8]). We know that it will return a new collection. To know what it will do, we need to look at the passed in function. The documentation for inc says
clojure.core/inc ([x]) Returns a number one greater than num. Does
not auto-promote longs, will throw on overflow. See also: inc'
I actually think that is a badly worded bit of documentation. Instead of saying "return a number one greater than num, it probably should either say, Return a number one greater than x or it should change the argument name to be ([num]), but yu get the idea - it simply increments its argument by 1.
So, map will apply inc to each item in the collection passed in as the second argument in turn and collect the result in a new sequence. Now we could represent this as [(inc 9) (inc 5) (inc 4) (iinc 8)], which is a vector of clojure forms (expressions) which can be evaluated. (inc 9) => 10, (inc 5) => 6 etc, which will result in [10 6 5 9]. The reason it is expressed as a vector of clojure forms is to emphasise that map returns a lazy sequence i.e. a sequence where the values do not exist until they are realised.
The key point to understand here is that map just goes through the sequences you provide and applies the function you provide to each item from the sequence and collects the results into a new sequence. The real work is done by the function passed in as the first argument to map. To understand what is actually happening, you just need to look at the function map is applying. As this is just a normal function, you can even just run it on its own and provide a test value i.e.
(inc 9)
This can be useful when the function is a bit more complicated than inc.
We cold just stop there as map can pretty much do everything we need. However, there are a few common processing patterns which occur frequently enough that we may want to abstract them out into their own functions, for eample reduce and filter. We could just implement this functionality in terms of map, but it may be complicated by having to track state or be less efficient, so they are abstracted out into their own functions. However, the overall pattern is pretty much the same. Filter is just like map, except the new sequence it generates only contains elements from the input collection which satisfy the predicate function passed in as the first argument. Reduce follows the same basic pattern, the passed in function is applied to elements from the collection to generate a new sequence. The big difference is that it 'reduces' the sequence in some way - either by reducing it to a new value or a new representation (such as hashmap or a set or whatever.
for example,
(reduce + (list 1 2 3 4 5))
follows the same basic pattern of applying the function supplied as the first argument to each of the items in the collection supplied as the second argument. Reduce is slightly different in that the supplied function must take two arguments and in each call following the first one, the first argument passed to the function represents the value returned from the last call. The example above can be written as
(reduce + 0 (list 1 2 3 4 5))
and executes as
(+ 0 1) => 1
(+ 1 2) => 3
(+ 3 3) => 6
(+ 6 4) => 10
(+ 10 5) => 15
so the return value will be 15. However reduce is actually more powerful than is obvious with that little example. The documentation states
clojure.core/reduce ([f coll] [f val coll]) Added in 1.0 f should be
a function of 2 arguments. If val is not supplied, returns the
result of applying f to the first 2 items in coll, then applying f
to that result and the 3rd item, etc. If coll contains no items, f
must accept no arguments as well, and reduce returns the result of
calling f with no arguments. If coll has only 1 item, it is
returned and f is not called. If val is supplied, returns the
result of applying f to val and the first item in coll, then
applying f to that result and the 2nd item, etc. If coll contains no
items, returns val and f is not called.
If you read that carefully, you will see that reduce can be used to accumulate a result which is carried forward with each application of the function. for example, you could use reduce to generate a map containing the sum of the odd and even numbers in a collection e.g.
(defn odd-and-even [m y]
(if (odd? y)
{:odd (+ (:odd m) y)
:even (:even m)}
{:odd (:odd m)
:even (+ (:even m) y)}))
now we can use reduce like this
(reduce odd-and-even {:odd 0 :even 0} [1 2 3 4 5 6 7 8 9 10])
and we get the result
{:odd 25, :even 30}
The execution is like this
(odd-and-even {:odd 0 :even o} 1) -> {:odd 1 :even 0}
(odd-and-even {:odd 1 :even 0} 2) => {:odd 1 :even 2}
(odd-and-even {:odd 1 :even 2} 3) => {:odd 4 :even 2}
(odd-and-even {:odd 4 :even 2) 4) => {:odd 4 :even 6}
....
map applies the function, in this case inc, to the list and returns the result. So its returning a new list which each value incremented by one.
Clojure documentation might be helpful.
There is no function that will print the intermediate values in all cases though there are a couple of tools that may help.
tools.trace helps some print intermediate values such as function calls:
=> (deftrace fubar [x v] (+ x v)) ;; To trace a function call and its return value
=> (fubar 2 3)
TRACE t1107: (fubar 2 3)
TRACE t1107: => 5
5
macroexpand-1 will show you the code generated by macro's such as -> and is essential in learning how to write macros:
=> (macroexpand-1 '(-> 42 inc inc dec inc))
=> (inc (dec (inc (inc 42))))
That second link is talking about reduce not map so the explanation doesn't apply. Map takes a sequence and builds a new sequence by essentially looping through it, calling the function you pass on an element and then adding the result to the list it's building. It iterates down the list until every element has been transformed and included.
reduce which that link is referring to takes an inital value and repeatedly changes that value by calling the function with it and the first value from the sequence, then looping around and calling the function with the updated value and the second item in the list, and then the third and so on until every item in the list has been used to change the value, which it then returns.
I'm reading a book on clojure, and I came by an example that I dont fully understand..
Here is the code in repl:
user=> (repeatedly 10 (rand-int 10))
ClassCastException java.lang.Integer cannot be cast to clojure.lang.IFn clojure.core/repeatedly/fn--4705 (core.clj:4642)
user=> (repeatedly 10 (partial rand-int 10))
(5 0 5 5 2 4 8 8 0 0)
My question is:
why is the partial needed here, and how does that fit into partial definition,
and repeatedly definition & syntax. Partial ...
Takes a function f and fewer than the normal arguments to f, and
returns a fn that takes a variable number of additional args. When
called, the returned function calls f with args + additional args.
So how does this fit in?
Partial is just an easier way of defining an anonymous function that fixes some of the arguments to a function and then passes the rest from the arguments to the created function.
In this case
user> (repeatedly 10 (partial rand-int 10))
(3 1 3 6 1 2 7 1 5 3)
is equivalent to:
user> (repeatedly 10 #(rand-int 10))
(9 5 6 0 0 5 7 6 9 6)
Partial is a misnomer here because partial is being used to fix in advance all the arguments (or rather the only one) to rand-int.
a more intersting use of partial illustrates it's function better:
(partial + 4)
produces the equivalent of:
(fn [& unfixed-args] (apply + (concat [4] unfixed-args)))
(it does not literally produce this)
The idea being to build a function that takes the un-fixed arguments, combines them with the fixed ones, and calls the function you passed to partial with enough arguments to work properly.
user> ((fn [& unfixed-args] (apply + (concat [4] unfixed-args))) 5 6 7 8 9)
39
user> ((partial + 4) 5 6 7 8 9)
39
I only use partial in practice when the number of arguments is variable. Otherwise I have a personal preference towards using the anonymous function reader form #( ... )
partial does not actually check which arities its first argument supports; an arguably more accurate docstring would say that it "takes a function f and some arguments to f". (Clearly if you supply too many arguments, the resulting partially applied function will be broken, though that will only be observable when you try to call it.) So that's why (partial rand-int 10) is ok even though the number of arguments to rand-int supplied is not "fewer than normal".
The reason why either partial or something like #(rand-int 10) is needed here is that repeatedly expects its final argument to be a function which it can repeatedly call, whereas (rand-int 10) would be a number.
Compare this to repeat which returns a sequence with the supplied item repeated the specified number of times (or infinitely many times in the unary case). Here (rand-int 10) would be an appropriate second argument, but of course it would be some particular number, so the result would look like (8 8 8 8 8 ...); repeatedly will make a separate call to (partial rand-int 10) for each item of the sequence returned, so you'll get from it a sequence of (likely different, independent) random numbers.
repeatedly signature we are interested in: (repeatedly number function)
In this case partial will simply wrap rand-int 10 into a function that can be returned and used by the outer function, in this case repeatedly.
Without partial(or #) inner expressions are resolved before the outer ones(there are exceptions, but let's keep it simple for now), so when repeatedly is called without partial, what's gonna be passed to it is a return value of rand-int, that is Int and not a function.
What's the one-level sequence flattening function in Clojure? I am using apply concat for now, but I wonder if there is a built-in function for that, either in standard library or clojure-contrib.
My general first choice is apply concat. Also, don't overlook (for [subcoll coll, item subcoll] item) -- depending on the broader context, this may result in clearer code.
There's no standard function. apply concat is a good solution in many cases. Or you can equivalently use mapcat seq.
The problem with apply concat is that it fails when there is anything other than a collection/sequential is at the first level:
(apply concat [1 [2 3] [4 [5]]])
=> IllegalArgumentException Don't know how to create ISeq from: java.lang.Long...
Hence you may want to do something like:
(defn flatten-one-level [coll]
(mapcat #(if (sequential? %) % [%]) coll))
(flatten-one-level [1 [2 3] [4 [5]]])
=> (1 2 3 4 [5])
As a more general point, the lack of a built-in function should not usually stop you from defining your own :-)
i use apply concat too - i don't think there's anything else in the core.
flatten is multiple levels (and is defined via a tree-walk, not in terms of repeated single level expansion)
see also Clojure: Semi-Flattening a nested Sequence which has a flatten-1 from clojure mvc (and which is much more complex than i expected).
update to clarify laziness:
user=> (take 3 (apply concat (for [i (range 1e6)] (do (print i) [i]))))
012345678910111213141516171819202122232425262728293031(0 1 2)
you can see that it evaluates the argument 32 times - this is chunking for efficiency, and is otherwise lazy (it doesn't evaluate the whole list). for a discussion of chunking see comments at end of http://isti.bitbucket.org/2012/04/01/pipes-clojure-choco-1.html