I am trying to convert heavily nested expressions in Clojure.
One example is the following:
(distinct (flatten (map keys (flatten (filter vector? (vals data))))))
The threaded form would be:
(->> data vals (filter vector?) flatten (map keys) flatten distinct)
Is it possible in Clojure to create a function or macro that help me automate getting the threaded form with the nested form as input? Or are there any third-party tools that I can use?
If you're using CIDER, I would recommend clj-refactor. It has refactoring capability for both -> and ->>, as well as a whole bunch of other stuff.
Related
I find myself writing a lot of clojure in this manner:
(defn my-fun [input]
(let [result1 (some-complicated-procedure input)
result2 (some-other-procedure result1)]
(do-something-with-results result1 result2)))
This let statement seems very... imperative. Which I don't like. In principal, I could be writing the same function like this:
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
The problem with this is that it involves recomputation of some-complicated-procedure, which may be arbitrarily expensive. Also you can imagine that some-complicated-procedure is actually a series of nested function calls, and then I either have to write a whole new function, or risk that changes in the first invocation don't get applied to the second:
E.g. this works, but I have to have an extra shallow, top-level function that makes it hard to do a mental stack trace:
(defn some-complicated-procedure [input] (lots (of (nested (operations input)))))
(defn my-fun [input]
(do-something-with-results (some-complicated-procedure input)
(some-other-procedure (some-complicated-procedure input)))))
E.g. this is dangerous because refactoring is hard:
(defn my-fun [input]
(do-something-with-results (lots (of (nested (operations (mistake input))))) ; oops made a change here that wasn't applied to the other nested calls
(some-other-procedure (lots (of (nested (operations input))))))))
Given these tradeoffs, I feel like I don't have any alternatives to writing long, imperative let statements, but when I do, I cant shake the feeling that I'm not writing idiomatic clojure. Is there a way I can address the computation and code cleanliness problems raised above and write idiomatic clojure? Are imperitive-ish let statements idiomatic?
The kind of let statements you describe might remind you of imperative code, but there is nothing imperative about them. Haskell has similar statements for binding names to values within bodies, too.
If your situation really needs a bigger hammer, there are some bigger hammers that you can either use or take for inspiration. The following two libraries offer some kind of binding form (akin to let) with a localized memoization of results, so as to perform only the necessary steps and reuse their results if needed again: Plumatic Plumbing, specifically the Graph part; and Zach Tellman's Manifold, whose let-flow form furthermore orchestrates asynchronous steps to wait for the necessary inputs to become available, and to run in parallel when possible. Even if you decide to maintain your present course, their docs make good reading, and the code of Manifold itself is educational.
I recently had this same question when I looked at this code I wrote
(let [user-symbols (map :symbol states)
duplicates (for [[id freq] (frequencies user-symbols) :when (> freq 1)] id)]
(do-something-with duplicates))
You'll note that map and for are lazy and will not be executed until do-something-with is executed. It's also possible that not all (or even not any) of the states will be mapped or the frequencies calculated. It depends on what do-something-with actually requests of the sequence returned by for. This is very much functional and idiomatic functional programming.
i guess the simplest approach to keep it functional would be to have a pass-through state to accumulate the intermediate results. something like this:
(defn with-state [res-key f state]
(assoc state res-key (f state)))
user> (with-state :res (comp inc :init) {:init 10})
;;=> {:init 10, :res 11}
so you can move on to something like this:
(->> {:init 100}
(with-state :inc'd (comp inc :init))
(with-state :inc-doubled (comp (partial * 2) :inc'd))
(with-state :inc-doubled-squared (comp #(* % %) :inc-doubled))
(with-state :summarized (fn [st] (apply + (vals st)))))
;;=> {:init 100,
;; :inc'd 101,
;; :inc-doubled 202,
;; :inc-doubled-squared 40804,
;; :summarized 41207}
The let form is a perfectly functional construct and can be seen as syntactic sugar for calls to anonymous functions. We can easily write a recursive macro to implement our own version of let:
(defmacro my-let [bindings body]
(if (empty? bindings)
body
`((fn [~(first bindings)]
(my-let ~(rest (rest bindings)) ~body))
~(second bindings))))
Here is an example of calling it:
(my-let [a 3
b (+ a 1)]
(* a b))
;; => 12
And here is a macroexpand-all called on the above expression, that reveal how we implement my-let using anonymous functions:
(clojure.walk/macroexpand-all '(my-let [a 3
b (+ a 1)]
(* a b)))
;; => ((fn* ([a] ((fn* ([b] (* a b))) (+ a 1)))) 3)
Note that the expansion doesn't rely on let and that the bound symbols become parameter names in the anonymous functions.
As others write, let is actually perfectly functional, but at times it can feel imperative. It's better to become fully comfortable with it.
You might, however, want to kick the tires of my little library tl;dr that lets you write code like for example
(compute
(+ a b c)
where
a (f b)
c (+ 100 b))
I'm trying to create a function that applies several processes to a map, including adding / updating some standard items to each map using "conj". I'm doing it by composing several other functions using "comp".
So I tried doing this
(defn everything [extra] (comp (partial conj {:data extra}) another-func) )
Which won't work because conj wants the extra data as the second argument, not the first.
I assume there should be a similarly straightforward way of composing a curried conj, but I can't quite figure out how to do it.
Easiest is just to write an anonymous function:
(defn everything [extra]
(comp #(conj % {:data extra}) another-func))
I'm inspired by clojure's 1.5 cond-> macro.
Similarily, I want to create a macro of the same idea, applied to the function map. However, I have no idea where to start.
For example, I can't find the source for cond->. (probably because it's not released yet)
Any suggestions?
There is the source of cond-> https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L6742
there are a variety of threading macros from the pallet project folks including apply-map-> which looks close to, though not exactly what you are looking for.
(letfn [(apply-map-
[arg f arg-coll]
`(let [arg# ~arg]
(apply ~f arg#
~#(butlast arg-coll)
(apply concat ~(last arg-coll)))))]
(defmacro apply-map->
"Apply in a threaded expression.
e.g.
(-> :a
(apply-map-> hash-map 1 {:b 2}))
=> {:a 1 :b 2}"
[arg f & arg-coll]
(apply-map- arg f arg-coll))
Perhaps there will be enough examples there for you to pick out what you need.
If I understand -- you want to write a macro that takes a list of partial function calls, and for each one, adds map (or apply map) to the beginning, and the previous result to the end?
While this doesn't directly answer how to write that macro, I wanted to point out that you have a couple of alternatives.
Factor out map
This is always true for pure functions:
(=
(map g (map f coll))
(map (comp g f) coll))
The refactored version only walks the collection once, and no intermediate collections need to be made.
Here's what it looks like with threading:
(=
(->> coll
(map f)
(map g))
(map #(->> % f g) coll))
Here's a concrete example in JS.
Transducers
Transducers are another pattern for doing this kind of thing in Clojure that work on more than just map. They're sort of an abstraction over reducer functions. Clojure's map / filter / reduce (etc.) will create a transducer if called without a collection. You can chain them with comp and use them in various contexts (lazy, eager, observable, whatever). Rich Hickey's talk on them is a good intro.
In Clojure, there are several option for composition of functions. There are composition functions for:
Apply: for 'unwrapping' arguments
Partial: for arguments that are not yet given
Comp: for piping consecutive results through multiple functions
Juxt: for applying one argument on multiple functions
However, AFAIK there are no such composition functions that include branching. Are there any functions that compose functions in a branching way, like a functional version of if or cond ?
Of course an if version is easy to make (though this implementation might not be the quickest):
(defn iff
([pred rtrue] (iff pred rtrue identity))
([pred rtrue rfalse]
(fn [& args]
(if (apply pred args)
(apply rtrue args)
(apply rfalse args)))))
There could be discussion about by default returning identity in the 'else' case is the right choice, or if nil should be returned in such case.
The use of such function could produce more easy to read code. Instead of #(if (string? %) (trim %) %) it would become (iff string? trim), or with a cond version:
(condf string? trim,
vector? (partial apply str),
:else identity)
Do other FP languages have such constructs ? I can imagine it might be handy in compositions with comp and juxt. Why doesn't Clojure ?
Bonus points for nice iff / condf implementations :)
I'm not sure if this is a direct match for what you're looking for (the question, to me, is somewhat vague), but you should look into Monads and Arrows.
Monads allow you to chain together functions with a specific "bind" function that defines how to chain them. It could do some sort of if/else pipelining, as in the Maybe and Either monads, or it could simulate state, as in the State monad.
Monads are built into Haskell (as monads) and F# (as "Workflows"). I have seen monad libraries for Clojure (check this out for one), and there are probably Arrow libraries too.
Well there could be many such composition pattern you can come up and ask why this isn't in the core language. The reason is obvious, it is not feasible. The core of the language provide you all the constructs to build such patterns. These sort of features are more of a contrib kind of thing rather than core of the language.
As far as implementation is concerned it would as simple as something shown below:
(defn condf [& args]
(let [chain (partition 2 args)]
(fn [& params]
(first (for [[p f] chain :when (or (= :else p) (apply p params))]
(apply f params))))))
(def my-func (condf string? clojure.string/trim
vector? (partial apply str)
:else identity))
(my-func "Ankur ") ==> "Ankur"
(my-func [1 2 3]) ==> "123"
(my-func '(1 2 3)) ==> (1 2 3)
This approaches the idea of Strategic Programming. You may find the following paper of interest
The Essence of Strategic Programming by
Ralf Lämmel and Eelco Visser and Joost Visser
http://homepages.cwi.nl/~ralf/eosp/
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.20.1969
I have a java.util.HashMap object m (a return value from a call to Java code) and I'd like to get a new map with an additional key-value pair.
If m were a Clojure map, I could use:
(assoc m "key" "value")
But trying that on a HashMap gives:
java.lang.ClassCastException: java.util.HashMap cannot be cast to clojure.lang.Associative
No luck with seq either:
(assoc (seq m) "key" "value")
java.lang.ClassCastException: clojure.lang.IteratorSeq cannot be cast to clojure.lang.Associative
The only way I managed to do it was to use HashMap's own put, but that returns void so I have to explicitly return m:
(do (. m put "key" "value") m)
This is not idiomatic Clojure code, plus I'm modifying m instead of creating a new map.
How to work with a HashMap in a more Clojure-ish way?
Clojure makes the java Collections seq-able, so you can directly use the Clojure sequence functions on the java.util.HashMap.
But assoc expects a clojure.lang.Associative so you'll have to first convert the java.util.HashMap to that:
(assoc (zipmap (.keySet m) (.values m)) "key" "value")
Edit: simpler solution:
(assoc (into {} m) "key" "value")
If you're interfacing with Java code, you might have to bite the bullet and do it the Java way, using .put. This is not necessarily a mortal sin; Clojure gives you things like do and . specifically so you can work with Java code easily.
assoc only works on Clojure data structures because a lot of work has gone into making it very cheap to create new (immutable) copies of them with slight alterations. Java HashMaps are not intended to work in the same way. You'd have to keep cloning them every time you make an alteration, which may be expensive.
If you really want to get out of Java mutation-land (e.g. maybe you're keeping these HashMaps around for a long time and don't want Java calls all over the place, or you need to serialize them via print and read, or you want to work with them in a thread-safe way using the Clojure STM) you can convert between Java HashMaps and Clojure hash-maps easily enough, because Clojure data structures implement the right Java interfaces so they can talk to each other.
user> (java.util.HashMap. {:foo :bar})
#<HashMap {:foo=:bar}>
user> (into {} (java.util.HashMap. {:foo :bar}))
{:foo :bar}
If you want a do-like thing that returns the object you're working on once you're done working on it, you can use doto. In fact, a Java HashMap is used as the example in the official documentation for this function, which is another indication that it's not the end of the world if you use Java objects (judiciously).
clojure.core/doto
([x & forms])
Macro
Evaluates x then calls all of the methods and functions with the
value of x supplied at the front of the given arguments. The forms
are evaluated in order. Returns x.
(doto (new java.util.HashMap) (.put "a" 1) (.put "b" 2))
Some possible strategies:
Limit your mutation and side-effects to a single function if you can. If your function always returns the same value given the same inputs, it can do whatever it wants internally. Sometimes mutating an array or map is the most efficient or easiest way to implement an algorithm. You will still enjoy the benefits of functional programming as long as you don't "leak" side-effects to the rest of the world.
If your objects are going to be around for a while or they need to play nicely with other Clojure code, try to get them into Clojure data structures as soon as you can, and cast them back into Java HashMaps at the last second (when feeding them back to Java).
It's totally OK to use the java hash map in the traditional way.
(do (. m put "key" "value") m)
This is not idiomatic Clojure code, plus I'm modifying m instead of creating a new map.
You are modifying a data structure that really is intended to be modified. Java's hash map lacks the structural sharing that allows Clojures map's to be efficiently copied. The generally idiomatic way of doing this is to use java-interop functions to work with the java structures in the typical java way, or to cleanly convert them into Clojure structures and work with them in the functional Clojure way. Unless of course it makes life easier and results in better code; then all bets are off.
This is some code I wrote using hashmaps when I was trying to compare memory characteristics of the clojure version vs java's (but used from clojure)
(import '(java.util Hashtable))
(defn frequencies2 [coll]
(let [mydict (new Hashtable)]
(reduce (fn [counts x]
(let [y (.toLowerCase x)]
(if (.get mydict y)
(.put mydict y (+ (.get mydict y) 1))
(.put mydict y 1)))) coll) mydict))
This is to take some collection and return how many times each different thing (say a word in a string) is reused.