I'm running into some limitations of Clojure macros. I wonder how to optimize the following code?
(defmacro ssplit-7-inefficient [x]
(let [t 7]
;; Duplicated computation here!
`(do [(first (split-with #(not (= '~t %)) '~x))
(drop 1 (second (split-with #(not (= '~t %)) '~x)))])))
(ssplit-7-inefficient (foo 7 bar baz))
;; Returns: [(foo) (bar baz)]
Here are some approaches that don't work:
(defmacro ssplit-7-fails [x]
(let [t 7]
`(do ((fn [[a b]] [a (drop 1 b)]) (split-with #(not (= '~t %)) '~x)))))
(ssplit-7-fails (foo 7 bar baz))
;; Error: Call to clojure.core/fn did not conform to spec.
(defmacro ssplit-7-fails-again [x]
(let [t 7]
`(do
(let [data (split-with #(not (= '~t %)) '~x)]
((fn [[a b]] [a (drop 1 b)]) data)))))
(ssplit-7-fails-again (foo 7 bar baz))
;; Error: Call to clojure.core/let did not conform to spec.
Note that split-with splits only once. You can use some destructuring to get what you want:
(defmacro split-by-7 [arg]
`((fn [[x# [_# & z#]]] [x# z#]) (split-with (complement #{7}) '~arg)))
(split-by-7 (foo 7 bar baz))
=> [(foo) (bar baz)]
In other use cases, partition-by can be also useful:
(defmacro split-by-7 [arg]
`(->> (partition-by #{7} '~arg)
(remove #{[7]})))
(split-by-7 (foo 7 bar baz))
=> ((foo) (bar baz))
It is not so easy to reason about macros in Clojure - (in my view macroexpand-1 alienates the code a lot - in contrast to Common Lisp's macroexpand-1 ...).
My way was first to build a helper function.
(defn %split-7 [x]
(let [y 7]
(let [[a b] (split-with #(not= y %) x)]
[a (drop 1 b)])))
This function uses destructuring so that the split-with is "efficient".
It does nearly exactly what the macro should do. Just that one has to quote
the argument - so that it works.
(%split-7 '(a 7 b c))
;;=> [(a) (b c)]
From this step to the macro is not difficult.
The macro should just automatically quote the argument when inserting into the helper function's call.
(defmacro split-7 [x]
`(%split-7 '~x))
So that we can call:
(split-7 (a 7 b c))
;; => [(a) (b c)]
Using this trick, even generalize the function to:
(defn %split-by [x y]able like this
(let [[a b] (split-with #(not= y %) x)]
[a (drop 1 b)]))
(defmacro split-by [x y]
`(%split-by '~x ~y))
(split-by (a 7 b c) 7)
;; => [(a) (b c)]
(split-by (a 7 b c 9 d e) 9)
;; => [(a 7 b c) (d e)]
The use of (helper) functions in the macro body - and even other macros - or recursive functions or recursive macros - macros which call other macros - shows how powerful lisp macros are. Because it shows that you can use the entirety of lisp when formulating/defining macros. Something what most language's macros usually aren't able to do.
Related
I have been writing a fairly simple piece of code to get the hang of Clojure and I've run into an issue where when I pass each line into a REPL in order (while using a test case to substitute the values that would be passed as part of the function), I get the expected result but when I try to compile it as a function I get the error Execution error (ClassCastException) at testenv.core/sum-of-two-largest-squares (core.clj:14). class clojure.lang.PersistentList cannot be cast to class clojure.lang.IFn (clojure.lang.PersistentList and clojure.lang.IFn are in unnamed module of loader 'app')
The relevant function is as follows (note that I've moved each step into variables in order to figure out the problem)
(defn sum-of-two-largest-squares [a b c]
(
(def x (into () (map math/abs [a b c])))
(def m (apply min x))
(def y (into () (map square (remove (fn [n] (= n m)) x))))
(apply + y)
)
)
You can't just put parenthesis around things without expecting it to change the meaning.
What works when run in a REPL is:
(defn abs [n] (java.lang.Math/abs n))
(defn square [n] (* n n))
(def x (into () (map abs [a b c])))
(def m (apply min x))
(def y (into () (map square (remove (fn [n] (= n m)) x))))
(apply + y)
...and strictly, this still works if injected into a function, as long as you take out the extra parenthesis (though it works slowly and with unwanted side effects due to the inappropriate use of def):
(defn sum-of-two-largest-squares [a b c]
(def x (into () (map abs [a b c])))
(def m (apply min x))
(def y (into () (map square (remove (fn [n] (= n m)) x))))
(apply + y)
)
(sum-of-two-largest-squares a b c)
However, a good-practice alternative would use let instead:
(defn abs [n] (java.lang.Math/abs n))
(defn square [n] (* n n))
(defn sum-of-two-largest-squares [a b c]
(let [x (into () (map abs [a b c]))
m (apply min x)
y (into () (map square (remove (fn [n] (= n m)) x)))]
(apply + y)))
Here is a more typical solution to this problem
(ns tst.demo.core
(:use tupelo.core tupelo.test))
(defn square [x] (* x x))
(defn sum-of-two-largest-squares [a b c]
(let-spy
[sorted (sort [a b c])
largest-two (rest sorted)
squares (mapv square largest-two)
result (apply + squares)]
result))
(dotest
(is= 41 (spyx (sum-of-two-largest-squares 3 4 5)))
)
with result:
-----------------------------------
Clojure 1.10.3 Java 15.0.2
-----------------------------------
Testing tst.demo.core
sorted => (3 4 5)
largest-two => (4 5)
squares => [16 25]
result => 41
(sum-of-two-largest-squares 3 4 5) => 41
Ran 2 tests containing 0 assertions.
0 failures, 0 errors.
It uses my favorite template project. Just change let-spy back to let once finished writing/debugging.
I have a file which contains some trusted clojure source code:
((+ a b) (* a b) (- a b))
For each of the items in the list I want to generate an anonymous function:
(fn [a b] (+ a b))
(fn [a b] (* a b))
(fn [a b] (- a b))
If I call the following marco
(defmacro create-fn
[args exprs]
`(fn ~args ~exprs))
directly with some clojure code it works perfectly:
user=> (macroexpand-1 '(create-fn [a b] (* a b)))
(clojure.core/fn [a b] (* a b))
But when I bind the context of the file to a local and try to map my macro it will not work. On access of the first generated function I get the error message "java.lang.RuntimeException: Unable to resolve symbol: a in this context"
(Please note that I had to put an extra eval into the macro to get the value of the symbol e which is used in the anonymous function used by map)
(defmacro create-fn
[args exprs]
`(let [e# (eval ~exprs)]
(fn ~args
e#)))
(let [exprs (read-string "((+ a b) (* a b) (- a b))")
fns (map
(fn [e] (create-fn [a b] e))
exprs)]
(first fns))
Any help is very much appreciated!
Let's look at the whole code after macro expansion. This code:
(let [exprs (read-string "((+ a b) (* a b) (- a b))")
fns (map
(fn [e] (create-fn [a b] e))
exprs)]
(first fns))
Expands to this, where e__900__auto__ is the symbol generated by e#:
(let [exprs (read-string "((+ a b) (* a b) (- a b))")
fns (map
(fn [e] (let [e__900__auto__ (eval e)]
(fn [a b] e__900__auto__))
exprs)]
(first fns))
Why doesn't this work? Well, one reason is that a and b aren't even in the scope of (eval e). You might be tempted to try this next:
(defmacro create-fn [args exprs] `(fn ~args (eval ~exprs)))
After expansion, the generated function looks like this:
(let [exprs (read-string "((+ a b) (* a b) (- a b))")
fns (map
(fn [e] (fn [a b] (eval e)))
exprs)]
(first fns))
This looks good, but it won't work because eval evaluates in an empty lexical environment. In other words, eval won't see a and b even with this code.
You could ditch the macro and just manually mangle the code into something you can eval, like this:
(map
(fn [e] (eval (concat '(fn [a b]) (list e))))
exprs)
Alternatively, you could declare the variables a and b as dynamic and then use binding to set them before evaluating the expressions.
(declare ^:dynamic a ^:dynamic b)
(let [exprs (read-string "((+ a b) (* a b) (- a b))")
fns (map
(fn [e] (fn [a1 b1] (binding [a a1 b b1] (eval e))))
exprs)]
(first fns))
If you do not want to have a and b in your namespace, you could set up another namespace and evaluate the code there.
My suggested solutions do not use macros. They're not useful here, because macros are expanded at compile time, but the expressions are read at runtime. If you really do want to use macros here, you'll need to move the read-string and file handling code inside the defmacro.
I have a sequence s and a list of indexes into this sequence indexes. How do I retain only the items given via the indexes?
Simple example:
(filter-by-index '(a b c d e f g) '(0 2 3 4)) ; => (a c d e)
My usecase:
(filter-by-index '(c c# d d# e f f# g g# a a# b) '(0 2 4 5 7 9 11)) ; => (c d e f g a b)
You can use keep-indexed:
(defn filter-by-index [coll idxs]
(keep-indexed #(when ((set idxs) %1) %2)
coll))
Another version using explicit recur and lazy-seq:
(defn filter-by-index [coll idxs]
(lazy-seq
(when-let [idx (first idxs)]
(if (zero? idx)
(cons (first coll)
(filter-by-index (rest coll) (rest (map dec idxs))))
(filter-by-index (drop idx coll)
(map #(- % idx) idxs))))))
make a list of vectors containing the items combined with the indexes,
(def with-indexes (map #(vector %1 %2 ) ['a 'b 'c 'd 'e 'f] (range)))
#'clojure.core/with-indexes
with-indexes
([a 0] [b 1] [c 2] [d 3] [e 4] [f 5])
filter this list
lojure.core=> (def filtered (filter #(#{1 3 5 7} (second % )) with-indexes))
#'clojure.core/filtered
clojure.core=> filtered
([b 1] [d 3] [f 5])
then remove the indexes.
clojure.core=> (map first filtered)
(b d f)
then we thread it together with the "thread last" macro
(defn filter-by-index [coll idxs]
(->> coll
(map #(vector %1 %2)(range))
(filter #(idxs (first %)))
(map second)))
clojure.core=> (filter-by-index ['a 'b 'c 'd 'e 'f 'g] #{2 3 1 6})
(b c d g)
The moral of the story is, break it into small independent parts, test them, then compose them into a working function.
The easiest solution is to use map:
(defn filter-by-index [coll idx]
(map (partial nth coll) idx))
I like Jonas's answer, but neither version will work well for an infinite sequence of indices: the first tries to create an infinite set, and the latter runs into a stack overflow by layering too many unrealized lazy sequences on top of each other. To avoid both problems you have to do slightly more manual work:
(defn filter-by-index [coll idxs]
((fn helper [coll idxs offset]
(lazy-seq
(when-let [idx (first idxs)]
(if (= idx offset)
(cons (first coll)
(helper (rest coll) (rest idxs) (inc offset)))
(helper (rest coll) idxs (inc offset))))))
coll idxs 0))
With this version, both coll and idxs can be infinite and you will still have no problems:
user> (nth (filter-by-index (range) (iterate #(+ 2 %) 0)) 1e6)
2000000
Edit: not trying to single out Jonas's answer: none of the other solutions work for infinite index sequences, which is why I felt a solution that does is needed.
I had a similar use case and came up with another easy solution. This one expects vectors.
I've changed the function name to match other similar clojure functions.
(defn select-indices [coll indices]
(reverse (vals (select-keys coll indices))))
(defn filter-by-index [seq idxs]
(let [idxs (into #{} idxs)]
(reduce (fn [h [char idx]]
(if (contains? idxs idx)
(conj h char) h))
[] (partition 2 (interleave seq (iterate inc 0))))))
(filter-by-index [\a \b \c \d \e \f \g] [0 2 3 4])
=>[\a \c \d \e]
=> (defn filter-by-index [src indexes]
(reduce (fn [a i] (conj a (nth src i))) [] indexes))
=> (filter-by-index '(a b c d e f g) '(0 2 3 4))
[a c d e]
I know this is not what was asked, but after reading these answers, I realized in my own personal use case, what I actually wanted was basically filtering by a mask.
So here was my take. Hopefully this will help someone else.
(defn filter-by-mask [coll mask]
(filter some? (map #(if %1 %2) mask coll)))
(defn make-errors-mask [coll]
(map #(nil? (:error %)) coll))
Usage
(let [v [{} {:error 3} {:ok 2} {:error 4 :yea 7}]
data ["one" "two" "three" "four"]
mask (make-errors-mask v)]
(filter-by-mask data mask))
; ==> ("one" "three")
I know that the -> form can be used to pass the results of one function result to another:
(f1 (f2 (f3 x)))
(-> x f3 f2 f1) ; equivalent to the line above
(taken from the excellent Clojure tutorial at ociweb)
However this form requires that you know the functions you want to use at design time. I'd like to do the same thing, but at run time with a list of arbitrary functions.
I've written this looping function that does it, but I have a feeling there's a better way:
(defn pipe [initialData, functions]
(loop [
frontFunc (first functions)
restFuncs (rest functions)
data initialData ]
(if frontFunc
(recur (first restFuncs) (rest restFuncs) (frontFunc data) )
data )
) )
What's the best way to go about this?
I must admit I'm really new to clojure and I might be missing the point here completely, but can't this just be done using comp and apply?
user> (defn fn1 [x] (+ 2 x))
user> (defn fn2 [x] (/ x 3))
user> (defn fn3 [x] (* 1.2 x))
user> (defn pipe [initial-data my-functions] ((apply comp my-functions) initial-data))
user> (pipe 2 [fn1 fn2 fn3])
2.8
You can do this with a plain old reduce:
(defn pipe [x fs] (reduce (fn [acc f] (f acc)) x fs))
That can be shortened to:
(defn pipe [x fs] (reduce #(%2 %1) x fs))
Used like this:
user> (pipe [1 2 3] [#(conj % 77) rest reverse (partial map inc) vec])
[78 4 3]
If functions is a sequence of functions, you can reduce it using comp to get a composed function. At a REPL:
user> (def functions (list #(* % 5) #(+ % 1) #(/ % 3)))
#'user/my-list
user> ((reduce comp functions) 9)
20
apply also works in this case because comp takes a variable number of arguments:
user> (def functions (list #(* % 5) #(+ % 1) #(/ % 3)))
#'user/my-list
user> ((apply comp functions) 9)
20
As a neophyte clojurian, it was recommended to me that I go through the Project Euler problems as a way to learn the language. Its definitely a great way to improve your skills and gain confidence. I just finished up my answer to problem #14. It works fine, but to get it running efficiently I had to implement some memoization. I couldn't use the prepackaged memoize function because of the way my code was structured, and I think it was a good experience to roll my own anyways. My question is if there is a good way to encapsulate my cache within the function itself, or if I have to define an external cache like I have done. Also, any tips to make my code more idiomatic would be appreciated.
(use 'clojure.test)
(def mem (atom {}))
(with-test
(defn chain-length
([x] (chain-length x x 0))
([start-val x c]
(if-let [e (last(find #mem x))]
(let [ret (+ c e)]
(swap! mem assoc start-val ret)
ret)
(if (<= x 1)
(let [ret (+ c 1)]
(swap! mem assoc start-val ret)
ret)
(if (even? x)
(recur start-val (/ x 2) (+ c 1))
(recur start-val (+ 1 (* x 3)) (+ c 1)))))))
(is (= 10 (chain-length 13))))
(with-test
(defn longest-chain
([] (longest-chain 2 0 0))
([c max start-num]
(if (>= c 1000000)
start-num
(let [l (chain-length c)]
(if (> l max)
(recur (+ 1 c) l c)
(recur (+ 1 c) max start-num))))))
(is (= 837799 (longest-chain))))
Since you want the cache to be shared between all invocations of chain-length, you would write chain-length as (let [mem (atom {})] (defn chain-length ...)) so that it would only be visible to chain-length.
In this case, since the longest chain is sufficiently small, you could define chain-length using the naive recursive method and use Clojure's builtin memoize function on that.
Here's an idiomatic(?) version using plain old memoize.
(def chain-length
(memoize
(fn [n]
(cond
(== n 1) 1
(even? n) (inc (chain-length (/ n 2)))
:else (inc (chain-length (inc (* 3 n))))))))
(defn longest-chain [start end]
(reduce (fn [x y]
(if (> (second x) (second y)) x y))
(for [n (range start (inc end))]
[n (chain-length n)])))
If you have an urge to use recur, consider map or reduce first. They often do what you want, and sometimes do it better/faster, since they take advantage of chunked seqs.
(inc x) is like (+ 1 x), but inc is about twice as fast.
You can capture the surrounding environment in a clojure :
(defn my-memoize [f]
(let [cache (atom {})]
(fn [x]
(let [cy (get #cache x)]
(if (nil? cy)
(let [fx (f x)]
(reset! cache (assoc #cache x fx)) fx) cy)))))
(defn mul2 [x] (do (print "Hello") (* 2 x)))
(def mmul2 (my-memoize mul2))
user=> (mmul2 2)
Hello4
user=> (mmul2 2)
4
You see the mul2 funciton is only called once.
So the 'cache' is captured by the clojure and can be used to store the values.