Suppose I want to define two mutually recursive functions within a local scope. I can do this with letfn:
(letfn
[(f [x] (if (= x 0) (g x) true))
(g [x] (if (= x 1) (f x) false))]
(f 0))
But letfn is quite restricted, compared to let, as it accepts only "function specs," not arbitrary expressions. My question is: what if I want to attach metadata (using with-meta) to both f and g, so that within f, I can read g's metadata, and within g, I can read f's metadata? Is this possible in Clojure?
(For context, I am trying to implement a fn-like macro that automatically attaches certain metadata to the function being created. I'd like these auto-annotated fns to be instantiable wherever a normal Clojure function is, including inside a letfn. But I don't see how I can define a letfn-like macro that attaches the metadata, because it would ultimately have to desugar to letfn, which cannot attach metadata.)
Don't forget about with-local-vars:
(with-local-vars [f (fn [x] (if (= x 0) (g x) true))
g (fn [x] (if (= x 1) (f x) false))]
(reset-meta! f {:f 3})
(reset-meta! g {:g 2})
with results:
(f 0) => false
(f 1) => true
f => #<Var: --unnamed-->
(var-get f) => #object[tst.demo.core$fn__20698$fn__20699 0x1eb2d718 "tst.demo.core$fn__20698$fn__20699#1eb2d718"]
(meta f) => {:f 3}
(meta g) => {:g 2}
You can also use var-get and var-set to access/change the value of the local vars.
I've found the following solution, in which letfn is used to define thunks that evaluate to the actual functions f and g:
(letfn [(f-thunk []
(with-meta (fn f [] (let [g (g-thunk)] (meta g))) {:f 3}))
(g-thunk []
(with-meta (fn g [] (let [f (f-thunk)] (meta f))) {:g 2}))]
(let [f (f-thunk) g (g-thunk)]
[(f) (g)]))
Related
Since one cannot apply a macro to a list, e.g.
;; does not work
(apply -> [expr1 expr2 expr3])
How does one generate such expression:
(-> expr1
expr2
expr3)
Where
expr1 is generated by (generate-expr1 f g h)
expr2 is generated by (generate-expr2 f g h)
expr3 is generated by (generate-expr3 f g h)
Context
I'm trying to design an embedded DSL, e.g.
["increment" "increment" "increment"]
Which is then converted into code, e.g.
(fn [n] (-> n inc inc inc))
You can generate it with a macro e.g.
(defmacro opfun [op-names]
(let [m {"increment" 'inc}
ops (map m op-names)]
`(fn [n#] (-> n# ~#ops))))
then
(opfun ["increment" "increment" "increment"])
if you need to provide your argument list at runtime you can interpret your structure directly e.g.
(defn build-eval [op-names]
(let [m {"increment" inc}
ops (map m op-names)]
(fn [n] (reduce (fn [acc f] (f acc)) n ops))))
I've written a probability function in Clojure that takes an optional hash-map of options:
(defn roll-lte
([n d] (/ n d))
([n d options]
(let [p (/ n d)
roll-type (:type options :normal)]
(cond
(= roll-type :advantage) (- (* p 2) (* p p))
(= roll-type :disadvantage) (* p p)
(= roll-type :normal) p
:else (throw (IllegalArgumentException. "Invalid roll type."))))))
This works as intended, but the idea is to write other functions that build off of this one -- for example:
(defn roll-gte
([n d] (roll-lte (- d n -1) d))
([n d options] (roll-lte (- d n -1) d options)))
The two arities in roll-lte make building off of the function awkward and repetitive, especially in cases like the above where options is simply being forwarded to roll-lte. Is there a more concise and less repetitive way to achieve this?
When I have functions with multiple arities, I usually try to have the lower-arity versions call the higher-arity versions with safe default arguments. The "main" implementation of the function usually ends up being the highest-arity body:
(defn roll-lte
([n d] (roll-lte n d nil))
([n d {:keys [type]
:or {type :normal}}]
(let [p (/ n d)]
(case type ;; used case instead of cond here
:advantage (- (* p 2) (* p p))
:disadvantage (* p p)
:normal p
(throw (IllegalArgumentException. "Invalid roll type."))))))
I also used :or in the options map destructuring above to set the default value for type, which allows the lower-arity functions to just pass a nil options map.
(defn roll-gte
([n d] (roll-gte n d nil))
([n d options] (roll-lte (- d n -1) d options)))
(roll-gte 3 4) ;=> 1/2
(roll-gte 3 4 {:type :advantage}) ;=> 3/4
I like my code to have a "top-down" structure, and that means I want to do exactly the opposite from what is natural in Clojure: functions being defined before they are used. This shouldn't be a problem, though, because I could theoretically declare all my functions first, and just go on and enjoy life. But it seems in practice declare cannot solve every single problem, and I would like to understand what is exactly the reason the following code does not work.
I have two functions, and I want to define a third by composing the two. The following three pieces of code accomplish this:
1
(defn f [x] (* x 3))
(defn g [x] (+ x 5))
(defn mycomp [x] (f (g x)))
(println (mycomp 10))
2
(defn f [x] (* x 3))
(defn g [x] (+ x 5))
(def mycomp (comp f g))
3
(declare f g)
(defn mycomp [x] (f (g x)))
(defn f [x] (* x 3))
(defn g [x] (+ x 5))
But what I would really like to write is
(declare f g)
(def mycomp (comp f g))
(defn f [x] (* x 3))
(defn g [x] (+ x 5))
And that gives me
Exception in thread "main" java.lang.IllegalStateException: Attempting to call unbound fn: #'user/g,
That would mean forward declaring works for many situations, but there are still some cases I can't just declare all my functions and write the code in any way and in whatever order I like. What is the reason for this error? What does forward declaring really allows me to do, and what are the situations I must have the function already defined, such as for using comp in this case? How can I tell when the definition is strictly necessary?
You can accomplish your goal if you take advantage of Clojure's (poorly documented) var behavior:
(declare f g)
(def mycomp (comp #'f #'g))
(defn f [x] (* x 3))
(defn g [x] (+ x 5))
(mycomp 10) => 45
Note that the syntax #'f is just shorthand (technically a "reader macro") that translates into (var f). So you could write this directly:
(def mycomp (comp (var f) (var g)))
and get the same result.
Please see this answer for a more detailed answer on the (mostly hidden) interaction between a Clojure symbol, such as f, and the (anonymous) Clojure var that the symbol points to, namely either #'f or (var f). The var, in turn, then points to a value (such as your function (fn [x] (* x 3)).
When you write an expression like (f 10), there is a 2-step indirection at work. First, the symbol f is "evaluated" to find the associated var, then the var is "evaluated" to find the associated function. Most Clojure users are not really aware that this 2-step process exists, and nearly all of the time we can pretend that there is a direct connection between the symbol f and the function value (fn [x] (* x 3)).
The specific reason your original code doesn't work is that
(declare f g)
creates 2 "empty" vars. Just as (def x) creates an association between the symbol x and an empty var, that is what your declare does. Thus, when the comp function tries to extract the values from f and g, there is nothing present: the vars exist but they are empty.
P.S.
There is an exception to the above. If you have a let form or similar, there is no var involved:
(let [x 5
y (* 2 x) ]
y)
;=> 10
In the let form, there is no var present. Instead, the compiler makes a direct connection between a symbol and its associated value; i.e. x => 5 and y => 10.
I think Alan's answer addresses your questions very well. Your third example works because you aren't passing the functions as arguments to mycomp. I'd reconsider trying to define things in "reverse" order because it works against the basic language design, requires more code, and might be harder for others to understand.
But... just for laughs and to demonstrate what's possible with Clojure macros, here's an alternative (worse) implementation of comp that works for your preferred syntax, without dealing directly in vars:
(defn- comp-fn-arity [variadic? args f & fs] ;; emits a ([x] (f (g x)) like form
(let [args-vec (if variadic?
(into (vec (butlast args)) ['& (last args)])
(apply vector args))
body (reduce #(list %2 %1)
(if variadic?
(apply list 'apply (last fs) args)
(apply list (last fs) args))
(reverse (cons f (butlast fs))))]
`(~args-vec ~body)))
(defmacro momp
([] identity)
([f] f)
([f & fs]
(let [num-arities 5
args-syms (repeatedly num-arities gensym)]
`(fn ~#(map #(apply comp-fn-arity (= % (dec num-arities)) (take % args-syms) f fs)
(range num-arities))))))
This will emit something kinda like comp's implementation:
(macroexpand '(momp f g))
=>
(fn*
([] (f (g)))
([G__1713] (f (g G__1713)))
([G__1713 G__1714] (f (g G__1713 G__1714)))
([G__1713 G__1714 G__1715] (f (g G__1713 G__1714 G__1715)))
([G__1713 G__1714 G__1715 & G__1716] (f (apply g G__1713 G__1714 G__1715 G__1716))))
This works because your (unbound) functions aren't being passed as values to another function; during compilation the macro expands "in place" as if you'd written the composing function by hand, as in your third example.
(declare f g)
(def mycomp (momp f g))
(defn f [x] (* x 3))
(defn g [x] (+ x 5))
(mycomp 10) ;; => 45
(apply (momp vec reverse list) (range 10)) ;; => [9 8 7 6 5 4 3 2 1 0]
This won't work in some other cases, e.g. ((momp - dec) 1) fails because dec gets inlined and doesn't have a 0-arg arity to match the macro's 0-arg arity. Again, this is just for the sake of example and I wouldn't recommend it.
I've been reading SICP and getting into lisps / clojure more and more, and I found myself wondering how apply would actually be implemented. Of course there are some silly ways like (defn apply [f xs] (eval (cons f xs))), but I can't find an example to look at covering the real implementation. I figured once I got to 4.1 in SICP it would be covered, but was disappointed to find out that they define apply in terms of the already existing underlying scheme implementation.
How would one go about implementing this from the ground up?
EDIT:
I think the way I asked this is a bit unclear. I know how apply is implemented in terms of the eval/apply interaction mentioned in SICP. What I'm referring to is the underlying apply in scheme that they fall back on within the definition of the metacircular version of apply. Basically ... how to call a function with a list of args, each passed individually, if you don't already have apply implemented in some base language.
Due to Clojure being hosted on the JVM platform (and being designed to have great Java interop), the peculiarities of the underlying platform shine through.
You can see in the source code for apply on JVM here: https://github.com/clojure/clojure/blob/clojure-1.9.0/src/clj/clojure/core.clj#L652
Notice how there is specific code for arities up to 4, for efficiency reasons.
Arities 5 and above are treated in a less efficient way.
(defn apply
"Applies fn f to the argument list formed by prepending intervening arguments to args."
{:added "1.0"
:static true}
([^clojure.lang.IFn f args]
(. f (applyTo (seq args))))
([^clojure.lang.IFn f x args]
(. f (applyTo (list* x args))))
([^clojure.lang.IFn f x y args]
(. f (applyTo (list* x y args))))
([^clojure.lang.IFn f x y z args]
(. f (applyTo (list* x y z args))))
([^clojure.lang.IFn f a b c d & args]
(. f (applyTo (cons a (cons b (cons c (cons d (spread args)))))))))
The ClojureScript implementation does the same, but looks quite different from the JVM implementation above:
(defn apply
"Applies fn f to the argument list formed by prepending intervening arguments to args."
([f args]
(if (.-cljs$lang$applyTo f)
(let [fixed-arity (.-cljs$lang$maxFixedArity f)
bc (bounded-count (inc fixed-arity) args)]
(if (<= bc fixed-arity)
(apply-to f bc args)
(.cljs$lang$applyTo f args)))
(apply-to-simple f (seq args))))
([f x args]
(if (.-cljs$lang$applyTo f)
(let [arglist (list* x args)
fixed-arity (.-cljs$lang$maxFixedArity f)
bc (inc (bounded-count fixed-arity args))]
(if (<= bc fixed-arity)
(apply-to f bc arglist)
(.cljs$lang$applyTo f arglist)))
(apply-to-simple f x (seq args))))
([f x y args]
(if (.-cljs$lang$applyTo f)
(let [arglist (list* x y args)
fixed-arity (.-cljs$lang$maxFixedArity f)
bc (+ 2 (bounded-count (dec fixed-arity) args))]
(if (<= bc fixed-arity)
(apply-to f bc arglist)
(.cljs$lang$applyTo f arglist)))
(apply-to-simple f x y (seq args))))
([f x y z args]
(if (.-cljs$lang$applyTo f)
(let [arglist (list* x y z args)
fixed-arity (.-cljs$lang$maxFixedArity f)
bc (+ 3 (bounded-count (- fixed-arity 2) args))]
(if (<= bc fixed-arity)
(apply-to f bc arglist)
(.cljs$lang$applyTo f arglist)))
(apply-to-simple f x y z (seq args))))
([f a b c d & args]
(if (.-cljs$lang$applyTo f)
(let [spread-args (spread args)
arglist (cons a (cons b (cons c (cons d spread-args))))
fixed-arity (.-cljs$lang$maxFixedArity f)
bc (+ 4 (bounded-count (- fixed-arity 3) spread-args))]
(if (<= bc fixed-arity)
(apply-to f bc arglist)
(.cljs$lang$applyTo f arglist)))
(apply-to-simple f a b c d (spread args)))))
I made a dynamic lisp language a while ago and I didn't expose apply. I did supply rest arguments and thus since I had eval and macros in the language I made several attempts to do this. I found out quickly that macros are useless so eval is the only solution. Your example has a flaw:
(defn mapply [f xs] (eval (cons f xs)))
(mapply cons '(1 (3)))
; ClassCastException java.lang.Long cannot be cast to clojure.lang.IFn
The reason is that the resulting expression being evaluated by eval becomes:
(cons 1 (3))
Instead of
(cons '1 '(3))
Thus to mimic it you need to make sure the already evaluated values doesn't get evaluates a second time around. We could fix that by quoting the values:
(defn m2apply [f xs] (eval (cons f (map #(list 'quote %) xs))))
(m2apply cons '(1 (3)))
; ==> (1 3)
Yey.. But you really are doing a lot more computing than you need. For a lexical interpreter that does have apply you only need to leak that as a primitive into the environment. And yes, it is the unimpressive apply whose only purpose is to call internals (primitives) and to evaluate user function bodies in an extended environment. In a language not already a lisp the apply and a whole set of primitives and data structures would be implemented in the implementation language and it would just expose that instead.
The way you implement apply is directly tied to how you implement function calls. If you compile your code, you have a protocol at runtime where you know how values are exchanged between function calls, and apply can emit code that satisfy to this protocol. We could do the same in a quick and dirty interpreter. Let's define a package:
(defpackage :interpreter (:use :cl))
(in-package :interpreter)
We define a function object, which has an optional name, a list of parameters, the code as well as a set of bindings being closed-over:
(defstruct fn name parameters code closed)
We also define a frame, which has a set of bindings and an optional parent frame:
(defstruct frame bindings parent)
Here we have a simple interpreter, and we put the current frame within the evaluation environment:
(defstruct env frame)
Bindings are either objects of type FN, or cons pairs. We write generic functions to manipulate them with a uniform API. Functions and variables share the same namespace:
(defgeneric name (object)
(:method ((fn fn)) (fn-name fn))
(:method ((pair cons)) (car pair)))
(defgeneric value (object)
(:method ((c cons)) (cdr c))
(:method ((fn fn)) fn))
We define two functions, my-apply and my-eval
(declaim (ftype function my-apply my-eval))
There is a global environment, which is simply:
(defparameter *global-frame*
(make-frame
:bindings (list (make-fn :name '+
:parameters '(x y)
;; built-in
:code (lambda (x y) (+ x y)))
(make-fn :name 'addition
:parameters '(x y)
:code '(+ x y)))
:parent nil))
The empty environment implicitly holds to the global frame:
(defgeneric frame (env)
(:method ((empty null)) *global-frame*)
(:method ((env env)) (env-frame env)))
Resolving a binding involves visiting parent frames:
(defun resolve (name frame &optional (on-error :error))
(labels ((recurse (frame)
(cond
(frame (or (find name (frame-bindings frame) :key #'name)
(recurse (frame-parent frame))))
((eql :error on-error) (error "Unknown: ~a" name)))))
(recurse frame)))
The evaluation function is the following one:
(defun my-eval (code env &aux (frame (frame env)))
(flet ((ev (exp) (my-eval exp env)))
(typecase code
(symbol (value (resolve code frame)))
(atom code)
(cons
(destructuring-bind (head . tail) code
(case head
(list (mapcar #'ev tail))
(let (destructuring-bind ((var val) expr) tail
(my-eval expr
(make-env :frame (make-frame :bindings `((,var . ,(ev val)))
:parent frame)))))
(thunk (make-fn :name nil
:parameters nil
:code (first tail)
:closed (frame-bindings frame)))
(apply (my-apply (ev (first tail))
(ev (second tail))
env))
(t (my-apply (resolve head (frame env))
(mapcar #'ev tail)
env))))))))
The evaluation functions accept the following terms:
(list <...>) builds a list containing the result of evaluation of its arguments
(apply <fn-expr> <arg-expr>), evaluate all arguments and call the my-apply primitive.
(let (<var> <val>) <expr>), local binding
(thunk <expr>) closes over current environment and produce an anonymous closure with no parameters which returns the value of <expr>
(<f> . <args>) function call
symbols are resolved for values, and other values are returned as-is.
The built-in my-apply knows how to bind parameters to values dynamically:
(defun my-apply (fn arguments env)
(assert (= (length arguments)
(length (fn-parameters fn)))
()
"Length mismatch when calling ~S with argsuments ~S"
fn
arguments)
(let ((code (fn-code fn)))
(typecase code
(function (apply code arguments))
(t (my-eval code
(make-env :frame
(make-frame :bindings (append (fn-closed fn)
(mapcar #'cons
(fn-parameters fn)
arguments))
:parent (frame env))))))))
For example:
(my-eval '(let (f (let (x 10) (thunk (addition x 5))))
(let (x 20) (apply f (list)))) nil)
=> 15
In the above example, f is a function that closes over the binding of x to 10, and calls addition. The binding that is made later is not seen by the closure. The call to apply resolves f and builds an empty list. The call to addition resolves to (+ 10 5), which itself eventually calls the CL function +. You can (trace my-eval) to see how things are evaluated. The above code is a bit messy.
I don't think you can define it from the ground up in the language: at some point your language needs a mechanism of actually calling a function on a bunch of arguments, and apply is pretty much that point.
That's why it's a primitive: asking how you implement apply is like asking how you implement cons or +: sooner or later the thing needs to bottom out and you call a function which is not defined in the language, or is only partly defined in the language: + for instance can probably be partly implemented in terms of checking types and extracting the actual machine numbers from them, but sooner or later you are going to ask the machine to add some machine numbers for you (or, OK, some equivalent operation if your machine does not support addition directly).
Sometimes I want to pass argument-value pairs to a higher-order function, where the value I should pass is determined by the argument I pass. I want to be able to pass the argument without explicitly specifying the accompanying value. In particular, I'm interested in the case where the argument is itself a function.
Generic Example:
Here's a very generic example, where my-foo and my-bar are functions that I'm passing to higher-foo:
(higher-foo my-foo :option4 args) ;good
(higher-foo my-bar :option13 args) ;good
(higher-foo my-foo :option13 args) ;how stupid are you?! my-foo requires :option4!
Question: Is there a "standard" method for making :option4 or :option13 to be inferable by higher-foo so that I can just write (higher-foo my-foo) and (higher-foo my-bar)?
More Specific Example:
Bear in mind that there are better alternatives to the following code, but I'm just trying to put forward a concrete example of what I'm talking about:
(defn seq-has? [f n someseq]
(every? (partial apply f)
(partition n 1 someseq)))
(defn monotonicity [a b]
(<= a b))
(defn generalized-fib [a b c]
(= c (+ a b)))
(seq-has? monotonicity 2 someseq) should return true if the sequence is monotonic, false otherwise. (seq-has? generalized-fib 3 someseq) should return true if the sequence follows the generalized Fibonacci form, false otherwise.
But the "2" and "3" bother me. I could have an arbitrary number of properties to test for, and I don't want to have to remember the appropriate "magic numbers" for such calls.
Note: I know of two ways to do this, and for my own personal use, I suppose they both work. But I'm interested in what is idiomatic or considered best practice in the community. I'll post my answers, but I'm hoping there are more solutions.
Just make the predicate function itself take variadic arguments, and have it do the partitioning / recurring. Your monotonic? for instance already exists in core, and is called <=
(<= 1 2 4 5)
=> true
(<= 1 2 1 5)
=> false
Here's the source for the 1, 2 and variadic arg versions:
(source <=)
(defn <=
"Returns non-nil if nums are in monotonically non-decreasing order,
otherwise false."
{:inline (fn [x y] `(. clojure.lang.Numbers (lte ~x ~y)))
:inline-arities #{2}
:added "1.0"}
([x] true)
([x y] (. clojure.lang.Numbers (lte x y)))
([x y & more]
(if (<= x y)
(if (next more)
(recur y (first more) (next more))
(<= y (first more)))
false)))
You can make a fib? work the same way, have it take variadic arguments and recur over triples:
(defn fib?
[a b & [c & r]]
(if (= c (+ a b))
(if r
(recur b c r)
true)
false))
(fib? 0 1 1)
=> true
(fib? 2 3 5 8 13)
=> true
Since you are asking for a standard way how a function determines a not passed argument from one argument:
(defn f
([arg0] (case a :foo (f a :bar)
:baz (f a :quux)))
([arg0 arg1] ...))
Depending on your use case a different dispatch construct than case may be a better fit.
For your generic example this implies that higher-foo should determine the correct :option in the desired overload like demonstrated above.
In your specific example, you can't determine the n from the passed function. You need a more specific datastructure:
(defn seq-has? [{:keys [f n]} s]
(every? (partial apply f)
(partition n 1 s)))
(def monotonicity
{:f <=
:n 2})
(def generalized-fib
{:f #(= (+ %1 %2) %3)
:n 3})
(seq-has? monotonicity [1 2 3])
;; => true
This solution seems like a hack to me. Is it considered common/idiomatic? Use meta-data on the functions that define the property you are looking for:
(defn higher-foo [foo & args]
(apply foo (:option (meta foo))
args))
(def my-foo
(with-meta
(fn [a b] (println "I'm doing something cool"))
{:option :option4}))
;using it:
user=> (higher-foo my-foo arg)