In the project I'm working on we often define custom defsomething-style macros for different purposes to hide boilerplate. One example is defhook which helps to define a hook handler for an event. Here's a simplified version of it (the actual version has more parameters and does some non-trivial things in defmethod, but that's irrelevant to my question):
(defmulti handle-hook
"This multimethod is called when an event was fired."
(fn [event context] event))
(defmacro defhook
"Define a hook for an event."
[event docstring & more]
`(let [body# (fn ~#more)]
(defmethod handle-hook ~event [event# context#]
(body# context#))))
(defhook "EntryDeleted"
"Hook called on entry deletion."
[context]
(log-deletion (:EntryID context)))
The main problem I have with this code is that defmethod does not support docstring, so I can't use the one for "EntryDeleted" in REPL or for automatic documentation generation. The last one is important for the project: there are defhooks and defhandlers that are exposed as external API and currently we have to maintain documentation separately (and manually).
So the simplest question is "how to attach docstring to a defmethod"?.
And the deeper one would be "how to attach/generate documentation for custom defsomething macros?"
If some of the existing tools for documentation generation supported this feature it would be great! Yet, neither of Marginalia, Codox or Autodoc seem to support something like that.
How to attach/generate documentation for custom defsomething macros?
Since docstrings are attached to vars, you'd generally have defsomething macros expand into a more primitive def form, e.g. defn, def. Then you just arrange for your defsomething's docstring to be attached to the underlying var.
How to attach docstring to a defmethod?
This is a special case - defmethod is not defining a new var; it's calling a Java method on a Java object. On the other hand, defmulti does create a var. One idea would be to extend the multi-function's docstring with the dispatch value and associated description. For example,
(defn append-hook-doc! [event docstring]
(let [hook-doc (str event " - " docstring)]
(alter-meta! #'handle-hook
(fn [m]
(update-in m [:doc] #(str % "\n\t" hook-doc))))))
...
(doc handle-hook)
-------------------------
user/handle-hook
This multimethod is called when an event was fired.
EntryDeleted - Hook called on entry deletion.
As the ! indicates, this form has a side-effect: multiple evaluations of a defining form that calls this will result in duplicate lines in #'handle-hook's docstring. You might avoid this by stashing some extra metadata in #'handle-hook to use as a marker for whether or not the doc has already been appended. Alternatively, you might stash the docstrings elsewhere and patch it all together in some auxiliary step, e.g. by delaying the expansion of (defmulti handle-hook ... until you have all the docstrings (although, this breaks the open extension of multi-methods wrt docstrings).
Related
(defn my-func [opts]
(assoc opts :something :else))
What i want to be able to do, is serialize a reference to the function (maybe via #'my-func ?) to a string in such a way that i can upon deserializing it, invoke it with args.
How does this work?
Edit-- Why This is Not a Duplicate
The other question asked how to serialize a function body-- the entire function code. I am not asking how to do that. I am asking how to serialize a reference.
Imagine a cluster of servers all running the same jar, attached to a MQ. The MQ pubs in fn-reference and fn-args for functions in the jar, and the server in the cluster runs it and acks it. That's what i'm trying to do-- not pass function bodies around.
In some ways, this is like building a "serverless" engine in clojure.
Weirdly, a commit for serializing var identity was just added to Clojure yesterday: https://github.com/clojure/clojure/commit/a26dfc1390c53ca10dba750b8d5e6b93e846c067
So as of the latest master snapshot version, you can serialize a Var (like #'clojure.core/conj) and deserialize it on another JVM with access to the same loaded code, and invoke it.
(import [java.io File FileOutputStream FileInputStream ObjectOutputStream ObjectInputStream])
(defn write-obj [o f]
(let [oos (ObjectOutputStream. (FileOutputStream. (File. f)))]
(.writeObject oos o)
(.close oos)))
(defn read-obj [f]
(let [ois (ObjectInputStream. (FileInputStream. (File. f)))
o (.readObject ois)]
(.close ois)
o))
;; in one JVM
(write-obj #'clojure.core/conj "var.ser")
;; in another JVM
(read-obj "var.ser")
As suggested on the comments, if you can just serialize a keyword label for the function and store/retrieve that, you are finished.
If you need to transmit the function from one place to another, you essentially need to send the function source code as a string and then have it compiled via eval on the other end. This is what Datomic does when a Database Function is stored in the DB and automatically run by Datomic for any new additions/changes to the DB (these can perform automatic data validation, for example). See:
http://docs.datomic.com/database-functions.html
http://docs.datomic.com/clojure/index.html#datomic.api/function
As similar technique is used in the book Clojure in Action (1st Edition) for the distributed compute engine example using RabbitMQ.
I'm creating a library for an API server, here's a simpliefied version of I have:
(defonce ^:dynamic *my-token* nil)
(defmacro def-my-token
[token1 & body]
`(binding [*my-token* ~token1] ~#body))
And the main "post" method:
(defn my-post-request [url1]
(try
(let [res (client/post (str "api/url/base" url1)
{:body (json/write-str (:secret my-token)) ; my-token should come from the macro
;......
And here's how I want to use it:
(defn -main [& args]
(def-my-token "fdsfdsfdsfds"
; now "my-token" should be created and visible in "my-post-request", shouldn't it?
(print
(my-post-request "/some_end_point"))))
But it says "Unable to resolve symbol: my-token in this context"
I wonder why? doens't def-my-token, being a macros, define it? why not? And how to fix that?
UPDATE:
Also without (defonce ^:dynamic *token* nil) it doesn't work. Why not?
Why isn't defining the macro enough?
Answer to your UPDATE:
According to the documentation for binding, you can only override already existing vars. That's why your solution doesn't work without establishing a root binding to your dynamic var.
Sidenote:
I would recommend doing what jmargolisvt said and use a plain def instead of defonce, as I've never seen any dynamic var definition in the wild using defonce.
EDIT:
doens't def-my-token, being a macros, define it? why not? And how to fix that?
Macros by themselves don't define things, they are small programs transforming your source code in the macro-expansion step of most Lisp REPL's. It could define anything you want it to, but then you should've wrote the def special form. What you used instead was binding which deals with already existing vars.
You might get more insight by toying with it in the REPL and/or reading the answer of this stackoverflow answer.
If you need some further explanation why overriding is needed:
It's practical to conceptualize vars as stacks. The root binding that you establish with using def is the first layer. Everything in your program will see this value unless you put "something" over it. As you can imagine in your example having *my-token* seen as nil from your functions would cause issues.
binding to the resuce!
It allows you put anything "on top" of the root binding (in your case nil) thread-locally inside of the body it, like so:
you bound *my-token*, not my-token. Try:
{:body (json/write-str (:secret *my-token*))
The asterisks are just a naming convention for dynamic vars, they are still part of the actual var name.
I am currenlty trying to provide a DSL by using Clojure macros. The users of my library must be able to extend it using that DSL. The library provides a multi-method and some baseline implementations using defmethod for some certain commands out of the box. Let's say, we have a multimethod:
(defmulti command command-name)
and some provided commands by library like:
(defmethod command "say-hello" [arg] (println "hello" arg))
(defmethod command "say-bye" [arg] (println "bye" arg))
The idea is to write a macro which generates these defmethods, so the users don't need to write defmethods themselves, instead, use my DSL. Regarding to this, I wrote the following macro:
(defmacro add-command [command-name command-impl]
`(defmethod command ~command-name '[arg] ~#command-impl))
What I get is, whenever I use my macro with, (add-command "new-command" (print "new-command")) the following exception "java.lang.IllegalArgumentException: Parameter declaration do should be a vector".
If I expand the macro:
(clojure.core/defmethod com.foo/on-error "new-command" [] (println "starting"))
Everything looks OK, except the argument vector, which is empty after expanding.
What am I missing?
If you want the implicit arg to exist in the body you need to write:
(defmacro add-command [command-name command-impl]
`(defmethod command ~command-name [~'arg] ~#command-impl))
instead. Anaphoric macros (macros that intoduce fixed names like arg here) are generally unusual/frowned upon in Clojure.
Please note that macroexpanding your example I get a different (and expected) result from yours:
(clojure.core/defmethod user/command "new-command" (quote [user/arg]) print "new-command")
Perhaps a possible solution to use (<! c) outside go macro could be done with macro and its macro expansion time :
This is my example:
(ns fourclojure.asynco
(require [clojure.core.async :as async :refer :all]))
(defmacro runtime--fn [the-fn the-value]
`(~the-fn ~the-value)
)
(defmacro call-fn [ the-fn]
`(runtime--fn ~the-fn (<! my-chan))
)
(def my-chan (chan))
(defn read-channel [the-fn]
(go
(loop []
(call-fn the-fn)
(recur)
)
))
(defn paint []
(put! my-chan "paint!")
)
And to test it:
(read-channel print)
(repeatedly 50 paint)
I've tried this solution in a nested go and also works. But I'm not sure if it could be a correct path
The reason about this question is releated to this other question Isn't core.async contrary to Clojure principles?, #aeuhuea comment that "It seems to me that this prevents simplicity and composability. Why is it not a problem?" and #cgrand response "The limitation of the go macro (its locality) is also a feature: it enforces source code locality of stateful operations."
But force to localize your code is not the same as "complect"?
Regarding the title of your question:
>!must be called in a go block because it's designed to. If you are interested in the go-block state-machine mechanics, I can highly recommend Timothy Baldridges Youtube videos on that http://www.youtube.com/channel/UCLxWPHbkxjR-G-y6CVoEHOw
Remember that there is always blocking take and put >!! and <!!. I don't know which part of your code is supposed to provide a "solution" for not being able to use <! and >! outside of a go block, however looping around events dispatched from a single channel is common practice. Here is a modified version of read-channel
(defn do-channel [f ch]
(go-loop []
(when-let [v (<! ch)]
(f v)
(recur))))
put! puts asynchronously, an effect that you usually don't intend. In your example, to put the string "paint" into the channel 50 times, I'd recommend a one-liner like this one:
(do-channel println (to-chan (repeat 50 "print")))
Here is a comment as an answer to your edit:
Channels are not designed to be used as mutable data-structures, period. They have a buffer and that buffer can be thought of as a mutable queue. However we don't use channels to store a value in there, just to take it out a few lines later again.
We use channels as helping construct that may be used to bring execution of two or more different pieces of source-code in two or more different places in line. E.g. a go-block here does not continue to execute until it has received a value produced by another go-block. >! and >!! help us to distinguish whether they are used in a thread-blocking context or in a go-block (blocking a spawned process).
Also, please refer to this answer: Clojure - Why does execution hang when doing blocking insert into channel? (core.async)
You should not use >!! or <!! inside of a go-block, neither transparently or nested in a function call. Rich Hickey himself has commented on that in a recent bug report (http://dev.clojure.org/jira/browse/ASYNC-29?focusedCommentId=32414&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-32414).
Looking at the source-code of >! you will see that it only throws an exception. As a matter of fact, go will replace >! with different source-code. go spawns a state-machine controlled process. Depending on the context you may want to make this explicitly known or nest the go block inside of a macro or function (like in the code examples that you have provided).
Regarding David Nolens (swannodettes) helpers: They have been implemented by Rich Hickey and Nolen himself into the core.async library. Nolen said himself that they are superseded in this presentation (http://www.youtube.com/watch?v=AhxcGGeh5ho). Notice that go-loop has been implemented since after Nolens commit.
I have this function which load correctly my namespace :
(defn load-module [module-name]
(use module-name)
)
And my "equivalent" macro that doesn't work :
(defmacro load-module-macro [module-name]
`(
(use '~module-name)
)
)
I don't understand the problem.
Moreover, I want to use this macro for load a module choose in configuration. In my config.clj I define a var with the namespace of my logger module which contains "save-data" function. Then I want to load the specified logger in my core program. So I can choose the logger to use directly in my configuration file (logger on disk, logger in database...). Is it the best way to do that ?
EDIT :
Error message
IllegalArgumentException Don't know how to create ISeq from: java.lang.Character clojure.lang.RT.seqFrom (RT.java:505)
No, in fact you don't want to use "use" directly in code at all. Use modifies the entire namespace it is called in and that could break your code in ways that are hardly predictable.
Instead what you should do is:
Implement a logging interface (Protocol), write a "meta-constructor" that dispatches whatever you set in config.clj as keyword. Code example
(defprotocol ILog
(save-data [this msg] "Logs message in msg."))
(defn create-file-log
"Returns an object implementing ILog, opens and flushes java.io.File file."
[file]
(let [f ... ;; create file writer here
]
(reify ILog
(save-data [this msg] ;; Write code that writes data to file here
))))
;; create other implementations like database here or elsewhere
(defn create-log
"Creates a log of of the type passed in type-kw."
[type-kw]
(case type-kw
:file (create-file-log "./app-log.txt")
;; other types
))
Now you would simply invoke create-log with whatever keyword is set in your config file and pass the returned object around to functions that need to do logging. Obviously, you could also def it as a global object but I don't recommend to do that.
Eventually you don't just want to set a keyword (type-kw) for the desired logging method in your config, but also other parameters like the file-name or a database uri so that you can pass something like
{:log-method :file
:data {:fname "app-log.txt"}}
or
{:log-method :db
:data {:uri "....
...to your create-log function that uses this structure to get the parameters for the reify constructors create-file-log, create-db-log, etc.
EDIT:
Because you don't like the switch statement, here is how to do it with multi-methods:
(defmulti create-log :logging-method)
(defmethod create-log :file
[arg-map]
(let [file (java.io.File. (:fname arg-map))]
(if (.exists file)
...
Then you simply have an entry in your config.clj
{...
:log {:logging-method :file
:fname "./log-file.txt"}}
To create a new logging type all you have to do now is to imagine an argument map like the one above and a constructor method for create-log.
I think you can also use multimethods for this. It's clojure's other polymorphism strategy, and might work well for your use case as you're only looking to implement a single method (save-data).
;; set up a config map
(def config {:logger :db-logger)
;; set up the dispatch function to read the logger from the config map
(defmulti save-data (fn [] (:logger config))
;; define methods as required - database logging
(defmethod save-data :db-logger []
(println "Save to database"))
;; some other logging - can be in another file
(defmethod save-data :other-logger []
(println "Save to other thing"))
Note: I'm still quite new to Clojure - so I'm not sure if this is a 'proper' way to use multimethods. Most of the examples I've seen dispatch on the type of the arguments, not on a config setting. Any experts, please correct me if I've got the wrong idea.