I'm trying to implement a (very) simple actor model prototype in clojure. Each actor would have, as the loop to handle the arrival of messages, a go-loop. I wanted to invoke, inside this go-loop, functions that would either take from or put into a channel. I really wanted parking behaviour on these inner channel operations, but I can't actually use <! or >! to do it so, because the operations wouldn't be directly inside the go-loop. Can I use macros to circumvent such limitation? If so, how would I do it?
To illustrate the issue simply, in the following snippet, how can I change foo so that the code actually compiles and I get parking behaviour?
(defn main-loop
[ch]
(async/go-loop [] (foo ch) (recur)))
(defn foo [ch] (>! ch "Hello, World!"))
You can use macros to write any code that you could write by hand, and that's all. So if you just want your functions to be convenient shorthand for using >! inline, then sure, you can replace the functions with macros. If you want to do something fancier, like map that function over a list of messages, then you can't, because that would involve bundling them up in a function to pass to map.
A simple example would just be
(defmacro foo [ch]
`(>! ~ch "Hello, World!"))
Whether this is the solution to your problem, or something else is (such as using >!!), depends on what you want to be able to do with these things.
As Carcigenicate said, the >! form must be inside a go loop as it is just a giant macro that does code generation/reformating. Note also, the >! cannot be inside a function called within the go loop, since the go macro can't see inside the called function to do the necessary reformatting.
The >!! form does briefly use a java thread & locks to do the necessary synchronization, but this is a microscopic cost that you shouldn't worry about.
For convenience, I like to think of the async functions as follows:
>! => "put-go" (must be in a go loop - parking semantics)
>!! => "put-now" (can be anywhere - blocking/immediate effect)
put! => "put-later" (can be anywhere - non-blocking/delayed effect)
You can see some examples of these functions in action in these unit tests.
Related
I'm learning Clojure via a pet project. The project would consist of several workers that would be called from other functions.
Each worker is defined in their own namespace as a set of functions (currently two: get-data for gathering data and write-data for writing the gathered data into a file).
In order to make the code a bit DRYer, I decided to write a macro that would gather functions from namespace into a map that can be passed around:
(ns clojure-bgproc.workers)
(defmacro gen-worker-info []
(let [get-data (ns-resolve *ns* 'get-data)
write-data (ns-resolve *ns* 'write-data)]
`(def ~(quote worker-info)
{:get-data ~get-data
:write-data ~write-data}
)
)
)
In my worker code, I use my macro (code abridged for clarity):
(ns clojure-bgproc.workers.summary
(:require [clojure-bgproc.workers :refer [gen-worker-info]]))
(defn get-data [params]
<...>
)
(defn write-data [data file]
;; <...>
)
(gen-worker-info)
While it does work (I get my get-data and write-data functions in clojure-bgproc.workers.summary/worker-info, I find it a bit icky, especially since, if I move the macro call to the top of the file, it doesn't work.
My question is, is there a more idiomatic way to do so? Is this idiomatic Clojure at all?
Thank you.
I think you're in a weird spot because you've structured your program wrong:
Each worker is defined in their own namespace as a set of functions
This is the real problem. Namespaces are a good place to put functions and values that you will refer to in hand-written code. For stuff you want to access programmatically, they are not a good storage space. Instead, make the data you want to access first-class by putting it into an ordinary proper data structure, and then it's easy to manipulate.
For example, this worker-info map you're thinking of deriving from the namespace is great! In fact, that should be the only way workers are represented: as a map with keys for the worker's functions. Then you just define somewhere a list (or vector, or map) of such worker maps, and that's your list of workers. No messing about with namespaces needed.
My go-to solution for defining the workers would be Protocols. I would also apply some of the well-tried frameworks for system lifecycle management.
Protocols provide a way of defining a set of methods and their signatures. You may think of them as similar, but more flexible than, interfaces in object-oriented programming.
Your workers will probably have some state and life-cycle, e.g., the workers may be running or stopped, acquiring and releasing a resource, and so on. I suggest you take a look at Integrant for managing a system with stateful components (i.e., workers).
I would argue for avoiding macros in this case. The adage data over functions over macros seems to apply here. Macros are not available at runtime, make debugging harder, and force all other programmers who look at your code to learn a new Domain-Specific Language, i.e., the one you defined with your macros.
I'm trying to advise a number of methods in one library with utility functions from another library, where some of the methods to be advised are defined with (defn) and some are defined with (defprotocol).
Right now I'm using this library, which uses (alter-var-root). I don't care which library I use (or whether I hand-roll my own).
The problem I'm running into right now is that protocol methods sometimes can be advised, and sometimes cannot, depending on factors that are not perfectly clear to me.
If I define a protocol, then define a type and implement that protocol in-line, then advising never seems to work. I am assuming this is because the type extends the JVM interface directly and skips the vars.
If, in a single namespace, I define a protocol, then advise its methods, and then extend the protocol to a type, the advising will not work.
If, in a single namespace, I define a protocol, then extend the protocol to a type, then advise the protocol's methods, the advising will work.
What I would like to do is find a method of advising that works reliably and does not rely on undefined implementation details. Is this possible?
Clojure itself doesn't provide any possibilities to advice functions in a reliable way, even those defined via def/defn. Consider the following example:
(require '[richelieu.core :as advice])
(advice/defadvice add-one [f x] (inc (f x)))
(defn func-1 [x] x)
(def func-2 func-1)
(advice/advise-var #'func-1 add-one)
> (func-1 0)
1
> (func-2 0)
0
After evaluation of the form (def func-2 func-1), var func-2 will contain binding of var func-1 (in other words its value), so advice-var won't affect it.
Eventhough, definitions like func-2 are rare, you may have noticed or used the following:
(defn generic-function [generic-parameter x y z]
...)
(def specific-function-1 (partial generic-function <specific-arg-1>))
(def specific-function-2 (partial generic-function <specific-arg-2>))
...
If you advice generic-function, none of specific functions will work as expected due to peculiarity described above.
If advising is critical for you, as a solution that may work, I'd suppose the following: since Clojure functions are compiled to java classes, you may try to replace java method invoke with other method that had desired behaviour (however, things become more complicated when talking about replacing protocol/interface methods: seems that you'll have to replace needed method in every class that implements particular protocol/interface).
Otherwise, you'll need explicit wrapper for every function that you want to advice. Macros may help to reduce boilerplate in this case.
Is is good practice to use Clojure nested let in the following way, or is it confusing ?
(defn a-fun [config]
(let [config (-> config (parse) (supply-defaults))]
;; do something with config
))
I noticed I have this pattern of parsing/checking/validating things quite often in my input functions that talk to the external world (in this case a Clojurescript library that exposes public functions, but I also had Compojure routes with this same feeling).
Is it confusing, because one has to understand the rules for bindings visibility (not sure what the exact wording is) ?
What would be the idiomatic way to do it ? Change the config name to parsed-config, put it in another function, something else completely ?
I would reach for this idiom when
the rebinding is the same kind of thing and
you want to make clear that the local binding supersedes the
global one.
For example
(defn fact [n]
(loop [n n, answer 1]
(if (pos? n)
(recur (dec n) (* answer n))
answer)))
This also stops you using the global binding by accident, as I was prone to do.
#Thumbnail's answer is good, but I personally would almost never shadow an outer binding with an inner one in this way. Even if you understand binding rules, and want to shadow an outer variable for a good reason, it's confusing for someone reading the code--which could very well be you, later, after you've forgotten how the code works.
Suppose I have a complex function, and I see the variable foo used somewhere in the middle of it. I look up and see a binding for it--perhaps as a function parameter, which would be obvious and easy to notice. If I don't notice that somewhere below that, the name was rebound, then I will misunderstand what's in the variable.
So I usually make up new, related names that correspond to the role of the different variables in the code. Sometimes the name differences are somewhat arbitrary.
I think these are good reasons not to shadow variables, and I think #Thumbnail gives go reasons to go ahead and shadow them. There are tradeoffs, and you have to decide what's best for your situation.
Short functions are probably better contexts for shadowing. Personally, I'd add a very noticeable comment if I did this sort of thing, or if I was doing it over and over again, maybe a very noticeable comment near the top of the file.
EDIT: As nha's comment made me realize, it can be more reasonable to shadow variables when the new binding occurs immediately after the previous binding; that makes it hard to miss the fact that the name is being redefined.
Another option would be to slightly rename the argument, keeping the general name for the "final" version of the data:
(defn a-fun [config-in]
(let [config (-> config-in (parse) (supply-defaults))]
;; do something with config
))
I also sometimes use the suffixes -arg, -orig, etc to differentiate various stages of processing.
I spent a couple of years doing Scheme "back in the day" and am now learning Clojure. One of the "best practices" in Scheme was to define helper functions within the parent function thus limiting their visibility from "outside." Of course back then TDD wasn't done (or known!) so testing such functions wasn't an issue.
I'm still tempted to structure Clojure functions this way; i.e., using letfn to bind helper functions within the main function. Of course testing such "local" functions is problematic. I realize I can define "private" functions, but this scopes the visibility to the namespace which helps, but is not as fine grained. If you come upon a letfn within another function it's pretty clear that the function is not available for general use.
So, my question is, can one test such local functions and if so, how? If not, then is there some convention to aid in code reading so that it's clear that a function has only one caller?
TIA,
Bill
The usual approach is to just put the functions in the namespace.
One option is using metadata:
user=> (defn ^{::square #(* % %)} cube [x]
#_=> (* x ((::square (meta #'cube)) x)))
#'user/cube
user=> (meta #'cube)
{…, :user/square #<user$fn__780 user$fn__780#2e62c3f9>}
user=> (cube 3)
27
It is of course possible to write a macro to make this prettier.
I'm using a 3rd-party library (clj-msgpack), and wish to extend a protocol for a type which the library also provides a handler for.
On its own, this is simple enough -- but is there any way to do this which wouldn't impact other users of this library running inside the same JVM? Something similar to a dynamic var binding (only taking effect under a given point on the stack) would be ideal.
At present, I'm doing an unconditional override but using a dynamic var to enable my modified behavior; however, this feels far too much like monkey-patching for my comfort.
For the curious, the (admitted abomination) I'm putting into place follows:
(in-ns 'clj-msgpack.core)
(def ^:dynamic *keywordize-strings*
"Assume that any string starting with a colon should be unpacked to a keyword"
false)
(extend-protocol Unwrapable
RawValue
(unwrap [o]
(let [v (.getString o)]
(if (and *keywordize-strings* (.startsWith v ":"))
(keyword (.substring v 1))
v))))
After some thought I see two basic approches (one of which I get from you):
Dynamic binding (as you are doing it now):
Some complain that dynamic binding holds to the principal of most supprise; "what? is behaves this way only when called from there?". While I don't personally hold to this being a bad-thing(tm) some people do. In this case it exacly matches your desire and so long as you have one point where you decide if you want keywordized-strings this should work. If you add a second point that changes them back and a code path that crosses the two well... your on your own. But hey, working code has it's merits.
Inheritance:
good'ol java style or using clojure's add-hoc heirarchies you could extend the type of object you are passing around to be keywordized-string-widgewhatzit that extends widgewhatzit and add a new handler for your specific subclass. This only works in some cases and forces a different object style on the rest of the design. Some smart people will also argue that it still follows the principal of most surprise because the type of the objects will be different when called via another code path.
Personally I would go with your existing solution unless you can change your whole program to use keywords instead of strings (which would of course be my first (potentially controversial) choice)