Goal: Construct a ClojureScript function that takes a string s and returns the unique channel with the name (str s "-chan") (if the channel doesn't exist, then create it). Here is my attempt:
(defn string-channel
[s]
(let [chan-name (symbol (str s "-chan"))]
(defonce chan-name (chan))
chan-name))
This yields an error. How do I accomplish this goal? Note that since I'm in ClojureScript, I am unable to use the eval construct if the solution involves a macro.
I would rather propose to keep these channels in an atom (since defining vars dynamically seems really needless here). In addition, keeping channels in one place seems to me more manageable.
(def channels (atom {}))
(defn string-channel [s]
(when-not (#channels s)
(swap! channels assoc s (chan)))
(#channels s))
It turns out the right approach was to use a defmacro:
(defmacro string-channel
[s]
`(do
(defonce ~(symbol (str s "-chan")) (chan))
~(symbol (str s "-chan"))))
Consider using memoize if the intent is to ensure a unique channel for each label s:
(def unique-channel (memoize (fn [s] (chan))))
Related
Consider a dataset like this:
(def data [{:url "http://www.url1.com" :type :a}
{:url "http://www.url2.com" :type :a}
{:url "http://www.url3.com" :type :a}
{:url "http://www.url4.com" :type :b}])
The contents of those URL's should be requested in parallel. Depending on the item's :type value those contents should be parsed by corresponding functions. The parsing functions return collections, which should be concatenated, once all the responses have arrived.
So let's assume that there are functions parse-a and parse-b, which both return a collection of strings when they are passed a string containing HTML content.
It looks like core.async could be a good tool for this. One could either have separate channels for each item ore one single channel. I'm not sure which way would be preferable here. With several channels one could use transducers for the postprocessing/parsing. There is also a special promise-chan which might be proper here.
Here is a code-sketch, I'm using a callback based HTTP kit function. Unfortunately, I could not find a generic solution inside the go block.
(defn f [data]
(let [chans (map (fn [{:keys [url type]}]
(let [c (promise-chan (map ({:a parse-a :b parse-b} type)))]
(http/get url {} #(put! c %))
c))
data)
result-c (promise-chan)]
(go (put! result-c (concat (<! (nth chans 0))
(<! (nth chans 1))
(<! (nth chans 2))
(<! (nth chans 3)))))
result-c))
The result can be read like so:
(go (prn (<! (f data))))
I'd say that promise-chan does more harm than good here. The problem is that most of core.async API (a/merge, a/reduce etc.) relies on fact that channels will close at some point, promise-chans in turn never close.
So, if sticking with core.async is crucial for you, the better solution will be not to use promise-chan, but ordinary channel instead, which will be closed after first put!:
...
(let [c (chan 1 (map ({:a parse-a :b parse-b} type)))]
(http/get url {} #(do (put! c %) (close! c)))
c)
...
At this point, you're working with closed channels and things become a bit simpler. To collect all values you could do something like this:
;; (go (put! result-c (concat (<! (nth chans 0))
;; (<! (nth chans 1))
;; (<! (nth chans 2))
;; (<! (nth chans 3)))))
;; instead of above, now you can do this:
(->> chans
async/merge
(async/reduce into []))
UPD (below are my personal opinions):
Seems, that using core.async channels as promises (either in form of promise-chan or channel that closes after single put!) is not the best approach. When things grow, it turns out that core.async API overall is (you may have noticed that) not that pleasant as it could be. Also there are several unsupported constructs, that may force you to write less idiomatic code than it could be. In addition, there is no built-in error handling (if error occurs within go-block, go-block will silently return nil) and to address this you'll need to come up with something of your own (reinvent the wheel). Therefore, if you need promises, I'd recommend to use specific library for that, for example manifold or promesa.
I wanted this functionality as well because I really like core.async but I also wanted to use it in certain places like traditional JavaScript promises. I came up with a solution using macros. In the code below, <? is the same thing as <! but it throws if there's an error. It behaves like Promise.all() in that it returns a vector of all the returned values from the channels if they all are successful; otherwise it will return the first error (since <? will cause it to throw that value).
(defmacro <<? [chans]
`(let [res# (atom [])]
(doseq [c# ~chans]
(swap! res# conj (serverless.core.async/<? c#)))
#res#))
If you'd like to see the full context of the function it's located on GitHub. It's heavily inspired from David Nolen's blog post.
Use pipeline-async in async.core to launch asynchronous operations like http/get concurrently while delivering the result in the same order as the input:
(let [result (chan)]
(pipeline-async
20 result
(fn [{:keys [url type]} ch]
(let [parse ({:a parse-a :b parse-b} type)
callback #(put! ch (parse %)(partial close! ch))]
(http/get url {} callback)))
(to-chan data))
result)
if anyone is still looking at this, adding on to the answer by #OlegTheCat:
You can use a separate channel for errors.
(:require [cljs.core.async :as async]
[cljs-http.client :as http])
(:require-macros [cljs.core.async.macros :refer [go]])
(go (as-> [(http/post <url1> <params1>)
(http/post <url2> <params2>)
...]
chans
(async/merge chans (count chans))
(async/reduce conj [] chans)
(async/<! chans)
(<callback> chans)))
First, I have no experience with CS and Clojure is my first language, so pardon if the following problem has a solution, that is immediately apparent for a programmer.
The summary of the question is as follows: one needs to create atoms at will with unknown yet symbols at unknown times. My approach revolves around a) storing temporarily the names of the atoms as strings in an atom itself; b) changing those strings to symbols with a function; c) using a function to add and create new atoms. The problem pertains to step "c": calling the function does not create new atoms, but using its body does create them.
All steps taken in the REPL are below (comments follow code blocks):
user=> (def atom-pool
#_=> (atom ["a1" "a2"]))
#'user/atom-pool
'atom-pool is the atom that stores intermediate to-be atoms as strings.
user=> (defn atom-symbols []
#_=> (mapv symbol (deref atom-pool)))
#'user/atom-symbols
user=> (defmacro populate-atoms []
#_=> (let [qs (vec (remove #(resolve %) (atom-symbols)))]
#_=> `(do ~#(for [s qs]
#_=> `(def ~s (atom #{}))))))
#'user/populate-atoms
'populate-atoms is the macro, that defines those atoms. Note, the purpose of (remove #(resolve %) (atom-symbols)) is to create only yet non-existing atoms. 'atom-symbols reads 'atom-pool and turns its content to symbols.
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(nil nil nil)
Here it is confirmed that there are no 'a1', 'a2', 'a-new' atoms as of yet.
user=> (defn new-atom [a]
#_=> (do
#_=> (swap! atom-pool conj a)
#_=> (populate-atoms)))
#'user/new-atom
'new-atom is the function, that first adds new to-be atom as string to `atom-pool. Then 'populate-atoms creates all the atoms from 'atom-symbols function.
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(#'user/a1 #'user/a2 nil)
Here we see that 'a1 'a2 were created as clojure.lang.Var$Unbound just by defining a function, why?
user=> (new-atom "a-new")
#'user/a2
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(#'user/a1 #'user/a2 nil)
Calling (new-atom "a-new") did not create the 'a-new atom!
user=> (do
#_=> (swap! atom-pool conj "a-new")
#_=> (populate-atoms))
#'user/a-new
user=> (for [s ['a1 'a2 'a-new]]
#_=> (resolve s))
(#'user/a1 #'user/a2 #'user/a-new)
user=>
Here we see that resorting explicitly to 'new-atom's body did create the 'a-new atom. 'a-new is a type of clojure.lang.Atom, but 'a1 and 'a2 were skipped due to already being present in the namespace as clojure.lang.Var$Unbound.
Appreciate any help how to make it work!
EDIT: Note, this is an example. In my project the 'atom-pool is actually a collection of maps (atom with maps). Those maps have keys {:name val}. If a new map is added, then I create a corresponding atom for this map by parsing its :name key.
"The summary of the question is as follows: one needs to create atoms at will with unknown yet symbols at unknown times. "
This sounds like a solution looking for a problem. I would generally suggest you try another way of achieving whatever the actual functionality is without generating vars at runtime, but if you must, you should use intern and leave out the macro stuff.
You cannot solve this with macros since macros are expanded at compile time, meaning that in
(defn new-atom [a]
(do
(swap! atom-pool conj a)
(populate-atoms)))
populate-atoms is expanded only once; when the (defn new-atom ...) form is compiled, but you're attempting to change its expansion when new-atom is called (which necessarily happens later).
#JoostDiepenmaat is right about why populate-atoms is not behaving as expected. You simply cannot do this using macros, and it is generally best to avoid generating vars at runtime. A better solution would be to define your atom-pool as a map of keywords to atoms:
(def atom-pool
(atom {:a1 (atom #{}) :a2 (atom #{})}))
Then you don't need atom-symbols or populate-atoms because you're not dealing with vars at compile-time, but typical data structures at run-time. Your new-atom function could look like this:
(defn new-atom [kw]
(swap! atom-pool assoc kw (atom #{})))
EDIT: If you don't want your new-atom function to override existing atoms which might contain actual data instead of just #{}, you can check first to see if the atom exists in the atom-pool:
(defn new-atom [kw]
(when-not (kw #atom-pool)
(swap! atom-pool assoc kw (atom #{}))))
I've already submitted one answer to this question, and I think that that answer is better, but here is a radically different approach based on eval:
(def atom-pool (atom ["a1" "a2"]))
(defn new-atom! [name]
(load-string (format "(def %s (atom #{}))" name)))
(defn populate-atoms! []
(doseq [x atom-pool]
(new-atom x)))
format builds up a string where %s is substituted with the name you're passing in. load-string reads the resulting string (def "name" (atom #{})) in as a data structure and evals it (this is equivalent to (eval (read-string "(def ...)
Of course, then we're stuck with the problem of only defining atoms that don't already exist. We could change the our new-atom! function to make it so that we only create an atom if it doesn't already exist:
(defn new-atom! [name]
(when-not (resolve (symbol name))
(load-string (format "(def %s (atom #{}))" name name))))
The Clojure community seems to be against using eval in most cases, as it is usually not needed (macros or functions will do what you want in 99% of cases*), and eval can be potentially unsafe, especially if user input is involved -- see Brian Carper's answer to this question.
*After attempting to solve this particular problem using macros, I came to the conclusion that it either cannot be done without relying on eval, or my macro-writing skills just aren't good enough to get the job done with a macro!
At any rate, I still think my other answer is a better solution here -- generally when you're getting way down into the nuts & bolts of writing macros or using eval, there is probably a simpler approach that doesn't involve metaprogramming.
The function below does 2 things -
Checks if the atom is nil or fetch-agin is true, and then fetches the data.
It processes the data by calling (add-date-strings).
What is a better pattern to separate out the above two concerns ?
(def retrieved-data (atom nil))
(defn fetch-it!
[fetch-again?]
(if (or fetch-again?
(nil? #retrieved-data))
(->> (exec-services)
(map #(add-date-strings (:time %)))
(reset! retrieved-data))
#retrieved-data))
One possible refactoring would be:
(def retrieved-data (atom nil))
(defn fetch []
(->> (exec-services)
(map #(add-date-strings (:time %)))))
(defn fetch-it!
([]
(fetch-it! false))
([force]
(if (or force (nil? #retrieved-data))
(reset! retrieved-data (fetch))
#retrieved-data)))
By the way, the pattern to seperate out concerns is called "functions" :)
To really separate the concerns I think it might be better to define a separate fetch and process function. So that in no way they are complected.
(def retrieved-data (atom nil))
(defn fetcher []
(->> (exec-services)
(map #(add-date-strings (:time %)))))
(defn fetch-again? [force]
(fn [data] (or force (nil? data))))
(defn fetch-it! [fetch-fn data fetch-again?]
(when (fetch-again? #data))
(reset! data (fetch-fn))))
;;Usage
(fetch-it! fetcher retrieved-data (fetch-again? true))
Notice that I also gave the data atom as an argument.
I'm trying to figure out the best way to troll a namespace for functions that contain a specific bit of metadata. I've come up with a solution, but it feels a little awkward and I'm not at all sure I'm going about it the right way. There's a second component to this as well: I don't just want the names of the functions, I want to find them and then execute them. Here's a snippet of what I'm doing presently:
(defn wrap-routes
[req from-ns]
(let [publics (ns-publics from-ns)
routes (->>
(keys publics)
(map #(meta (% publics)))
(filter #(= (:route-handler %) true))
(map #(:name %)))
resp (first
(->>
(map #((% publics) req) routes)
(filter #(:status %))))]
(or resp not-found)))
As you can see, I'm doing all sorts of gymnastics to see if my metadata is attached to any functions in a given namespace and then am doing extra work after that to get the actual function back. I'm sure there must be a better way. So my question is, how would you do this?
(defn wrap-routes [req from-ns]
(or (first (filter :status
(for [[name f] (ns-publics from-ns)
:when (:route-handler (meta f))]
(f req))))
not-found))
You can do something like this:
(defn wrap-routes
[req from-ns]
(->> (ns-publics from-ns)
(filter #(:route-handler (meta (%1 1))))
(map #((%1 1) req))
(filter #(:status %))
first
(#(or % not-found))))
I'm attempting to write a macro which will call java setter methods based on the arguments given to it.
So, for example:
(my-macro login-as-fred {"Username" "fred" "Password" "wilma"})
might expand to something like the following:
(doto (new MyClass)
(.setUsername "fred")
(.setPassword "wilma"))
How would you recommend tackling this?
Specifically, I'm having trouble working out the best way to construct the setter method name and have it interpreted it as a symbol by the macro.
The nice thing about macros is you don't actually have to dig into the classes or anything like that. You just have to write code that generates the proper s-expressions.
First a function to generate an s-expression like (.setName 42)
(defn make-call [name val]
(list (symbol (str ".set" name) val)))
then a macro to generate the expressions and plug (~#) them into a doto expression.
(defmacro map-set [class things]
`(doto ~class ~#(map make-call things))
Because it's a macro it never has to know what class the thing it's being called on is or even that the class on which it will be used exists.
Please don't construct s-expressions with list for macros. This will seriously hurt the hygiene of the macro. It is very easy to make a mistake, which is hard to track down. Please use always syntax-quote! Although, this is not a problem in this case, it's good to get into the habit of using only syntax-quote!
Depending on the source of your map, you might also consider to use keywords as keys to make it look more clojure-like. Here is my take:
(defmacro configure
[object options]
`(doto ~object
~#(map (fn [[property value]]
(let [property (name property)
setter (str ".set"
(.toUpperCase (subs property 0 1))
(subs property 1))]
`(~(symbol setter) ~value)))
options)))
This can then be used as:
user=> (macroexpand-1 '(configure (MyClass.) {:username "fred" :password "wilma"}))
(clojure.core/doto (MyClass.) (.setUsername "fred") (.setPassword "wilma"))
Someone (I believe Arthur Ulfeldt) had an answer posted that was almost correct, but it's been deleted now.
This is a working version:
(defmacro set-all [obj m]
`(doto ~obj ~#(map (fn [[k v]]
(list (symbol (str ".set" k)) v))
m)))
user> (macroexpand-1 '(set-all (java.util.Date.) {"Month" 0 "Date" 1 "Year" 2009}))
(clojure.core/doto (java.util.Date.) (.setMonth 0) (.setDate 1) (.setYear 2009))
user> (set-all (java.util.Date.) {"Month" 0 "Date" 1 "Year" 2009})
#<Date Fri Jan 01 14:15:51 PST 3909>
You have to bite the bullet and use clojure.lang.Reflector/invokeInstanceMethod like this:
(defn do-stuff [obj m]
(doseq [[k v] m]
(let [method-name (str "set" k)]
(clojure.lang.Reflector/invokeInstanceMethod
obj
method-name
(into-array Object [v]))))
obj)
(do-stuff (java.util.Date.) {"Month" 2}) ; use it
No need for a macro (as far as I know, a macro would not allow to circumvent reflection, either; at least for the general case).