Say I wanted to factor out some common code between my client-side *.cljs and my server-side *.clj, e.g. various data structures and common operations, can I do that ? Does it make sense to do it ?
I wrote the cljx Leiningen plugin specifically to handle Clojure/ClojureScript code sharing for a Clojure data visualization library.
95% of non-host-interop code looks the same, and cljx lets you automatically rewrite that last 5% by specifying rewrite rules using core.logic.
Most of the time, though, it's simple symbol substitutions; clojure.lang.IFn in Clojure is just IFn in ClojureScript, for instance.
You can also use metadata to annotate forms to be included or excluded when code is generated for a specific platform.
Update: as of clojure 1.7, check out Clojure reader conditionals or cljc. I've used cljc with great success to share a lot of code between server and browser very easily.
Great question! I've been thinking a lot about this as well lately and have written a few apps to experiment.
Here's my list of what types things you might want to share and pros/cons of each:
Most of my client cljs files contains code that manipulates the dom. So, it wouldn't make sense to share any of that with server
Most of the server side stuff deals with filesystem and database calls. I suppose you might want to call the database from the client (especially if you're using one of the no-sql db's that support javascript calls). But, even then, I feel like you should choose to either call db from client or call db from server and, therefore, it doesn't make much sense to share the db code either.
One area where sharing is definitely valuable is being able to share and pass clojure data structures (nested combinations of lists, vectors, sets, etc) between client and server. No need to convert to json (or xml) and back. For example, being able to pass hiccup-style representations of the dom back and forth is very convenient. In gwt, I've used gilead to share models between client and server. But, in clojure, you can simply pass data structures around, so there's really no need to share class definitions like in gwt.
One area that I feel I need to experiment more is sharing state between client and server. In my mind there are a few strategies: store state on client (single page ajax type applications) or store state on server (like legacy jsp apps) or a combo of both. Perhaps the code responsible for updating state (the atoms, refs, agents or whatever) could be shared and then state could be passed back and forth over request and response to keep the two tiers in synch? So far, simply writing server using REST best practices and then having state stored on client seems to work pretty well. But I could see how there might be benefits to sharing state between client and server.
I haven't needed to share Constants and/or Properties yet, but this might be something that would be good to reuse. If you put all your app's global constants in a clj file and then wrote a script to copy it over to cljs whenever you compiled the clojurescript, that should work fine, and might save a bit of duplication of code.
Hope these thoughts are useful, I'm very interested in what others have found so far!
The new lein-cljsbuild plugin for Leiningen has built-in support for sharing pure Clojure code.
Wrote a quick bit of code to copy a subset of my server clojure code over to my clojurescript code, renaming as .cljs before building:
(ns clj-cljs.build
(use
[clojure.java.io]
)
(require
[cljs.closure :as cljsc]
)
)
(defn list-files [path]
(.listFiles (as-file path))
)
(defn copy-file* [from to]
;(println " coping " from " to " to)
(make-parents to)
(copy from to)
)
(defn rename [to-path common-path f]
(str to-path common-path (.replaceAll (.getName f) ".clj" ".cljs"))
)
(defn clj-cljs* [files common-path to-path]
(doseq [i (filter #(.endsWith (.getName %) ".clj") files)]
(copy-file* i (file (rename to-path common-path i)))
)
(doseq [i (filter #(.isDirectory %) files)]
(clj-cljs* (list-files i) (str common-path (.getName i) "/") to-path)
)
)
(defn build [{:keys [common-path clj-path cljs-path js-path module-name]}]
(clj-cljs* (list-files (str clj-path common-path)) common-path cljs-path)
(cljsc/build
cljs-path
{
:output-dir js-path
:output-to (str js-path module-name ".js")
}
)
)
(defn build-default []
(build
{
:clj-path "/home/user/projects/example/code/src/main/clojure/"
:cljs-path "/home/user/projects/example/code/src/main/cljs/"
:js-path "/home/user/projects/example/code/public/js/cljs/"
:common-path "example/common/" ; the root of your common server-client code
:module-name "example"
}
)
)
This question predates cljc, but since I stumbled upon it, I thought I would mention Clojure reader conditionals.
Related
Why can't we simply convert Clojure code to string and send it over TCP and evaluate on the other side(nrepl)?
For example : This is a hashmap {"foo" "bar", 1 "spam"} whose BENCODE encoding is d3:foo3:bari1e4:spame.
If we convert it to string -> {\"foo\" \"bar\", 1 \"spam\"}
and evaluate on the other side instead of using BENCODE as shown below.
(eval (read-string "{\"foo\" \"bar\", 1 \"spam\"}"))
; ⇒ {"foo" "bar", 1 "spam"}
I am new to the Clojure world. This might be a stupid question but anyway.
nREPL's maintainer here. There are several reasons why nREPL uses bencode by default:
We needed a data format that supports data streaming easily (you won't find many streaming JSON parsers)
We needed a data format that could easily be supported by many clients (support for formats like JSON and EDN is tricky in editors like Emacs and vim). I can tell you CIDER would not have existed if 8 years ago we had to deal with JSON there. :-)
Bencode is so simple that usually you don't even need to rely on a third-party library for it (many clients/servers have their own implementation of the encoding/decoding in less than 100 lines of code) - this means means that clients/servers have one less 3rd party library. Tools like nREPL can't really have runtime deps, as they conflict with the user application deps.
EDN didn't exist back when nREPL was created
Btw, these days nREPL supports EDN and JSON (via the fastlane library) as well, but I think that bencode is still the best transport in most cases.
For people looking for the answer, read the # Motivation section in https://github.com/clojure/tools.nrepl/blob/master/src/main/clojure/clojure/tools/nrepl/bencode.clj
This is very well written.
I have the following code that loads edn data from my resources folder:
(defn load-data
[]
(->> (io/resource "news.edn")
slurp
edn/read-string))
I'd like to test this by mocking out the file reading part, so far I have this:
(deftest loading-data
(is (= (edn/read-string (prn-str {:articles [{:title "ASAP Rocky released" :url "http://foo.com"}]})) (load-data))))
But I know this very flaky test because if the edn file name changes or it's contents or updated, the test will fail. Any ideas?
You can mock out a call to function with with-redefs, that "Temporarily redefines Vars while executing the body". E.g.,
(deftest load-data-test
(with-redefs [slurp (constantly "{:a \"b\"}")]
(is (= (load-data) {:a "b"}))))
This way the slurp in load-data in the scope of with-redefs returns "{:a \"b\"}".
What about this function are you wanting to gain confidence for? Are you worried that news.edn won't exist? Are you worried slurping and or edn reading won't work on a resource?
My advice is to test your separate concerns separately
If you're worried about news.edn not existing assert against it's existence
If you're worried about the rest of the function not converting from edn add a new signature to accept a resource then provide another resource at test time to assert against
If you're worried about the shape of the file maybe have a test that runs against the data before it's put in news.edn
Then when you come back to these tests years later you'll see clear reasons for failures instead of a test that fell over because of N possible reasons that are unknown till debug time
As other's have suggested: Mock less things.
It might be a reflex you learned from building Java tests.
If you have composed your functions correctly, you can test them individually without the need for side-effects (like reading a file).
If you mock slurp in your example, you are not testing anything meaningful: You would essentially test if the standard function edn/read-string works as intended.
Rewrite your load-data to accept the name of the file to load as argument (and later call it with news.edn in your "main"). This makes it way more functional and this way you can easily test load-data the way you test it right now, but you pass down some test-news.edn from your test resources. And there is no need to mock anything for the happy path.
This way you can also write test for other scenarios: what if the file is missing? Or the .edn file is malformed. What if you pass some resource that loads forever? etc.
I'm working on a script that needs to read tens of thousands of files from the disk. I'm trying to understand the best way to do this. I've run into a problem when I use map to do this using two packages clj-glob and clojure-mail:
(def sent-mail-paths
(->> (str maildir-path "/*/_sent_mail/*")
(glob) ;; returns files using clojure.java.io/as-file
(map str) ;; i just want the paths
))
(def msgs
(->> sent-mail-paths ;; 30K + paths
(map mail/file->message)))
where the glob function in the first block comes from clj-glob and uses as-file to return a set of file objects (see here). I only want the path strings, so I do (map str). The mail/file->message function in the second block uses with-open along with the java FileInputStream class to read the files (see here).
The trouble I am encountering is that this code causes an error the moment I try to process the files in the resulting lazy sequence by doing evensomething like:
(count msgs)
The error is:
(Too many open files in system)
The only way I've been able to get the job done here is to use doseq:
(def msgs (->> list-of-paths ;; 30K+ paths
(map mail/file->message)))
(def final (atom []))
(doseq [x result]
(swap! final conj (mail/file->message x)))
My question is whether this is the best (only?) way to accomplish this process without opening thousands and thousands of files at once? I don't fully understand why I can't use the lazy sequence that is returned by map. Why does that end up opening tons of files.
One thing, incidentally, that I noticed is that clj-glob, which is not a well-maintained package, does not use with-open when it calls as-file...
Even if you open/close the files correctly, there's a chance that during the execution of the program you hit an internally defined limit on the number of file descriptors that you program can have (this is common on long-lived programs such as microservices).
You can read here on how to look up what that limit is currently and how to increase it: https://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
Is there any way to import a record type, that works in Clojure as well as ClojureScript?
As far as I can tell it's (ns x (:import y [A B])) for Clojure, (ns x (:require y :refer [A B])) for ClojureScript, and each is invalid for the respective other.
Ignoring the specifics on the syntax of requiring records, there are two main ways to write ns declarations (or any platform specific code) for multiple Clojure variants while sharing the majority of your code.
CLJX is a Clojure preprocessor that runs before the Clojure compiler. You write platform specific code prefixed with #+clj or #+cljs in a .cljx file. It can run on pretty much any Clojure code, and will spit out multiple platform specific files which the respective Clojure Compilers can handle.
Reader Conditionals are a feature in Clojure 1.7 and are available in recent releases of ClojureScript. This is similar in spirit to cljx, but is integrated into the Clojure Compiler. You write code with reader conditionals like #?(:clj 1 :cljs 2) in files with a .cljc extension.
Now back to your specific question, you can achieve this with Reader Conditionals like so:
(ns myapp.music-store
(:require #?(:clj [myapp.cool-music]
:cljs [myapp.cool-music :refer [Vinyl]]))
#?(:clj
(:import [myapp.cool_music Vinyl])))
I wrote a longer blog post about this too: Requiring records in Clojure and ClojureScript
I've written a very simple date library in Clojure and I'd like to be able to use it both when my code runs in a JVM and when it is compiled to Javascript with ClojureScript. The fly in the ointment is how to detect which runtime environment the code is in so that a macro might use the platform's own date string parsing mechanism. Each platform's date objects
How should one write a library that may be reused across different deployment platforms like the JVM, JS, CLR etc.? I know that one might cheat and parse date strings with a regex but that will not easily cover the case of parsing month names from the large variety of human languages which the built-in date parsing libraries solve quite nicely.
Use resolve to find out which vars defined:
user=> (resolve '*clojurescript-version*)
nil
user=> (resolve '*clojure-version*)
#'clojure.core/*clojure-version*
user=> *clojure-version*
{:major 1, :minor 5, :incremental 1, :qualifier nil}
You can use these misc vars:
http://clojuredocs.org/clojure_core/clojure.core/clojure-version
Returns clojure version as a printable string.
http://clojuredocs.org/clojure_core/clojure.core/clojure-version
The version info for Clojure core, as a map containing :major :minor
:incremental and :qualifier keys. Feature releases may increment
:minor and/or :major, bugfix releases will increment :incremental.
Possible values of :qualifier include "GA", "SNAPSHOT", "RC-x" "BETA-x"
Clojure has special Conditional reader rule to switch b/w runtimes.
In your example, it would something like:
(def ^:dynamic *runtime*
#?(:clj :clojure)
#?(:cljs :clojurescript))
(print *runtime*) ;; :clojure