custom accept function in start-server clojure - clojure

I'm trying to run clojure.core.server/start-server but instead of using the repl I want a custom function as accept option. I'm following this post where a repl server is executed as
clojure -X clojure.core.server/start-server :name '"server"' :port 5555 :accept clojure.core.server/repl :server-daemon false
What requirements do I need to pass a function to accept opt? What if I only want to print the request from any connection? Using something like clojure.core/println didn't work
clojure -X clojure.core.server/start-server :name '"server"' :port 5555 :accept clojure.core/println :server-daemon false
btw, I can't even run start-server on the repl itself, I've got the error everytime a made a request. Is it possible to run it from the repl?
(clojure.core.server/start-server {:name "server" :port 9000 :accept clojure.core.server/repl :server-daemon false})
#object[java.net.ServerSocket 0x25b865b5 "ServerSocket[addr=localhost/127.0.0.1,localport=9000]"]
user=> Exception in thread "Clojure Connection server 1" java.lang.ClassCastException: class clojure.core.server$repl cannot be cast to class clojure.lang.Named (clojure.core.server$repl and clojure.lang.Named are in unnamed module of loader 'app')
at clojure.core$namespace.invokeStatic(core.clj:1612)
at clojure.core.server$accept_connection.invokeStatic(server.clj:73)
at clojure.core.server$start_server$fn__8998$fn__8999$fn__9001.invoke(server.clj:117)
at clojure.lang.AFn.run(AFn.java:22)
at java.base/java.lang.Thread.run(Thread.java:833)

The start-server function states, that :accept must be a namespaced
symbol (so this is the reason, why you see the error in your REPL
(you are passing a variable; use 'clojure.core/println instead --
note the leading ').
As for passing your own function, it helps to take a look at the
function clojure.core.server/accept-connection, what is actually going
on. So basically your :accept function gets called with *in*,
*out*, *err* redirected (and also a binding of *session* with some
book-keeping) and the :args. So you want to read/write on
stdin/stdout in your function and parameterize with the given arguments.
E.g. create a file src/so.clj with the following content:
(ns so)
(defn accept
[prefix]
(loop []
(println prefix (read-line))
(recur)))
Then start the server from the same root:
clojure -X clojure.core.server/start-server :name '"server"' :port 5555 \
:accept so/accept \
:args '[">>>"]' \
:server-daemon false
Note the :accept symbol (it's coming from the shell, so Clojure reads
that as a symbol) points to the namespace and the function within. Make
sure, things match up with the class-path (src is a default source
path for the Clojure CLI).
Also note the passing of the :args represented as an EDN array.
Inside accept-connection the call to your function is then (apply accept args), so you can assume regular arguments in your accept-fn.
Then test your server with e.g. telnet:
% telnet localhost 5555
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
hello
>>> hello
world
>>> world

Related

Shared env vars between profiles when using yogthos/config in clojure?

The https://github.com/yogthos/config approach let's you lay out per-profile env variables in separate files, like the below , in a project.clj .
Per the below, one can use lein with-profile prod uberjar or lein with-profile dev repl and the like.
But my issue is I have been unable to figure out how to place some common values into a shared area, accessible by dev, stage, prod profiles.
Basic example
(defproject edn-config-test "0.1.0-SNAPSHOT"
...
:profiles {:shared {:resource-paths ["config/shared"]}
:dev {:resource-paths ["config/dev"]}
:stage {:resource-paths ["config/stage"]}
:prod {:resource-paths ["config/prod"]}}
...
(with files)
config/shared/config.edn
config/dev/config.edn
config/stage/config.edn
config/prod/config.edn
I tried this without luck
lein with-profile shared,prod lein , borrowing from the composite approach in
https://github.com/technomancy/leiningen/blob/stable/doc/PROFILES.md#composite-profiles
When I do that, I only get variables in prod profile, for example.
I think it is a limitation of config. I tried this (more explicit):
:profiles {:dev {:resource-paths ["config/shared" "config/dev"]}
:prod {:resource-paths [ "config/prod" "config/shared"]}}
However, the last file wins and the first is ignored. So for :dev the shared stuff is ignored, and for :prod the prod stuff is ignored (like it doesn't exist):
config/dev/config.edn => {:special-val :dev-val}
config/prod/config.edn => {:special-val :prod-val}
cat config/shared/config.edn => {:shared-val 42}
and results:
> lein with-profile prod run
(:shared-val env) => 42
(:special-val env) => nil
> lein with-profile dev run
(:shared-val env) => nil
(:special-val env) => :dev-val
Perhaps you'd like to submit an enhancement PR to the project?
Here is the problem. It uses io/resource to read config.edn, which implicitly expects there to be only one file config.edn anywhere on the classpath:
(defn- read-config-file [f]
(try
(when-let [url (io/resource f)]
(with-open [r (-> url io/reader PushbackReader.)]
(edn/read r))) ...
(read-config-file "config.edn")
So you'd have to get away from the hard-coded filename config.edn, and make something like config-dev.edn, config-prod.edn, and config-shared.edn. At least then they could all live in a single ./resources dir.

with-redefs-fn fails to pick up binding from do-seq?

There's something fundamental I'm not getting here. I expected the following test to pass. But the 2nd test case "staging"/"staging" fails. Its as if with-redefs-fn is failing to advance through the test-case instances. But the logging says everything is fine. This is confusing.
(deftest test-bad-derive-s3-environment
(testing "variants of props environments"
(doseq [test-case [{:env "qa1" :expect "qa1"}
{:env "dev" :expect "qa1"}
{:env "staging" :expect "staging"}]]
(log/infof "test-case %s" test-case)
(with-redefs-fn {#'config/environment (fn [] (:env test-case))}
(let [actual (fs/derive-s3-environment (config/environment))
_ (log/infof "within redefs :env %s :expect %s" (:env test-case) (:expect test-case))]
#(is (= actual (:expect test-case))))))))
...
lein test com.climate.test.mapbook.filestore
2016-05-03 16:16:29,353 INFO filestore:288 - test-case {:env "qa1", :expect "qa1"}
2016-05-03 16:16:29,355 INFO EnvConfig:98 - Loading config properties from /export/disk0/wb/etc/env.properties
2016-05-03 16:16:29,357 INFO EnvConfig:98 - Loading config properties from /export/disk0/wb/etc/local.properties
2016-05-03 16:16:29,358 INFO filestore:288 - within redefs :env qa1 :expect qa1
2016-05-03 16:16:29,359 INFO filestore:288 - test-case {:env "staging", :expect "staging"}
2016-05-03 16:16:29,359 INFO filestore:288 - within redefs :env staging :expect staging
lein test :only com.climate.test.mapbook.filestore/test-bad-derive-s3-environment
FAIL in (test-bad-derive-s3-environment) (filestore.clj:29)
variants of props environments
expected: (= actual (:expect test-case))
actual: (not (= "qa1" "staging"))
2016-05-03 16:16:29,364 INFO filestore:288 - test-case {:env "dev", :expect "qa1"}
2016-05-03 16:16:29,364 INFO filestore:288 - within redefs :env dev :expect qa1
Why does my with-redefs-fn fail to redefine the config/environment function in terms of the current test-case?
First of all, notice that your final test instance has an :expect of "qa1" – the same as the first test instance – so it should actually fail if the code worked as you intended it to; its passing is a symptom of the same problem as the second instance's failing.
Now for the fix – there are two options:
Just use with-redefs instead of with-redefs-fn:
(with-redefs [config/environment (fn [] (:env test-case))]
…)
Most of the time this is what you want to do and you can consider with-redefs-fn to be an implementation detail behind with-redefs – although strictly speaking it does have some utility of its own in that it can redefine dynamically constructed collections of Vars.
Use with-redefs-fn, but move the inner let form inside the anonymous function:
(with-redefs-fn {…}
#(let […]
(is …)))
Finally, the reason these work and the version from the question text does not:
with-redefs-fn is a function, so at runtime its arguments will be evaluated before it is actually invoked with their runtime values passed in. In particular, the let expression that you pass in as the second argument will be evaluated before the redefinition takes place, and so the local called actual will get the result of evaluating (config/environment) before the redefinition as its value, and that value will be installed in the anonymous closure created in the let's body. That closure, however, will then be called with the redefinition in place, and so it will take its notion of the "actual" value from before the redefinition and compare it with the expectation set after the redefinition, resulting in the observed behaviour.
Moving the let inside the closure, as in the second approach above, fixes this mismatch problem – the let local's value is computed with the redefinition in place and all is well. The first approach using with-redefs expands to the second approach.
The log printouts are fine, because they are only concerned with the doseq local and never examine any Vars. If they did, they would only see the pre-redefinition values.

Difference between lein repl (-main "something") and lein run "something")

Newbie Clojure and leiningen question:
Given the code snippet in my project below, this works from the lein repl :
==> (-main "something")
produces the expected "Command: something ... running ... done"
but doesn't work from the command line:
me pallet1]lein run "something"
produces "Command: something ... error: not resolved as a command"
Why? / how do I fix it?
To reproduce:
lein new eg
Then edit the generated project file, adding :main eg.core to define the main function, and edit the generated src/eg/core.clj file, and paste this in:
core.clj
(ns eg.core)
(defn something [] (println "Something!"))
(defn run-command-if-any [^String commandname]
(printf "Command: %s ..." commandname)
(if-let [cmd (ns-resolve *ns* (symbol commandname))]
(
(println "running ...") (cmd) (println "done.")
)
(println "error: not resolved as a command.")
))
(defn -main [ commandname ] (run-command-if-any commandname))
Then
lein repl
eg.core=> (-main "something")
works (ie prints "Something!) , but
lein run something
doesn't (ie prints the "error: not resolved" message)
The problem is that when you run it from lein your default namespace is "user" namespace:
(defn -main [ commandname ] (println *ns*))
Prints #<Namespace user>. So it doesn't contain something function because it is from another namespace. You have several choices:
Pass fully qualified function name: your-namespace/something instead of something.
Use your-namespace instead of *ns*: (ns-resolve 'your-namespace (symbol commandname))
Change namespace to your-namespace in -main.
Example of method 3:
(defn -main [ commandname ]
(in-ns 'your-namespace)
(run-command-if-any commandname))
Also you if you want to call several functions one by one you should use do:
(do (println "Hello")
(println "World"))
Not just braces like ( (println "hello") (println "World"))
the lein exec plugin is very useful for scripting such things in the context of a project. I have used this extensively for writing Jenkins jobs in clojure and other scripting situations
lein exec -pe '(something ...) (something-else) (save-results)'

Clojure throws ClassCastException on java.util.Collection

I'm trying to use an MPD library in Java in Clojure. Everything has gone well, but these methods that return a java.util.Collection get bad reception in REPL. Let's define
(def mpd (org.bff.javampd.MPD. "localhost" ))
(def pl (.getMPDPlaylist mpd))
(def db (.getMPDDatabase mpd))
And now some methods play ok:
(.getSongList pl) ; returns List<MPDSong>
works well. But for instance every db (MPDDatabase) method return Collection<MPDSong> (according to their API):
(.findAlbum db "Crises") ; returns Collection<MPDSong>
java.lang.ClassCastException (NO_SOURCE_FILE:0)
Doesn't work that well. Why is that, how to fix it?
Stack trace follows:
hello.hello=> (.findAlbum db "Crises")
java.lang.ClassCastException (NO_SOURCE_FILE:0)
hello.hello=> (.printStackTrace *e)
java.lang.ClassCastException (NO_SOURCE_FILE:0)
at clojure.lang.Compiler.eval(Compiler.java:5440)
at clojure.lang.Compiler.eval(Compiler.java:5391)
at clojure.core$eval.invoke(core.clj:2382)
at clojure.main$repl$read_eval_print__5624.invoke(main.clj:183)
at clojure.main$repl$fn__5629.invoke(main.clj:204)
at clojure.main$repl.doInvoke(main.clj:204)
at clojure.lang.RestFn.invoke(RestFn.java:422)
at user$eval13$acc__808__auto____14$fn__16.invoke(NO_SOURCE_FILE:1)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.ClassCastException
at java.lang.Class.cast(Class.java:2990)
at clojure.lang.Reflector.boxArg(Reflector.java:364)
at clojure.lang.Reflector.boxArgs(Reflector.java:397)
at clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:55)
at clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28)
at hello.hello$eval44.invoke(NO_SOURCE_FILE:8)
at clojure.lang.Compiler.eval(Compiler.java:5424)
... 9 more
nil
Looks like the API Documentation is invalid (checking using clojure.contrib.repl-utils):
user> (show MPDDatabase "findAlbum$")
=== public org.bff.javampd.MPDDatabase ===
[ 1] findAlbum : Collection (MPDAlbum)
and you need to a intermediate MPDAlbum object:
user> (.findAlbum db (MPDAlbum. "Crisis"))
#<ArrayList []>

Issue reading command-line-args in script

I'm attempting to write a library to do some domain-specific stuff. I'd like to add some scripts to this library that can be run straight from the commandline.
Directory layout:
+- project.clj
+- src/
| +- my-lib.clj
| +- my-lib/
| +- my-sub-1.clj
+- scripts/
+- run-command.sh
The src/my-lib.clj file loads the my-lib/my-sub-1.clj file:
(ns my-lib
(:use [my-lib.my-sub-1])
)
The src/my-lib/my-sub-1.clj file contains the functions I want to make available. One of these functions is called "convert" and takes two arguments: an input filename and an output filename; it converts the fileformat. To use it:
(convert "input-file.txt" "output-file.txt")
The (do-something-interesting) and (convert) functions in src/my-lib/my-sub-1.clj look like this:
(defn do-something-interesting
[input-file output-file]
(magic-happens-here input-file output-file))
(defn convert
[args]
(let [infile (first args)
outfile (second args)]
(do-something-interesting infile outfile)))
My goal at the moment: to create a script "run-command.sh" in the "scripts" directory that takes two arguments: the input filename and output filename. It should be possible to run the script with:
./run-command.sh input-file.txt output-file.txt
I do have run-command.sh working, as long as I hard-code the filenames in that script by using the (do-something-interesting) function instead of (convert). I haven't been able yet to read from the argument list...
The run-command.sh script that works:
#!/bin/sh
java -cp "../lib/*":"../src":$CLASSPATH clojure.main -e "
(use '[my-lib my-sub-1])
(do-something-interesting "path-to-input-file" "path-to-output-file")
"
... but what doesn't work:
#!/bin/sh
java -cp "../lib/*":"../src":$CLASSPATH clojure.main -e "
(use '[my-lib my-sub-1])
(convert *command-line-args*)
"
The error I get is:
Exception in thread "main" java.lang.IllegalArgumentException: No implementation of method: :reader of protocol: #'clojure.contrib.io/Streams found for class: nil (NO_SOURCE_FILE:0)
at clojure.lang.Compiler.eval(Compiler.java:5435)
at clojure.lang.Compiler.eval(Compiler.java:5386)
at clojure.core$eval.invoke(core.clj:2382)
at clojure.main$eval_opt.invoke(main.clj:235)
at clojure.main$initialize.invoke(main.clj:254)
at clojure.main$null_opt.invoke(main.clj:279)
at clojure.main$main.doInvoke(main.clj:354)
at clojure.lang.RestFn.invoke(RestFn.java:422)
at clojure.lang.Var.invoke(Var.java:369)
at clojure.lang.AFn.applyToHelper(AFn.java:165)
at clojure.lang.Var.applyTo(Var.java:482)
at clojure.main.main(main.java:37)
Caused by: java.lang.IllegalArgumentException: No implementation of method: :reader of protocol: #'clojure.contrib.io/Streams found for class: nil
at clojure.core$_cache_protocol_fn.invoke(core_deftype.clj:471)
at clojure.contrib.io$eval32$fn__33$G__23__38.invoke(io.clj:118)
at bioclojure.vcf$header.invoke(vcf.clj:22)
at bioclojure.vcf$column_header.invoke(vcf.clj:55)
at bioclojure.vcf$column_names.invoke(vcf.clj:61)
at bioclojure.vcf$vcf2tsv.invoke(vcf.clj:169)
at bioclojure.vcf$convert.invoke(vcf.clj:185)
at user$eval474.invoke(NO_SOURCE_FILE:3)
at clojure.lang.Compiler.eval(Compiler.java:5419)
... 11 more
I've tried "use"ing clojure.contrib.io within the script file run-command.sh itself, and within the top library file my-lib.clj, but no luck so far...
If anyone could help me out that'd be great.
jan.
You should consider using the "gen-class" feature to handle command
line arguments in a compiled Clojure/Java function, instead of
on-the-fly evaluating Clojure code via Clojures main/repl function:
(ns commandline
(:gen-class))
(defn -main [& args]
(convert args))
Use lein jar to create the Jar file of your Application and pass
command line arguments in your shell skript to the main function:
#!/bin/bash
java -cp "../lib/*":../YOURPROJECT.jar:$CLASSPATH commandline "$#"
That's because you don't specify cli arguments. You have to call java .... clojure.main some-script.clj a b c. Then a, b and c will be contained in *command-line-args*.
I found the solution thanks to kotarak's suggestion. In run-command.sh I need to refer to the arguments using the bash-way instead of the clojure way. And I don't even need that separate (convert) function.
What the script looked like:
#!/bin/sh
java -cp "../lib/*":"../src":$CLASSPATH clojure.main -e "
(use '[my-lib my-sub-1])
(convert *command-line-args*)
"
What it should look like:
#!/bin/sh
java -cp "../lib/*":"../src":$CLASSPATH clojure.main -e "
(use '[my-lib my-sub-1])
(do-something-interesting \"$1\" \"$2\")
"