A Clojure project building several related small tools - clojure

I'm working on a Clojure project with a common library of functionality, but I want to use this library from a number of small command-line tools.
Let's call them tool1, tool2, tool3.
Each of these tools wraps the bulk of the library.
How should I organize my Clojure source code and what do I need in my project.clj file?
Right now my code is in
src/projectname/core.cljc ; a default executable
cljc/projectname/lib1.cljc
cljc/projectname/lib2.cljc ; cljc because I want to use from clojurescript too later
etc.
Where should I put the tool1.clj, tool2.clj etc?
My defproject is
(defproject
....
:aot [projectname.core]
:main project.core )
What do I need to put in to tell it that I want to make the three stand-alone separate, executable tools?

You can do this using the lein-exec library.
First, set up ~/bin like this (or similar):
> ls -ldF ~/bin/l*
lrwxrwxrwx 1 alan alan 26 Oct 31 08:29 /home/alan/bin/lein -> /home/alan/cool/tools/lein*
lrwxrwxrwx 1 alan alan 31 Oct 31 08:29 /home/alan/bin/lein-exec -> /home/alan/cool/tools/lein-exec*
lrwxrwxrwx 1 alan alan 33 Oct 31 08:29 /home/alan/bin/lein-exec-p -> /home/alan/cool/tools/lein-exec-p*
I have symlinks to the actual files, but you can copy them directly. Make sure ~/bin is in your path, of course.
Then just write an executable clojure "script" like the following. Of course, it must be executable:
> ls -l say-hello
-rwxrwxr-x 1 alan alan 212 Nov 2 09:10 say-hello
> cat say-hello
#!/usr/bin/env lein-exec
(defn say-hello [name]
(println (format "Hello from the command line, %s!" name)))
(do
; *command-line-args* = <cmd> <arg1> <arg2> ...
(say-hello (second *command-line-args*)))
And off we go!
> ./say-hello buckaroo
Hello from the command line, buckaroo!
As for project organization, I would start simple (as with any project!). Maybe start off with just 1 source file, with different functions as entrypoints for each "script". As the project grows, it'll be easier to see where you would like to break out different namespaces/files.
Update
You can also do this with lein uberjar, and then invoking java directly:
(ns clj.core
(:gen-class))
(defn say-hello [name]
(println (format "Hello from the command line, %s!" name)))
(defn -main [& args]
(say-hello (first args)))
> lein uberjar
Compiling 1 source files to /home/alan/clj/target/uberjar/classes
Compiling clj.core
Created /home/alan/clj/target/uberjar/clj-0.1.0-SNAPSHOT.jar
Created /home/alan/clj/target/uberjar/clj-0.1.0-SNAPSHOT-standalone.jar
> java -jar /home/alan/clj/target/uberjar/clj-0.1.0-SNAPSHOT-standalone.jar pardner
Hello from the command line, pardner!
So just put the java command into a shell script to kick things off. Also note that :gen-class is required and that args doesn't include the script name anymore. Of course, you must deploy both the script files and the uberjar.
Update #2
You can also use this version:
Source:
(ns clj.core
(:gen-class))
(defn say-howdy [args]
(println (format "Howdy, %s!" (first args))))
(defn give-reply [args]
(println (format "Back at ya, %s!" (first args))))
(defn -main [& args]
(let [method-name (first args)
message (second args) ]
(cond
(= method-name "say-howdy" ) (say-howdy (rest args))
(= method-name "give-reply") (give-reply (rest args))
:else (throw (NoSuchMethodException. (str "clj.core: invalid method='" method-name \')))))
)
scripts:
> ls -l say*
-rwxrwxr-x 1 alan alan 212 Nov 2 16:24 say-hello
-rwxrwxr-x 1 alan alan 104 Nov 2 16:35 say-howdy
-rwxrwxr-x 1 alan alan 105 Nov 2 16:35 say-reply
> cat ./say-howdy
#!/bin/bash
java -jar /home/alan/clj/target/uberjar/clj-0.1.0-SNAPSHOT-standalone.jar say-howdy pardner
> cat ./say-reply
#!/bin/bash
java -jar /home/alan/clj/target/uberjar/clj-0.1.0-SNAPSHOT-standalone.jar give-reply $1
Run tools:
~/clj > ./say-howdy
Howdy, pardner!
~/clj > ./say-reply
Back at ya, null!
~/clj > ./say-reply buckaroo
Back at ya, buckaroo!
Making the scripts smarter and deciding how to deploy the N scripts and single JAR file is left as an exercise for the reader.
;)

Related

Unittest in Clojure doesn't compile when I test my own functions

I am new to Clojure and now I'm trying to use some unittesting.
I have a sample project with this structure:
core.clj in src/hello contains
(ns hello.core
(:gen-class))
(defn side-eq [x]
(if (< 0 (count x))
(= (.charAt x 0) (.charAt x (- (count x) 1)))
false))
core.clj in test/hello contains
(ns hello.core
(:use clojure.test)
(:require [hello.core :refer :all])
(:gen-class))
(use 'clojure.test)
(deftest side-eq-tests (
is (= false (side-eq "aws"))))
(run-tests)
when I execute tests, it throws
java.lang.RuntimeException: Unable to resolve symbol: side-eq in this context
When I test something like
is (= 1 1)
then everything works fine.
What is going on?
You should not have multiple files with the same namespace. Rename your tests to something else. The idiomatic name here would be hello.core-test In test/hello/core_test.clj
A variation which I prefer is to begin all testing namespacese with the tst.* prefix, which avoids the hyphen-underscore conversion & confusion (e.g. demo.core-test vs demo.core_test.clj. So your files look like so:
> d **/*.clj
-rwxrwxr-x 1 alan alan 1024 Jan 5 19:00 project.clj*
-rwxrwxr-x 1 alan alan 84 Jan 5 15:29 src/demo/core.clj*
-rwxrwxr-x 1 alan alan 248 Jan 7 12:42 test/tst/demo/core.clj*
and the code looks like:
(ns demo.core)
(defn -main []
(println "main - enter")
)
and
(ns tst.demo.core
(:use demo.core tupelo.core tupelo.test)
(:require
[tupelo.misc :as tm] ))
....

Why I get no output when use (sh "top") in clojure?

I try to get output from sh command but don't see any.
(:use [clojure.java.shell :only [sh]])
(sh "ls" "-aul"); <-- WORKS
=>
{:exit 0,
:out "total 136
-rw-r--r-- 1 snaggs staff 113 Jan 8 14:17 .babelrc
drwxr-xr-x 15 snaggs staff 510 Jan 12 14:14 .hg
The command (sh "top") - no output. I know that top command is like listener. So how can I configure sh to get output from top. Suppose tail -f should be the same behavior.
Thanks,
First make sure, that you run your version of top in "batch mode", which just prints to stdout ad infinitum. For Linux and FreeBSD this is top -b. Next sh can not be used for this problem, as it waits for the command to finish. So the same can be done via the ProcessBuilder from Java. E.g.
(require '[clojure.java.io :as io])
; start `top -b`
(let [process (-> (ProcessBuilder. ["top" "-b"]) .start)]
; get the stdout of the process
(with-open [stdout (io/reader (.getInputStream process))]
(loop []
; read a line, handle it, until EOF
(when-let [line (.readLine stdout)]
(println line) ; work here
(recur)))))
(note this lacks a final (.destroy process) call)
Original answer why (sh "top") does not return
Not strictly a Clojure problem. You have to start top with the following params for the linux top: -n1 (only one iteration, then stop) and -b (batchmode, just write to stdout). Other versions might differ here.
user=> (use '[clojure.java.shell :only [sh]])
nil
user=> (sh "top" "-n1" "-b")
{:err "",
:exit 0,
:out "top - 14:06:47 up 126 days, 5:21, 5 users, load average: 0.93, 0.67, 1.12 ,,,
In your example, you just ran top as with you would do from the command line. So it starts and runs waiting for you to quit or kill it - blocking your REPL.

How to substitute path to home for "~"?

If I pass in a path from the command line, "~" expands to my home directory:
(defn -main
"I don't do a whole lot ... yet."
[& args]
(doseq [arg args]
(println arg)))
britannia:uberjar srseverance$ java -jar args-0.1.0-SNAPSHOT-standalone.jar ~/158.clj
/Volumes/Macintosh HD/Users/srseverance/158.clj
But if I try to use a path-file containing ~, I can't find the file.
user> (with-open [r (clojure.java.io/reader "~/158.clj")]
(doall (line-seq r)))
FileNotFoundException ~/158.clj (No such file or directory) java.io.FileInputStream.open0 (FileInputStream.java:-2)
How do I take a string like, "~/158.clj" and get back something clojure.java.io/reader can use, such as "/Volumes/Macintosh HD/Users/srseverance/158.clj"?
You can define
(defn expand-home [s]
(if (.startsWith s "~")
(clojure.string/replace-first s "~" (System/getProperty "user.home"))
s))
and use it to resolve home directory:
(clojure.java.io/reader (expand-home "~/158.clj"))]
You could also look into fs library definition of expand-home, which solves the ~foo problem outlined in bfontaine's comment below:
(let [homedir (io/file (System/getProperty "user.home"))
usersdir (.getParent homedir)]
(defn home
"With no arguments, returns the current value of the `user.home` system
property. If a `user` is passed, returns that user's home directory. It
is naively assumed to be a directory with the same name as the `user`
located relative to the parent of the current value of `user.home`."
([] homedir)
([user] (if (empty? user) homedir (io/file usersdir user)))))
(defn expand-home
"If `path` begins with a tilde (`~`), expand the tilde to the value
of the `user.home` system property. If the `path` begins with a
tilde immediately followed by some characters, they are assumed to
be a username. This is expanded to the path to that user's home
directory. This is (naively) assumed to be a directory with the same
name as the user relative to the parent of the current value of
`user.home`."
[path]
(let [path (str path)]
(if (.startsWith path "~")
(let [sep (.indexOf path File/separator)]
(if (neg? sep)
(home (subs path 1))
(io/file (home (subs path 1 sep)) (subs path (inc sep)))))
path)))
Addressing bfontaine's comment, we can get correct results for ~user and ~root by asking the system instead:
(require '[clojure.java.shell :refer [sh]])
(defn bash [command]
(sh "bash" "-c" command))
(defn expand [path]
(-> (str "echo -n " path)
bash
:out))
(expand "~")
;; => /home/teodorlu
(expand "~teodorlu")
;; => /home/teodorlu
(expand "~root")
;; => /root
Though, just use this for trusted code!
(expand "`cat ~/.passwords`")
;; => All my passwords!

Concatenate two files and redirect output with Clojure using terminal

I'm trying to reach out to the terminal in Clojure to concatenate two binary files together.
So I'm trying to do something like: cat file1 file2 > target
I've started looking at conch but I can't seem to get cat to treat my inputs as file paths rather than strings, e.g.
(def files '["/tmp/file1" "/tmp/file2"])
(defn add-to-target [files target]
(cat {:in files :out (java.io.File. target)}))
(add-to-target files "/tmp/target")
The result written to the /tmp/target file is:
/tmp/file1
/tmp/file2
I'm happy to try other (perhaps more Clojure idiomatic) ways to do this.
Thanks in advance.
Here you go:
(ns user
(:require [clojure.java.io :as io]))
(defn catto [f1 f2 out]
(with-open [o (io/output-stream out)]
(io/copy (io/file f1) o)
(io/copy (io/file f2) o)))
;; in REPL
;; > (catto "station.mov" "super.pdf" "zzz.bin")
Take a look at clojure.java.io docs.

Zip a file in clojure

I want to zip a file in clojure and I can't find any libraries to do it.
Do you know a good way to zip a file or a folder in Clojure?
Must I use a java library?
There is a stock ZipOutputStream in Java which can be used from Clojure. I don't know whether there is a library somewhere. I use the plain Java functions with a small helper macro:
(defmacro ^:private with-entry
[zip entry-name & body]
`(let [^ZipOutputStream zip# ~zip]
(.putNextEntry zip# (ZipEntry. ~entry-name))
~#body
(flush)
(.closeEntry zip#)))
Obviously every ZIP entry describes a file.
(require '[clojure.java.io :as io])
(with-open [file (io/output-stream "foo.zip")
zip (ZipOutputStream. file)
wrt (io/writer zip)]
(binding [*out* wrt]
(doto zip
(with-entry "foo.txt"
(println "foo"))
(with-entry "bar/baz.txt"
(println "baz")))))
To zip a file you might want to do something like this:
(with-open [output (ZipOutputStream. (io/output-stream "foo.zip"))
input (io/input-stream "foo")]
(with-entry output "foo"
(io/copy input output)))
All compression and decompression of files can be done with a simple shell command which we can access through clojure.java.shell
Using the same method you can also compress and decompress any compression type you would usually from your terminal.
(use '[clojure.java.shell :only [sh]])
(defn unpack-resources [in out]
(clojure.java.shell/sh
"sh" "-c"
(str " unzip " in " -d " out)))
(defn pack-resources [in out]
(clojure.java.shell/sh
"sh" "-c"
(str " zip " in " -r " out)))
(unpack-resources "/path/to/my/zip/foo.zip"
"/path/to/store/unzipped/files")
(pack-resources "/path/to/store/archive/myZipArchiveName.zip"
"/path/to/my/file/myTextFile.csv")
You can import this (gzip) https://gist.github.com/bpsm/1858654
Its quite interesting.
Or more precisely, you can use this
(defn gzip
[input output & opts]
(with-open [output (-> output clojure.java.io/output-stream GZIPOutputStream.)]
(with-open [rdr (clojure.java.io/reader input)]
(doall (apply clojure.java.io/copy rdr output opts)))))
You can use rtcritical/clj-ant-tasks library that wraps Apache Ant, and zip with a single command.
Add library dependency [rtcritical/clj-ant-tasks "1.0.1"]
(require '[rtcritical.clj-ant-tasks :refer [run-ant-task]])
To zip a file:
(run-ant-task :zip {:destfile "/tmp/file-zipped.zip"
:basedir "/tmp"
:includes "file-to-zip"})
Note: run-ant-task(s) functions in this library namespace can be used to run any other Apache Ant task(s) as well.
For more information, see https://github.com/rtcritical/clj-ant-tasks