I'm attempting to use the following code to base64 encode an image and then email it using a third-party email service.
(ns application.encode
(:import org.apache.commons.codec.binary.Base64)
(:require [clojure.java.io :as io]))
(defn encode [file-path]
(let [content (String.
(Base64/encodeBase64
(.getBytes
(slurp (clojure.java.io/resource "public/test.gif")))))]
;; email file contents
)
However, the image is being corrupted and its size is being doubled. I'm able to verify this either by sending it via email or spitting it to a local file.
What am I doing incorrectly?
UPDATE: In case it's useful, the following Ruby code does what I'm trying to do above and writing its output to a file, then slurping it allows me to send the email as desired. (This isn't a solution, of course, but I wanted to make sure what I'm trying to do was even possible using the file in question.)
encoded = Base64.encode64(File.read('resources/public/test.gif'))
First of all, significant increase in size is expected for base64 but not exactly doubling of course.
I'm not a Clojure developer but I beliveve that the issue is aroud slurp + getBytes. Unfortunately you can't just read file as string and use .getBytes and expect to get original file contents because of various issues introduced by encoding. You should read file directly as bytes array. This SO answer suggest there is no such standard function but slurp-bytes from the second answer looks quite promising to me.
Please see this closely related question and answer from this week: Slurping http://foobar.mp3 that redirects to http://fizzbar.mp3 in Clojure
Basically, slurp is for text files only.
For full information, please bookmark and keep open in a browser tab the Clojure Cheatsheet. Also, see the specific docs for the slurp function.
A nice example can be found in the Clojure Cookbook: https://github.com/clojure-cookbook/clojure-cookbook/blob/master/04_local-io/4-19_handle-binary-files.asciidoc
More examples can be found on ClojureDocs: https://clojuredocs.org/clojure.java.io/input-stream
Related
Why can't we simply convert Clojure code to string and send it over TCP and evaluate on the other side(nrepl)?
For example : This is a hashmap {"foo" "bar", 1 "spam"} whose BENCODE encoding is d3:foo3:bari1e4:spame.
If we convert it to string -> {\"foo\" \"bar\", 1 \"spam\"}
and evaluate on the other side instead of using BENCODE as shown below.
(eval (read-string "{\"foo\" \"bar\", 1 \"spam\"}"))
; ⇒ {"foo" "bar", 1 "spam"}
I am new to the Clojure world. This might be a stupid question but anyway.
nREPL's maintainer here. There are several reasons why nREPL uses bencode by default:
We needed a data format that supports data streaming easily (you won't find many streaming JSON parsers)
We needed a data format that could easily be supported by many clients (support for formats like JSON and EDN is tricky in editors like Emacs and vim). I can tell you CIDER would not have existed if 8 years ago we had to deal with JSON there. :-)
Bencode is so simple that usually you don't even need to rely on a third-party library for it (many clients/servers have their own implementation of the encoding/decoding in less than 100 lines of code) - this means means that clients/servers have one less 3rd party library. Tools like nREPL can't really have runtime deps, as they conflict with the user application deps.
EDN didn't exist back when nREPL was created
Btw, these days nREPL supports EDN and JSON (via the fastlane library) as well, but I think that bencode is still the best transport in most cases.
For people looking for the answer, read the # Motivation section in https://github.com/clojure/tools.nrepl/blob/master/src/main/clojure/clojure/tools/nrepl/bencode.clj
This is very well written.
I have the following code that loads edn data from my resources folder:
(defn load-data
[]
(->> (io/resource "news.edn")
slurp
edn/read-string))
I'd like to test this by mocking out the file reading part, so far I have this:
(deftest loading-data
(is (= (edn/read-string (prn-str {:articles [{:title "ASAP Rocky released" :url "http://foo.com"}]})) (load-data))))
But I know this very flaky test because if the edn file name changes or it's contents or updated, the test will fail. Any ideas?
You can mock out a call to function with with-redefs, that "Temporarily redefines Vars while executing the body". E.g.,
(deftest load-data-test
(with-redefs [slurp (constantly "{:a \"b\"}")]
(is (= (load-data) {:a "b"}))))
This way the slurp in load-data in the scope of with-redefs returns "{:a \"b\"}".
What about this function are you wanting to gain confidence for? Are you worried that news.edn won't exist? Are you worried slurping and or edn reading won't work on a resource?
My advice is to test your separate concerns separately
If you're worried about news.edn not existing assert against it's existence
If you're worried about the rest of the function not converting from edn add a new signature to accept a resource then provide another resource at test time to assert against
If you're worried about the shape of the file maybe have a test that runs against the data before it's put in news.edn
Then when you come back to these tests years later you'll see clear reasons for failures instead of a test that fell over because of N possible reasons that are unknown till debug time
As other's have suggested: Mock less things.
It might be a reflex you learned from building Java tests.
If you have composed your functions correctly, you can test them individually without the need for side-effects (like reading a file).
If you mock slurp in your example, you are not testing anything meaningful: You would essentially test if the standard function edn/read-string works as intended.
Rewrite your load-data to accept the name of the file to load as argument (and later call it with news.edn in your "main"). This makes it way more functional and this way you can easily test load-data the way you test it right now, but you pass down some test-news.edn from your test resources. And there is no need to mock anything for the happy path.
This way you can also write test for other scenarios: what if the file is missing? Or the .edn file is malformed. What if you pass some resource that loads forever? etc.
Suppose we wanted to write a cheating quine in clojure, we could do:
(ns cheating-quine)
... stuff here doesn't really matter ...
(println (slurp *file*))
Now, this does not work in Lein Figwheel because the value of file ends up being something like /tmp/form-init########.clj, and contains bootstrapping code of some sort.
Question: how can we get this "cheating" quine to work in clojurescript?
Note: the goal is NOT to write a quine. The goal is to write a cljs program which has access to the file it's defined in. This whole "cheating quine" thing is trying to explain the problem better then my previous failed attempt of
Clojure + Clojurescript: Macro to read code of current file
A cheating solution is to distribute the source code (e.g. put it in resources and serve it as a file) and compile a bootstrap to load and execute code, and get it to pull down your file, and in the file load the same file... that way it can do a HTTP request for the code that is currently running, and execute it. This is essentially how ClojureScript browser REPLs such as http://clojurescript.io/ work... They compile and execute incoming code on the fly. clojuresript.io is a good working example of ClojureScript bootstrapping if you are interested in it, but a word of caution, there is quite a lot going on so expect to invest quite some time :)
So I have been working on a Clojure tutorial and it's pretty fun so far. Unfortunately, every time I close my REPL out, I lose all of the defn and def that I created in the previous session.
So, to save time in the future, is it possible to have the Clojure REPL save everything I've typed to a file so I can dig out what I need for future uses?
I think most people work by using their IDE to send fragments of code to the REPL, rather than doing a lot of direct hacking in the REPL. This way you have a better idea of the current state of the program.
If you use Emacs and SLIME, you can save the REPL buffer to a file just like saving any other buffer.
If you are using a Unix-like operating system you can start your REPL using rlwrap. This would give you Ctrl-R for accessing commands from history in an easy way, as well as storing the history itself.
You'd just have to install rlwrap and prepend it to the line where you start REPL, e.g. rlwrap lein repl
Couple of tips:
I do a lot of coding in the REPL, but as soon as I write something that works then I copy/paste it into a source file. Then I can just run or reload that source file whenever I want to get a REPL environment back to a fully configured state.
Some REPLs (e.g. the Eclipse Counterclockwise REPL that I use) keep history from previous sessions so you can up-arrow to the earlier REPL commands you want to repeat.
This is actually a harder problem than it seems because a dump like this must take into account symbol and vars dependencies and output to file in the correct order of symbols. That said since code is data one should be able to write a function that does that.
I am trying to go through the process of creating a jar file from a simple clojure file. Below is my clojure code:
(ns app.first (:gen-class))
(refer 'clojure.core)
(defn -main [& args] (println "this program worked!"))
I am using these instructions to create the jar file: http://en.wikibooks.org/wiki/Clojure_Programming/Tutorials_and_Tips
I see the error "java.io.FileNotFoundException: Could not locate app/hello__init.class or app/hello.clj on classpath: (NO_SOURCE_FILE:0)" when I try to complete the (compile 'app.first) step.
The only difference between my attempt and the link is the name of my file (first.clj instead of hello.clj).
Can anyone see where I am going wrong? Or for that matter, all I want to do is learn how to create a jar from a clojure file, so if anyone knows of a better/easier way to do that, let me know.
It's better to use Leiningen for such tasks - it allows to maintain dependencies, and packs all necessary components into jar file
I'm rusty on this, but I heard about other people with similar problems.
I think it's helpful to remember that the classpath you indicate points to the root of your class tree, and package names end up creating subdirectories within that tree. Awkwardly stated, but I hope you get the idea. Thus, I think you need to do some kind of gymnastics with creating directories to match the "app.first" -> "/app/first" hierarchy.
Sorry, that's as close as I come to a sensible and useful answer. Hope this helps you.
EDIT:
The Prime Directive of Computer Science: It only works if you do everything right! I spent almost 10 minutes fiddling with this but was finally successful.
Here's what I needed to do to get your program to compile:
created a directory app, and within that, first.clj with your code.
checked for the *compile-path* by doing (pr *compile-path) within Clojure. It said "classes".
created a second directory classes parallel to app.
in the shell, did export CLASSPATH=.:./classes
in Clojure, did (compile 'app.first)
... and I found a bunch of class files in classes. JARring those should be a snap.
I found it very helpful to run (doc compile) because that reminded me of the requirement to have a directory to satisfy the requirement for a *compile-path*.