Clojure immutability practice - clojure

I´m trying to understand the Clojure´s immutability best practice and I have this simple example where I constantly re-declaring(updating) "new-orders" but I´m not sure if this is the right way.
(defrecord Order [fplate splate])
(def new-orders clojure.lang.PersistentQueue/EMPTY)
(defn add-order [orders order]
(conj orders order))
(defn cook [order] ())
(defn cook-order [orders]
(cook (first orders)) (pop orders))
;;order1
(def o1 (->Order "Soup" "Fish&Chips"))
(def new-orders (add-order new-orders o1))
;;order2
(def o2 (->Order "Salad" "Hamburger"))
(def new-orders (add-order new-orders o2))
;;order3
(def o3 (->Order "Rice" "Steak"))
(def new-orders (add-order new-orders o3))
;;cook order
(def new-orders (cook-order new-orders))
(peek new-orders)
Thanks,
R.

This definitely is not the right way to do functional programming in Clojure, or any other FP language. Your code is still essentially imperative, though you have used Clojure as the implementation language.
The right way is to create new data structure with modified content whenever you need a new version of some data structure. Something like:
(let [a []
b (conj a "order1")
c (conj b "order2")]
(println c))
In this case conj creates a new data structure by adding a new element to the structure given as parameter to conj. The old structure is not modified in any way, which means it's immutable.
If you really really need to have some sort of state, Clojure has primitives for that, like atom for example. But first try to write functional code without a modifiable central state.

Here is a more typical way of doing this example. It uses the spyx-pretty function from the Tupelo library, but you could substitute println if you want:
(ns tst.demo.core
(:require [tupelo.core :as t] ))
(defrecord Order [fplate splate])
(def orders-queue (atom []))
(defn add-order [order]
(swap! orders-queue conj order))
(defn cook [order] (println "cooking: " (pr-str order)))
(add-order (->Order "Soup" "Fish&Chips")) ; order1
(t/spyx-pretty orders-queue)
(add-order (->Order "Salad" "Hamburger")) ; order2
(t/spyx-pretty orders-queue)
(add-order (->Order "Rice" "Steak")) ; order3
(t/spyx-pretty orders-queue)
; cook orders
(newline)
(doseq [order #orders-queue]
(cook order))
with results:
orders-queue =>
#<Atom#4147f771: [{:fplate "Soup", :splate "Fish&Chips"}]>
orders-queue =>
#<Atom#4147f771:
[{:fplate "Soup", :splate "Fish&Chips"}
{:fplate "Salad", :splate "Hamburger"}]>
orders-queue =>
#<Atom#4147f771:
[{:fplate "Soup", :splate "Fish&Chips"}
{:fplate "Salad", :splate "Hamburger"}
{:fplate "Rice", :splate "Steak"}]>
cooking: #tst.demo.core.Order{:fplate "Soup", :splate "Fish&Chips"}
cooking: #tst.demo.core.Order{:fplate "Salad", :splate "Hamburger"}
cooking: #tst.demo.core.Order{:fplate "Rice", :splate "Steak"}

Related

How can I filter JSON data by a given date in Clojure?

I have many JSON objects, and I am trying to filter those objects by the date. These objects are being parsed from several JSON files using Cheshire.core, meaning that the JSON objects are in a collection. The date is being passed in in the following format "YYYY-MM-DD" (eg. 2015-01-10). I have tried using the filter and contains? functions to do this, but I am having no luck so far. How can I filter these JSON objects by my chosen date?
Current Clojure code:
(def filter-by-date?
(fn [orders-data date-chosen]
(contains? (get (get orders-data :date) :date) date-chosen)))
(prn (filter (filter-by-date? orders-data "2017-12-25")))
Example JSON object:
{
"id":"05d8d404-b3f6-46d1-a0f9-dbdab7e0261f",
"date":{
"date":"2015-01-10T19:11:41.000Z"
},
"total":{
"GBP":57.45
}
}
JSON after parsing with Cheshire:
[({:id "05d8d404-b3f6-46d1-a0f9-dbdab7e0261f",
:date {:date "2015-01-10T19:11:41.000Z"},
:total {:GBP 57.45}}) ({:id "325bd04-b3f6-46d1-a0f9-dbdab7e0261f",
:date {:date "2015-02-23T10:15:14.000Z"},
:total {:GBP 32.90}})]
First, I'm going to assume you've parsed the JSON first into something like this:
(def parsed-JSON {:id "05d8d404-b3f6-46d1-a0f9-dbdab7e0261f",
:date {:date "2015-01-10T19:11:41.000Z"},
:total {:GBP 57.45}})
The main problem is the fact that the date as stored in the JSON contains time information, so you aren't going to be able to check it directly using equality.
You can get around this by using clojure.string/starts-with? to check for prefixes. I'm using s/ here as an alias for clojure.string:
(defn filter-by-date [date jsons]
(filter #(s/starts-with? (get-in % [:date :date]) date)
jsons))
You were close, but I made a few changes:
You can't use contains? like that. From the docs of contains?: Returns true if key is present in the given collection, otherwise returns false. It can't be used to check for substrings; it's used to test for the presence of a key in a collection.
Use -in postfix versions to access nested structures instead of using multiple calls. I'm using (get-in ...) here instead of (get (get ...)).
You're using (def ... (fn [])) which makes things more complicated than they need to be. This is essentially what defn does, although defn also adds some more stuff as well.
To address the new information, you can just flatten the nested sequences containing the JSONs first:
(->> nested-json-colls ; The data at the bottom of the question
(flatten)
(filter-by-date "2015-01-10"))
#!/usr/bin/env boot
(defn deps [new-deps]
(merge-env! :dependencies new-deps))
(deps '[[org.clojure/clojure "1.9.0"]
[cheshire "5.8.0"]])
(require '[cheshire.core :as json]
'[clojure.string :as str])
(def orders-data-str
"[{
\"id\":\"987654\",
\"date\":{
\"date\":\"2015-01-10T19:11:41.000Z\"
},
\"total\":{
\"GBP\":57.45
}
},
{
\"id\":\"123456\",
\"date\":{
\"date\":\"2016-01-10T19:11:41.000Z\"
},
\"total\":{
\"GBP\":23.15
}
}]")
(def orders (json/parse-string orders-data-str true))
(def ret (filter #(clojure.string/includes? (get-in % [:date :date]) "2015-01-") orders))
(println ret) ; ({:id 987654, :date {:date 2015-01-10T19:11:41.000Z}, :total {:GBP 57.45}})
You can convert the date string to Date object using any DateTime library like joda-time and then do a proper filter if required.
clj-time has functions for parsing strings and comparing date-time objects. So you could do something like:
(ns filter-by-time-example
(:require [clj-time.coerce :as tc]
[clj-time.core :as t]))
(def objs [{"id" nil
"date" {"date" "2015-01-12T19:11:41.000Z"}
"total" nil}
{"id" "05d8d404-b3f6-46d1-a0f9-dbdab7e0261f"
"date" {"date" "2015-01-10T19:11:41.000Z"}
"total" {"GBP" :57.45}}
{"id" nil
"date" {"date" "2015-01-11T19:11:41.000Z"}
"total" nil}])
(defn filter-by-day
[objs y m d]
(let [start (t/date-time y m d)
end (t/plus start (t/days 1))]
(filter #(->> (get-in % ["date" "date"])
tc/from-string
(t/within? start end)) objs)))
(clojure.pprint/pprint (filter-by-day objs 2015 1 10)) ;; Returns second obj
If you're going to repeatedly do this (e.g. for multiple days) you could parse all dates in your collection into date-time objects with
(map #(update-in % ["date" "date"] tc/from-string) objs)
and then just work with that collection to avoid repeating the parsing step.
(ns filter-by-time-example
(:require [clj-time.format :as f]
[clj-time.core :as t]
[cheshire.core :as cheshire]))
(->> json-coll
(map (fn [json] (cheshire/parse-string json true)))
(map (fn [record] (assoc record :dt-date (f/format (get-in record [:date :date])))))
(filter (fn [record] (t/after? (tf/format "2017-12-25") (:dt-date record))))
(map (fn [record] (dissoc record :dt-date))))
Maybe something like this? You might need to change the filter for your usecase but as :dt-time is now a jodo.DateTime you can leverage all the clj-time predicates.

Howto include clojure.spec'd functions in a test suite

Is there anyway to include clojure.spec'd functions in a generalized test suite? I know we can register specs and directly spec functions.
(ns foo
(:require [clojure.spec :as s]
[clojure.spec.test :as stest]))
(defn average [list-sum list-count]
(/ list-sum list-count))
(s/fdef average
:args (s/and (s/cat :list-sum float? :list-count integer?)
#(not (zero? (:list-count %))))
:ret number?)
And later, if I want to run generative tests against that spec'd function, I can use stest/check.
=> (stest/check `average)
({:spec #object[clojure.spec$fspec_impl$reify__14282 0x68e9f37c "clojure.spec$fspec_impl$reify__14282#68e9f37c"], :clojure.spec.test.check/ret {:result true, :num-tests 1000, :seed 1479587517232}, :sym edgar.core.analysis.lagging/average})
But i) is there anyway to include these test runs in my general test suite? I'm thinking of the kind of clojure.test integration that test.check has. The closest thing that I can see ii) is the stest/instrument (see here) function. But that seems to just let us turn on checking at the repl. Not quite what I want. Also, iii) are function specs registered?
(defspec foo-test
100
;; NOT this
#_(prop/for-all [v ...]
(= v ...))
;; but THIS
(stest/some-unknown-spec-fn foo))
Ok, solved this one. Turns out there's no solution out of the box. But some people on the clojure-spec slack channel have put together a defspec-test solution for clojure.spec.test and clojure.test.
So given the code in the question. You can A) define the defspec-test macro that takes your test name and a list of spec'd functions. You can then B) use it in your test suite.
Thanks Clojure community!! And hopefully such a utility function makes it into the core library.
A)
(ns foo.test
(:require [clojure.test :as t]
[clojure.string :as str]))
(defmacro defspec-test
([name sym-or-syms] `(defspec-test ~name ~sym-or-syms nil))
([name sym-or-syms opts]
(when t/*load-tests*
`(def ~(vary-meta name assoc
:test `(fn []
(let [check-results# (clojure.spec.test/check ~sym-or-syms ~opts)
checks-passed?# (every? nil? (map :failure check-results#))]
(if checks-passed?#
(t/do-report {:type :pass
:message (str "Generative tests pass for "
(str/join ", " (map :sym check-results#)))})
(doseq [failed-check# (filter :failure check-results#)
:let [r# (clojure.spec.test/abbrev-result failed-check#)
failure# (:failure r#)]]
(t/do-report
{:type :fail
:message (with-out-str (clojure.spec/explain-out failure#))
:expected (->> r# :spec rest (apply hash-map) :ret)
:actual (if (instance? Throwable failure#)
failure#
(:clojure.spec.test/val failure#))})))
checks-passed?#)))
(fn [] (t/test-var (var ~name)))))))
B)
(ns foo-test
(:require [foo.test :refer [defspec-test]]
[foo]))
(defspec-test test-average [foo/average])
The above example can fail in the case where :failure is false due to how stest/abbrev-result tests for failure. See CLJ-2246 for more details. You can work around this by defining your own version of abbrev-result. Also, the formatting of failure data has changed.
(require
'[clojure.string :as str]
'[clojure.test :as test]
'[clojure.spec.alpha :as s]
'[clojure.spec.test.alpha :as stest])
;; extracted from clojure.spec.test.alpha
(defn failure-type [x] (::s/failure (ex-data x)))
(defn unwrap-failure [x] (if (failure-type x) (ex-data x) x))
(defn failure? [{:keys [:failure]}] (not (or (true? failure) (nil? failure))))
;; modified from clojure.spec.test.alpha
(defn abbrev-result [x]
(let [failure (:failure x)]
(if (failure? x)
(-> (dissoc x ::stc/ret)
(update :spec s/describe)
(update :failure unwrap-failure))
(dissoc x :spec ::stc/ret))))
(defn throwable? [x]
(instance? Throwable x))
(defn failure-report [failure]
(let [expected (->> (abbrev-result failure) :spec rest (apply hash-map) :ret)]
(if (throwable? failure)
{:type :error
:message "Exception thrown in check"
:expected expected
:actual failure}
(let [data (ex-data (get-in failure
[::stc/ret
:result-data
:clojure.test.check.properties/error]))]
{:type :fail
:message (with-out-str (s/explain-out data))
:expected expected
:actual (::s/value data)}))))
(defn check?
[msg [_ body :as form]]
`(let [results# ~body
failures# (filter failure? results#)]
(if (empty? failures#)
[{:type :pass
:message (str "Generative tests pass for "
(str/join ", " (map :sym results#)))}]
(map failure-report failures#))))
(defmethod test/assert-expr 'check?
[msg form]
`(dorun (map test/do-report ~(check? msg form))))
Here's a slightly modified version of grzm's excellent answer that works with [org.clojure/test.check "0.10.0-alpha4"]. It uses the new :pass? key that comes from this PR: https://github.com/clojure/test.check/commit/09927b64a60c8bfbffe2e4a88d76ee4046eef1bc#diff-5eb045ad9cf20dd057f8344a877abd89R1184.
(:require [clojure.test :as t]
[clojure.string :as str]
[clojure.spec.alpha :as s]
[clojure.spec.test.alpha :as stest])
(alias 'stc 'clojure.spec.test.check)
;; extracted from clojure.spec.test.alpha
(defn failure-type [x] (::s/failure (ex-data x)))
(defn unwrap-failure [x] (if (failure-type x) (ex-data x) x))
;; modified from clojure.spec.test.alpha
(defn abbrev-result [x]
(if (-> x :stc/ret :pass?)
(dissoc x :spec ::stc/ret)
(-> (dissoc x ::stc/ret)
(update :spec s/describe)
(update :failure unwrap-failure))))
(defn throwable? [x]
(instance? Throwable x))
(defn failure-report [failure]
(let [abbrev (abbrev-result failure)
expected (->> abbrev :spec rest (apply hash-map) :ret)
reason (:failure abbrev)]
(if (throwable? reason)
{:type :error
:message "Exception thrown in check"
:expected expected
:actual reason}
(let [data (ex-data (get-in failure
[::stc/ret
:shrunk
:result-data
:clojure.test.check.properties/error]))]
{:type :fail
:message (with-out-str (s/explain-out data))
:expected expected
:actual (::s/value data)}))))
(defn check?
[msg [_ body :as form]]
`(let [results# ~body
failures# (remove (comp :pass? ::stc/ret) results#)]
(if (empty? failures#)
[{:type :pass
:message (str "Generative tests pass for "
(str/join ", " (map :sym results#)))}]
(map failure-report failures#))))
(defmethod t/assert-expr 'check?
[msg form]
`(dorun (map t/do-report ~(check? msg form))))
Usage:
(deftest whatever-test
(is (check? (stest/check `whatever
;; optional
{:clojure.spec.test.check/opts {:num-tests 10000}})))

Is equality properly defined on chan?

I want to maintain a collection of channels, with the ability to add and remove channels. Is the equality defined so I can conj and disj correctly?
In other words, will this always work?
=> (def chan-collection (atom #{}))
=> (def my-chan-1 (chan))
=> (def my-chan-2 (chan))
=> #chan-collection
#{}
=> (swap! chan-collection conj my-chan-1)
=> #chan-collection
#{#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel#6ec3a2f6>}
=> (swap! chan-collection conj my-chan-2)
=> #chan-collection
#{#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel#382830a1>
#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel#6ec3a2f6>}
=> (swap! chan-collection disj my-chan-1)
=> #chan-collection
#{#<ManyToManyChannel clojure.core.async.impl.channels.ManyToManyChannel#382830a1>}
=> (swap! chan-collection disj my-chan-2)
=> #chan-collection
#{}
Yes this is true, and changing it would break everything.
Channels are identical? if they are the same chan object, and therefore are equal in all cases. All other comparisons of chans are explicitly not equal, and this is good for you. You want removal to remove the exact chan you ask to be removed rather than some equivalent chan with the same contents. So it's fortunate that chans that are not identical are also not equal
user> (= (chan) (chan))
false
user> (identical? (chan) (chan))
false
user> (identical? my-chan-1 (chan))
false
user> (identical? my-chan-1 my-chan-1)
true
user>
user> (= my-chan-1 my-chan-1)
true
In the general "Clojure world" this same property is true of all things that are Identities rather than values. Identities have values that change over time so it doesn't make sense to say that two identities are equal if they happen to contain the same value at the moment you ask, even though this may only be true for you and never for anyone else. comparing the values in identities makes much more sense. for example like chans, atoms with the same values are also not equal and this is a fundamental property of clojure that will never change.
user> (let [a (atom 1)]
(= a a))
true
user> (= (atom 1) (atom 1))
false
Provided you want to remove them by giving the exact chan you want removed as an argument to disj, as you are doing above, rather than some other notion like "remove the channel with 42 in it"
If we do the same setup:
user> (require '[clojure.core.async :refer [<! <!! >! chan]])
nil
user> (def chan-collection (atom #{}))
#'user/chan-collection
user> (def my-chan-1 (chan))
#'user/my-chan-1
user> (def my-chan-2 (chan))
#'user/my-chan-2
user> (swap! chan-collection conj my-chan-1 my-chan-2)
#{#object[clojure.core.async.impl.channels.ManyToManyChannel 0x35b61c71 "clojure.core.async.impl.channels.ManyToManyChannel#35b61c71"] #object[clojure.core.async.impl.channels.ManyToManyChannel 0x240e86d5 "clojure.core.async.impl.channels.ManyToManyChannel#240e86d5"]}
and then ask to have "the empty chan" removed:
user> (swap! chan-collection disj (chan))
#{#object[clojure.core.async.impl.channels.ManyToManyChannel 0x35b61c71 "clojure.core.async.impl.channels.ManyToManyChannel#35b61c71"] #object[clojure.core.async.impl.channels.ManyToManyChannel 0x240e86d5 "clojure.core.async.impl.channels.ManyToManyChannel#240e86d5"]}
we can verify that it does nothing.

Using clojure schedulers

How to evaluate a function correctly every minute using at-at and chime?
Here are my tests:
(require '[overtone.at-at :refer :all]
'[chime :refer [chime-at]]
'[clj-time.periodic :refer [periodic-seq]]
'[clj-time.core :as t])
;; 1. Use of future
(defonce data1 (atom {:num 1}))
(defonce updater
(future
(while true
(swap! data1 update-in [:num] inc)
(Thread/sleep 60000))))
;; 2. Using at-at
(defonce data2 (atom {:num 1}))
(def my-pool (mk-pool))
(every 60000 #(swap! data2 update-in [:num] inc) my-pool)
;; 3. Using chime
(defonce data3 (atom {:num 1}))
(chime-at (periodic-seq (t/now) (-> 60 t/seconds))
(fn [] (swap! data3 update-in [:num] inc))
{:error-handler (fn [e] (str e))})
After 5 minutes:
#data1
;;=> {:num 5}
#data2
;;=> {:num 8}
#data3
;;=> {:num 1}
Why is at-at counting to fast?
Why is chimenot counting at all?
Thank you!
Not sure what's up with at-at.
As for Chime, chime-at calls the callback function with the time of the current chime, so you'll need to amend your callback to something like
(fn [time] (swap! data3 update-in [:num] inc))
With (fn [] …) you'll get an ArityException at each chime and your :error-handler swallows those. (Chime's default handler prints a stack trace; NB. depending on your setup that stack trace may or may not be visible in your REPL window – for example with a fairly typical Emacs/CIDER setup you might have to switch to an *nrepl-server* buffer to see it.)
(Incidentally, in 1.7 alphas you can use update :num instead of update-in [:num].)

How to do hooks in Clojure

I have a situation where I am creating and destroying objects in one clojure namespace, and want another namespace to co-ordinate. However I do not want the first namespace to have to call the second explicitly on object destruction.
In Java, I could use a listener. Unfortunately the underlying java libraries do not signal events on object destruction. If I were in Emacs-Lisp, then I'd use hooks which do the trick.
Now, in clojure I am not so sure. I have found the Robert Hooke library https://github.com/technomancy/robert-hooke. But this is more like defadvice in elisp terms -- I am composing functions. More over the documentation says:
"Hooks are meant to extend functions you don't control; if you own the target function there are obviously better ways to change its behaviour."
Sadly, I am not finding it so obvious.
Another possibility would be to use add-watch, but this is marked as alpha.
Am I missing another obvious solution?
Example Added:
So First namespace....
(ns scratch-clj.first
(:require [scratch-clj.another]))
(def listf (ref ()))
(defn add-object []
(dosync
(ref-set listf (conj
#listf (Object.))))
(println listf))
(defn remove-object []
(scratch-clj.another/do-something-useful (first #listf))
(dosync
(ref-set listf (rest #listf)))
(println listf))
(add-object)
(remove-object)
Second namespace
(ns scratch-clj.another)
(defn do-something-useful [object]
(println "object removed is:" object))
The problem here is that scratch-clj.first has to require another and explicitly push removal events across. This is a bit clunky, but also doesn't work if I had "yet-another" namespace, which also wanted to listen.
Hence I thought of hooking the first function.
Is this solution suitable to your requirements?
scratch-clj.first:
(ns scratch-clj.first)
(def listf (atom []))
(def destroy-listeners (atom []))
(def add-listeners (atom []))
(defn add-destroy-listener [f]
(swap! destroy-listeners conj f))
(defn add-add-listener [f]
(swap! add-listeners conj f))
(defn add-object []
(let [o (Object.)]
(doseq [f #add-listeners] (f o))
(swap! listf conj o)
(println #listf)))
(defn remove-object []
(doseq [f #destroy-listeners] (f (first #listf)))
(swap! listf rest)
(println #listf))
Some listeners:
(ns scratch-clj.another
(:require [scratch-clj.first :as fst]))
(defn do-something-useful-on-remove [object]
(println "object removed is:" object))
(defn do-something-useful-on-add [object]
(println "object added is:" object))
Init binds:
(ns scratch-clj.testit
(require [scratch-clj.another :as another]
[scratch-clj.first :as fst]))
(defn add-listeners []
(fst/add-destroy-listener another/do-something-useful-on-remove)
(fst/add-add-listener another/do-something-useful-on-add))
(defn test-it []
(add-listeners)
(fst/add-object)
(fst/remove-object))
test:
(test-it)
=> object added is: #<Object java.lang.Object#c7aaef>
[#<Object java.lang.Object#c7aaef>]
object removed is: #<Object java.lang.Object#c7aaef>
()
It sounds a lot like what you're describing is callbacks.
Something like:
(defn make-object
[destructor-fn]
{:destructor destructor-fn :other-data "data"})
(defn destroy-object
[obj]
((:destructor obj) obj))
; somewhere at the calling code...
user> (defn my-callback [o] (pr [:destroying o]))
#'user/my-callback
user> (destroy-object (make-object my-callback))
[:destroying {:destructor #<user$my_callback user$my_callback#73b8cdd5>, :other-data "data"}]
nil
user>
So, here is my final solution following mobytes suggestion. A bit more work, but
I suspect that I will want this in future.
Thanks for all the help
;; hook system
(defn make-hook []
(atom []))
(defn add-hook [hook func]
(do
(when-not
(some #{func} #hook)
(swap! hook conj func))
#hook))
(defn remove-hook [hook func]
(swap! hook
(partial
remove #{func})))
(defn clear-hook [hook]
(reset! hook []))
(defn run-hook
([hook]
(doseq [func #hook] (func)))
([hook & rest]
(doseq [func #hook] (apply func rest))))
(defn phils-hook []
(println "Phils hook"))
(defn phils-hook2 []
(println "Phils hook2"))
(def test-hook (make-hook))
(add-hook test-hook phils-hook)
(add-hook test-hook phils-hook2)
(run-hook test-hook)
(remove-hook test-hook phils-hook)
(run-hook test-hook)