I what the idiomatic way of handling optional query parameters in Compojure and assign undefined query parameters to a default value.
I've tried this (which perhaps obviously doesn't work):
(GET "/something/:id" [id q :<< as-int :or {q (System/currentTimeMillis)}]
....)
I'd like this route to match both:
curl /something/123
and
curl /something/123?q=234542345
(Note a similar question has been posted here but it uses Swagger)
There are lots of options you could take. This is one I might choose.
(GET "/something/:id" req
(let [{:keys [id q] :or {q (System/currentTimeMillis)}} (:params req)]
,,,))
Ultimately, though, which one I chose would depend on whatever led to the most readable (a subjective measurement, but that's the way it goes) code.
Related
TL;DR: Is the following a good pattern for a library?
(def ^{:dynamic true} *var*)
(defn my-fn [{:keys [var]}]
(do-smth (or var *var*)))
--
Say I want to write a sentiment analysis library.
Is it good design in get-sentiment fn to accept optional sentiment labels but provide default one as dynamic var?
(def ^:dynamic *sentiment-level-labels*
["Very Negative" "Negative" "Neutral" "Positive" "Very Positive"])
;;...
(defn get-sentiment-scores
"Takes text and gives back a 0 to 4 sentiment score for each sentences."
[text]
;;...)
(defn get-sentiment
"Gives back a sentiment map with sentences scores,
average score, rounded score and labeled score.
Can accepts custom sentiment level labels under :labels opt."
[text & {:keys [labels]}]
(let [scores (get-sentiment-scores text)
average-score (get-average scores)
rounded-score (Math/round average-score)
label (get (or labels *sentiment-level-labels*) rounded-score)]
{:scores scores
:average-score average-score
:rounded-score rounded-score
:label label}))
Clojure library coding standards official page says:
If you present an interface that implicitly passes a parameter via
dynamic binding (e.g. db in sql), also provide an identical interface
but with the parameter passed explicitly.
https://dev.clojure.org/display/community/Library+Coding+Standards
In my example, I provided only one interface but with opt argument.
Is this okay? Are there better ways to handle this?
Thank you!
Dynamic vars are full or pitfalls. They push your API code towards implicit environmental coupling, and often force your calling code to add a lot of (binding ...) clauses, which kind of defeats the purpose of concision for using Dynamic vars in the first place. They also lead to tricky edge cases if control is passed from one thread to another.
In your case, I would recommend simply passing the labels in a params map argument:
(def default-sentiment-level-labels
["Very Negative" "Negative" "Neutral" "Positive" "Very Positive"])
(defn get-sentiment
"Gives back a sentiment map with sentences scores,
average score, rounded score and labeled score.
Can accepts custom sentiment level labels under :labels opt."
[text {:as params, :keys [labels] :or {labels default-sentiment-labels}}]
...))
Note that the usage of a map can be interesting, because a map is opaque to intermediaries: you can have various components of your algorithm read only from the params map the keys that concern them.
Im sure I have read somwhere how it is possible to use the reconciler to test query expressions in Om Next directly but im not able to find the source again or figure out if this is possible based on the Om documentation. Is this possible to do so and if it is, how?
What I have right now to test is using the parser but I was hoping for a better way using the reconciler:
(parser {:state (atom state)} (om/get-query MyQuery))
This is how I currently find the value of top level keywords:
(defn query [kw]
(let [res (my-parser {:state my-reconciler} `[[~kw _]])]
(when (not-empty res) (apply val res))))
So in your case you could try:
(my-parser {:state my-reconciler} (om/get-query MyQuery))
It looks like the value for :state can either be a state you give it as in your example, or the reconciler itself as in my example.
It depends on what you mean by "test query expressions in Om Next directly"? The code you wrote above is the only way to check how the parser will interpret the query you give it.
If you're wanting to see how the app state will be normalized and denormalized using the queries you provide, maybe the documentation for idents and om/tree->db is closer to what you're looking for.
I am trying to 'purify' some of my Clojure functions. I would like to get to a point where all my side-effecting code is explicitly declared in one function. It's easy to get some data at the start and to write it to a db at the end and just have a pure function transforming in between. However, the usual situation is that the transformation function requires another DB read somewhere in the middle of the logic:
(defn transform-users
[users]
(let [ids (map :id users)
profiles (db/read :profiles ids)]
(profiles->something profiles)))
(->> (db/read :users)
(transform-users)
(db/write :something)
Obviously this is a very simple example but the point is, how do I get the side-effecting db/read function out of there, how can I make transform-users pure (and as a benefit, easily testable)?
One thing you could do here would be a dependency-injection-like approach of supplying the (potentially) side-effectful function as an optional parameter, e.g.:
(defn transform-users
[users & {:keys [ids->profiles]
:or {ids->profiles #(db/read :profiles %)}]
(let [ids (map :id users)
profiles (ids->profiles ids)]
(profiles->something profiles)))
This should be easily testable since you can mock the injected functions without a lot of effort. And as a bonus, by supplying the default value, you're documenting what you're expecting and making the function convenient to call.
Why couple the reading of the profiles with transforming profiles?
(->> (db/read :users)
(map :id)
(db/read :profiles)
(profile->something)
(db/write :something)
(This also exposes the fact that you are doing two round trips to the db. Where is db/read :user-profiles ?)
(->> (db/read :user-profiles)
(profile->something)
(db/write :something)
or perhaps:
(->> (read-profiles-from-users)
(profile->something)
(db/write :something)
I am working through the first edition of this book and while I enjoy it, some of the examples given seem out-dated. I would give up and find another book to learn from, but I am really interested in what the author is talking about and want to make the examples work for myself, so I am trying to update them as I go along.
The following code is a map/reduce approach to analyzing text that depends on clojure.contrib. I have tried changing the .split function to re-seq with #"\w+", used line-seq instead of read-lines, and changed the .toLowerCase to string/lower-case. I tried to follow my problems to the source code and read the docs thoroughly to learn that the read-lines function closes after you consume the entire sequence and that line-seq returns a lazy sequence of strings, implementing java.io.BufferedReader. The most helpful thing for my problem was post about how to read files after clojure 1.3. Even still, I can't get it to work.
So here's my question: What dependencies and/or functions do I need to change in the following code to make it contemporary, reliable, idiomatic Clojure?
First namespace:
(ns chapter-data.word-count-1
(:use clojure.contrib.io
clojure.contrib.seq-utils))
(defn parse-line [line]
(let [tokens (.split (.toLowerCase line) " ")]
(map #(vector % 1) tokens)))
(defn combine [mapped]
(->> (apply concat mapped)
(group-by first)
(map (fn [[k v]]
{k (map second v)}))
(apply merge-with conj)))
(defn map-reduce [mapper reducer args-seq]
(->> (map mapper args-seq)
(combine)
(reducer)))
(defn sum [[k v]]
{k (apply + v)})
(defn reduce-parsed-lines [collected-values]
(apply merge (map sum collected-values)))
(defn word-frequency [filename]
(map-reduce parse-line reduce-parsed-lines (read-lines filename)))
Second namespace:
(ns chapter-data.average-line-length
(:use rabbit-x.data-anal
clojure.contrib.io))
(def IGNORE "_")
(defn parse-line [line]
(let [tokens (.split (.toLowerCase line) " ")]
[[IGNORE (count tokens)]]))
(defn average [numbers]
(/ (apply + numbers)
(count numbers)))
(defn reducer [combined]
(average (val (first combined))))
(defn average-line-length [filename]
(map-reduce parse-line reducer (read-lines filename)))
But when I compile and run it in light table I get a bevy of errors:
1) In the word-count-1 namespace I get this when I try to reload the ns function after editing:
java.lang.IllegalStateException: spit already refers to: #'clojure.contrib.io/spit in namespace: chapter-data.word-count-1
2) In the average-line-length namespace I get similar name collision errors under the same circumstances:
clojure.lang.Compiler$CompilerException: java.lang.IllegalStateException: parse-line already refers to: #'chapter-data.word-count-1/parse-line in namespace: chapter-data.average-line-length, compiling:(/Users/.../average-line-length.clj:7:1)
3) Oddly, when I quit and restart light table, copy and paste the code directly into the files (replacing what's there) and call instances of their top level functions the word-count-1 namespace runs fine, giving me the number of occurrences of certain words in the test.txt file but the average-line-length name-space gives me this:
"Warning: *default-encoding* not declared dynamic and thus is not dynamically rebindable, but its name suggests otherwise. Please either indicate ^:dynamic *default-encoding* or change the name. (clojure/contrib/io.clj:73)...
4) At this point when I call the word-frequency functions of the first namespace it returns nil instead of the number of word occurrences and when I call the average-line-length function of the second namespace it returns
java.lang.NullPointerException: null
core.clj:1502 clojure.core/val
As far as I can tell, clojure.contrib.io and clojure.contrib.seq-utils are no longer updated, and in fact they may be conflicting with clojure.core functions like spit. I would recommend taking out those dependencies and seeing if you can do this using only core functions. spit should just work -- the error that you're getting is caused by useing clojure.contrib.io, which contains its own spit function, which looks to be roughly equivalent; perhaps the current version in clojure.core is a "new and improved" version of clojure.contrib.io/spit.
Your problem with the parse-line function looks to be caused by the fact that you've defined two functions with the same name, in two different namespaces. The namespaces don't depend on one another, but you can still run into a conflict if you load both namespaces in a REPL. If you only need to use one at a time, try using one of them, and then when you want to use the other one, make sure you do a (remove-ns name-of-first-ns) first to free up the vars so there is no conflict. Alternatively, you could make parse-line a private function in each namespace, by changing (defn parse-line ... to (defn- parse-line ....
EDIT: If you still need any functions that were in clojure.contrib.io or clojure.contrib.seq-utils that aren't available in core or elsewhere, you can always copy the source over into your namespace. See clojure.contrib.io and clojure.contrib.seq-utils on github.
So I am using congomongo (the fetch function near the end) to pull some documents from a mongodb collection. I want to pass options through to the fetch call, so i can do something like (posts :limit 1) and have {:limit 1} get passed through to fetch. I am doing hand rolled "memoization" with #posts, because i want to be able to reset the cache, which to my understanding cant be done with clojure.core/memoize.
Now, the problem I see here is that (fetch :posts options) call is non trivial, and I would really rather not hammer my datastore if dosync has to retry the transaction. I am a total clojure/fp noob though, and I am not sure how to get around that problem. Also, since I am a noob, if I am doing anything else here that makes you cringe, I would love to find out how to write this properly.
(def posts (ref nil))
(defn reset-posts [] (dosync alter posts nil))
(defn fetch-posts [& options]
(let [options (apply array-map options)]
(or #posts
(dosync alter posts (fetch :posts options)))))
I'm not convinced that your transaction blocks ((dosync alter...) do what you think!
user=> (def posts (ref nil))
#'user/posts
user=> (dosync (ref-set posts [1 2 3 4 5]))
[1 2 3 4 5]
user=> #posts
[1 2 3 4 5]
user=> (dosync alter posts nil)
nil
user=> #posts
[1 2 3 4 5]
In reset-posts, you probably want (dosync (ref-set posts nil)), and in fetch-posts, the syntax fix would be (dosync (ref-set posts (fetch :posts options))).
However, there's a race condition in fetch-posts, a check-then-act. Might not be that big of a deal; not sure who uses fetch-posts, but moving the or #posts bit inside the transaction would avoid a situation where 2 concurrent transactions both end up committing the alter.
With regard to retries of fetch-posts, yeah, that could happen, though your cache solution avoids most of them. I'm not sure there's a way around it without locking, though. Usually with I/O stuff in transactions you'd farm it out to an agent, but the transaction's success depends on the return value from fetch, so it's not clear to me how that'd work.
So you're introducing the ref because you want to be able to not blow up memory when time passes, 'cause just using memoize around fetch-posts may lead to this, sooner or later, right ?
Maybe you could try an alternate approach : let fetch-posts be "pure", memoize-free. With this scenario, someone can call fetch-posts blindlessly, without having to fear OutOfMemoryExceptions.
Indeed, maybe for some usecases, it may be sufficient to "cache the value" in a local of the calling code.
But the story does not end here, or I would not have taken the time to answer :-) : you can pretty easily have your "localized in time" memoize by rebinding fetch-posts using clojure.core/binding : from then, all the code in the same thread in the call stack will benefit from the bound memoized fetch-posts.
If you're using clojure 1.3 alpha, you'll need to declare the fetch-posts var as rebindable explicitly via the :dynamic metadata.
;; most simple definition
(defn ^:dynamic fetch-posts [& options]
(let [options (apply array-map options)]
(fetch :posts options)))
;; a la carte caching by the calling code (lexically scoped)
(let [posts (apply fetch-posts options)] ...)
;; a la carte caching by the calling code (dynamically scoped)
(binding [fetch-posts (memoize fetch-posts)] ...)
My last guess would be that you'd want to "memoize" in posts, in your original version, by indexing the posts by a key which would be the options seq, right ? Some maybe your code was not right ? (or you made the assumption that fetch-posts would always be called with the same args over and over ?)
Another idea. Use an agent to serialize write-access to posts and then ensure the call to fetch is only done when it is nil :
(def posts (agent nil))
(defn reset-posts [] (send posts (constantly nil)))
(defn fetch-posts [& options]
(let [options (apply array-map options)]
(send-off posts #(or % (fetch :posts options)))
(await-for (Long/MAX_VALUE) posts)
#posts))
Another approach which might be useful to move extensive computations outside the dosync is to use delay.
(defn fetch-posts
[& options]
#(dosync (or #posts (ref-set posts (delay (apply fetch :posts options))))))
Also note that your original code is not thread-safe since you access the ref outside the dosync and modify it based on this value afterwards in the dosync. But the value might have changed already between the deref and the dosync. Eg. by another thread calling fetch-posts in parallel.
Also the agent approach is questionable, because you cannot reliably read an agent. The value you get is consistent, but the access is not synchronised. Consider Laurent's example: between await-for and the deref another thread might already call reset-posts and you get nil instead of the post data. In this example this is probably a) far fetched and b) maybe a case one has to consider anyway, but there might be other use cases where this introduces a subtle race condition in more critical code.
tl;dr: Be careful what you do! Clojure is not magically thread-safe. Reason thoroughly about your solution and be aware of the implications.