Clojure annotations and Integers - clojure

I am adding Swagger annotations to JaxRs annotated services.
I have the following:
(^{
GET true
Path "/{who}"
ApiOperation {:value "Get a hello" :notes "simple clojure GET"}
Produces ["text/plain; charset=UTF-8"]
ApiResponses {:value [(ApiResponse {:code 200 :message "yay!"})]}
}
If I decompile the produced class the annotations look like this:
#ApiResponses({#com.wordnik.swagger.annotations.ApiResponse(code=200L, message="yay!")})
#Produces({"text/plain; charset=UTF-8"})
#ApiOperation(value="Get a hello", notes="simple clojure GET")
#Path("/{who}")
#GET(true)
notes that in the first annotation code = 200L
During runtime, this value must be an int, and I cannot figure out how to make this happen
if I try
ApiResponses {:value [(ApiResponse {:code (int 200) :message "yay!"})]}
I get a compilation error (using the maven swagger plugin)
Exception in thread "main" java.lang.ClassCastException: clojure.lang.Var cannot be cast to java.lang.Class, compiling:(pocclj/resourceclj.clj:14)
I have tried
(def success (int 200))
...
ApiResponses {:value [(ApiResponse {:code success :message "yay!"})]}
Which produces this compilation error:
Exception in thread "main" java.lang.IllegalArgumentException: Unsupported annotation value: success of class class java.lang.Integer, compiling:(pocclj/resourceclj.clj:14)
I have tried a bunch of other stuff (deref etc) but cant find the secret sauce.
I am fairly new to clojure and desperate for some help on this.
Thanks in advance
Martin

You are setting the type of ':code' correctly. Which can be tested independently:
user> (def something ^{:ApiResponses {:code (int 200) :message "yay!"}} {:some :data :goes :here})
#'user/something
user> (meta something)
{:ApiResponses {:code 200, :message "yay!"}}
user> (-> (meta something) :ApiResponses :code type)
java.lang.Integer
And without the cast the metadata contains the wrong type:
user> (def something-wrong ^{:ApiResponses {:code 200 :message "yay!"}} {:some :data :goes :here})
#'user/something-wrong
user> (meta something)
{:ApiResponses {:code 200, :message "yay!"}}
user> (-> (meta something-wrong) :ApiResponses :code type)
java.lang.Long
From the exception it looks like perhaps the call to ApiResponse is crashing. If ApiResponse is a macro that expects a number and not an s-expression, then I could see it not handling this properly. If it is a function you would need to look into why it is crashing.
If I provide a stub implementation for ApiResponse then It works for me:
user> (definterface Fooi (Something []))
user.Fooi
user> (def ApiResponse identity)
#'user/ApiResponse
user> (deftype Foo []
Fooi
(Something
^{GET true
Path "/{who}"
ApiOperation {:value "Get a hello" :notes "simple clojure GET"}
Produces ["text/plain; charset=UTF-8"]
ApiResponses {:value [(ApiResponse {:code (int 200) :message "yay!"})]}}
[this] (identity this)))
user.Foo

I don't know about ApiResponse, or much about annotations really, but: it looks like some macro (deftype?) is producing annotations for you, and you need it to see 200 as an int. Clojure doesn't have int literals, so the only way to hand an Integer object directly to a macro is through some other macro that calls it. It's not really possible to do this in a nice way; as far as I know you have to either use eval, or be very narrow by aiming specifically at int literals. Here's a sketch of a solution:
user> (use 'clojure.walk)
user> (defmacro with-int-literals [named-ints & body]
(prewalk-replace (into {}
(for [[k v] (partition 2 named-ints)]
[k (int v)]))
`(do ~#body)))
user> (map class (second (macroexpand-1 '(with-int-literals [x 100, y 200] [x y]))))
(java.lang.Integer java.lang.Integer)
So if you wrap your entire deftype (or whatever macro is generating these annotations) with a with-int-literals form, you can produce integers for it instead of longs. I don't actually know that this will work; perhaps there's something in the annotation processor that's fundamentally unable to handle ints for some reason. But at least this way you can offer it ints and hope for the best.
Edit
Since you actually need int literals in metadata, rather than in "ordinary" code, prewalk won't actually look at the data you care about. You'll have to write a version of walk that treats metadata in a sensible way, and then use it instead of prewalk here.

Related

Clojure function self reference to get own metadata

When I attach some metadata to a function and then call it I am not able to access those metadata within that function
(let [I (fn I [x] (println I) (println (meta I)))]
(let [f (with-meta I {:rr 5})]
(println I)
(println f)
(f I)))
I see that the self reference from within the function is not the function instance actually invoked and thus no metadata is available through that self reference. I need the self reference to give me the function instance actually invoked to access those metadata
I think that the problem is that your conflating the value of the function and the identity of the function together. It's a thing many other languages do so it's natural when you're learning Clojure. In your example, I has a reference to itself, and looks up the metadata from that reference, which returns nil. You then create f which is the same as I, but with some metadata. So when you run f it looks up the metadata on I and returns nil. Defining f doesn't change I at all, it just creates a new thing in terms of the old thing. If you want to change something you need to introduce a reference type that you can change. There are several of these, but usually to store functions you'd use a Var (see here for reference)
(defn i [] (meta i))
(i) ;;=> nil
(alter-var-root #'i with-meta {:rr 5})
(i) ;;=> {:rr 5}
Here we define a function in the current namespace called i which just returns it's own metadata. We call it to get nil. Then we alter the global reference with some new metadata, and call it again.
If you wanted a more lexically scoped example, you could use an atom as below:
(let [i (atom nil)
f (fn [] (meta #i))]
(reset! i f)
(prn 'before '>> (#i))
(swap! i with-meta {:rr 5})
(prn 'after '>> (#i)))
However, other than learning how these things fit together, I'm not sure what the goal is. It's probably a bad idea to try and use these structures in a real program that you plan on maintaining.
Rather accidentally, I found a trick that enables functions to read it own metadata. It appears, the Clojure compiler generates metadata support code differently when the original function definition has custom metadata. If it is present, (meta fn-name) works inside the body of the function, otherwise it does not. For example, the following produces the result desired by the OP:
*clojure-version*
;;=> {:major 1, :minor 10, :incremental 0, :qualifier nil}
(let [f1 ^{:foo true} (fn f [] (meta f))
f2 (with-meta f1 {:bar true})]
(prn (f1))
(prn (f2)))
;;=> {:foo true}
;;=> {:bar true}
;;=> nil
We can examine the code generated for a function without the metadata in the original definition - there is just the invoke method
(require '[clojure.pprint :as p])
(let [ff (fn f [] (meta f))]
(p/pprint (seq (.getDeclaredMethods (class ff)))))
;;=> (#object[java.lang.reflect.Method 0x2b56b137 "public java.lang.Object user$eval2171$f__2172.invoke()"])
;;=> nil
And when the metadata is present, additional methods (meta and withMeta) are generated to deal with the metadata.
(let [ff ^:foo (fn f [] (meta f))]
(p/pprint (seq (.getDeclaredMethods (class ff)))))
;;=> (#object[java.lang.reflect.Method 0x3983bd83 "public clojure.lang.IObj user$eval2175$f__2176.withMeta(clojure.lang.IPersistentMap)"]
;;=> #object[java.lang.reflect.Method 0x547d182d "public clojure.lang.IPersistentMap user$eval2175$f__2176.meta()"]
;;=> #object[java.lang.reflect.Method 0x62c3d0fe "public java.lang.Object user$eval2175$f__2176.invoke()"])
;;=> nil
Welcome to Clojure, #xstreamer!
I'm going to suggest something different from what (precisely) you're asking for. I don't know how querying the function's metadata from within the function should work, really. So I'm going to suggest defining the function first, and redefining the function metadata afterwards. This is fairly simple in Clojure.
(defn f
"Boring doc"
[])
(meta #'f)
;; => {:arglists ([]),
;; :doc "Boring doc",
;; :line 32,
;; :column 1,
;; :file "C:/Users/teodorlu/IdeaProjects/th-scratch/src/th/play/core.clj",
;; :name f,
;; :ns #object[clojure.lang.Namespace 0x3b402f0c "th.play.core"]}
Now, redefine it!
(alter-meta! #'f assoc :rr 5)
(meta #'f)
;; => {:arglists ([]),
;; :doc "Boring doc",
;; :line 32,
;; :column 1,
;; :file "C:/Users/teodorlu/IdeaProjects/th-scratch/src/th/play/core.clj",
;; :name f,
;; :ns #object[clojure.lang.Namespace 0x3b402f0c "th.play.core"],
;; :rr 5}
Where assoc sets a value in a map.
(assoc {} :rr 5)
;; {:rr 5}
(assoc {:some :stuff} :more :stuff)
;; {:some :stuff, :more :stuff}
References
If you're confused by the #'f, this is how you get the var representing the binding of f, instead of just the value it refers to. For more information about vars and how to use them, refer to the official reference on vars and the less terse guide from 8th light.

Clojure Spec Not Validating Data Properly

I am super confused by Clojure Spec. When I run in the repl by entering:
(require '[clojure.spec.alpha :as s])
And then add:
(s/valid? even? 10)
I get //true. And when I run:
(s/valid? even? 11)
//False. Ok so that works. Then when I require spec in my core.clj as:
(ns spam-problem.core
(:require [clojure.spec.alpha :as s]
[clojure.spec.gen.alpha :as gen]))
And try a simple validation to get it to throw an error, nothing happens:
(defn -main
"I don't do a whole lot ... yet."
[& args]
(s/valid? even? 11))
I have no idea what I'm doing wrong here and am very confused about how spec is supposed to work. I am running this by using command lein run. Is there another way you're supposed to run it?
I understand what you are feeling because once I got into Spec it caused me the same thoughts. What really helped me to solve the problem in my mind is to considering Spec being not a final library but rather a framework. In my projects, usually I've got a special module with high-level wrappers above basic spec capabilities. I believe, you might do the same: define a function that takes data, spec and raises those error message you desire to have in terms of your business-logic. Here is a small example of my code:
(ns project.spec
(:require [clojure.spec.alpha :as s]))
;; it's better to define that value is a constant
(def invalid :clojure.spec.alpha/invalid)
(defn validate
"Either returns coerced data or nil in case of error."
[spec value]
(let [result (s/conform spec value)]
(if (= result invalid)
nil
result)))
(defn spec-error
"Returns an error map for data structure that does not fit spec."
[spec data]
(s/explain-data spec data))
Now, let's prepare some specs:
(defn x-integer? [x]
(if (integer? x)
x
(if (string? x)
(try
(Integer/parseInt x)
(catch Exception e
invalid))
invalid)))
(def ->int (s/conformer x-integer?))
(s/def :opt.visits/fromDate ->int)
(s/def :opt.visits/toDate ->int)
(s/def :opt.visits/country string?)
(s/def :opt.visits/toDistance ->int)
(s/def :opt.visits/params
(s/keys :opt-un [:opt.visits/fromDate
:opt.visits/toDate
:opt.visits/country
:opt.visits/toDistance]))
And here are some usage examples:
(let [spec :opt.visits/params
data {:some :map :goes :here}]
(if-let [cleaned-data (validate spec data)]
;; cleaned-data has values coerced from strings to integers,
;; quite useful for POST parameters
(positive-logic cleaned-data)
;; error values holds a map that describes an error
(let [error (spec-error spec data)]
(error-logic-goes-here error))))
What might be improved here is to have a combo-function with both validate and error functionality. Such a function could return a vector of two values: success flag and either result or error data structure as follows:
[true {:foo 42}] ;; good result
[false {:error :map}] ;; bad result
The Spec library does not dictate a single way of processing data; that's why it's really good and flexible.
valid? is a predicate that returns true or false. Your program isn’t doing anything with the return value. Try printing it to console or using s/assert if you want to throw an exception:
If (check-asserts?) is false at runtime, always returns x. Defaults to
value of 'clojure.spec.check-asserts' system property, or false if not
set. You can toggle check-asserts? with (check-asserts bool).
So you may need to set (s/check-asserts true) to have s/assert throw exceptions:
(clojure.spec.alpha/assert even? 3)
=> 3
(clojure.spec.alpha/check-asserts?)
=> false
(clojure.spec.alpha/check-asserts true)
=> true
(clojure.spec.alpha/assert even? 3)
ExceptionInfo Spec assertion failed
val: 3 fails predicate: :clojure.spec.alpha/unknown
clojure.core/ex-info (core.clj:4739)

Using Clojure's data structure with MapDB

I tried to use directly Clojure's hashmap with MapDB and ran into weird behaviour. I checked Clojure and MapDB sources and couldn't understand the problem.
First everything looks fine:
lein try org.mapdb/mapdb "1.0.6"
; defining a db for the first time
(import [org.mapdb DB DBMaker])
(defonce db (-> (DBMaker/newFileDB (java.io.File. "/tmp/mapdb"))
.closeOnJvmShutdown
.compressionEnable
.make))
(defonce fruits (.getTreeMap db "fruits-store"))
(do (.put fruits :banana {:qty 2}) (.commit db))
(get fruits :banana)
=> {:qty 2}
(:qty (get fruits :banana))
=> 2
(first (keys (get fruits :banana)))
=> :qty
(= :qty (first (keys (get fruits :banana))))
=> true
CTRL-D
=> Bye for now!
Then I try to access the data again:
lein try org.mapdb/mapdb "1.0.6"
; loading previsously created db
(import [org.mapdb DB DBMaker])
(defonce db (-> (DBMaker/newFileDB (java.io.File. "/tmp/mapdb"))
.closeOnJvmShutdown
.compressionEnable
.make))
(defonce fruits (.getTreeMap db "fruits-store"))
(get fruits :banana)
=> {:qty 2}
(:qty (get fruits :banana))
=> nil
(first (keys (get fruits :banana)))
=> :qty
(= :qty (first (keys (get fruits :banana))))
=> false
(class (first (keys (get fruits :banana))))
=> clojure.lang.Keyword
How come the same keyword be different with respect to = ?
Is there some weird reference problem happening ?
The problem is caused by the way equality of keywords works. Looking at the
implementation of the = function we see that since keywords are not
clojure.lang.Number or clojure.lang.IPersistentCollection their equality is
determined in terms of the Object.equals method. Skimming the source of
clojure.lang.Keyword we learn that keywords don't not override
Object.equals and therefore two keywords are equal iff they are the same
object.
The default serializer of MapDB is org.mapdb.SerializerPojo, a subclass of
org.mapdb.SerializerBase. In its documentation we can read that
it's a
Serializer which uses ‘header byte’ to serialize/deserialize most of classes
from ‘java.lang’ and ‘java.util’ packages.
Unfortunately, it doesn't work that well with clojure.lang classes; It doesn't
preserve identity of keywords, thus breaking equality.
In order to fix it let's attempt to write our own serializer using the
EDN format—alternatively, you could consider, say, Nippy—and use
it in our MapDB.
(require '[clojure.edn :as edn])
(deftype EDNSeralizer []
;; See docs of org.mapdb.Serializer for semantics.
org.mapdb.Serializer
(fixedSize [_]
-1)
(serialize [_ out obj]
(.writeUTF out (pr-str obj)))
(deserialize [_ in available]
(edn/read-string (.readUTF in)))
;; MapDB expects serializers to be serializable.
java.io.Serializable)
(def edn-serializer (EDNSeralizer.))
(import [org.mapdb DB DBMaker])
(def db (.. (DBMaker/newFileDB (java.io.File. "/tmp/mapdb"))
closeOnJvmShutdown
compressionEnable
make))
(def more-fruits (.. db
(createTreeMap "more-fruits")
(valueSerializer (EDNSeralizer.))
(makeOrGet)))
(.put more-fruits :banana {:qty 2})
(.commit db)
Once the more-fruits tree map is reopened in a JVM with EDNSeralizer defined
the :qty object stored inside will be the same object as any other :qty
instance. As a result equality checks will work properly.

Use a get parameter as a function name

this is my clojure-ring handler
(defn handler [req]
(let [distr (get-in req [:query-params "dist"])]
(def sortie (describe (eval ((resolve (symbol distr)) 1 3))
0.5 0.25 3))
{:status 200
:headers {"Content-Type" "text/html"}
:body (str "<p>" (print-str sortie) "<p>")}
))
(def app
(-> #'handler
(ring.middleware.stacktrace/wrap-stacktrace)
(wrap-spy)
(wrap-params)
))
the "dist" key maps to a string.
:query-params {"dist" "gaussian-dist"}
I want to use it as a function name.
The whole
(eval ((resolve (symbol distr)) 1 3))
returns a map
When I replace distr by the string, it works as desired.
=> (eval ((resolve (symbol "gaussian-dist")) 1 3))
{:Distribution :actuarial.distributions/Gaussian, :Parameters {:sigma 3, :mu 1}}
Edit:
This is how I ended up solving my problem:
Thanks again, I'm a beginner at Clojure
I got through it using a macro
(defmacro get-map [map name]
`(get ~map ~name))
(defn handler [req]
(let [distr-name (get-in req [:query-params "dist"])
distr-map ((get-map distributions-map distr-name) 1 3)
sortie (describe distr-map 0.5 0.25 3)
]
{:status 200
:headers {"Content-Type" "text/html"}
:body (str "<p>" sortie "<p>")}))
It seems a bit of a risky proposition from a security and maintainability standpoint to let the Internet call any symbol in your namespace as a function, so my thought would be to instead do something like:
Prepare a white-list of functions users should be able to request, in the form of a map from query parameter strings to functions.
(def distributions {"gaussian-dist" gaussian-dist})
Make a default function which either makes an error message or just picks some default distribution.
When you get a query, just look up the function you need in distributions using get and the default function to handle the case of query strings not found.
(let [distr (get-in req [:query-params "dist"])
sortie ((get distributions distr default-fn) 1 3)]
It's also kinda risky to def a var like sortie every request, as that opens you up to race conditions. That's why it seemed more natural to let that name too unless there's more to the handler than I'm seeing here. You might also be able to do what you want with (def ^:dynamic *sortie*) and binding, which isolates the bindings of different threads from one another to prevent that.

How to parse URL parameters in Clojure?

If I have the request "size=3&mean=1&sd=3&type=pdf&distr=normal" what's the idiomatic way of writing the function (defn request->map [request] ...) that takes this request and
returns a map {:size 3, :mean 1, :sd 3, :type pdf, :distr normal}
Here is my attempt (using clojure.walk and clojure.string):
(defn request-to-map
[request]
(keywordize-keys
(apply hash-map
(split request #"(&|=)"))))
I am interested in how others would solve this problem.
Using form-decode and keywordize-keys:
(use 'ring.util.codec)
(use 'clojure.walk)
(keywordize-keys (form-decode "hello=world&foo=bar"))
{:foo "bar", :hello "world"}
Assuming you want to parse HTTP request query parameters, why not use ring? ring.middleware.params contains what you want.
The function for parameter extraction goes like this:
(defn- parse-params
"Parse parameters from a string into a map."
[^String param-string encoding]
(reduce
(fn [param-map encoded-param]
(if-let [[_ key val] (re-matches #"([^=]+)=(.*)" encoded-param)]
(assoc-param param-map
(codec/url-decode key encoding)
(codec/url-decode (or val "") encoding))
param-map))
{}
(string/split param-string #"&")))
You can do this easily with a number of Java libraries. I'd be hesitant to try to roll my own parser unless I read the URI specs carefully and made sure I wasn't missing any edge cases (e.g. params appearing in the query twice with different values). This uses jetty-util:
(import '[org.eclipse.jetty.util UrlEncoded MultiMap])
(defn parse-query-string [query]
(let [params (MultiMap.)]
(UrlEncoded/decodeTo query params "UTF-8")
(into {} params)))
user> (parse-query-string "size=3&mean=1&sd=3&type=pdf&distr=normal")
{"sd" "3", "mean" "1", "distr" "normal", "type" "pdf", "size" "3"}
Can also use this library for both clojure and clojurescript: https://github.com/cemerick/url
user=> (-> "a=1&b=2&c=3" cemerick.url/query->map clojure.walk/keywordize-keys)
{:a "1", :b "2", :c "3"}
Yours looks fine. I tend to overuse regexes, so I would have solved it as
(defn request-to-keywords [req]
(into {} (for [[_ k v] (re-seq #"([^&=]+)=([^&]+)" req)]
[(keyword k) v])))
(request-to-keywords "size=1&test=3NA=G")
{:size "1", :test "3NA=G"}
Edit: try to stay away from clojure.walk though. I don't think it's officially deprecated, but it's not very well maintained. (I use it plenty too, though, so don't feel too bad).
I came across this question when constructing my own site and the answer can be a bit different, and easier, if you are passing parameters internally.
Using Secretary to handle routing: https://github.com/gf3/secretary
Parameters are automatically extracted to a map in :query-params when a route match is found. The example given in the documentation:
(defroute "/users/:id" [id query-params]
(js/console.log (str "User: " id))
(js/console.log (pr-str query-params)))
(defroute #"/users/(\d+)" [id {:keys [query-params]}]
(js/console.log (str "User: " id))
(js/console.log (pr-str query-params)))
;; In both instances...
(secretary/dispach! "/users/10?action=delete")
;; ... will log
;; User: 10
;; "{:action \"delete\"}"
You can use ring.middleware.params. Here's an example with aleph:
user=> (require '[aleph.http :as http])
user=> (defn my-handler [req] (println "params:" (:params req)))
user=> (def server (http/start-server (wrap-params my-handler)))
wrap-params creates an entry in the request object called :params. If you want the query parameters as keywords, you can use ring.middleware.keyword-params. Be sure to wrap with wrap-params first:
user=> (require '[ring.middleware.params :refer [wrap-params]])
user=> (require '[ring.middleware.keyword-params :refer [wrap-keyword-params])
user=> (def server
(http/start-server (wrap-keyword-params (wrap-params my-handler))))
However, be mindful that this includes a dependency on ring.