In CLojure how to call xml-> with arbitrary preds - clojure

I want to create a function that allows me to pull contents from some feed, here's what I have... zf is from here
(:require
[clojure.zip :as z]
[clojure.data.zip.xml :only (attr text xml->)]
[clojure.xml :as xml ]
[clojure.contrib.zip-filter.xml :as zf]
)
(def data-url "http://api.eventful.com/rest/events/search?app_key=4H4Vff4PdrTGp3vV&keywords=music&location=Belgrade&date=Future")
(defn zipp [data] (z/xml-zip data))
(defn contents[cont & tags]
(assert (= (zf/xml-> (zipp(parsing cont)) (seq tags) text))))
but when I call it
(contents data-url :events :event :title)
I get an error
java.lang.RuntimeException: java.lang.ClassCastException: clojure.lang.ArraySeq cannot be cast to clojure.lang.IFn (NO_SOURCE_FILE:0)

(Updated in response to the comments: see end of answer for ready-made function parameterized by the tags to match.)
The following extracts the titles from the XML pointed at by the URL from the question text (tested at a Clojure 1.5.1 REPL with clojure.data.xml 0.0.7 and clojure.data.zip 0.1.1):
(require '[clojure.zip :as zip]
'[clojure.data.xml :as xml]
'[clojure.data.zip.xml :as xz]
'[clojure.java.io :as io])
(def data-url "http://api.eventful.com/rest/events/search?app_key=4H4Vff4PdrTGp3vV&keywords=music&location=Belgrade&date=Future")
(def data (-> data-url io/reader xml/parse))
(def z (zip/xml-zip data))
(mapcat (comp :content zip/node)
(xz/xml-> z
(xz/tag= :events)
(xz/tag= :event)
(xz/tag= :title)))
;; value of the above right now:
("Belgrade Early Music Festival, Gosta / Purcell: Dido & Aeneas"
"Belgrade Early Music Festival, Gosta / Purcell: Dido & Aeneas"
"Belgrade Early Music Festival, Gosta / Purcell: Dido & Aeneas"
"VIII Early Music Festival, Belgrade 2013"
"Kevlar Bikini"
"U-Recken - Tree of Life Pre event"
"Green Day"
"Smallman - Vrane Kamene (Crows Of Stone)"
"One Direction"
"One Direction in Serbia")
Some comments:
The clojure.contrib.* namespaces are all deprecated. xml-> now lives in clojure.data.zip.xml.
xml-> accepts a zip loc and a bunch of "predicates"; in this context, however, the word "predicate" has an unusual meaning of a filtering function working on zip locs. See clojure.data.zip.xml source for several functions which return such predicates; for an example of use, see above.
If you want to define a list of predicates separately, you can do that too, then use xml-> with apply:
(def loc-preds [(xz/tag= :events) (xz/tag= :event) (xz/tag= :title)])
(mapcat (comp :content zip/node) (apply xz/xml-> z loc-preds))
;; value returned as above
Update: Here's a function which takes the url and keywords naming tags as arguments and returns the content found at the tags:
(defn get-content-from-tags [url & tags]
(mapcat (comp :content zip/node)
(apply xz/xml->
(-> url io/reader xml/parse zip/xml-zip)
(for [t tags]
(xz/tag= t)))))
Calling it like so:
(get-content-from-tags data-url :events :event :title)
gives the same result as the mapcat form above.

Related

No view is being rendered in re-frame app

Here's a simple re-frame app that I tried to create based on the existing example project in re-frame's github repo. But it is only displaying things from the html file. Seems like no event is being dispatched. Can anyone point out what am I doing wrong? Thanks.
(ns simple.core
(:require [reagent.core :as reagent]
[re-frame.core :as rf]
[clojure.string :as str]))
(rf/reg-event-db
:rand
(fn [db [_ _]]
(assoc db :winner ( + 2 (rand-int 3)))))
(rf/reg-sub
:winner
(fn [db _]
(:winner db)))
(def participants ["Alice" "Bob" "Ellie"])
(defn winners-name
[idx]
(get participants idx))
(defn show-winner
[]
[:h1
(winners-name
(#(rf/subscribe [:winner])))])
(defn ui
[]
[:div
[:h1 "Lottery"]
[show-winner]])
(defn ^:export run
[]
(rf/dispatch-sync [:rand])
(reagent/render [ui]
(js/document.getElementById "app")))
The :rand handler will produce nil most times since you are adding 2 to the generated value and the participants vector only has 3 entries.
The issue is caused because of a pair of extra parenthesis around the deref thing. So the function winners-name is treating it as a list instead of an integer.
(winners-name
(#(rf/subscribe [:winner]))

How to convert HTML tag with style to Hiccup? React problems

I'm trying to parse HTML with CSS into Hiccup in a Reagent project. I am using Hickory. When I parse HTML with inline CSS, React throws an exception.
(map
as-hiccup (parse-fragment "<div style='color:red'>test</div>")
)
The above generates [:div {:style color:red} "test"] & Reactjs returns exception from Reactjs:
Violation: The style prop expects a mapping from style properties to values, not a string.
I believe [:div {:style {"color" "red"}} "test"] must be returned instead.
Here is the code view:
(ns main.views.job
(:require [reagent.core :as reagent :refer [atom]]
[hickory.core :refer [as-hiccup parse parse-fragment]]))
(enable-console-print!)
(defn some-view [uid]
[:div
(map as-hiccup (parse-fragment "<div style='color:red'>test</div>"))
])
The whole repo is here and it works. I added the parsing from style tag to a map for React in the core.cljs file:
(ns hickory-stack.core
(:require [clojure.string :as s]
[clojure.walk :as w]
[reagent.core :as reagent :refer [atom]]
[hickory.core :as h]))
(enable-console-print!)
(defn string->tokens
"Takes a string with syles and parses it into properties and value tokens"
[style]
{:pre [(string? style)]
:post [(even? (count %))]}
(->> (s/split style #";")
(mapcat #(s/split % #":"))
(map s/trim)))
(defn tokens->map
"Takes a seq of tokens with the properties (even) and their values (odd)
and returns a map of {properties values}"
[tokens]
{:pre [(even? (count tokens))]
:post [(map? %)]}
(zipmap (keep-indexed #(if (even? %1) %2) tokens)
(keep-indexed #(if (odd? %1) %2) tokens)))
(defn style->map
"Takes an inline style attribute stirng and converts it to a React Style map"
[style]
(tokens->map (string->tokens style)))
(defn hiccup->sablono
"Transforms a style inline attribute into a style map for React"
[coll]
(w/postwalk
(fn [x]
(if (map? x)
(update-in x [:style] style->map)
x))
coll))
;; Test Data
(def good-style "color:red;background:black; font-style: normal ;font-size : 20px")
(def html-fragment
(str "<div style='" good-style "'><div id='a' class='btn' style='font-size:30px;color:white'>test1</div>test2</div>"))
;; Rendering
(defn some-view []
[:div (hiccup->sablono
(first (map h/as-hiccup (h/parse-fragment html-fragment))))])
(reagent/render-component [some-view]
(. js/document (getElementById "app")))

Is there some way to let compojure support type conversion automatically?

Now can use compojure this way:
(GET ["/uri"] [para1 para2]
)
Para1 and para2 are all of type String.
I would like to let it know the type correcttly,like this:
(GET ["/uri"] [^String para1 ^Integer para2]
)
It can convert para1 to be Sting and para2 to Integer.
Is there some library or good way to do this?
This is possible as of Compojure 1.4.0 using the syntax [x :<< as-int]
This is not currently possible with only Compojure.
You could use Prismatic schema coercion.
(require '[schema.core :as s])
(require '[schema.coerce :as c])
(require '[compojure.core :refer :all])
(require '[ring.middleware.params :as rparams])
(def data {:para1 s/Str :para2 s/Int s/Any s/Any})
(def data-coercer (c/coercer data c/string-coercion-matcher ))
(def get-uri
(GET "/uri" r
(let [{:keys [para1 para2]} (data-coercer (:params r))]
(pr-str {:k1 para1 :k2 (inc para2)}))))
(def get-uri-wrapped
(let [keywordizer (fn [h]
(fn [r]
(h (update-in r [:params] #(clojure.walk/keywordize-keys %)))))]
(-> get-uri keywordizer rparams/wrap-params)))
Here is a sample run:
(get-uri-wrapped {:uri "/uri" :query-string "para1=a&para2=3" :request-method :get})
{:status 200,
:headers {"Content-Type" "text/html; charset=utf-8"},
:body "{:k1 \"a\", :k2 4}"}

Hiccup template function

I'm trying to add the following Hiccup template function to my file
(defn d3-page [title js body & {:keys [extra-js] :or {extra-js []}}]
(html5
[:head
[:title title]
(include-css "/css/nv.d3.css"))
(include-css "/css/style.css")]
[:body
(concat
[body]
[(include-js "http://d3js.org/d3.v3.min.js")
(include-js (str "https://raw.github.com"
"/novus/nvd3"
"/master/nv.d3.min.js")]
(map include-js extra-js)
[(include-js "/js/script.js")
(javascript-tag js)])]))
but keep getting an unmatched delimiter when I run lein ring server. This comes from Clojure Data Cookbook, so I am surprised to find an error and suspect the error is just on my end. Below is the rest of the code in the file:
(ns web-viz.web
(:require [compojure.route :as route]
[compojure.handler :as handler]
[clojure.string :as str])
(:use compojure.core
ring.adapter.jetty
[ring.middleware.content-type :only
(wrap-content-type)]
[ring.middleware.file :only (wrap-file)]
[ring.middleware.file-info :only
(wrap-file-info)]
[ring.middleware.stacktrace :only
(wrap-stacktrace)]
[ring.util.response :only (redirect)]
[hiccup core element page]
[hiccup.middleware :only (wrap-base-url)]))
(defn d3-page...as above
...)
(deftype Group [key values])
(deftype Point [x y size])
(defn add-label [chart axis label]
(if-not (nil? label)
(.axisLabel (aget chart axis) label)))
(defn add-axes-labels [chart x-label y-label]
(doto chart (add-label "xAxis" x-label)
(add-label "yAxis" y-label)))
(defn populate-node [selector chart groups transition continuation]
(-> (.select js/d3 selector)
(.datum groups)
(.transition)
(.duration (if transition 500 0))
(.call chart)
(.call continuation)))
(defn force-layout-plot []
(d3-page "Force-Directed Layout"
"webviz.force.force_layout();"
[:div#force.chart [:svg]]))
(defroutes site-routes
(GET "/force" [] (force-layout-plot))
(GET "/force/data.json" []
(redirect "/data/census-race.json"))
(route/resources "/")
(route/not-found "Page not found"))
(def app (-> (handler/site site-routes)))
In line 5
(include-css "/css/nv.d3.css"))
There's an extra ) there.
And in line 13,
"/master/nv.d3.min.js")]
There's one ) missing. Should be
"/master/nv.d3.min.js"))]
You should use an editor which can do the matching of braces, parentheses, and brackets, etc. automatically.

How to parse URL parameters in Clojure?

If I have the request "size=3&mean=1&sd=3&type=pdf&distr=normal" what's the idiomatic way of writing the function (defn request->map [request] ...) that takes this request and
returns a map {:size 3, :mean 1, :sd 3, :type pdf, :distr normal}
Here is my attempt (using clojure.walk and clojure.string):
(defn request-to-map
[request]
(keywordize-keys
(apply hash-map
(split request #"(&|=)"))))
I am interested in how others would solve this problem.
Using form-decode and keywordize-keys:
(use 'ring.util.codec)
(use 'clojure.walk)
(keywordize-keys (form-decode "hello=world&foo=bar"))
{:foo "bar", :hello "world"}
Assuming you want to parse HTTP request query parameters, why not use ring? ring.middleware.params contains what you want.
The function for parameter extraction goes like this:
(defn- parse-params
"Parse parameters from a string into a map."
[^String param-string encoding]
(reduce
(fn [param-map encoded-param]
(if-let [[_ key val] (re-matches #"([^=]+)=(.*)" encoded-param)]
(assoc-param param-map
(codec/url-decode key encoding)
(codec/url-decode (or val "") encoding))
param-map))
{}
(string/split param-string #"&")))
You can do this easily with a number of Java libraries. I'd be hesitant to try to roll my own parser unless I read the URI specs carefully and made sure I wasn't missing any edge cases (e.g. params appearing in the query twice with different values). This uses jetty-util:
(import '[org.eclipse.jetty.util UrlEncoded MultiMap])
(defn parse-query-string [query]
(let [params (MultiMap.)]
(UrlEncoded/decodeTo query params "UTF-8")
(into {} params)))
user> (parse-query-string "size=3&mean=1&sd=3&type=pdf&distr=normal")
{"sd" "3", "mean" "1", "distr" "normal", "type" "pdf", "size" "3"}
Can also use this library for both clojure and clojurescript: https://github.com/cemerick/url
user=> (-> "a=1&b=2&c=3" cemerick.url/query->map clojure.walk/keywordize-keys)
{:a "1", :b "2", :c "3"}
Yours looks fine. I tend to overuse regexes, so I would have solved it as
(defn request-to-keywords [req]
(into {} (for [[_ k v] (re-seq #"([^&=]+)=([^&]+)" req)]
[(keyword k) v])))
(request-to-keywords "size=1&test=3NA=G")
{:size "1", :test "3NA=G"}
Edit: try to stay away from clojure.walk though. I don't think it's officially deprecated, but it's not very well maintained. (I use it plenty too, though, so don't feel too bad).
I came across this question when constructing my own site and the answer can be a bit different, and easier, if you are passing parameters internally.
Using Secretary to handle routing: https://github.com/gf3/secretary
Parameters are automatically extracted to a map in :query-params when a route match is found. The example given in the documentation:
(defroute "/users/:id" [id query-params]
(js/console.log (str "User: " id))
(js/console.log (pr-str query-params)))
(defroute #"/users/(\d+)" [id {:keys [query-params]}]
(js/console.log (str "User: " id))
(js/console.log (pr-str query-params)))
;; In both instances...
(secretary/dispach! "/users/10?action=delete")
;; ... will log
;; User: 10
;; "{:action \"delete\"}"
You can use ring.middleware.params. Here's an example with aleph:
user=> (require '[aleph.http :as http])
user=> (defn my-handler [req] (println "params:" (:params req)))
user=> (def server (http/start-server (wrap-params my-handler)))
wrap-params creates an entry in the request object called :params. If you want the query parameters as keywords, you can use ring.middleware.keyword-params. Be sure to wrap with wrap-params first:
user=> (require '[ring.middleware.params :refer [wrap-params]])
user=> (require '[ring.middleware.keyword-params :refer [wrap-keyword-params])
user=> (def server
(http/start-server (wrap-keyword-params (wrap-params my-handler))))
However, be mindful that this includes a dependency on ring.