I have the following Clojure code with a render function which renders a html page using enlive-html. Depending on the selected language a different html template is used.
As you can see, there is a lot of code duplication and I would like to remove it.
I was thinking of writing some macros but, if I understand correctly, the language (i.e. lang parameter) is not available at macro execution time because it is provided in the request and that is at execution time and not at compilation time.
I also tried to modify enlive in order to add i18n support at some later point but my Clojure skills are not there yet.
So the questions are:
How can I remove the code duplication in the code below?
Is enlive-html the way to go or should I use another library?
Is there a library similar to enlive with support for i18n?
Thanks!
See the code here:
(ns myapp.core
(:require [net.cgrand.enlive-html :as e))
(deftemplate not-found-en "en/404.html"
[{path :path}]
[:#path] (e/content path))
(deftemplate not-found-fr "fr/404.html"
[{path :path}]
[:#path] (e/content path))
(defn getTemplate [page lang]
(case lang
:en (case page
:page/not-found not-found-en)
:fr (case page
:page/not-found not-found-fr)))
(defn render [lang [page params]]
(apply (getTemplate page lang) params))
On the one hand, it is not too hard to write a macro that will generate the exact code you have here for an arbitrary set of languages. On the other hand, there is probably a better approach than using deftemplate - things that are defd are things you expect to refer to by name in the source code, whereas you just want this thing created and used automatically. But I'm not familiar with the enlive API so I can't say what you should do instead.
If you decide to stick with the macro instead, you could write something like:
(defmacro def-language-404s [languages]
`(do
~#(for [lang languages]
`(deftemplate ~(symbol (str "not-found-" lang)) ~(str lang "/404.html")
[{path# :path}]
[:#path] (e/content path#)))
(defn get-template [page# lang#]
(case page#
:page/not-found (case lang#
~#(for [lang languages
clause [(keyword lang)
(symbol (str "not-found-" lang))]]
clause))))))
user> (macroexpand-1 '(def-language-404s [en fr]))
(do
(deftemplate not-found-en "en/404.html"
[{path__2275__auto__ :path}]
[:#path] (content path__2275__auto__))
(deftemplate not-found-fr "fr/404.html"
[{path__2275__auto__ :path}]
[:#path] (content path__2275__auto__))
(defn get-template [page__2276__auto__ lang__2277__auto__]
(case page__2276__auto__
:page/not-found (case lang__2277__auto__
:en not-found-en
:fr not-found-fr))))
After quite a bit of Macro-Fu I got to a result that I'm happy with. With some help from a few nice stackoverflowers I wrote the following macros on top of enlive:
(ns hello-enlive
(:require [net.cgrand.enlive-html :refer [deftemplate]]))
(defn- template-name [lang page] (symbol (str "-template-" (name page) "-" (name lang) "__")))
(defn- html-file [lang page] (str (name lang) "/" (name page) ".html"))
(defn- page-fun-name [page] (symbol (str "-page" (name page))))
(defmacro def-page [app languages [page & forms]]
`(do
~#(for [lang languages]
`(deftemplate ~(template-name lang page) ~(html-file lang page)
~#forms))
(defn ~(page-fun-name page) [lang#]
(case lang#
~#(for [lang languages
clause [(keyword lang) (template-name lang page)]]
clause)))
(def ^:dynamic ~app
(assoc ~app ~page ~(page-fun-name page)))
))
(defmacro def-app [app-name languages pages]
(let [app (gensym "app__")]
`(do
(def ~(vary-meta app merge {:dynamic true}) {})
~#(for [page# pages]
`(def-page ~app ~languages ~page#))
(defn ~app-name [lang# [page# params#]]
(apply (apply (get ~app page#) [lang#]) params#)))))
...which are then used like this:
The html templates are stored in a tree like this
html/fr/not-found.html
html/fr/index.html
html/en/not-found.html
html/en/index.html
...
...and the rendering logic looks like this:
(def-app my-app [:en :it :fr :de]
[ [:page/index [] ]
;... put your rendering here
[:page/not-found [{path :path}]
[:#path] (content path)]])
...and the usage look like this:
...
(render lang [:page/index {}])
(render lang [:page/not-found {:path path}])
...
The result, although it can probably be improved, I think is pretty nice, without duplication and boilerplate code.
Related
Given Java instance obj and a member name (string) "Foo", and a map conf, I am trying to generate Clojure code that will look like this:
(if (get conf "Foo")
(set! (.Foo obj) (get conf "foo")
obj)
And also, if I know that "SomeEnum" is a Java enum name, a code like this:
(if (get conf "SomeEnum")
(set! (.someEnum obj)(Enum/valueOf SomeEnum (get conf "SomeEnum")))
obj)
Here is what I came up with:
(defmacro set-java [obj conf obj-name]
`(if (get ~conf ~obj-name)
(set! (. ~obj ~(symbol obj-name)) (get ~conf ~obj-name))
~obj))
(defn lowercase-first [s]
(apply str (Character/toLowerCase (first s)) (rest s)))
(defmacro set-java-enum [obj conf obj-name]
`(if (get ~conf ~obj-name)
(set! (. ~obj ~(symbol (lowercase-first obj-name)))
(Enum/valueOf ~(symbol obj-name) (get ~conf ~obj-name)))
~obj))
Testing with macroexpand seems to give the right result, but after trying:
(defn ^Policy map->policy [conf]
(-> (Policy.)
(set-java-enum conf "CommitLevel")
(set-java conf "durableDelete")
(set-java conf "expiration")
(set-java conf "generation")
(set-java-enum conf "GenerationPolicy")
(set-java-enum conf "RecordExistsAction")
(set-java conf "respondAllOps")))
I got a weird endless loop of reflection warnings.
--- edit ---
after battelling with that for quite a while, I gave up threading (->) and ended with:
(defmacro set-java [obj conf obj-name]
`(when (get ~conf ~obj-name)
(set! (. ~obj ~(symbol obj-name)) (get ~conf ~obj-name))))
(defn lowercase-first [s]
(apply str (Character/toLowerCase ^Character (first s)) (rest s)))
(defmacro set-java-enum [obj conf obj-name]
`(when (get ~conf ~obj-name)
(set! (. ~obj ~(symbol (lowercase-first obj-name)))
(Enum/valueOf ~(symbol obj-name) (get ~conf ~obj-name)))))
(defn map->write-policy [conf]
(let [wp (WritePolicy. (map->policy conf))]
(set-java-enum wp conf "CommitLevel")
(set-java wp conf "durableDelete")
;; more non threaded object manipulation
wp))
So I am still not sure what was the reflection warning endless loop about, but this is hopefully handful as well, and can be improved further.
I think this is due to your interpolating ~obj multiple times in each macro. Is your "loop" actually endless, or is it just ~128 or ~256 steps long?
In any case, the fix for that particular issue (whether or not it's the root cause of the problem you describe) is to wrap the form in (let [obj# ~obj] ...) and then refer to obj# below, so the argument is only interpolated once.
You can (should!) do this with conf and obj-name too, but they're probably not actively causing problems, at least with the usage code you provided.
I want to create a function that allows me to pull contents from some feed, here's what I have... zf is from here
(:require
[clojure.zip :as z]
[clojure.data.zip.xml :only (attr text xml->)]
[clojure.xml :as xml ]
[clojure.contrib.zip-filter.xml :as zf]
)
(def data-url "http://api.eventful.com/rest/events/search?app_key=4H4Vff4PdrTGp3vV&keywords=music&location=Belgrade&date=Future")
(defn zipp [data] (z/xml-zip data))
(defn contents[cont & tags]
(assert (= (zf/xml-> (zipp(parsing cont)) (seq tags) text))))
but when I call it
(contents data-url :events :event :title)
I get an error
java.lang.RuntimeException: java.lang.ClassCastException: clojure.lang.ArraySeq cannot be cast to clojure.lang.IFn (NO_SOURCE_FILE:0)
(Updated in response to the comments: see end of answer for ready-made function parameterized by the tags to match.)
The following extracts the titles from the XML pointed at by the URL from the question text (tested at a Clojure 1.5.1 REPL with clojure.data.xml 0.0.7 and clojure.data.zip 0.1.1):
(require '[clojure.zip :as zip]
'[clojure.data.xml :as xml]
'[clojure.data.zip.xml :as xz]
'[clojure.java.io :as io])
(def data-url "http://api.eventful.com/rest/events/search?app_key=4H4Vff4PdrTGp3vV&keywords=music&location=Belgrade&date=Future")
(def data (-> data-url io/reader xml/parse))
(def z (zip/xml-zip data))
(mapcat (comp :content zip/node)
(xz/xml-> z
(xz/tag= :events)
(xz/tag= :event)
(xz/tag= :title)))
;; value of the above right now:
("Belgrade Early Music Festival, Gosta / Purcell: Dido & Aeneas"
"Belgrade Early Music Festival, Gosta / Purcell: Dido & Aeneas"
"Belgrade Early Music Festival, Gosta / Purcell: Dido & Aeneas"
"VIII Early Music Festival, Belgrade 2013"
"Kevlar Bikini"
"U-Recken - Tree of Life Pre event"
"Green Day"
"Smallman - Vrane Kamene (Crows Of Stone)"
"One Direction"
"One Direction in Serbia")
Some comments:
The clojure.contrib.* namespaces are all deprecated. xml-> now lives in clojure.data.zip.xml.
xml-> accepts a zip loc and a bunch of "predicates"; in this context, however, the word "predicate" has an unusual meaning of a filtering function working on zip locs. See clojure.data.zip.xml source for several functions which return such predicates; for an example of use, see above.
If you want to define a list of predicates separately, you can do that too, then use xml-> with apply:
(def loc-preds [(xz/tag= :events) (xz/tag= :event) (xz/tag= :title)])
(mapcat (comp :content zip/node) (apply xz/xml-> z loc-preds))
;; value returned as above
Update: Here's a function which takes the url and keywords naming tags as arguments and returns the content found at the tags:
(defn get-content-from-tags [url & tags]
(mapcat (comp :content zip/node)
(apply xz/xml->
(-> url io/reader xml/parse zip/xml-zip)
(for [t tags]
(xz/tag= t)))))
Calling it like so:
(get-content-from-tags data-url :events :event :title)
gives the same result as the mapcat form above.
Trying to load a particular template based on what :server-name returns in the request:
(ns rosay.views.common
(:use noir.core)
(:require [noir.request :as req]
[clojure.string :as string]
[net.cgrand.enlive-html :as html]))
(defn get-server-name
"Pulls servername for template definition"
[]
(or (:server-name (req/ring-request)) "localhost"))
(defn get-template
"Grabs template name for current server"
[tmpl]
(string/join "" (concat [(get-server-name) tmpl])))
(html/deftemplate base (get-template "/base.html")
[]
[:p] (html/content (get-template "/base.html")))
It works for localhost which returns /home/usr/rosay/resources/localhost/base.html, but when I test against a different host say "hostname2" I see where get-template is looking at /home/usr/rosay/resources/hostname2/base.html but when it renders in the browser it always points back to ../resources/localhost/base.html.
Is there a macro or different way to handle this use-case?
As mentioned in the comments, deftemplate is a macro that defines the template as a function in your namespace - only once, when it's first evaluated. You can easily write some code to lazily create the templates, and eliminate some of the overhead by caching the template once it's created:
(def templates (atom {}))
(defmacro defservertemplate [name source args & forms]
`(defn ~name [& args#]
(let [src# (get-template ~source)]
(dosync
(if-let [template# (get templates src#)]
(apply template# args#)
(let [template# (template src# ~args ~#forms)]
(swap! templates assoc src# template#)
(apply template# args#)))))))
In your case you'd then be able to say (defservertemplate base "/base.html"....
You can probably tidy this up a bit. All you really need to know is that deftemplate just calls template, and you can use that directly if you want.
I tried to create function to scrape and tags from HTML page, whose URL I provide to a function, and this works as it should. I get sequence of <h3> and <table> elements, when I try to use select function to extract only table or h3 tags from resulting sequence,
I get (), or if I try to map those tags I get (nil nil nil ...).
Could you please help me to resolve this issue, or explain me what am I doing wrong?
Here is the code:
(ns Test2
(:require [net.cgrand.enlive-html :as html])
(:require [clojure.string :as string]))
(defn get-page
"Gets the html page from passed url"
[url]
(html/html-resource (java.net.URL. url)))
(defn h3+table
"returns sequence of <h3> and <table> tags"
[url]
(html/select (get-page url)
{[:div#wrap :div#middle :div#content :div#prospekt :div#prospekt_container :h3]
[:div#wrap :div#middle :div#content :div#prospekt :div#prospekt_container :table]}
))
(def url "http://www.belex.rs/trgovanje/prospekt/VZAS/show")
This line gives me headache :
(html/select (h3+table url) [:table])
Could you please tell me what am I doing wrong?
Just to clarify my question: is it possible to use enlive's select function to extract only table tags from result of (h3+table url) ?
As #Julien pointed out, you will probably have to work with the deeply nested tree structure that you get from applying (html/select raw-html selectors) on the raw html. It seems like you try to apply html/select multiple times, but this doesn't work. html/select parses html into a clojure datastructure, so you can't apply it on that datastructure again.
I found that parsing the website was actually a little involved, but I thought that it might be a nice use case for multimethods, so I hacked something together, maybe this will get you started:
(The code is ugly here, you can also checkout this gist)
(ns tutorial.scrape1
(:require [net.cgrand.enlive-html :as html]))
(def *url* "http://www.belex.rs/trgovanje/prospekt/VZAS/show")
(defn get-page [url]
(html/html-resource (java.net.URL. url)))
(defn content->string [content]
(cond
(nil? content) ""
(string? content) content
(map? content) (content->string (:content content))
(coll? content) (apply str (map content->string content))
:else (str content)))
(derive clojure.lang.PersistentStructMap ::Map)
(derive clojure.lang.PersistentArrayMap ::Map)
(derive java.lang.String ::String)
(derive clojure.lang.ISeq ::Collection)
(derive clojure.lang.PersistentList ::Collection)
(derive clojure.lang.LazySeq ::Collection)
(defn tag-type [node]
(case (:tag node)
:tr ::CompoundNode
:table ::CompoundNode
:th ::TerminalNode
:td ::TerminalNode
:h3 ::TerminalNode
:tbody ::IgnoreNode
::IgnoreNode))
(defmulti parse-node
(fn [node]
(let [cls (class node)] [cls (if (isa? cls ::Map) (tag-type node) nil)])))
(defmethod parse-node [::Map ::TerminalNode] [node]
(content->string (:content node)))
(defmethod parse-node [::Map ::CompoundNode] [node]
(map parse-node (:content node)))
(defmethod parse-node [::Map ::IgnoreNode] [node]
(parse-node (:content node)))
(defmethod parse-node [::String nil] [node]
node)
(defmethod parse-node [::Collection nil] [node]
(map parse-node node))
(defn h3+table [url]
(let [ws-content (get-page url)
h3s+tables (html/select ws-content #{[:div#prospekt_container :h3]
[:div#prospekt_container :table]})]
(for [node h3s+tables] (parse-node node))))
A few words on what's going on:
content->string takes a data structure and collects its content into a string and returns that so you can apply this to content that may still contain nested subtags (like <br/>) that you want to ignore.
The derive statements establish an ad hoc hierarchy which we will later use in the multi-method parse-node. This is handy because we never quite know which data structures we're going to encounter and we could easily add more cases later on.
The tag-type function is actually a hack that mimics the hierarchy statements - AFAIK you can't create a hierarchy out of non-namespace qualified keywords, so I did it like this.
The multi-method parse-node dispatches on the class of the node and if the node is a map additionally on the tag-type.
Now all we have to do is define the appropriate methods: If we're at a terminal node we convert the contents to a string, otherwise we either recur on the content or map the parse-node function on the collection we're dealing with. The method for ::String is actually not even used, but I left it in for safety.
The h3+table function is pretty much what you had before, I simplified the selectors a bit and put them into a set, not sure if putting them into a map as you did works as intended.
Happy scraping!
Your question is hard to understand, but I think your last line should simply be
(h3+table url)
This will return a deeply nested data structure containing scraped HTML that you can then dive into with the usual Clojure sequence APIs. Good luck.
I'm attempting to write a macro which will call java setter methods based on the arguments given to it.
So, for example:
(my-macro login-as-fred {"Username" "fred" "Password" "wilma"})
might expand to something like the following:
(doto (new MyClass)
(.setUsername "fred")
(.setPassword "wilma"))
How would you recommend tackling this?
Specifically, I'm having trouble working out the best way to construct the setter method name and have it interpreted it as a symbol by the macro.
The nice thing about macros is you don't actually have to dig into the classes or anything like that. You just have to write code that generates the proper s-expressions.
First a function to generate an s-expression like (.setName 42)
(defn make-call [name val]
(list (symbol (str ".set" name) val)))
then a macro to generate the expressions and plug (~#) them into a doto expression.
(defmacro map-set [class things]
`(doto ~class ~#(map make-call things))
Because it's a macro it never has to know what class the thing it's being called on is or even that the class on which it will be used exists.
Please don't construct s-expressions with list for macros. This will seriously hurt the hygiene of the macro. It is very easy to make a mistake, which is hard to track down. Please use always syntax-quote! Although, this is not a problem in this case, it's good to get into the habit of using only syntax-quote!
Depending on the source of your map, you might also consider to use keywords as keys to make it look more clojure-like. Here is my take:
(defmacro configure
[object options]
`(doto ~object
~#(map (fn [[property value]]
(let [property (name property)
setter (str ".set"
(.toUpperCase (subs property 0 1))
(subs property 1))]
`(~(symbol setter) ~value)))
options)))
This can then be used as:
user=> (macroexpand-1 '(configure (MyClass.) {:username "fred" :password "wilma"}))
(clojure.core/doto (MyClass.) (.setUsername "fred") (.setPassword "wilma"))
Someone (I believe Arthur Ulfeldt) had an answer posted that was almost correct, but it's been deleted now.
This is a working version:
(defmacro set-all [obj m]
`(doto ~obj ~#(map (fn [[k v]]
(list (symbol (str ".set" k)) v))
m)))
user> (macroexpand-1 '(set-all (java.util.Date.) {"Month" 0 "Date" 1 "Year" 2009}))
(clojure.core/doto (java.util.Date.) (.setMonth 0) (.setDate 1) (.setYear 2009))
user> (set-all (java.util.Date.) {"Month" 0 "Date" 1 "Year" 2009})
#<Date Fri Jan 01 14:15:51 PST 3909>
You have to bite the bullet and use clojure.lang.Reflector/invokeInstanceMethod like this:
(defn do-stuff [obj m]
(doseq [[k v] m]
(let [method-name (str "set" k)]
(clojure.lang.Reflector/invokeInstanceMethod
obj
method-name
(into-array Object [v]))))
obj)
(do-stuff (java.util.Date.) {"Month" 2}) ; use it
No need for a macro (as far as I know, a macro would not allow to circumvent reflection, either; at least for the general case).