Is it possible to append a value to an attribute using enlive?
example: I have this
edit
and would like this
edit
I am currently doing this:
(html/defsnippet foo "views/foo.html" [:#main]
[ctxt]
[:a] (html/set-attr :href (str "/item/edit/" (ctxt :id))))
But I would prefer not to embed the URL into my code, by just appending the id to the existing URL
(html/defsnippet foo "views/foo.html" [:#main]
[ctxt]
[:a#href] (html/append (ctxt :id)))
#ddk answer is spot on but you may prefer a more generic way to solve the problem
(defn update-attr [attr f & args]
(fn [node]
(apply update-in node [:attrs attr] f args))))
and then
(update-attr :href str "123")
You could always write your own append-attr in the same vein as set-attr. Here is my attempt
(defn append-attr
[& kvs]
(fn [node]
(let [in-map (apply array-map kvs)
old-attrs (:attrs node {})
new-attrs (into {} (for [[k v] old-attrs]
[k (str v (get in-map k))]))]
(assoc node :attrs new-attrs))))
Which gives the following, when appending "/bar" to href, on enlive's representation of A link
((append-attr :href "/bar")
{:tag :a, :attrs {:href "/foo"}, :content "A link"})
;=> {:tag :a, :attrs {:href "/foo/bar"}, :content "A link"}
Related
There is a map containing sequences. The sequences contain items.
I want to remove a given item from any sequence that contains it.
The solution I found does what it should, but I wonder if there is a better
or more elegant way to achieve the same.
my current solution:
(defn remove-item-from-map-value [my-map item]
(apply merge (for [[k v] my-map] {k (remove #(= item %) v)})))
The test describe the expected behaviour:
(require '[clojure.test :as t])
(def my-map {:keyOne ["itemOne"]
:keyTwo ["itemTwo" "itemThree"]
:keyThree ["itemFour" "itemFive" "itemSix"]})
(defn remove-item-from-map-value [my-map item]
(apply merge (for [[k v] my-map] {k (remove #(= item %) v)})))
(t/is (= (remove-item-from-map-value my-map "unknown-item") my-map))
(t/is (= (remove-item-from-map-value my-map "itemFive") {:keyOne ["itemOne"]
:keyTwo ["itemTwo" "itemThree"]
:keyThree ["itemFour" "itemSix"]}))
(t/is (= (remove-item-from-map-value my-map "itemThree") {:keyOne ["itemOne"]
:keyTwo ["itemTwo"]
:keyThree ["itemFour" "itemFive" "itemSix"]}))
(t/is (= (remove-item-from-map-value my-map "itemOne") {:keyOne []
:keyTwo ["itemTwo" "itemThree"]
:keyThree ["itemFour" "itemFive" "itemSix"]}))
I'm fairly new to clojure and am interested in different solutions.
So any input is welcome.
I throw in the specter
version for good measure. It keeps the vectors inside the map
and it's really compact.
(setval [MAP-VALS ALL #{"itemFive"}] NONE my-map)
Example
user=> (use 'com.rpl.specter)
nil
user=> (def my-map {:keyOne ["itemOne"]
#_=> :keyTwo ["itemTwo" "itemThree"]
#_=> :keyThree ["itemFour" "itemFive" "itemSix"]})
#_=>
#'user/my-map
user=> (setval [MAP-VALS ALL #{"itemFive"}] NONE my-map)
{:keyOne ["itemOne"],
:keyThree ["itemFour" "itemSix"],
:keyTwo ["itemTwo" "itemThree"]}
user=> (setval [MAP-VALS ALL #{"unknown"}] NONE my-map)
{:keyOne ["itemOne"],
:keyThree ["itemFour" "itemFive" "itemSix"],
:keyTwo ["itemTwo" "itemThree"]}
i would go with something like this:
user> (defn remove-item [my-map item]
(into {}
(map (fn [[k v]] [k (remove #{item} v)]))
my-map))
#'user/remove-item
user> (remove-item my-map "itemFour")
;;=> {:keyOne ("itemOne"),
;; :keyTwo ("itemTwo" "itemThree"),
;; :keyThree ("itemFive" "itemSix")}
you could also make up a handy function map-val performing mapping on map values:
(defn map-val [f data]
(reduce-kv
(fn [acc k v] (assoc acc k (f v)))
{} data))
or shortly like this:
(defn map-val [f data]
(reduce #(update % %2 f) data (keys data)))
user> (map-val inc {:a 1 :b 2})
;;=> {:a 2, :b 3}
(defn remove-item [my-map item]
(map-val (partial remove #{item}) my-map))
user> (remove-item my-map "itemFour")
;;=> {:keyOne ("itemOne"),
;; :keyTwo ("itemTwo" "itemThree"),
;; :keyThree ("itemFive" "itemSix")}
I think your solution is mostly okay, but I would try to avoid the apply merge part, as you can easily recreate a map from a sequence with into. Also, you could also use map instead of for which I think is a little bit more idiomatic in this case as you don't use any of the list comprehension features of for.
(defn remove-item-from-map-value [m item]
(->> m
(map (fn [[k vs]]
{k (remove #(= item %) vs)}))
(into {})))
Another solution much like #leetwinski:
(defn remove-item [m i]
(zipmap (keys m)
(map (fn [v] (remove #(= % i) v))
(vals m))))
Here's a one-liner which does this in an elegant way. The perfect function for me to use in this scenario is clojure.walk/prewalk. What this fn does is it traverse all of the sub-forms of the form that you pass to it and it transforms them with the provided fn:
(defn remove-item-from-map-value [data item]
(clojure.walk/prewalk #(if (map-entry? %) [(first %) (remove #{item} (second %))] %) data))
What the remove-item-from-map-value fn will do is it will check if current form is a map entry and if so, it will remove specified key from its value (second element of the map entry, which is a vector containing a key and a value, respectively).
The best this about this approach is that is is completely extendable: you could decide to do different things for different types of forms, you can also handle nested forms, etc.
It took me some time to master this fn but once I got it I found it extremely useful!
Is there an easy way in Clojure (maybe using specter) to filter collections depending on whether the an arbitrarily nested key with a known name contains an element ?
Ex. :
(def coll [{:res [{:a [{:thekey [
"the value I am looking for"
...
]
}
]}
{:res ...}
{:res ...}
]}])
Knowing that :a could have a different name, and that :thekey could be nested somewhere else.
Let's say I would like to do :
#(find-nested :thekey #{"the value I am looking for"} coll) ;; returns a vector containing the first element in coll (and maybe others)
use zippers.
in repl:
user> coll
[{:res [{:a [{:thekey ["the value I am looking for"]}]} {:res 1} {:res 1}]}]
user> (require '[clojure.zip :as z])
nil
user> (def cc (z/zipper coll? seq nil coll))
#'user/cc
user> (loop [x cc]
(if (= (z/node x) :thekey)
(z/node (z/next x))
(recur (z/next x))))
["the value I am looking for"]
update:
this version is flawed, since it doesn't care about :thekey being the key in a map, or just keyword in a vector, so it would give unneeded result for coll [[:thekey [1 2 3]]]. Here is an updated version:
(defn lookup-key [k coll]
(let [coll-zip (z/zipper coll? #(if (map? %) (vals %) %) nil coll)]
(loop [x coll-zip]
(when-not (z/end? x)
(if-let [v (-> x z/node k)] v (recur (z/next x)))))))
in repl:
user> (lookup-key :thekey coll)
["the value I am looking for"]
user> (lookup-key :absent coll)
nil
lets say we have the same keyword somewhere in a vector in a coll:
(def coll [{:res [:thekey
{:a [{:thekey ["the value I am looking for"]}]}
{:res 1} {:res 1}]}])
#'user/coll
user> (lookup-key :thekey coll)
["the value I am looking for"]
which is what we need.
I know how to extract one attribute using zip-xml/attr, but how to extract multiple attributes?
e.g I have the following
<table>
<column name="col1" type="varchar" length="8"/>
<column name="col2" type="varchar" length="16"/>
<column name="col3" type="int" length="16"/>
<table>
And the expected result is. A silly way is to call zip-xml/attr for each attribute, but is there any elegant way to do that?
[["co11" "varchar" 8] [["co12" "varchar" 16] [["co13" "int" 16]
My advice is to use a tree-walking function to extract the interesting data from the XML tree. clojure.walk has several of these, but here I use tree-seq from core clojure to just produce a seq of nodes and work on that. This function takes two functions - a branch? predicate which checks if a node can have children and a children function which gets them. I use :content for both, as tags with no nested tags produce nil, which is a falsey value and so it works also as a predicate.
(->> (clojure.xml/parse "res/doc.xml") ;;source file for your xml
(tree-seq :content :content) ;; Produce a seq by walking the tree
(filter #(= :column (:tag %))) ;;Take only :column tags
(mapv (comp vec vals :attrs)))
;;Collect the values of the :attrs maps into vectors
;;and collect those into a vector with mapv
Your desired output had unmatched square brackets, but I assume it should be like
[["col1" "varchar" "8"] ["col2" "varchar" "16"] ["col3" "int" "16"]]
which was my return value. However, this is potentially brittle - you're relying on the maps returned by clojure.xml/parse preserving the ordering of the attributes in the XML in order to know what the data means. That's not really part of the contract of maps. As an implementation detail it creates clojure.lang.PersistentStructMaps which apparently do have this feature, but it might not always be so.
Alternatively you could use just (mapv :attrs) to keep the whole of the map in there.
The right solution depends on how large and complex the XML is and to some extent, what you know about its structure. If it needs to be very generic, then you need to have quite a lot of logic to navigate the nodes etc. However, if it is a known format and you know what nodes you are interested in, its pretty straight-forward.
I used clojure.zip to create a zipper from the XML file and then use clojure.data.zip.xml to extract the nodes/paths I was interested in. I then defined simple helper functions to process specific nodes. This was pretty much my first bit of clojure and I've not yet gone back to it to re-factor it and refine/clarify some of my very rough clojure idioms based on what I've learnt since, but in the spirit of an example being worth 1000 words, here it is -
(ns arcis.models.nessus
(:use [taoensso.timbre :only [trace debug info warn error fatal]])
(:require [arcis.util :as util]
[arcis.models.db :as db]
[clojure.java.io :as io]
[clojure.xml :as xml]
[clojure.zip :as zip]
[clojure.data.zip.xml :as zx]))
(def nessus-host-keys [:hostname :host_fqdn
:system_type :operating_system
:operating_system_unsupported])
(def used-nessus-host-keys (conj nessus-host-keys
:host_start :host_end
:items :traceroute_hop_0 :traceroute_hop_1
:traceroute_hop_2 :traceroute_hop_3
:traceroute_hop_4 :traceroute_hop_5
:traceroute_hop_6 :traceroute_hop_7
:traceroute_hop_8 :traceroute_hop_9
:traceroute_hop_10 :traceroute_hop_11
:traceroute_hop_12 :traceroute_hop_13
:traceroute_hop_14 :traceroute_hop_15
:traceroute_hop_16 :traceroute_hop_17
:host_ip :patch_summary_total_cves
:cpe_0 :cpe_1 :cpe_2 :cpe_3 :cpe_4 :cpe_5
:cpe_6 :cpe_7 :cpe_8 :cpe_9))
(def nessus-item-keys [:port :svc_name :protocol :severity :plugin_id
:plugin_output])
(def used-nessus-item-keys (conj nessus-item-keys
:plugin_details
:plugin_name
:plugin_family))
(def nessus-plugin-keys [:plugin_id :plugin_name :plugin_family :fname
:script_version :plugin_type :exploitability_ease
:vuln_publication_date :cvss_temporal_data
:solution :cvss_temporal_score :risk_factor
:description :cvss_vector :synopsis
:patch_publication_date :exploit_available
:plugin_publication_date :plugin_modification_date
:cve :bid :exploit_framework_canvas :edb_id
:exploit_framework_metasploit :exploit_framework_core
:metasploit_name :canvas_package :osvdb :cwe
:cvss_temporal_vector :cvss_base_score :cpe
:exploited_by_malware])
(def used-nessus-plugin-keys (conj nessus-plugin-keys
:xref :see_also :cert
:attachment :iava :stig_severity :hp
:secunia :iawb :msft))
(def show-unprocessed true)
(defn log-unprocessed [title vls]
(if (and show-unprocessed
(seq vls))
(println (str "Unprocessed " title ": " vls))))
;;; parse nessus report
(defn parse-xref [xref]
{:xref (first (:content xref))})
(defn parse-see-also [see-also]
{:see_also (first (:content see-also))})
(defn parse-plugin [plugin]
{(util/db-keyword (name (:tag plugin))) (first (:content plugin))})
(defn parse-contents [cont]
(let [xref (mapv parse-xref (filter #(= (:tag %) :xref) cont))
see-also (mapv parse-see-also (filter #(= (:tag %) :see-also) cont))
details (reduce merge {}
(map parse-plugin
(remove #(or (= (:tag %) :xref)
(= (:tag %) :see-also)) cont)))]
(assoc details
:see_also see-also
:xref xref)))
(defn fix-item-keywords [item]
(let [ks (keys item)]
(into {}
(for [k ks]
[(util/db-keyword (name k))
(k item)]))))
(defn parse-item [item]
(let [attrs (fix-item-keywords (:attrs item))
contents (parse-contents (:content item))]
(assoc attrs
:plugin_output (:plugin_output contents)
:plugin_details (assoc (dissoc contents :plugin_output)
:plugin_id (:plugin_id attrs)
:plugin_family (:plugin_family attrs)))))
(defn parse-properties [props]
(into {}
(for [p props]
[(util/db-keyword (:name (:attrs p)))
(first (:content p))])))
(defn parse-host [h]
(let [items (map first (zx/xml-> h :ReportItem))
properties (:content (first (zx/xml1-> h :HostProperties)))]
(assoc (parse-properties properties)
:hostname (zx/attr h :name)
:items (mapv parse-item items))))
(defn parse-hosts [hosts]
(mapv parse-host hosts))
(defn parse-file [f]
(let [root (zip/xml-zip (xml/parse (io/file f)))
report-xml (zx/xml1-> root :Report)
hosts (zx/xml-> report-xml :ReportHost)]
{:report_name (zx/attr report-xml :name)
:policy (zx/text (zx/xml1-> root :Policy :policyName))
:hosts (parse-hosts hosts)}))
;;; insert nessus records into db
(defn mk-host-rec [scan-id host]
(let [[id err] (db/get-sequence-nextval "host_seq")]
(if (nil? err)
(assoc (util/build-map host nessus-host-keys)
:ipv4 (:host_ip host)
:scan_start (util/from-nessus-date (:scan_start host))
:scan_end (util/from-nessus-date (:scan_end host))
:total_cves (:patch_summary_total_cves host)
:id id
:scan_id scan-id)
nil)))
(defn insert-patches [p]
(when (seq p)
(db/insert-nessus-host-patch (first p))
(recur (rest p))))
(defn insert-host-patch [id host]
(let [p-keys (filter #(re-find #"patch_summary_*" %) (map name (keys host)))
recs (map (fn [s]
{:id (first (db/get-sequence-nextval "patch_seq"))
:host_id id
:summary ((keyword (str "patch_summary_txt_" s)) host)
:cve_num ((keyword (str "patch_summary_cve_num_" s)) host)
:cves ((keyword (str "patch_summary_cves_" s)) host)})
(filter seq
(map #(second (re-find #"patch_summary_txt_(.*)" %))
p-keys)))]
(insert-patches recs)
(util/remove-keys host (map keyword p-keys))))
(defn mk-item-rec [host-id item]
(let [[id err] (db/get-sequence-nextval "item_seq")]
(assoc (util/build-map item nessus-item-keys)
:host_id host-id
:id id)))
(defn insert-item [host-id item]
(let [rec (mk-item-rec host-id item)
not-done (keys (util/remove-keys item used-nessus-item-keys))]
(log-unprocessed "Item Keys" not-done)
(db/insert-nessus-report-item rec)
(:plugin_id item)))
(defn mk-plugin-rec [item]
(let [rec (util/build-map (:plugin_details item) nessus-plugin-keys)
not-used (keys (util/remove-keys (:plugin_details item)
used-nessus-plugin-keys))]
(log-unprocessed "Plugin Keys" not-used)
(assoc rec
:vuln_publication_date (util/from-nessus-date
(:vuln_publication_date rec))
:patch_publication_date (util/from-nessus-date
(:patch_publication_date rec))
:plugin_publication_date (util/from-nessus-date
(:plugin_publication_date rec))
:plugin_modification_date (util/from-nessus-date
(:plugin_modificaiton_date rec)))))
(defn insert-xref [plugin-id xrefs]
(when (seq xrefs)
(let [xref {:id (first (db/get-sequence-nextval "xref_seq"))
:plugin_id plugin-id
:xref (:xref (first xrefs))}]
(db/insert-nessus-xref xref)
(recur plugin-id (rest xrefs)))))
(defn insert-see-also [plugin-id see-also]
(when (seq see-also)
(let [sa {:id (first (db/get-sequence-nextval "ref_seq"))
:plugin_id plugin-id
:reference (:see_also (first see-also))}]
(db/insert-nessus-ref sa)
(recur plugin-id (rest see-also)))))
(defn insert-plugin [item]
(let [rec (mk-plugin-rec item)
xref (:xref (:plugin_details item))
see-also (:see_also (:plugin_details item))]
(if (seq xref)
(insert-xref (:plugin_id rec) xref))
(if (seq see-also)
(insert-see-also (:plugin_id rec) see-also))
(db/upsert-nessus-plugin rec)))
(defn insert-items [host-id items plugin-set]
(if (empty? items)
plugin-set
(let [p (insert-item host-id (first items))]
(if-not (contains? plugin-set p)
(insert-plugin (first items)))
(recur host-id (rest items) (conj plugin-set p)))))
(defn insert-host [scan-id host plugin-set]
(if-let [h-rec (mk-host-rec scan-id host)]
(let [[v err] (db/insert-nessus-host h-rec)
items (:items host)]
(if (nil? err)
(let [host2 (insert-host-patch (:id h-rec) host)]
(log-unprocessed "Host Keys" (keys (util/remove-keys
host2 used-nessus-host-keys)))
(insert-items (:id h-rec) items plugin-set))
plugin-set))
plugin-set))
(defn insert-hosts
([id hosts]
(insert-hosts id hosts #{}))
([id hosts plugins]
(if (empty? hosts)
plugins
(let [plugin-set (insert-host id (first hosts) plugins)]
(recur id (rest hosts) plugin-set)))))
(defn mk-scan-record [id report]
{:id id
:name (:report_name report)
:scan_dt (util/to-sql-date)
:policy (:policy report)
:entered_dt (util/to-sql-date)})
(defn store-report [update-plugins report]
(let [[id err] (db/get-sequence-nextval "nscan_seq")
scan-rec (mk-scan-record id report)]
(if (nil? err)
(let [[v e] (db/insert-nessus-scan scan-rec)]
(if (nil? e)
(if update-plugins
(let [plugin-list (set (first (db/select-nessus-plugin-ids)))]
[(insert-hosts id (:hosts report) plugin-list) nil])
[(insert-hosts id (:hosts report)) nil])
[v e]))
[id err])))
(defn process-nessus-report [update-plugins filename]
(let [report (parse-file filename)]
(println (str "Report: " (:report_name report)
"\nPolicy: " (:policy report)
"\nHost Records: " (count (:hosts report))))
(store-report update-plugins report)))
Magos's answer using tree-seq is perfectly fine, but there's no reason to abandon zippers; filtering using zippers is more succinct and the arguably the "clojure" way. (note this example uses data.xml ([org.clojure/data.xml "0.0.8"]) instead of clojure.xml).
(require '[clojure.data.zip.xml :as zf])
(require '[clojure.zip :as z])
(def ex
"<table>
<column name=\"col1\" type=\"varchar\" length=\"8\"/>
<column name=\"col2\" type=\"varchar\" length=\"16\"/>
<column name=\"col3\" type=\"int\" length=\"16\"/>
</table>")
(let [x (z/xml-zip (clojure.data.xml/parse-str ex))]
(->> (zf/xml-> x :column) ;;equivalent to (->> treeseq ... filter)
flatten
(keep :attrs)
(map vals)))
;>>> (("col1" "varchar" "8") ("col2" "varchar" "16") ("col3" "int" "16"))
But the xml-> macro simply applies functions in order, so you can do the following:
(let [x (z/xml-zip (clojure.data.xml/parse-str ex))]
(->> (zf/xml-> x :column #(keep :attrs %))
(map vals)))
;>>> (("col1" "varchar" "8") ("col2" "varchar" "16") ("col3" "int" "16"))
I'm having trouble finding solution to the following problem:
Lets say I have a map:
(def defaults {
:name "John"
:surname "Doe"
:info {:date-of-birth "01-01-1980"
:registered [{:type "newsletter" :name "breaking news" }]}
})
And then I pass a similar structured map but I want to conjoin the vectors and overwrite the rest of the keys:
(def new {
:name "Peter"
:info {:date-of-birth "11-01-1986"
:registered [{:type "alert" :name "mobile-alert" }]}
})
And I want this result:
{:name "Peter"
:surname "Doe"
:info {:date-of-birth "11-01-1986"
:registered [{:type "newsletter" :name "breaking news" }
{:type "alert" :name "mobile-alert" }]}}
Now I can do this easily by using static syntax like:
(reduce conj (get-in defaults [:info :registered]) (get-in new [:info :registered]))
(There is probably a better way...) But I was hoping more of a dynamic function with the following properties:
Keep all keys from both maps, without knowing the structure
Update any keys with the values from the right map
if the val of a key is a vector, then conj the vector with the vector of the right map (if the appropriate key exists of course)
Thanks for the help in advance :)
You should definitely look at merge-with function. This is possible implementation:
(defn deep-merge [a b]
(merge-with (fn [x y]
(cond (map? y) (deep-merge x y)
(vector? y) (concat x y)
:else y))
a b))
Here's a possible implementation for this kind of functionality. It is at least a starting point, you may need some extra validations depending on the possible structure of your data (e.g. what if the overriding map's value is a vector but the value in the default map is not even a collection?).
(declare merge-maps)
(defn merge-map [x [k v]]
(cond (vector? v)
(assoc x k (vec (reduce conj (x k) v)))
(map? v)
(assoc x k (merge-maps (x k) v))
:esle
(assoc x k v)))
(defn merge-maps [x y]
(reduce merge-map x y))
(merge-maps defaults new)
;= {:info {:date-of-birth "11-01-1986",
;= :registered [{:name "breaking news", :type "newsletter"}
;= {:name "mobile-alert", :type "alert"}]},
;= :name "Peter",
;= :surname "Doe"}
I want to traverse a vector tree that represents hiccup data structures:
[:div {:class "special"} [:btn-grp '("Hello" "Hi")]]
Then I want to dispatch on the keyword of the vector, if a multimethod has been defined for the keyword, then it would return another set of vectors, which will replace the original tag.
For example, the above structure will be transformed to:
[:div {:class "special"} [:div [:button "Hello"] [:button "Hi"]]]
The custom multimethod will receive the list ("hello" "hi") as parameters. It will then return the div containing the buttons.
How do I write a function that traverses the vector and dispatches on the keyword with everything else in the form as parameter, and then replaces the current form with the returned form ?
(ns customtags
(:require [clojure.walk :as walk]))
(def customtags (atom {}))
(defn add-custom-tag [tag f]
(swap! customtags assoc tag f))
(defn try-transform [[tag & params :as coll]]
(if-let [f (get #customtags tag)]
(apply f params)
coll))
(defmacro defcustomtag [tag params & body]
`(add-custom-tag ~tag (fn ~params ~#body)))
(defn apply-custom-tags [coll]
(walk/prewalk
(fn [x]
(if (vector? x)
(try-transform x)
x)) coll))
Using it:
(require '[customtags :as ct])
(ct/defcustomtag :btn-grp [& coll] (into [:div] (map (fn [x] [:button x]) coll)))
(ct/defcustomtag :button [name] [:input {:type "button" :id name}])
(def data [:div {:class "special"} [:btn-grp "Hello" "Hi"]])
(ct/apply-custom-tags data)
[:div {:class "special"} [:div [:input {:type "button", :id "Hello"}] [:input {:type "button", :id "Hi"}]]]