I have a set of values in Clojure that I want to structure similar to record. I'm trying to figure out the best way of handling a set of those records.
So I have a for example a record:
(defrecord Link [page url])
Whats the best datastructure to hold a collection of these records that I can step through recursively, while continuously updating the collection?
Previously I've done this on a single value using sequence, then concat'ing new links on the end as i process them recursively. But now I want to hold more information about each link.
Edit For Clarity
I had previously been using maps, however I think I've been confusing myself by trying to use a nested map with like
#{:rootlink "http://www.google.co.uk" :links nestedmapoflinks}
which is confusing me when i try to re curse through it.
Below is the code that I've been using, below is what currently works with a sequence of links but no other info about the link.
(defn get-links
[url]
(map :href (map :attrs (html/select (fetch-url url) [:a])))))
(defn process-links
[links]
(if (not (empty? links))
(do
(if (not (is-working (first links)))
(do
(println (str (first links) " is not working"))
(recur (rest links)))
(do
(println (str (first links) " is working"))
(recur (concat (rest links) (get-links (first links)))))))))
I think I have to add each item into the map with
{:rootlink "http://www.google.co.uk" :link "http://someurlontherootlinkpage.com"}
instead of trying to work with the nested map.
However, the reason I mentioned records, because I was struggling to merge two maps together using the first method of map creation. I'm still a bit confused about the best structure to use to recurse through the map.
Final Update
Ok, so after much wrangling I finally came up with this below piece of code which returns a seq of vectors made up of:
["root link address" "link"]
["http://www.google.co.uk" "http://www.google.co.uk/examplelink"]
Code:
(defn get-links
[url]
(map #(vector url %)(map :href (map :attrs (html/select (fetch-url url) [:a])))))
The code is now on my github available in my profile.
I think you are getting confused between using Tree type structure or a flat structure.
Lets say you have a list of links as a vector of maps:
[ {:root nil :link "A.COM"} {:root nil :link "B.COM"} ]
Now you map over it and using your get-link method you get:
[ [ {:root nil :link "A.COM"} {:root "A.COM" :link "Aa.COM"} {:root "A.COM" :link "Ab.COM"} ] [ {:root nil :link "B.COM"} {:root "B.COM" :link "Ba.COM"} {:root "B.COM" :link "Bb.COM"}] ]
Now you can call flatten on this result to get a flat list of link instead of nested map in vector.
You can repeat this process recursively till you exit condition met.
Related
I'm developing a mini-social media API where the user is allowed to insert a new profile, connect two profiles together (like friends) and then receive recommendations based on the "friends of my friends" rule.
Right now I'm trying to create the API for Profile.
I have an atom that holds a list of maps, one for each profile.
(def profiles (atom ()))
(defn create [request]
(swap! profiles conj {:id (get-in request [:body "id"])
:name (get-in request [:body "name"])
:age (get-in request [:body "age"])
:recommendable (get-in request [:body "recommendable"])
:friends (list)
})
(created "")
)
I was trying to develop the find-by-id for the GET http verb for the API when I stumbled into a problem. How can I get the values from the maps within said list so I can apply functions to it?
For instance, here I was trying to use the filter function to return me only the maps that contained a given id. But I keep getting an error:
(defn find-by-id [id]
(filter #(= (:id %) id) profiles)
)
Dont know how to create ISeq from: clojure.lang.Atom
It seems to me that filter is not applicable to an Atom.
Same thing happens to remove:
(defn delete-by-id [id]
(swap! profiles (remove #(= (:id %) id) profiles))
)
When I try with #profiles I get an empty array as a result. And to make things worst when I tried the filter function using REPL it worked just fine.
Which leaves me wondering what am I missing here.
Could anyone please tell me what's going on?
Thanks in advance.
The first one fails because, as it says, atoms aren't a sequence, which filter is expecting.
You need to get the sequence out of the atom before you can filter it:
; I'm dereferencing the atom using # to get the list of profiles that it holds
(defn find-by-id [id]
(filter #(= (:id %) id) #profiles))
Note though, this isn't optimal. You're relying on the state of profiles that can change at seemingly random times (if you have asynchronous processes swap!ping it). It may complicate debugging since you can't get a good handle on the data before it's passed to filter. It also isn't good for the function to rely on profiles being an atom, since that's irrelevant to its function, and you may change your design later. It would be more future proof to make this function rely purely on its parameters and have no knowledge of the atom:
(defn find-by-id [id profiles]
(filter #(= (:id %) id) profiles))
; Then call it like this. I renamed your atom here
(find-by-id some-id #profile-atom)
Your second example fails because swap! accepts a function as its second argument. I think you meant to use reset!, which changes the value of the atom regardless of what it was before:
(defn delete-by-id [id]
(reset! profiles (remove #(= (:id %) id) #profiles)))
Although, this isn't optimal either. If you want to update an atom based on a previous state, use swap! instead and supply an updating function:
(defn delete-by-id [id]
(swap! profile-atom (fn [profiles] (remove #(= (:id %) id)) profiles)))
Or, slightly more succinctly:
(defn delete-by-id [id]
(swap! profile-atom (partial remove #(= (:id %) id))))
I'm partially applying remove to make a function. The old state of the atom is passed as the last argument to remove.
Consider the following clojurescript code where the specter, reagent and re-frame frameworks are used, an external React.js grid component is used as a view component.
In db.cls :
(def default-db
{:cats [{:id 0 :data {:text "ROOT" :test 17} :prev nil :par nil}
{:id 1 :data {:text "Objects" :test 27} :prev nil :par 0}
{:id 2 :data {:text "Version" :test 37} :prev nil :par 1}
{:id 3 :data {:text "X1" :test 47} :prev nil :par 2}]})
In subs.cls
(register-sub
:cats
(fn [db]
(reaction
(select [ALL :data] (t/tree-visitor (get #db :cats))))))
result from select:
[{:text "ROOT", :test 17}
{:text "Objects", :test 27}
{:text "Version", :test 37}
{:text "X1", :test 47}]
In views.cls
(defn categorymanager []
(let [cats (re-frame/subscribe [:cats])]
[:> Reactable.Table
{:data (clj->js #cats)}]))
The code above works as expected.
Instead of displaying the data with the react.js component I want to go through each of the maps in the :cats vector and display the :text items in html ul / li.
I started as follows:
(defn categorymanager2 []
(let [cats (re-frame/subscribe [:cats])]
[:div
[:ul
(for [category #cats]
;;--- How to continue here ?? ---
)
))
Expected output:
ROOT
Objects
Version
X1
How do I loop through a subscribed collection in re-frame and display the data as a list-item? ( = question for title ).
First, be clear why you use key...
Supplying a key for each item in a list is useful when that list is quite dynamic - when new list items are being regularly added and removed, especially if that list is long, and the items are being added/removed near the top of the list.
keys can deliver big performance gains, because they allow React to more efficiently redraw these changeable lists. Or, more accurately, it allows React to avoid redrawing items which have the same key as last time, and which haven't changed, and which have simply shuffled up or down.
Second, be clear what you should do if the list is quite static (it does not change all the time) OR if there is no unique value associated with each item...
Don't use :key at all. Instead, use into like this:
(defn categorymanager []
(let [cats (re-frame/subscribe [:cats])]
(fn []
[:div
(into [:ul] (map #(vector :li (:text %)) #cats))])))
Notice what has happened here. The list provided by the map is folded into the [:ul] vector. At the end of it, no list in sight. Just nested vectors.
You only get warnings about missing keys when you embed a list into hiccup. Above there is no embedded list, just vectors.
Third, if your list really is dynamic...
Add a unique key to each item (unique amoung siblings). In the example given, the :text itself is a good enough key (I assume it is unique):
(defn categorymanager []
(let [cats (re-frame/subscribe [:cats])]
(fn []
[:div
[:ul (map #(vector :li {:key (:text %)} (:text %)) #cats)]])))
That map will result in a list which is the 1st parameter to the [:ul]. When Reagent/React sees that list it will want to see keys on each item (remember lists are different to vectors in Reagent hiccup) and will print warnings to console were keys to be missing.
So we need to add a key to each item of the list. In the code above we aren't adding :key via metadata (although you can do it that way if you want), and instead we are supplying the key via the 1st parameter (of the [:li]), which normally also carries style data.
Finally - part 1 DO NOT use map-indexed as is suggested in another answer.
key should be a unique value associated with each item. Attaching some arb integer does nothing useful - well, it does get rid of the warnings in the console, but you should use the into technique above if that's all you want.
Finally - part 2 there is no difference between map and for in this context.
They both result in a list. If that list has keys then no warning. But if keys are missing, then lots of warnings. But how the list was created doesn't come into it.
So, this for version is pretty much the same as the map version. Some may prefer it:
(defn categorymanager []
(let [cats (re-frame/subscribe [:cats])]
(fn []
[:div
[:ul (for [i #cats] [:li {:key (:text i)} (:text i)])]])))
Which can also be written using metadata like this:
(defn categorymanager []
(let [cats (re-frame/subscribe [:cats])]
(fn []
[:div
[:ul (for [i #cats] ^{:key (:text i)}[:li (:text i)])]])))
Finally - part 3
mapv is a problem because of this issue:
https://github.com/Day8/re-frame/wiki/Using-%5Bsquare-brackets%5D-instead-of-%28parentheses%29#appendix-2
Edit: For a much more coherent and technically correct explanation of keys and map, see Mike Thompson's answer!
Here's how I would write it:
(defn categorymanager2 []
(let [cats (re-frame/subscribe [:cats])]
(fn []
[:div
[:ul
(map-indexed (fn [n cat] ;;; !!! See https://stackoverflow.com/a/37186230/500207 !!!
^{:key n}
[:li (:text cat)])
#cats)]])))
(defn main-panel []
[:div
[categorymanager2]])
A few points:
See the re-frame readme's Subscribe section, near the end, which says:
subscriptions can only be used in Form-2 components and the subscription must be in the outer setup function and not in the inner render function. So the following is wrong (compare to the correct version above)…
Therefore, your component was ‘wrong’ because it didn't wrap the renderer inside an inner function. The readme has all the details, but in short, not wrapping a component renderer that depends on a subscription inside an inner function is bad because this causes the component to rerender whenever db changes—not what you want! You want the component to only rerender when the subscription changes.
Edit: seriously, see Mike Thompson's answer. For whatever reason, I prefer using map to create a seq of Hiccup tags. You could use a for loop also, but the critical point is that each [:li] Hiccup vector needs a :key entry in its meta-data, which I add here by using the current category's index in the #cats vector. If you don't have a :key, React will complain in the Dev Console. Note that this key should somehow uniquely tie this element of #cats to this tag: if the cats subscription changes and gets shuffled around, the result might not be what you expect because I just used this very simple key. If you can guarantee that category names will be unique, you can just use the :test value, or the :test value, or something else. The point is, the key must be unique and must uniquely identify this element.
(N.B.: don't try and use mapv to make a vector of Hiccup tags—re-frame hates that. Must be a seq like what map produces.)
I also included an example main-panel to emphasize that
parent components don't need the subscriptions that their children component need, and that
you should call categorymanager2 component with square-brackets instead of as a function with parens (see Using [] instead of ()).
Here's an ul / li example:
(defn phone-component
[phone]
[:li
[:span (:name #phone)]
[:p (:snippet #phone)]])
(defn phones-component
[]
(let [phones (re-frame/subscribe [:phones])] ; subscribe to the phones value in our db
(fn []
[:ul (for [phone in #phones] ^{:key phone} [phone-component phone] #phones)])))
I grabbed that code from this reframe tutorial.
Also map is preferable to for when using Reagent. There is a technical reason for this, it is just that I don't know what it is.
I'm trying to figure out an idiomatic, performant, and/or highly functional way to do the following:
I have a sequence of maps that looks like this:
({:_id "abc" :related ({:id "123"} {:id "234"})}
{:_id "bcd" :related ({:id "345"} {:id "456"})}
{:_id "cde" :related ({:id "234"} {:id "345"})})
The :id fields can be assumed to be unique within any one :_id.
In addition, I have two sets:
ids like ("234" "345") and
substitutes like ({:id "111"} {:id "222"})
Note that the fact that substitutes only has :id in this example doesn't mean it can be reduced to a collection of ids. This is a simplified version of a problem and the real data has other key/value pairs in the map that have to come along.
I need to return a new sequence that is the same as the original but with the values from substitutes replacing the first occurrence of the matching id from ids in the :related collections of all of the items. So what the final collection should look like is:
({:_id "abc" :related ({:id "123"} {:id "111"})}
{:_id "bcd" :related ({:id "222"} {:id "456"})}
{:_id "cde" :related ({:id "234"} {:id "345"})})
I'm sure I could eventually code up something that involves nesting maps and conditionals (thinking in iterative terms about loops of loops) but that feels to me like I'm not thinking functionally or cleverly enough given the tools I might have available, either in clojure.core or extensions like match or walk (if those are even the right libraries to be looking at).
Also, it feels like it would be much easier without the requirement to limit it to a particular strategy (namely, subbing on the first match only, ignoring others), but that's a requirement. And ideally, a solution would be adaptable to a different strategy down the line (e.g. a single, but randomly positioned match). The one invariant to strategy is that each id/sub pair should used only once. So:
Replace one, and one only, occurrence of a :related value whose :id matches a value from ids with the corresponding value from substitutes, where the one occurrence is the first (or nth or rand-nth...) occurrence.
(def id-mapping (zipmap ids
(map :id substitutes)))
;; id-mapping -> {"345" "222", "234" "111"}
(clojure.walk/prewalk-replace id-mapping original)
Assuming that the collection is called results:
(require '[clojure.zip :as z])
(defn modify-related
[results id sub]
(loop [loc (z/down (z/seq-zip results))
done? false]
(if (= done? true)
(z/root loc)
(let [change? (->> loc z/node :_id (= id))]
(recur (z/next (cond change?
(z/edit loc (fn [_] identity sub))
:else loc))
change?)))))
(defn modify-results
[results id sub]
(loop [loc (z/down (z/seq-zip results))
done? false]
(if (= done? true)
(z/root loc)
(let [related (->> loc z/node :related)
change? (->> related (map :_id) set (#(contains? % id)))]
(recur (z/next (cond change?
(z/edit loc #(assoc % :related (modify-related related id sub)))
:else loc))
change?)))))
(defn sub-for-first
[results ids substitutes]
(let [subs (zipmap ids substitutes)]
(reduce-kv modify-results results subs)))
I'm generating json as literally as I can in clojure. My problem is that certain branches of the json are only present if given parameters are given. Here is a sample of such a condition
(defn message-for
[name uuid & [generated-uuids]]
{:message {:id (generate-uuid)
:details {:name name}
:metadata {:batch (merge {:id uuid}
(when generated-uuids (let [batches (map #(array-map :id %) generated-uuids)]
{:generatedBatches batches})))}}})
Unfortunately the when/let part is quite ugly. This same could be achieved using when-let as following but it doesn't work because my map returns [] instead of a nil.
(defn message-for
[name uuid & [generated-uuids]]
{:message {:id (generate-uuid)
:details {:name name}
:metadata {:batch (merge {:id uuid}
(when-let [batches (map #(array-map :id %) generated-uuids)]
{:generatedBatches batches}))}}})
Any ideas if I could somehow make when-let consider an empty list/array/seq as false so I could clean up my code a bit?
not-empty returns its argument if it is not empty.
When using when-let with a collection, always use not-empty
to retain the collection type
make refactoring easier
expressivenes
(when-let [batches (not-empty (map ...))]
...)
In your case I'd however prefer something like this:
...
:metadata {:batch (cond-> {:id uuid}
(seq generated-uuids)
(assoc :generatedBatches (map ...)))}
...
Notice that all three of the advantages listed above where met, without a nested let.
Also notice a new advantage
easier to extend with more conditions lateron
seq returns nil on an empty input sequence so you could do:
(when-let [batches (seq (map #(array-map :id %) generated-uuids))]
{:generatedBatches batches}))}}})
Here is an example where I use html/text directly inside a selector vector.
(:use [net.cgrand.enlive-html :as html])
(defn fetch-url [url]
(html/html-resource (java.net.URL. url)))
(defn parse-test []
(html/select
(fetch-url "https://news.ycombinator.com/")
[:td.title :a html/text]))
Calling (parse-test) returns a data structure containing Hacker News Headlines :
("In emergency cases a passenger was selected and thrown out of the plane. [2004]"
"“Nobody expects privacy online”: Wrong."
"The SCUMM Diary: Stories behind one of the greatest game engines ever made" ...)
Cool!
Would it be possible to end the selector vector with a custom function that would give me back the list of article URLs.
Something like: [:td.title :a #(str "https://news.ycombinator.com/" (:href (:attrs %)))]
EDIT:
Here is a way to achieve this. We could write our own select function:
(defn select+ [coll selector+]
(map
(peek selector+)
(html/select
(fetch-url "https://news.ycombinator.com/")
(pop selector+))))
(def href
(fn [node] (:href (:attrs node))))
(defn parse-test []
(select+
(fetch-url "https://news.ycombinator.com/")
[:td.title :a href]))
(parse-test)
As you suggest in your comment, I think it's clearest to keep the selection and the transformation of nodes separate.
Enlive itself provides both selectors and transformers. Selectors to find nodes, and transformers to, um, transform them. If your intended output was html, you could probably use a combination of a selector and a transformer to achieve your desired result.
However, seeing as you are just looking for data (a sequence of maps, perhaps?) - you can skip the transform bit, and just use a sequence comprehension, like this:
(defn parse-test []
(for [s (html/select
(fetch-url "https://news.ycombinator.com/")
[:td.title :a])]
{:title (first (:content s))
:link (:href (:attrs s))}))
(take 2 (parse-test))
;; => ({:title " \tStartup - Bill Watterson, a cartoonist's advice ",
:link "http://www.zenpencils.com/comic/128-bill-watterson-a-cartoonists-advice"}
{:title "Drug Agents Use Vast Phone Trove Eclipsing N.S.A.’s",
:link "http://www.nytimes.com/2013/09/02/us/drug-agents-use-vast-phone-trove-eclipsing-nsas.html?hp&_r=0&pagewanted=all"})