I'm using Datomic, although it doesn't particularly matter for this question. But it typically returns namespaced keys (and enum values are returned as namespaces keywords also). I want to translate the potentially nested structure to strip the namespaces from the keys and from values (and also string-ify the values of enums). I'm doing this because I'll return the result in a JSON REST API and the namespacing doesn't make much sense in that context. Here's a simple example structure:
{
:person/name "Kevin"
:person/age 99
:person/gender :gender/M
:person/address {
:address/state :state/NY
:address/city "New York"
:address/zip "99999"
}
}
And I'm hoping to translate to:
{
:name "Kevin"
:age 99
:gender "M"
:address {
:state "NY"
:city "New York"
:zip "99999"
}
}
One thing I know I can do is use (postwalk-replace {:person/name :name :person/age :age :person/gender :gender :person/address :address :address/city :city :address/state :state :address/zip :zip} the-entity) and that covers the keys, but not the values.
What other options do I have?
You can use clojure.walk/postwalk. Simple version doesn't differentiate between keywords as keys or values in a map, simply converts all keys to strings:
(def data {:person/name "Kevin"
:person/age 99
:person/gender :gender/M
:person/address {:address/state :state/NY
:address/city "New York"
:address/zip "99999"}})
(clojure.walk/postwalk
(fn [x]
(if (keyword? x)
(name x)
x))
data)
;; => => {"name" "Kevin", "age" 99, "gender" "M", "address" {"state" "NY", "city" "New York", "zip" "99999"}}
To implement exactly what you want you need to handle keys and values in a map separately:
(defn transform-keywords [m]
(into {}
(map (fn [[k v]]
(let [k (if (keyword? k) (keyword (name k)) k)
v (if (keyword? v) (name v) v)]
[k v]))
m)))
(clojure.walk/postwalk
(fn [x]
(if (map? x)
(transform-keywords x)
x))
data)
;; => => {:name "Kevin", :age 99, :gender "M", :address {:state "NY", :city "New York", :zip "99999"}}
As a side note: in my experience, the impedance mismatch between namespace-qualified and non-namespace-qualified keys at the boundary of your system can be an ongoing pain; what's more, having namespaced-qualified keys has significant advantages regarding code clarity (very good data traceability).
So I wouldn't give up namespace-qualified keys too readily. If EDN's syntax for namespacing (with dots and slashes) doesn't suit the consumers of your API, you may even want to use something more conventional like underscores (e.g :person_name instead of :person/name); a bit uglier, but it still gives you most of the benefits of namespace-qualified keys, you will not even need to transform the data structures, and Datomic won't mind.
Related
I start learn Clojure and need help with task.
I have to write this function:
(data-table student-tbl)
;; => ({:surname "Ivanov", :year "1996", :id "1"}
;; {:surname "Petrov", :year "1996", :id "2"}
;; {:surname "Sidorov", :year "1997", :id "3"})
I must use let, map, next, table-keys and data-record functions.
In this case:
student-tbl => (["id" "surname" "year" "group_id"] ["1" "Ivanov" "1998"] ["2" "Petrov" "1997"] ["3" "Sidorov" "1996"])
(table-keys student-tbl) => [:id :surname :year :group_id]
(data-record [:id :surname :year :group_id] ["1" "Ivanov" "1996"]) => {:surname "Ivanov", :year "1996", :id "1"}
I wrote this:
(defn data-table [tbl]
(let [[x] (next tbl)]
(data-record (table-keys tbl) x)
))
(data-table student-tbl) => {:surname "Ivanov", :year "1998", :id "1"}
How I can use map for right result?
First, here is how you should probably write this in practice. Then I'll show you your mistake so you can learn for your homework.
One way:
(defn data-table
[[headers & data]]
(let [headers (map keyword headers)
data-record (partial zipmap headers)]
(map data-record data)))
The key takeaways here are:
destructure the input to go ahead and separate headers from data
build the headers once, using the core keyword function
compose a function which always takes the same set of headers, and then map that function over our data
note that there are no external functions, which is always a nice thing when we can get away with it
Now, to make your way work, what you need to do is map the data-record function over x. First, the let binding should bind (next tbl) to x, not [x] (the way you're doing it, you only get the first element of the data set (Ivanov, 1998, 1).
In this example, ignore the data-record zipmap and table-keys binding in the let. They're there to make this example work, and you can remove them safely.
(defn data-table-newb
[tbl]
(let [table-keys #(map keyword (first %))
headers (table-keys tbl)
data-record zipmap
x (next tbl)]
(map #(data-record headers %) x)))
Essentially, you compute your table headers at the beginning, then create a new anonymous function that calls data-record and gives it your computed headers and an individual vector of data. You apply that function over every element of your data list, which you have bound to x.
Removing the unnecessary functions which are defined elsewhere, you get:
(defn data-table-newb
[tbl]
(let [headers (table-keys tbl)
x (next tbl)]
(map #(data-record headers %) x)))
Say I have a list of maps that looks like the following:
(def my-map '({:some-key {:another-key "val"}
:id "123"}
{:some-key {:another-key "val"}
:id "456"}
{:some-other-key {:a-different-key "val2"}
:id "789"})
In my attempt to filter this map by :another-key, I tried this:
(filter #(= "val" ((% :some-key) :another-key)) my-map)))
However, this will throw a NullPointerException on the map entry that doesn't contain the key I'm filtering on. What would be the optimal way to filter this map, excluding entries that don't match the filtered schema entirely?
Your first lookup of the key :some-key will return nil if the map key is not in the map. Calling nil will result in the NPE you see.
The solution is easy, just make the keyword lookup itself in the map which work even if given a nil:
(def my-map '({:some-key {:another-key "val"}
:id "123"}
{:some-key {:another-key "val"}
:id "456"}
{:some-other-key {:a-different-key "val2"}
:id "789"}))
(filter #(= "val" (:another-key (% :some-key))) my-map)
You can also use get-in:
(filter #(= "val" (get-in % [:some-key :another-key])) my-map)
And if your list could potentially have nil items:
(filter #(= "val" (:another-key (:some-key %))) my-map)
Explanation:
(:k nil);; => nil
(nil :k);; => NPE
({:k 4} :k);; => 4
(:k {:k 4});; => 4
;; BTW, you can also specify the "not found" case:
(:k nil :not-there);; => :not-there
See also the clojure style guide.
I'm trying to figure out an idiomatic, performant, and/or highly functional way to do the following:
I have a sequence of maps that looks like this:
({:_id "abc" :related ({:id "123"} {:id "234"})}
{:_id "bcd" :related ({:id "345"} {:id "456"})}
{:_id "cde" :related ({:id "234"} {:id "345"})})
The :id fields can be assumed to be unique within any one :_id.
In addition, I have two sets:
ids like ("234" "345") and
substitutes like ({:id "111"} {:id "222"})
Note that the fact that substitutes only has :id in this example doesn't mean it can be reduced to a collection of ids. This is a simplified version of a problem and the real data has other key/value pairs in the map that have to come along.
I need to return a new sequence that is the same as the original but with the values from substitutes replacing the first occurrence of the matching id from ids in the :related collections of all of the items. So what the final collection should look like is:
({:_id "abc" :related ({:id "123"} {:id "111"})}
{:_id "bcd" :related ({:id "222"} {:id "456"})}
{:_id "cde" :related ({:id "234"} {:id "345"})})
I'm sure I could eventually code up something that involves nesting maps and conditionals (thinking in iterative terms about loops of loops) but that feels to me like I'm not thinking functionally or cleverly enough given the tools I might have available, either in clojure.core or extensions like match or walk (if those are even the right libraries to be looking at).
Also, it feels like it would be much easier without the requirement to limit it to a particular strategy (namely, subbing on the first match only, ignoring others), but that's a requirement. And ideally, a solution would be adaptable to a different strategy down the line (e.g. a single, but randomly positioned match). The one invariant to strategy is that each id/sub pair should used only once. So:
Replace one, and one only, occurrence of a :related value whose :id matches a value from ids with the corresponding value from substitutes, where the one occurrence is the first (or nth or rand-nth...) occurrence.
(def id-mapping (zipmap ids
(map :id substitutes)))
;; id-mapping -> {"345" "222", "234" "111"}
(clojure.walk/prewalk-replace id-mapping original)
Assuming that the collection is called results:
(require '[clojure.zip :as z])
(defn modify-related
[results id sub]
(loop [loc (z/down (z/seq-zip results))
done? false]
(if (= done? true)
(z/root loc)
(let [change? (->> loc z/node :_id (= id))]
(recur (z/next (cond change?
(z/edit loc (fn [_] identity sub))
:else loc))
change?)))))
(defn modify-results
[results id sub]
(loop [loc (z/down (z/seq-zip results))
done? false]
(if (= done? true)
(z/root loc)
(let [related (->> loc z/node :related)
change? (->> related (map :_id) set (#(contains? % id)))]
(recur (z/next (cond change?
(z/edit loc #(assoc % :related (modify-related related id sub)))
:else loc))
change?)))))
(defn sub-for-first
[results ids substitutes]
(let [subs (zipmap ids substitutes)]
(reduce-kv modify-results results subs)))
I've got two databases that I'm attempting to keep in sync using a bit of Clojure glue code.
I'd like to make something like a clojure.set/difference that operates on values projected by a function.
Here's some sample data:
(diff #{{:name "bob smith" :favourite-colour "blue"}
{:name "geraldine smith" :age 29}}
#{{:first-name "bob" :last-name "smith" :favourite-colour "blue"}}
:name
(fn [x] (str (:first-name x) " " (:last-name x))))
;; => {:name "geraldine smith" :age 29}
The best I've got is:
(defn diff
"Return members of l who do not exist in r, based on applying function
fl to left and fr to right"
[l r fl fr]
(let [l-project (into #{} (map fl l))
r-project (into #{} (map fr r))
d (set/difference l-project r-project)
i (group-by fl l)]
(map (comp first i) d)))
But I feel that this is a bit unwieldly, and I can't imagine it performs very well. I'm throwing away information that I'd like to keep, and then looking it up again.
I did have a go using metadata, to keep the original values around during the set difference, but I can't seem put metadata on primitive types, so that didn't work...
I'm not sure why, but I have this tiny voice inside my head telling me that this kind of operation on the side is what monads are for, and that I should really get around to finding out what a monad is and how to use it. Any guidance as to whether the tiny voice is right is very welcome!
(defn diff
[l r fl fr]
(let [r-project (into #{} (map fr r))]
(set (remove #(contains? r-project (fl %)) l))))
This no longer exposes the difference operation directly (it is now implicit with the remove / contains combination), but it is succinct and should give the result you are looking for.
example usage and output:
user> (diff #{{:name "bob smith" :favourite-colour "blue"}
{:name "geraldine smith" :age 29}}
#{{:first-name "bob" :last-name "smith" :favourite-colour "blue"}}
:name
(fn [x] (str (:first-name x) " " (:last-name x))))
#{{:age 29, :name "geraldine smith"}}
I have 2 data structures like the ones below
(ns test)
(def l
[{:name "Sean" :age 27}
{:name "Ross" :age 27}
{:name "Brian" :age 22}])
(def r
[{:owner "Sean" :item "Beer" }
{:owner "Sean" :item "Pizza"}
{:owner "Ross" :item "Computer"}
{:owner "Matt" :item "Bike"}])
I want to have get persons who dont own any item . (Brian in this case so [ {:name "Brian" :age 22}]
If this was SQL I would do left outer join or not exists but I not sure how to do this in clojure in more performant way.
While Chuck's solution is certainly the most sensible one, I find it interesting that it is possible to write a solution in terms of relational algebraic operators using clojure.set:
(require '[clojure.set :as set])
(set/difference (set l)
(set/project (set/join r l {:owner :name})
#{:name :age}))
; => #{{:name "Brian", :age 22}}
You basically want to do a filter on l, but negative. We could just not the condition, but the remove function already does this for us. So something like:
(let [owner-names (set (map :owner r))]
(remove #(owner-names (% :name)) l))
(I think it reads more nicely with the set, but if you want to avoid allocating the set, you can just do (remove (fn [person] (some #(= (% :owner) (person :name)) r)) l).)