I start learn Clojure and need help with task.
I have to write this function:
(data-table student-tbl)
;; => ({:surname "Ivanov", :year "1996", :id "1"}
;; {:surname "Petrov", :year "1996", :id "2"}
;; {:surname "Sidorov", :year "1997", :id "3"})
I must use let, map, next, table-keys and data-record functions.
In this case:
student-tbl => (["id" "surname" "year" "group_id"] ["1" "Ivanov" "1998"] ["2" "Petrov" "1997"] ["3" "Sidorov" "1996"])
(table-keys student-tbl) => [:id :surname :year :group_id]
(data-record [:id :surname :year :group_id] ["1" "Ivanov" "1996"]) => {:surname "Ivanov", :year "1996", :id "1"}
I wrote this:
(defn data-table [tbl]
(let [[x] (next tbl)]
(data-record (table-keys tbl) x)
))
(data-table student-tbl) => {:surname "Ivanov", :year "1998", :id "1"}
How I can use map for right result?
First, here is how you should probably write this in practice. Then I'll show you your mistake so you can learn for your homework.
One way:
(defn data-table
[[headers & data]]
(let [headers (map keyword headers)
data-record (partial zipmap headers)]
(map data-record data)))
The key takeaways here are:
destructure the input to go ahead and separate headers from data
build the headers once, using the core keyword function
compose a function which always takes the same set of headers, and then map that function over our data
note that there are no external functions, which is always a nice thing when we can get away with it
Now, to make your way work, what you need to do is map the data-record function over x. First, the let binding should bind (next tbl) to x, not [x] (the way you're doing it, you only get the first element of the data set (Ivanov, 1998, 1).
In this example, ignore the data-record zipmap and table-keys binding in the let. They're there to make this example work, and you can remove them safely.
(defn data-table-newb
[tbl]
(let [table-keys #(map keyword (first %))
headers (table-keys tbl)
data-record zipmap
x (next tbl)]
(map #(data-record headers %) x)))
Essentially, you compute your table headers at the beginning, then create a new anonymous function that calls data-record and gives it your computed headers and an individual vector of data. You apply that function over every element of your data list, which you have bound to x.
Removing the unnecessary functions which are defined elsewhere, you get:
(defn data-table-newb
[tbl]
(let [headers (table-keys tbl)
x (next tbl)]
(map #(data-record headers %) x)))
Related
Let's say I have several vectors
(def coll-a [{:name "foo"} ...])
(def coll-b [{:name "foo"} ...])
(def coll-c [{:name "foo"} ...])
and that I would like to see if the names of the first elements are equal.
I could
(= (:name (first coll-a)) (:name (first coll-b)) (:name (first coll-c)))
but this quickly gets tiring and overly verbose as more functions are composed. (Maybe I want to compare the last letter of the first element's name?)
To directly express the essence of the computation it seems intuitive to
(apply = (map (comp :name first) [coll-a coll-b coll-c]))
but it leaves me wondering if there's a higher level abstraction for this sort of thing.
I often find myself comparing / otherwise operating on things which are to be computed via a single composition applied to multiple elements, but the map syntax looks a little off to me.
If I were to home brew some sort of operator, I would want syntax like
(-op- (= :name first) coll-a coll-b coll-c)
because the majority of the computation is expressed in (= :name first).
I'd like an abstraction to apply to both the operator & the functions applied to each argument. That is, it should be just as easy to sum as compare.
(def coll-a [{:name "foo" :age 43}])
(def coll-b [{:name "foo" :age 35}])
(def coll-c [{:name "foo" :age 28}])
(-op- (+ :age first) coll-a coll-b coll-c)
; => 106
(-op- (= :name first) coll-a coll-b coll-c)
; => true
Something like
(defmacro -op-
[[op & to-comp] & args]
(let [args' (map (fn [a] `((comp ~#to-comp) ~a)) args)]
`(~op ~#args')))
Is there an idiomatic way to do this in clojure, some standard library function I could be using?
Is there a name for this type of expression?
For your addition example, I often use transduce:
(transduce
(map (comp :age first))
+
[coll-a coll-b coll-c])
Your equality use case is trickier, but you could create a custom reducing function to maintain a similar pattern. Here's one such function:
(defn all? [f]
(let [prev (volatile! ::no-value)]
(fn
([] true)
([result] result)
([result item]
(if (or (= ::no-value #prev)
(f #prev item))
(do
(vreset! prev item)
true)
(reduced false))))))
Then use it as
(transduce
(map (comp :name first))
(all? =)
[coll-a coll-b coll-c])
The semantics are fairly similar to your -op- macro, while being both more idiomatic Clojure and more extensible. Other Clojure developers will immediately understand your usage of transduce. They may have to investigate the custom reducing function, but such functions are common enough in Clojure that readers can see how it fits an existing pattern. Also, it should be fairly transparent how to create new reducing functions for use cases where a simple map-and-apply wouldn't work. The transducing function can also be composed with other transformations such as filter and mapcat, for cases when you have a more complex initial data structure.
You may be looking for the every? function, but I would enhance clarity by breaking it down and naming the sub-elements:
(let [colls [coll-a coll-b coll-c]
first-name (fn [coll] (:name (first coll)))
names (map first-name colls)
tgt-name (first-name coll-a)
all-names-equal (every? #(= tgt-name %) names)]
all-names-equal => true
I would avoid the DSL, as there is no need and it makes it much harder for others to read (since they don't know the DSL). Keep it simple:
(let [colls [coll-a coll-b coll-c]
vals (map #(:age (first %)) colls)
result (apply + vals)]
result => 106
I don't think you need a macro, you just need to parameterize your op function and compare functions. To me, you are pretty close with your (apply = (map (comp :name first) [coll-a coll-b coll-c])) version.
Here is one way you could make it more generic:
(defn compare-in [op to-compare & args]
(apply op (map #(get-in % to-compare) args)))
(compare-in + [0 :age] coll-a coll-b coll-c)
(compare-in = [0 :name] coll-a coll-b coll-c)
;; compares last element of "foo"
(compare-in = [0 :name 2] coll-a coll-b coll-c)
I actually did not know you can use get on strings, but in the third case you can see we compare the last element of each foo.
This approach doesn't allow the to-compare arguments to be arbitrary functions, but it seems like your use case mainly deals with digging out what elements you want to compare, and then applying an arbitrary function to those values.
I'm not sure this approach is better than the transducer version supplied above (certainly not as efficient), but I think it provides a simpler alternative when that efficiency is not needed.
I would split this process into three stages:
transform items in collections into the data in collections you want to operate
on - (map :name coll);
Operate on transformed items in collections, returning collection of results - (map = transf-coll-a transf-coll-b transf-coll-c)
Finally, selecting which result in resulting collection to return - (first calculated-coll)
When playing with collections, I try to put more than one item into collection:
(def coll-a [{:name "foo" :age 43} {:name "bar" :age 45}])
(def coll-b [{:name "foo" :age 35} {:name "bar" :age 37}])
(def coll-c [{:name "foo" :age 28} {:name "bra" :age 30}])
For example, matching items by second char in :name and returning result for items in second place:
(let
[colls [coll-a coll-b coll-c]
transf-fn (comp #(nth % 1) :name)
op =
fetch second]
(fetch (apply map op (map #(map transf-fn %) colls))))
;; => false
In transducers world you can use sequence function which also works on multiple collections:
(let
[colls [coll-a coll-b coll-c]
transf-fn (comp (map :name) (map #(nth % 1)))
op =
fetch second]
(fetch (apply sequence (map op) (map #(sequence transf-fn %) colls))))
Calculate sum of ages (for all items at the same level):
(let
[colls [coll-a coll-b coll-c]
transf-fn (comp (map :age))
op +
fetch identity]
(fetch (apply sequence (map op) (map #(sequence transf-fn %) colls))))
;; => (106 112)
New to clojure. Trying to solve the following problem with a java background. I need to transform table to a hash-map that maps products to all the cities that sell the product. So the output should be.
{"Pencil": ("Oshawa" "Toronto")
"Bread": ("Ottawa" "Oshawa" "Toronto")}
(def table [
{:product "Pencil"
:city "Toronto"
:year "2010"
:sales "2653.00"}
{:product "Pencil"
:city "Oshawa"
:year "2010"
:sales "525.00"}
{:product "Bread"
:city "Toronto"
:year "2010"
:sales "136,264.00"}
{:product "Bread"
:city "Oshawa"
:year "nil"
:sales "242,634.00"}
{:product "Bread"
:city "Ottawa"
:year "2011"
:sales "426,164.00"}])
This is what I have so far. I write this code into the repl.
(let [product-cities {}]
(for [row table]
(if (= (contains? product-cities (keyword (row :product))) true)
(println "YAMON") ;;To do after. Add city to product if statement is true
(into product-cities {(keyword (row :product)) (str (row :city))}))))
However, the outcome is the following:
({:Pencil "Toronto"}
{:Pencil "Oshawa"}
{:Bread "Toronto"}
{:Bread "Oshawa"}
{:Bread "Ottawa"})
My if statement keeps returning false. I see that there are semi-circle brackets around the many hash-maps. I can't figure out why it's not returning one hashmap and why there are many hashmap? Thanks
EDIT:
QUESTION 2:
Transform table to a hash-map that maps products to the city that has the highest sale. For example, the output should look like:
{"Pencil": "Toronto"
"Bread": "Ottawa"}
I think a different strategy is needed than building up a value but here's what I'm thinking:
(reduce (fn [product-cities {:keys [product city sales]}]
(update-in product-cities [product] (fnil conj []) {(keyword city) sales}))
{}
table)
This produces the following output:
{"Bread"
[{:Toronto "136,264.00"}
{:Oshawa "242,634.00"}
{:Ottawa "426,164.00"}],
"Pencil"
[{:Toronto "2653.00"}
{:Oshawa "525.00"}]}
I could then use the reduce function again but only add the city with max sales. I don't think this is the most efficient way.
You seem to have some misconceptions about how clojure works. Coming from java it can be hard to know how to do stuff, just because it's so different. This small problem serves nicely as an introduction to how to build up a value, and I'll try to explain each part of the solution.
Immutability
It's a common pattern in java to define a variable that will hold the final result, then loop through something while adding to that variable.
That's what you're trying to do with your product-cities local. When you define a local with let in clojure it never changes, so to build up a value you need another pattern.
Adding something to a map
First let's take a look at how to "add something to a map". In clojure what you actually do is make a new map with the thing added. The old map doesn't change. We still sometimes phrase it as adding to a map, but that's just shorthand for "make a new map with the thing added".
assoc takes a map, a key and a value and returns a new map with the value added at the key. If there's already a value there it will be overwritten. We want multiple things for each key, so it's not the right thing in this case.
update is similar, but it takes a map, a key, a function and optionally arguments to that function. It will call the function with the value that's already at key as the first argument and (if present) the arguments supplied. The returned map will have the return value of the function as the new value at key. Some examples might make this clearer.
;; The function - is called with the old value of :foo and the argument supplied
;; (- 10 3)
(update {:foo 10} :foo - 3) ;=> {:foo 7}
If there's nothing already at key, the function will be called with nil as the first argument. That's what nil means, nothing.
(update {} :foo + 5) ;=> Null pointer exception. Same as (+ nil 5)
Null pointers are no good. There's a trick for avoiding them. fnil is a higher order function that takes a function and arguments. It returns a new function that will substitute a nil argument for the arguments supplied.
;; The nil here is substituted with 0, so no NPE
((fnil + 0) nil 5) ;=> 5
;; If the arg is not nil, the zero is not used.
((fnil + 0) 5 5) ;=> 10
;; So now we can update the value at :foo regardless of whether it's already there.
(update {} :foo (fnil + 0) 5) ;=> {:foo 5}
conj adds something to a collection. If that collection is a vector, it adds it at the end.
(conj [] :foo) ;=> :foo
(conj [:foo] :bar) ;=> [:foo :bar]
To add things to the map we can combine these:
(update {} "product" (fnil conj []) "city") ;=> {"product ["city"]"}
(update {"product" ["city"]} "product" (fnil conj []) "another city")
;;=> {"product" ["city" "another city"]}
Building up a value
We need to do some looping somehow. A for in clojure is a list comprehension however, and not a for loop. It will return a sequence of things, so it's not the right thing to use when you want to build up a value.
One way to do it is with loop.
With a loop you define binding names paired with their initial value. The bindings in loop can be thought of as "the things that are going to change during the loop".
One difference between loop and traditional loops is that to break out of the loop you simply don't do anything, and you need to specifically use recur to keep looping.
;; the bindings are pairs of names and initial values
(loop [product-cities {} ; This is the accumulator that we're gonna build up.
rows table] ; and these are the rows, we're gonna go through them one by one.
;; Here's the base case, if there are no rows left we return the accumulator.
(if (empty? rows)
product-cities
;; If there are rows left, we need to add to the accumulator.
(let [row (first rows)
city (:city row)
product (:product row)
new-accumulator (update product-cities product (fnil conj []) city)]
;; recur takes as many arguments as the pairs we defined
;; and "jumps" back to the loop
(recur new-accumulator (rest rows)))))
;;=> {"Pencil" ["Toronto" "Oshawa"], "Bread" ["Toronto" "Oshawa" "Ottawa"]}
This can be made nicer with some destructuring.
(loop [product-cities {}
[{:keys [city product] :as row} & rows] table]
(if (nil? row)
product-cities
(recur (update product-cities product (fnil conj []) city) rows)))
;;=> {"Pencil" ["Toronto" "Oshawa"], "Bread" ["Toronto" "Oshawa" "Ottawa"]}
loop is not much used in normal clojure though. It's too general and usually you want something more specific. Like in this case, you want to build up a value while looping through every thing in a sequence of things. That's what reduce does.
Reduce takes three arguments, a function, an initial value and a collection. The function, called a "reducing function" takes two arguments; the accumulated value so far and an item. It is called once for each item in the collection.
So the final implementation becomes:
(reduce (fn [product-cities {:keys [product city]}]
(update product-cities product (fnil conj []) city))
{}
table)
Edit:
About your comment on the other answer. update was added in clojure 1.7.0 so you're presumable on an older version. You can use update-in instead (though you should consider upgrading). It's called in exactly the same way except the key is in a vector.
(reduce (fn [product-cities {:keys [product city]}]
(update-in product-cities [product] (fnil conj []) city))
{}
table)
You need to go through the table one by one and accumulate (conj) the cities under the product:
(reduce
(fn [acc {:keys [product city]}]
(update acc product conj city))
{}
table)
;; => {"Pencil" ("Oshawa" "Toronto"), "Bread" ("Ottawa" "Oshawa" "Toronto")}
The use of the updating function (conj in this case) can be a bit tricky, so here is an alternative formation of the update:
(update acc product (fn [cities]
(conj cities city)))
Instead of {} I started out with:
{"Pencil" []
"Bread" []}
That might make it easier to see that for each entry in the table the update is updating the product key ("Pencil" or "Bread") by putting the latest city on the end (that's what conj does) of the sequence. When it was working I just replaced with {}, using the fact that update will insert a new key if one is not there.
I think of for as being generative. It takes a sequence as input and at each step generates a new thing - hence you end up with a sequence of new things. There is no updating of product-cities possible with generation.
reduce is more useful for what you want to do as you get an 'accumulator' that can be slightly modified at each step. Actually you are creating a new 'accumulator' each time, by modifying the one that is passed in, so in reality you are not modifying anything at all: Clojure being a functional language, its all about creating new things.
I'm trying to figure out an idiomatic, performant, and/or highly functional way to do the following:
I have a sequence of maps that looks like this:
({:_id "abc" :related ({:id "123"} {:id "234"})}
{:_id "bcd" :related ({:id "345"} {:id "456"})}
{:_id "cde" :related ({:id "234"} {:id "345"})})
The :id fields can be assumed to be unique within any one :_id.
In addition, I have two sets:
ids like ("234" "345") and
substitutes like ({:id "111"} {:id "222"})
Note that the fact that substitutes only has :id in this example doesn't mean it can be reduced to a collection of ids. This is a simplified version of a problem and the real data has other key/value pairs in the map that have to come along.
I need to return a new sequence that is the same as the original but with the values from substitutes replacing the first occurrence of the matching id from ids in the :related collections of all of the items. So what the final collection should look like is:
({:_id "abc" :related ({:id "123"} {:id "111"})}
{:_id "bcd" :related ({:id "222"} {:id "456"})}
{:_id "cde" :related ({:id "234"} {:id "345"})})
I'm sure I could eventually code up something that involves nesting maps and conditionals (thinking in iterative terms about loops of loops) but that feels to me like I'm not thinking functionally or cleverly enough given the tools I might have available, either in clojure.core or extensions like match or walk (if those are even the right libraries to be looking at).
Also, it feels like it would be much easier without the requirement to limit it to a particular strategy (namely, subbing on the first match only, ignoring others), but that's a requirement. And ideally, a solution would be adaptable to a different strategy down the line (e.g. a single, but randomly positioned match). The one invariant to strategy is that each id/sub pair should used only once. So:
Replace one, and one only, occurrence of a :related value whose :id matches a value from ids with the corresponding value from substitutes, where the one occurrence is the first (or nth or rand-nth...) occurrence.
(def id-mapping (zipmap ids
(map :id substitutes)))
;; id-mapping -> {"345" "222", "234" "111"}
(clojure.walk/prewalk-replace id-mapping original)
Assuming that the collection is called results:
(require '[clojure.zip :as z])
(defn modify-related
[results id sub]
(loop [loc (z/down (z/seq-zip results))
done? false]
(if (= done? true)
(z/root loc)
(let [change? (->> loc z/node :_id (= id))]
(recur (z/next (cond change?
(z/edit loc (fn [_] identity sub))
:else loc))
change?)))))
(defn modify-results
[results id sub]
(loop [loc (z/down (z/seq-zip results))
done? false]
(if (= done? true)
(z/root loc)
(let [related (->> loc z/node :related)
change? (->> related (map :_id) set (#(contains? % id)))]
(recur (z/next (cond change?
(z/edit loc #(assoc % :related (modify-related related id sub)))
:else loc))
change?)))))
(defn sub-for-first
[results ids substitutes]
(let [subs (zipmap ids substitutes)]
(reduce-kv modify-results results subs)))
I'm generating json as literally as I can in clojure. My problem is that certain branches of the json are only present if given parameters are given. Here is a sample of such a condition
(defn message-for
[name uuid & [generated-uuids]]
{:message {:id (generate-uuid)
:details {:name name}
:metadata {:batch (merge {:id uuid}
(when generated-uuids (let [batches (map #(array-map :id %) generated-uuids)]
{:generatedBatches batches})))}}})
Unfortunately the when/let part is quite ugly. This same could be achieved using when-let as following but it doesn't work because my map returns [] instead of a nil.
(defn message-for
[name uuid & [generated-uuids]]
{:message {:id (generate-uuid)
:details {:name name}
:metadata {:batch (merge {:id uuid}
(when-let [batches (map #(array-map :id %) generated-uuids)]
{:generatedBatches batches}))}}})
Any ideas if I could somehow make when-let consider an empty list/array/seq as false so I could clean up my code a bit?
not-empty returns its argument if it is not empty.
When using when-let with a collection, always use not-empty
to retain the collection type
make refactoring easier
expressivenes
(when-let [batches (not-empty (map ...))]
...)
In your case I'd however prefer something like this:
...
:metadata {:batch (cond-> {:id uuid}
(seq generated-uuids)
(assoc :generatedBatches (map ...)))}
...
Notice that all three of the advantages listed above where met, without a nested let.
Also notice a new advantage
easier to extend with more conditions lateron
seq returns nil on an empty input sequence so you could do:
(when-let [batches (seq (map #(array-map :id %) generated-uuids))]
{:generatedBatches batches}))}}})
I'm trying to read metadata for a collection of functions in Clojure, but the var or reader special forms do not work unless they are directly dealing with the symbol.
; this works
(var my-fn)
; this doesn't
(defn val-it [x] (var x))
(val-it my-fn)
Is there any way to get this to work within the scope of another function?
resolve returns the Var or class object corresponding the given symbol in the context of the current namespace. ns-resolve allows you to specify which namespace to resolve the symbol in.
(resolve 'my-fn)
;= #'some.ns/my-fn
If the symbol cannot be resolved to a Var, nil is returned.
(var my-fn) does deal directly with the symbol because it is a special form (the reader receives the form unevaluated).
The metadata you want to read is stored in the var object, not in the function object.
Thus your goal, to read metadata from a list of function objects, is only achievable by traversing all existing vars and comparing their value by equality. I'd only recommend it if the function objects are the only way to start.
(defn meta-of
"Returns a hashmap mapping the function objects in fn-objs to
a set of metadata of vars containing it."
[fn-objs]
(let [fn-objs (set fn-objs)]
(reduce (fn [acc ns]
(reduce (fn [acc var]
(let [val (var-get var)]
(cond-> acc
(contains? fn-objs val)
(update-in [val] (fnil conj #{}) (meta var)))))
acc
(vals (ns-interns ns)))) {} (all-ns))))
(def foo inc) ;; Now there are two vars that have the inc function as their value
(meta-of [inc])
{#<core$inc clojure.core$inc#66d7e31d> ;; <- function object
#{{:ns #<Namespace clojure.core>, ;; <- metadata in clojure.core namespace
:name inc,
:file "clojure/core.clj",
:column 1,
:line 881,
:arglists ([x]),
:added "1.2",
:inline #<core$inc__inliner clojure.core$inc__inliner#24f87069>,
:doc
"Returns a number one greater than num. Does not auto-promote\n longs, will throw on overflow. See also: inc'"}
{:ns #<Namespace user>, ;; <- metadata in user namespace
:name foo,
:file "/tmp/form-init1078564431656334911.clj",
:column 1,
:line 1}}}