I have a vector of entities (maps), I have filtered this vector to those entities with a particular key. Then I apply a function to update some values in those entities, and now have a subset that I'd like to merge with the original superset.
world [{a} {b} {c} {d} {e}]
a [{b} {d} {e}] ; (filter matches? world)
b [{b'} {d'} {e'}] ; (map func a)
new-world [{a} {b'} {c} {d'} {e'}] ; (merge world b)
How can I merge my update b with the original world?
My map structure is:
{:id identity
:attrs {:property1 {:param1 value
:param2 value}
:property2 {}}
There can be a variable number of properties. Properties can have 0 ({}) or more params. The only part of the map that is changing are the values, and only some of them.
I have to peer into each map to identify it, and then merge accordingly? What is an idiomatic way to do this?
Clojure's map function can take multiple sequences, but won't compare my subset b to the whole superset world, only b number of world entities, and stop once b is exhausted.
> (map #(str %1 " " %2) [1 2 3 4 5] [6 7 8])
("1 6" "2 7" "3 8")
Have I chosen the wrong structure for my data in the first place? Would I be better off without a vector and just one map?
{:entid1
{:property1 {:param1 value :param2 value}
:property2 {}}
:entid2
{:property1 {:param1 value :param2 value}
:property2 {:param1 value}
:property3 {:param1 value :param2 value :param3 value}}}
This question looks similar but does not merge my vectors correctly.
Real world implementation
My actual code is part of a game I'm writing to familiarise myself with Clojure (ClojureScript in this instance).
The game state (world) is as follows:
[{:id :player,
:attrs
{:gravity {:weight 10},
:jump {:height 60, :falling false, :ground true},
:renderable {:width 37, :height 35},
:position {:x 60, :y 565},
:walk {:step 4, :facing :right},
:input {},
:solid {:blocked false}}}
{:id wall1,
:attrs
{:solid {},
:position {:x 0, :y 0},
:renderable {:width 20, :height 600}}}
{:id wall2,
:attrs
{:solid {},
:position {:x 780, :y 0},
:renderable {:width 20, :height 600}}}
{:id platform3,
:attrs
{:solid {},
:position {:x 20, :y 430},
:renderable {:width 600, :height 20}}}]
I update the world on each animation frame, then re-render the canvas.
(defn game-loop []
(ui/request-frame game-loop)
(-> world
system/update!
ui/render))
system/update! is:
(defn update! [world]
(let [new-world (->> #world
(phys/move #player/input-cmd))]
(player/clear-cmd)
(reset! world new-world)
#world))
My plan was to have a chain of systems updating the world in the thread last macro. I am in the process of writing phys/move.
This means that systems have to update the world, instead of just returning their effect on the world (vector of changes).
I'm contemplating if it's more manageable to have the system/update! function manage the effects on the world (applying the updates). So systems only return their list of changes. Something like:
(defn update! [world]
(let [updates []
game #world]
(conj updates (phys/move #player/input-cmd game))
(conj updates (camera/track :player game))
; etc.
(reset! world
(map #(update-world updates %) world))
#world))
Repeated (conj updates (func world)) feels clunky though. This is possibly more work than just merging updates (or returning an modified entity) in a system function.
How to elegantly pass state changes between my system functions (phys/move, camera/track), and subsystem functions (walk, jump)?
My stab at applying Joaquin's map only approach to my move system:
(ns game.phys)
(def step 4)
(def height 50)
(defn walk?
"is this a walk command?"
[cmd]
(#{:left :right} cmd))
(defn walks?
"update? equivalent"
[entity]
(contains? (:attrs entity) :position))
(defn walk
[cmd entity]
(if (walks? entity)
(let [x (get-in entity [:attrs :position :x]
op (if (= cmd :left) - +)
update {:attrs {:position {:x (op x step)}
:walk {:facing cmd}}}]
(merge entity update))
entity))
; cmd is a deref'ed atom that holds player input commands
; e.g. :left :right: :jump
(defn move
[cmd world]
(cond
(walk? cmd) (map #(walk input-cmd %) world)
(jump? cmd) (map #(jump height %) world)
:else world))
It is simpler than all that (having a vector of maps is something pretty common, and it may fit your problem properly). Instead of doing a filter and then a map, just do a map:
(defn update? [entity] ...)
(defn update-entity [entity] ...)
(defn update-world [world]
(let [update #(if (matches? %) (update-entity %) %)]
(map update world)))
This pattern of updating something or just leaving it as it is is pretty common, and it is idiomatic to make the functions that update something return the updated thing, or the old thing if it did nothing, so at the end it would be something like:
(defn update-entity?
"Predicate that returns if an entity needs updating"
[entity] ...)
(defn update-entity
"Updates an entity if it is needed.
Otherwise returns the unchanged entity"
[entity]
(if (update-entity? entity)
(assoc entity :something :changed)
entity))
(defn update-world [world]
(map update-entity world))
You can see some examples of this behavior in the game logic of this snake game:
https://github.com/joakin/cnake/blob/master/src/cnake/game.cljs#L54-L66
https://github.com/joakin/cnake/blob/master/src/cnake/game.cljs#L102-L114
Related
Getting data from the database as a list of maps (LazySeq) leaves me in need of transforming it into a map of maps.
I tried to 'assoc' and 'merge', but that didn't bring the desired result because of the nesting.
This is the form of my data:
(def data (list {:structure 1 :cat "A" :item "item1" :val 0.1}
{:structure 1 :cat "A" :item "item2" :val 0.2}
{:structure 1 :cat "B" :item "item3" :val 0.4}
{:structure 2 :cat "A" :item "item1" :val 0.3}
{:structure 2 :cat "B" :item "item3" :val 0.5}))
I would like to get it in the form
=> {1 {"A" {"item1" 0.1}
"item2" 0.2}}
{"B" {"item3" 0.4}}
2 {"A" {"item1" 0.3}}
{"B" {"item3" 0.5}}}
I tried
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply merge-with into))
This gives
{1 {"A" {"item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
By merging I lose some entries, but I can't think of any other way. Is there a simple way? I was even about to try to use specter.
Any thoughts would be appreciated.
If I'm dealing with nested maps, first stop is usually to think about update-in or assoc-in - these take a sequence of the nested keys. For a problem like this where the data is very regular, it's straightforward.
(assoc-in {} [1 "A" "item1"] 0.1)
;; =>
{1 {"A" {"item1" 0.1}}}
To consume a sequence into something else, reduce is the idiomatic choice. The reducing function is right on the edge of the complexity level I'd consider an anonymous fn for, so I'll pull it out instead for clarity.
(defn- add-val [acc line]
(assoc-in acc [(:structure line) (:cat line) (:item line)] (:val line)))
(reduce add-val {} data)
;; =>
{1 {"A" {"item1" 0.1, "item2" 0.2}, "B" {"item3" 0.4}},
2 {"A" {"item1" 0.3}, "B" {"item3" 0.5}}}
Which I think was the effect you were looking for.
Roads less travelled:
As your sequence is coming from a database, I wouldn't worry about using a transient collection to speed the aggregation up. Also, now I think about it, dealing with nested transient maps is a pain anyway.
update-in would be handy if you wanted to add up any values with the same key, for example, but the implication of your question is that structure/cat/item tuples are unique and so you just need the grouping.
juxt could be used to generate the key structure - i.e.
((juxt :structure :cat :item) (first data))
[1 "A" "item1"]
but it's not clear to me that there's any way to use this to make the add-val fn more readable.
You may continue to use your existing code. Only the final merge has to change:
(defn deep-merge [& xs]
(if (every? map? xs)
(apply merge-with deep-merge xs)
(apply merge xs)))
(->> data
(map #(assoc {} (:structure %) {(:cat %) {(:item %) (:val %)}}))
(apply deep-merge))
;; =>
{1
{"A" {"item1" 0.1, "item2" 0.2},
"B" {"item3" 0.4}},
2
{"A" {"item1" 0.3},
"B" {"item3" 0.5}}}
Explanation: your original (apply merge-with into) only merge one level down. deep-merge from above will recurse into all nested maps to do the merge.
Addendum: #pete23 - one use of juxt I can think of is to make the function reusable. For example, we can extract arbitrary fields with juxt, then convert them to nested maps (with yet another function ->nested) and finally do a deep-merge:
(->> data
(map (juxt :structure :cat :item :val))
(map ->nested)
(apply deep-merge))
where ->nested can be implemented like:
(defn ->nested [[k & [v & r :as t]]]
{k (if (seq r) (->nested t) v)})
(->nested [1 "A" "item1" 0.1])
;; => {1 {"A" {"item1" 0.1}}}
One sample application (sum val by category):
(let [ks [:cat :val]]
(->> data
(map (apply juxt ks))
(map ->nested)
(apply (partial deep-merge-with +))))
;; => {"A" 0.6000000000000001, "B" 0.9}
Note deep-merge-with is left as an exercise for our readers :)
(defn map-values [f m]
(into {} (map (fn [[k v]] [k (f v)])) m))
(defn- transform-structures [ss]
(map-values (fn [cs]
(into {} (map (juxt :item :val) cs))) (group-by :cat ss)))
(defn transform [data]
(map-values transform-structures (group-by :structure data)))
then
(transform data)
Say I have a board like this
(def board {:a 10 :b 12})
And a list of functions like this
(def funcs [(assoc board :a 2)
(assoc board :b 4)
(assoc board :a (inc (get board :a)))])
How would I go about applying each operation in the list to my board, having it update each time in a functional fashion.
Evaluating funcs in my repl, gives a list of return values after calling each function, but the board itself remains unchanged each time
user=> funcs
[{:a 2, :b 12} {:a 10, :b 4} {:a 11, :b 12}
Ideally I'd like the value of the board to update each time a function is ran
so after running the all commands the final state of the board should be:
{:a 3 :b 4}
I'm aware this is possible using tail recursion but I'd like to avoid doing so as I'm almost certain a combination of reduce/apply/map will do the trick
One of the defining traits of clojure is that its data structures are immutable. That means that board will never change, functions that operate on data structures return modified copies of the data structure. So what you've done in funcs is make a vector of three different boards, the original board with a fuction applied.
Usually what you want is to apply a bunch of functions to some initial value, where each function takes the returned value of the function before, then use the returned value for something. Usually passing it around in function parameters.
;; First of all, there's a function for that
(assoc board :a (inc (get board :a)))
;; The function update takes a map and a key and a function
;; It applies the function to value currently at key,
;; then sets key in the retuned "copy" of the map to be the return value of the function.
;; Equivalent to the above
(update board :a inc)
If you want an updated board with these functions applied, you need to to pass the return value through the functions, because the original board never changes, they all just return updated copies of their input.
(def updated-board
;; The innermost forms are evaluated first.
(update (assoc (assoc board :a 2) :b 4) :a inc))
This can be made more readable by using the -> or "thread first" macro. It takes an initial value and forms which are missing the first argument, then rewrites the code to "thread" the return value of each one as the first argument to the next.
(def updated-board-threaded
(-> board
(assoc :a 2)
(assoc :b 4)
(update :a inc)))
;; You can expand the macro to see for yourself
;; that the two examples are exactly equivalent.
(macroexpand-1 '(-> board
(assoc :a 2)
(assoc :b 4)
(update :a inc)))
;;=> (update (assoc (assoc board :a 2) :b 4) :a inc)
This is the way to "think in clojure", functions usually just take immutable values and return other immutable values.
But sometimes you need something mutable, and clojure delivers this in the form of atoms. An atom can be thought of as a mutable box that contains an immutable value.
It uses the functions swap! and reset! to apply controlled mutation. And the function deref to get the current value.
(def board (atom {:a 10, :b 12}))
;; I'll define a function that takes a board and returns an updated version of it.
(defn do-stuff-with-board [b]
(-> b
(assoc :a 2)
(assoc :b 4)
(update :a inc)))
;; Get the current value of board.
(deref board) ;;=> {:a 10, :b 12}
;; Swap takes an atom and a function and
;; sets the value of the atom to be the return value of the function
(swap! board do-stuff-with-board)
;; Now the mutable board atom contains a new immutable value.
(deref board) ;;=> {:a 3, :b 4}
;; derefing an atom is a very usual operation, so there's syntax sugar for it
;; Equivalent to (deref board)
#board ;;=> {:a 3, :b 4}
reset! sets the value of board to be another value, like = in "normal" languages. It's not usually idiomatic to do this as it kinda says to the reader that the new value of the atom has nothing to do with the old one, but clojure is pragmatic, and sometimes it's what you need.
(reset! board "And now for something completely different.")
;; The value is now a string.
#board ;;=> "And now for something completely different."
As an aside. The data structures are not actually deep copies of each other, there is magic behind the scenes to make it almost as efficient as updating the data structure in place, but from the perspective of the programmer they are equivalent to deep copies in other languages.
I'd like to suggest a different approach to #madstap's fine answer.
In ...
(def funcs [(assoc board :a 2)
(assoc board :b 4)
(assoc board :a (inc (get board :a)))])
... the elements such as (assoc board :b 4) are not functions: they are function calls, which, as #madstap points out, fail to modify whatever board refers to.
We can, without too much trouble, turn them into proper functions:
(def funcs [(fn [board] (assoc board :a 2))
(fn [board] (assoc board :b 4))
(fn [board] (assoc board :a (inc (get board :a))))])
The boards here are locals. Any distinct identifier would do as well:
(def funcs [(fn [b] (assoc b :a 2))
(fn [b] (assoc b :b 4))
(fn [b] (assoc b :a (inc (get b :a))))])
We can write a function to compose them:
(defn compose [fs]
(fn [x] (reduce (fn [a f] (f a)) x fs)))
This is a simplified version of the standard comp. It applies the functions first to last instead of last to first.
Now, for example, if
(def board {:a 10 :b 12})
... then
((compose funcs) board)
;{:a 3, :b 4}
Furthermore, we can modify compose to show the chain of results:
(defn compositions [fs]
(fn [x] (reductions (fn [a f] (f a)) x fs)))
((compositions funcs) board)
;({:a 10, :b 12} {:a 2, :b 12} {:a 2, :b 4} {:a 3, :b 4})
Notice that compose and compositions are completely generic - they just do stuff with functions.
Given a collection"
[{:key "key_1" :value "value_1"}, {:key "key_2" :value "value_2"}]
I would like to convert this to:
{"key_1" "value_1" "key_2" "value_2"}
An function to do this would be:
(defn long->wide [xs]
(apply hash-map (flatten (map vals xs))))
I might simplify this using the threading macro:
(defn long->wide [xs]
(->> xs
(map vals)
(flatten)
(apply hash-map)))
This still requires explicit definition of the function argument which I am not doing anything with other than passing to the first function. I might then rewrite this using comp to remove this:
(def long->wide
(comp (partial apply hash-map) flatten (partial map vals)))
This however requires repeated use of partial which to me is a lot of noise in the function.
Is there a some function in clojure that combines comp and ->> so I can create a higher order function without repeated use of partial, and also which out having to create a new function?
Since many of the answers here already don't answer the original question, but
suggest different approaches, I put that one back up too.
I'd go with reduce and destructuring:
(reduce
(fn [m {:keys [key value]}]
(assoc m key value))
{}
[{:key "key_1" :value "value_1"}, {:key "key_2" :value "value_2"}])
Note, that this will also work with string keys (which you mentioned in the comments) (note :strs):
(reduce
(fn [m {:strs [key value]}]
(assoc m key value))
{}
[{"key" "key_1" "value" "value_1"}, {"key" "key_2" "value" "value_2"}])
Another (point-free) version, when using keywords:
(partial (into {} (map (juxt :key :value))))
Since you mentioned in the comments, that you are using values from a DB, there might also be the chance, that you can switch to just return value tuples. Then the whole operation is just:
(into {} [["key_1" "value_1"]["key_2" "value_2"]])
Also note, that the use of vals on a map and expecting "insertion order" is
dangerous. Small maps are ordered only by accident:
user=> (take 3 (zipmap (range 3) (range 3)))
([0 0] [1 1] [2 2])
user=> (take 3 (zipmap (range 100) (range 100)))
([0 0] [65 65] [70 70])
An other alternative to the nice answers is also:
(apply hash-map (mapcat vals [{:key "key_1" :value "value_1"}, {:key "key_2" :value "value_2"}]))
or:
((comp #(apply hash-map %) #(mapcat vals %)) [{:key "key_1" :value "value_1"}, {:key "key_2" :value "value_2"}])
which are exactly the same.
As with clojure, so many ways to solve most problems.
(partial #(reduce (fn [r m] (assoc r (m :key) (m :value)))
{}
%)))
Not sure if the creation of anonymous functions violates your condition or not but this isn't adding functions to the namespace so I thought I'd throw it out there. This also has the benefit of not requiring the keys in the input maps to be keywords as :key and :value can be replaced with values of any type since the map is in the function position. For example:
(partial #(reduce (fn [r m] (assoc r (m "key") (m "value")))
{}
%)))
I am building a minesweeper where the gameboard is a 2d array.
To generate the board I need to get a 2d array, randomly place some bombs, calculate all neighboring bombs for each field, then pack all together with another value (is-revealed) into a Map that should be the final value of each field.
It seems that to do all of this, I need the help of functions like map, filter, nth or zip, but instead of working on lists, they should work on a 2d array. I already started building those, but it seems like the wrong path to go.
Is there some cool abstraction that helps me use the existing functions on the 2d array? Or are there any functions already existing, that deal with 2d arrays?
edit: The way I was doing it - before thinking to myself, that there should be a better way - is something like this:
(defn mapmap [f xxs]
(for [xs xxs]
(map f xs)))
(defn mapmap-coords [f xxs]
(for [[i xs] (map-indexed list xxs)]
(map-indexed (fn [j x] (f j i)) xs)))
(defn get-value [board x y]
(if (or (< x 0) (< y 0) (< (dec (count board)) y))
false
(let [xs (nth board y)]
(if (< (dec (count xs)) x)
false
(nth xs x)))))
for works for nested sequential data.
user> (def game-state
(let [game '[[_ _ _ _]
[1 1 _ _]
[! 1 _ _]
[1 1 _ _]]]
(for [[i row] (map-indexed list game)
[j cell] (map-indexed list row)
:when (not= '_ cell)]
{:x i :y j :value cell})))
#'user/game-state
user> (pprint game-state)
({:x 1, :y 0, :value 1}
{:x 1, :y 1, :value 1}
{:x 2, :y 0, :value !}
{:x 2, :y 1, :value 1}
{:x 3, :y 0, :value 1}
{:x 3, :y 1, :value 1})
nil
We can use reduce to build up the data structure again (of course any transformations to the state could have been done in between).
user> (let [empty-game (vec (repeat 4 '[_ _ _ _]))
fill-cell (fn [game, {:keys [x y value]}] (assoc-in game [x y] value))
game (reduce fill-cell empty-game game-state)]
(doseq [row game] (apply println row)))
_ _ _ _
1 1 _ _
! 1 _ _
1 1 _ _
nil
If you're willing to drop down into Java types for your implementation, consider using
clojure.core/to-array-2d to create your 2d array and then
clojure.core/aget to index into it.
aget will give you a more natural method for accessing the array, but you'll still need to build further abstractions to get neighbours and perform updates to (i.e., regenerate) this data structure.
I'm having trouble finding solution to the following problem:
Lets say I have a map:
(def defaults {
:name "John"
:surname "Doe"
:info {:date-of-birth "01-01-1980"
:registered [{:type "newsletter" :name "breaking news" }]}
})
And then I pass a similar structured map but I want to conjoin the vectors and overwrite the rest of the keys:
(def new {
:name "Peter"
:info {:date-of-birth "11-01-1986"
:registered [{:type "alert" :name "mobile-alert" }]}
})
And I want this result:
{:name "Peter"
:surname "Doe"
:info {:date-of-birth "11-01-1986"
:registered [{:type "newsletter" :name "breaking news" }
{:type "alert" :name "mobile-alert" }]}}
Now I can do this easily by using static syntax like:
(reduce conj (get-in defaults [:info :registered]) (get-in new [:info :registered]))
(There is probably a better way...) But I was hoping more of a dynamic function with the following properties:
Keep all keys from both maps, without knowing the structure
Update any keys with the values from the right map
if the val of a key is a vector, then conj the vector with the vector of the right map (if the appropriate key exists of course)
Thanks for the help in advance :)
You should definitely look at merge-with function. This is possible implementation:
(defn deep-merge [a b]
(merge-with (fn [x y]
(cond (map? y) (deep-merge x y)
(vector? y) (concat x y)
:else y))
a b))
Here's a possible implementation for this kind of functionality. It is at least a starting point, you may need some extra validations depending on the possible structure of your data (e.g. what if the overriding map's value is a vector but the value in the default map is not even a collection?).
(declare merge-maps)
(defn merge-map [x [k v]]
(cond (vector? v)
(assoc x k (vec (reduce conj (x k) v)))
(map? v)
(assoc x k (merge-maps (x k) v))
:esle
(assoc x k v)))
(defn merge-maps [x y]
(reduce merge-map x y))
(merge-maps defaults new)
;= {:info {:date-of-birth "11-01-1986",
;= :registered [{:name "breaking news", :type "newsletter"}
;= {:name "mobile-alert", :type "alert"}]},
;= :name "Peter",
;= :surname "Doe"}