I'm a newbie to clojure and i'm trying to convert a messages that come in a particular format into another.
ie, i have to convert something like:
{
:image-url ["https://image.png"],
:topic "Some title",
:id "88ebaf91-a01d-4683-9aa7-629bb3ecea01",
:short-description "Some Description",
:mobile-deeplink "https://deeplink.com/link",
:partner-name "partner"}
Into something like
{
:title "Some title",
:id "88ebaf91-a01d-4683-9aa7-629bb3ecea01",
:content {
:url ["https://image.png"],
:description "Some Description",
:deeplink "https://deeplink.com/link",
:partner "partner"}}
So in effect, there is a combination of renaming keys and nesting the flat map
What I have done so far was something on the lines of:
(let [message-map {
:image-url :purl
:topic :title
:partner-name :partner
:short-description :description
:mobile-deeplink :deeplink}]
(defn- map-to-body
[message]
(-> message
(clojure.set/rename-keys message-map)
;;some sort of (assoc-in) <- this is where i need help in
)))
Combining assoc-in, a path conversion table, and reduce could be more self-describing and maintainable. You could choose to reduce over either the conversion table or the input message, whichever makes more sense for the data you have.
(defn transform [m]
(let [pp '([:image-url [:content :url]]
[:topic [:title]]
[:id [:id]]
[:short-description [:content :description]]
;; etc.
)]
(reduce
(fn [o [mk ok]]
(assoc-in o ok (get m mk)))
{}
pp)))
You could chain-assoc-in here, but I think you are easier off
using select-keys. select-keys lets you extract only the keys
from a map into a new map, you need. So you can select :id/:title for
the outer map and the rest to assoc to :content.
E.g.
(require 'clojure.set)
(defn transform
[message]
(let [message-map {:image-url :url
:topic :title
:partner-name :partner
:short-description :description
:mobile-deeplink :deeplink}
renamed (clojure.set/rename-keys message message-map)]
(assoc ; XXX
(select-keys renamed [:title :id])
:content (select-keys renamed [:url :description :deeplink :partner]))))
(def src {:image-url ["https://image.png"],
:topic "Some title",
:id "88ebaf91-a01d-4683-9aa7-629bb3ecea01",
:short-description "Some Description",
:mobile-deeplink "https://deeplink.com/link",
:partner-name "partner"})
(def tgt {:title "Some title",
:id "88ebaf91-a01d-4683-9aa7-629bb3ecea01",
:content {
:url ["https://image.png"],
:description "Some Description",
:deeplink "https://deeplink.com/link",
:partner "partner"}})
(assert (= (transform src) tgt))
Related
I'm using lacinia+pedestal to set up a graphql service, most of the queries I've seen in the tutorial need an arg (iD) e.g. games_by_id, but I'd like to retrieve all objects without an arg:
(defn resolve-all-drivers
[drivers-map context args value]
drivers-map)
schema-data.edn
:all_drivers
{:type (list Driver)
:description "Get all the drivers"
:resolve :query/all-drivers}
}
schema:
:Driver {:description "A collection of drivers"
:fields {:id {:type (non-null ID)}
:name {:type (non-null String)}
:email {:type (non-null String)}}}
In GraphiQL:
{
all_drivers {
name
}
}
Any idea how I can change this to give me a whole list without args?
Updated the resolver map:
(defn resolver-map
[component]
(let [trips-data (-> (io/resource "trips-data.edn")
slurp
edn/read-string)
trips-map (entity-map trips-data :trips)
cars-map (entity-map trips-data :cars)
drivers-map (get trips-data :drivers)]
{:query/trip-by-id (partial resolve-trip-by-id trips-map)
:query/drivers-by-id (partial resolve-drivers-by-id drivers-map)
:query/all-drivers (partial resolve-all-drivers drivers-map)
:Trip/cars (partial resolve-trip-cars cars-map)
:Car/trips (partial resolve-car-trips trips-map)}))
I have many JSON objects, and I am trying to filter those objects by the date. These objects are being parsed from several JSON files using Cheshire.core, meaning that the JSON objects are in a collection. The date is being passed in in the following format "YYYY-MM-DD" (eg. 2015-01-10). I have tried using the filter and contains? functions to do this, but I am having no luck so far. How can I filter these JSON objects by my chosen date?
Current Clojure code:
(def filter-by-date?
(fn [orders-data date-chosen]
(contains? (get (get orders-data :date) :date) date-chosen)))
(prn (filter (filter-by-date? orders-data "2017-12-25")))
Example JSON object:
{
"id":"05d8d404-b3f6-46d1-a0f9-dbdab7e0261f",
"date":{
"date":"2015-01-10T19:11:41.000Z"
},
"total":{
"GBP":57.45
}
}
JSON after parsing with Cheshire:
[({:id "05d8d404-b3f6-46d1-a0f9-dbdab7e0261f",
:date {:date "2015-01-10T19:11:41.000Z"},
:total {:GBP 57.45}}) ({:id "325bd04-b3f6-46d1-a0f9-dbdab7e0261f",
:date {:date "2015-02-23T10:15:14.000Z"},
:total {:GBP 32.90}})]
First, I'm going to assume you've parsed the JSON first into something like this:
(def parsed-JSON {:id "05d8d404-b3f6-46d1-a0f9-dbdab7e0261f",
:date {:date "2015-01-10T19:11:41.000Z"},
:total {:GBP 57.45}})
The main problem is the fact that the date as stored in the JSON contains time information, so you aren't going to be able to check it directly using equality.
You can get around this by using clojure.string/starts-with? to check for prefixes. I'm using s/ here as an alias for clojure.string:
(defn filter-by-date [date jsons]
(filter #(s/starts-with? (get-in % [:date :date]) date)
jsons))
You were close, but I made a few changes:
You can't use contains? like that. From the docs of contains?: Returns true if key is present in the given collection, otherwise returns false. It can't be used to check for substrings; it's used to test for the presence of a key in a collection.
Use -in postfix versions to access nested structures instead of using multiple calls. I'm using (get-in ...) here instead of (get (get ...)).
You're using (def ... (fn [])) which makes things more complicated than they need to be. This is essentially what defn does, although defn also adds some more stuff as well.
To address the new information, you can just flatten the nested sequences containing the JSONs first:
(->> nested-json-colls ; The data at the bottom of the question
(flatten)
(filter-by-date "2015-01-10"))
#!/usr/bin/env boot
(defn deps [new-deps]
(merge-env! :dependencies new-deps))
(deps '[[org.clojure/clojure "1.9.0"]
[cheshire "5.8.0"]])
(require '[cheshire.core :as json]
'[clojure.string :as str])
(def orders-data-str
"[{
\"id\":\"987654\",
\"date\":{
\"date\":\"2015-01-10T19:11:41.000Z\"
},
\"total\":{
\"GBP\":57.45
}
},
{
\"id\":\"123456\",
\"date\":{
\"date\":\"2016-01-10T19:11:41.000Z\"
},
\"total\":{
\"GBP\":23.15
}
}]")
(def orders (json/parse-string orders-data-str true))
(def ret (filter #(clojure.string/includes? (get-in % [:date :date]) "2015-01-") orders))
(println ret) ; ({:id 987654, :date {:date 2015-01-10T19:11:41.000Z}, :total {:GBP 57.45}})
You can convert the date string to Date object using any DateTime library like joda-time and then do a proper filter if required.
clj-time has functions for parsing strings and comparing date-time objects. So you could do something like:
(ns filter-by-time-example
(:require [clj-time.coerce :as tc]
[clj-time.core :as t]))
(def objs [{"id" nil
"date" {"date" "2015-01-12T19:11:41.000Z"}
"total" nil}
{"id" "05d8d404-b3f6-46d1-a0f9-dbdab7e0261f"
"date" {"date" "2015-01-10T19:11:41.000Z"}
"total" {"GBP" :57.45}}
{"id" nil
"date" {"date" "2015-01-11T19:11:41.000Z"}
"total" nil}])
(defn filter-by-day
[objs y m d]
(let [start (t/date-time y m d)
end (t/plus start (t/days 1))]
(filter #(->> (get-in % ["date" "date"])
tc/from-string
(t/within? start end)) objs)))
(clojure.pprint/pprint (filter-by-day objs 2015 1 10)) ;; Returns second obj
If you're going to repeatedly do this (e.g. for multiple days) you could parse all dates in your collection into date-time objects with
(map #(update-in % ["date" "date"] tc/from-string) objs)
and then just work with that collection to avoid repeating the parsing step.
(ns filter-by-time-example
(:require [clj-time.format :as f]
[clj-time.core :as t]
[cheshire.core :as cheshire]))
(->> json-coll
(map (fn [json] (cheshire/parse-string json true)))
(map (fn [record] (assoc record :dt-date (f/format (get-in record [:date :date])))))
(filter (fn [record] (t/after? (tf/format "2017-12-25") (:dt-date record))))
(map (fn [record] (dissoc record :dt-date))))
Maybe something like this? You might need to change the filter for your usecase but as :dt-time is now a jodo.DateTime you can leverage all the clj-time predicates.
I'm using Datomic, although it doesn't particularly matter for this question. But it typically returns namespaced keys (and enum values are returned as namespaces keywords also). I want to translate the potentially nested structure to strip the namespaces from the keys and from values (and also string-ify the values of enums). I'm doing this because I'll return the result in a JSON REST API and the namespacing doesn't make much sense in that context. Here's a simple example structure:
{
:person/name "Kevin"
:person/age 99
:person/gender :gender/M
:person/address {
:address/state :state/NY
:address/city "New York"
:address/zip "99999"
}
}
And I'm hoping to translate to:
{
:name "Kevin"
:age 99
:gender "M"
:address {
:state "NY"
:city "New York"
:zip "99999"
}
}
One thing I know I can do is use (postwalk-replace {:person/name :name :person/age :age :person/gender :gender :person/address :address :address/city :city :address/state :state :address/zip :zip} the-entity) and that covers the keys, but not the values.
What other options do I have?
You can use clojure.walk/postwalk. Simple version doesn't differentiate between keywords as keys or values in a map, simply converts all keys to strings:
(def data {:person/name "Kevin"
:person/age 99
:person/gender :gender/M
:person/address {:address/state :state/NY
:address/city "New York"
:address/zip "99999"}})
(clojure.walk/postwalk
(fn [x]
(if (keyword? x)
(name x)
x))
data)
;; => => {"name" "Kevin", "age" 99, "gender" "M", "address" {"state" "NY", "city" "New York", "zip" "99999"}}
To implement exactly what you want you need to handle keys and values in a map separately:
(defn transform-keywords [m]
(into {}
(map (fn [[k v]]
(let [k (if (keyword? k) (keyword (name k)) k)
v (if (keyword? v) (name v) v)]
[k v]))
m)))
(clojure.walk/postwalk
(fn [x]
(if (map? x)
(transform-keywords x)
x))
data)
;; => => {:name "Kevin", :age 99, :gender "M", :address {:state "NY", :city "New York", :zip "99999"}}
As a side note: in my experience, the impedance mismatch between namespace-qualified and non-namespace-qualified keys at the boundary of your system can be an ongoing pain; what's more, having namespaced-qualified keys has significant advantages regarding code clarity (very good data traceability).
So I wouldn't give up namespace-qualified keys too readily. If EDN's syntax for namespacing (with dots and slashes) doesn't suit the consumers of your API, you may even want to use something more conventional like underscores (e.g :person_name instead of :person/name); a bit uglier, but it still gives you most of the benefits of namespace-qualified keys, you will not even need to transform the data structures, and Datomic won't mind.
I have a reagent atom which consists a vector of key/value maps.
How can I remove a key/value pair with certain key value which happens to be UUID? I have tried:
(swap! state/items (dissoc #state/items id))
but I get:
core.cljs:270 Uncaught Error: No protocol method IMap.-dissoc defined for type cljs.core/LazySeq: ({:id #uuid "e1f9341f-bc02-4c17-a594-b9b5ede72214", :description "foo bar"})
I think you need to use remove. So this should help:
(def data [{:id "e1f9341f-bc02-4c17-a594-b9b5ede72214" :description "Foo"} {:id "e1f9341f-bc02-4c17-a594-b9b5ede72214" :description "Bar"}] )
(remove #(= (:id %) "e1f9341f-bc02-4c17-a594-b9b5ede72214") data)
;; => ()
The swap! function takes the old value and returns the updated value. So data from above will be the old value. Your function you pass to swap! will thus look like this:
(fn [old]
(remove #(= (:id %) "e1f9341f-bc02-4c17-a594-b9b5ede72214") old))
I have data in clojure which I have grouped by application into following format:
(["name1" [{:application "name1", :date "date1", :description "desc1"}
{:application "name1", :date "date2", :description "desc2"}]]
["name2" [{:application "name2", :date "date1", :description "desc1"}
{:application "name2", :date "date2", :description "desc2"}]]
... and so on)
I need to count the number of events for each application (i.e. the number of maps in each vector) and produce a list of maps in the format:
({:application "name1", :count 2} {:application "name2", :count2} ... etc )
I have the following code to produce a list of application names and a list of the counts for each application name but I am struggling with how to get them back into the format above.
(let[
application-list (map first group-by-app)
grouped-data (map second group-by-app)
count-list (map count grouped-data)]
Any help would be greatly appreciated.
Thanks in advance.
David
for offers a really succinct, but expressive way to do something with the elements of sequences to build up a new sequence. I'd certainly recommend it here.
(def group-by-app '(["name1" [{:application "name1", :date "date1", :description "desc1"}
{:application "name1", :date "date2", :description "desc2"}]]
["name2" [{:application "name2", :date "date1", :description "desc1"}
{:application "name2", :date "date2", :description "desc2"}]]))
(for [[application events] group-by-app]
{:application application, :count (count events)})
; => ({:application "name1", :count 2} {:application "name2", :count 2})
For completeness, however, it's worth noting that map can map a function over multiple sequences at the same time. So, you could recombine the intermediate data you produced with a function of the application and the count.
(let [application-list (map first group-by-app)
grouped-data (map second group-by-app)
count-list (map count grouped-data)]
(map
(fn [app event-count]
{:application app :count event-count})
application-list count-list))
group-by produces a map, which is a sequence of key value pairs. You can simply map that sequence.
user=> (def foo #{{:application "name1", :date "date1", :description "desc1"}
#_=> {:application "name1", :date "date2", :description "desc2"}
#_=> {:application "name2", :date "date1", :description "desc1"}
#_=> {:application "name2", :date "date2", :description "desc2"}})
#'user/foo
user=> (map #(let [[key tuples] %] [key (count tuples)]) (group-by :application foo))
(["name1" 2] ["name2" 2])
You can turn that back into a map if that's what you need.
user=> (into {} (map #(let [[key tuples] %] [key (count tuples)]) (group-by :application foo)))
{"name1" 2, "name2" 2}
(reduce
(fn [acc [_ maps]]
(let [c (count maps)]
(into acc
(map #(assoc % :count c) maps))))
'()
grouped)
grouped is the grouped data you provided.