I'm working on a Compojure application that serves json files. I'm using clojure.java.jdbc library to retrieve the files from sqlite database:
(defn grid-result [z x y]
(j/with-connection db-specs
(j/with-query-results results ["SELECT grid FROM grids WHERE zoom_level = ? AND tile_column = ? AND tile_row = ?" z x y]
(doall results))))
The result of the query is this:
({:grid #<byte[] [B#7e88dd33>})
In order to display it as json, I update the headers, and strip the query result to byte array and use ByteArrayInputStream:
(defn grid-response [grid]
{:status 200
:headers {"Content-Type" "application/json"}
:body (new java.io.ByteArrayInputStream (:grid (first grid)))})
This is how the request is handled:
(defroutes app-routes
(GET "/api/:z/:x/:y.grid.json" [z x y] (grid-response (grid-result z x y))
(route/not-found "Page not found"))
All above results in:
"xœ«VJ/ÊLQ²ŠVR (éŒ1jĨ£FŒ1jĨ£FŒ1jĨÄEÊPP¨¨ªQl„
Å®€ƒQ#F5b¤«£”ZYÈ�gh`jndaXƒf†æ0†Œa¨[�7ð+k"
Json that I'm trying to retrieve has a format of:
grid({"keys": ["""], "data": {"105803": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UFGH0011"}, "106178": {"predicti_9": 0.0265, "prediction": "6B1", "OA01CDOLD": "15UHFE0001"}, "106171": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UHFC0001"}, "105721": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UFGC0013"}, "106170": {"predicti_9": 0.0257, "prediction": "3B2", "OA01CDOLD": "15UHFB0001"}}, "grid": [" ", " "]});
What I've also tried:
Changed grid-response function to convert byte array to String:
:body (String. (:grid (first grid)) "UTF-8")
Results in ((with UTF-8 or without):
x??VJ/?LQ??VR?(?1j??F?1j??F?1j??E?PP????Ql?
???Q#F?5b?????ZY?�gh`jndaX?f??0??a?[�7?
Mapped each char to str:
(apply str (map #(char (bit-and % 255)) (:grid (first grid))))
Results in:
xœ«VJ/ÊLQ²ŠVR (éŒ1jĨ£FŒ1jĨ£FŒ1jĨÄEÊPP¨¨ªQl„
Å®€ƒQ#F5b¤«£”ZYÈ�gh`jndaXƒf†æ0†Œa¨[�7ð+k
(Same as with ava.io.ByteArrayInputStream).
Any advice on how to convert that byte stream to json will be greatly appreciated.
The grids are compressed with zlib. So this is the solution I used:
(defn zlib-decompress
[input]
(with-open [input (-> input io/input-stream InflaterInputStream.)]
(slurp input)))
This function returns a string representation of the json, so when I set the response's content type to "application/json", everything works.
You're trying to read GZIP compressed data. You can try using something like:
(java.io.BufferedInputStream.
(java.util.zip.GZIPInputStream.
(new java.io.ByteArrayInputStream (:grid (first grid)))))
to decompress it before displaying it.
Related
I am new to functional programming and have a use case where I have a list of Books, I want to iterate over it, do some mapping and return a new List.
My code
(defn map-book [books]
((doseq [x books]
(let [lst (create-response x)]
(println "data -> " (json/encode lst))
(json/encode lst)))))
(defn create-response [book]
(let [updated-book (merge {"book-name" (.getBookName book)
"book-page" (.getBookPageCount book)}]
updated-book))
when I try to run this, I am able to get json decoded response in terminal due to println but not the json list as a response from the function.
have been stucked around for some time on this.
What I want is something like :
[{
"book-name": "rick boy",
"book-page": 110
},
{
"book-name": "poor boy",
"book-page": 124
}
]
but am getting something like, when I run my unit test:
#object[book_service.book$map_book 0x7915bca3 [book_service.book$map_book#7915bca3]
thanks for any help!
If you want to return new list (or vector), you should avoid doseq (which is for side effects- for example, printing, and always returns nil) and use map or for instead.
I guess you want to return JSON string for given data and your json library is actually Cheshire:
Dependencies: [cheshire "5.11.0"]
Require in ns: [cheshire.core :as json]
Book class:
public class Book {
String bookName;
Long pageCount;
public Book(String name, Long pages) {
bookName = name;
pageCount = pages;
}
public String getBookName() {
return bookName;
}
public Long getBookPageCount() {
return pageCount;
}
}
Clojure code:
(defn get-book-map [book]
{"book-name" (.getBookName book)
"book-page" (.getBookPageCount book)})
(defn encode-books [books]
(json/encode (map get-book-map books)))
Test:
(encode-books [(Book. "rick boy" 110)
(Book. "poor boy" 124)])
=> "[{\"book-name\":\"rick boy\",\"book-page\":110},{\"book-name\":\"poor boy\",\"book-page\":124}]"
(json/decode *1)
=> ({"book-name" "rick boy", "book-page" 110} {"book-name" "poor boy", "book-page" 124})
Instaparse can pprint nice error messages to the REPL
=> (negative-lookahead-example "abaaaab")
Parse error at line 1, column 1:
abaaaab
^
Expected:
NOT "ab"
but I can not find a built-in function to get the message as a String. How to do that?
You could always wrap it using with-out-str:
(with-out-str
(negative-lookahead-example "abaaaab"))
You may also be interested in using with-err-str documented here.
(with-err-str
(negative-lookahead-example "abaaaab"))
I can't remember if instaparse writes to stdout or stderr, but one of those will do what you want.
Let's look at the return type of parse in the failure case:
(p/parse (p/parser "S = 'x'") "y")
=> Parse error at line 1, column 1:
y
^
Expected:
"x" (followed by end-of-string)
(class *1)
=> instaparse.gll.Failure
This pretty printing behavior is defined like this in Instaparse:
(defrecord Failure [index reason])
(defmethod clojure.core/print-method Failure [x writer]
(binding [*out* writer]
(fail/pprint-failure x)))
In the REPL this prints as a helpful human-readable description, but it can also be treated as a map:
(keys (p/parse (p/parser "S = 'x'") "y"))
=> (:index :reason :line :column :text)
(:reason (p/parse (p/parser "S = 'x'") "y"))
=> [{:tag :string, :expecting "x", :full true}]
And you could do this:
(with-out-str
(instaparse.failure/pprint-failure
(p/parse (p/parser "S = 'x'") "y")))
=> "Parse error at line 1, column 1:\ny\n^\nExpected:\n\"x\" (followed by end-of-string)\n"
Or write your own version of pprint-failure that builds a string instead of printing it.
I need to collect and transfer :c/name values into nested vector to first level same way.
Input example:
[:a/name "name" :a/vals [{:b/val [{:c/name "one"}{:c/name "two"}]}
{:b/val [{:c/name "three"}]}]]
Output:
[:a/name :a/vals "one, two, three"]
This produces the output from the input, is this what you want?
(defn f [[k1 _ k2 rels]]
[k1 k2
(clojure.string/join ", "
(map :c/name (apply concat (mapcat vals rels))))])
I want to take text (sample below) and convert it into a nested data structure that can be walked:
Arbirarchy !
dog, my friend
Bailey the Great
cats, my enemies
Robert the Terrible
Trev the Diner
Gombas the Tasty
Lenny Lion
Alligators
Sadly I have none
Is this the solution?
(defn parse [s]
{(re-find #"(?m)^.+" s)
(map parse (map #(str/replace % #"(?m)^\s{2}" "")
(map first (re-seq #"(?m)(^\s{2}.+(\n\s{4}.+$)*)" s))))})
your string (supposed that "Gombas" is indented):
(def s "hierarchy\n dog\n Bailey\n cats\n Robert\n Trev\n Gombas")
test
(parse s)
-> {"hierarchy" ({"dog" ({"Bailey" ()})}
{"cats" ({"Robert" ()}
{"Trev" ()}
{"Gombas" ()})})}
I'm trying to wrap knockout.js in clojurescript but its turning to be very difficult. The problem that I'm having is the reference to the 'this' variable. I'm thinking of giving up and just using javascript directly.
I've taken examples off http://knockoutjs.com/examples/helloWorld.html and http://knockoutjs.com/examples/contactsEditor.html
I've managed to wrap easy functions with some macros. For example:
var ViewModel = function() {
this.firstName = ko.observable("Bert");
this.lastName = ko.observable("Bertington");
this.fullName = ko.computed(function() {
// Knockout tracks dependencies automatically. It knows that fullName depends on firstName and lastName, because these get called when evaluating fullName.
return this.firstName() + " " + this.lastName();
}, this);
};
becomes:
(defviewmodel data
(observing :first_name "Bert")
(observing :last_name "Bertington")
(computing :name [:first_name :last_name]
(str :first_name " " :last_name)))
However, for something harder like:
var BetterListModel = function () {
this.itemToAdd = ko.observable("");
this.allItems = ko.observableArray(["Fries", "Eggs Benedict", "Ham", "Cheese"]); // Initial items
this.selectedItems = ko.observableArray(["Ham"]); // Initial selection
this.addItem = function () {
if ((this.itemToAdd() != "") && (this.allItems.indexOf(this.itemToAdd()) < 0)) // Prevent blanks and duplicates
this.allItems.push(this.itemToAdd());
this.itemToAdd(""); // Clear the text box
};
this.removeSelected = function () {
this.allItems.removeAll(this.selectedItems());
this.selectedItems([]); // Clear selection
};
this.sortItems = function() {
this.allItems.sort();
};
};
ko.applyBindings(new BetterListModel());
I'm not sure what I can do in clojurescript to match code like this: this.allItems.push(this.itemToAdd())
Any thoughts?
After lots of trial and error, I figured out how to have the same structure for clojurescript as for javascript.
The this-as macro has a few idiosyncrasies and only works when the method is put into the class
for example I want to create something that looks like this in javascript:
var anobj = {a: 9,
get_a: function(){return this.a;}};
I have to do a whole lot more coding to get the same object in clojurescript:
(def anobj (js-obj))
(def get_a (fn [] (this-as me (.-a me))))
(aset anobj "a" 9)
(aset anobj "get_a" get_a)
which is seriously ugly for a language as beautiful as clojure. Things get worse when you have got functions that link to each other, like what happens in knockout.
I found that the best way to create an js-object with lots of this's in there is to define a __init__ method, add it to the class and then run it, then remove it from the class. For example, if I wanted to make another object:
var avobj = {a: this,
b: 98,
c: this.a
get_a: function(){return str(this.a) + str(this.c);}};
written as clojurescript with and __init__ method looks like this:
(def avobj (js-obj))
(def av__init__
#(this-as this
(aset this "a" this)
(aset this "b" 9)
(aset this "c" (.-a this))
(aset this "get_a" (fn [] (str (.-a this) (.-c this))))))
(aset avobj "__init__" av__init__)
(. avobj __init__)
(js-delete stuff "__init__")
There's still a whole bunch more code than javascript... but the most important thing is that you get the same object as javascript. Setting all the variables using this form also allows the use of macros to simplify. So now I have defined a macro:
(defmacro defvar [name & body]
(list 'do
(list 'def name
(list 'map->js
{
:__init__
(list 'fn []
(list 'this-as 'this
(list 'aset 'this "a" "blah")))
}))
;(. js/console log ~name)
(list '. name '__init__)
(list 'js-delete name "__init__")))
and with map->js taken from jayq.utils:
(defn map->js [m]
(let [out (js-obj)]
(doseq [[k v] m]
(aset out (name k) v))
out))
Now I can write code like this:
(defvar avobj
a this
b 9
c (.-a this)
get_a (fn [] (str (.-a this) (.-c this))))
and for the answer to knockout:
(defvar name_model
first_name (observable "My")
last_name (observable "Name")
name (computed (fn [] (str (. this first_name) " " (. this last_name)))))
(. js/ko (applyBindings name_model));
Which is really nice for me as it matches javascript really well and its entirely readable!
If you need an explicit reference to JavaScript's this dynamic binding, ClojureScript provides a this-as macro:
https://github.com/clojure/clojurescript/blob/master/src/clj/cljs/core.clj#L324