I have a simple text file which contains simple email strings such as:
email1#example.com
email2#example.com
email3#example.com
email4#example.com
I want to be able to return the contents of this file as a json response through my REST api which is a simple GET request.
Is there a way I can read the file line by line (I can do that) and append to a JSON object such that I can easily render the contents on a webpage.
{
"emails": [
"email1#example.com",
"email2#example.com",
"email3#example.com",
"email4#example.com"
]
}
I want my REST API to look like:
(GET "/emails" [] {
"emails": [
"email1#example.com",
"email2#example.com",
"email3#example.com",
"email4#example.com"
]
})
but I want to render the JSON upon request as the file can be modified.
You can use the following:
(GET "/emails" []
(clojure.pprint/cl-format nil
"{\"emails\": [~{~S~^,~}]}"
(clojure.string/split-lines (slurp "/path/to/addresses.txt"))))
I came up with the following solution using the Chesire library
(GET "/emails" []
(generate-string {:emails (clojure.string/split-lines (slurp "/path/to/file"))} {:pretty true}))
Related
I have the following code, in which I want to send an InputStream of a file in the function fetch-items, which handles the route /fetch-items.
(defn id->image [image-id]
(let [image (.getInputStream (gfs/find-by-id fs image-id))] image))
(defn item-resp [item]
(assoc item :_id (str (:_id item))
:images (into [] (map id->image (:image-ids item))))
)
(defn fetch-items [req]
(res/response
(map item-resp (find fs "items" {}))))
Here's my request in the client side, using cljs-ajax:
(ajax-request
{:uri "http://localhost:5000/fetch-items"
:method :get
:handler #(prn (into [] %))
:format (json-request-format)
:response-format (raw-response-format)
}
)
But the response I get on the client is this:
[:failure :parse] [:response nil] [:status-text "No reader function for tag object. Format should have been EDN"]
:original-text "{:_id \"5e63f5c591585c30985793cd\", :images [#object[com.mongodb.gridfs.GridFSDBFile$GridFSInputStream 0x22556652 \"com.mongodb.gridfs.GridFSDBFile$GridFSInputStream#22556652\"]]}{:_id \"5e63f5d891585c30985793d0\", :images [#object[com.mongodb.gridfs.GridFSDBFile$GridFSInputStream 0x266ae6c0 \"com.mongodb.gridfs.GridFSDBFile$GridFSInputStream#266ae6c0\"]]}{:_id \"5e63f5e891585c30985793d3\", ...
Why would the response say that the format should have been edn? How do I extract this file/image out in the client side?
--- EDIT ----
Doing the following:
(IOUtils/toString image "utf-8")
returns a string of size 1594 bytes, which is much smaller than the expected image size.
I think this is because it's converting the file object to base64 and not the actual chunk of data associated with it.
How do I make it convert the actual GridFS chunk to base64 string and not the file object?
It seems that you are building a response and directly putting an reference to an InputStream object into the response, without encoding the contents of the stream into an array of bytes and serializing the contents on the response.
You'll need to find a way to read the contents of the stream and encode it in the response (maybe send them encoded as base 64?)
On the other end, the client seems to be expecting an EDN response, and when it found the string #object, it complained that it didn't have a way to read an object with such a tag.
Here's a simple example of how to read an EDN string with a tagged literal, you can extend it so you decode the image in the client (note I'm using Java in the decoder, you'll need a different implementation on JS):
(defn b64decode [s]
(->> s .getBytes (.decode (java.util.Base64/getDecoder)) String.))
(def message "{:hello :world :msg #base64str \"SGV5LCBpdCB3b3JrcyE=\"}")
;; Now we can read the EDN string above adding our handler for #base64str
(clojure.edn/read-string {:readers {'base64str b64decode}} message)
;; => {:hello :world, :msg "Hey, it works!"}
I have been struggling with signing requests for private GDAX endpoints for a while. Everything I have tried results in a 400 response with a message of "invalid signature." I have read their documentation on the matter several times, which can be found here. My current code is below. I'm using clj-http for making requests. I'm using their /time endpoint response for the timestamp, and I'm using pandect for the sha256 HMAC generation. I've tried converting the secret-decoded to a string using String. before passing it to sha256-hmac. I've also examined the request using clj-http's debug flag. It looks to me that I am following their directions precisely, but something must be wrong. I've done a lot of online searching before posting here. Any help would be greatly appreciated.
(defn get-time
[]
(-> (str (:api-base-url config) "/time")
(http/get {:as :json})
:body))
(defn- create-signature
([timestamp method path]
(create-signature timestamp method path ""))
([timestamp method path body]
(let [secret-decoded (b64/decode (.getBytes (:api-secret config)))
prehash-string (str timestamp (clojure.string/upper-case method) path body)
hmac (sha256-hmac prehash-string secret-decoded)]
(-> hmac
.getBytes
b64/encode
String.))))
(defn- send-signed-request
[method path & [opts]]
(let [url (str (:api-base-url config) path)
timestamp (long (:epoch (get-time)))
signature (create-signature timestamp method path (:body opts))]
(http/request
(merge {:method method
:url url
:as :json
:headers {"CB-ACCESS-KEY" (:api-key config)
"CB-ACCESS-SIGN" signature
"CB-ACCESS-TIMESTAMP" timestamp
"CB-ACCESS-PASSPHRASE" (:api-passphrase config)
"Content-Type" "application/json"}
:debug true}
opts))))
(defn get-accounts []
(send-signed-request "GET" "/accounts"))
(send-signed-request "GET" "/accounts")))
I've figured out the issue. Just in case anyone happens to have this very specific problem, I'm posting the solution. My error was that I was using the sha256-hmac function from pandect, which returns a string hmac, then I was converting that to a byte array, base64 encoding it, and converting it back to a string. Somewhere in those conversions, or perhaps in the pandect function's conversion, the value is altered in an erroneous way.
What works is using the sha256-hmac* function (note the asterisk) from pandect, which returns a raw byte array hmac, then base64 encoding that result directly, and converting it to a string. Below is the corrected, working code snippet, with which I was able to make a request to a private GDAX endpoint.
(defn create-signature
([timestamp method path]
(create-signature timestamp method path ""))
([timestamp method path body]
(let [secret-decoded (b64/decode (.getBytes (:api-secret config)))
prehash-string (str timestamp (clojure.string/upper-case method) path body)
hmac (sha256-hmac* prehash-string secret-decoded)]
(-> hmac
b64/encode
String.))))
I need to create on the fly excel file on request and serve it to user via response, using clojure.ring.
I use docjure to create an excel file and write it to the output stream
(see this function: https://github.com/mjul/docjure/blob/master/src/dk/ative/docjure/spreadsheet.clj#L86), and I get the output stream from using piped-input-stream (see https://github.com/ring-clojure/ring/blob/1.5.0/ring-core/src/ring/util/io.clj#L11).
The relevant part of code:
(defn excel-response
[params]
(-> (response (piped-input-stream (fn [out-stream]
(create-excel-into-stream out-stream
params))))
(assoc :headers {"Content-Type"
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"})))
Using this function, I always get an empty .xlsx file for some reasons.
It seems like piped-input-stream closes before I can serve it as the body of my response.
How do I use it properly, so I can write to output stream, pass it to input stream and then serve as the body of response?
Your excel-response function looks correct to me, I've written something very similar before. What does your create-excel-into-stream function look like?
This has worked for me in the past.
(ns my.data.handler
(:require [clojure.data.csv :as csv]
[clojure.java.io :as io]))
(defn create-excel-into-stream
[out-stream params]
(let [excel-data (get-excel-data params)]
(with-open [writer (io/writer out-stream)]
(csv/write-csv writer excel-data))))
I'm trying to make a very basic pedestal app to display a list of items, or a text message if the list is empty.
I thought what I wanted was :
a data model where [:root :items] would be my list of items
a template function for the page ([:root])
a static template function for the empty list
a dynamic template function for a list with elements
at the start of the application, send a message that will initialize the list to be empty
in rendering, respond to :
[node-create [] :map] by doing ... nothing ?
[node-create [:root] :map] by rendering the template for the full page
[node-create [:root :items] :map] by adding the template for the empty list
To be clear, what I want to display in the end would be (without elements)
<html>
<div>My list</div>
<p>There is nothing in the list</p>
</html>
and when there is something in the list :
<html>
<div>My list</div>
<ul>
<li>item1</li>
<li>item2</li>
</ul>
</html>
Am I on the right track here ?
I'm stuck at initializing the data model to be what I want. The closest I got is this :
;; behavior.clj
(defn init-root-transform [old-value message]
{})
(defn init-items-transform [old-value message]
[])
(def example-app
{:version 2
;; :transform [[:set-value [:greeting] set-value-transform]]})
:transform [[:bootstrap [:root] init-app-transform]
[:bootstrap-systems [:root :items] init-items-transform]
]})
;; start.cljs
... skipped...
(app/begin app)
(p/put-message (:input app) {msg/type :bootstrap msg/topic [:root]})
(p/put-message (:input app) {msg/type :bootstrap-systems msg/topic [:root :items]})
The problem is that, this way, I get a data model with a single :root node containing the value {:items [] }.
It might sounds like a very silly question, but are those two data models the same ?
[:root :systems] => []
[:root] => {:systems []}
And when I boot my application, the generated rendering deltas are :
[node-create [] :map]
[node-create [:root] :map]
[value [:root] nil {:items [] }]
I don't think this is suitable for rendering my list of items, is it ?
UPDATE :
#solussd implied that the two datamodels are the same ; however, when the following delta is generated :
[value [:root] nil {:items []}]
I still can't handle it. If I add a rendering config like this :
[value [:root :items] render-items-list]
Then the render-items-list function is not called.
If I add a rendering config like this :
[value [:root] render-root]
Then the render-root function will be called, but obviously, not with the proper 'path', and I suppose it would also be called when any other change to the ':root' element in the data model is changed.
Any idea how I can fix that is welcome...
Yes, those two datamodels are the same.
The node-create and value deltas are sufficient to send the sequence of items to your renderer, but you're rendering granularity is the whole list of items.
Your render config might look like this:
(defn render-config
[]
[:node-create [:root :items] render-item-list]
[:value [:root :items] update-item-list])
If you wanted to use a template for each item in the list, e.g.,
<ul template="list-item" fields="id:id,content:item"/>
in the html template file), you might want to output deltas at the individual list item level and probably have them be keyvals in a map, e.g. a model like:
{:root {:items {:1 <item> :2 <item> ...}}}.
Hope this helps.
I don't have a computer to check this right now, but I think if you add an emit pair to your dataflow definition like so it will allow you to trigger your renderer:
{... :emit [[#{:root :items}] (app/default-emitter :main)]}
This basically tells the dataflow to publish changes to [:root :items] as separate rendering deltas under the path [:main :root :items]. In your rendering config you should then specify this path:
[... [:value [:main :root :items] render-items] ...]
You will need to add the rendering path [:main] in a similar way, i.e.
[... [:node-create [:main] render-root] ...]
This will trigger your main template when your rendering deltas start arriving to the renderer.
Hopefully this helps.
I'm currently doing some REST API stuff in clojure, and I am using the ring.middleware.format library with compojure to transform JSON to and from clojure data structures.
I am having a huge issue, in that and JSON posted to the ring app will have all arrays replaced with the first item that was in the array. I.E. it will turn this JSON posted to it from
{
"buyer":"Test Name",
"items":[
{"qty":1,"size":"S","product":"Red T-Shirt"},
{"qty":1,"size":"M","product":"Green T-Shirt"}
],
"address":"123 Fake St",
"shipping":"express"
}
to this
{
"buyer": "Test Name",
"items": {
"qty": 1,
"size": "M",
"product": "Green T-Shirt"
},
"address": "123 Fake St",
"shipping": "express"
}
It does it for any arrays, including when an array is the root element.
I am using the following code in clojure to return the json:
(defroutes app-routes
(GET "/"
[]
{:body test-data})
(POST "/"
{data :params}
{:body data}))
;{:body (str "Printing " (count (data :jobs)) " jobs")}))
(def app
(-> (handler/api app-routes)
(wrap-json-params)
(wrap-json-response)))
The GET route has no issues with arrays and outputs properly, so it has to be either the way I am getting the data or the wrap-restful-params middleware.
Any ideas?
I am having similar problem with ring-json-params. So I ended up using raw request body and parsing the JSON string myself.
I used the folliwng code:
(defroutes app-routes
(POST "/"
{params :body}
(slurp params)))
I use the clj-json.core library for parsing JSON.
Hope this helps. If you figured out a better way then please share. I am a Clojure/Compojure newbie!!!
I ran into this problem recently and actually figured out what the problem is: it's that you have wrap-nested-params middleware getting evaluated after your wrap-json-params middleware, which causes objects stored in JSON arrays/Clojure vectors to be flattened by grabbing the first element contained within.
user=> bod-map
{:address "100 Bush Street", :contacts [{:name "Dudely Mcdooderson", :title "engineer", :tax_id "555"}], :name "Dudely Inc.", :tax_id "5234234"}
user=> (ring.middleware.nested-params/nested-params-request {:params bod-map})
{:params {"tax_id" "5234234", "name" "Dudely Inc.", "contacts" {:name "Dudely Mcdooderson", :title "engineer", :tax_id "555"}, "address" "100 Bush Street"}}
You can get around this by just making sure that wrap-nested-params is evaluated first in your middleware ordering.
I know it's been some time but I did just stumble upon the same issue. The only way I got it to work was to use the ring.middleware.format/wrap-restful-format before the compojure.handler/api.
I'm not sure why that is but if I put the compujure.handler/api wrapper first, it messes up the array parameteres