I'm looking for a way to let a ring server - by request - grab an image from an URL, process it and serve the modified version.
This is how I got so far:
(require '[clj-http.client :as client]
'[ring.adapter.jetty :refer [run-jetty])
(import javax.imageio.ImageIO)
(def handler (-> (client/get "http://.../some-img.jpg" {:as :stream})
:body
ImageIO/read
.. ;; do some processing with the BufferedImage here
.. ;; and serve the modified version))
(run-jetty handler {:port 55555})
Especially I'm having troubles performing the last step inside the threading macro.
To return bytes as the response in Ring you need to provide either java.io.File or java.io.InputStream as the body content:
(defn jpeg-response [image-data]
(-> image-data
(ring.util.response/response)
(ring.util.response/content-type "image/jpeg")))
I haven't found a way to obtain an InputStream from BufferedImage directly without creating an intermediate byte arrays. Maybe this is a limitation of the Java Image API due to complexity required to implement a "pull" approach to get a stream of image bytes in the desired format.
Instead there is a "push" API where ImageIO.write method requires some kind of output for image bytes. It might be a java.io.File or java.io.OutputStream. Thus you need to first store the bytes somewhere (java.io.File or java.io.ByteArrayOutputStream) and then use them as your response body:
With file (so storing the image on the disk first - you need to remember to clean it up):
(let [image (ImageIO/read "...")
image-file (java.io.File. "...")]
(ImageIO/write image "jpg" image-file)
(jpeg-response image-file))
In-memory byte array:
(let [image (ImageIO/read "...")
image-output-stream (ByteArrayOutputStream.)]
(ImageIO/write image "jpg" image-output-stream)
(jpeg-response (ByteArrayInputStream. (.toByteArray image-output-stream))))
Related
I'm trying to use the Github Gist API to get a list of all of my Gists like so:
(ns epi.core)
(.then (.fetch js/window "https://api.github.com/users/seisvelas/gists")
(fn [data] (.log js/epi data)))
js/epi is just console.log except provided by the blogging platform I'm using (epiphany.pub).
When I call that API from curl it works fine; however, when done in cljs instead of giving me the body of the response, this gives me [object Response]. Does anyone know how I can get the body text of the response?
TL;DR
(-> (.fetch js/window "https://api.github.com/users/seisvelas/gists")
(.then #(.json %)) ; Get JSON from the Response.body ReadableStream
(.then #(.log js/epi %))
is what I'd write
From ClojureScript, a JavaScript call like data.body() can be invoked with
(.body data)
and a JavaScript property access like data.body with
(.-body data)
One of those should work in your case. However, the fetch API requires a bit more if you want to get JSON from the body, which I assume you do based on the endpoint.
If you're dealing with promise chains, you might also want to consider using -> (thread-first) so it reads top to bottom.
See this Gist for more about threading promise chains.
There is a library wrapping js fetch API called lamdaisland.fetch. This library uses transit as default encoding format, so you need to specify accept format when working with github API.
This library contains kitchen-async.promise as its dependency, so you can require the kitchen-async.promise in your ClojureScript source code.
(ns fetch.demo.core
(:require [kitchen-async.promise :as p]
[lambdaisland.fetch :as fetch]))
(p/try
(p/let [resp (fetch/get
"https://api.github.com/users/seisvelas/gists"
{:accept :json
:content-type :json})]
(prn (:body resp)))
(p/catch :default e
;; log your exception here
(prn :error e)))
Seems like .fetch returns a Response object, and you need to get the attribute body from it for the body. https://developer.mozilla.org/en-US/docs/Web/API/Response
Something like (.body data)
I'm still in a learning phase for Cljs and Om. I'm looking into writing comopnent test. Some components have cljs-http calls to an API I created. When testing, I do not want those API calls to actually send the request, so I'm looking into mocking the request and returning a fixture. Here's an example component I have:
(defn async-component [data owner]
(reify
IWillMount
(will-mount [_]
(let [resp (go ((<! (async-call "/") :body))]
(om/update! data [:objects] resp)))
IRender
(render [_]
[:ul
(om/build-all item-component data)])))
(defn async-call [path]
(http/get path {:keywordize-keys true}))
Please don't mind if the code is actually syntactically correct, I'm just showing the gist of it.
What I now want to do is test this async-component and the API call to see if it will render the fixture that I mock the request with. How is this done? I know cljs.test has the async block to test async code with, but all example show it testing actual code blocks that only have a go in it, not in a larger context.
Here is a way you might use mocking to test your component:
(deftest test-async-component
(cljs.test/async done
(with-redefs
[async-call (fn [path]
(let [mock-ch (async/chan 1)
fixture-data {:body {:fixture-with path :and "foobar"}})]
(async/put! mock-ch fixture-data)
mock-ch)]
; At this point we successfully mocked out our data source (the API call)
; the only task that remains is to render our Om component into DOM and inspect it.
; As this task requires utility fns I will reuse the ones in this blog post:
; http://lab.brightnorth.co.uk/2015/01/27/unit-and-browser-testing-om-clojurescript-applications/
(let [c (new-container!)
initial-data {:objects [{:initial-object 42}]}]
; This will mount and render your component into the DOM residing in c.
(om/root async-component initial-data {:target c})
(testing "fixture data gets put into the DOM"
(is (= "foobar" (text (sel1 c :ul)))))
; You can add more tests in this manner, then finally call 'done'.
(done)))))
The steps taken in the above code in English:
Write async-call's mock fn that returns a channel (the same interface as the original one) prefilled with fixture data.
Mock out the original fn (you need to refer it or fully qualify the ns).
Create a new virtual DOM for unit testing purposes.
Specify the mock data that doesn't come from the API, if any.
Render your component into DOM (this will call async-call when om/will-mount runs, taking the fixture-data off the chan).
Observe DOM contents.
I am working with a Java library in clojure and I would like to attach some extra information to those objects while I do stuff to them. I was thinking of stashing the info as metadata although I'm not committed to that and if you have any better suggestions, I would love to hear them.
The basic example would be to add a name to a BufferedImage and store it as metadata. The goal is to process the image and then be able to pull the name back out.
(ns test.clj
(:import [java.awt BufferedImage]))
; define a new image and try to attach data
(let [image (new BufferedImage 200 200 1)]
(with-meta image {:name "myawesomepicture"}))
;generates the following error
;ClassCastException java.awt.image.BufferedImage
;cannot be cast to clojure.lang.IObj
;clojure.core/with-meta (core.clj:214)
There is a recent post that defines a type that can attach metadata to an array but this wraps the values which will trigger type errors when trying to use the object as shown below. (box is a metadata wrapping function from the previous link; show is a function from mikera's imagez library that displays a BufferedImage).
(use 'mikera.image.core)
(def image1 (new BufferedImage 200 200 1))
(show image1) ; no problem
(def image2 (box image1 {:name "test"})) ; add metadata using box wrapper function
(show image2) ; ClassCastException Box cannot be cast to java.awt.image.BufferedImage
I would appreciate any thoughts on using clojure to tag Java objects with custom data - whether as metadata or fields or whatever.
thanks,
zach cp
My question is, how can I idiomatically read the body of a Ring request if it has already been read?
Here's the background. I'm writing an error handler for a Ring app. When an error occurs, I want to log the error, including all relevant information that I might need to reproduce and fix the error. One important piece of information is the body of the request. However, the statefulness of the :body value (because it is a type of java.io.InputStream object) causes problems.
Specifically, what happens is that some middleware (the ring.middleware.json/wrap-json-body middleware in my case) does a slurp on the body InputStream object, which changes the internal state of the object such that future calls to slurp return an empty string. Thus, the [content of the] body is effectively lost from the request map.
The only solution I can think of is to preemptively copy the body InputStream object before the body can be read, just in case I might need it later. I don't like this approach because it seems clumsy to do some work on every request just in case there might be an error later. Is there a better approach?
I have a lib that sucks up the body, replaces it with a stream with identical contents, and stores the original so that it can be deflated later.
groundhog
This is not adequate for indefinitely open streams, and is a bad idea if the body is the upload of some large object. But it helps for testing, and recreating error conditions as a part of the debugging process.
If all you need is a duplicate of the stream, you can use the tee-stream function from groundhog as the basis for your own middleware.
I adopted #noisesmith's basic approach with a few modifications, as shown below. Each of these functions can be used as Ring middleware.
(defn with-request-copy
"Transparently store a copy of the request in the given atom.
Blocks until the entire body is read from the request. The request
stored in the atom (which is also the request passed to the handler)
will have a body that is a fresh (and resettable) ByteArrayInputStream
object."
[handler atom]
(fn [{orig-body :body :as request}]
(let [{body :stream} (groundhog/tee-stream orig-body)
request-copy (assoc request :body body)]
(reset! atom request-copy)
(handler request-copy))))
(defn wrap-error-page
"In the event of an exception, do something with the exception
(e.g. report it using an exception handling service) before
returning a blank 500 response. The `handle-exception` function
takes two arguments: the exception and the request (which has a
ready-to-slurp body)."
[handler handle-exception]
;; Note that, as a result of this top-level approach to
;; error-handling, the request map sent to Rollbar will lack any
;; information added to it by one of the middleware layers.
(let [request-copy (atom nil)
handler (with-request-copy handler request-copy)]
(fn [request]
(try
(handler request)
(catch Throwable e
(.reset (:body #request-copy))
;; You may also want to wrap this line in a try/catch block.
(handle-exception e #request-copy)
{:status 500})))))
I think you're stuck with some sort of "keep a copy around just in case" strategy. Unfortunately it looks like :body on the request must be an InputStream and nothing else (on the response it can be a String or other things which is why I mention it)
Sketch: In a very early middleware, wrap the :body InputStream in an InputStream that resets itself on close (example). Not all InputStreams can be reset, so you may need to do some copying here. Once wrapped, the stream can be re-read on close, and you're good. There's memory risk here if you have giant requests.
Update: here's an half-baked attempt, inspired in part by tee-stream in groundhog.
(require '[clojure.java.io :refer [copy]])
(defn wrap-resettable-body
[handler]
(fn [request]
(let [orig-body (:body request)
baos (java.io.ByteArrayOutputStream.)
_ (copy orig-body baos)
ba (.toByteArray baos)
bais (java.io.ByteArrayInputStream. ba)
;; bais doesn't need to be closed, and supports resetting, so wrap it
;; in a delegating proxy that calls its reset when closed.
resettable (proxy [java.io.InputStream] []
(available [] (.available bais))
(close [] (.reset bais))
(mark [read-limit] (.mark bais read-limit))
(markSupported [] (.markSupported bais))
;; exercise to reader: proxy with overloaded methods...
;; (read [] (.read bais))
(read [b off len] (.read bais b off len))
(reset [] (.reset bais))
(skip [n] (.skip bais)))
updated-req (assoc request :body resettable)]
(handler updated-req))))
I'm trying to write a web service that takes requests, puts them into a queue, and then processes them in batches of 2. The response can be sent straight away, and I'm trying to use Lamina as follows (though not sure it's the right choice)...
(def ch (channel))
(def loop-forever
(comp doall repeatedly))
(defn consumer []
(loop-forever
(fn []
(process-batch
#(read-channel ch)
#(read-channel ch)))))
(def handler [req]
(enqueue ch req)
{:status 200
:body "ok"})
But this doesn't work... :( I've been through all the Lamina docs but can't get my head around how to use these channels. Can anyone confirm if Lamina supports this kind of behaviour and advise on a possible solution?
The point of lamina is that you don't want to loop forever: you want lamina's scheduler to use a thread from its pool to do work for you whenever you have enough data to do work on. So instead of using the very, very low-level read-channel function, use receive to register a callback once, or (more often) receive-all to register a callback for every time a channel receives data. For example:
(def ch (lamina/channel))
(lamina/receive-all (lamina/partition* 2 channel)
(partial apply process-batch))
(defn handler [req]
(lamina/enqueue ch req)
{:status 200
:body "ok"})