I am follwing the example here http://patternhatch.com/2013/06/12/messaging-using-clojure-and-zeromq/
I have verified that I can serialize MarketData and have built the protobuf for it.
Instead of using chesire serialization I decided to try my new learned protobuf serialization knowledge. When I modified the functions in that example into their gpb versions, when I run
(future-call market-data-publisher-gpb)
It seems ok. However, when I run the client
(get-market-data-gpb 100)
Nothing happens. I have two questions:
1) Is there some sort of graphical or otherwise debugger for Clojure?
2) If someone can point me in the right direction as to what I am doing wrong on my modfied example that would also be helpful.
I seem to remember that over ZMQ with a [protobuf] binary data payload required a different set of calls?
(ns clj-zmq.core
(:import [org.jeromq ZMQ])
)
(use 'flatland.protobuf.core)
(import com.example.Example$MarketData)
(def MarketData (protodef Example$MarketData))
(def ctx (ZMQ/context 1))
(defn market-data-publisher-gpb
[]
(let [s (.socket ctx ZMQ/PUB)
market-data-event (fn []
{:symbol (rand-nth ["CAT" "UTX"])
:size (rand-int 1000)
:price (format "%.2f" (rand 50.0))})]
(.bind s "tcp://127.0.0.1:6666")
(while :true
(.send s ( protobuf-dump(market-data-event))))))
; Client
(defn get-market-data-gpb
[num-events]
(let [s (.socket ctx ZMQ/SUB)]
(.subscribe s "")
(.connect s "tcp://127.0.0.1:6666")
(dotimes [_ num-events]
(println (protobuf-load MarketData (.recv s))))
(.close s)))
Both Eclipse Counterclockwise and IntelliJ Cursive have Clojure debug support.
Also, your address looks bad - should be "tcp://127.0.0.1:6666".
Related
Suppose I have a very simple .clj file on disk with the following content:
(def a 2)
(def b 3)
(defn add-two [x y] (+ x y))
(println (add-two a b))
From the context of separate program, I would like to read the above program as a list of S-Expressions, '((def a 2) (def b 3) ... (add-two a b))).
I imagine that one way of doing this involves 1. Using slurp on (io/file file-name.clj) to produce a string containing the file's contents, 2. passing that string to a parser for Clojure code, and 3. injecting the sequence produced by the parser to a list (i.e., (into '() parsed-code)).
However, this approach seems sort of clumsy and error prone. Does anyone know of a more elegant and/or idiomatic way to read a Clojure file as a list of S-Expressions?
Update: Following up on feedback from the comments section, I've decided to try the approach I mentioned on an actual source file using aphyr's clj-antlr as follows:
=> (def file-as-string (slurp (clojure.java.io/file "src/tcl/core.clj")))
=> tcl.core=> (pprint (antlr/parser "src/grammars/Clojure.g4" file-as-string))
{:parser
{:local
#object[java.lang.ThreadLocal 0x5bfcab6 "java.lang.ThreadLocal#5bfcab6"],
:grammar
#object[org.antlr.v4.tool.Grammar 0x5b8cfcb9 "org.antlr.v4.tool.Grammar#5b8cfcb9"]},
:opts
"(ns tcl.core\n (:gen-class)\n (:require [clj-antlr.core :as antlr]))\n\n(def foo 42)\n\n(defn parse-program\n \"uses antlr grammar to \"\n [program]\n ((antlr/parser \"src/grammars/Clojure.g4\") program))\n\n\n(defn -main\n \"I don't do a whole lot ... yet.\"\n [& args]\n (println \"tlc is tcl\"))\n"}
nil
Does anyone know how to transform this output to a list of S-Expressions as originally intended? That is, how might one go about squeezing valid Clojure code/data from the result of parsing with clj-antlr?
(import '[java.io PushbackReader])
(require '[clojure.java.io :as io])
(require '[clojure.edn :as edn])
;; adapted from: http://stackoverflow.com/a/24922859/6264
(defn read-forms [file]
(let [rdr (-> file io/file io/reader PushbackReader.)
sentinel (Object.)]
(loop [forms []]
(let [form (edn/read {:eof sentinel} rdr)]
(if (= sentinel form)
forms
(recur (conj forms form)))))))
(comment
(spit "/tmp/example.clj"
"(def a 2)
(def b 3)
(defn add-two [x y] (+ x y))
(println (add-two a b))")
(read-forms "/tmp/example.clj")
;;=> [(def a 2) (def b 3) (defn add-two [x y] (+ x y)) (println (add-two a b))]
)
Do you need something like this?
(let [exprs (slurp "to_read.clj")]
;; adding braces to form a proper list
(-> (str "(" (str exprs")"))
;; read-string is potentially harmful, since it evals the string
;; there exist non-evaluating readers for clojure but I don't know
;; which one are good
(read-string)
(prn)))
Consider a dataset like this:
(def data [{:url "http://www.url1.com" :type :a}
{:url "http://www.url2.com" :type :a}
{:url "http://www.url3.com" :type :a}
{:url "http://www.url4.com" :type :b}])
The contents of those URL's should be requested in parallel. Depending on the item's :type value those contents should be parsed by corresponding functions. The parsing functions return collections, which should be concatenated, once all the responses have arrived.
So let's assume that there are functions parse-a and parse-b, which both return a collection of strings when they are passed a string containing HTML content.
It looks like core.async could be a good tool for this. One could either have separate channels for each item ore one single channel. I'm not sure which way would be preferable here. With several channels one could use transducers for the postprocessing/parsing. There is also a special promise-chan which might be proper here.
Here is a code-sketch, I'm using a callback based HTTP kit function. Unfortunately, I could not find a generic solution inside the go block.
(defn f [data]
(let [chans (map (fn [{:keys [url type]}]
(let [c (promise-chan (map ({:a parse-a :b parse-b} type)))]
(http/get url {} #(put! c %))
c))
data)
result-c (promise-chan)]
(go (put! result-c (concat (<! (nth chans 0))
(<! (nth chans 1))
(<! (nth chans 2))
(<! (nth chans 3)))))
result-c))
The result can be read like so:
(go (prn (<! (f data))))
I'd say that promise-chan does more harm than good here. The problem is that most of core.async API (a/merge, a/reduce etc.) relies on fact that channels will close at some point, promise-chans in turn never close.
So, if sticking with core.async is crucial for you, the better solution will be not to use promise-chan, but ordinary channel instead, which will be closed after first put!:
...
(let [c (chan 1 (map ({:a parse-a :b parse-b} type)))]
(http/get url {} #(do (put! c %) (close! c)))
c)
...
At this point, you're working with closed channels and things become a bit simpler. To collect all values you could do something like this:
;; (go (put! result-c (concat (<! (nth chans 0))
;; (<! (nth chans 1))
;; (<! (nth chans 2))
;; (<! (nth chans 3)))))
;; instead of above, now you can do this:
(->> chans
async/merge
(async/reduce into []))
UPD (below are my personal opinions):
Seems, that using core.async channels as promises (either in form of promise-chan or channel that closes after single put!) is not the best approach. When things grow, it turns out that core.async API overall is (you may have noticed that) not that pleasant as it could be. Also there are several unsupported constructs, that may force you to write less idiomatic code than it could be. In addition, there is no built-in error handling (if error occurs within go-block, go-block will silently return nil) and to address this you'll need to come up with something of your own (reinvent the wheel). Therefore, if you need promises, I'd recommend to use specific library for that, for example manifold or promesa.
I wanted this functionality as well because I really like core.async but I also wanted to use it in certain places like traditional JavaScript promises. I came up with a solution using macros. In the code below, <? is the same thing as <! but it throws if there's an error. It behaves like Promise.all() in that it returns a vector of all the returned values from the channels if they all are successful; otherwise it will return the first error (since <? will cause it to throw that value).
(defmacro <<? [chans]
`(let [res# (atom [])]
(doseq [c# ~chans]
(swap! res# conj (serverless.core.async/<? c#)))
#res#))
If you'd like to see the full context of the function it's located on GitHub. It's heavily inspired from David Nolen's blog post.
Use pipeline-async in async.core to launch asynchronous operations like http/get concurrently while delivering the result in the same order as the input:
(let [result (chan)]
(pipeline-async
20 result
(fn [{:keys [url type]} ch]
(let [parse ({:a parse-a :b parse-b} type)
callback #(put! ch (parse %)(partial close! ch))]
(http/get url {} callback)))
(to-chan data))
result)
if anyone is still looking at this, adding on to the answer by #OlegTheCat:
You can use a separate channel for errors.
(:require [cljs.core.async :as async]
[cljs-http.client :as http])
(:require-macros [cljs.core.async.macros :refer [go]])
(go (as-> [(http/post <url1> <params1>)
(http/post <url2> <params2>)
...]
chans
(async/merge chans (count chans))
(async/reduce conj [] chans)
(async/<! chans)
(<callback> chans)))
I have had an issue with my code due to a very twisted behaviour of a function.
I use google-api to stream data in BigQuery. In Java, you create an object called Bigquery.Tabledata.InsertAll (a request) and then you execute it
TableDataInsertAllResponse response = request.execute();
(sample code from Google)
But as you can see, the execution is something that has side effect while returning a response.
I reproduced it in Clojure (in a chunked fashion)
(defn load-bq
[client project-id dataset-id table-id data & {:keys [chunk-size] :or {chunk-size 500}}]
(let [chunks (partition-all chunk-size data)
_ (str "Now streaming data into table : [" project-id ":" dataset-id "." table-id "]")]
(map (partial atomic-load-bq client project-id dataset-id table-id) chunks)))
If I try to stream in repl, it works fine. But surpinsingly a doall does not work in code like in a let or with a do.
Here a function to illustrate the principle
(def load-this-table
[... data]
(let [_ (doall (load-bq ... data))]
(load-bq ... data)
(do (load-bq ...data))))
Here, nothing will be loaded.
I found a trick that works though it is a bit far-fetched :
(def load-this-table
[... data]
(let [_ (println (load-bq ... data))]
(println (load-bq ... data))))
Here both line will execute. of course i need only one streaming so I chose a single solution here.
How to force evaluation of this code without having to use println ?
I could use what force evaluation behing println or any more general core function.
I have the impression that the difference is not really linked to lazyness but more to a more fundamental difference bewteen Clojure and Java. And maybe that the response have to be "taken" by the client.
Thanks !
Where in the clojurescript library can I access a function to compile snippets of clojure into js?
I need this to run in the clojure (not clojurescript) repl:
(->js '(fn [x y] (+ x y)))
=> "function(x,y){return x+y}"
Snippet compilation from Clojure REPL
(require '[cljs.analyzer.api :refer [analyze empty-env]])
(require '[cljs.compiler.api :refer [emit]])
(let [ast (analyze (empty-env) '(defn plus [a b] (+ a b)))]
(emit ast))
;; result
"cljs.user.plus = (function cljs$user$plus(a,b){\nreturn (a + b);\n});\n"
Snippet compilation from ClojureScript REPL:
(require '[cljs.js :refer [empty-state compile-str]])
(compile-str (empty-state) "(defn add [x y] (+ x y))" #(println (:value %)))
;; Output (manually formatted for easier reading)
cljs.user.add = (function cljs$user$add(x,y){
return (x + y);
});
compile-str takes a callback as the last argument. It will be called with a map either with a key :value containing result JS as a string or :error with the compilation error.
In both cases org.clojure/tools.reader is needed on your classpath.
there is a lightweight alternative: https://github.com/kriyative/clojurejs which creates the right output asked by the question.
Examples can be seen here: https://github.com/kriyative/clojurejs/wiki/Examples
i have a big problem my doctor want form me to make calculator using new language(clojure)but i don't know anything about it i read some information from www.clojure.org but i still have a problem how to save the code in a file to run it another time and i need the path how to connect java to clojure i found this code :
(ns rayne.main
(:gen-class)
(:import (javax.swing JFrame JTextField JButton JOptionPane)
(java.awt.event ActionListener)
(java.awt GridLayout)))
(def numbers (ref []))
(def times-clicked (ref 0))
(defn calc [nseq op]
(let [n1 (first nseq)
n2 (last nseq)]
(cond
(= op "+") (+ n1 n2)
(= op "*") (* n1 n2)
(= op "-") (- n2 n1)
(= op "/") (/ n1 n2))))
(defn add-op-button [op text button]
(.addActionListener button
(proxy [ActionListener] []
(actionPerformed [e]
(dosync
(ref-set times-clicked (inc #times-clicked))
(if (= #times-clicked 2)
(do
(let [result (.toString (calc #numbers op))
result2 (read-string result)]
(.setText text result)
(ref-set numbers [])
(ref-set times-clicked 0)))
(do
(ref-set numbers (conj #numbers (read-string (.getText text))))
(.setText text ""))))))))
(defn -main []
(let [frame (JFrame. "Calculator")
add-button (JButton. "+")
sub-button (JButton. "-")
mul-button (JButton. "*")
div-button (JButton. "/")
clr-button (JButton. "Clear")
text-field (JTextField.)]
(add-op-button "+" text-field add-button)
(add-op-button "-" text-field sub-button)
(add-op-button "*" text-field mul-button)
(add-op-button "/" text-field div-button)
(doto frame
(.setLayout (GridLayout. 1 5))
(.add text-field)
(.add add-button)
(.add sub-button)
(.add mul-button)
(.add div-button)
(.setSize 500 100)
(.setVisible true))))
so when i try to test it i don't know how it work .
please i need some one to help me in this problem and send me a link to install a clojure program to execute such a file.
thank's for all
Sounds like a great reason to lean a fun new language.
for getting started with new clojure pojects the leiningen tool can get you to the point of compiling and running code very quickly. (i'm assuming mac or linux here)
install leiningen
lein new project-name
put your code in the project-name/src/ ... /core.clj file
lein uberjar
java -jar name-of-jar-file
repeat, hack, and have some fun!
Leiningen can also start a repl for you which will speed up you iterations and integrates well (through slime/swank) with emacs.
here is a good tutorial on leiningen
Clojure.org has a nice section called Getting Started that is about ... getting started. From getting the clojure zip file to debugging and profiling. Very, very basic.
Also has links to more advanced resources.