Search on Google shows some results for clojure profiling. Profiling tool for Clojure?
This link shows that profiling will blow up if large code is there. http://richhickey.github.com/clojure-contrib/profile-api.html. Is there any stable profiling tool for clojure?
I need to profile huge library made in clojure with some clojure functions of upto 400 lines.
We've used YourKit successfully. No special steps - just set it up like any other JVM app. Of course, interpreting the results is a bit of an art form. You can play with the filters to sometimes get it to ignore clojure core.
I found it useful to set it up on a swank server which let me profile parts of the app in the REPL (and turn things on and off between runs).
Which do you need to do - profile? or speed up the code?
If the latter, check out random pausing.
Related
I hope you are doing good and i really appreciate your help here for my query.
We have our system T3000 written in C++ (http://www.temcocontrols.com/ftp/software/9TstatSoftware.zip and codes are available here https://github.com/temcocontrols/T3000_Building_Automation_System).
I am trying to integrate 'BIRT reporting tool' in my C++ application. I want to create report based on the data available in our T3000 system. I think BIRT is embeddable (??). We don't need to compile and change the project, just need to be able to call it from T3000.exe mainly.
My thinking is we may put one menu label in existing T3000 and try to display report in user single click.
Can you please help me to solve my issue with 'BIRT' ? I really appreciate your answer.
Regards
Raju
Well, the answer depends on what your definition of "embeddable" is.
BIRT is written in pure Java.
I could think of 3 different ways:
Of course it is possible to integrate Java code into an existing C/C++ program (see Embed Java into a C++ application?).
You could just use the BIRT runtime engine and generate the report as PDF or HTML from the command line (that means, basically you call the java executable from your program with several arguments). See Birt - How to run report engine on the console? and http://eclipser-blog.blogspot.de/2008/02/automatic-generation-of-birt-reports.html for more information.
You could run a Java web server like Tomcat in a second process and then start your report by calling a http URL (e.g. you could use the included Servlet example). See http://www.eclipse.org/birt/documentation/integrating/viewer-usage.php
Similar to 3. (see below)
Some notes:
The second option is slow, due to the Java and BIRT engine startup overhead (this may take several seconds). With the first and third option, the startup overhead is or can be minimized to only once (and for each report).
For the second and third option it may be necessary to modify the existing code of the example programs to suit your needs.
The first option is probably the best for an industry-quality solution, but it is also the most difficult to develop.
Anyway, Java skills are necessary IMHO.
If you plan to run this on a SOC instead of a PC, take performance into account.
Is a Java-based solution well-suited for this kind of hardware? BIRT needs quite a lot of RAM and CPU (for a SOC). Hardware like the Raspi 3 should handle this quite easily, I reckon.
I integrated the BIRT runtime into an existing Python application (all this running on an application server) in a fourth way: I wrote a listener program that listens on a TCP socket for BIRT tasks. It uses a pool of worker processes (written in Java) which in turn use the BIRT report engine to generate the output. The client program (here: written in Python) opens a TCP connection to the listener and uses this socket to tell it which report to generate (including report parameters and destination file name). The listener program then in turn chooses a worker process for the task and gives the task to the worker process.
So, basically, this fourth option is similar to the third one, with two differences:
The communication is socket-based (instead of http), allowing bi-di communication.
The architecture is multi-processes instead of multi-threading. We choose this because very large reports could cause out-of-memory errors for otherwise unrelated reports that just happen to run at the same time. It's the same basic architecture Oracle chose for their reports server.
However, developing the programs took months.
HVB: I have to give you more than a simple thanks for the explanation above, this info will save us time I am sure. Raju will be sharing our experience after we get into the project a little deeper so others can benefit.
I have been learning Clojure a bit recently. Is there such a thing in Clojure world as Scala-like worksheets, into which I can put any code and get it evaluated as soon as I save it? Or maybe there's a similar solution for Clojure?
I am now working with lein repl and, sadly, can't say it's the most usable tool ever.
In Lisp development in general (and Clojure in particular) the preferred programming style is what's usually dubbed interactive programming - the developer keeps an image of the app loaded at all times and interacts with it via a REPL. You can easily modify the loaded code on the fly and test changes immediately in the REPL (that's not easy at all with Scala - one has to resort to something like JRebel to do it). I find the Scala worksheets a pretty primitive solution in comparison...
The workflow that I follow in Clojure is:
I open nREPL.el in Emacs - this loads my lein2 project with all of its dependencies and gives me a REPL which I can use the try out stuff
I write some code in source code and load the changed functions (maybe by evaluating a top level form with C-M-x
Afterwards I'd press C-x C-z to jump back to the REPL and I try out the new code in it
I go back to step 2
Basically the Clojure REPL is much more powerful than the Scala REPL and I personally consider it hugely superior to the Scala IDE worksheets. Once you get used to the interactive incremental style of programming Lisp offers everything else starts to look strangely complex by comparison. I'm not familiar with Eclipse's CounterClockWise Clojure plugin, but I'm pretty sure it offers similar functionality to Emacs's nREPL.el.
You might want to take a look at the autoexpect plugin for Leiningen. Every time you save any file in the working directory, the plugin compiles and runs your code; as a bonus, it will evaluate any "expect" function calls which can serve as tests. This is very helpful for test driven development and is a nice compliment to working with the REPL as described in the other answer (I often use one or the other or both together depending on how many test cases I have in place).
I should note that running autoexpect is far faster than running "lein test" or "lein run" repeatedly, due to the startup cost of the JVM and Leiningen.
It sounds like what you are looking for is the Clojure Koan plugin. This a worksheet-style problem-solving exercise tool that watches your edits and provides instant feedback on the correctness of your work.
For actual development workflow I second the advice others here have provided on tooling and interactive environment setup, but since you specifically said you are learning Clojure, this can be a fun approach. There is also a web application called 4Clojure that you might have fun playing with.
However you will eventually (or right away) want to get a smooth and convenient development environment set up, and I haven't seen any mention so far of a few important tools. One person mentioned Nrepl. If you like Emacs, there's a slime/swank-like interaction mode that jacks into nrepl called nrepl.el that provides very nice integration between editing files and messing around in the repl. Similarly there is VimClojure, and you can find plugins for IntelliJ (LaClojure) and Eclipse (Counterclockwise) that are also popular and well-maintained.
Someone mentioned autoexpect. If you are looking for a good TDD setup, I would recommend midje. If you are using a 2.0 preview release of leiningen there are a few issues with the lazytest integration being in flux, and lazytest itself is or should be deprecated. But I prefer midje over expectations, personally, and these problems will surely be worked out in the 2.0 release of lein-midje. The stable version of lein-midje that works with the non-preview (1.x) leiningen has autotest-like functionality using lazytest.
As nobody has mentioned it, cursive is really nice these days, although I was sad to move back to a full blown IDE. You can easily do the same thing as Emacs with the built in repl, just move into the namespace that you are working with and synchronise every time you make changes. It's a really nice way of working.
Note: I'm a backend (Java) developer by trade and work in Clojure in my spare time, so forgive me for my ignorance.
I'm trying to get my head around Clojurescript and how it could potentially fit in with projects I'm working on, or plan to work on in the future. As I've grown up with the "classic" web development mindset (e.g. Clojure running the backend, distributing data to the frontend via JSON to be processed in JS or returning a HTML page for the browser to render), I'm having trouble trying to understand how Clojurescript might make things better than this model.
Could anyone explain to me what the general approach to Clojurescript/Clojure development would be, seeing as the "Clojurescript One" project moniker signifies that application development will be unified under one language (as such)
What tasks would normally be done in the Clojurescript portion of the application?
What tasks would normally be done in the Clojure (e.g. backend) portion of the application?
Any help would be appreciated, or if anyone can point me towards some diagrams or explanations or anything - that would be great too!
I think Clojure/ClojureScript applications will be structured very similarly to X backend technology + JavaScript.
One big benefit with architecting applications with Clojure and ClojureScript - a richer data format than JSON (you can represent hash-maps and sets with arbitrary keys) without losing compactness.
JavaScript is a fine, fine language but ClojureScript offers quite a few benefits. It's semantically simpler (functional), ships with a rich standard library, a robust battle tested application library (Google Closure), and all the benefits you get from the tasteful application of syntactic abstraction via macros.
That said, it's still very much alpha software and the tooling still needs a lot of work.
A bit of background about me, I have developed with Clojurescript, JQuery, Vaadin, Servlets, JSP, and many other web technologies.
1) Clojurescript is much harder to learn than any other web technology I have used as you need, Java, Clojure, Closure (with an s ;), Closure Lib, and Closurescript specific knowledge.
2) Clojurescript doesn't make sense for a small app. It only makes sense when you will have ALOT of client side processing
3) Clojurescript's only use as far as I see is as a better javascript (which is why it is better suited to larger apps) as the minifier part of Clojurescript is available for javascript too
4) Only the client end would be written in Javascript, the server would be in Clojure/Java servlets
Maybe Ganelon micro-framework (which incidentally I am author of) will suit your needs - the execution model is similiar to Vaadin's: server-side Clojure code pushes UI updates to the browser through AJAX/JavaScript, but we don't store application state in session by default.
The demo and docs are available at http://ganelon.tomeklipski.com/
For me, clojure and clojurescript offers cleaner code than mixed stack. There is only one language to think of and code is quite easy to read.
At the backend clojure does things that java usually would do. Input validation, saving to database and above all, implementing business logic. Our backend also validates incoming / outgoing data by types using prismatic schemas.
Frontend in short: We get pretty code using ClojureScript and it is fast to write. We are using ClojureScript version of material-ui when writing UI components. We have to write less code when compared to the JavaScript and I find our UI component code to be easier to read than JavaScript counterpart. One of the main reasons is shorter closing tags and less noise by coding language. Development with ClojureScript is quite fast.
Of course ClojureScript is used for simple imput validation like RegExp for phone numbers etc.
One of the disadvantages of clojure which you have probably noticed is long lines after giving proper names to the functions. I havent found silver bullet how to cope with that.
As dnolen said: ClojureScript is still developing. It is way better now than it was 6 months ago, so you'll have to keep checking its maturity now and then.
does anyone know of a good profiling tool or library for Clojure?
I would prefer something that could be used from the REPL, along the lines of (with-profiling ...) in Allegro Common Lisp back in the day.
Is there anything along those lines?
Or do you have any experience with (non-commercial) Java profilers that work well with Clojure?
I found VisualVM (see here and here) to be very convenient. Its use with Clojure has been described about a year ago in this blog post; as far as I can see, it's not outdated in any way.
Note that the GUI from which one starts the VisualVM profiler has a prominent text area where one can enter classes / packages to be excluded from profiling -- I find the results rather more useful when clojure.* is on that list.
There's a newish Clojure library which offers profiling: https://github.com/ptaoussanis/timbre
Just found profile in Clojure contrib.
It doesn't work for large amounts of code (it blew up with OutOfMemoryError on a Project Euler solution which VisualVM handled just fine) and it requires you to insert profiling calls in the functions you want to profile.
Still, it's a better alternative to VisualVM in the cases where you just want to profile a couple of functions.
Quick heads-up that I've deprecated Timbre's profiling for a new dedicated Clojure + ClojureScript profiling lib at https://github.com/ptaoussanis/tufte.
That's basically a refinement of the stuff from Timbre, plus dedicated docs.
The README includes a comparison with JVM tools like VisualVM, YourKit, etc.
I just downloaded and built the libraries/executables of Google Performance Tools. Before I run the CPU profiler on the application that I want to investigate, I want to learn how to use the tools properly perhaps on a sample application. What would be a good example to run the Google CPU profiler on? Thanks in advance.
The following paragraph appears in the README.windows file distributed with perftools 1.3:
The heap-profiler has had a preliminary port to Windows. It has not been well tested, and probably does not work at all when Frame Pointer Optimization (FPO) is enabled -- that is, in release mode. The other features of perftools, such as the cpu-profiler and leak-checker, have not yet been ported to Windows at all.
In my experience, for performance tuning, stack-sampling is the method of choice.
Google perftools contains a stack-sampler, and I believe its visual analyzer can be made to show the cost of individual statements, not just functions.
What you need to know is the percent of time the stack contains that statement, because that is how much time would be saved if the statement were removed.